Does "Disallow: /xmlrpc.php" in robots.txt affect moz tools ability to fetch DA?
-
Just checked a website for Domain Authority using Moz' tool, however it returned 1 for DA, which should be unlikely. I have been trying to find the problem and found "Disallow: /xmlrpc.php" in robots.txt. Could this affect Moz' tools ability to get the required data?
-
I see. Thanks for the info, Tawny.
-
It's not necessarily that those links you're seeing with other tools but not with Moz's tools aren't relevant; it could just be that we haven't crawled and indexed the sites linking back to yours recently enough to include links from them in our index. If you're seeing 0 internal links reported, then that definitely means we haven't fully indexed your site, or if we have it was long enough ago that the linking data is no longer stored in the index, and we'll need to re-crawl and re-index.
The best thing you can do to get your site indexed by our tools is to search for it and the sites linking to it in Open Site Explorer. That should prompt our tools to visit your site and crawl and index it within the next few index updates.
-
Thank you for the breakdown of how the DA calculation works. The dilemma here is that with other tools, such as Raven Tools, it finds almost 300 backlinks for the website where Moz states it found none. Does this imply that none of those backlinks are relevant? The webpage is also linked internally, and moz returns a value of 0 for that also.
-
Hey there, Tawny from Moz's Help Team here.
That Disallow command shouldn't be affecting your DA score; that's calculated based on our own metrics and your backlink profile.
Domain Authority is a score (on a 100-point scale) developed by Moz that predicts how well a website will rank on search engines. It is made up of an aggregate of metrics (MozRank, MozTrust, general link profile, and more) that each have an impact on this score. it can be difficult to directly influence your DA, but one way to move toward increasing it would be to improve your overall SEO. A great place to start with that could be to focus on link building! You can learn more about link building here and get a more detailed explanation of Domain Authority through this resource.
In general, I'd recommend keeping in mind that all of these metrics are relative with respect to the other sites included in our index. The metrics will probably have some additional context when compared to sites similar to your own, including your competitors.
Rand does a great job of breaking down how to interpret DA fluctuations through this post.
I think you've already pinpointed the reason we don't have a DA score of higher than 1 for your site — you're not seeing any link information in our index. If we don't have any linking data for your site, your DA will naturally drop, as a big part of our calculation is unavailable.
Just a few points on how we compile our index:
- We grab the most recent index.
- We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains).
- We start crawling from the top down until we've crawled ~130 billion URLs
The idea here is that we're focusing on the highest-quality links we can find, coming from the most prominent pages of authoritative sites. So, while you may not see every link for a site within our index, we're aiming to report the most valuable ones available!
Most new sites and links will be indexed by our spiders and available in Mozscape and Open Site Explorer within 60 days, but some take even longer for many reasons - including the crawl-ability of sites, the number of inbound links to them, and the depth of pages in subdirectories. This also has the effect of slightly biasing our index in favor of newer links. Links are only cached in the index for around 180 days. After that, unless the tools have a compelling reason to return to your site and the sites linking to yours, re-crawl, and re-index them, those links can fall back out of our index. That doesn't mean they don't exist and aren't out there on the web, just that our tools don't see them anymore.
You can see our most recently updated schedule here as well as some more technical metrics on our Mozscape API Updates page. You can also see when the last and next updates happened on the Open Site Explorer (OSE) homepage at any time.
Since Moz focuses on quality of links over quantity, we are always focused on the most relevant links to display to our users. It's possible that Moz's index will leave out some of the lower-quality (non-link juice providing) links out of our index because of this. So, that might explain why you may see some discrepancies with what other tools may be showing.
You can read more about how we build our index in our guide here.
I know that's a ton of information, so if you've still got questions, feel free to drop us a line at help@moz.com and we'll do our best to sort things out for ya.
-
its a .fi domain. I have tried with many .fi domains and it works. Also Moz shows that it is getting 0 link data from the webpage, which means something might be blocking moz's tool from crawling.
-
What type of TLD is it, i.e. co.uk or .com etc? There are some that aren't supported by Moz and for those you'll see a DA of 1.
-
Hi,
Thanks for your reply.
The website is not new, and the domain was created 4.2014.
I doubted it would be that, but cant figure out what else it could be...Perhaps I have to accept the fate offered by moz tool?...
-
Hi,
I don’t think that’s the reason of a low DA, xmlrpc.php it’s a file that for security lots of websites block even in htaccess for prevent access.
Do you check the domain age? It's a new website?
Regards
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
On Page Grade & Page Optimization Tools not working
Hello! I have an issue with my on page grader and Page optimization. Over the past two weeks i get the attached errors. I searched the forums but after contacting our developer I am in a bit of a standstill. Could someone help on what triggers this? Would highly appreciate it. Best,
Moz Bar | | pranbg
Pran c5CSsHK c5CSsHK0 -
Page Optimization problem in Moz
Hello there, I've been using Moz Pro for a year now, and encountered I've encountered small issues here and there (like some features not working currently, but they start working after a few hours tops) but now I have a bigger issue. Page Optimization feature stopped working and it's dead for five days now. Even my older research is gone. (check the images) Anyone else having this problem, or is it just me? It says that the crawler is blocked or the page isn't working, and when I check the pages in other platforms with crawlers it all goes smooth. All of the pages are working, and all of the crawlers (tested even in Google console) are doing fine. I even had the Moz crawler on 04.07.2019 as the standard week crawl and I got the results. Everything else is working just fine, I have a problem with the Page Optimization feature only. Thanks in advance, Ivan Ga4LKkQ
Moz Bar | | BMGEmployee0 -
How can I be sure Moz Campaign for blog is working?
I set up a Moz Campaign to track my blog domain (made a separate one for our website) and added in keywords I've optimized our blog posts for. I've spent about 2 months optimizing pages and there's been no change in the numbers I see in the dashboard (specifically Search Visibility and Top National Keywords) and Rankings module. It all basically says either 0 or '--'. I think I've set this campaign up right because the website campaign has been changing and showing different results in the same amount of time. I'm going to ask the stupid question -- is this because I've set up the blog campaign wrong, my site is blocking Moz from accessing blog data or my SEO efforts simply aren't working? Please help. I need a sanity check here. Thank so much in advance!
Moz Bar | | Visier0 -
Editing Keywords in Moz Keyword Tracking Under Campaigns
Hi all, Some questions for Moz Pro Campaigns Setup Q1
Moz Bar | | SP1
As part of Moz Pro, we've created a campaign to track a set number of keywords and it seems that 2-3 of the keywords have been entered as a misspelling so I wanted to know how I can edit them to be corrected? Under "Add & Manage Keyword" I do see the keywords however do I need to click on "Show More" in order to scroll over to the keywords I need to edit / correct? Q2Eager to know if duplicated keywords are omitted when they are entered into the campaign. So if someone had entered in "abc123" twice in the list of 600 keywords will the 2nd instance of abc123 be removed when the keyword list is submitted into Moz? Cheers,
SP0 -
Keyword Difficulty tool showing Google search volume or Bing?
Hi, I plugged in a bunch of keywords to know their search volume, chose Google from the drop down and then chose English from the next dropdown and clicked on Check dificulty. It shoes me "Bing Search Volume (Exact Match)" on top but shows the name of the keyword and "for Google CA" on the left. Is the search volume for Google? or Bing? and the difficulty as well? Google or Bing? "
Moz Bar | | SpaMedica0 -
Suggestion: Moz Domain Authority should take disavow into account
Since Moz is trying to predict how Google ranks your site, and Google claims to take the disavow file into account, I'd like to suggest that Moz allow webmasters to upload their disavow file. I imagine this data would be useful to Moz in determining Domain Authority (they may even think of other ways to use it and might even help come to a conclusion on the great debate) and it gives a chance for sites to improve their Moz DA when they are bombarded by spammy links. I'd love to hear the community's thoughts on this idea, as well as the what Wizards of Moz have to say.
Moz Bar | | YairSpolter1 -
Does anyone else have issues with Moz's keyword search volume tool for Google's search engine?
It will show the search volume for Bing even when Google is selected. Then, if you select Bing, you'll get the same data as it shows for when you selected "google". So basically, this tool does not work for Google's search engine. Or it is most likely not a reliable way to perform keyword research. Anyone else notice this? Does Moz even offer a way to submit a support ticket to get this fixed?
Moz Bar | | ShokIdeaGroup1 -
Does anyone have a good article or video on how to read the SEO MOZ crawl report column by column?
I am trying to find a good how-to on how to read and analyze each column of the SEO MOZ crawl report, specifically, the excel sheet it allows you to export. What I'm really trying to get to the bottom of is what the "Yes" indiciates under rel-cononical. If it says "yes," does this mean that the link in question has been canonoicalized?
Moz Bar | | armcwill0