Moz can't crawl our site
-
Moz can't crawl our site because of an error in the robots.txt, we've tried everything in the troubleshooting guide but nothing works - I believe its a server error but have no idea how to fix it pls help
-
Hi there!
We would be happy to look into this for you! Can you please send us an email over at help@moz.com so we can pinpoint what is going on here?
We would also need the URL of the site you are trying to crawl.
We look forward to hearing back from you soon!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Disavow File Format and MOZ Spam Score Updates
Hi, Is there a defined file format for Google disavow file name? Does it has to be disavowlinks.txt or can we do this like domain-name-date.txt ? Also, since Google does not share their data with Moz, how does MOz updates its spam score after we disavow the bad links? Do we need to connect Google search console with Moz?
Link Explorer | | Sunil-Gupta1 -
The New Link Explorer (which will replace Open Site Explorer) is Now in Beta
Howdy Moz friends, Today, the Moz team is making a new tool -- Link Explorer -- available in private beta for Moz Pro subscribers (including those taking a free trial). The tool is still a little ways out from public launch, but we wanted to get your feedback to help make it the best product it can be. What's Link Explorer? In essence, it's a replacement for Open Site Explorer (Moz's tool for link discovery, competitive analysis, and link building) that addresses many of its most pernicious challenges, such as: Daily updates - no longer will you need to wait a month or two to see a new DA score or the links you built last week. Link Explorer updates every 24 hours with all the new links we've discovered that day, and gives a new DA score each night. A MUCH bigger index - OSE has always been known for having quality links, but quantity has been an issue. No longer. The new tool's link index is more than 20 times larger than Open Site Explorer's, covering trillions of links across hundreds of billions of pages, while maintaining a focus on high quality domains and pages. Additional functionality - new graphs (like link growth over time), new charts (like gained+lost links), new filters and sorts, and some new kinds of data coming soon. Improved metrics - Domain Authority and Page Authority have both been upgraded to have better correlations with Google's rankings (and they now update every 24 hours) Insanely fast - page load times on the new tool are almost as fast as Google's results 🙂 Less time waiting means more time to dig into results Link Tracking Lists - check a box next to any links of interest and you can build lists in the tool to track them over time, see whether/when they link, prioritize your outreach efforts, and (in the future) get aggregated data and alerts about those links There's much more to come, but we'd love for you to check out Link Explorer, find bugs, report things you love (and don't), and help us make it the best possible product for you and your teams. You can leave feedback here in this Q+A thread, email help@moz.com, or send feedback through the feedback form in Link Explorer.
Link Explorer | | randfish18 -
Different numbers in Moz bar vs. OSE
Hi, On a Google SERP, when i activate the MOZ toolbar one of the sites says it has 1,258 links / 667 RDs. But when I open this same site up in OSE I get 3 inbound links (see attch). What's going on? 0Au9H
Link Explorer | | sanjosepainting0 -
Incorrect crawl errors
A crawl of my websites has indicated that there are some 5XX server errors on my website: Error Code 608: Page not Decodable as Specified Content Encoding
Link Explorer | | LiamMcArthur
Error Code 803: Incomplete HTTP Response Received
Error Code 803: Incomplete HTTP Response Received
Error Code 608: Page not Decodable as Specified Content Encoding
Error Code 902: Network Errors Prevented Crawler from Contacting Server The five pages in question are all in fact perfectly working pages and are returning HTTP 200 codes. Is this a problem with the Moz crawler?1 -
Inbound Links - How accurate or up-to-date is Open Site Explorer?
When I type my domain into OSE I get a list of linking domains. Most of them I have seen before and I know were active last year. However, when I click through, many of these links have been removed (which is what I wanted/requested) and some of the pages return 404. Google (via link:www.mydomain.com) doesn't show these links as active either. My question is why does OSE show them? Thanks
Link Explorer | | neilmac0 -
Learn how to use Open Site Explorer to find opportunities in your links' anchor text. Get your Daily SEO Fix!
In today's Daily SEO Fix, Tori dives into using Open Site Explorer to find opportunities—particularly potential keyword ranking opportunities—in your links' anchor text. Watch "How to Use Open Site Explorer to Check Your Anchor Text" to get started! The Daily SEO Fix is an ongoing series of Moz tool tips and tricks in under 2 minutes. To watch all of our videos so far, and to subscribe to future ones, make sure to check out the Daily SEO Fix channel on YouTube!
Link Explorer | | MattRoney4 -
Dofollow link from Moz user profile still not showing in OpenSiteExplorer
Hi everyone! Refering to my last question http://moz.com/community/q/dofollow-link-from-moz-user-profile-not-showing-in-opensiteexplorer, the last update was on March 10 and I'm still searching my dofollow link from my Moz user profile http://moz.com/community/users/625691 after have 200 MozPoints, Opensiteexplorer only found the nofollow link from the icon, but nothing from the anchor text of my URL. Is possible that only take the first link to my blog and ignore the rest if it have the same href? 😞 Thanks! fTdwaO6.png
Link Explorer | | rubenalonsoes1 -
How can I get a Moz crawl report of 404 errors on my site
I have a Moz subscription and I see dead links on my website that link externally. Is there a Moz crawl report which will show me these 404 errors and which pages on my site those 404 links are on?
Link Explorer | | Marbanasin0