Crawl diagnostic issue?
-
I'am sorry if my English isn't very good, but this is my problem at the moment:
On two of my campagnes I get a weird error on Moz Analytics:
605 Page Banned by robots.txt, X-Robots-Tag HTTP Header, or Meta Robots Tag
Moz Analytics points to an url that starts with: http:/**/None/**www.????.com. We don't understand how Moz indexed this non-existing page that starts with None? And how can we solve this error?
I hope that someone can help me.
-
Hi MOZ,
I'am sorry that I have not previously responded. The problem has been solved. Thanks!
Also thanks to Pixel for the response!
Greetz,
Sam
-
Hi Nettt!
I apologize for any confusion and can confirm there is no issue on your side. One of our crawlers failed causing some campaigns crawled on Aug 29th attempt to follow the strange /None/ URL you are seeing in your diagnostics. I've submitted a re-crawl for all of your campaigns affected so you should see updated data by this Friday.
Hope this helps!
-
"I have checked the URL, and it is not our own website that has the error."
is this the problem?
Could you take a screen grab of the problem it might help better.
-
Thanks for the respons, Pixelbypixel!
I have checked the URL, and it is not our own website that has the error.
We have checked the robots.txt and it should not cause any problem. We have n't recently changed it.
I Think that Moz is causing it, but I am not sure..
-
Is the URL correct on Moz pro? It also seems like your robots.txt is blocking Moz which you may want to look into.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Infinite scrolling issue?
Hi Guys, Reviewing this E-commerce page - https://tinyurl.com/ybjjwr65 Based on this Google article: https://webmasters.googleblog.com/2014/02/infinite-scroll-search-friendly.html It mentions: Make sure that you or your content management system produces a paginated series (component pages) to go along with your infinite scroll. How would you check this, is there a tool to conduct this test? Cheers.
Intermediate & Advanced SEO | | kayl870 -
How do I know if I am correctly solving an uppercase url issue that may be affecting Googlebot?
We have a large e-commerce site (10k+ SKUs). https://www.flagandbanner.com. As I have begun analyzing how to improve it I have discovered that we have thousands of urls that have uppercase characters. For instance: https://www.flagandbanner.com/Products/patriotic-paper-lanterns-string-lights.asp. This is inconsistently applied throughout the site. I directed our website vendor to fix the issue and they placed 301 redirects via a rule to the web.config file. Any url that contains an uppercase character now displays as a lowercase. However, as I use screaming frog to monitor our site, I see all these 301 redirects--thousands of them. The XML sitemap still shows the the uppercase versions. We have had indexing issues as well. So I'm wondering what is the most effective way to make sure that I'm not placing an extra burden on Googlebot when they index our site? Should I have just not cared about the uppercase issue and let it alone?
Intermediate & Advanced SEO | | webrocket0 -
Http - Https Issue
Hey there Mozzers, I have a site that few months ago went from being http - https. All the links redirect perfect but after scanning my site with Screaming Frog i get a bunch of 503 errors. After looking into my website I see that a lot of links in my content and menu have as a link the http url. For example my homepage has content that interlinks to the http version of the site. And even though when I test it it redirects correctly after scanning with Screaming frog it reports back as 503. Any ideas what's going on? Thanks in advance
Intermediate & Advanced SEO | | Angelos_Savvaidis0 -
Dealing with close content - duplicate issue for closed products
Hello I'm dealing with some issues. Moz analyses is telling me that I have duplicate on some of my products pages. My issue is that: Concern very similar products IT products are from the same range Just the name and pdf are different Do you think I should use canonical url ? Or it will be better to rewrite about 80 descriptions (but description will be almost the same) ? Best regards.
Intermediate & Advanced SEO | | AymanH0 -
Issues with Sub domains for dealers
I'm starting a new SEO project and am feeling a little overwhelmed due to the scale of it. I am not sure where to start and hope that someone has some ideas. Thousands of dealer websites reside as sub domains on gravelymower.com/ (e.g. http://quality-mowers.gravelymower.com/) The particular sub domain mentioned above is not showing up at all for any searches and is not cached by Google: http://webcache.googleusercontent.com/search?q=cache:http://quality-mowers.gravelymower.com/ I realize that pretty much zero SEO best practices are followed on page and the location is not on the page, but why is this sub domain not even being indexed by Google? Any help is appreciated. Thanks!
Intermediate & Advanced SEO | | BridgelineDigital880 -
Rel canonical issues on wordpress posts
Our site has 500 rel canonical issues. This is the way i understand the issues. All our blog posts automatically include a rel=canonical to themselves.
Intermediate & Advanced SEO | | acs111
eg a blog about content marketing has: Should this tag point to one of the main pages instead so the link juice is sent back to our home page?0 -
Will the Canonical tag fix this issue?
I recently joined promoz and I've been busy working through the issues raised brought to light during the crawls of our Magento site, www.unitedbmwonline.com. One of many issues were the 10,000+ Duplicate Page Titles which I believe are the result of not using Canonical tags when setting up the store. This is now corrected and hopefully I'll see a significant drop in this value after this next crawl. Am I correct in this assumption? Cheers, Steve
Intermediate & Advanced SEO | | SteveMaguire0
