Has any on else experienced a spike in crawl errors?
-
Hi,
Since the last time our sites were crawled in SEOmoz they are all showing a spike in Errors. (Mainly duplicate page titles and duplicate content).
We haven't changed anything to the structure of the sites but they are all using the same content management system.
The image is an example of what we are witnessing for all our sites based on the same system.
Is anyone else experiencing anything similar? or does anyone know of any changes that SEOmoz has implemented which may be affecting this?
Anthony.
-
Thanks for all your replies.
We haven't changed anything on any of the sites. We use our own CMS which has not changed either.
Webmaster tools doesn't show the same errors as SEOMoz.
We appear to be in the same situation as Mike. We know that we have duplicate titles and content but we have taken care of our duplicate issues using canonical and no index tags, which drastically reduced our errors. For some reason SEOmoz doesn't seem to have paid heed to them on it's latest crawl.
Thanks Mike. At least we are not on our own.
Maybe I should see if this is rectified after the next SEOMoz crawl before I pursue this any further?
-
This leads me to a problem then. As per Dave (the author of the article), "using canonical tags will result in duplicate errors being suppressed. If one page refers to another as a duplicate, than that pair will not be reported as duplicates. Also, if two pages both refer to the same third page as their canonical, then they will not be reported as duplicates of each other, either."
But now that this change has gone into effect I have 2000+ more duplicate content errors appearing and they are all pages with rel="canonical" pointing to the original page. So, as he stated earlier in the post this has caused "the most negative customer experience we anticipate: having a behind-the-scenes change of our duplicate detection heuristic causing a sudden rash of incorrect "duplicate page" errors to appear for no apparent good reason."
Is this something that will eventually correct itself or is this something that will need tweaking of the new detection method?
-
We did change the way we detected duplicate content earlier this month. Here's a blog post about it at http://www.seomoz.org/blog/visualizing-duplicate-web-pages.
Hope this helps explain things for you! Let me know if you have any more questions.
-
I saw a huge spike after the last crawl. In my case, the canonicals we set on our site months ago to handle some duplicate content issues appear not to be seen by Seomoz's crawl. Though when I check for duplicate title & meta issues in Webmaster Tools I don't see the offending pages that SEOMoz is showing me. That leads me to believe something is happening with either how the SEOMoz system is reporting or how their bot is crawling.
-
What CMS are you using?
Did you add any menus to your home or sub-pages (ie footer menus or anything like that?)
Have you gone into the Errors and see what pages are being duplicated?
Have you implemented rel=canonical on the pages?
Is your CMS creating Titles for you or are they manually created?
Have you checked WMT to see if the duplicate issue is there too? (under html improvements)
-
No spikes in either of our campaigns.
You said that yours were related to duplicate page titles / content which likely means your CMS is generating duplicate pages. Could be related to reviews, sorting, comments etc..
Have had a chance to research the errors and see if those pages actually exist? We had an issue with Oscommerce and page sorting causing this same problem, we fixed it by implementing rel canonical tags.
Hope that helps
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do I exclude my blog subfolder from being crawled with my main domain (www.) folder?
I am trying to setup two separate campaigns for my blog and for my main site. While it is easy enough to do through the wizard the results I am getting for my main site still include pages that are in my blog sub folder. Please Advise!
Moz Pro | | sameufemia0 -
How to solve duplicate page title & content error
I got lot of errors in Duplicate page title - 5000 Here the result page is same and content is also same,but it differs only with page no in meta title Title missing error In seomoz report i got empty msg - title,meta desc,meta robots,meta refresh But if i check the link which i got error it shows all meta tags..we have added all meta tags in our site..But i dont no why i got title missing error . 404 error In this report,if i click the link which i got error, it goes to main page of our site. But the url differs. eg: The error link is :www.example.com/buy/requirement-2-0-inmumbai-property it automatically goes to www.example.com page Let me know how to solve these issues.
Moz Pro | | Rajesh.Chandran0 -
Why would my site return an error when using Open Site Explorer to crawl it?
I have built several new sites over the last few months for others, but recently built a new one for myself. I have gone through most of the checklists from this site to address on-page SEO, and now I am looking at link building. When using Open Site Explorer, I receive an error saying that no information about the URL is available, even when I add competitor sites. Wondering if this is a common issue and if there is a convenient remedy? thanks!
Moz Pro | | MindSpark0 -
Increase of 404 error after change of encoding
Hello, We just have launch a new version of our website with a new utf-8 encoding. Thing is, we use comma as a separator and since the new website went live, I have a massive increase of 404 error of comma-encoded URL. Here is an example : http://web.bons-de-reduction.com/annuaire%2C321-sticker%2Csite%2Cpromotions%2C5941.html instead of : http://web.bons-de-reduction.com/annuaire,321-sticker,site,promotions,5941.html I check with Screaming Frog SEO and Xenu, I can't manage to find any encoded URL. Is anyone have a clue on how to fix that ? Thanks
Moz Pro | | RetailMeNotFr0 -
Crawl Errors from URL Parameter
Hello, I am having this issue within SEOmoz's Crawl Diagnosis report. There are a lot of crawl errors happening with pages associated with /login. I will see site.com/login?r=http://.... and have several duplicate content issues associated with those urls. Seeing this, I checked WMT to see if the Google crawler was showing this error as well. It wasn't. So what I ended doing was going to the robots.txt and disallowing rogerbot. It looks like this: User-agent: rogerbot Disallow:/login However, SEOmoz has crawled again and it still picking up on those URLs. Any ideas on how to fix? Thanks!
Moz Pro | | WrightIMC0 -
Campaign crawl re - schedule
Hello, On the last crawl of a website of mine, seomoz pointed out about 1500 errors (ouch!) on my site. I have made some corrections and i just want to see if they are at the right way but the next crawl is in a week. Is there any way so i can force a crawl before the scheduled date? Thanks!
Moz Pro | | Tz_Seo0 -
Excluding parameters from seomoz crawl?
I'm getting a ton of duplicate content errors because almost all of my pages feature a "print this page" link that adds the parameter "printable=Y" to the URL and displays a plain text version of the same page. Is there any way to exclude these pages from the crawl results?
Moz Pro | | AmericanOutlets0 -
Crawl Issues
My website - qtmoving.com - has 26 articles and when the SEOmoz did a crawl it only found 13 articles. Can someone please give me some insight as to why not all pages are being crawled.
Moz Pro | | CohesiveMarketing0