Why so many crawl errors?
-
Our site is showing it has a ton of crawl errors in the back end, mostly concerning duplicate content within our blog. The content is unique however. We know this for certain because it's done in house or put together by some of the freelance writers we work with.
The site is for an RV dealership and we're using a template-based system from a well known company.
Any ideas on what may be causing this?
-
Hi Darrin,
Can you provide your website URL or a few examples of the URLs that are getting the duplicate content error?
If you can't provide them, I would recommend referencing http://www.seomoz.org/help/fixing-crawl-diagnostic-issues which says the following about fixing duplicate page content:
"Duplicate Page Content
Duplicate Content means there are pages that are identical (or nearly identical) to content on other pages of your site, which can force your pages to unnecessarily compete with each other for rankings.
Here are some things you can do about duplicate pages:
Delete content that is similar on each page.
Add some new and unique content to each page that is on the report. This can be done by adding more information, ideas, product descriptions, or anything that can make it differ from other pages on the domain.
You can also add a rel=canonical to one of the duplicate pages. Here are a few ways to do this:
Add a rel="canonical" link in between the and elements. This should be done on the version of the page you want to be ranking or that non-canonical version of the two (or multiple) pages.
To specify a canonical link to the page http://www.seomoz.org/blog.php?item=seomoz-iscool, create a element that looks like this: <link rel="canonical"href="<a href="http://www.seomoz.com/blog.php?item=seomoz-iscool">http://www.seomoz.com/blog.php?item=seomoz-iscool"/></link rel="canonical"href="<a>
Copy this link into the section of all non-canonical versions of the page, such as http://www.seomoz.com/blog.php?item=seomoz-iscool&sort=fun.
Keep in mind that that canonicals will stop the pages from ranking against each other, but they will still show up as duplicate content from a UI perspective, so we will still count them as duplicate."
Thanks,
Mike
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Redirect and Redirect Error in Moz Crawl
Hello, We have a wordpress blog attached to our magento website located at domain.co.uk/blog/ Moz was coming back showing we had multiple page versions on show (http and https) So i updated the htaccess file to what is below. This has fixed most of the errors, however the homepage is being a little tricky. Moz is now saying that the page is redirecting and redirecting again http://www.domain.co.uk/blog to
On-Page Optimization | | ATP
http://www.domain.co.uk/blog/ to
https://www.domain.co.uk/blog/ BEGIN WordPress <ifmodule mod_rewrite.c="">RewriteEngine On
RewriteBase /blog/</ifmodule> RewriteCond %{HTTPS} !=on
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301] RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /blog/index.php [L] END WordPress Within wordpress settings the urls are set up as follows Wordpress Address URL: https://www.domain.co.uk/blog Site Address URL: https://www.domain.co.uk/blog i tried to add a trailing / to these but it gets automatically removed. So i am assuming that wordpress is serving up https://www.domain.co.uk/blog **RewriteBase /blog/ **is re-directing it to / then my https rewrite is re-directing it again I am not sure where exactly to fix this, could anybody advise? Many thanks,0 -
Is the meta title written only for google (try to stuff in as many keywords as possible) or is there a user experience aspect as well?
Is the meta title written only for google (try to stuff in as many keywords as possible) or is there a user experience aspect as well?
On-Page Optimization | | whiteonlySEO0 -
How many product subcategories are ok?
Let's say I have a sea glass ankle bracelet. On my site, my main keyword is "Sea Glass Jewelry" and have ranked relatively well for this, but this main page has over 200 products in it. I thought that if the URL has the keywords in it, it would be beneficial. I also have a section for all my bracelets, so it would be there and then, a more specific ankle bracelets category. So, technically, an ankle bracelet will show up 3x. Sea Glass Jewelry (all products go here) Bracelets (all bracelets go here) Ankle Bracelets (only ankle bracelets) The URL is only attached to the main category so to speak. If you click the ankle bracelets category, the url will still revert back to the original main category: seaglassjewelry/sterlinganklebracelet so I don't believe there is duplicate content. I have had my domain for years and it has ranked well until someone hacked into my site 2 years back. I have never been able to recover from this loss. Since then, I have tried to optimize my site, but nothing seems to be working and I just want to make sure that I am not hurting my ranking by doing this. Can someone confirm this is the best way to do it or make a suggestion? Thank you.
On-Page Optimization | | tiffany11030 -
Too many page links warning... but each link has canonical back to main page? Is my page OK?
The Moz crawl warns me many of my pages have too many links, like this page http://www.webjobz.com/jobs/industry/Accounting ...... has 269 links but many of the links are like this /jobs/jobtitles/Accounting?k=&w=3&hiddenLocationID=463170&depth=2 and are used to refine search criteria.... when you click on those links they all have a canonical link back to http://www.webjobz.com/jobs/industry/Accounting Is my page being punished for this? Do I have to put "no follow" tags on every link I do not want the bots to follow and if I do so is Roger (moz bot) not going to count this as a link?
On-Page Optimization | | Webjobz0 -
Why do I suddenly have so many more page errors?
I've been working diligently to remove page errors that appear after each scan of my website. I was down to 3 or 4 errors after the last crawl, now this week I'm getting 80 plus. I haven't changed anything since last week (page titles, etc.) so what's up?
On-Page Optimization | | janettapp0 -
Crawling - Blue Notice - Canonical
Hi, I have 270x blue notices within crawl diagnostics in SEOMoz Pro labelled rel=canonical. My site has the rel=canonical tag set-up as I was advised to do so. See www.comparecurrency.co.uk Are these notices suggesting I have to remove the tag? Can somebody please explain this notice to me .. Thanks Olly
On-Page Optimization | | ojkingston0 -
How to Resolve Google Crawling Issues for My eCommerce Website?
I want to resolve Google crawling issues for my eCommerce website. My website is as follow. http://www.vistastores.com/ Google have crawled only 97 webpages from my website. My website is quite old. (~More than 6 months) But, Google have indexed only 97 webpages. I have created one campaign over SEOmoz tool and found some errors over there. So, I just assumed that due to it Google did not crawled my website. But, I have created one another campaign for my competitor website to know actual status and reason behind it. I found that, my competitor website have more error compare to me but, Google have crawled maximum pages compare to me. So, What is reason behind it? How can I improve my crawling rate and index maximum webpages to Google? [6133009604_af85d29730_b.jpg](img src=) 6133009604_af85d29730_b.jpg 6133009604_af85d29730_b.jpg 6139706697_4e252fdb82_b.jpg
On-Page Optimization | | CommercePundit0 -
Too many On-Page Links on a WP based Website
Hi, I've already browsed through various of the Q&As on the "too many On-Page links" issue, but I would really need some advice concerning a WP Site with a dropdown navigation. As outlined in the on-page report, every site has about 180 outgoing links, which pretty much is the number of site featured in the navigation. Even though the 100 link limit is somewhat outdated I'm still worried about the distribution of linkjuice from the starting page and how Google perceives the importance of the various pages. Would it make sense to adapt the structure of the navigation, so that the starting page only links to the 5 category pages and the category pages only link to the detail pages they contain? The site has good rankings for several pages and I assume that Google can tell that the large number of links is caused by the navigation. But with every page having appr. 180 links it may be difficult for Google to tell, which of those pages are the most important regarding internal link structure... Looking foward to your opinion and insights! Cheers, Chris
On-Page Optimization | | adwordize0