Why so many crawl errors?
-
Our site is showing it has a ton of crawl errors in the back end, mostly concerning duplicate content within our blog. The content is unique however. We know this for certain because it's done in house or put together by some of the freelance writers we work with.
The site is for an RV dealership and we're using a template-based system from a well known company.
Any ideas on what may be causing this?
-
Hi Darrin,
Can you provide your website URL or a few examples of the URLs that are getting the duplicate content error?
If you can't provide them, I would recommend referencing http://www.seomoz.org/help/fixing-crawl-diagnostic-issues which says the following about fixing duplicate page content:
"Duplicate Page Content
Duplicate Content means there are pages that are identical (or nearly identical) to content on other pages of your site, which can force your pages to unnecessarily compete with each other for rankings.
Here are some things you can do about duplicate pages:
Delete content that is similar on each page.
Add some new and unique content to each page that is on the report. This can be done by adding more information, ideas, product descriptions, or anything that can make it differ from other pages on the domain.
You can also add a rel=canonical to one of the duplicate pages. Here are a few ways to do this:
Add a rel="canonical" link in between the and elements. This should be done on the version of the page you want to be ranking or that non-canonical version of the two (or multiple) pages.
To specify a canonical link to the page http://www.seomoz.org/blog.php?item=seomoz-iscool, create a element that looks like this: <link rel="canonical"href="<a href="http://www.seomoz.com/blog.php?item=seomoz-iscool">http://www.seomoz.com/blog.php?item=seomoz-iscool"/></link rel="canonical"href="<a>
Copy this link into the section of all non-canonical versions of the page, such as http://www.seomoz.com/blog.php?item=seomoz-iscool&sort=fun.
Keep in mind that that canonicals will stop the pages from ranking against each other, but they will still show up as duplicate content from a UI perspective, so we will still count them as duplicate."
Thanks,
Mike
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can the Lightboxes on My Site be Crawled?
I'm trying to optimize my site, but I have lightboxes and I don't know if they are visible to the search engines. If they aren't, could you suggest something that I could do? THANK YOU so much!!!!! My site is lymphexpo.com
On-Page Optimization | | bosleypalmer0 -
Single Page on my client's website is not crawling and indexing new changes. What could be the possible reason?
I made several changes on client's website on different pages, changed titles, add content on few pages, moved blog from subdomain to sub directory. Everything is crawled but there is one page on the website (not part of the blog) that isn't getting crawled in Google and picking up changes. The last crawl of the website is 2 days back whereas that page was last crawled on 30th sep. I just wanted to know the possible reasons and has anyone encountered this before?
On-Page Optimization | | MoosaHemani0 -
Should a company worry about how many domains it maps to the same home page?
I seem to be at logger heads with developers regarding domain mapping. The scenario: I have a company with one site on a primary domain name, but all the other domains they own are mapped using a tool provided by their hosting vendor. But. what I see is a keyword loaded domain that shows it has been 'mapped' to the primary domain, but you can type into the browser this keyword loaded domain and it will serve up in your browser that same home page you see on the PRIMARY DOMAIN. So, picture this - you are looking at the home page on wwww.keyworddomain.com and see the same home page as www.primarycompanydomain.com - but if you select anything from the menu at www.keyworddomain.com you will be taken immediately to www.primarycompanydomain.com/page-you-selected I just get a feeling this is not right as I can search Google for www.keyworddomain.com and Google lists the site home page on that domain. But when I click through from the listed result, I am taken to www.primarycompanydomain.com which is ideally where I want to be and I would want Google to focus on this domain, and I have told it to do so within the feature included within Google Webmaster Tools. The developers say there is nothing wrong. There argument - why would a hosting company provide this domain mapping feature if it was not best practice. My argument - but Google is listing that domain URL (www.keyworddomain.com) despite the fact it takes me through to www.primarycompanydomain.com - will Google not think this strange despite me telling it via GWMT that www.primarycompanydomain.com is the one and only domain I am working on. Tell me if I am going mad or not, and who is right and who is wrong. Appreciate all your answers.
On-Page Optimization | | ICTADVIS0 -
Big problem with my new crawl report
I am owner of small opencart online store. I installed http://www.opencart.com/index.php?route=extension/extension/info&extension_id=6182&filter_search=seo. Today my new crawl report is awful. The number of errors is up by 520 (30 before), up with 1000 (120 before), notices up with 8000 (1000 before). I noticed that the problem is with search. There is a lot duplicate content in search only. What to do ?
On-Page Optimization | | ankali0 -
I have more pages in my site map being blocked by the robot file than I have being allowed to be crawled. Is Google going to hate me for this?
Using some rules to block all pages which start with "copy-of" on my website because people have a bad habit of duplicating new product listings to create our refurbished, surplus etc. listings for those products. To avoid Google seeing these as duplicate pages I've blocked them in the robot file, but of course they are still automatically generated in our sitemap. How bad is this?
On-Page Optimization | | absoauto0 -
Reducing number crawl-able links?
Hello, I just like to ask for best practice when it comes to reduce number of internal links on a site with a mega menu. Since the mega menu lists all categories and all their subcategories it creates a problem when all categories are linking to all categories directly.. Would the method below reduce the number of links and preventing the link juice flowing directly from category to category? [(link built with JavaScript and the html5 "data-" attribute) Thinking of using these links to categories in the menu not directly below the parent category.](#)
On-Page Optimization | | AJPro0 -
PANDA Attack: Too many on page links
Hey guys! I have a bit of a dilemma...one of my sites got hit by Panda 😞 The content itself contains about 10 links, however since the site is a process directory, at the bottom of the page you will find that the visitor can also browse process directory by name or page and then beneath this there are 80 links :s My concern is that if i remove this I will lose internal link juice! HELP! What approach should I take? I was thinking of either reducing the number of links OR hiding it by using Java ORRRR removing the links entirely. Advice anyone? This is a page as an example: http://www.processlibrary.com/directory/files/csrsc/25349/ All pages are like this!
On-Page Optimization | | OrangeGuys0 -
My report indicated that I have 340 crawl warnings. Not sure how to fix them. Please provide links on where I need to go to fix them.
My report indicated that I have 340 crawl warnings. Not sure how to fix them. Please provide links on where I need to go to fix them. http://pro.seomoz.org/campaigns/95663/issues#notice-issues
On-Page Optimization | | cyaindc0