SEOMoz Crawler vs Googlebot Question
-
I read somewhere that SEOMoz’s crawler marks a page in its Crawl Diagnostics as duplicate content if it doesn’t have more than 5% unique content.(I can’t find that statistic anywhere on SEOMoz to confirm though).
We are an eCommerce site, so many of our pages share the same sidebar, header, and footer links. The pages flagged by SEOMoz as duplicates have these same links, but they have unique URLs and category names. Because they’re not actual duplicates of each other, canonical tags aren’t the answer. Also because inventory might automatically come back in stock, we can’t use 301 redirects on these “duplicate” pages.
It seems like it’s the sidebar, header, and footer links that are what’s causing these pages to be flagged as duplicates. Does the SEOMoz crawler mimic the way Googlebot works? Also, is Googlebot smart enough not to count the sidebar and header/footer links when looking for duplicate content?
-
Google bot definitely won't ding you for duplicate content based upon you sidebar, header, and footer links. I've managed several ecommerce sites and have never had this problem.
However, it may still ding you for duplicate content if you have several matching pages with you main column being the same.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Question Regarding Website Architecture
Hello All, Our website currently has a general solutions subdirectory, which then links to each specific solution, following the path /solutions/ => /solutions/solution1/. As our solutions can be quite complex, we are adding another subdirectory to target individuals by profession. I would like to link from our profession pages to the varying solutions that help. As both subdirectories will be top level pages in the main menu, would linking from our professions to **solutions **be poor architecture? In this case the path would look like: /professions/ => /professions/profession1/ => /solutions/solution1/. Thanks!
Technical SEO | | Tom3_150 -
Google Results Title vs My Page Title
I'm having some trouble with my titles of a new site, it has been online for around two months now and i'm getting weird titles from most indexed pages. Since my site is focused on finding courses, the course title format is the following: URL: https://www.maseducacion.com/estudios/programacion-curricular--tecnigrap-2982
Technical SEO | | JoaoCJ
My Title: Course - Institute | Mybrand
Google Search Title: Course - Institute | Mybrand - Educativa Half of my results have that word at the end, don't know where it comes from, that word is only included in two links. Any idea on how to fix it?0 -
H1 Tags the same as Title Tags and other meta questions
I run an ecom store that has about 800 live products. When everything got set up, no one set up the title tags correctly. So I am going through to update them in bulk. What I was going to do was to take the product name (which serves as the H1 tag), use that with a postfix | CompanyName. If length is an issue I trim it down. But the question is, will having essentially duplicate information in here be an issue? Also, when someone was setting up meta descriptions, they often used basically the product name or a half sentence. Would it be better to remove the descriptions and allow google to decide? I even had some that were literally just the brand name of the product, which I already removed.
Technical SEO | | ShockoeCommerce0 -
Can Googlebot crawl the content on this page?
Hi all, I've read the posts in Google about Ajax and javascript (https://support.google.com/webmasters/answer/174992?hl=en) and also this post: http://moz.com/ugc/can-google-really-access-content-in-javascript-really. I am trying to evaluate if the content on this page, http://www.vwarcher.com/CustomerReviews, is crawlable by Googlebot? It appears not to be. I perused the sitemap and don't see any ugly Ajax URLs included as Google suggests doing. Also, the page is definitely indexed, but appears the content is only indexed via its original source (Yahoo!, Citysearch, Google+, etc.). I understand why they are using this dynamic content, because it looks nice to an end-user and requires little to no maintenance. But, is it providing them any SEO benefit? It appears to me that it would be far better to take these reviews and simply build them into HTML. Thoughts?
Technical SEO | | danatanseo0 -
Canonical question
I have at least three duplicate main pages on my website: www.augustbullocklaw.com www.augustbullocklaw.com/index augustbullocklaw.com I want the first one, www.augustbullocklaw.com to be the main page. I put this code on the index page and uploaded it to my site: http://www.augustbullocklaw.com/canonical-version-of-page/" rel="canonical" /> This code now appears on all three pages shown above. Did I do this correctly? I surmise that www.augustbullocklaw.com is pointing to itself. Is that ok? I don't know how to take the cononical code off the page that is the page I want to be the main page. (I don't know how to remove it from www.augustbullocklaw.com, but leave it on www.augustbullocklaw.com/index and augustbullocklaw.com) Thanks
Technical SEO | | Augster990 -
X-cart page crawling question.
I have an x-cart site and it is showing only 1 page being crawled. I'm a newbie, is this common? Can it be changed? If so, how? Thanks.
Technical SEO | | SteveLMCG0 -
Adding Google + to SEOmoz
I wanted to add my google + signature to every post I make on SEOmoz and I think every user should do the same... Two reasons why... Google helps our existence so we should help theirs. If someone likes what I wrote or vice versa we should be able to follow each other in a simple click. In my opinion all blogs forum posts etc... should Lead to a user not a website, this will prevent spam and help people network. In other words blog spammers and forum spammers will be SOL (Which they all ready are lol).
Technical SEO | | SEODinosaur0 -
Frustrating Local Seo Question
I have just lost the top stop on Google Places (Pack 7) for one of my clients. The company that took it came out of nowhere to take the top stop. Now I don't mind getting out ranked when someone has worked hard to out rank me but what is frustrating is this site scores terrible across the board on just about every score that is supposed to determine on and off page ranking factors. The only place it outscores is in the link department but the site is so new there is no data available in OSE. The site has 0's and 1's for PA, MR and MT plus the keyword (kitchen remodeling) that it is taking the top spot for scores and F in the SEOMoz on page report card while the page that I optimized for this very same keyword scores an A. Plus this site has zero citations according to Bright Local Data. What Gives? Here is a little bit of data: Keyword: Kitchen Remodeling My Clients website is http://www.tandmkitchens.com Competitors website: http://www.njkitchendesigns.com I'm kicking his butt in every category except Domain Age and inbound link total, his inbound link total is 409 but it's only from 2 domains.
Technical SEO | | fun52dig
Any thoughts on how this is possible would be greatly appreciated. Thanks all and Merry Christmas!
Gary1