SEOMoz Crawler vs Googlebot Question
-
I read somewhere that SEOMoz’s crawler marks a page in its Crawl Diagnostics as duplicate content if it doesn’t have more than 5% unique content.(I can’t find that statistic anywhere on SEOMoz to confirm though).
We are an eCommerce site, so many of our pages share the same sidebar, header, and footer links. The pages flagged by SEOMoz as duplicates have these same links, but they have unique URLs and category names. Because they’re not actual duplicates of each other, canonical tags aren’t the answer. Also because inventory might automatically come back in stock, we can’t use 301 redirects on these “duplicate” pages.
It seems like it’s the sidebar, header, and footer links that are what’s causing these pages to be flagged as duplicates. Does the SEOMoz crawler mimic the way Googlebot works? Also, is Googlebot smart enough not to count the sidebar and header/footer links when looking for duplicate content?
-
Google bot definitely won't ding you for duplicate content based upon you sidebar, header, and footer links. I've managed several ecommerce sites and have never had this problem.
However, it may still ding you for duplicate content if you have several matching pages with you main column being the same.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
302 Redirect Question
After running a site crawl. I found two 302 redirects. The two redirects go from: site.com to www.site.com & site.com/products to www.site.com/products How do I fix the 302 redirect and change it to a 301 redirect? I have no clue where to start. Thanks.
Technical SEO | | Ryan_1320 -
Ajax Pagination in Magento Question
Hi, We just launched our new theme for Magento and my developer stated the pagination uses Ajax. Previously I had the developers set up rel prev/next for all our pages (categories/ecommerce site) that had multiples. He said it's not required with Ajax. Is this correct? Example: https://www.bestpricenutrition.com/whey.html and when you go to Page 2, the URL shows: https://www.bestpricenutrition.com/whey.html? I want to make sure these pages are set up correctly.
Technical SEO | | vetofunk0 -
Webmaster tools question
Hi all. I have a question regarding http vs https. I have an https site and was wondering how to tell google in Webmaster tools to combine and use https. I have setup all sites in Webmaster tools. Both www and non www for both http and https. I see where to set up the www vs the non www but don't quite understand how to do the https part. I want all traffic to: https://www-creative -technology-solutions.com Thanks
Technical SEO | | twoacejr0 -
Https vs http two different domains?
If i visit mywebsite.com.au, www.mywebsite.com.au and http://www.mywebsite.com.au - i get one website BUT if I visit https://www.mywebsite.com.au I get a different website - I also get a untrusted website warning The logo in the bottom right of the https: website is the name of the webdesigner where the website is hosted. Is this a normal practice?
Technical SEO | | GardenBeet0 -
HTTPS vs HTTP
A few months ago we switched all product page urls in our e-commerce site to https: Recently working on the site I was appalled at how slow our pages were loading and on investigating further with our hosting partner they advised to switch back to http instead of https for all of the site content to help page speed. We obviously still need to use https in the cart and check-out. I think that that Google will be pushing all commerce web pages to https but for now I need to improve page load speed. Will the switch back from https to http impair our keywords? https://www.silverandpewtergifts.com/
Technical SEO | | silverpewter0 -
301 vs 500 Errors for discontinued products
I have a client that has a around 15 "products" (they are pages containing details of the products rather than e-Commerce products) that have been discontinued. The client has suggested 301s but unless the alternative products are replacement products am I correct that we should be using a 500 error?
Technical SEO | | MentorDigital0 -
Wordpress question
I was curious when i run an OSE report on certain websites and their name.wordpress.com shows up with a PA of whatever and a DA of 100. But when I created my wordpress site and post on it, it only has a PA and DA of 1. is this because SEOmoz has not indexed it yet? It is a month old. http://shiftinsurance.wordpress.com/ Can anyone help pls?
Technical SEO | | greasy0 -
Home Page Indexing Question/Problem
Hello Everyone, Background: I recently decided to change the preferred domain settings in WM Tools from the non www version of my site to the www version. I did this because there is a redirect from the non www to the www and I've built all of my internal links with the www. Everything I read on SEO Moz seemed to indicate that this was a good move. Traffic has been down/volatile but I think it's attributable mostly to a recent site change/redesign. Having said that the preferred domain change did seem to drop traffic an additional notch. I made the move two weeks ago. Here is the question: When I google my site, the home page shows up as the site title without the custom title tags I've written. The page that displays in the SERP is still the non www version of the site. a site:www.mysite.com search shows an internal page first but doesn't return the home page as a result. All other pages pop up indexed with the www version of the page. a site:mysite.com (notice lack of www) search DOES SHOW my home page and my custom title tags but with a non www version of the page. All other pages pop up indexed with the www version of the page. Any one have thoughts on this? Is this a classic example of waiting on Google to catch up with the changes to my tiny little site?
Technical SEO | | JSOC0