SEOMoz Crawler vs Googlebot Question
-
I read somewhere that SEOMoz’s crawler marks a page in its Crawl Diagnostics as duplicate content if it doesn’t have more than 5% unique content.(I can’t find that statistic anywhere on SEOMoz to confirm though).
We are an eCommerce site, so many of our pages share the same sidebar, header, and footer links. The pages flagged by SEOMoz as duplicates have these same links, but they have unique URLs and category names. Because they’re not actual duplicates of each other, canonical tags aren’t the answer. Also because inventory might automatically come back in stock, we can’t use 301 redirects on these “duplicate” pages.
It seems like it’s the sidebar, header, and footer links that are what’s causing these pages to be flagged as duplicates. Does the SEOMoz crawler mimic the way Googlebot works? Also, is Googlebot smart enough not to count the sidebar and header/footer links when looking for duplicate content?
-
Google bot definitely won't ding you for duplicate content based upon you sidebar, header, and footer links. I've managed several ecommerce sites and have never had this problem.
However, it may still ding you for duplicate content if you have several matching pages with you main column being the same.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Backlink quality vs quantity: Should I keep spammy backlinks?
Regarding backlinks, I'm wondering which is more advantageous for domain authority and Google reputation: Option 1: More backlinks including a lot of spammy links Option 2: Fewer backlinks but only reliable, non-spam links I've researched this topic around the web a bit and understand that the answer is somewhere in the middle, but given my site's specific backlink volume, the answer might lean one way or the other. For context, my site has a spam score of 2%, and when I did a quick backlink audit, roughly 20% are ones I want to disavow. However, I don't want to eliminate so many backlinks that my DA goes down. As always, we are working to build quality backlinks, but I'm interested in whether eliminating 20% of backlinks will hurt my DA. Thank you!
Technical SEO | | LianaLewis1 -
Link to AMP VS AMP Google Cache VS Standard page?
Hi guys, During the link building strategy, which version should i prefer as a destination between: to the normal version (php page) to the Amp page of the Website to the Amp page of Google Cache The main doubt is between AMP of the website or standard Version. Does the canonical meta equals the situation or there is a better solution? Thank you so mutch!
Technical SEO | | Dante_Alighieri0 -
Googlebots and cache
Our site checks whether visitors are resident in the same country or live abroad. If it recognises that the visitor comes from abroad, the content is made more appropriate for them. Basically, instead of encouraging the visitor to come and visit a showroom, it tells them that we export worldwide. It does this by IP checking. So far so good! But I noticed that if I look at cached pages in Google's results, that the cached pages are all export pages. I've also used Google Webmaster Tools (Search Console) and rendered pages as Google - and they also render export pages. Does anybody have a solution to this?
Technical SEO | | pulcinella2uk
Is it a problem?
Can Google see the properly (local - as in UK) version of the site?0 -
Questions about the Sandbox and 301 Redirects
Does the sandbox still exist? What if you have a brand new URL and do a 301 redirect from another website because the name of the service business changed? Thanks for any insight and help.
Technical SEO | | SDSLaw0 -
Redirect Impact - Moving From SEOmoz to Moz
Hey Guys, This has been on my mind ever since the big announcement, so today I did some searching around for some posts/talk about what the impact of their full site redirect has been for them and didn't find anything. Have they posted on this or are there any threads that I'm missing out on? I'd love to hear more about what the impact has been or any general thoughts/insights people may have. Thanks!
Technical SEO | | TakeLessons
Jon1 -
Request to seomoz staff
Hi SeoMoz personnel I am pro member for few months here in the community. You have all googles in the rank tracking tool and the keyword analysis tool. Except from one. You dont have the google.com.cy and I was wondering if is problem to add it. If no its ok I track my results on the google.gr and google.com. But I would love if you could add it. Is also one plus for me when I promote the software to my local clients.... Thank you
Technical SEO | | nyanainc0 -
Sub pages Vs subdomain Pagerank flow.
I Have Question about Pagerank flow:
Technical SEO | | tommytai
If ihave a site :domain.com and i have 2 solutions like:
Solution #1: Quote: | domain.com/blog and domain.com/video if i try to do :
Root domain only link to
domain.com/video and
domain.com/blog | Solution #2: Quote: | Root domain only link to
video.domain.com and
blog.domain.com | So <acronym title="Google PageRank">Pr</acronym> domain.com/blog = <acronym title="Google PageRank">Pr</acronym> blog.domain.com ?
and <acronym title="Google PageRank">PR</acronym> domain.com/video = <acronym title="Google PageRank">Pr</acronym> video.domain.com ? And don't know why a subdomain of blogspot or Wordpress ranking easier than a new domain like:keyword.wordpress.com and keyword.com So What Wordpress pass to keyword.wordpress.com ?0 -
Subdirectories vs subdomains
Hi SEO gurus 🙂 Anyone has input on what's better? blog.domain.com vs domain.com/blog store.domain.com vs domain.com/store etc I think the subdir (/xyz) will concentrate authority on the same subdomain so should be better? However sometimes it is tidier on the server to maintain online stores or blogs in a separate strucutre so subdomains work better in that sense. I just want to make sure that doesn't affect SEO? Cheers!
Technical SEO | | hectorpn0