SeoMoz crawler giving false positives?
-
SeoMoz crawler indicated a few times that my site has a duplicate home page error (http://mysite.com and www.mysite.com)
I eliminated the the couple remaining internal links that pointed to http://mysite on a couple pages (all other internal links point to http://www.mysite.com)
I ran the crawl again and it said no errors this time. I naturally thought the duplicate page error problem was fixed.
However this morning I got the regularly scheduled crawl report from SeoMoz that said again I have those duplicate error pages. No changes were made to any of my site's pages between the crawls.
That makes me wonder if the crawler is providing false positives at times or was wrong when it said on the crawl a couple days ago that I don't have any errors (no duplicate page error).
Now, I don't know what to think.
-
Hey,
Our crawler actually requests the page http://mysite.com first but then finds all your links to www.mysite.com
You will want to contact the person responsible for hosting or developing your site in order to make these changes.
Have a great day!
Kenny
-
Thanks for the explanation. Could you answer a couple questions?
1 - If all internal site links go to www.mysite.com (none link to http://mysite.com), how does a duplicate page even happen? I don't understand how this happened to begin with if I don't have any such internal link to http://mysite.com.
2 - Can you recommend a service who can fix the htaccess page for me to create the 301 redirect? I'm not sure I want the hosting service doing it and making a mistake.
Thanks!
-
Hey,
That third campaign is actually a subdomain setup to crawl non-www. No duplicate content errors were presented because there are not any links to follow since all the links contain the subdomain www in them.
Root domain campaigns are distinguished with an astrick before the domain name.
-
Thanks - I initially thought that was it.
But if you see my 3rd campaign of the crawl, it runs it for the root domain and it shows no duplicates.
-
Hey,
I just looked into the issue that you are experiencing with our crawler. The reason the the discrepancy is because you actually have two separate campaigns running for the same site. One is set to crawl the root domain and one the subdomain.
The root domain campaign actually still presents these errors and has week over week but the sub-domain campaign is setup for the www version of your site and that's why these errors are not present, because the crawler won't even attempt to crawl off of www.
It is advisable to perform a 301 redirect as the other commenters mention.
Hope that helps!
Kenny
-
My point is the inconsistency in the SeoMoz crawler reports.
I got two SeoMoz crawl reports today - one was the regularly scheduled one which said I have duplicate home pages (as noted) and the crawl I started a couple hours ago said there are no errors.
So...how do you tell which one is right? Both cannot be since there were no changes to my website pages between the crawls.
thx
-
Hi,
If needed - this is the .htaccess code to help fix this issue; (Make sure and back up .htaccess before making any chages)
Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^yourdomainhere.com [NC]
RewriteRule ^(.*)$ http://www.yourdomainhere.com/$1 [L,R=301]
The above code would redirect all traffic from non www to www version of your site fixing dup content issues in that regard
Source ;http://www.webconfs.com/how-to-redirect-a-webpage.php
PS Spaces between lines not needed (funky formatting here)
Hope this helps
-
You need to redirect one of your home pages to the other. www.mysite.com is different to the crawl robot as my site.com. In addition to having the issue with seomoz, you are losing serp value for your home page because you are dividing up the SEO value. Do a 301 redirect from one to the other and voila....problem solved.
Please make sure you give me the thumbs up for the help!! Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
False Soft 404s, Shadow Bans, and Old User Generated Content
What are the best ways to keep old user generated content (UGC) pages from being falsely flagged by Google as soft 404s? I have tried HTML site maps to make sure no page is an orphaned but that has not solved the problem. Could crawled currently not indexed by explained by a shadow ban from Google? I have had problems with Google removing pages from SERPs without telling me about it. It looks like a lot of content is not ranking due to its age. How can one go about refreshing UGC without changing the work of the user?
Technical SEO | | STDCarriers0 -
Giving a final nudge to lead the serps
hi
Technical SEO | | ciznerguy
might be a weird one to ask. working on an ecommerce site, using all new inbound marketing best practices the site positioned on many keywords on the second page of results. the site started to generate income. Assuming budget is not an issue here, do you have any suggestions on how to give it the final push. any ideas will be considered.0 -
Google showing a Cached option but then giving a 404
2 weeks ago my home page plus some others had a 301 redirect to another domain for about 1 week (due to a hack).The original pages were then de-indexed and the new bad domain was indexed and in effect stole my rankings.Then the 301 was removed/cleaned from my domain and the bad domain was fully de-indexed via a request I made (this was 1 week ago).Then my pages came back into the index but without any ranking power.Now when I perform a search for my domain my home page is listed with an option to view the Cache. Clicking on the Cache brings up a 404 error.So why is Google showing the Cached option but doesn't have the cached file? How do I get Google to properly update it's Cache or show a cached copy?
Technical SEO | | Dantek0 -
SEOMOZ and non-duplicate duplicate content
Hi all, Looking through the lovely SEOMOZ report, by far its biggest complaint is that of perceived duplicate content. Its hard to avoid given the nature of eCommerce sites that oestensibly list products in a consistent framework. Most advice about duplicate content is about canonicalisation, but thats not really relevant when you have two different products being perceived as the same. Thing is, I might have ignored it but google ignores about 40% of our site map for I suspect the same reason. Basically I dont want us to appear "Spammy". Actually we do go to a lot of time to photograph and put a little flavour text for each product (in progress). I guess my question is, that given over 700 products, why 300ish of them would be considered duplicates and the remaning not? Here is a URL and one of its "duplicates" according to the SEOMOZ report: http://www.1010direct.com/DGV-DD1165-970-53/details.aspx
Technical SEO | | fretts
http://www.1010direct.com/TDV-019-GOLD-50/details.aspx Thanks for any help people0 -
Why doesn't SEOmoz see internal/external links on my site?
My SEOmoz analysis that my site contains neither external or internal lnks. I have used other tools and they have all seen the internal and external links on the pages. There aren't many but they are there. Why isn't SEOmoz seeing them?
Technical SEO | | iain0 -
Dear Support, my client is a large bank and his site is https. and seomoz does not give any data. what can i do ? thank you
Dear Support, what to do in case of https pages. seomoz does not give any data about it. please help asap. Thank you.
Technical SEO | | SebestynMrton0 -
Webmaster tools lists a large number (hundreds)of different domains linking to my website, but only a few are reported on SEOMoz. Please explain what's going on?
Google's webmaster tools lists hundreds of links to my site, but SEOMoz only reports a few of them. I don't understand why that would be. Can anybody explain it to me? Is there someplace to I can go to alert SEOMoz to this issue?
Technical SEO | | dnfealkoff0 -
Duplicate page content errors in SEOmoz
Hi everyone, we just launched this new site and I just ran it through SEOmoz and I got a bunch of duplicate page content errors. Here's one example -- it says these 3 are duplicate content: http://www.alicealan.com/collection/alexa-black-3inch http://www.alicealan.com/collection/alexa-camel-3inch http://www.alicealan.com/collection/alexa-gray-3inch You'll see from the pages that the titles, images and small pieces of the copy are all unique -- but there is some copy that is the same (after all, these are pretty much the same shoe, just a different color). So, why am I getting this error and is there any best way to address? Thanks so much!
Technical SEO | | ketanmv
Ketan0