SeoMoz crawler giving false positives?
-
SeoMoz crawler indicated a few times that my site has a duplicate home page error (http://mysite.com and www.mysite.com)
I eliminated the the couple remaining internal links that pointed to http://mysite on a couple pages (all other internal links point to http://www.mysite.com)
I ran the crawl again and it said no errors this time. I naturally thought the duplicate page error problem was fixed.
However this morning I got the regularly scheduled crawl report from SeoMoz that said again I have those duplicate error pages. No changes were made to any of my site's pages between the crawls.
That makes me wonder if the crawler is providing false positives at times or was wrong when it said on the crawl a couple days ago that I don't have any errors (no duplicate page error).
Now, I don't know what to think.
-
Hey,
Our crawler actually requests the page http://mysite.com first but then finds all your links to www.mysite.com
You will want to contact the person responsible for hosting or developing your site in order to make these changes.
Have a great day!
Kenny
-
Thanks for the explanation. Could you answer a couple questions?
1 - If all internal site links go to www.mysite.com (none link to http://mysite.com), how does a duplicate page even happen? I don't understand how this happened to begin with if I don't have any such internal link to http://mysite.com.
2 - Can you recommend a service who can fix the htaccess page for me to create the 301 redirect? I'm not sure I want the hosting service doing it and making a mistake.
Thanks!
-
Hey,
That third campaign is actually a subdomain setup to crawl non-www. No duplicate content errors were presented because there are not any links to follow since all the links contain the subdomain www in them.
Root domain campaigns are distinguished with an astrick before the domain name.
-
Thanks - I initially thought that was it.
But if you see my 3rd campaign of the crawl, it runs it for the root domain and it shows no duplicates.
-
Hey,
I just looked into the issue that you are experiencing with our crawler. The reason the the discrepancy is because you actually have two separate campaigns running for the same site. One is set to crawl the root domain and one the subdomain.
The root domain campaign actually still presents these errors and has week over week but the sub-domain campaign is setup for the www version of your site and that's why these errors are not present, because the crawler won't even attempt to crawl off of www.
It is advisable to perform a 301 redirect as the other commenters mention.
Hope that helps!
Kenny
-
My point is the inconsistency in the SeoMoz crawler reports.
I got two SeoMoz crawl reports today - one was the regularly scheduled one which said I have duplicate home pages (as noted) and the crawl I started a couple hours ago said there are no errors.
So...how do you tell which one is right? Both cannot be since there were no changes to my website pages between the crawls.
thx
-
Hi,
If needed - this is the .htaccess code to help fix this issue; (Make sure and back up .htaccess before making any chages)
Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^yourdomainhere.com [NC]
RewriteRule ^(.*)$ http://www.yourdomainhere.com/$1 [L,R=301]
The above code would redirect all traffic from non www to www version of your site fixing dup content issues in that regard
Source ;http://www.webconfs.com/how-to-redirect-a-webpage.php
PS Spaces between lines not needed (funky formatting here)
Hope this helps
-
You need to redirect one of your home pages to the other. www.mysite.com is different to the crawl robot as my site.com. In addition to having the issue with seomoz, you are losing serp value for your home page because you are dividing up the SEO value. Do a 301 redirect from one to the other and voila....problem solved.
Please make sure you give me the thumbs up for the help!! Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Forum post multiple pages gives meta description duplicate.
My website has a forum that is using the title of the posts as a Meta Description.The problem is that when a posts becomes long and separates in pages Google tells me that i have duplicate meta description issues because the 2nd page and the 3rd page are using the same meta description.What is the best course of action here?
Technical SEO | | Angelos_Savvaidis0 -
Is there a reason why a host would be reluctant to give up Cpanel access info?
Granted, a strange question here... My client lost her cpanel login credentials, or never bothered to get them (she didn't even know she had a hosting account). Apparently she has a friend who is hosting her website for her, free of charge. I need to get into the cpanel, but they are being extremely difficult. The client asked them and they didn't want to give it to her either. Still trying, but is there any reason why they would be so difficult? How does it benefit them? It can't be because they're afraid of losing her account because she isn't paying them anything. Totally confused by this. Any ideas?
Technical SEO | | Masbro1 -
Redirect Impact - Moving From SEOmoz to Moz
Hey Guys, This has been on my mind ever since the big announcement, so today I did some searching around for some posts/talk about what the impact of their full site redirect has been for them and didn't find anything. Have they posted on this or are there any threads that I'm missing out on? I'd love to hear more about what the impact has been or any general thoughts/insights people may have. Thanks!
Technical SEO | | TakeLessons
Jon1 -
SEOMoz Crawler vs Googlebot Question
I read somewhere that SEOMoz’s crawler marks a page in its Crawl Diagnostics as duplicate content if it doesn’t have more than 5% unique content.(I can’t find that statistic anywhere on SEOMoz to confirm though). We are an eCommerce site, so many of our pages share the same sidebar, header, and footer links. The pages flagged by SEOMoz as duplicates have these same links, but they have unique URLs and category names. Because they’re not actual duplicates of each other, canonical tags aren’t the answer. Also because inventory might automatically come back in stock, we can’t use 301 redirects on these “duplicate” pages. It seems like it’s the sidebar, header, and footer links that are what’s causing these pages to be flagged as duplicates. Does the SEOMoz crawler mimic the way Googlebot works? Also, is Googlebot smart enough not to count the sidebar and header/footer links when looking for duplicate content?
Technical SEO | | ElDude0 -
How to create GA Profile for SEOMoz
Hi ! I get this : Oh No! We did not find any Google Analytics profiles associated with this account. So I was wondering how to create GA Profiles for SEOMOZ. I'm admin of 3 websites in my GA accounts ... Thanks !
Technical SEO | | ovivo0 -
Does CSS position effect the value of a link?
For example, take a look at http://www.dueds.com and scroll all the way to the bottom of the page. See the link in the bottom left? Does the fact that it is pushed all the way down to the bottom make the link worth less than if it was directly under the social media buttons?
Technical SEO | | adriandg0 -
Does using parentheses affect the crawlers?
Quick question: if you using a parantheses around a keyword, do search bots still recognize the keyword? Fox ex: Welcome to a website about the National Basketball Association (NBA). Will the bots recognize that I'm trying to optimize to NBA and not (NBA)? Is this different for tags vs. actual body copy?
Technical SEO | | BPIAnalytics2 -
Internal Link Counts in SEOMoz Report?
Hi, We ran a site diagnostic and it came back with thousands of pages that have more than 100 internal links on a page; however, the actual number of links on those pages seems to be far less than what was reported. Any ideas? Thanks! Phil UPDATE: So we've looked at the source code and realized that for each product we link to the product page in multiple ways - from the product image, product title and price. So we have three internal links to the same page from each product listing, which is being counted by the SEOMoz crawler as hundreds of links on each page. But in terms of the Googlebot, is this as egregious as having hundreds of links to different pages or does it not matter as much?
Technical SEO | | beso1
