Moz crawler finding my homepage multiple times
-
Hi and thank you in advance for your help!
I have a Moz Pro campaign running (I am a complete Moz novice by the way) for one of my websites (balloonsutah.com). After crawling my site, the Moz crawler informed me that I have 3 pages with duplicate content. While I am not sure why exactly this is happening, the crawler indexed my homepage 3 times under different url's.
-balloonsutah.com
-balloonsutah.com/
-balloonsutah.com/index.htmlI checked my FTP server and I cannot figure out for the life of me why the crawler is finding anything other than the index.html file.
I suppose I need to do something regarding a rel="Canonical" but I am not terribly familiar with that either.
Any suggestions would be greatly appreciated!
Keenan -
You're welcome!
-
Great answer! I appreciate the time you spent spelling everything out in detail. Thank you!
-
First things first, I did check all web addresses. They all exist. You probably need to provide more detail whether or not you are using a CMS for your web pages.
All 3 pages have different page authority. That is, one of the version is ranking higher than the other versions. I did a quick check of that via Moz toolbar. Looks like the index.html has the highest authority.
Note that all 3 versions you listed, has 2 other versions. The one with the www, and the one without the www. Judging from the moz toolbar, looks like you rank better for the one without the 'www' . Rel canonical is is good option, but in this case I would try to do a 301 redirect from the server side first. Again, not sure how much access you have to the server side. You might need to contact your web admin.host company etc.
You can read about redirects more over here. --> http://moz.com/learn/seo/redirection. If you don't have access to the server you can try doing the rel canonical. Read more here --> http://moz.com/learn/seo/duplicate-content
Example. you have www.example.com/page1.htm, /page2.htm, page3.htm. They all have same exact content. Lets say that pag1.htm is your main version. You can do the following in the header section of page2, and page 3.htm
"This tag tells Bing and Google that the given page should be treated as though it were a copy of the URL www.example.com/pag1.htm/ and that all of the links and content metrics the engines apply should actually be credited toward the provided URL."
I would recommend not to delete all the other version, but instead do a 301 redirect, or a rel canonical, as they all of some kind of page authority, except index.html has the highest. (the non www version). But you need to make that decision. But looks like that's what you want to be the main one anyway.
ALSO,
You can tell google which version you prefer to google in GWT. This informs google which version you prefer. You can read more here.
https://support.google.com/webmasters/answer/44231?hl=en
"Once you tell us your preferred domain name, we use that information for all future crawls of your site and indexing refreshes. For instance, if you specify your preferred domain as http://www.example.com and we find a link to your site that is formatted as http://example.com, we follow that link as http://www.example.com instead. In addition, we'll take your preference into account when displaying the URLs. If you don't specify a preferred domain, we may treat the www and non-www versions of the domain as separate references to separate pages."
"Note: Once you've set your preferred domain, you may want to use a 301 redirect to redirect traffic from your non-preferred domain, so that other search engines and visitors know which version you prefer."
You cannot control the www and non www versons of your website, but you can control, making duplicate pages, especially of your home page. I am guessing that that is something that was done by your CMS. Index.html was probably done by you. FURTHERMORE, I think .com/ & .com is the one and the same thing. and you probably had to decide, when you were making a new campaign in moz. They probably asked you to put down your web address for your domain, and your probably put something like, "balloonsutah.com"Not exactly sure, why it showed you .com & .com/, but it makes sense that they would show you, .com, and /index.html, as they are two different pages, even though it has the same content. It still is two different URL's.
I probably wouldn't worry too much about it. But I'll let one of the moz members answer about .com &.com/. I would perhaps concern myself more about 301 redirects, and rel canonicals.
Hope I helped.
-
Thank you for the help!
-
Hello Keenan-price,
Welcome to the Moz community!
Moz is reporting these duplicates correctly. Each of the listed URLs are seen as unique URLs and unique pages. This is a common problem when a website does not have the proper canonical tags and 301 redirects in place for these URLs.
You'll want to decide on how your website should be displayed (which URL you prefer) and implement the canonical tag and 301 redirects.
the 301 redirects could be done with your .htaccess file, depending on your site environment. The canonical tags would depend on your site's environment (wordpress, custom development, ect).
Also, make sure to go into your Google Webmaster Tools account and specify a single page as being the correct page, once you've decided on how you want the URL to be displayed.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Moz on-page grader can't see keyword in H1
Hi, Trying to do grade for whatiswhere.com, keyword 'poi search'. The tool can't see keyword in h1, but It is there. Could you please check what is the problem? Thank you, Andrei.
Moz Bar | | Anazar_20010 -
Moz Keyword Explorer how long does it take to gather metrics
Looking at some low volume keywords, I get lots of "Gathering Metrics". These don't seem to be updating with metrics. How long does this typically take, should I set and come back in a couple of hours, overnight? Should I leave the screen open, or will this stop it working? Justin
Moz Bar | | GrouchyKids0 -
Keyword research in Moz - Am I missing something?…
I'm fairly new at SEO and am still trying out MOZ. I'm finding myself wishing that Moz had a broader set of tools for keyword research, but am wondering if maybe I'm just missing something. Am I correct in saying that there is no place within Moz that you can get keyword ideas, select/organize keywords, find new/related keywords? I've appreciated the keyword difficulty tool and have played around with Moz Analytics keyword tracking, but these don't help with the broader keyword research aspect. The various tutorials talk about using Google AdWords Keyword Planner and other tools to do that job. But with all that Moz can do, it just seems backwards to do lots of copying-pasting from other tools into Moz, or to have to rely on old excel spreadsheets to organize keywords. It seems that life would be much easier if Moz would be able to pull that kind of external research information in, like Raven does. That way you do most of your research within Moz – allowing you to organize/evaluate potential keywords BEFORE choosing the ones you would like to test for difficulty or track. That way you could simply select the desired keywords and add them to your tracked list. The way it is now, even the one research tool that Moz does have (keyword difficulty) does not allow for an easy way to track or manage a chosen set of keywords. For example, after doing some keyword difficulty analysis, I've chose 5 keywords. To add these 5 to the tracked keywords in a specific campaign I can't just select them and say "track keywords > in xxx campaign". From what I've understood, I must copy-and-paste them into the manage keyword section of the desired campaign. To me this all feels a bit awkward. But since I'm new at this, maybe I'm just missing something? Or maybe it's unrealistic to expect a single tool to address all my SEO needs?
Moz Bar | | Mike_E0 -
Moz bar chrome
Hi
Moz Bar | | Emanuele_Ricci
moz bar does not work on chrome, it works correctly on firefox.
The bar is there but not showing any data...
Any idea? It can be something that is blocking moz bar on chrome browser or what else????
Thanks1 -
Why does it show (in Moz) that we don't have any meta descriptions on our site when we do?
We have over 1,330 pages that say we are "Missing Meta Description Tag" when I spot check all of them have meta descriptions? Can you please explain to me why Moz is picking up that we do not have meta descriptions when we do. Our website http://www.betheboss.ca Please help. I would like a accurate measure of meta descriptions that are missing.
Moz Bar | | BeTheBoss0 -
Why moz does not update actual keyword possition
I managed to get first place on google by keyword: vervo.lv kravu pārvadājumi in google.lv (it is in first place all this week) But in moz reports it shows just second position? I added so far about 100 keywords to monitor so if it shows incorrect this one how can i trust to other keyword results. Report says it updated on 3. september and it already was in first place. I really need to monitor keywords but if it does not show actual positions it is useless.
Moz Bar | | vervo0 -
Moz reporting appropriate Canonical tag usage but no canonical tag on page !?
I take it this means that the page in question has been referenced via a different pages canonical tag but that the page in question itself does not have a self referencing canonical tag (and that it should do) cheers dan
Moz Bar | | Dan-Lawrence0 -
Best way to submit multiple, simultaneous urls to SEOMOZ
I have a pro membership, and am looking to get inbound link data on multiple urls. Is there a way to submit multiple urls at once?
Moz Bar | | cmcilwain0