Should I care about this Webmaster Tools Message
-
Here is the message:
"Googlebot found an extremely high number of URLs on your site: http://www.uncommongoods.com/"
Should i try to do anything about this? We are not having any indexation issues so we think Google is still crawling our whole site. What could be some possible repercussions of ignoring this?
Thanks Mozzers!
-Zack
-
Hi Zack,
Just checking in to see if you had run any other tests to determine why this message might have appeared (e.g. a site crawl to find duplication / bad canonicalisation, etc.)? Let me know if you still need assistance.
Cheers,
Jane
-
Also agree with David, but it wold not hurt to ensure that your XML sitemaps are up-to-date and an accurate reflection of your site's content. It also does not hurt to clean up any potential duplicate URL issues that your site may have. Otherwise you do not need to put a lot of weight into this message from GWT.
-
David is spot on aobut the titles and urls. Large DB driven sites tend to cause big issues off of simple mistakes in coding or content entry.
I have to say "Crawl it" again.
-
Since it is an ecommerce store with a lot of products, its natural for you to have a lot of URLs. As long as your content, titles, and urls are unique, I wouldn't pay a lot of attention to it.
Did a site search, and it came up with "About 181,000 results" that were indexed. If you know that number to be excessive, I would begin checking to see if you have cms generated URLs, that may be causing such a high page count.
-
I would crawl the site and look for and spider traps or potential duplicate content. I make it a point to do so every time I see one of these notices and I have found things after these notices that could potentially or havecontributed to a Panda filter.
Although I do recommend at least weekly crawls on larger sites, these notices area good prompt to do it if you have been lax on your audits.
-
If Google sends you a message, I'd listen.
Generally, these messages comes when you don't have something set correctly in your parameters. Have you set canonicals for your items?
Here are some resources to go through.
From Google: https://support.google.com/webmasters/answer/76401?hl=en
A Good Product Forum Discussion: https://productforums.google.com/forum/?hl=en#!category-topic/webmasters/crawling-indexing--ranking/CCORJBI-mEg
A Good Product Forum Discussion that has John Mueller (works for Google) on it: https://productforums.google.com/forum/#!topic/webmasters/1aTAjsRbIOU
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Website structure - best tools to analyse and plan, visually
Hi - I am about to analyse and then re-plan the structure of a website and think it would be best to do it graphically - in the form of a chart. Are there any tools you would recommend to visualise the structure of an existing website (perhaps something that can scan and then represent a websites) - or plan out a new/revised website? Thanks in advance, Luke
Intermediate & Advanced SEO | | McTaggart0 -
Should We Remove Content Through Google Webmaster Tools?
We recently collapsed an existing site in order to relaunch it as a much smaller, much higher quality site. In doing so, we're facing some indexation issues whereas a large number of our old URLs (301'd where appropriate) still show up for a site:domain search. Some relevant notes: We transitioned the site from SiteCore to Wordpress to allow for greater flexibility The Wordpress CMS went live on 11/22 (same legacy content, but in the new CMS) The new content (and all required 301s) went live on 12/2 The site's total number of URLS is currently at 173 (confirmed by ScreamingFrog) As of posting this question, a site:domain search shows 6,110 results While it's a very large manual effort, is there any reason to believe that submitting removal requests through Google Webmaster Tools would be helpful? We simply want all indexation of old pages and content to disappear - and for Google to treat the site as a new site on the same old domain.
Intermediate & Advanced SEO | | d50-Media0 -
Removal tool - no option to choose mobile vs desktop. Why?
Google's removal tool doesn't give a person the option to tell them which index - mobile friendly, or desktop/laptop - the url should be removed from. Why? I may have a fundamental misunderstanding. The way I thought it works is that when you have a dynamically generated page based on the user agent, (ie, the SAME URL but different formatting for smartphones as for desktop/laptop) then the Google mobile bot will index the mobile friendly version and the desktop bot will index the desktop version -- so Google will have 2 different indexed results for the same url. That SEEMS to be validated by the existence of the words 'mobile-friendly' next to some of my mobile friendly page descriptions on mobile devices. HOWEVER, if that's how it works--why would Google not allow a person to remove one of the urls and keep the other? Is it because Google thinks a mobile version of a website must have all of the identical pages as the desktop version? What if it doesnt? What if a website is designed so that some of the slower pages simply aren't given a mobile version? Is it possible that Google doesn't really save results for a mobile friendly page if there is a corresponding desktop page-- but only checks to see if it renders ok? That is, it keeps only one indexed copy of each url, and basically assumes the mobile title and actual content is the same and only the formatting is different? That assumption isn't always true -- mobile devices lend themselves to different interactions with the user - but it certainly could save Google billions of dollars in storage. Thoughts?
Intermediate & Advanced SEO | | friendoffood0 -
Should I delete 'data hightlighter' mark-up in webmaster tools after added schema.org mark-up?
LEDSupply.com is my site, and before becoming familiar with schema mark-up I used the 'data-highlighter' in webmaster tools to mark-up as much of the site as I could. Now that Schema is set-up I'm wondering if having both active is bad and am thinking I should delete the previous work with the 'data highlighter' tool. To delete or not to delete? Thank you!
Intermediate & Advanced SEO | | saultienut0 -
Best tools for identifying internal duplicate content
Hello again Mozzers! Other than the Moz tool, are there any other tools out there for identifying internal duplicate content? Thanks, Luke
Intermediate & Advanced SEO | | McTaggart0 -
Whole site blocked by robots in webmaster tools
My URL is: www.wheretobuybeauty.com.auThis new site has been re-crawled over last 2 weeks, and in webmaster tools index status the following is displayed:Indexed 50,000 pagesblocked by robots 69,000Search query 'site:wheretobuybeauty.com.au' returns 55,000 pagesHowever, all pages in the site do appear to be blocked and over the 2 weeks, the google search query site traffic declined from significant to zero (proving this is in fact the case ).This is a Linux php site and has the following: 55,000 URLs in sitemap.xml submitted successfully to webmaster toolsrobots.txt file existed but did not have any entries to allow or disallow URLs - today I have removed robots.txt file completely URL re-direction within Linux .htaccess file - there are many rows within this complex set of re-directions. Developer has double checked this file and found that it is valid.I have read everything that google and other sources have on this topic and this does not help. Also checked webmaster crawl errors, crawl stats, malware and there is no problem there related to this issue.Is this a duplicate content issue - this is a price comparison site where approx half the products have duplicate product descriptions - duplicated because they are obtained from the suppliers through an XML data file. The suppliers have the descriptions from the files in their own sites.Help!!
Intermediate & Advanced SEO | | rrogers0 -
Webmaster tool parameters
Hey forum, About my site, idealchooser.com. Few weeks ago I've defined a parameter "sort" at the Google Webmaster tool that says effect: "Sorts" and Crawl: "No URLs". The logic is simple, I don't want Google to crawl and index the same pages with a different sort parameter, only the default page without this parameter. The weird thing is that under "HTML Improvement" Google keeps finding "Duplicate Title Tag" for the exact same pages with a different sort parameter. For example: /shop/Kids-Pants/16//shop/Kids-Pants/16/?sort=Price/shop/Kids-Pants/16/?sort=PriceHi These aren't old pages and were flagged by Google as duplicates weeks after the sort parameter was defined. Any idea how to solve it? It seems like Google ignores my parameters handling requests. Thank you.
Intermediate & Advanced SEO | | corwin0
