How to resolve - Googlebot found an extremely high number of URLs
-
Hi,
We got this message from Google Webmaster “Googlebot found an extremely high number of URLs on your site”. The sample URLs provided by Google are all either noindex or have a canonical.
- http://www.myntra.com/nike-stylish-show-caps-sweaters
- http://www.myntra.com/backpacks/f-gear/f-gear-unisex-black-&-purple-calvin-backpack/162453/buy?src=tn&nav_id=541
- http://www.myntra.com/kurtas/alma/alma-women-blue-floral-printed-kurta/85178/buy?nav_id=625
Also we have specified the parameters on these URLs as representative URL in Google Webmaster - URL parameters.
Your comments on how to resolve this issue will be appreciated.
Thank You
Kaushal Thakkar
-
Hi Kaushal,
Thanks for the question.
There are a few ways to deal with this problem which are recommended by Google here. In summary, you can:
- Use parameter handling as you have done
- Add the nofollow attribute to problematic URLs
- Block problematic URLs in robots.txt
There is also a thread in the Google webmaster forums which may be useful to you:
Overall, it comes down to having a good site architecture and cutting down / removing / blocking URLs that you don't care about from a search perspective.
I hope that helps a bit!
Paddy
-
Thank you David, Its been more than 10 months since these parameters have been specified in webmaster. This and other activities like noindex and canonicals helped us to reduce the indexed URL count from 32 million to 1.2 million. As the url index reduced this warning from google stopped for 4 months. However we started receiving this message again from february 2014.
Thanks
Kaushal
-
"we have specified the parameters on these URLs as representative URL in Google Webmaster - URL parameters."
How long ago was this done? Since there are so many URL's, it may take a while for them to recrawl and index the representative URL's per your request.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
URL masking
Hi all, Within my organization we maintain a bunch of websites, say child1.com, child2.com and child3.com. It was recently suggested that we merge all three into a single one, say mother.com. The marketing community within my organization strongly argued against this, mainly due to risks related to brand awareness, rank and traffic loss for each of these websites. But we resigned to the idea that we must do it, so we started considering the next proper steps. Now, my understanding is that setting up redirects is crucial in order to cushion the fall and mitigate the losses. However, some people are suggesting url masking. To me personally, url masking doesn't sound like a white hat practice, maybe it's borderline grey, but the bottom line is I need some advice on this topic. Could someone kindly address the following: 1. How is url masking different from url redirect? 2. Is url masking different from url cloaking? 3. Would google penalize us for implementing url masking? 4. Would that have any impact on our PPC campaigns? 5. Are there any documented cases of successful and google-sanctioned websites that are actively using url masking? 6. Are there any pitfalls to using this strategy? Thank you
White Hat / Black Hat SEO | | SimonaCretu0 -
How to redirect 301 from high authority sites to own website?
How to redirect 301 from high authority sites to own website? If anyone know can tell me, such gigs are selling on the Fiverr.
White Hat / Black Hat SEO | | jefjaa0 -
Why to add a product id in the url
Hello ! shop.com/en/2628-buy-key-origin-the-sims-4-seasons/ Why will people use a product id in the link? Is there any advantage like better ranking or else?
White Hat / Black Hat SEO | | kh-priyam0 -
Googlebot crawling AJAX website not always uses _escaped_fragment_
Hi, I started to investigate googlebot crawl log of our website, and it appears that there is no 1:1 correlation between a crawled URL with escaped_fragment and without it.
White Hat / Black Hat SEO | | yohayg
My expectation is that each time that google crawls a URL, a minute or so after, it suppose to crawl the same URL using an escaped_fragment For example:
Googlebot crawl log for https://my_web_site/some_slug Results:
Googlebot crawled this URL 17 times in July: http://i.imgur.com/sA141O0.jpg Googlebot crawled this URL additional 3 crawls using the escaped_fragment: http://i.imgur.com/sOQjyPU.jpg Do you have any idea if this behavior is normal? Thanks, Yohay sOQjyPU.jpg sA141O0.jpg0 -
Can I 301 redirect old URLs to staging URLs (ex. staging.newdomain.com) for testing?
I will temporarily remove a few pages from my old website and redirect them to a new domain but in staging domain. Once the redirection is successful, I will remove the redirection rules in my .htaccess and get the removed pages back to live. Thanks in advance!
White Hat / Black Hat SEO | | esiow20130 -
How to remove trailing slashes in URLs using .htaccess (Apache)?
I want my URLs to look like these: http://www.domain.com/buy http://www.domain.com/buy/shoes http://www.domain.com/buy/shoes/red Thanks in advance!
White Hat / Black Hat SEO | | esiow20130 -
How long we can keep 302 redirection for a webpage url?
Hi Friends, I have a webpage featuring a product. I have created a new domain featuring the same product and the page is under construction. I am planning to do 302 redirection from the new domain to the existing domain for the time being. How long can I keep the 302 redirection from the new domain to existing domain? Is there any fixed time period/ duration that we can keep the 302 redirection for a webpage? I am planning to make few more pages (privacy policy, about us, etc) from the new domain 302 redirected to the existing domain. Is it possible? If so, how long can I keep the same? May I know which redirect is safe to use in this case, 302 or 301 redirect?
White Hat / Black Hat SEO | | zco_seo0 -
Will two numbers on a local listing affect me?
The reason I'm asking this question is. I was on the phone with a Google Rep yesterday for one of my google places. It was in reference to map maker and the fact that I only wanted one number on the listing. About a month ago I had it, so I deleted the listing's local number and than had an 877 number. The problem is when I checked on Friday the Local number was then added back by a Google moderator in Map Maker. So, now there's two numbers on the listing. He told me that would not affect my NAP info, which I don't believe.
White Hat / Black Hat SEO | | PeterRota
He also went on to say that Google goes through listings and if they have an 800 number they may delete it and replace place it with a local number. Has anyone delt with this and can Verify what he says to be true? Additionally, will my NAP be affected if this is the case? Thanks.0