Googlebot found an extremely high number of URLs on your site:
-
Website: www.gobol.in
Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again
Here's a list of sample URLs with potential problems.
- http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
- http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612
Please help
-
Thanks Dirk
-
Hi Ravi,
Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use
Disallow: /catalogsearch/
On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file
rgds,
Dirk
-
Thanks Martijn
-
Hi Ravi,
This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.
The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How important are clean URLs?
Just wanting to understand the importance of clean URLs in regards to SEO effectiveness. Currently, we have URLs for a site that reads as follows: http://www.interhampers.com.au/c/90/Corporate Gift Hampers Should we look into modifying this so that the URL does not have % or figures?
On-Page Optimization | | Gavo1 -
How can i block the below URLs
Google indexed plugins pages for my website. Please check below. How can stop them to be indexed on google.? http://www.ayurjeewan.com/wp-content/plugins/LayerSlider/static/skins/glass/ http://www.ayurjeewan.com/wp-content/plugins/LayerSlider/static/skins/borderlesslight3d/ http://www.ayurjeewan.com/wp-content/plugins/LayerSlider/static/skins/defaultskin/ My robots.txt file is - User-agent: * Disallow: /wp-admin/
On-Page Optimization | | MasonBaker0 -
Does anyone know of an api for on site SEO?
I have searched for one, but really cannot find one that fits my needs. I am looking at making an on site grader / service that will check pages and point out SEO problems. One that I have found that I like is seorch.eu but they do not have an api. I do not want to reinvent the wheel if I do not have to. Also, the api does not have to be free, or it does not even have to be an api, it can be a self hosted application too.
On-Page Optimization | | LesleyPaone0 -
URL for a new website
Hi, I am creating a new website for a client. Is it best to include the keywords from the most common search in the domain name, they would like: forenamesurname.com but should I be recommending: weddingmakeupbyforename.com Does it make much difference to search rankings if the keyword is in the domain name? Thanks v much
On-Page Optimization | | danieldunn100 -
Does this site have a duplicate content issue?
Google WMT is showing me only 2 short meta descriptions under "HTML Improvements" but I believe http://www.customgia.com may have a content duplication issue. Numerous keywords are used repeatedly across many product descriptions. To make matters worse, every product page has a "Design It!" button that sends the user to a flash-based jewelry designer in which they can edit the product's appearance. I'm not sure if these "designer pages" are adding unnecessary and potentially damaging duplicate content but it's certainly a possibility. There are many items on this site that are similar to one another but not the same. The product description tend to use the same phrases over and over again - words like crystal, Swarovski, beaded, design it, customize, change, pearl, glass beads, iridescent, pearl, drop earrings are used a lot. What I'm stuck on is whether or not I should be focusing on a content duplication issue as the primary SEO problem or if there is something bigger. Thank you for any assistance you can provide!
On-Page Optimization | | rja2140 -
Number of occurances of a keyword
At the moment my site is down due to issue at the datat centre so please don't ask for teh url as it will be some time before it is back up and running. On one of my pages I am targeting two related keyword phrases e.g. "How to use Widgets" and "Using Widgets" Each of these phrases appears once each in the Tile tag, H1 tag and meta description and two or three times in the body text. Which I beileve is current best practice. However the word Widgets appears more 60 times in total could this be hurting the rankings of the other two phrases? Many Thanks
On-Page Optimization | | spes1230 -
Brand Name URL Redirecting to Actual URL
So we have already built a site under a parent company's URL: parentcompany.com And now we have their branded product lines in directories: parentcompany.com/brand-name1, and parentcompany.com/brand-name2 We also own the actual URL Brand Name 1 (which is also the exact description of the product): brandname1.com We do not yet own the URL for Brand Name 2 (which is also the exact description of the product): brandname2.com. This is because a squatter is sitting on it and is asking $10,000+ for it. What we are trying to determine is how valuable these brand name URLs are since they will be redirecting and not the actual site's primary domain name. Anybody know how much of an effect owning those and redirecting has on ranking for those brand names that are also very descriptive of the products? Would we be smarter to spend $10,000 on adwords or 10,000 on the domain? Thanks!
On-Page Optimization | | grayloon1 -
Number of links in breadcrumb
Does google take the number of links in breadcrumb as a indicator to the depth of the page in the site structure? That will mean that a page with 5 links in the breadcrumb is a page very far away from the index, therefore a less important page.
On-Page Optimization | | seo.academy0