How long does Google take to reduce the index size?
-
A few months ago, we have incorporated our custom search in our website www.ergodotisi.com . We hadn't been paying a lot of attention to our webmaster analytics, to find out a few months later than the Google Index had grown from 2K- 3K pages to one million because it was crawling all combinations of search filters. We have now followed the right instructions to add noindex meta tags and blocked most search result pages from the robot.txt. We allow indexing of some main categories by setting new seo-friendly url structures. A few weeks have passed and the index size has only reduced to 700K. How long does it take before it removes most of the duplicated search result pages from the index? Is it still crawling those pages but has not fully decided to remove most of them? How bad is this for SEO?
-
How long does it take before it removes most of the duplicated search result pages from the index?
Every site is different but I have seen it take 6 - 9 months for pages to drop out.
Is it still crawling those pages but has not fully decided to remove most of them?
It's possible. As Gaston has already pointed out, search engines will need to access those files again to see you want them noindexed.
How bad is this for SEO?
It temporarily dilutes the amount of SEO equity available to flow to pages you DO want indexed.
-
Hello there,
Did you left some time, without blocking those pages, to google bot to recrawl them?
If you implemented at the same time the noindex tag and the disallow in the robots.txt you are not letting google know that those pages should be deindexed.
Remember that blocking pages in the robots.txt avoid to be scanned again and the new robots tag is not seeng by google bot.My advise is to let google bot recrawl all those pages and wait a few days, may be 2-3 weeks. Slowly the amount of indexed pages will decrease.
Hope i've helped.
Best luck.
GR.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does this index well
hi i have been looking at this template but as the content will be generated from their database will it index well - am i better to build a static equivalent using something like visual composer as i am not that technical: http://realhomes.inspirythemes.biz/listing/
On-Page Optimization | | neilhenderson0 -
When do Panda ranking factors apply when Google deindexes a page
Here is 2 scenarios Scenario 1 Lets say I have a site with a ton of pages (100,000+) that all have off site duplicate content. And lets say that those pages do not contain any rel="noindex" tags on them. Google then decides to de-index all those pages because of the duplicate content issue and slaps me with a Panda penalty. Since all those pages are no longer indexed by Google does the Panda Penalty still apply even though all those pages have been deindexed? Scenario 2 I add a rel="noindex" to all those 100,000+ off site duplicate content pages. Since Google sees that I have decided to not index them does the Panda penalty come off? What I am getting at is that I have realized that I have a ton of pages with off site duplicate content, even though those pages are already not indexed by Google does me by simply adding the rel="noindex" tag to them tell Google that I am trying to get rid of duplicate content and they lift the Panda penalty? The pages are useful to my users so I need them to stay. Since in both scenarios the pages are not indexed anyways, will Google acknowledge the difference in that I am removing them myself and lift the panda ban? Hope this makes sense
On-Page Optimization | | cbielich0 -
Make Google reindex the website after on-page optimization
Hello Moz Community, I just finished the on-page optimization for a new project and I would like to know how can I make Google to reindex the new link structure, titles and meta tags? Thank you!
On-Page Optimization | | CosminC0 -
Boat broker - issues with duplicate content and indexing search results
Hello, I have read a lot about optimising product pages and not indexing search results or category pages as ideally a person should be directed straight to a product page. I am interested in how best to approach a site that is listing second hand products for sale - essentially a marketplace of second hand goods (in my case, www.boatshed.com - international boat brokers). For example, we currently have 5 Colvic Sailer 26 boats for sale across the world - that is 5 boats of the same make and model but differing years, locations, sellers and prices. My concern is with search results and 'category' pages. Unlike typical e-commerce sites, when someone searches for a 'Colvic sailer 26 for sale' I want them to go to a search results style page as it is more useful for them to see a list of boats than one random one that Google decides is most important (or possibly one it can match by location). Currently we have 3 different URL types to show search results style pages (i.e. paginated lists of boats that include name, image and short description):
On-Page Optimization | | pbscreative
manufacturer URL's e.g. http://www.boatshed.com/colvic-manufacturer-145.html
category URL's e.g. barges http://www.boatshed.com/barges-category-55.html
and normal search results e.g. dosearch.php?form_boattype_textbox=&.... I have noindexed the search results pages but our category and manufacturer URLs show up in search results and ultimately these are pages I want people to land on. I am however getting duplicate content warnings in Moz. Most boats are in several categories and all will come up on 1 manufacturer and one manufacturer and model page. Both sets of URL's are in my opinion needed; lots of users search for exact makes / models and lots of users just search for the type of boat e.g. 'barge for sale' so both sets of landing pages are useful. Any suggestions or thoughts greatly appreciated Thanks Ben0 -
How does Google Detect which keywords my website should show up for in the SE?
When I checked my Google Webmaster Tools I found that my website is showing up for keywords that I didn't optimize for ... for example I optimize my website for "funny pictures with captions", and the website is showing up for "funny images with captions". I know that this is good, but the keyword is dancing all around, sometimes I search for "funny pictures with captions" and I show up in the 7th page, and some time I don't show up. and the same goes for the other keyword. of course I am optimizing for more than two keywords but the results is not consistent. my question is how does Google decide which keywords you website should show up for? Is it the on-page keywords?, or is it the off-page anchor text keywords? Thank you in advance ...
On-Page Optimization | | FarrisFahad
FarrisFahad0 -
Indexed iframed content behind login
Hi, I have a question regarding iframed content. I would like to get my non cms content which is served via an iframe solution (from the same domain) behind a anonymous or personal login indexed by search engines. How can we make this work? I've looked at the following solutions: http://googlewebmastercentral.blogspot.nl/2008/10/first-click-free-for-web-search.htmlhttp://productforums.google.com/forum/#!topic/webmasters/l9n8oGLQRkUBut I would like the content to be crawlable deeper than the just one page (if this is possible using the iframe solution).We could also setup different new pages in our CMS with the same content...Any suggestions?Thanks!Arnout
On-Page Optimization | | hellemans0 -
Branded keyword domain not appearing in Google
Hi, I have a site called www.emv-cards.com but the site has a very poor showing on Google for the search term 'emv cards' and this does not make sense to me. It has position #6 on Bing but is not in Google's top 50. Any assistance would be appreciated. regards, George.
On-Page Optimization | | sirgeorge0 -
How can I reduce my webpage load time?
According to google 'On average, pages in your site take 4.6 seconds to load (updated on Apr 3, 2011). This is slower than 71% of sites. These estimates are of low accuracy (fewer than 100 data points). The chart below shows how your site's average page load time has changed over the last few months. For your reference, it also shows the 20th percentile value across all sites, separating slow and fast load times.' My website: http://ablemagazine.co.uk I've installed Cache plugins, Minify plugins, reduce the amount of posts on my main page. But my website is still taking too long to load and I'm afraid I'm being penalised for it. Any tips?
On-Page Optimization | | craven220