How long does Google take to reduce the index size?
-
A few months ago, we have incorporated our custom search in our website www.ergodotisi.com . We hadn't been paying a lot of attention to our webmaster analytics, to find out a few months later than the Google Index had grown from 2K- 3K pages to one million because it was crawling all combinations of search filters. We have now followed the right instructions to add noindex meta tags and blocked most search result pages from the robot.txt. We allow indexing of some main categories by setting new seo-friendly url structures. A few weeks have passed and the index size has only reduced to 700K. How long does it take before it removes most of the duplicated search result pages from the index? Is it still crawling those pages but has not fully decided to remove most of them? How bad is this for SEO?
-
How long does it take before it removes most of the duplicated search result pages from the index?
Every site is different but I have seen it take 6 - 9 months for pages to drop out.
Is it still crawling those pages but has not fully decided to remove most of them?
It's possible. As Gaston has already pointed out, search engines will need to access those files again to see you want them noindexed.
How bad is this for SEO?
It temporarily dilutes the amount of SEO equity available to flow to pages you DO want indexed.
-
Hello there,
Did you left some time, without blocking those pages, to google bot to recrawl them?
If you implemented at the same time the noindex tag and the disallow in the robots.txt you are not letting google know that those pages should be deindexed.
Remember that blocking pages in the robots.txt avoid to be scanned again and the new robots tag is not seeng by google bot.My advise is to let google bot recrawl all those pages and wait a few days, may be 2-3 weeks. Slowly the amount of indexed pages will decrease.
Hope i've helped.
Best luck.
GR.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Parsing jQuery Links as Real Links
While trying to diagnose a recent Google penalty I found out that links were being parsed by Google even though they were made using jQuery. I had the linkify plugin on my site and configured it to convert URLs to links on all of my pages. Today I found links to other sites of mine from sites that should not have been linking to them and found that the links came from pages whose links were generated via jQuery. This makes me wonder, how do I know if Google is counting javascript generated links? Is it possible that my native ad widgets are creating links that Google might count? Since I don't own any of the sites that advertise via the widgets I don't know how to tell if they are getting link juice or not. It used to be that Google didn't parse javascript, so you could add as many links to your site via javascript as you wanted without being seen by Google as linking to those sites. Does anyone know of a jQuery plugin that does turn URLs into clickable links that Google won't parse as real links?
On-Page Optimization | | STDCarriers0 -
Page title in Google search is defferent
Hello, Google changes the title of the main page only for my sites in this way: What I configured: My page title | my site name How it shows in Google: My site name: My page title If I checked some meta tags analyzer it will show my configured page title and also in Bing.com So what do you thing about it. Best Regards, Housam
On-Page Optimization | | anubis20 -
Strange ranking occurrences in Google (NL)
Mozzers, I got a question for all of you. Recently (about 3 months ago) I launched a renewed website for a costumer of mine. Since then rankings have been improving and some decreasing a little but overall it went quite allright. _Still, we now have some issues with some new pages and I really don't know what to do any more. _ The Case:
On-Page Optimization | | JarnoNijzing
For starters lets say the company sells vacuum cleaners. We now make pages for specific product ranges e.g. Miele Diamond vacuum cleaners which in turn tells something about this product range and has links to different pages of that series, for instance the Miele Black Diamond Silent Vacuum Cleaner. Why did we do this? We already ranked for specific product pages but also wanted to rank for more general terms and thus product ranges. What happened? We now rank perfectly well for the product pages itself but for some reason the Miele Diamond Vacuum Cleaners page doesn't rank at all or not as it should. Why is this strange? Because we applied the exact same tactic for some other product ranges on the same website and it worked like a charm (part of the reason why we started to do this for all product-ranges). I could really use some help here. If you want I can message you the pagelink in PM but I won't post it here for several reasons. The Vacuum Cleaners in this example or not the real products though but used as an exemplar. I really do hope to hear from you with some advice or request for more information. Regards
Jarno0 -
What does Google consider a "Duplicate Title Tag?"
Do the title tags have to be exactly the same, or can they have some of the same keywords but different context? Hypothetical example: Home Page = Raising a Kitten, Tips & Tricks for a Healthy Cat Sub-Page = How to Cat-Proof your Home when Raising a Kitten Since both title tags has "raising a kitten," "cat" and "tips" would this be considered a "Duplicate Title Tag" even though the pages have completely different content in them? Thanks in advance!
On-Page Optimization | | Scratch_MM0 -
Google plus authorship is driving me truly mad!
Ok permision to vent first 😉 Aaaaahhhhhh!!!!! Fu@king Google authorship, for fuc@s sake why so fuc@ing useless....Ok vent over.... Mission: I wanted to add Google+ authorship images to appear in the serps so I followed this guide to the letter:
On-Page Optimization | | Nightwing
https://plus.google.com/authorship I then tested it my authorship link on page http://www.netconstruct.co.uk/services/digital-marketing/ work via the testing tool http://www.google.com/webmasters/tools/richsnippets And i get a thumbs down(arggg!!) no authorship data recognised but here's the mark up:
Author: [xxxxxxx](https://plus.google.com/u/0/114149997094688010790/?<br /> rel=author) onpafe http://www.netconstruct.co.uk/services/digital-marketing/ So please can someone give me any insight into why this is not working 😞 Grazie,
David !!IGNORE!! Spotted a gap in the authorship code afre the the question mark! Now it recognisies authorship mark up!0 -
Latent semantic Indexing - Does this help rankings/relevance?
Hi, Does semantically related words to the target term on a page help with rankings/relevance? If your after the term 'PC Screen' and you use the term 'PC Monitor' will go make the connection and also reward you because of the relevance? Anyone do this and have you seen any positives? I've just started to try this out lately and have been combining it with Wordle.net to give me an indication of where the content piece is heading and how aggressive the content leans towards certain words (makes things a little more interesting then calculating densities).
On-Page Optimization | | Bondara0 -
No index?
Hi, I have about 600 posts and most of them are not really optimized (some of them are flash photo gallery). Should I do no index them? They are just too many to optimize them. My website is http://www.soobumimphotography.com/ Thank you
On-Page Optimization | | BistosAmerica0 -
Google's Page Layout Algorithm Change
Hello Everyone, Google says they've implemented this change because they are answering the complaints of users who have to search for actual content after they've clicked on a result. They go on to say users want to see content right away. Now while most of this talk is about ads, I wonder if this will also apply to websites that are image and flash heavy above the fold with very little content. I am working on a few auto dealer sites where 99% of the content above the fold are flash banners and images. Below all of this noise you can find about 200 words of text talking about their dealerships. I'd love to know everyone's thoughts on this...Does the new page layout algorithm change apply to only ads or to images and flash as well? Thanks
On-Page Optimization | | wparlaman0