How long does Google take to reduce the index size?
-
A few months ago, we have incorporated our custom search in our website www.ergodotisi.com . We hadn't been paying a lot of attention to our webmaster analytics, to find out a few months later than the Google Index had grown from 2K- 3K pages to one million because it was crawling all combinations of search filters. We have now followed the right instructions to add noindex meta tags and blocked most search result pages from the robot.txt. We allow indexing of some main categories by setting new seo-friendly url structures. A few weeks have passed and the index size has only reduced to 700K. How long does it take before it removes most of the duplicated search result pages from the index? Is it still crawling those pages but has not fully decided to remove most of them? How bad is this for SEO?
-
How long does it take before it removes most of the duplicated search result pages from the index?
Every site is different but I have seen it take 6 - 9 months for pages to drop out.
Is it still crawling those pages but has not fully decided to remove most of them?
It's possible. As Gaston has already pointed out, search engines will need to access those files again to see you want them noindexed.
How bad is this for SEO?
It temporarily dilutes the amount of SEO equity available to flow to pages you DO want indexed.
-
Hello there,
Did you left some time, without blocking those pages, to google bot to recrawl them?
If you implemented at the same time the noindex tag and the disallow in the robots.txt you are not letting google know that those pages should be deindexed.
Remember that blocking pages in the robots.txt avoid to be scanned again and the new robots tag is not seeng by google bot.My advise is to let google bot recrawl all those pages and wait a few days, may be 2-3 weeks. Slowly the amount of indexed pages will decrease.
Hope i've helped.
Best luck.
GR.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google AMP or CDN?
Hello. I'm running a CMS that cannot currently support both CDN and Google AMP. I would have to choose one or the other. Does anyone have any insight on which may be the better choice until I can figure out how to have both? I installed CDN first to reduce the time it took for my pages/images to load. I'd like to have AMP because it can do the same, and perhaps be a little more Google friendly (their product). I would appreciate any thoughts. Thanks! Steve
On-Page Optimization | | recoil0 -
When do Panda ranking factors apply when Google deindexes a page
Here is 2 scenarios Scenario 1 Lets say I have a site with a ton of pages (100,000+) that all have off site duplicate content. And lets say that those pages do not contain any rel="noindex" tags on them. Google then decides to de-index all those pages because of the duplicate content issue and slaps me with a Panda penalty. Since all those pages are no longer indexed by Google does the Panda Penalty still apply even though all those pages have been deindexed? Scenario 2 I add a rel="noindex" to all those 100,000+ off site duplicate content pages. Since Google sees that I have decided to not index them does the Panda penalty come off? What I am getting at is that I have realized that I have a ton of pages with off site duplicate content, even though those pages are already not indexed by Google does me by simply adding the rel="noindex" tag to them tell Google that I am trying to get rid of duplicate content and they lift the Panda penalty? The pages are useful to my users so I need them to stay. Since in both scenarios the pages are not indexed anyways, will Google acknowledge the difference in that I am removing them myself and lift the panda ban? Hope this makes sense
On-Page Optimization | | cbielich0 -
Long url > 115
Hi, in my web code I have link to my images that are resize online and the link is very long. like this src="http://img.espectador.com/mediadelivery/?fn=&i_enc=1&i_a=L2hvbWUvZXNwZWN0YWRvci93d3cvaW1hZ2VuZXMvMjUwMTY2XzEzNDk5NTQ0NjFfY29uc3RydWNjaW9uLmpwZw==&i_cl=1&i_tr=100&i_q=70&i_rt=0&i_w=250&i_h=188&i_wtmrk=" alt="Paro parcial de Sunca" border="0"/> I have a lot of warning in my reports with this and I would like to omit this warnings How can I do that? noindex? nofollow? Thanks The original page that contain that code is this http://www.espectador.com/noticias/250166/paro-parcial-de-sunca Thanks
On-Page Optimization | | informatica8100 -
Blocking Google seeing outbound links?
Apart from rewriting the outbound url to look like a folder 'abc.co.uk/out/link1' and blocking the folder 'out' in the robots.txt file, along with also nofollowing the links as well, is there anything else you can do?
On-Page Optimization | | activitysuper0 -
Discrepancy between SeoMoz vs Google Webmaster tools
SeoMoz reports over 70 4xx client server errors on my site, but Google Web Master Tools does not report any broken links. There are not any broken links on any of the pages that it is reporting. Could there be another reason for the 4xx errors besides broken links?
On-Page Optimization | | AndyHawkins0 -
How Can I Get Yahoo to Index My Site?
How Can I Get Yahoo to Index My Site? I have installed Bing webmaster tool two months ago -- is Yahoo that slow. My site has been out since May 2010 and for a year and a half, I only have 40 pages index. HELP!!!!
On-Page Optimization | | AppleCapitalGroup0 -
Blog Comment IPs Seen By Google?
I have a page on a client's site for testimonials (a dental practice). The page is actually a post on a Wordpress install where customers can enter their testimonials as WP comments. In an effort to encourage more clients to give more testimonials I was considering setting up an iPad or other tablet at the receptionist's desk where patients would be able to enter their successes as comments on the page. If I made sure the patients all used unique names and emails in the Wordpress comments, would Google still see all the comments are from the same IP and view this as suspicious?
On-Page Optimization | | jargomang0 -
Getting pages indexed by Google
Hi SEOMoz, I relaunched a site back in February of this year (www.uniquip.com) with about 1 million URL's. Right now I'm seeing that Google is not going past 110k indexed URL's (based on sitemaps). Do you have any tips on what I can do to make the site more likeable by Google and get more indexed URL's? All the the part pages can be browsed to by going to: http://www.uniquip.com/product-line-card/suppliers/sw-a/p-1 I've tried to make the content as unique as possible by adding random testimonials and random "related part numbers" see here: http://www.uniquip.com/id/246172/electronic-components/infineon/microcontrollers-mcu/sabc161pilfca Do I need to wait more time and be more patient with Google? It just seems like I'm only getting a few thousand URL's per day at the most. Would it help me if I implemented a breadcrumb on all part pages? Thanks, -Carlos
On-Page Optimization | | caneja0