How long does Google take to reduce the index size?
-
A few months ago, we have incorporated our custom search in our website www.ergodotisi.com . We hadn't been paying a lot of attention to our webmaster analytics, to find out a few months later than the Google Index had grown from 2K- 3K pages to one million because it was crawling all combinations of search filters. We have now followed the right instructions to add noindex meta tags and blocked most search result pages from the robot.txt. We allow indexing of some main categories by setting new seo-friendly url structures. A few weeks have passed and the index size has only reduced to 700K. How long does it take before it removes most of the duplicated search result pages from the index? Is it still crawling those pages but has not fully decided to remove most of them? How bad is this for SEO?
-
How long does it take before it removes most of the duplicated search result pages from the index?
Every site is different but I have seen it take 6 - 9 months for pages to drop out.
Is it still crawling those pages but has not fully decided to remove most of them?
It's possible. As Gaston has already pointed out, search engines will need to access those files again to see you want them noindexed.
How bad is this for SEO?
It temporarily dilutes the amount of SEO equity available to flow to pages you DO want indexed.
-
Hello there,
Did you left some time, without blocking those pages, to google bot to recrawl them?
If you implemented at the same time the noindex tag and the disallow in the robots.txt you are not letting google know that those pages should be deindexed.
Remember that blocking pages in the robots.txt avoid to be scanned again and the new robots tag is not seeng by google bot.My advise is to let google bot recrawl all those pages and wait a few days, may be 2-3 weeks. Slowly the amount of indexed pages will decrease.
Hope i've helped.
Best luck.
GR.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Review snippets not shown on google search results
Hi, In Moz it shows that we have a review snippet for a keyword/page, but it is not shown on google SERP. Can anyone explain why it isnt shown on Google search results, and what we should do in order to get it shown ?
On-Page Optimization | | jensatlieto0 -
Can Google read this code?
I'm working on some basic on-page SEO content for a website within my company and I need some guidance as far as 1. whether Google can read the small amount of existing text (not optimized) and if it isn't spiderable, then 2. what code should be there instead. Here is the site: https://www.le-velgear.com/store/catalog The text I'm referring to is toward the bottom of the page (isn't it always?) and says this: Designed for a Thriving Lifestyle The Le-Vel Gear store is an extension of the LV Life, the Thrive product line, and the world's largest health and wellness Movement, which you helped create. Living a life you deserve includes looking good while showing the world your pride in being a Thriver...Check out all the new and incredible gear and tools and take your Thriving lifestyle to the next level!!! When I "View Source," I cannot see the text, however, the text is highlight-able with my cursor and I can see it when I "Inspect Element" in a container that says Thanks in advance for any help!
On-Page Optimization | | lizzyr0 -
Page title in Google search is defferent
Hello, Google changes the title of the main page only for my sites in this way: What I configured: My page title | my site name How it shows in Google: My site name: My page title If I checked some meta tags analyzer it will show my configured page title and also in Bing.com So what do you thing about it. Best Regards, Housam
On-Page Optimization | | anubis20 -
Google search: 'define:____'
See: http://screencast.com/t/oFSzIt5rRm Thrilled that Google is pulling our content over wikipedia (in this instance). Wondering how we can assure more success like this. Mike Corso
On-Page Optimization | | Mike_c
Gartner.com1 -
Google crawler showing cache of another page
For the page http://www.thinkdigit.com/top-products/Laptops-and-PCs/top-10-laptops-124.php google is showing another page in cache (http://www.thinkdigit.com/top-products/Ultrabooks/top-10-ultrabooks-153.php). Please let me know how this happened and how to correct it.
On-Page Optimization | | 9dot90 -
Long URL's
So I'm super new at SEO and learning a lot. I'm a small business owner and enjoy doing it myself. Are long URL's good or bad? Like this: http://www.farnorthkennel.com/german-shepherd-puppies-the-girls/long-haired-german-shepherd-puppies-lava Is that too long? The german-shepherd-puppies-the-girls is an actual page with actual content. Do those hurt me?
On-Page Optimization | | Joshlaska0 -
Prevent Indexing of URLs Based on Tags
I started my website as a blog over at Posterous, but decided to turn it into a full scale business website with a self-hosted WordPress theme. Shortly after transitioning from Posterous to WordPress, I noticed that Google was indexing not only my old blog posts, but the URLs of my blog posts based on the tags they have. Is there any reason why this is a problem? I'm sure it shouldn't qualify as duplicate content, but for some reason it just feels a bit sloppy to me to have all of these pages indexed...Is this a non-issue? Should I just be more discriminating with my use of 'tags' if it bothers me? JiGLH.png
On-Page Optimization | | williammarlow0 -
Why are some of page indexed and others not
I have created a site structure like this: domain/for-sale/brand domain/for-sale/brand-model domain/for-sale/brand-model/pg1 domain/for-sale/brand-model/pg2 domain/for-sale/brand-model/pg3 etc.... I cannot understand why the domain/for-sale/brand-model does not seem to be indexed, yet the domain/for-sale/brand-model/pg6 is? This is a new site, but I cannot understand why this URL would be indexed without the others... Any ideas? My home pages has links to the domain/for-sale/brand, this page has links to domain/for-sale/brand-model1, domain/for-sale/brand-model2 etc, each of these pages have links to domain/for-sale/brand-model/pg1, domain/for-sale/brand-model/pg2 etc...
On-Page Optimization | | MirandaP0