How long does Google take to reduce the index size?
-
A few months ago, we have incorporated our custom search in our website www.ergodotisi.com . We hadn't been paying a lot of attention to our webmaster analytics, to find out a few months later than the Google Index had grown from 2K- 3K pages to one million because it was crawling all combinations of search filters. We have now followed the right instructions to add noindex meta tags and blocked most search result pages from the robot.txt. We allow indexing of some main categories by setting new seo-friendly url structures. A few weeks have passed and the index size has only reduced to 700K. How long does it take before it removes most of the duplicated search result pages from the index? Is it still crawling those pages but has not fully decided to remove most of them? How bad is this for SEO?
-
How long does it take before it removes most of the duplicated search result pages from the index?
Every site is different but I have seen it take 6 - 9 months for pages to drop out.
Is it still crawling those pages but has not fully decided to remove most of them?
It's possible. As Gaston has already pointed out, search engines will need to access those files again to see you want them noindexed.
How bad is this for SEO?
It temporarily dilutes the amount of SEO equity available to flow to pages you DO want indexed.
-
Hello there,
Did you left some time, without blocking those pages, to google bot to recrawl them?
If you implemented at the same time the noindex tag and the disallow in the robots.txt you are not letting google know that those pages should be deindexed.
Remember that blocking pages in the robots.txt avoid to be scanned again and the new robots tag is not seeng by google bot.My advise is to let google bot recrawl all those pages and wait a few days, may be 2-3 weeks. Slowly the amount of indexed pages will decrease.
Hope i've helped.
Best luck.
GR.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Long-tail with few searches vs. Generic with many
Our business is a contract packager/manufacturer of products sold to very prominent brands who sell through retail. For example, we make the sunscreen under a brand’s name, which you might then find on the shelf in Target or CVS. As I’ve optimized our pages, I’ve attempted to go long-tail, which has been simply to add “…contract packaging” or a variation after the particular product. So, instead of trying to compete in “sunscreen”, which would pit me against big-box distributors and prominent brands and sellers of sunscreen, I’ve optimized for “sunscreen manufacturers.” “Sunscreen” has 31K – 72K searches, with an 81 Difficulty and 67 Potential. “Sunscreen manufacturers” has a low 13 Difficulty and a decent 54 Potential, but only 51 – 100 searches. Some of my terms have only 0 – 10 searches, but I’ve been thinking that it’s better to compete for fewer but more qualified / buyer-intent searches and have generally lower Difficulty. Can you please tell me if this is a smart strategy, or if I should instead try to compete in higher-volume terms but much greater Difficulty? Thanks a lot for everyone's help.
On-Page Optimization | | Beau_W0 -
Google Console returning zero data
Hi. I verified my site both www. and non www. with google search console a while back and yet for neither domain am I seeing any data at all under Search Appearance and under index status (under google index) it is saying 0 pages indexed even though I can see in google search there are over 116. Any idea why this might be? Thanks
On-Page Optimization | | CosiCrawley0 -
How to reduced dns requests on magneto website
How to reduced dns requests on magneto website, really stuck, read many documents but no where near resolving the issues.
On-Page Optimization | | Mikaai0 -
Multiple Sizes of eCommerce Product Best Practice
I sell a product that comes in 9 different sizes, two materials, and two different shapes. People often search for this product by size, material and shape as follows: #9 material1 square widget #5 material2 circle widget The dilemma I'm facing is should I create 1 page for each of these products resulting in 36 different pages, or should I create one page that the users can select size shape and material? I'm thinking that from a usability stance, the 36 different pages are easier to navigate and determine price on, but I'm afraid that going the route that is easier for the customer to use in this case could hurt me duplicate content wise. I'm all about making a good user experience, but don't want to hurt myself because the content on all 9 sizes is basically the same. Are images of the product enough to be considered non-duplicate content? I also list out the dimensions of each product, but beyond that there isn't much to delineate the content. My plan is to create one page with all the content that relates to all of the products as a top level page with links to the individual products broken down, but just wanted to get some feedback from you guys before making the effort.
On-Page Optimization | | kadesmith0 -
What is everyone doing to reduce the number of links on a page?
Some clients of mine have sites that are throwing the "too many links on one page" error and we're not just talking a little more than the status quo 100 links, it's much more. I believe it could be due to the fly-out navigation. My Solution: shorten the Tier 2 categories in the left nav down to 5 and add a "View All" link after the 5th and remove top nav fly-outs. I'm not sure if these are best practices or the best for usability though?
On-Page Optimization | | LisaS130 -
Issues with Product Pages Getting Index In Google
I just started working here the other week and one of the big issue is that a lot of the product pages are not getting index in google. We have an xml.gz site map they submitted a long time ago. My guess is it might be something with not enough content on the pages? Here are a few example of pages that are not getting index in google. http://www.rockymountainatvmc.com/p/43/-/439/716/-/33097/Alpinestars-Dual-Motorcycle-Gloves http://www.rockymountainatvmc.com/p/47/-/201/803/-/28948/Camelbak-Blowfish-2013 http://www.rockymountainatvmc.com/p/46/-/203/836/-/6996/MSR-Head-Case http://www.rockymountainatvmc.com/p/44/54/208/764/80/1220/Galfer-Brake-Pad-Sintered-Metal There are 100's that are not indexed just trying to figure out what we need to do! We are working on new content to them all but we have over 5000 products so it will take a long time. We also have the reviews on the pages and are looking at starting a Q&A on page to help get more unique content.
On-Page Optimization | | DoRM0 -
Google's Page Layout Algorithm Change
Hello Everyone, Google says they've implemented this change because they are answering the complaints of users who have to search for actual content after they've clicked on a result. They go on to say users want to see content right away. Now while most of this talk is about ads, I wonder if this will also apply to websites that are image and flash heavy above the fold with very little content. I am working on a few auto dealer sites where 99% of the content above the fold are flash banners and images. Below all of this noise you can find about 200 words of text talking about their dealerships. I'd love to know everyone's thoughts on this...Does the new page layout algorithm change apply to only ads or to images and flash as well? Thanks
On-Page Optimization | | wparlaman0 -
Does Google still see masked domains as duplicate content?
Older reads state the domain forwarding or masking will create duplicate content but Google has evolved quite a bit and I'm wondering if that is still the case? Not suggesting that a 301 is not the proper way to redirect something but my question is: Does Google still see masked domains as duplicate content? Is there any viable use for domain masking other than for affiliates?
On-Page Optimization | | TracyWeb0