How long does Google take to reduce the index size?
-
A few months ago, we have incorporated our custom search in our website www.ergodotisi.com . We hadn't been paying a lot of attention to our webmaster analytics, to find out a few months later than the Google Index had grown from 2K- 3K pages to one million because it was crawling all combinations of search filters. We have now followed the right instructions to add noindex meta tags and blocked most search result pages from the robot.txt. We allow indexing of some main categories by setting new seo-friendly url structures. A few weeks have passed and the index size has only reduced to 700K. How long does it take before it removes most of the duplicated search result pages from the index? Is it still crawling those pages but has not fully decided to remove most of them? How bad is this for SEO?
-
How long does it take before it removes most of the duplicated search result pages from the index?
Every site is different but I have seen it take 6 - 9 months for pages to drop out.
Is it still crawling those pages but has not fully decided to remove most of them?
It's possible. As Gaston has already pointed out, search engines will need to access those files again to see you want them noindexed.
How bad is this for SEO?
It temporarily dilutes the amount of SEO equity available to flow to pages you DO want indexed.
-
Hello there,
Did you left some time, without blocking those pages, to google bot to recrawl them?
If you implemented at the same time the noindex tag and the disallow in the robots.txt you are not letting google know that those pages should be deindexed.
Remember that blocking pages in the robots.txt avoid to be scanned again and the new robots tag is not seeng by google bot.My advise is to let google bot recrawl all those pages and wait a few days, may be 2-3 weeks. Slowly the amount of indexed pages will decrease.
Hope i've helped.
Best luck.
GR.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google ranking content for phrases that don't exist on-page
I am experiencing an issue with negative keywords, but the “negative” keyword in question isn’t truly negative and is required within the content – the problem is that Google is ranking pages for inaccurate phrases that don’t exist on the page. To explain, this product page (as one of many examples) - https://www.scamblermusic.com/albums/royalty-free-rock-music/ - is optimised for “Royalty free rock music” and it gets a Moz grade of 100. “Royalty free” is the most accurate description of the music (I optimised for “royalty free” instead of “royalty-free” (including a hyphen) because of improved search volume), and there is just one reference to the term “copyrighted” towards the foot of the page – this term is relevant because I need to make the point that the music is licensed, not sold, and the licensee pays for the right to use the music but does not own it (as it remains copyrighted). It turns out however that I appear to need to treat “copyrighted” almost as a negative term because Google isn’t accurately ranking the content. Despite excellent optimisation for “Royalty free rock music” and only one single reference of “copyrighted” within the copy, I am seeing this page (and other album genres) wrongly rank for the following search terms: “free rock music”
On-Page Optimization | | JCN-SBWD
“Copyright free rock music"
“Uncopyrighted rock music”
“Non copyrighted rock music” I understand that pages might rank for “free rock music” because it is part of the “Royalty free rock music” optimisation, what I can’t get my head around is why the page (and similar product pages) are ranking for “Copyright free”, “Uncopyrighted music” and “Non copyrighted music”. “Uncopyrighted” and “Non copyrighted” don’t exist anywhere within the copy or source code – why would Google consider it helpful to rank a page for a search term that doesn’t exist as a complete phrase within the content? By the same logic the page should also wrongly rank for “Skylark rock music” or “Pretzel rock music” as the words “Skylark” and “Pretzel” also feature just once within the content and therefore should generate completely inaccurate results too. To me this demonstrates just how poor Google is when it comes to understanding relevant content and optimization - it's taking part of an optimized term and combining it with just one other single-use word and then inappropriately ranking the page for that completely made up phrase. It’s one thing to misinterpret one reference of the term “copyrighted” and something else entirely to rank a page for completely made up terms such as “Uncopyrighted” and “Non copyrighted”. It almost makes me think that I’ve got a better chance of accurately ranking content if I buy a goat, shove a cigar up its backside, and sacrifice it in the name of the great god Google! Any advice (about wrongly attributed negative keywords, not goat sacrifice ) would be most welcome.0 -
Why did I take a nose dive on november 6?
Note: Domain listed below is NSFW Hi guys and girls, I dont know WHY this seemed to have happened for my little webshop www.reallovesexdolls.com At this moment I raised my paid per click campaign so i still get visitors. But that costs too much money. Dont know what I did wrong, everything seemed so great. How can I make a fast recovery?
On-Page Optimization | | MartinePeters0 -
Google webmaster markup validation error
Type: Schema product Property: Image Error: Missing required field "name (fn)". Google Webmaster is showing this error when I try to validate markup on webmaster. This is my domain - www(dot)wishpicker(dot)com Would be great if someone could please help with this. Thanks
On-Page Optimization | | bansheeviv
Prakul0 -
Blogposts and Google Rankings
I'm pretty new to all of this, my new website is starting to come along nicely in the SERPs, now appear on page 4 for my main key phrases were I didn't appear last week at all. I'm now on a mission to get myself ranked higher but my website is pretty much done... apart from the ongoing blogposts that I need to add, I'm a wedding photographer so every shoot is going on to it. My question, my blogposts aren't necessarily targeting my main key phrase, it might be mentioned once or twice but ultimately it'll most likely be the venue that steals the targeting. By adding the blogposts (I still have 36 to do) and using my keyphrase sparing, will this help to elevate me from page 4 in google? I know that I still need to build backlinks etc to but just something I can't seem to find an answer for on google hehe.
On-Page Optimization | | MartinWardPhoto0 -
Hi.. Can a E-commerce site have a Google Authorship.
Hi Can a E-commerce site have a Google Authorship, and if yes i have learned google requires a face for Google Authorship, as they are applying Facial recognition with the authorship. If So, how can an e-commerce website have an individual's face. ?
On-Page Optimization | | usef4u0 -
Can Rankings in Google differ so much from computer to computer.
I was telling my friend via facebook to go on my website, I told him to search 'nightlife forum' in google. To which, I believed it was 11th, top of second page. On his computer, its currently ranking at 1st place is it possible to have a difference of 10 places? even though he lives in the same city as me. Would be good to see what it ranks on your computers too google "nightlife forum" look for www.talknightlife.co.uk (don't get confused with the .COM one out there) Cheers Guys
On-Page Optimization | | Lukescotty0 -
Stumped on why Google is not showing main site pages anymore
Recently had sites homepage listing taken off first page for brand name search even though search term is not competitive. Does anyone have any ideas?
On-Page Optimization | | Luia0 -
How to design a site map page for users (not for Google)
I would like to design a site map for my visitors so they can have a quick view on the whole content of the website. 2 questions : 1 - is this kind of site map can help in terms of SEO ? 2 - if so, what are the best practices to design it ? Thanks in advance.
On-Page Optimization | | betadvisor0