Why is google not deindexing pages with the meta noindex tag?
-
On our website www.keystonepetplace.com we added the meta noindex tag to category pages that were created by the sorting function.
Google no longer seems to be adding more of these pages to the index, but the pages that were already added are still in the index when I check via site:keystonepetplace.com
Here is an example page: http://www.keystonepetplace.com/dog/dog-food?limit=50
How long should it take for these pages to disappear from the index?
-
Google might have already crawled the pages but not indexed them yet. Be patient , if you have enough links coming in and the pages are less than 3 levels deep they will all be crawled and indexed in no time.
-
I guess it depends on the urgency of your situation. If you were just trying to clean things up then it's okay to wait for Google to re-crawl and solve the problem. But if you have been affected by panda and your site is not ranking then I personally would consider that an urgent enough need to use the tool.
-
This link almost makes it seem like I shouldn't use the webmaster tools removal.
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1269119
-
The crawlers have so many billions of webpages to get to. We have more than 50,000 on our site; there's about 8,000 that they check more regularly than the others - some are just really deep on the site and hard to get to.
-
You can remove entire category directories from the index in one command using the tool. But the urls won't be removed from the cache, just the index. To remove them from the cache you'll need to enter each url individually. I think that if you are trying to clear things up for Panda reasons, just removing from the index is enough. However, I'm currently trying to decide if it will speed things up to remove from the cache as well.
-
Ok. That makes sense. I wonder why it takes so long? I'll start the long process of the manual removal.
-
Streamline Metrics has got it right.
I've seen pages take MONTHS to drop out of the index after being noindexed. It's best to use the URL removal tool in WMT (not to be confused with the disavow tool) to tell Google to not only deindex the pages but to remove them from the cache as well. I have found that when you do this the pages are gone within 12 hours.
-
In your experience how long does this normally take?
-
Yes it was around December 2nd or 3rd that we added the noindex tags. It just seemed like google wasn't removing any pages yet from the index. It did stop google from adding more of these pages though.
-
It all depends on how long it takes Google to re-crawl those pages with the no index tag on them.
I would do this along with the steps you have already taken in order to help speed the process up if you are in a hurry
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1663419
-
Do you know when you added the noindex tags? Google will need to recrawl the pages to see the noindex tags before removing them. I just looked at one your category pages and it looks like it was cached by Google on December 1st, and there was no noindex tag on that page. Depending on how big your site is and how often your site is crawled will determine when they will be removed from the index. Here's Google's official explanation -
"When we see the noindex meta tag on a page, Google will completely drop the page from our search results, even if other pages link to it. Other search engines, however, may interpret this directive differently. As a result, a link to the page can still appear in their search results.
Note that because we have to crawl your page in order to see the noindex meta tag, there's a small chance that Googlebot won't see and respect the noindex meta tag. If your page is still appearing in results, it's probably because we haven't crawled your site since you added the tag. (Also, if you've used your robots.txt file to block this page, we won't be able to see the tag either.)
If the content is currently in our index, we will remove it after the next time we crawl it. To expedite removal, use the URL removal request tool in Google Webmaster Tools."
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=93710
-
Or canonical or by robots.txt
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
No: 'noindex' detected in 'robots' meta tag
Pages on my site show No: 'noindex' detected in 'robots' meta tag. However, when I inspect the pages html, it does not show noindex. In fact, it shows index, follow. Majority of pages show the error and are not indexed by Google...Not sure why this is happening. The page below in search console shows the error above...
Technical SEO | | Sean_White_Consult0 -
Home Page Deindexed overnight?
Hi, Hope you guys can help. I run an e-commerce site https://alloywheels.com Last night our home page (and a few other pages, but not all) were de-indexed by Google. The site has been ranking (UK) for years in P1 for the "alloy wheels" keyword and on the whole been running very successfully. However recently I have noticed from fluctuation on the "alloy wheels" keyword, dropping to P3 then P5 then back to P3, but this morning I noticed we were not even ranking on the first page. When I check inside Search Console there are no messages or warnings but the "/" page was de-indexed. There were a few other key pages that were also de-indexed. I have request reindexing and they have come back, P7 for the home page for "alloy wheels" The only thing I have changed was I realised yesterday there was no robots.txt on the site and was being recommended by web.dev to add one, so I did. It was just an allow all: User-agent: *
Technical SEO | | JamesDolden
Disallow Sitemap: https://alloywheels.com/sitemap.xml I ran tests on the robots.txt before it was uploaded and it all came green. I have removed the robots.txt for now. Has anybody seen anything like this before? With the recent ranking fluctuation I am not sure whether it is to do with that, the robots.txt or something different altogether? Thanks in advance, James0 -
Implemented google adwords via tag manager do it still require to paste script at thank you page?
Hi All Experts, I have implemented google adwords with tag manager, so now query is still it is required to place the google adwords scripts at thank you page?
Technical SEO | | varo0 -
Canonical Tags on Parameter Pages With Hreflang
Hey Everyone: We are currently implementing hreflang tags on our site, and we have many parameter pages with hreflang tags; however, I am afraid these may be counted as duplicate content without canonical tags. example.com/utm_source=tpi href='http://example.com/de" hreflang="de" rel="alternate" href='http://example.com/nl" hreflang="nl" rel="alternate" href='http://example.com/fr" hreflang="fr" rel="alternate" href='http://example.com/it" hreflang="it" rel="alternate" I have two questions 1. Do I need a canonical tag pointing to example.com ? 2. On the homepage without the parameter, should I add self referencing hreflang tags? (href="http://example.com/" hreflang="es" Thanks so much for your help! Kyle
Technical SEO | | TeespringMoz0 -
How do I find which pages are being deindexed on a large site?
Is there an easy way or any way to get a list of all deindexed pages? Thanks for reading!
Technical SEO | | DA20130 -
Meta description tag problems according to an seo tool
hi, my site is www.in2town.co.uk I am using an seo tool to check on my site and how to improve the seo. The tool is here. http://www.juxseo.com/report/view/51ebf9deab900 for some reason it has brought up errors, it claims i have not got a meta description even though i have and have doubled checked in my source code the errors it has brought up is as follows, and i would like to know if this is a fault of the seo tool or am i doing something wrong Does the description tag exist?0/1 <a id="sub_toggle_12" class="sub_toggle contract_sub"></a>Hide Info Description Tag: Explanation: The meta description tag does not help your rankings but it is your opportunity to encourage prospects to click. The meta description should describe the content of your web page, include a strong call to action, and include your keyword. Action: Make sure you are using the meta description tag. It is found in the section of your page. checkboxIs there only one description tag?0/2<a id="sub_toggle_13" class="sub_toggle expand_sub"></a>More InfocheckboxIs your description less than 156 characters?0/1<a id="sub_toggle_14" class="sub_toggle expand_sub"></a>More InfocheckboxIs your keyword in the description tag?0/3 <a id="sub_toggle_15" class="sub_toggle expand_sub"></a>More Info it also says about the canocial tag which it claims i have more than one Is the canonical tag optmized? Is there only one canonical tag?0/4 <a id="sub_toggle_10" class="sub_toggle contract_sub"></a>Hide Info Explanation: You only need one of these to direct a search engine. Don't muddy the waters. Action: Make sure you only have one canonical tag. This only applies if you use the canonical tag. any help and advice would be greatregards
Technical SEO | | ClaireH-1848860 -
Google webmaster tool doestn allow me to send 'URL and all linked pages"
Hello! I made a lot of optimization changes in my site ( seo urls, and a lot more ) , I always use Google Webmaster tools, fetch as Google Bot to refresh my site but now it doesnt allow me to 'Send URL and all linked pages' check the attachment Thank you
Technical SEO | | matiw0 -
My blog page isn't ranking in Google
Hi, I noticed that my blog page on my site isn't in Google when i search for full URL link http://www.asggutter.com/blog/ instead i see page that isn't even working asggutter.com/sitemap.xml screen shot http://screencast.com/t/6OVFLwL8nTL How i can i fix that. Thanks
Technical SEO | | tonyklu0