NOINDEX content still showing in SERPS after 2 months
-
I have a website that was likely hit by Panda or some other algorithm change. The hit finally occurred in September of 2011. In December my developer set the following meta tag on all pages that do not have unique content:
name="robots" content="NOINDEX" />
It's been 2 months now and I feel I've been patient, but Google is still showing 10,000+ pages when I do a search for site:http://www.mydomain.com
I am looking for a quicker solution. Adding this many pages to the robots.txt does not seem like a sound option. The pages have been removed from the sitemap (for about a month now). I am trying to determine the best of the following options or find better options.
- 301 all the pages I want out of the index to a single URL based on the page type (location and product). The 301 worries me a bit because I'd have about 10,000 or so pages all 301ing to one or two URLs. However, I'd get some link juice to that page, right?
- Issue a HTTP 404 code on all the pages I want out of the index. The 404 code seems like the safest bet, but I am wondering if that will have a negative impact on my site with Google seeing 10,000+ 404 errors all of the sudden.
- Issue a HTTP 410 code on all pages I want out of the index. I've never used the 410 code and while most of those pages are never coming back, eventually I will bring a small percentage back online as I add fresh new content. This one scares me the most, but am interested if anyone has ever used a 410 code.
Please advise and thanks for reading.
-
Just wanted to let you know that submitting all the sites I wanted removed into an XML sitemap worked. I then submitted that sitemap to webmaster tools and listed it in the robots.txt. When doing query "site:domain.com" index pages went from 20k+ down to 700 in a matter of days.
-
I could link to them then, but what about creating a custom sitemap for just content that I want removed? Would that have the same effect?
-
If they are not linked to then spiders will not find the noindex code. They could suffer in the SERPs for months and months.
-
If all these pages are under a directory structure than you have the option to remove a complete directory in URL removal option. See if that is feasible in your case.
-
I suppose I'll wait longer. Crawl rate over the last 90 days is a high of 3,285 and average of 550 with a low of 3 according to webmaster tools.
-
Yeah the pages are low PR and are not linked to at all from the site. I've never heard of removing a page via webmaster tools. How do I do that? I also have to remove several thousand.
*edit: It looks like I have to remove them one at a time which is not feasible in my case. Is there a faster way?
-
If you want a page out of the index fast the best way is to do it through webmaster tools. It's easy and lasts for about six months. Then, if they find your page again it will register the noindex and you should be fine.
As EGOL said, if it's a page that isn't crawled very often then it could be a LONG time before it gets deindexed.
-
I removed some pages from the index and used the same line of code...
name="robots" content="NOINDEX" />
My pages dropped from the index within 2 or 3 days - but this is a site that has very heavy spider activity.
If your site is not crawled very much or these are low PR pages (such as PR1, PR2) it could take google a while to revisit and act upon your noindex instructions - but two months seems a bit long.
Is your site being crawled vigorously? Look in webmaster tools to see if crawling declined abruptly when your rankings fell. Check there also for crawl problems.
If I owned your site and the PR of these pages is low I would wait a while longer before doing anything. If my patience was wearing thin I would do the 301 redirect because that will transfer the linkjuice from those pages to the target URL of the redirect - however, you might wait quite a while to see the redirect take effect. That's why my first choice would be to wait longer.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Domain migration via Wordpress - organic still 60% down four months later
Hi guys!Almost four months ago I performed a Wordpress domain migration. Three pet-based sites were migrated into a new pet-based one that incorporated them all - the new site is petskb.com and 240 posts were migrated.The site migration was performed via 301 wildcard re-directs using the .htaccess files in the old domains, which are still in place and working. I also used the site move tool in GSC. Afterwards, I performed an audit of the new site to ensure that all the old urls were being re-directed to the new one, which they were (and are). There have been no manual actions reported in GSC.The results have been very poor. A small few of the articles that were in the top 10 moved over and I quickly claimed the same positions in the new site. Most did not though and still sit >100 in the SERP or absolutely nowhere (or even omitted) using the main keyword.I've created about 60 new articles (using the same SEO analysis I did previously) since that time on the new site and not one of them has ranked <100 in all that time, whereas on the old sites they would initially rank somewhere in the top 50 after a couple of days and work their way up over the months. These new posts haven't moved though. The posts that were published on the new site four months ago are still in the exact same position.So, I've created new content, re-submitted the sitemap and manually requested re-indexing of the posts. Nothing has changed. I've hired SEO's and not one has found any problems with my site or how I performed the migration. Clearly there is a problem though. The original posts that were ranking previously and all the new posts have not moved in the SERP. There were a few spammy links pointing to the new domain but nothing significant, I did disavow these though - no more than on the old sites though.As a test, I created a new post on another domain which has no posts with the same long-tail keyword as one that has been on my new site for almost four months. The one I posted on the new domain out-ranked the one on petskb after just two days.Can anyone help? If you can I will personally travel to where you live and buy you several beers.Thanks,Matt
Intermediate & Advanced SEO | | mattpettitt1 -
Recent drop in SERP from #1 to #2\.
Our #1 priority keyword, which we've ranked #1 for years, suddenly we're #2. And, the page listed above us doesn't seem to be even compete. Moz On-Page has us at an A and them at a C. When I review the html, I don't even seem any exactly keyword match or matching text on the page. I checked are ranking last week and didn't notice any change - so I've narrowed it down to something changing in the last 4-5 days. Also of note, when I test, We're #1 on mobile, #2 on desktop. Sorry to not list the url's. omitted intentionally. Any thoughts would be much appreciated.
Intermediate & Advanced SEO | | FX4nWOO0 -
Duplicate content on product pages
Hi, We are considering the impact when you want to deliver content directly on the product pages. If the products were manufactured in a specific way and its the same process across 100 other products you might want to tell your readers about it. If you were to believe the product page was the best place to deliver this information for your readers then you could potentially be creating mass content duplication. Especially as the storytelling of the product could equate to 60% of the page content this could really flag as duplication. Our options would appear to be:1. Instead add the content as a link on each product page to one centralised URL and risk taking users away from the product page (not going to help with conversion rate or designers plans)2. Put the content behind some javascript which requires interaction hopefully deterring the search engine from crawling the content (doesn't fit the designers plans & users have to interact which is a big ask)3. Assign one product as a canonical and risk the other products not appearing in search for relevant searches4. Leave the copy as crawlable and risk being marked down or de-indexed for duplicated contentIts seems the search engines do not offer a way for us to serve this great content to our readers with out being at risk of going against guidelines or the search engines not being able to crawl it.How would you suggest a site should go about this for optimal results?
Intermediate & Advanced SEO | | FashionLux2 -
Content question please help
Would content behind a drop down on this site Https://www.homeleisuredirect.com/pool_tables/english_pool_tables/ you have to click the - more about English pool tables text under the video Work just as well for SEO as content on the page like this site http://www.pooltablesonline.co.uk/uk-slate-bed-pool-tables.asp
Intermediate & Advanced SEO | | BobAnderson0 -
2 pages optimised for same keyword... what should I do?
Hi, I have two pages appearing in positions 11 and 12 for the keyword: 80 btl mortgage. These are: https://www.commercialtrust.co.uk/btl/landlord-advice/mortgages/btl-mortgage-80-ltv/ https://www.commercialtrust.co.uk/btl/product-types/80-buy-to-let-mortgages/ Both pages are good, provide useful information and I would not wish to remove one of them. However, I am concerned that the reason neither one of the pages is on page 1 is because the keywords targeted on both pages is essentially the same. Should I reoptimise one of them for other variations of 80 BTL mortgage keywords? (e.g. 80% LTV Buy to Let Mortgage, 80 Buy to Let Mortgage, etc etc) Or, is there another solution I haven't yet thought of? I welcome your insights! Thanks! Amelia
Intermediate & Advanced SEO | | CommT0 -
Press Release and Duplicate Content
Hello folks, We have been using Press Releases to promote our clients business for a couple of years and we have seen great results in referral traffic and SEO wise. Recently one of our clients requested us to publish the PR on their website as well as blast it out using PRWeb and Marketwire. I think that this is not going to be a duplicate content issue for our client's website since I believe that Google can recognize which content has been published first, but I will be more than happy to get some of the Moz community opinions. Thank you
Intermediate & Advanced SEO | | Aviatech0 -
Duplicate content
Is there manual intervention required for a site that has been flagged for duplicate content to get back to its original rankings, once the duplicated content has been removed? Background: Our site recently experienced a significant drop in traffic around the time that a chunk of content from other sites (ie. duplicate) went live. While it was not an exact replica of the pages on other sites, there was quite a bit of overlap. That content has since been removed, but our traffic hasn't improved. What else can we do to improve our ranking?
Intermediate & Advanced SEO | | jamesti0 -
Image and Content Management
My boss has decided that on our new website we are building, that he wants all content and images managed by not allowing copying content and/or saving images. Some of the information and images is proprietary, yet most is available for public viewing, but never the less, he wants it prohibited from copy and/or saving. We would still want to keep the content indexable and use appropriate alt tags etc... I wanted to find out if there is any SEO reason and facts to why this would not be a good idea?Would implementing code to prohibit (or at least make it difficult) to save images and copy content, penalize us?
Intermediate & Advanced SEO | | KJ-Rodgers0