Removing a Page From Google index
-
We accidentally generated some pages on our site that ended up getting indexed by google. We have corrected the issue on the site and we 404 all of those pages. Should we manually delete the extra pages from Google's index or should we just let Google figure out that they are 404'd? What the best practice here?
-
Thanks Ryan. I fully understand what you are saying and will be careful while making the change.
-
Hi Atul,
Generally speaking I am uncomfortable advising others on specific changes to the .htaccess file. If you make even a slight error while working with the file, your site security can be compromised, not to mention your SEO. There are also many factors to consider such as which mods are enabled on your particular server along with other configuration issues. Lastly, the order in which your code is placed in the file can effect it's operation so it's not like adding a meta tag to the section of an html document.
If you are on managed hosting, my recommendation is to ask your web host to make the change. If you are not on managed hosting, I recommend asking the developer who manages the site to make the change.
If you still insist on making the change yourself, try
Redirect gone /ABC/xyz.html
-
After reading your answer, i searched for methods on how to generate 410 error.
Lets say i want to remove a page named xyz.html.
Which of the following entry in .htaccess is correct
Redirect gone xyz.html
or
Redirect gone /xyz/
If xyz were in a folder named ABC,
would it be correct
Redirect gone /ABC/xyz.html
Thaks
-
Thanks everyone! We are just going to leave it as is. Google will eventually flush it out. Ryan - because of the 90 days we can't remove the URL's. I will need them back sooner than that when we actually put products in those states. Thanks again! helpful....as usual!
-
Remove URL tool will just expedite the inevitable. There is no downside in doing so.
I agree with everything you shared Esko up to this point. Aside from the time spent to remove the page, there is another downside. The URL you remove will not appear in SERPs again for 90 days after being manually removed.
If your URL was mysite.com/blue-widgets then your site will not have another /blue-widgets page listed again for 90 days. I can share it is a headache as an SEO trying to figure out why a page is not being indexed, and later learning I did not ask all the right questions i.e. "Prior to hiring my services, have you or anyone with access to your WMT account used the URL Removal tool within the past 90 days?". That otherwise obscure question now is asked regularly of my clients. Painful lesson.
Also, I wanted to share another helpful link I located from Google: When NOT to use the URL Removal tool.
-
Google will completely drop the page from the index after the next time they crawl it. Using the Remove URLs tool in Google Webmaster Tools will only expedite removal.
Best practice is to 404 (Not Found) or 410 (Gone) the page first of all.
Remove URL tool will just expedite the inevitable. There is no downside in doing so.
-
Should we manually delete the extra pages from Google's index or should we just let Google figure out that they are 404'd? What the best practice here?
The best practice would be to generate a 410 error (GONE) for the pages and Google will remove them from their index fairly quickly.
The next best practice would be to leave the pages as 404s and Google will still remove them from their index but it will take a bit longer.
A 410 is used to inform Google and others the page is definitely gone. A 404 merely states the page is unavailable now. It could be available later.
The removal tool should only be used if it is a major concern for the search result to appear in SERPs. An example would be if confidential information was leaked.
-
I think it's always good to let Google know as they might remove it sooner. But there's no guarantee either way. Though if you can, you should 301 your content to a new/similar page rather than just let it 404.
-
I understand HOW to remove a page. I want to know whether it's better for me to manually remove it or let Google remove it on its own.
-
Remove a page from Google's Index
Use Google Webmaster Tools www.google.com/webmasters/tools/
http://www.google.com/support/webmasters/bin/answer.py?answer=1663419
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Specific page does not index
Hi, First question: Working on the indexation of all pages for a specific client, there's one page that refuses to index. Google Search console says there's a robots.txt file, but I can't seem to find any tracks of that in the backend, nor in the code itself. Could someone reach out to me and tell me why this is happening? The page: https://www.brody.be/nl/assistentiewoningen/ Second question: Google is showing another meta description than the one our client gave in in Yoast Premium snippet. Could it be there's another plugin overwriting this description? Or do we have to wait for it to change after a specific period of time? Hope you guys can help
Intermediate & Advanced SEO | | conversal0 -
Password Protected Page(s) Indexed
Hi, I am wondering if my website can get a penalty if some password protected pages are showing up when I search on google: site:www.example.com/sub-group/pass-word-protected-page That shows that my password protected page was indexed either before or after adding the password protection. I've seen people suggest no indexing the page. Is that the best method to take care of this? What if we are planning on pushing the page live later on? All of these pages have no title tag, meta description, image alt text, etc. Should I add them for each page? I am wondering what is the best step, especially if we are planning on pushing the page(s) live. Thanks for any help!
Intermediate & Advanced SEO | | aua0 -
Major Landing page removed from Google SERP and replace homepage URL.How do I fix it?
Hi Major Landing page removed from Google SERP and replace homepage URL.How do I fix it? In an SPA website (angularJS), Why it happens?
Intermediate & Advanced SEO | | cafegardesh0 -
Google Adsbot crawling order confirmation pages?
Hi, We have had roughly 1000+ requests per 24 hours from Google-adsbot to our confirmation pages. This generates an error as the confirmation page cannot be viewed after closing or by anyone who didn't complete the order. How is google-adsbot finding pages to crawl that are not linked to anywhere on the site, in the sitemap or linked to anywhere else? Is there any harm in a google crawler receiving a higher percentage of errors - even though the pages are not supposed to be requested. Is there anything we can do to prevent the errors for the benefit of our network team and what are the possible risks of any measures we can take? This bot seems to be for evaluating the quality of landing pages used in for Adwords so why is it trying to access confirmation pages when they have not been set for any of our adverts? We included "Disallow: /confirmation" in the robots.txt but it has continued to request these pages, generating a 403 page and an error in the log files so it seems Adsbot doesn't follow robots.txt. Thanks in advance for any help, Sam
Intermediate & Advanced SEO | | seoeuroflorist0 -
Organic Listings showing Google Tag Manager + Google Page Title...?
I'm a bit stumped with this. I optimise all my titles etc for Australia - and now the organic liatings are showing something strange. For example ( we sell health supplements ) Meta title = "My Product , Buy Online Australia" If I type "My Product" - the title in the organic listings says "My Product - My Company Limited" - and the only place I can see it getting that from is a combination of Meta Data used in Google Tag Manager + the Name on my Google places page. This is much more obvious for categories.. but it's a pain in the butt. If I type "My Product Australia" Then the original "My Product , Buy Online Australia" comes up. Any ideas on policy etc? I have taken the "Limited" off the Google business page - so hopefully this will change over time - but I can't find any information on why google would do something like this. If you had shed any light on this - would be much appreciated.
Intermediate & Advanced SEO | | s_EOgi_Bear0 -
New Web Page Not Indexed
Quick question with probably a straightforward answer... We created a new page on our site 4 days ago, it was in fact a mini-site page though I don't think that makes a difference... To date, the page is not indexed and when I use 'Fetch as Google' in WT I get a 'Not Found' fetch status... I have also used the'Submit URL' in WT which seemed to work ok... We have even resorted to 'pinging' using Pinglar and Ping-O-Matic though we have done this cautiously! I know social media is probably the answer but we have been trying to hold back on that tactic as the page relates to a product that hasn't quite launched yet and we do not want to cause any issues with the vendor! That said, I think we might have to look at sharing the page socially unless anyone has any other ideas? Many thanks Andy
Intermediate & Advanced SEO | | TomKing0 -
Certain Product Pages Not Indexing
Hey All, We discovered an issue where new product pages on our site were not getting indexed because a "noindex" tag was inadvertently being added to section when those pages were created. We removed the noindex tag in late April and some of the pages that had not been previously indexed are now showing up, but others are still not getting indexed and I'd appreciate some help on why this could be. Here is an example of a page that was not in the index but is now showing after removal of noindex: http://www.cloud9living.com/san-diego/gaslamp-quarter-food-tour And here is an example of a page that is still not showing in the index: http://www.cloud9living.com/atlanta/race-a-ferrari UPDATE: The above page is now showing after I manually submitted it in WMT. I had previously submitted another page like a month ago and it was still not indexing so I thought the manual submission was a dead end. However, it just so happens that the above URL just had its Page Title and H1 updated to something more specific and less duplicative so I am currently running a test to see if that's the problem with these pages not indexing. Will update this soon. Any suggestions? Thanks!
Intermediate & Advanced SEO | | GManSEO0 -
Best way to stop pages being indexed and keeping PageRank
If for example on a discussion forum, what would be the best way to stop pages such as the posting page (where a user posts a topic or message) from being indexed AND not diluting PageRank too? If we added them to the Disallow on robots.txt, would pagerank still flow through the links to those blocked pages or would it stay concentrated on the linking page? Your ideas and suggestions will be greatly appreciated.
Intermediate & Advanced SEO | | Peter2640