Tens of duplicate homepages indexed and blocked later: How to remove from Google cache?
-
Hi community,
Due to some WP plugin issue, many homepages indexed in Google with anonymous URLs. We blocked them later. Still they are in SERP. I wonder whether these are causing some trouble to our website, especially as our exact homepages indexed. How to remove these pages from Google cache? Is that the right approach?
Thanks
-
Hi Nigel,
Thanks for the suggestion. I'm going to use "Remove URLs" tool from GSC. They have been created due to a bug in the Yoast SEO plugin. Very unfortunate and we paid for no mistake from our end.
Removing from SERP means removing from Google index also? Or Google will still consider them and just stops showing us? My intention is: Anyway we blocked them, but whether they will cause some distraction to our ranking efforts being there in results being cached.
Thanks
-
Thanks!
A agree - I have just done a similar clean up by:
1. Don't let them be created
2. Redirect all previous versions!One site I just worked on had 8 versions of the home page! lol
http
https
/index.php
/index.php/A mess!
We stopped them all being created and 301'd all versions just in case they were indexed anywhere or linked externally.
Cheers
-
It is assuredly true that, just like in any number of fields (medicine) - in SEO, prevention is better than cleanup based methodology. If your website doesn't take its medicine, you get problems like this one
I think your advice here was really good
-
Good solid advice
They can be created in any number of ways but it's normally simple enough to specify the preferred URL on the server then move any variations in htaccess, such as those with www (if the none www is preferred), those with a trailing slash at the end etc.
The self canonical on all will sort out any other duplicates.
As for getting rid of them - the search console way is the quickest. If they don't exist after that then the won't be reindexed unless they are linked from somewhere else. In such cases, they will 301 from htaccess so it shouldn't be a problem.
if you 410 you will lose any benefit from those links going to the pages and it's a bad experience for a visitor. Always 301 do not 410 if it is a version.
410s are fine for old pages you never want to see in the index again but not for a home page version.
Regards
Nigel
-
It's likely that you don't have access to edit the coding on these weird plugin URLs. As such, normal techniques like using a Meta no-index tag in the HTML may be non-viable.
You could use the HTTP header (server level stuff) to help you out. I'd advise adding two strong directives to the afflicted URLs through the HTTP header so that Google gets the message:
-
Use the X-Robots deployment of the no-index directive on the affected URLs, at the HTTP header (not the HTML) level. That linked pages tells you about the normal HTML implementation, but also about the X-Robots implementation which is the one you need (scroll down a bit)
-
Serve status code 410 (gone) on the affected URLs
That should prompt Google to de-index those pages. Once they are de-indexed, you can use robots.txt to block Google from crawling such URLs in the future (which will stop the problem happening again!)
It's important to de-index the URLs before you do any robots.txt stuff. If Google can't crawl the affected URLs, it can't find the info (in the HTTP header) to know that it should de-index those pages
Once Google is blocked from both indexing and crawling these pages, they should begin to stop caching them too
Hope that helps
-
-
+1 for "Make sure that they are not created in the first place" haha
-
Hi again vtmoz!
1. Make sure that they are not created in the first place
2. Make sure that they are not in the sitemap
3. Go to search console and remove any you do not want - it will say temporary removal but they will not come back if they are not in the structure or the sitemap.More:
https://support.google.com/webmasters/answer/1663419?hl=en
Note: Always self canonicalize the home page to stop versions with UTM codes (created by Facebook, Twitter etc) appearing in SERPS
Regards
Nigel
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does Google use dateModified or date Published in its SERPs?
I was curious as to the prioritization of dateCreated / datePublished and dateModified in our microdata and how it affects google search results. I have read some entries online that say Google prioritizes dateModified in SERPs, but others that claim they prioritize datePublished or dateCreated. Do you know (or could you point me to some resources) as to whether Google uses dateModified or date Published in its SERPs? Thanks!
Algorithm Updates | | Parse.ly0 -
What are the top tips for winning on Google, Bing, and Yahoo?
We just launched a new site that is starting to be indexed well in Google, but Bing and Yahoo are lagging a bit. I understand that the search engines are different algorithms and will take different lengths of time to index, rank, etc. What I'm curious about is are there any other tips / advice / things to keep in mind when trying to rank on the different search engines? Thanks!!
Algorithm Updates | | Emily_A0 -
Google Penguin update
When Google Penguin update will run again. The last time was in October 2013 and I'm still really curious now. Or have they stopped this and this is now continuously just like the panda?
Algorithm Updates | | NECAnGeL0 -
Google Places/Points of Interest Rankings?
Does anyone have an idea on how Google ranks or determines the 'Points of Interests' that come up when searching about places/cities?
Algorithm Updates | | CarlLarson0 -
Webpage is ranking on google.ie / google.co.uk but not google.com?
One of our site webpage appears to be found in the first few pages on google.ie / google.co.uk but not on google.com. Is there such a thing being penalised on a specific Google domain? Traffic is healthy despite this but I want to rank well for the page in google.com. Any ideas?
Algorithm Updates | | notnem0 -
Is it normal to receive 2 mails from Google?
I filed a reconsideration request that was answered in less than a week. Subsequently I was told that no manual penalty was in place but various algorithm factors might be causing my heavy drops in ranking. Then I got a second email which was even more specific. This was great, really heartening stuff and a total surprise as it was very helpful. Is it normal to receive 2 emails from Google with such clear information? I have been very pleased by the comments they have made as it has shown me that they're more customer focused than I had been led to believe by all the research I had done pre reconsideration request. Has anyone else had a clear outline of what they needed to fix and has their site subsequently rebounded post fixing?
Algorithm Updates | | swimwithfishes0 -
Decent rankings in Google, nothing in Bing and Yahoo
Hi there, I'm in the process of SEOing a site in a very competitive sector, the short term loans market. The URL for the site is http://www.piggy-bank.co.uk. I've managed to get a fair bit of success in Google for some very competitive keywords like short term loans, short term lender etc but in Bing and Yahoo I'm having no luck at all, with only 2 visits in the past month and no decent rankings!! I think I'm doing everything right, with regular new content on the site, decent technical SEO, semantic site structure, regular site map upload, a 10 year old domain, holistic link building through guest blogging etc, but still no luck at all. Looking at the webmaster tools in Bing, 95% of the URLs are indexed, but I'm getting such a low impression count, and obviously, an even lower click through. Am I missing something really obvious? Does anyone have any suggestions to improve my Bing and Yahoo rankings? I've worked on 100s of other sites and Yahoo and Bing tend to be the easy win to make the client happy 😉 Thanks in advance for your help. Dan
Algorithm Updates | | djslimited1 -
Duplicate Content & www.3quarksdaily.com, why no penalty?
Does anyone have a theory as to why this site does not get hit with a DC penalty? The site is great, and the information is good but I just cannot understand the reason that this site does not get hit with a duplicate content penalty as all articles are posted elsewhere. Any theories would be greatly appreciated!
Algorithm Updates | | KMack0