Old URLs Appearing in SERPs
-
Thirteen months ago we removed a large number of non-corporate URLs from our web server. We created 301 redirects and in some cases, we simply removed the content as there was no place to redirect to.
Unfortunately, all these pages still appear in Google's SERPs (not Bings) for both the 301'd pages and the pages we removed without redirecting. When you click on the pages in the SERPs that have been redirected - you do get redirected - so we have ruled out any problems with the 301s.
We have already resubmitted our XML sitemap and when we run a crawl using Screaming Frog we do not see any of these old pages being linked to at our domain.
We have a few different approaches we're considering to get Google to remove these pages from the SERPs and would welcome your input.
- Remove the 301 redirect entirely so that visits to those pages return a 404 (much easier) or a 410 (would require some setup/configuration via Wordpress). This of course means that anyone visiting those URLs won't be forwarded along, but Google may not drop those redirects from the SERPs otherwise.
- Request that Google temporarily block those pages (done via GWMT), which lasts for 90 days.
- Update robots.txt to block access to the redirecting directories.
Thank you.
Rosemary
One year ago I removed a whole lot of junk that was on my web server but it is still appearing in the SERPs.
-
You're right - I'm worrying about something that isn't yet a problem.
Thank you
-
In my experience, the best way to absolutely get rid of them is to use the 410 permanently gone status code, then resubmit them for indexation (possibly via an XML sitemap submission, and you can also use Google's crawl testing tool in Search Console to double-check). That said, even with 410, Google can take their time.
The other option is to recreate 200 pages there and use the meta robots noindex tag on the page to specifically exclude them. The temporary block in Google Search Console can work, too, but, it's temporary and I can't say whether it will actually extend the time that the redirected pages appear in the index via the site: command.
All that said, if the pages only show via a site: command, there's almost no chance anyone will see them
-
Ok, Rand - one last questions.
I do think one year is a long time to have old results and if I was going to do a test to get Google to stop showing them in their SERPs what would you do? --- Let's say a client asked you to have these URLs disappear
The 79 pages that appear in the /eichler/ directory are from a personal site so I don't care what happens with those pages in the SERPs.
My ideas are:
-
Remove the 301 redirect entirely so that visits to those pages return a 404 (much easier) or a 410 (would require some setup/configuration via Wordpress). This of course means that anyone visiting those URLs won't be forwarded along, but Google may not drop those redirects from the SERPs otherwise.
-
Request that Google temporarily block those pages (done via GWMT), which lasts for 90 days.
-
Update robots.txt to block access to the redirecting directories.
-
Remove the 301 redirect entirely so that visits to those pages return a 404 (much easier) or a 410 (would require some setup/configuration via Wordpress). This of course means that anyone visiting those URLs won't be forwarded along, but Google may not drop those redirects from the SERPs otherwise.
-
Request that Google temporarily block those pages (done via GWMT), which lasts for 90 days.
-
Update robots.txt to block access to the redirecting directories.
-
-
14 months! Wow. That is a long time indeed. Although, now that I look, Moz redirected OpenSiteExplorer just about a year ago, and we still have URLs showing for the site: command in Google too (https://www.google.com/search?q=site%3Aopensiteexplorer.org) so I suppose it's not that uncommon.
Glad to hear traffic and rankings are solid. Let us know if we can help out in the future!
-
Thank you Rand. It has been 14 months since these pages were moved and I'd never seen Google retain pages anywhere near this long.
You're right of course, there has been no impact to traffic for our site as these pages weren't about our search business.
Thanks for taking a look at our issue.
Rosemary
-
Oh gosh - it's my pleasure! Thanks for being part of the Moz community
I'm honored to help out.
As for the URLs - looks like everything's fine. Google often maintains old URLs in a searchable index form long after they've been 301'd, but for every query I tried, they're clearly pulling up the correct/new version of the page, so those redirects seem to be working just great. You're simply seeing the vestigal remnants of them still in Google (which isn't unusual - we had URLs from seomoz.org findable via site: queries for many months after moving to Moz, but the right, new pages were all ranking for normal queries and traffic wasn't being hurt).
Some examples:
- https://www.google.com/search?q=Enter+the+World+of+Eichler+Design
- https://www.google.com/search?q=Eichler+History+flashbacks
- https://www.google.com/search?q=eichler+resources+on+the+web+books
Unless you're also seeing a loss in search traffic/rankings, I wouldn't sweat it much. They'll disappear eventually from the site: query, too. It just takes a while.
-
Wow - do I ever feel privileged to have you respond! Thank you Rand.
You can see a batch of redirected URLs here < site:totheweb.com eichler >
I appreciate any suggestions.
Rosemary
-
Hi Rosemary - can you share some examples of the URLs and the queries that bring them up in search results? If so, we can likely do a diagnosis of what might be going on with Google and why the pages aren't correctly showing the redirected-to URLs.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Old domain to new domain
Hi, A website on server A is no longer required. The owner has redirected some URLS of this website (via plugin) to his new website on server B -but not all URLS. So when I use COMMAND site:website A , I see a mixture of redirected URLS and not redirected URLS.Therefore two websites are still being indexed in some form and causing duplication. However, weirdly when I crawl with Screaming Frog I only see one URL which is 301 redirected to the new website. I would have thought I'd see lots of URLs which hadn't been redirected. How come it is different to using the site:command? Anyway, how do I move to the new website completely without the old one being indexed anymore. I thought I knew this but have read so many blogs I've confused myself! Should I: Redirect all URLS via the HTACESS file on old website on server A? There are lots of pages indexed so a lot of URLs. What if I miss some? or Point the old domain via DNS to server B and do the redirects in website B HTaccess file? This seems more sensible but does this method still retain the website rankings? Thanks for any help
Technical SEO | | AL123al0 -
New URL Structure
Hi Guy's, For our webshop we're considering a new URL structure because longtail keywords to rank so well. Now we have /category (main focus keywords)
Technical SEO | | Happy-SEO
/product/the-product345897345123/ (nice to rank on, not that much volume) We have over 500 categories and every one of them is placed after our domain. Because i think it's better to work with a good structure and managed a way to make categories and sub-categories. The 500 categories may be the case why not every one of them is ranking so well, so that was also the choice of thinking about a new structure. So the new URL structure will be: /category (main focus keywords)
/category/subcat/ (also main focus keywords) Everything will be redirect (301, good way), so i think there won't be to much problems. I'm thinking about what to do with the /product/ URL. Because now it will be on the same level as the subcategories, and i'm affraid that when it's on that level, Google will give the same value to both of them. My options that i'm considering are: **Old way **
/product/the-product-345897345123/ .html (seen this on big webshops)
/product/the-product-345897345123.html/ Level deeper SKU /product/the-product/345897345123/ What would you suggest? The new structure would be 20 categories 500+ sub's devided under main categories 5000+ products Thanks!0 -
URL Redirect
Hi All, So we have employees who can own their own domains for business, however, one employee has a domain that links back to our main site, but when it does, the URL and Page title of our main site, still say his own domain. IE: www.johndoe.com links to www.mysite.com except the url and itle still say www.johndoe.com What are the implications of this? Thank you
Technical SEO | | PeteEllard0 -
URL structure
Hello Guys, Quick Question regarding URL strucutre One of our client is an hotel chain, thye have a group site www.example.com and each property is located in a subfolder: www.example.com/example-boston.html , www.example.com/example-ny.html etc. My quesion is : where is better to place the language extension at a subfolder level?
Technical SEO | | travelclickseo
Should i go for www.example.com/en/example-ny.html or it is preferable to specify the language after the property name www.example.com/example-ny/en/accommodation.html? Thanks and Regards, Alessio0 -
Removing URL Parentheses in HTACCESS
Im reworking a website for a client, and their current URLs have parentheses. I'd like to get rid of these, but individual 301 redirects in htaccess is not practical, since the parentheses are located in many URLs. Does anyone know an HTACCESS rule that will simply remove URL parantheses as a 301 redirect?
Technical SEO | | JaredMumford0 -
URL parameter reduction plug in
Anyone know of a good plug-in that reduces the amount parameters used in URLs? I need one for an ASP based system and a PHP based system
Technical SEO | | matmox0 -
Special characters in URL
Hello everybody, my question focus on special parameters in URL. I i am working for a website that use a lot of special entities in their URLS. For instance: www.mydomain.com/mykeyword1-mykeyword2%2C-1%2Cpage1.html I am about to make 301 redirect rules for all these urls to clean ones. IE: www.mydomain.com/mykeyword1-mykeyword2%2C-1%2Cpage1
Technical SEO | | objectif-mars
would become:
www.mydomain.com/mykeyword1-mykeyword.html I just wanted to know if anybody has already done this kind of "cleanup" and if i could expect a positive boost or not. Thanks0 -
SERP data went away??
As of a day ago, the SERPs in Google are showing our listing with NO meta description at all and the incorrect title. Plus the Title is varying based on the keywords searched. Info: Something I just had done was have the multiple versions of their home page (duplicate content, about 40 URLs or so) 301 redirected the the appropriate place. I think they accidentally did 302s. Anyone seen this before? Thanks
Technical SEO | | poolguy0