Website blog is hacked. Whats the best practice to remove bad urls
-
Hello
So our site was hacked which created a few thousand spam URLs on our domain. We fixed the issue and changed all the spam urls now return 404. Google index shows a couple of thousand bad URLs.
My question is-
What's the fastest way to remove the URLs from google index. I created a site map with sof the bad urls and submitted to Google. I am hoping google will index them as they are in the sitemap and remove from the index, as they return 404.
Any tools to get a full list of google index? ( search console downloads are limited to 1000 urls). A Moz site crawl gives larger list which includes URLs not in Google index too. Looking for a tool that can download results from a site: search.
Any way to remove the URLs from the index in bulk? Removing them one by one will take forever.
Any help or insight would be very appreciated.
-
Technically 404 means "temporarily unavailable but coming back later" so you might want to consider Status 410 instead of 404. You could also supplement it with Meta no-index, if you can't use the HTML implementation then fire the no-index directive through the HTTP header using X-robots:
https://developers.google.com/search/reference/robots_meta_tag (scroll down a little to find the relevant part)
E.g:
"HTTP/1.1 200 OK
Date: Tue, 25 May 2010 21:42:43 GMT
(…)
X-Robots-Tag: noindex
(…)"... something like that.
You can't use Search Console to remove URLs from Google at all. The remove URL tool, only removes URLs one at a time and it only does so 'temporarily', the URLs pop back again after a bit. The best thing you can do is give Google some harsher directives and hope they listen, in a month or two most of those should be gone
Don't use robots.txt on the URLs as, if Google can't crawl them it won't find the 410s or the no-index directives
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEO best practice : HTTP to HTTPS
What's the best practice to switch from an all HTTP site to an all HTTPS site ?
Technical SEO | | Crocodesign
No changes to the site structure, just a full site switch to SSL.
Right now, the site is reachable with HTTP and with HTTPS. http://crocodesign.be --> https://crocodesign.be
http://www.crocodesign.be --> https://crocodesign.be
https://www.crocodesign.be --> https://crocodesign.be CMS : Wordpress 3.9
Server type : Apache
Preferred method : .htaccess0 -
Best practice around removing large section of the website
We are looking at removing a large section of our website that is getting low/no traffic. My current thought of removing this would be to delete the pages and add 301 redirects to a similar page within the site that is not being deleted. This will be removing 400+ pages, does it this make sense? Or should we point them to the homepage? Finally should we do this in one batch or should we slowly remove the pages over the course of a couple weeks. Thanks - appreciate the help in understanding the best practice in terms of SEO.
Technical SEO | | webactive0 -
Duplicate blog URLs in Magenton
On one my sites Moz is picking up 4483 duplicate content pages. The majority of these are from our blog and video sections on our site. We're using a URL shortener and it appears that some of the pages are the full version of the URL then the shortened version. However if you go to the full version you get redirected to the shorter one. So I would assume that the Moz crawler should get the same redirect? We're also getting pagination being shown as duplicate pages, which I would half expect, but the URLs Magento is creating are truly bizarre: e.g http://www.xxx.com/uk/blog/cat/view/identifier/news/page/news/index.php/alarms-doorbells/?p=2 Alarms and doorbells is one of our product categories, which is displayed in the LHN on the blog page but has nothing to do with the blog itself. On another site on the same Magento instance, with the same content (they're for two different regions) we're show as having 248 duplicate pages, again in the video and news section, but this is a completely different scale of issue. Has anyone else encountered issues like these? I'm probably going to put a noindex in place on these two sections until we can get a solution in place as we're completely unranked in google on this site. Thanks
Technical SEO | | ahyde0 -
Blog.domain or domain.com/blog
My client can't do domain.com/blog because he's on wix. I'm thinking blog.domain.com. Do you have any resources for the pros and cons of this? I understand that google looks at them very similarly now, is that true for google +?
Technical SEO | | tylerfraser0 -
600+ 404 Errors: Best Practice for Redirects?
Hi All, I've just checked my GWMT profile for one of my client's sites and found that there are currently over 600 404 Error notifications! This is not that surprising given that we very recently redesigned and launched their new corporate site, which previously had a ton of "junk" legacy pages. I was wondering if it would work in terms of efficient SEO to simply apply a 301 redirect from the 404 page to our root to solve this issue? If not what would be a good solution? Thanks in advance for all your great advice!
Technical SEO | | G2W1 -
Updating content on URL or new URL
High Mozzers, We are an event organisation. Every year we produce like 350 events. All the events are on our website. A lot of these events are held every year. So i have an URL like www.domainname.nl/eventname So what would you do. This URL has some inbound links, some social mentions and so on. SO if the event will be held again in 2013. Would it be better to update the content on this URL or create a new one. I would keep this URL and update it because of the linkvalue and it is allready indexed and ranking for the desired keyword for that event. Cheers, Ruud
Technical SEO | | RuudHeijnen0 -
Remove Bad Links Or Build New
Hello, After deeply assessing our back links we have come to the conclusion that we have too many links that have been devalued and also some spammy looking links.... Our next question is do we remove these bad links and start a fresh or do we just build new white hat links?? Thanks, Scott
Technical SEO | | ScottBaxterWW0 -
Language/country redirect best practice?
Hi, What is SEO best practice when it comes to redirecting users from www.domain.com to their specific language/country, let's say www.domain.com/de for Germany? From what I heard in on of the whiteboard fridays, it seems to be Javascript based on IP and browser language, and then set a cookie - correct? Or should we let our users manually select their language/country at the first visit? Any suggestion appreciated, thanks!
Technical SEO | | rtora0