Website blog is hacked. Whats the best practice to remove bad urls
-
Hello
So our site was hacked which created a few thousand spam URLs on our domain. We fixed the issue and changed all the spam urls now return 404. Google index shows a couple of thousand bad URLs.
My question is-
What's the fastest way to remove the URLs from google index. I created a site map with sof the bad urls and submitted to Google. I am hoping google will index them as they are in the sitemap and remove from the index, as they return 404.
Any tools to get a full list of google index? ( search console downloads are limited to 1000 urls). A Moz site crawl gives larger list which includes URLs not in Google index too. Looking for a tool that can download results from a site: search.
Any way to remove the URLs from the index in bulk? Removing them one by one will take forever.
Any help or insight would be very appreciated.
-
Technically 404 means "temporarily unavailable but coming back later" so you might want to consider Status 410 instead of 404. You could also supplement it with Meta no-index, if you can't use the HTML implementation then fire the no-index directive through the HTTP header using X-robots:
https://developers.google.com/search/reference/robots_meta_tag (scroll down a little to find the relevant part)
E.g:
"HTTP/1.1 200 OK
Date: Tue, 25 May 2010 21:42:43 GMT
(…)
X-Robots-Tag: noindex
(…)"... something like that.
You can't use Search Console to remove URLs from Google at all. The remove URL tool, only removes URLs one at a time and it only does so 'temporarily', the URLs pop back again after a bit. The best thing you can do is give Google some harsher directives and hope they listen, in a month or two most of those should be gone
Don't use robots.txt on the URLs as, if Google can't crawl them it won't find the 410s or the no-index directives
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What's best practice for cart pages?
i don't mean e-commerce site in general, but the actual cart page itself. What's best practice for the links that customers click to add products to the cart, and the cart page itself? Also, I use vanity URLs for my cart links which redirect to the actual cart page with the parameters applied. Should I use use 301 or 302 redirects for the links? Do I make the cart page's canonical tag point back to the store home page so that I'm not accruing link juice to a page that customers don't actually want to land on from search? I'm kinda surprised at the dearth of information out there on this, or maybe I'm not looking in the right places?
Technical SEO | | VM-Oz0 -
Redirect our www.website.com to website. Com/target-keyword
Dear moz community I have been analyzing the websites that rank in top 20 for our target keywords. All of the top 20 sites except us have their websites re directing to websites.com/target keyword. This is due to probably because they have multiple city's and one of the target keywords term is cityname + word. My question is and idea - make a 301 to our www.website.com to /city-keyword and start linking to that page with new links. Would that bring any benefit? Seems that it's a very strong ranking signal. Any threats that I must take into account? We currently rank as #9
Technical SEO | | advertisingcloud0 -
What are best options for website built with navigation drop-down menus in JavaScript, to get those menus indexed by Google?
This concerns f5.com, a large website with navigation menus that drop down when hovered over. The sub nav items (example: “DDoS Protection”) are not cached by Google and therefore do not distribute internal links properly to help those sub-pages rank well. Best option naturally is to change the nav menus from JS to CSS but barring that, is there another option? Will Schema SiteNavigationElement work as an alternate?
Technical SEO | | CarlLarson0 -
404 Best Practices
Hello All, So about 2 months ago, there was a massive spike in the number of crawl errors on my site according to Google Webmaster tools. I handled this by sending my webmaster a list of the broken pages with working pages that they should 301 redirect to. Admittedly, when I looked back a couple weeks later, the number had gone down only slightly, so I sent another list to him (I didn't realize that you could 'Mark as fixed' in webmaster tools) So when I sent him more, he 301 redirected them again (with many duplicates) as he was told without really digging any deeper. Today, when I talked about more re-directs, he suggested that 404's do have a place, that if they are actually pages that don't exist anymore, then a ton of 301 re-directs may not be the answer. So my two questions are: 1. Should I continue to relentlessly try to get rid of all 404's on my site, and if so, do I have to be careful not to be lazy and just send most of them to the homepage. 2. Are there any tools or really effective ways to remove duplicate 301 redirect records on my .htaccess (because the size of it at this point could very well be slowing down my site). Any help would be appreciated, thanks
Technical SEO | | CleanEdisonInc0 -
Should I use canonicals? Best practice?
Hi there, I've been working on a pretty dated site. The product pages have tabs that separate the product information, e.g., a tab for specifications, a tab for system essentials, an overview tab that is actually just a copy of the product page. Each tab is actually a link to a completely separate page, so product/main-page is split into product/main-page/specs, product/main-page/resources, etc. Wondering if canonicals would be appropriate in this situation? The information isn't necessarily duplicate (except for the overview tabs) but with each tab as a separate page, I would imagine that's diluting the value of the main page? The information all belongs to the main page, shouldn't it be saying "I'm a version of the main page"?
Technical SEO | | anneoaks0 -
How do I remove Links to my website???
Hi Guys, Please can anyone help!! Can anyone tell me how on earth I can remove links to my website? My website has been hit by the new penguin update and the company that was doing my SEO seems to have built a lot of spammy links!! How can I remove these links??? Please can anyone help Thanks Gareth
Technical SEO | | GAZ090 -
Give your top 3 of best optimized websites
Hey gents & ladies, Give your top 3 of websites that in your eyes are optimized in a good way? Tell me why you think the website is that good and notice the keywords.
Technical SEO | | PlusPort0 -
Promoting a blog or a blog article
Hi what is the best way to promote a blog or a blog article. What i want to do is to find a site when i can put part of the article on that site and then have a link going to my blog for the article. Can anyone recommend any sites that do this please or the best ways to promote a new article from a blog
Technical SEO | | ClaireH-1848860