Solved PayDay hack - but SERPs show URLs - what should I do?

[Ocularis]

We had the PayDay hack - and solved it completely. The problem is - the SERPs have over 3,000 URLs pointing to 404 on our website all of which have urls that are like this:

<cite>www.onssi.com/2012/2/post1639/payday-loan-companies-us</cite>‎

What should I do? Should I disavow every one of the 3,000? No Follow?

[Dr-Pete]

Oh, they'lre still indexed - got it. Yeah, that's a lot tougher. Ultimately, Google has to re-crawl these URLs, and since they're bad URLs and have no internal links and only spammy inbound links, that can take a while.

You can remove the URLs in Google Webmaster Tools, but that's a one-by-one process, so it's mostly for the worst culprits. Another option would be to make an XML sitemap with just these bad URLs. Encourage Google to recrawl them and process the 404s. The sitemap would also help tell you how many of the URLs were indexed and to track that number (more reliably than "site:" will). Unfortunately, you may have to make that list manually.

[Ocularis]

Thanks, Dr. Meyers!

So I implemented what Tom said a few weeks ago, and it still hasn't resolved:

https://www.google.com/search?num=100&site=&source=hp&q=site%3Aonssi.com+payday&oq=site%3Aonssi.com+payday&gs_l=hp.3...1466.4885.0.5093.21.21.0.0.0.0.196.1331.6j7.13.0....0...1c.1.26.hp..19.2.292.1LdlO8AoNoU

The page those go to does throw a 404, so I don't know when the listings should go away but it's pretty frustrating to see that they haven't yet.

Do you have any other suggestions on how to fix this?

[Dr-Pete]

I'm not sure you'll see a big difference here between the 404/410 (I've heard some mixed data recently), but definitely agree with Tom that, once Google honors either one, you've essentially cut the inbound link at that point. Making note of the link sources seems smart, but I'd also hesitate to disavow all these sites for now. Google is going to have to reprocess this and it may take a few days (or a couple of weeks) for the 404s to sink in. A link to a page that doesn't exist generally shouldn't harm you, though.

[Ocularis]

Thanks, this is really helpful!

How would I serve the 410 instead of 404? Should I use a regex match for payday and other words that wouldn't appear in regular onssi.com urls and then serve that based on the regex?

Also - speaking of useful tools, is there a tool for getting all 3,000 (that's only for payday - not even talking about other keywords) results without having to go ten by ten?

Thanks so much for the help!

[TomRayner]

Hi there Itamar

If I am reading this correctly, are the (now removed) URLs appearing on your own site?

If so, in order to tell Google that these pages are well and truly gone, I would serve a 410 response on those pages, rather than a 404.  This is a response code that tells Google the page has gone permanently, and so it will encourage the crawler not to try to revisit the URL in the future.

That means that any external links pointing to that page will become obsolete and shouldn't be counted in your link profile.  That's the theory, anyway.  For that reason, I would hold off on disavowing those links for the time being.  I'd make a note of it, but if everything goes OK with the 410 response, Google probably won't count those external links towards your site anyway.  Just in case, make sure you get all of the external links and save them in an excel sheet, so if you do need to disavow them in the future, you have them to hand.

I've double checked the onssi.com site in the safe browsing tool (see for yourself here) and it looks as though Google thinks the site is safe - ie it doesn't think it's hacked, which is great.  In any case, you may want to run the site through the malware review process just to be absolutely sure.

Hope this helps.