Weird Behaviour

[LeeMiller]

Not sure if I've got this right, but I was just playing around looking at what of the top seo guys were up to and I found this site by someone called Scott D Smith. So I open up his site in site explorer and take a look at his link profile.

This guy has a bunch of .gov back links but whenever I try to see what the link is it just downloads a file and when I looked at the file it's an MS Access file.

Is this right, seems like something is amiss or is this some secret seo tactic I've not heard about?

lol

Lee

fyi - http://www.opensiteexplorer.org/links?site=www.scottdsmith.co.uk%2F

Oh and also the boing links they're displaying the same behaviour too.

[Nick_Ker]

I had to double check with the other link tracking things I use. I thought I had seen similar "weird" links elsewhere but that was not the case.

I apologize for accidentally floating a conspiracy theory - but it is a good one if I do say so myself.

[carinoverturf]

Haha! Sorry to dispel the mystery with boring reality! I did like your conspiracy theory though - much more exciting!!

Carin

[LeeMiller]

Carin,

thanks for clarifying the issue.

Although I'm still wondering why the links are there in the first place?

Lee

[carinoverturf]

Sure! Glad it was helpful!

Please post any questions you guys still have and I'll answer as best I can

Thanks!!

Carin

[ShaMenz]

Awesome Carin!

Thanks for responding so quickly with a great explanation of what's happening.

Sha

[carinoverturf]

Hey Lee,

Sha just gave me the heads up about this thread so I wanted to jump in and see if I can clarify what's going on with these downloadable links.

We made some improvements to the Linkscape crawler to make it fresher, crawl deeper and crawl more diverse domains - however, the deeper part ended up bringing to light a bug we had in the crawler. Once we started crawling deeply into websites, we started encountering more downloadable files which our crawler had no idea what to do with. They thought it was a link so they crawled it, but then when trying to associate it with a domain, it didn't know how to properly handle it and it ended up causing weird associations with domains previously crawled by the crawler.

We have been able to implement a few fixes, but, unfortunately, they take a bit of time to propagate through into the index - a full month to crawl and several weeks to process.

There were two solutions we found after investigating this problem. First, don't count binary files as a link - this has been done and should be part of our next index scheduled to launch 10/18. This should address about 70% of the issue. Second, update the crawler to disregard download files if it does encounter them. This update was just recently deployed to our crawlers and still needs about a month to propagate and go through processing. The affects of this fix probably won't be seen for another two index updates.

I hope this helps clear up some of the confusion going on here - most likely the weird "phantom" links you're seeing are a result of this bug we discovered in our updated crawler. If you're still seeing odd behavior after the next index update scheduled 10/18, please email the Customer Service team! We love the feedback as it makes our crawler be even better!

Thanks,

Carin

[LeeMiller]

Nicks and my though exactly Irving, Nick thinks its some kind of database injection and I tend to agree, the sites are using cms's, and several appear to running older versions of WP, so probably using a vulnerability of some kind, its common with php based sites.

[LeeMiller]

Already done Ryan

[irvingw]

but how did they get there in the first place, hacking sites I assume. The govt is not going to allow seo links/files on their site voluntarily.

[RyanKent]

I agree with Sha. These links are usually a form of Glitch with the Linkscape crawler which OSE uses to report information. Please follow Sha's suggestion and report this issue to help@seomoz.org. This will allow the team to not only resolve the issue you are seeing, but hopefully they can fix the root cause and improve the tool for everyone.

[Nick_Ker]

Frustrating indeed - seems like every time I read that something like comment spam/forums/directories is no longer useful, within days I will find a site that seems to be able to make effective use of it.

From what Sha says, OSE was showing too many of those wacky gov links and they did disappear in the next update for the site I mentioned. I did see a few in Yahoo SE and elsewhere, so I guess the files/links/whatever are really there but probably ignored by search engines.  I still wonder just how those got there. Hacking for links seems like a lot of trouble - especially for an otherwise legitimate site.

[ShaMenz]

Hi Lee,

As I understand it the problem is just a handling issue - the crawler is seeing downloadable files and recognizing them as regular links.

Sha

[LeeMiller]

Hi Sha,

are you saying these links aren't genuine, Mmmm is like a ghost result?

Lee

[ShaMenz]

Hi Lee,

There was a file handling issue with the changeover of SEOmoz crawlers month before last. I know that the SEOmoz Engineering team are aware of the problem and had thought that a fix was on the way.

This particular site seems to be one that has slipped through the net.

The best thing to do is to email the SEOmoz Help Team help [at] seomoz.org and give them as much detail about the particular OSE search you are looking up.

Hope that helps

Sha

[LeeMiller]

Yes I agree Nick, have to admire the guys balls if that the case as the site is a US gov website, (Department of Housing and urban Developement).

But you've got a good point with the database injection theory and have to wonder about the site, if they're using an outdated cms to run their website. All the sites that are exhibiting this behaviour have dynamic url's which to me says cms!

Is pretty annoying, when peeps like Rand say forums and blogs are not good for seo and here we see not only hundreds of blog and forum links, but also illegal means of gathering links.

pmsl had to laugh I just discovered an image file that was a backlink.

Thanks for your thoughts though, good to chat

[Nick_Ker]

I have see a lot of those too. Sometimes they are .mobi, PDF, or other odd file formats. I have not been able to track down exactly what they are, but I suspect they are part of some sort of shady automatic link building scheme. I had a client not too long ago who previously fell for a couple of SEO scams right before hiring me. A few weeks into it, I started seeing the .gov and .edu links like you describe. A week or two later the clients' rankings fell off the map completely. Not sure if it was from those, because there was also a lot of other bad SEO - too many exact match anchor text links from really bad directories and that sort of thing.

My theory is that some black hatters use some form of hacking (code injection?) to get these garbage files with links inside onto some high rank .gov sites. I have even seen some coming from NASA.gov.

I have also seen those gibberish odd file links in the profiles of some sites who rank high and you just cant figure out why. So whatever it is, it seems to work short term at least.

Another strange thing I noticed was that many of them had page titles that were extremely spammy - casinos, viagra, etc.