Tons of Crappy links in new OSE (Open Site Explorer)
-
I am starting to miss the old OSE. I've found that for a lot of the pages on our site, the new OSE is showing WAY more links and most of them are garbage nonsense links from China, Russia, and the rest of the internet Wild West.
For instance, in the old OSE, this page used to show 9 linking domains:
http://www.uncommongoods.com/gifts/by-recipient/gifts-for-him
It now shows 454 links. Some of the new links (about 5 of them) are legitimate. The other 400+ are garbage. Some are porn sites, most of them don't even open a web page, they just initiate some shady download. I've seen this for other sites as well (like Urban Outfitters) This is making it much harder for me to do backlink analysis on bc I have no clue how many "Normal" links they have. Is anyone else having this problem ? Any way to filter all this crap out ? See attached screenshot of the list of links I'm getting from OSE.
-
Ok thank you. I will email directly.
-
Hey Zack,
Sorry to hear you're still having problems - we've seen an improvement on most sites at this point. Would you want to send me info on the site you're searching and any filters you are using?
If you don't feel comfortable posting that info on this thread, feel free to email me directly: carin@seomoz.org.
Thanks!
Carin
-
Hey Carin,
I just wanted to follow up on this...I'm still seeing these spammy binary files show up as links. Unfortunately it makes OSE quite useless for me in regards to exploring our own backlinks.
What is the status of this problem? Has there been any headway ? Why does our site have problems but most others don't?
Thanks!
-Zack
-
Hey Zack,
Thanks so much for understanding! We are doing everything we can to get the bug resolved. Binary files are the downloadable files you see as links - .pdf, .exe, .img, etc.
I'm really sorry, but we don't have a URL to the old OSE. I saw Steven's response as a workaround - is that possible or are there too many file types to filter out?
Our crawlers that provide the metrics to OSE are always crawling, but will take about a month for our fix to propagate through to all the pages we crawl. Once we have removed these links from our crawlers, then we'll have to process the metrics. This is why it's looking like late September for the fix to show up.
I really appreciate your patience and understanding, we're doing everything we can to fix it!!
Thanks,
Carin
-
Hey Carin-
Thank you so much for this in-depth response. Glad to hear that you guys are aware of it and trying to sort it out. Very interesting info...I'd never hear of "binary" links before but I hope you guys can figure out how to handle these. Seems like a tough task to tackle, just by looking at my CSV it looks like these come in several different forms and they could be hard to identify..I have a few questions:
1. Is there by chance a URL you could give me that points to the old OSE ?
2. How often does OSE crawl? Is it a constant process or are there scheduled crawls?
Thanks!!
-Zack
-
Hey Zack, I saw the ticket you filed was answered by Aaron, but I just wanted to follow up with you as well. We have made some really exciting changes to the crawler, but, unfortunately, there is a pretty obvious bug as well...
The reason for the “questionable” links coming from the Internet Wild West is due to the crawler reaching much deeper into sites where there are more download (i.e. binary) links. The first issue is the crawler is counting a binary file as a link, but the larger issue, is that the crawler doesn’t really know how to handle these types of files. This bug is causing some links to be improperly associated with certain domains. This is probably what you're seeing with all the crazy links from China and Russia which don't actually link to the site you're researching.
There are two steps to addressing this issue: changing how the crawler sees these file types and then fixing how the crawler handles these file types. We have made improvements to our algorithm so that we will be handle the majority of these files correctly, however, this update will need about a month to propagate. The fix for this issue probably won’t be seen for two more updates, meaning late September. Our improvements should catch most of the issues, but there still could be a few cases we haven't addressed. If this happens, don't hesitate to let us know; we love feedback since it helps us improve and make our index even better!
The next step is to fix how our crawlers handle binary file links and prevent them from being improperly associated with certain domains. We are in the process of working through that issue right now. We’re doing everything we can to resolve this bug as we know it is alarming to see these “questionable” links associated with your sites.I hope this helps and thanks so much for being patient :)Thanks,Carin
-
2 ways:
- Get as CSV and spend the time going through it
- Wait it out
-
OK cool good info, hope they fix it soon!! Any good ideas on how you can filter this crap[ out ?
-
Hello Zack,
That is an issue that they are working on, I know this because I already discussed this with one of their help desk people. Here is the page that describes the changes: http://www.seomoz.org/blog/brand-new-open-site-explorer-is-here
In addition to that, here is some additional information I can share with you:
you may see “questionable” links with weird file extensions. This is due to the crawler reaching much deeper into sites where there are more download links. We are looking into fixing this bug as soon as we can so these won’t be counted as links.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Tumblr and Link Equity
Hi Moz Community, I've recently decided to start a project where I gather 1,000 great examples of something that's searched for often, and am thinking that posting it to a Tumblr site like the following website did could be a great way to pass link equity back to my main site (with a little "site by [my site]" somewhere in the header or footer). While I was super pumped about this idea today, and have now gathered almost 500 of my examples (mentioned above), I am not seeing link equity passed from this site, even on the non-redirected links here: http://gothamlogos.tumblr.com/ Anyone have any experience with projects like this? I've checked read the Moz Tumblr and SEO article from a few years ago, which makes it seem like this should be an SEO "win"... But using the Moz Pro account tools, I'm not seeing any of these non-redirect links (ordinary links) giving any value to anyone on this example site. Thanks so much in advance, Zack
Moz Pro | | Zack2 -
How do I see all links to my site in Web Explore?
It appears that the Web Explore looks for exact matches to a page. I'd like to see all matches to any page. Will regular expressions work?
Moz Pro | | Leverage_Marketing0 -
Open Site Explorer - Link Export Not Showing Same Number of Links
Noob question. I'm using Open Site Explorer to hunt down some external links for a website. In the attached image (post-pressing the filter button), it says I have 889 external links, but after I've downloaded the CSV, I only get 695 links. Why the difference in number? And if I'm doing something wrong, how do I make sure I'm getting the full amount of external links to my client's website? Thanks! GG6wURf.jpg
Moz Pro | | EEE31 -
External Followed Links History, number of links go down
I was reviewing Historical Domain Analysis and found that in last 2 month we lost almost 10000 external followed links. What this could be? is this real or just question seomoz crawling? 30voy1g.jpg
Moz Pro | | ctam0 -
Open Site Explorer - SEO - Company Back Links not visible?
Hi Guys, I am relatively new to the SEO community and have what is hopefully a pretty quick and simple question? I have recently outsourced some of my SEO campaigns to an Australian SEO group which were referred to me buy a friend, and i do not have the time to manage all sites SEO. On pitch the SEO company said they had in excess of 7000 domains, and they would implement a massive back linking strategy for me anywhere upwards of 200 links a month and all legit. Initially there were some basic header and title tag changes needed on the site, and I am now in month 4 of my campaign. Looking forward to using SEOMOZ service and specifically Open Site Explorer, I entered my URL but to my disappointment I could only see 4 links and two I was responsible for. I spoke to the SEO company who responded Open Site Explorer wasn't a good indication of back links and that a lot of their sites were not on the network because of the structure of their linking being only one way. I would appreciate a second opinion (external of this company) on this because of my short time learning and dabbling with SEO. On a side note thoroughly enjoying learning SEO and my journey as part of the SEOMOZ community. Appreciate any feedback or responses I get. Kind regards Bodie http://www.berkeleyriver.com.au
Moz Pro | | Bodie0 -
Domain authority decrease after open site explorer update. Reasons?
Hi! I just noticed a decrease on our domain authority and also the page authority of several of our pages on the SeoMoz toolbar. For instance, DA went from 34 to 29. I was scared and confused, so I went to check it on the opensite explorer and saw that it had been updated. I want to ask you guys if the same happened with your metrics or is it just us. Because the strange thing about it is that we have been doing some good optimization work lately and also some linkbuilding, and actually our SERP’s have been improving in general, and some specific ones have been performing great. Our website is http://www.inmonova.com as a reference. Can anyone cast some light into this?
Moz Pro | | inmonova0 -
Why doesn't the BBB / Trustlink.org links show up in the Link Analysis?
I am curious why one of my client's main competitors (www.allbayhardwood.com) shows links from the Better Business Bureau and Trustlink.org (associated with BBB) but links from those sources do not show up for his domain (www.sanjosehardwoodfloors.com). He has been a BBB Acredited Business since 12/2010 and on file with them for probably as long as they have had the online version, which seems like plenty of time for the link to have been picked up. BBB has a very nice domain authority and it would be great to see these links show up. (they don't show up in webmaster tools either) Is there something I am missing? Thanks in advance guys and gals! (I know the site has other SEO issues - just getting started on pounding everything out.)
Moz Pro | | SnoBaer0 -
When do I see the new Linkscape?
Hi I have seen a Tweet and item on the latest news saying that the Linkscape has been updated. I'm still seeing a report generated at the start of May though. What do I need to do to get my grubby hands on the latest data? Chris
Moz Pro | | P4D0