Specific external links
-
So...we have an A+ rating with the BBB and have for some time. We have a link from their page. We also use sites like pricegrabber and reseller rankings and have links from those.
What I can't figure out is why links from the BBB and Pricegrabber don't even show up in open site explorer but the reseller ratings links are everywhere. The BBB one is easily the oldest and the other two are roughly the same age (i.e. at least a couple years old.)
-
The Open Site Explorer indexed is completely refreshed every few weeks, and currently consists of 105 billion links. While that's quite a large data set and mostly comparable with other indexes in the space, it's still only a fraction of the links on the web and only a portion of the links Google undoubtedly knows about.
The way OSE finds links is that it prioritizes pages with lots of inbound links and links it found during previous crawls. Because BBB pages are often found deep-deep-deep under layers of navigation and usually don't have a lot of external links pointing to them, they aren't easy pages for crawlers to find on a regular basis. BBB has an especially difficult site architecture, and sadly this issue comes up a lot with that particular site
We're working on including more historical data in the index, so that even if we didn't crawl the link in the past two weeks, we'll still include it if we are reasonably confident it still exist. This will help alleviate some of these concerns.
The good news (or bad depending on how you look at it) is that while OSE doesn't show every link, it tends to show the majority of the most valuable links.
Regardless, thanks for the feedback and best of luck with your SEO.
-
I share your frustration in wanting to be able to track links in OSE. Someone at Moz told me a while back that OSE only reaches 20-25% of links out there. I've just come to tell myself that if I know the link exists then I don't need to see it in OSE.
-
I wonder if Moz understands and follows the Robots.txt file? I know at one point they never, back in the day when I used Moz most of my errors came from URL's which were hidden by robots which would not of caused errors for Google.
& what your saying Oren makes perfect sense if it's correct. Why would they want to crawl aggregated content.
-
Perhaps a Mozzer can verify this, but I remember someone saying they Moz doesn't crawl some big sites that aggregate lots of data (twitter, BBB, yelp, domain listings, etc. etc.) because of the size of their servers, and the likelihood that the user put it there themselves. They have smaller servers than say, Google or Bing.
-
Hi,
Firstly it could be that Open Site Explorer has not had time to pick them up, if it's years old then let's not forget the OSE manually crawls the internet and is not Google's crawler meaning there will be sections of the web which won't be crawled by Moz.
If it's old then there may be no links pointing to these pages which would not give OSE a reason to crawl them, certainly if it's years. Don't worry about it, a missing link on OSE is one less your competitors can find I had a similar issue with a few links from BOTW but in time it updated and works fine but Open Site Explorer is not flawless.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate content issues with file download links (diff. versions of a downloadable application)
I'm a little unsure how canonicalisation works with this case. 🙂 We have very regular updates to the application which is available as a download on our site. Obviously, with every update the version number of the file being downloaded changes; and along with it, the URL parameter included when people click the 'Download' button on our site. e.g. mysite.com/download/download.php?f=myapp.1.0.1.exe mysite.com/download/download.php?f=myapp.1.0.2.exe mysite.com/download/download.php?f=myapp.1.0.3.exe, etc In the Moz Site Crawl report all of these links are registering as Duplicate Content. There's no content per se on these pages, all they do is trigger a download of the specified file from our servers. Two questions: Are these links actually hurting our ranking/authority/etc? Would adding a canonical tag to the head of mysite.com/download/download.php solve the crawl issues? Would this catch all of the download.php URLs? i.e. Thanks! Jon
Moz Pro | | jonmc
(not super up on php, btw. So if I'm saying something completely bogus here...be kind 😉 )0 -
Why are my month old links not showing in WMT or OSE?
Hey Everyone! I've been working hard on boosting my ranking after launching my new site 2 months ago. I realize that this process normally takes some time however I've come across a questions that I'm not sure I know that answer to... I had some really solid back links added about a month ago but nothing is showing up yet in WMT on them. When I run an OSE I don't see any links there (not even under newly discovered). Can someone please help with some insight on why they might now be showing. Is it just a time thing or should I be doing something different?Any insight will be very helpful. Thanks
Moz Pro | | MDesigns0 -
Why is Link Count smaller than Internal Links in Crawl Test report?
We recently ran the crawl test report and for most of our pages we are getting 1150 internal links but 40-50 as the link count. Why is there such a big disparity?
Moz Pro | | usdmseo0 -
Why is my internal followed links is 0?
My internal followed links is 0. Also for top pages the report say "block by robots.txt" but the Moz bot is able to crawl my site and generate report for 10K pages. Please help me understand why. http://www.opensiteexplorer.org/comparisons?site=www.findyogi.com
Moz Pro | | namansr0 -
How do you track new inbound links?
Hello Fellow SEOs, I've been charged with managing and tracking my company's linkbuilding efforts. I'd love to hear from others experienced in this area: -is there a way in OSE to sort links by date? -if this isn't possible, how can you tell what new links come in every month? Is this a manual process of downloading and comparing inbound links every week? This seems pretty tedious and I would love to hear what you all do and if there is a better way. I do have a Google Alert out for my company name, but that doesn't necessarily give me what I need. Thank you!
Moz Pro | | Jenny8675309
Jenny0 -
Link reporting.
Is there a way in the Pro reporting where I can see a summary of the number of incoming links by type (blogs / news / wiki / dir / forums etc)? Even better, could the report give me an average Page Rank for each link type? Thanks,
Moz Pro | | CarlDarby0 -
Sudden, drastic drop in # of links and root domains on website?
Noticed that my links and root domain numbers had dropped by ~40%, seemingly overnight. Any idea what could have contributed to this (Toolbar update???)?
Moz Pro | | wlefevre0 -
Individual Link Value
We understand the PA, DA, trust and all of that. My question is, is there a process or formula anyone uses that shows an individual links value as to the link juice it passes. The old Domain Juice seemed to be that, but after further investigation (And Rand setting me straight) I now understand it's not a good metric. Today, we use PA divided by the number of external links on that page to get some sense of an individual links actual value to the site or page we link to. I understand this is a very sloppy system, but seems to be the only choice we have? It's based on this simple thought. If you get a back link on two different pages, and both are equal in every way, except one has 3 outbound links and the other has 30, the link from the page with 3 will be significantly stronger as far as passing juice. So... anyone using something to determine an individual links value? I did ask the SEO staff, and they do not current have it.
Moz Pro | | MBayes0