How to download full list of internal links properly with OSE?
-
Hi Moz Community,
I am having issues getting a full list of my internal links when I go to the Open Site Explorer. When I export the internal links into excel, I am getting half of my URLs and 80% of what I am getting is absolutely crazy URLs that are super super old.
Plus, these old URLs are not even showing in my campaign crawls. So the crawls themselves are different.
Any help on this would be greatly appreciated, since I am trying to get a full picture on the internal link structure of my site.
Thanks!
-
Agree, I love Screaming Frog!
-
Hi Ryan,
It would be best to use Screaming Frog if you want a true indication of your internal link structure.
https://www.screamingfrog.co.uk/seo-spider/
Cheers,
David
-
Hey there! Tawny from Moz's Help Team here. I think I can help explain why you're not seeing all the links you might expect, and why you're seeing some older links that you're not seeing in your main Campaign Site Crawl.
Open Site Explorer and the Link Analysis page of Moz Pro Campaigns are both tied to our Mozscape index, which tends to update roughly once a month.
Just a few points on how we compile our index:
-
We grab the most recent index.
-
We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains).
-
We start crawling from the top down until we've crawled ~130 billion URLs
The idea here is that we're focusing on the highest-quality links we can find, coming from the most prominent pages of authoritative sites. So, while you may not see every link for a site within our index, we're aiming to report the most valuable ones available!
Most new sites and links will be indexed by our spiders and available in Mozscape and Open Site Explorer within 60 days, but some take even longer for many reasons, including the crawl-ability of sites, the number of inbound links to them, and the depth of pages in subdirectories. This tends to bias our index in favor of newer links. Linking data is only stored in the index for about 180 days, so unless the crawler has a compelling reason to return to your site and the sites linking to yours and rediscover those links, they can fall back out of our index again. That doesn't mean they're not out there affecting your SEO, just that our tools don't see them anymore.
You can see our most recently updated schedule here as well as some more technical metrics on our Mozscape API Updates page. You can also see when the last and next updates happened on the Open Site Explorer (OSE) homepage at any time.
Since Moz focuses on quality of links over quantity, we are always focused on the most relevant links to display to our users. It's possible that Moz's index will leave out some of the lower-quality (non-link juice providing) links out of our index because of this. So, that might explain why you may see some discrepancies with what other tools may be showing.
You can read more about how we build our index in our guide here.
I know this is a ton of information, so if you have any questions or if I didn't make anything clear enough, please don't hesitate to ask! You can always drop us a line at help@moz.com and we'll do our best to clear up any questions you might have.
Cheers! -
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Moz link explorer is how much reliable and accurate as compared to others link explorer tools like Ahref
Hi.. I need to check backlinks for one of my client website duplicatebills through Moz Pro link explorer tool then its show that linking domain to this website are 90 but when i cross check these links from other tools like Ahref free backlinking tool then its show a different figure (ahref show that linking domain to this website are 103).
Link Explorer | | dbills
I am confused that which one correct. And MOZ link explorer tools in reliable to which extent?0 -
Discrepancies between google search console and OSE
Hi google search console is showing root linking domains at 297 but OSE only shows 19 This was highlighted because OSE dropped from 50 to 19 since last scan Regards Paul
Link Explorer | | phes0 -
Is OSE having problems?
The healthpage says it is operational but I'm getting a ton of 'There was an error getting your data' messages. First I thought, it just happens to smaller pages, since testing it with moz.com worked. But even pages like nbc.com give me this error. I tried logging out an in again and even swithcing browsers - without success. Anyone else having problems like this right now? Cheers, Chris
Link Explorer | | chrwald0 -
Drop in Site Links
Hi all, I've noticed recently a dramatic drop in the number of external links showing in moz across all campaigns (including tracked competitors) Is this to do with the new indexing? Thanks
Link Explorer | | wearehappymedia1 -
Quality/Established blog showing DA1 PA1 on OSE
Hello there, We have come across what looks like a high quality blog in our niche (interior design). The blog has been established for a while and is well designed. The content is of a good quality The blog URL: http://www.apartmentnumber4.com The blog has a good following across its social media accounts. However, Open Site Explorer is showing DA1 and PA1 Does anyone know why that is?
Link Explorer | | roberthseo0 -
How can Ahrefs be better than OSE ?
Dear Sirs and Contributors, I know this question is going to cause revolution, but I am paying for a subscription to moz every single month and I see that Open Site Explorer doesn't find baacklinks for some of my new clients when aHrefs can find more than 16 !! Why is this ? How can a free tool like ahrefs give me more information than OSE for which I pay for ? ;-(
Link Explorer | | Tintanus1 -
Facebook likes in OSE when there is no Facebook Like button on the web page
In OSE how are page specific facebook likes calculated when the page does not have a Facebook like/share button on it? I have seen many pages with multiple Facebook likes counted against it in OSE but cannot see how people could be liking the page.
Link Explorer | | TheHutGroup0