How to download full list of internal links properly with OSE?
-
Hi Moz Community,
I am having issues getting a full list of my internal links when I go to the Open Site Explorer. When I export the internal links into excel, I am getting half of my URLs and 80% of what I am getting is absolutely crazy URLs that are super super old.
Plus, these old URLs are not even showing in my campaign crawls. So the crawls themselves are different.
Any help on this would be greatly appreciated, since I am trying to get a full picture on the internal link structure of my site.
Thanks!
-
Agree, I love Screaming Frog!
-
Hi Ryan,
It would be best to use Screaming Frog if you want a true indication of your internal link structure.
https://www.screamingfrog.co.uk/seo-spider/
Cheers,
David
-
Hey there! Tawny from Moz's Help Team here. I think I can help explain why you're not seeing all the links you might expect, and why you're seeing some older links that you're not seeing in your main Campaign Site Crawl.
Open Site Explorer and the Link Analysis page of Moz Pro Campaigns are both tied to our Mozscape index, which tends to update roughly once a month.
Just a few points on how we compile our index:
-
We grab the most recent index.
-
We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains).
-
We start crawling from the top down until we've crawled ~130 billion URLs
The idea here is that we're focusing on the highest-quality links we can find, coming from the most prominent pages of authoritative sites. So, while you may not see every link for a site within our index, we're aiming to report the most valuable ones available!
Most new sites and links will be indexed by our spiders and available in Mozscape and Open Site Explorer within 60 days, but some take even longer for many reasons, including the crawl-ability of sites, the number of inbound links to them, and the depth of pages in subdirectories. This tends to bias our index in favor of newer links. Linking data is only stored in the index for about 180 days, so unless the crawler has a compelling reason to return to your site and the sites linking to yours and rediscover those links, they can fall back out of our index again. That doesn't mean they're not out there affecting your SEO, just that our tools don't see them anymore.
You can see our most recently updated schedule here as well as some more technical metrics on our Mozscape API Updates page. You can also see when the last and next updates happened on the Open Site Explorer (OSE) homepage at any time.
Since Moz focuses on quality of links over quantity, we are always focused on the most relevant links to display to our users. It's possible that Moz's index will leave out some of the lower-quality (non-link juice providing) links out of our index because of this. So, that might explain why you may see some discrepancies with what other tools may be showing.
You can read more about how we build our index in our guide here.
I know this is a ton of information, so if you have any questions or if I didn't make anything clear enough, please don't hesitate to ask! You can always drop us a line at help@moz.com and we'll do our best to clear up any questions you might have.
Cheers! -
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why moz is not indexing link from 21 march any body knows
I have lot of website few of them are really big sites. i want to know Why moz is not indexing link from 21 march any body knows please let me. is there some kind of maintenance going on
Link Explorer | | Arenapile3210 -
Link explorer section showing different linking domains numbers
Hello Moz, I recently activated my trial version, when I go into link explorer section. The linking domain option on Moz bar showing a different number. when i check the site into ahref, it's showing 14 linking domains, and when i check in Moz it's only showing me 4 linking domains. what is the problem? Which tool showing the accurate result. I am confused. please help.
Link Explorer | | NewDisplayAsi0 -
I'm doing a link audit and I want to download a spreadsheet of all the links to my website in OSE. There has to be a way to do that right?!
I'm doing a link audit and I want to download a spreadsheet of all the links to my website in OSE. There has to be a way to do that right?!
Link Explorer | | jennie.evans0 -
Moz showing +3M new inbound links. But nowhere to find them?
Hi there Moz is showing a +3M backlink rise in the dashboard for one of our domains. The site has always had around 350K backlinks and in Ahrefs and Majestic it still shows this number. But Moz shows a growth of +3M in the last two weeks. Is there a way to see where these backlinks come from in OSE? I can't seem to understand how it is possible to see this somehow. Can it be a mistake of Moz maybe?
Link Explorer | | snorkel1 -
How to interpret a possible spam link?
Hallo, Spam analysis is suggesting that my website may have a spam problem (score is 5). When I analyse the single spam links, I am puzzled. the links to my site are actually links in disqus comments I wrote myself on the linking site, in order to aswer REAL question of real people giving them honest and useful answers (and linking my site :-), like the link pointed out in the attached image, also pasted herebelow: http://www.androidiani.com/news/servizi-premium-indesiderati-lantitrust-contro-tim-vodafone-wind-e-3-213235 Most links have this same nature. Has this behaviour (which I consider natural, in good faith) been regarded as spammy? Should avoid linking my site in the comments I write on external sites? the links from comunicatistampagratis.it (towards the end of the list) are from a free press release website: you send them the press release, they publish it. Should I avoid using this free press release site? Should I avoid using any free press release site at all (since they are regarded as spammy)? is there a rule to learn (since apparently I have not been very good so far at using my good faith to separate good links from spammy links) whether a link can be spammy and therefore try and avoid it? thank you for you help.
Link Explorer | | micvitale0 -
Open Site Explorer - Just Discovered Inbound Links
Hello Moz Forum, Thanks in advance for any help. My website was recently featured in a bunch of high DA journals and magazines. I don't see the inbound links on the open site explorer, but I do see them in recently discovered. My website's Domain Authority hasn't changed a wink. How long does it take, typically to see the recently discovered links under the inbound links portion? Thanks! j1pqVRt
Link Explorer | | FutureSolutionsMedia0 -
Why I am not seeing any inbound link to my website?
While using the Open Site Explorer tool, I don't get any resultfor Inbound Links, while with other services and the Webmaster Tools I can see which websites are linking to mine. What I can do to see these results here on the MOZ analytics software? The website is http://b5footballcup.com Thanks for the support. Luca
Link Explorer | | lucab50 -
Why are none of the links we've received from press and other sources from the last two months showing on our Inbound Links in OSE???
It's really hard to judge if the DA of our site according to MOZ is 1) up to date and 2) accurate when for example there are SO many inbound links still not recorded, that were published months ago.
Link Explorer | | TVape0