How to download full list of internal links properly with OSE?
-
Hi Moz Community,
I am having issues getting a full list of my internal links when I go to the Open Site Explorer. When I export the internal links into excel, I am getting half of my URLs and 80% of what I am getting is absolutely crazy URLs that are super super old.
Plus, these old URLs are not even showing in my campaign crawls. So the crawls themselves are different.
Any help on this would be greatly appreciated, since I am trying to get a full picture on the internal link structure of my site.
Thanks!
-
Agree, I love Screaming Frog!
-
Hi Ryan,
It would be best to use Screaming Frog if you want a true indication of your internal link structure.
https://www.screamingfrog.co.uk/seo-spider/
Cheers,
David
-
Hey there! Tawny from Moz's Help Team here. I think I can help explain why you're not seeing all the links you might expect, and why you're seeing some older links that you're not seeing in your main Campaign Site Crawl.
Open Site Explorer and the Link Analysis page of Moz Pro Campaigns are both tied to our Mozscape index, which tends to update roughly once a month.
Just a few points on how we compile our index:
-
We grab the most recent index.
-
We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains).
-
We start crawling from the top down until we've crawled ~130 billion URLs
The idea here is that we're focusing on the highest-quality links we can find, coming from the most prominent pages of authoritative sites. So, while you may not see every link for a site within our index, we're aiming to report the most valuable ones available!
Most new sites and links will be indexed by our spiders and available in Mozscape and Open Site Explorer within 60 days, but some take even longer for many reasons, including the crawl-ability of sites, the number of inbound links to them, and the depth of pages in subdirectories. This tends to bias our index in favor of newer links. Linking data is only stored in the index for about 180 days, so unless the crawler has a compelling reason to return to your site and the sites linking to yours and rediscover those links, they can fall back out of our index again. That doesn't mean they're not out there affecting your SEO, just that our tools don't see them anymore.
You can see our most recently updated schedule here as well as some more technical metrics on our Mozscape API Updates page. You can also see when the last and next updates happened on the Open Site Explorer (OSE) homepage at any time.
Since Moz focuses on quality of links over quantity, we are always focused on the most relevant links to display to our users. It's possible that Moz's index will leave out some of the lower-quality (non-link juice providing) links out of our index because of this. So, that might explain why you may see some discrepancies with what other tools may be showing.
You can read more about how we build our index in our guide here.
I know this is a ton of information, so if you have any questions or if I didn't make anything clear enough, please don't hesitate to ask! You can always drop us a line at help@moz.com and we'll do our best to clear up any questions you might have.
Cheers! -
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved learning about links...
Hello! I'm wanting to gain some insight about backlinks, and wondering why a client's digital ads don't count towards our external link count. We don't see the domain listed in the Linking Domains and were just curious. Any information would be appreciated as we're always trying to learn! Thanks!
Link Explorer | | Ebbeka0 -
Are Unlinked Mentions Coming Back in the New OSE?
Hi Mozzers, Just a quickie, with the unlinked mentions being retired, I was wondering if Moz have any plans to bring this feature back through the new link explorer? Many thanks! 🙂
Link Explorer | | Corbec8881 -
Inbound links not found in Moz Pro(though they exist)
These sites have many inbound links but Moz Pro finds no such links. I also used SimilarWebPro Trial to confirm the existing of such links. (these sites have IP block so it requires an VPN to access with Japan IP) osakado.cc
Link Explorer | | Yuji_m
oskd.biz Could anyone figure out why Moz can't find those links to those sites? Any tricks in place to hide those links from SeoMoz? If I pay to upgrade SimilarWebPro but Id rather stay with SeoMoz Pro. thanks in advance.0 -
Help needed interpreting inbound link analysis please
Hi I am trying to determine how effective our store blog is and am trying to analyse the inbound links it provides to our store domain but am confused with the results, can anyone help please? For information the blog is on a subdomain of the store site, has plenty of relevant original content, and has been regularly updated over several years in line with basic Yoast SEO best practice. When I look at analysis of inbound links and linking domains for the store site in Moz Pro, the blog subdomain does not show as providing any inbound links. Google Search Console shows similar results, with no inbound links to the store domain showing from the blog subdomain in either 'search traffic > links to your site', or 'search traffic > inbound links'. Open Site Explorer for the store domain also shows no inbound links from the blog subdomain to the store with 'link source > only external' selected. But if I change to 'link source > only internal' then all of the inbound links from the blog subdomain to the main store that I would expect to see appear. Should I be concerned by the fact I can only find these inbound links under this one specific search and not in any of the others? Does this suggest a problem in the passing of any current and future SEO benefit from subdomain blog to main domain? Or should I not worry about it too much? I am looking to begin a full overhaul of all SEO for the store and blog from the ground up and want to be sure the connections between the blog subdomain and store are functioning properly before I start. Any help in unpicking this would be really appreciated Thanks very much
Link Explorer | | Warren_330 -
How to interpret a possible spam link?
Hallo, Spam analysis is suggesting that my website may have a spam problem (score is 5). When I analyse the single spam links, I am puzzled. the links to my site are actually links in disqus comments I wrote myself on the linking site, in order to aswer REAL question of real people giving them honest and useful answers (and linking my site :-), like the link pointed out in the attached image, also pasted herebelow: http://www.androidiani.com/news/servizi-premium-indesiderati-lantitrust-contro-tim-vodafone-wind-e-3-213235 Most links have this same nature. Has this behaviour (which I consider natural, in good faith) been regarded as spammy? Should avoid linking my site in the comments I write on external sites? the links from comunicatistampagratis.it (towards the end of the list) are from a free press release website: you send them the press release, they publish it. Should I avoid using this free press release site? Should I avoid using any free press release site at all (since they are regarded as spammy)? is there a rule to learn (since apparently I have not been very good so far at using my good faith to separate good links from spammy links) whether a link can be spammy and therefore try and avoid it? thank you for you help.
Link Explorer | | micvitale0 -
How to get backlinks from Https websites with OSE?
Hi guys, Any ideas how can we get/check the backlinks from securized websites (https) using Open Site Explorer? It seems we cannot change the protocol (http:// by default). Thanks,
Link Explorer | | Netsociety
Ben1 -
Unable to download CSV Report
Hi, I have the email notifying me that my report is ready to download but the link simply opens a blank webpage. If I go to 'Recent CSV Reports' I can see my report listed but if I click the download button nothing happens. Can anyone suggest a solution? Thanks Hugh
Link Explorer | | HLConsulting0