How to download full list of internal links properly with OSE?
-
Hi Moz Community,
I am having issues getting a full list of my internal links when I go to the Open Site Explorer. When I export the internal links into excel, I am getting half of my URLs and 80% of what I am getting is absolutely crazy URLs that are super super old.
Plus, these old URLs are not even showing in my campaign crawls. So the crawls themselves are different.
Any help on this would be greatly appreciated, since I am trying to get a full picture on the internal link structure of my site.
Thanks!
-
Agree, I love Screaming Frog!
-
Hi Ryan,
It would be best to use Screaming Frog if you want a true indication of your internal link structure.
https://www.screamingfrog.co.uk/seo-spider/
Cheers,
David
-
Hey there! Tawny from Moz's Help Team here. I think I can help explain why you're not seeing all the links you might expect, and why you're seeing some older links that you're not seeing in your main Campaign Site Crawl.
Open Site Explorer and the Link Analysis page of Moz Pro Campaigns are both tied to our Mozscape index, which tends to update roughly once a month.
Just a few points on how we compile our index:
-
We grab the most recent index.
-
We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains).
-
We start crawling from the top down until we've crawled ~130 billion URLs
The idea here is that we're focusing on the highest-quality links we can find, coming from the most prominent pages of authoritative sites. So, while you may not see every link for a site within our index, we're aiming to report the most valuable ones available!
Most new sites and links will be indexed by our spiders and available in Mozscape and Open Site Explorer within 60 days, but some take even longer for many reasons, including the crawl-ability of sites, the number of inbound links to them, and the depth of pages in subdirectories. This tends to bias our index in favor of newer links. Linking data is only stored in the index for about 180 days, so unless the crawler has a compelling reason to return to your site and the sites linking to yours and rediscover those links, they can fall back out of our index again. That doesn't mean they're not out there affecting your SEO, just that our tools don't see them anymore.
You can see our most recently updated schedule here as well as some more technical metrics on our Mozscape API Updates page. You can also see when the last and next updates happened on the Open Site Explorer (OSE) homepage at any time.
Since Moz focuses on quality of links over quantity, we are always focused on the most relevant links to display to our users. It's possible that Moz's index will leave out some of the lower-quality (non-link juice providing) links out of our index because of this. So, that might explain why you may see some discrepancies with what other tools may be showing.
You can read more about how we build our index in our guide here.
I know this is a ton of information, so if you have any questions or if I didn't make anything clear enough, please don't hesitate to ask! You can always drop us a line at help@moz.com and we'll do our best to clear up any questions you might have.
Cheers! -
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
My backlinks are not indexing , Added in the link tracking list too
I have waited for three days to index my backlinks on moz but still they are not indexed in moz backlink index 😞 How to index backlinks in moz so that I can track all are ok ?
Link Explorer | | esdda81111 -
Moz Pro: Filter inbound links by partial anchor text?
My site has been targeted by a spam farm with hundreds of different domains, all linking to images on our CDN with similar variations of anchor text, eg: get free high quality hd wallpapers wedding cake makers
Link Explorer | | James_NZ
get free high quality hd wallpapers hairstyle makeover
get free high quality hd wallpapers living room cafe
etc Is it possible within Moz Pro to filter all incoming links with anchor text including "free high quality hd wallpapers" so that I can disavow all of the domains en masse? So far I've only been able to display/download the list of links exactly matching the full anchor text which is very time-consuming with 100+ variations. Regards,
James0 -
Most of the backlinks not listed
Hi, I'm running a website since one year and a half, and I'm periodically watching with Moz OSE tools the number of backlinks it has. It still shows only 11 backlinks from 8 domains, even if my actual backlinks are much more and come from about 40 domains. I've already searched the forum and read that Moz scraper updates once per month, but all these backlinks are like 6 months old, and still no trace. Many other competitor tools, including the simple google search console can see all the backlinks, only Moz can't retrieve them. Do you have any idea? Somebody else in the same situation? Or is there a way to submit to Moz links to pages containing that backlinks?
Link Explorer | | fabio_m
Thanks
Regards0 -
Where can I see the list of the new backlinks reported?
In my dashboard I see a +875 in external links. Now they are 1,130. This sounds worrying rother than good news! I'd like to get an explaination but how can I see where these links come from? And when? How can be explained a steep increase in the link count from the last index? Also, a minor question, where can I find the increase percentage? I am able to see only the absolute count.
Link Explorer | | 2mlab0 -
Do other sites linking to my page with a low domain authority affect my rankings?
Hello, my page authority 30, I see other page with an authority of 21 for example linking to us. Is this good or bad, does this affect our ranking? A lot of these don't have anchor text, I'm guessing we should reach out to these webmaster to include anchor text with the keywords we want to drive traffic to this page or make as descriptive as possible? Thanks! Laura
Link Explorer | | lauramrobinson320 -
OSE results vs. Google
Is OSE still a worthy analyzer of the factors breaking down Google rank of a page or website? More so over the past six months, I've seems to noticed how the page comparison in OSE seems to have less and less correspondence to Google rank. For instance, I'm analyzing sites where the top result has a Page Authority rank of 11, Subdomain MozRank of 3.53, outranking a page coming in in spot #14 with Page Authority of 36, Subdomain MozRank of 3.94. I see this frequently, that by the factors measured in OSE, a certain page with favorable scores is being outranked by a page with poor OSE scores. Of course, I realize how sophisticated Google's algorithm is and the OSE can't reverse engineer the whole thing. Also, there are lots of other factors, such as personalized search results, banklink velocity, and who's in my G+ circles, that isn't taken into account in OSE. Yet, I see this more-and-more, where websites kicking ass in OSE measurements rank poor, and poor websites ranked in OSE come out near the top in Google. So, given this, where OSE is apparently a less effective tool to help analyze the factors building Google rank as Google evolves and values brand, what should a chap do? I already know the predictable answers of do RCS, build your community, and so on. But, I want to know what's driving the juice behind these top ranking websites, and OSE doesn't seem to be cutting it anymore... Your thoughts?
Link Explorer | | ExploreConsulting0 -
Inbound links export report link redirect to OSE
Hi there, I have received email notification that my new inbound links CSV report is ready to export and download. I follow the OSE link but I am redirected straight back to the main page of OSE not the download. Is there a problem? Are the access problems due to the Moz.com and Research Tools Outage. reported on Moz Health? David
Link Explorer | | David-E-Carey0 -
Updating OSE data
Hello Moz, I was wondering if anyone knows how Moz goes about updating their OSE data? I've been doing some link building that past couple of weeks, and I notice that the number of "just discovered" links will update every time I gain a new link, however, the "Linking domains" number doesn't (or maybe, rarely) update. I've checked through the "Linking domains" list to see if the link is there, but none of the links I've just recently obtained have shown up under that metric. Does anyone have any insight on this or is anyone experiencing the same issue? Thanks for reading!
Link Explorer | | GalcoIndustrial1