How to download full list of internal links properly with OSE?
-
Hi Moz Community,
I am having issues getting a full list of my internal links when I go to the Open Site Explorer. When I export the internal links into excel, I am getting half of my URLs and 80% of what I am getting is absolutely crazy URLs that are super super old.
Plus, these old URLs are not even showing in my campaign crawls. So the crawls themselves are different.
Any help on this would be greatly appreciated, since I am trying to get a full picture on the internal link structure of my site.
Thanks!
-
Agree, I love Screaming Frog!
-
Hi Ryan,
It would be best to use Screaming Frog if you want a true indication of your internal link structure.
https://www.screamingfrog.co.uk/seo-spider/
Cheers,
David
-
Hey there! Tawny from Moz's Help Team here. I think I can help explain why you're not seeing all the links you might expect, and why you're seeing some older links that you're not seeing in your main Campaign Site Crawl.
Open Site Explorer and the Link Analysis page of Moz Pro Campaigns are both tied to our Mozscape index, which tends to update roughly once a month.
Just a few points on how we compile our index:
-
We grab the most recent index.
-
We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains).
-
We start crawling from the top down until we've crawled ~130 billion URLs
The idea here is that we're focusing on the highest-quality links we can find, coming from the most prominent pages of authoritative sites. So, while you may not see every link for a site within our index, we're aiming to report the most valuable ones available!
Most new sites and links will be indexed by our spiders and available in Mozscape and Open Site Explorer within 60 days, but some take even longer for many reasons, including the crawl-ability of sites, the number of inbound links to them, and the depth of pages in subdirectories. This tends to bias our index in favor of newer links. Linking data is only stored in the index for about 180 days, so unless the crawler has a compelling reason to return to your site and the sites linking to yours and rediscover those links, they can fall back out of our index again. That doesn't mean they're not out there affecting your SEO, just that our tools don't see them anymore.
You can see our most recently updated schedule here as well as some more technical metrics on our Mozscape API Updates page. You can also see when the last and next updates happened on the Open Site Explorer (OSE) homepage at any time.
Since Moz focuses on quality of links over quantity, we are always focused on the most relevant links to display to our users. It's possible that Moz's index will leave out some of the lower-quality (non-link juice providing) links out of our index because of this. So, that might explain why you may see some discrepancies with what other tools may be showing.
You can read more about how we build our index in our guide here.
I know this is a ton of information, so if you have any questions or if I didn't make anything clear enough, please don't hesitate to ask! You can always drop us a line at help@moz.com and we'll do our best to clear up any questions you might have.
Cheers! -
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved How to remove these spammy links
I've noticed that my website's spam score has increased recently, and upon further investigation, I found some spammy links pointing to my site. and my website www.fastlandoffer.com Moz Smap Score is 16
Link Explorer | | FredericoSeo
5 I haven't engaged in any spammy tactics, so I'm not sure how they got there. Does anyone have any tips or suggestions on how to effectively remove these spammy links from my website and improve its overall credibility and search engine rankings? Any insights or experiences would be greatly appreciated! Thanks in advance!0 -
Link Explorer is not working for particular domain
Hello, Linking Domain section in Link Explorer is not working fine for sewingmachineprice.com. When I put this domain to analyze for backlinks it says, "You entered the URL https://sewingmachineprice.com/ which redirects to www.hugedomains.com/domain_profile.cfm?d=sewingmachineprice&e=com. Click here to analyze www.hugedomains.com/domain_profile.cfm?d=sewingmachineprice&e=com instead." In fact, this domain is not redirected to any third domain or URL. I personally checked it and can access this website without any redirection.
Link Explorer | | Jacksmorrin0 -
Question about link report
Hey all, SEO noob here, back with another question. I downloaded a link report, and I don't understand the columns Links to Page, Outbound Domains from Page, and Outbound Links from Page. Is Links to Page the number of links from the source page to the target page? Is Outbound Domains from Page the number of domains linked to by the source page? Is Outbound Links from Page the number of outbound links to any domain or page from the source page? Thanks in advance! AK
Link Explorer | | AndyKubrin0 -
Are Unlinked Mentions Coming Back in the New OSE?
Hi Mozzers, Just a quickie, with the unlinked mentions being retired, I was wondering if Moz have any plans to bring this feature back through the new link explorer? Many thanks! 🙂
Link Explorer | | Corbec8881 -
Moz Spam Score 9/17 when there are no links
Hi I was just looking at a competitors new site in OSE and it had a DA and PA of 1 which I would expect and also no links, which again I would expect, basically the guy used to design stuff for me and thinks he can do a better job so he is giving it ago. But he has a spam score of 9/17 but no links - how is this even possible. Thanks Andy
Link Explorer | | Andy-Halliday0 -
Open Site Explorer -> Just Discovered -> Internal Links: Only shown 1 url link per day
Hi, We have a page that have like 10-15 new content/pages per day.
Link Explorer | | Inspiringinterns
eg. domain.com/post-id-01/, omain.com/post-id-02/, omain.com/post-id-03/ etc
But somehow, under Just-Discovered, internal links only show 1 per day when it suppose to display like 10-15.
The layout for each pages is exact same except for the content changes and the url's ID changes. Our competitor on the other hand, they have like 5 new page per day, but somehow Moz shows their internal links as 25 per days. Each of their new url have 4 more prefix number.
Eg:
new-page-01?track=990
new-page-01?track=991
new-page-01?track=992 etc
The above example url is technically as 1 page if without the "?" affix. Whiles all our new pages/url have unique id, but Moz only show 1 a day. Please advise on how to get Moz to shows all the new pages in Just-Discovered (internal link). Thanks, Cam0 -
Dofollow link from Moz user profile not showing in OpenSiteExplorer
Hi everyone! I'm was searching my "new" dofollow link from my Moz user profile http://moz.com/community/users/625691 now I have more than 200 MozPoints, but in OpenSiteExplorer is only showing my "nofollow" link of the icon (the "M" icon in the left of the URL in the profile). Why not appears the dofollow link? Thanks!
Link Explorer | | rubenalonsoes1 -
OSE - Link Opportunities
I'm checking out this new Link Opportunities feature for one of our sites and what I'm seeing right now is pretty disappointing. For Reclaim Links, everything listed is an internal link on our site. It's got tons of URLs from our old Iciniti structure (replatformed to Magento mid-March). It's crawling tons of stuff that's blocked in robots. There are no links from external domains in the first 5 pages. For Unlinked Mentions, it's showing tons of mentions - 33,566 to be exact, a ridiculous number - mostly from news sites like Forbes, WSJ, Guardian, CNN, etc. These sites are not mentioning us. It's set up to look only for our brand name or domain name, so I don't know how it's thinking there are all these nonexistent mentions. What's going on with Link Opportunities?
Link Explorer | | Kingof50