Crawling and Links - Showing Links
-
Hello
I have a campaign in Moz that shows a new site as having no domain authority and no links.
The site does have links to it, but they don't show up on OSE.
Does Moz count links in the comments section of blogs (or is this counted as social, when the commenting system is one that requires facebook login?) I know blog comments are not great for SEO, I just use them when the post/link is 100% genuinely relevant. The site does have actual links on websites not just comment links, so it isn't showing any type of link.
What is the best way to see the links the site has?
Thanks!
-
Hey Radi,
The link metrics section of your campaign pulls from our Mozscape index, which updates about once a month.
Most new sites and links will be indexed by our spiders and available in Mozscape and Open Site Explorer within 60 days, but some take even longer for a variety of reasons, including the crawl-ability of sites, the amount of inbound links to them, and the depth of pages in subdirectories.
Just so you know, here's how we compile our index:
- We grab the most recent index
- We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains)
- We start crawling from the top down until we've crawled 180,000,000,000 pages (which is about 35% the amount in Google's index)
Therefore, if the site is not linked to by one of these seed URLs (or one of the URLs linked to by them in the next update), then it won't show up in our index.
We update our Mozscape Index every 4-8 weeks. Crawling the entire Internet to look for links takes ~2-4 weeks, but our crawlers are always collecting data. When we need to put the index together, we grab all the data they have collected and start processing, which can take up to 3 weeks to determine which of those links are the most important.
Mozscape focuses on a breadth-first approach. This means that we almost always have content from the homepage of websites, externally linked-to pages, and pages higher up in a site's information hierarchy. But deep pages that are buried beneath many layers of navigation are sometimes missed and it may be several index updates until we catch all of these.
If our crawlers or data sources are blocked from reaching those URLs, they may not be included in our index (though links that point to those pages will still be available). Finally, the URLs seen by Mozscape must be linked to by other documents on the web, or our index will not include them.
I hope this information helps! While the site and links may not be indexed yet, give it some time - maybe we'll see them in OSE next month!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Internal Links is reporting "0" value
Open Site Explorer is reporting "0" value under "Total Internal Links" as well as "Internal Links" metrics for my website. I have a good interlinking of pages throughout the site.
Link Explorer | | anupjain
Is there an issue with OSE (Open Site Explorer) crawler ? OR There are some crawling problems with our Website's structure ?
Can anybody please give an idea or recommendations on this ?
Domain: https://www.traveltoindia.org/0 -
Links in link lists not being discovered by Moz
I uploaded a couple of link lists to Moz about a week ago and have been tracking them. Most of the links on the list are from significant 90+ DA domains such as Wikipedia, Medium, etc. So far, Moz only seems to have discovered about 4 the 200 or so links I loaded. Do we know how long it takes for these to be discovered by Moz? Quite new to this, so any help understanding timelines for this would be appreciated.
Link Explorer | | RobertPhillips180 -
Apart from spying on competitors back link what else can be done in MOZ?
I've recently started trail version in MOZ and I want to explore every possible way that can help me in building my website better. My website Sanctum Consulting (Visas and Immigration)
Link Explorer | | Manifeat90 -
What happened to my links in Open Site Explorer.
Hello, A while ago I had a problem with open site explorer saying that I had a problems with redirects on my home page. There was 302 redirect and it was giving me information from my www domain rather then from non www domain. With your awesome help, this issue is sorted now and when I type in my non www domain, it doesn't give me that weird message anymore. I have a different issue now. For some reason when I enter non www domain it doesn't show links from www version. I presume that this is affecting my site authority for non www version. Does it mean that link juice was not passed on to non www version from www version? Authority dropped by two points, but I'm not sure why. I'm more interested to know why it doesn't show my links from www version on non www version. And is that affecting my sites rankings. Thank you, Regards, Armands
Link Explorer | | A_Fotografy0 -
Why does OSE show different link equity for homepage with and without trailing slash?
Hi all, I am working with a developer who believes OSE (Open Site Explorer) is flawed. His reasoning is that OSE shows completely different link equity and domain authority for these two URLs [company name is purposely withheld]: http://www.mydomain.com/us/ and http://www.mydomain.com/us NOTE: http://www.mydomain.com/us has been properly 301-redirecting to http://www.mydomain.com/us/ for years. The developer's argument is that the URL that is being 301-redirected shouldn't have any link equity at all, therefore OSE is flawed. Here is the response I would like to give him. Please feel free to poke holes, provide feedback or flat out correct me if I am wrong: Dear inquisitive developer, Any URL that is being linked to by another site that has link equity will get link equity passed to it. This is true even if the URL is a page that doesn't exist. What it can't do (if it doesn't exist) is then pass that link equity on to another page. Moz's OSE is showing separate link equity for the two versions of this URL because other sites have linked to both versions. The fact that one 301-redirects to the other is good and ensures that link equity from one is passing to the preferred URL. The 301-redirect does not, however, remove link equity from the first URL, but rather passes the existing link equity from one to the other. Consequently, Moz's OSE is not flawed, but rather is displaying the link equity of one URL that is benefiting the other via a 301-redirect. Sincerely, Dana
Link Explorer | | danatanseo3 -
Is there some way to tell the Moz crawler not to crawl URL's with particular dynamic tags such as "?redirect-to:http//" ?
We are encountering an issue where the crawler is finding a ton of pages from our wordpress login url that has this dynamic tag in it to kinds of different blog entries. It's madness. I can't figure out what is causing these URLs to generate to be crawled in the first place! Does this sound familiar to anyone out there, any constructive suggestions? Robots text or maybe meta robots tags that would resolve this crawl issue?
Link Explorer | | RegistrarCorp0 -
Why does facebook, twitter, pinterest show up for links?
Hello, As I have used site explorer on competition sites, I am curious what I need to do to have my facebook, twitter, and pinterest links to shop up as a backlink? I see some other sites have it, even if its a nofollow, but curious if I am doing something wrong? I have had accounts since last year. Just curious if I am doing something wrong, Thank you Chris
Link Explorer | | Berner0 -
Does the inbound links report include links to all pages of the domain being researched?
If I enter 'abc.com' am I only getting results for 'abc.com' or will I get results for the internal pages of 'abc.com' as well (i.e. 'abc.com/page1.html)? There is a bit of a discrepancy between these results and inbound link results in semrush for example. Then again it seems whenever you use different tools to measure the same thing you get wildly varying results. How do you all deal with that?
Link Explorer | | AISEO0