Does OSE crawl our site often? Or do we have an other problem? (internal links)
-
I am wondering if i can find out when OpenSiteExplorer crawled our entire website for the last time.
A few months ago we added some internal textlinks to all of our productpages. Those are links to the brandpage of that particular product. Some of our brands do have up to 1500 products, so there should be at least 1500 internal links pointing to that brandpage.But OSE still gives me a wrong count. It says there is only 1 internal link pointing to that brandpage.
Is it because OSE never crawled our site again? Or am i missing something here?(maybe too many internal links to one specific brandpage is not okay?)I'll hope you guys can help me out with this one.
-
Hi there!
For our inbound links, here is how we compile our index:
<code>- We grab the most recent index. - We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains). - We start crawling from the top down until we've crawled 90,000,000,000 pages (which is about 35% the amount in Google's index).</code>
Therefore, if the site is not linked to by one of these seed URLs (or one of the URLs linked to by them in the next update) then it won't show up in our index. Sorry!
We update our Mozscape Index every 4 weeks. Crawling the entire Internet to look for links takes 2-3 weeks, but our crawlers are always collecting data. When we need to put the index together, we grab all the data they have collected and start processing which can take up to 3 weeks to determine which of those links are the most important. You can see our most recently updated schedule here: http://moz.com/products/api/updates
Mozscape focuses on a breadth-first approach. Therefore we almost always have content from the homepage of websites, externally linked-to pages, and pages higher up in a site's information hierarchy. However, deep pages that are buried beneath many layers of navigation are sometimes missed and it may be several index updates before we catch all of these.
If our crawlers or data sources are blocked from reaching those URLs, they may not be included in our index (though links that point to those pages will still be available). Finally, the URLs seen by Mozscape must be linked-to by other documents on the web or our index will not include them.
For now, the best thing you can do to help your domain become indexed is to work on link building for links from sites with high mozrank.
Hope this helps clear things up!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Why Links Overview is showing '0' "Internal, followed links" when we have many?
Hi, Our site is showing '0' internal links on Links Overview when all other metrics appear to be updating correctly. Any idea why might this be happening? Or are there any issues with the MOZ getting the data from our site when it does a crawl?
Link Explorer | | Chris_Mc0 -
New Links Indexing Issue
Respected Team, I am facing new links indexing issue for one of my site, Is there any known indexing issue which is already addressed by Moz? Please help.
Link Explorer | | red-dr50 -
Archive problem
Hello. I cannot move notifications to the archive, nothing happens after clicking the Archive button. And I can’t go into the archive itself. What could be the problem?
Link Explorer | | architecturalcgi0 -
Open Site Explore page titles show as "No Title"
This has been asked many times but I cannot find an answer. On Open Site Explorer, most URLs I enter show as "No title" which means you have to hover over the URL to see which page is being referred to. I know you'd want an example, so here's your own site Moz.com 🙂 CWrkxZQ
Link Explorer | | clifra0 -
OSE: Missing back-links and High DA website with low DA backlinks
I have gone through a website with 81 DA; where its link equity back-links with highest DA of 70. Hardly 5 back-links 60 to 70 DA and just 25 root domains. This seems very unusual to me. Can someone clarify this for me? Can a website with low DA back-links can make high DA? If so what could be other major contributing factors for this?
Link Explorer | | vtmoz0 -
Can't see the link I earned from Moz in OSE
Hi Mozzers, A month ago I got the nofollow tag removed from the first custom link on my Moz profile but I can't see this in our links on OSE? SEO Spyglass shows it just fine. Is this related to the delays in the back end of Moz or have I set up my profile wrong? Cheers! Jamie
Link Explorer | | SanjidaKazi0 -
Why I am not seeing any inbound link to my website?
While using the Open Site Explorer tool, I don't get any resultfor Inbound Links, while with other services and the Webmaster Tools I can see which websites are linking to mine. What I can do to see these results here on the MOZ analytics software? The website is http://b5footballcup.com Thanks for the support. Luca
Link Explorer | | lucab50 -
Open Site Explorer not reporting all 301 redirected links
Our site had over 2,000 root domains linking to it as reported in MOZ Open Site Explorer (and Google Webmaster Tools). We then changed the domain and made sure that 301 redirects were set up for all pages across the site. That was about 1 month ago. Open Site Explorer is now reporting less than 300 linking root domains. For the links that it is reporting the majority of these are being 301 redirected to the new URLs (some of them we changed the links directly). However the majority of the links that are being 301 redirected are not being reported. It is reporting most of the links coming from the old to the new domain. Google Webmaster Tools is reporting over 1,000 linking root domains to the new domain (it has a max of 1,000). We did notice that MOZ took some time to update the domain authority of the new domain. It was 1 for along time and it has now jumped up to 46 ( it was previously at 74). Maybe this is a time delay thing and eventually Open Site Explorer will report all of the 301 redirected links? It is a bit frustrating at the moment as we can't fully analyse the links to the site to try to focus on the high domain linking sites to get them to change the link directly. Also, If all links are being 301 redirected to the new domain should the authority not be close to where it was previously after one month? Our search traffic has dropped considerably since the launch of the new site and hasn't returned yet, so just wondering if the 301 redirected links pass on as much value as thre original direct links. Thanks, Damien
Link Explorer | | james.harris0