How effective is OSE in crawling press release links?
-
How effective is OSE in crawling press release links?
We have released a few press releases recently (over the last couple of months) and OSE doesn't seem to have found them.
-
Hey There,
That could be a possibility. It is hard to say definitively given the nature of web crawlers. They just crawl links as they see them in random succession, so a lot of factors come into play.
Best,
Nick
SEOmoz -
Our releases have appeared on big sites like the Financial Post.
Is it possible they just get buried under other news so OSE can't find them? I know Google indexes these pages, we get the alerts.
-
Hey there,
Just so you know, here's how we compile our index: - We grab the most recent index. - We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains). - We start crawling from the top down until we've crawled 59,000,000,000 pages (which is about 25% the amount in Google's index).
Therefore, if the site is not linked to by one of these seed URLs (or one of the URLs linked to by them in the next update) then it won't show up in our index. Sorry!
We update our Linkscape Index every 4 weeks. Crawling the entire Internet to look for links takes 2-3 weeks, but our crawlers are always in motion. When we need to start processing, we grab all the data they have collected and start processing which can take up to 3 weeks to determine which of those links are the most important. You can see our most recently updated schedule here: http://seomoz.zendesk.com/entries/345964-linkscape-update-schedule
Linkscape focuses on a breadth-first approach. Therefore we almost always have content from the homepage of websites, externally linked-to pages, and pages higher up in a site's information hierarchy. However, deep pages that are buried beneath many layers of navigation are sometimes missed and it may be several index updates before we catch all of these.
If our crawlers or data sources are blocked from reaching those URLs, they may not be included in our index (though links that point to those pages will still be available). Finally, the URLs seen by Linkscape must be linked-to by other documents on the web or our index will not include them.
For now, the best thing you can do to help your domain become indexed is to work on link building for links from sites with high mozrank.
Best,
Nick
SEOmoz -
This all depends on where the press releases have been posted.
If you've got the urls of the sites they're on it may be worth looking at these in OSE to see if SEOmoz has them indexed. However, don't forget that the SEOmoz index is not the same as google's. Just because it's not showing on OSE doesn't mean that G hasn't seen it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why might my websites crawl rate....explode?
Hi Mozzers, I have a website with approx 110,000 pages. According to search console, Google will usually crawl, on average, anywhere between 500 - 1500 pages per day. However, lately the crawl rate seems to have increased rather drastically: 9/5/16 - 923
Reporting & Analytics | | Silkstream
9/6/16 - 946
9/7/16 - 848
9/8/16 - 11072
9/9/16 - 50923
9/10/16 - 60389
9/11/16 - 17170
9/12/16 - 79809 I was wondering if anyone could offer any insight into why may be happening and if I should be concerned?
Thanks in advance for all advice.0 -
How long before links are found in analytics?
I'm new to website building and have recently set up some links by submitting my website in directories and have also got some people in the same industry to link to my website. But they aren't being found in moz analytics. Does it take time or should they be getting found?
Reporting & Analytics | | PaddyCoop0 -
Disavow links.??.Any One
Hi There, Just checking Google webmasters for one website. I have download a list links that was show in webmasters under "Links to Your Site". And found that there are few not found urls that or expired domain. May be that belongs to very old links building on bookmarking websites. I am thinking to submit all these links list in disavow links. Your thoughts.?
Reporting & Analytics | | lucidsoftech0 -
Can I view lost links?
Hello! One of the blogs I'm tracking has literally lost like half of the number of external followed links. So, I have two questions really 1) is that common at all? it seems odd that that many would be removed 2) is there a way to see the specific links that were lost? Thanks!
Reporting & Analytics | | yandl
Kevin0 -
Total Linking Root Domins
I'm still very new to link building and SEO, (only one month in!) I have a reasonable starting point as our web site is fairly well designed but I have a long way to go to catch up in what is a very competitive market! (Music Retail). My question is regarding Open Site explorer and its ability to accurately reflect back links. Looking at total quantity of unique back link domains Open Site currently has me at 36 domains, Google webmaster tools has me at 187!! The one site that all my 'watched' competitors has is DMOZ, this gives them a great boost on the SEOmoz and open site rankings. I have 3 links on DMOZ and they all show up on my Webmaster tools yet don't show on either SEOmoz or open site. I've left it for a couple of months to see if it caught up but no real change. My worry is that there seems little value in the SEOmoz tools if their not at least as complete as Google webmaster is to allow me a fair comparison! Is there anything wrong with my setup on SEOmoz??
Reporting & Analytics | | rattleanddrum0 -
Subdomain and relative link paths cause crawl errors
I have a Wordpress blog on our subdomain and we use relative paths on our domain. It appears as though Google bot is crawling from the subdomain categories back to the domain relative paths. This of course results in hundreds of 404 pages. Any suggestions as to how to resolve this issue without changing the relative path structure of our domain? I can provide more information if need be. While I realize these issues are not that pressing, I'd obviously like to remove as many errors as possible. If anyone has encountered this problem, especially in Wordpress I'd really like to hear your solution or lack there of. Thank you in advance.
Reporting & Analytics | | BethA0 -
Too Many On-Page Links
I'm getting the warning of "Too many on-page links". I have a number of affiliate marketing sites, all Wordpress, all with sidebars. In the header navigation bar is a link that offers "reviews" and links to each product/brand. This is not a drop down link but leads to a page with product links. Also, in the sidebar I have links that also lead to the products/brands. Redundant yes, but in the beginning this seemed to be a good practice in site design. It would be easy to simply remove the widget that contains the link process in the sidebar. The problem may simply be that the sidebar is the same for every page in the site(s). Is this hurting total SEO as my sites are all over 95-percent indexed. One site in particular does very well in traffic and sales so is removing these links potentially going to improve my SEO and ultimately by success? Thanks, Don
Reporting & Analytics | | JavaManOne0 -
On what report do I get to know where do the external links come from?
I need to get a list of the external links. I don't find that report, where do I get it?
Reporting & Analytics | | carloscontinua0