How effective is OSE in crawling press release links?
-
How effective is OSE in crawling press release links?
We have released a few press releases recently (over the last couple of months) and OSE doesn't seem to have found them.
-
Hey There,
That could be a possibility. It is hard to say definitively given the nature of web crawlers. They just crawl links as they see them in random succession, so a lot of factors come into play.
Best,
Nick
SEOmoz -
Our releases have appeared on big sites like the Financial Post.
Is it possible they just get buried under other news so OSE can't find them? I know Google indexes these pages, we get the alerts.
-
Hey there,
Just so you know, here's how we compile our index: - We grab the most recent index. - We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains). - We start crawling from the top down until we've crawled 59,000,000,000 pages (which is about 25% the amount in Google's index).
Therefore, if the site is not linked to by one of these seed URLs (or one of the URLs linked to by them in the next update) then it won't show up in our index. Sorry!
We update our Linkscape Index every 4 weeks. Crawling the entire Internet to look for links takes 2-3 weeks, but our crawlers are always in motion. When we need to start processing, we grab all the data they have collected and start processing which can take up to 3 weeks to determine which of those links are the most important. You can see our most recently updated schedule here: http://seomoz.zendesk.com/entries/345964-linkscape-update-schedule
Linkscape focuses on a breadth-first approach. Therefore we almost always have content from the homepage of websites, externally linked-to pages, and pages higher up in a site's information hierarchy. However, deep pages that are buried beneath many layers of navigation are sometimes missed and it may be several index updates before we catch all of these.
If our crawlers or data sources are blocked from reaching those URLs, they may not be included in our index (though links that point to those pages will still be available). Finally, the URLs seen by Linkscape must be linked-to by other documents on the web or our index will not include them.
For now, the best thing you can do to help your domain become indexed is to work on link building for links from sites with high mozrank.
Best,
Nick
SEOmoz -
This all depends on where the press releases have been posted.
If you've got the urls of the sites they're on it may be worth looking at these in OSE to see if SEOmoz has them indexed. However, don't forget that the SEOmoz index is not the same as google's. Just because it's not showing on OSE doesn't mean that G hasn't seen it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why might my websites crawl rate....explode?
Hi Mozzers, I have a website with approx 110,000 pages. According to search console, Google will usually crawl, on average, anywhere between 500 - 1500 pages per day. However, lately the crawl rate seems to have increased rather drastically: 9/5/16 - 923
Reporting & Analytics | | Silkstream
9/6/16 - 946
9/7/16 - 848
9/8/16 - 11072
9/9/16 - 50923
9/10/16 - 60389
9/11/16 - 17170
9/12/16 - 79809 I was wondering if anyone could offer any insight into why may be happening and if I should be concerned?
Thanks in advance for all advice.1 -
Tracking Mailto Links Through Event Tracking
Hi I am trying to get event tracking working on some mailto links on a website using Universal Analytics. I've read various resources as nothing seems to mention mailto links. Does it not work on tracking outbound clicks on emails? Thanks in advance.
Reporting & Analytics | | Karen_Dauncey0 -
Webmaster Tools V. OSE Backlinks to Disavow
Hey Everybody, So I am getting to the point where I need to disavow some backlinks. I am noticing a discrepancy between the OSE backlinks and the GWMT backlinks. My question is do I put both sets of links in the Disavow tool? Just the links from GWMT? Just the links from OSE? Etc... Etc... Thanks Everybody!
Reporting & Analytics | | HashtagHustler0 -
Difference between External links and anchor text?
What is the exact meaning of external links & anchor text and what is different between these?
Reporting & Analytics | | surabhi60 -
Difference in incoming links between web masters tool and moz
Why are incoming links to my site in Google Webmasters tools totally different then in MOZ? Which is correct? According to Google as of this week I have hundreds of links coming from linkarena (dot) com (Never heard) Does any one know who they are or if they are hurting me? I definitely did not pay them to put me on there. It looks like a German site. Thank you.
Reporting & Analytics | | omakad0 -
What is the best tool to look at Internal Links?
My site has on average 220 internal links a page largely due to how the site is built ie every page shows every link via the menu system, which isnt great, but from what i have read there isnt much i can do about it. However, this week 3 pages jumped to over 400 links and the home page over 500 internal links, GWT will only show me 200 of them, Is there a tool that i can use that will list all my internal links, so i can find out why this has happened? Thanks Ash
Reporting & Analytics | | AshShep10 -
Link Analysis Past 6 Months
I'm analyzing inbound links for a site and I was wondering if there is a way to see how many of the links were created in a certain time period? Example how many inbound links were sent to this sub domain during the past 6 months? I would think there would be a fairly simple way to do this, but not sure. Any help is greatly appreciated.
Reporting & Analytics | | seantgreen0 -
Tracking Effects of Internal Linking
Can anyone suggest the best way to track the effectiveness of internal linking on a website? Thanks!
Reporting & Analytics | | RishadShaikh590