How effective is OSE in crawling press release links?
-
How effective is OSE in crawling press release links?
We have released a few press releases recently (over the last couple of months) and OSE doesn't seem to have found them.
-
Hey There,
That could be a possibility. It is hard to say definitively given the nature of web crawlers. They just crawl links as they see them in random succession, so a lot of factors come into play.
Best,
Nick
SEOmoz -
Our releases have appeared on big sites like the Financial Post.
Is it possible they just get buried under other news so OSE can't find them? I know Google indexes these pages, we get the alerts.
-
Hey there,
Just so you know, here's how we compile our index: - We grab the most recent index. - We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains). - We start crawling from the top down until we've crawled 59,000,000,000 pages (which is about 25% the amount in Google's index).
Therefore, if the site is not linked to by one of these seed URLs (or one of the URLs linked to by them in the next update) then it won't show up in our index. Sorry!
We update our Linkscape Index every 4 weeks. Crawling the entire Internet to look for links takes 2-3 weeks, but our crawlers are always in motion. When we need to start processing, we grab all the data they have collected and start processing which can take up to 3 weeks to determine which of those links are the most important. You can see our most recently updated schedule here: http://seomoz.zendesk.com/entries/345964-linkscape-update-schedule
Linkscape focuses on a breadth-first approach. Therefore we almost always have content from the homepage of websites, externally linked-to pages, and pages higher up in a site's information hierarchy. However, deep pages that are buried beneath many layers of navigation are sometimes missed and it may be several index updates before we catch all of these.
If our crawlers or data sources are blocked from reaching those URLs, they may not be included in our index (though links that point to those pages will still be available). Finally, the URLs seen by Linkscape must be linked-to by other documents on the web or our index will not include them.
For now, the best thing you can do to help your domain become indexed is to work on link building for links from sites with high mozrank.
Best,
Nick
SEOmoz -
This all depends on where the press releases have been posted.
If you've got the urls of the sites they're on it may be worth looking at these in OSE to see if SEOmoz has them indexed. However, don't forget that the SEOmoz index is not the same as google's. Just because it's not showing on OSE doesn't mean that G hasn't seen it.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Huge Decline in Links
Good Morning Everyone, Looking for some feedback as to why all of my Backlink metrics might be way down (as well as rankings)... Please see the details below that show some of the metrics from MOZ reports from August and from October. Does anyone know why these metrics are all so down? We have not done any link removal exercises or anything that would cause this drop --- please let me know if there is any insight any of you have as to what is the reason for this drop. Thanks Linking C Blocks August 3: 124 October 14: 23 External Followed Links August 3: 4486 October 14: 1558 Total External Links August 3: 4795 October 14: 1680 Total Links August 3: 21338 October 14: 17809 Followed Linking Root Domains August 3: 323 October 14: 116 Total Linking Root Domains August 3: 442 October 14: 143
Reporting & Analytics | | Prime850 -
Google Web Master Tools show that my site has been crawled, but search results show old title tags, ect.
Does the index report reflect what will be displayed on Google results? Google seems to be indexing my site every Sunday...
Reporting & Analytics | | 928shopper0 -
How to Detect Links within PDFs
Hi All, I have a funny situation that I would like some advice on handling... There are a handful of domains that were created several years ago in support of an offline to online campaign. These domains are simply vanity domains that use an IFrame at 100% to show the content of another page. Essentially, the content of the sites I manage are embedded into the frame on the vanity URL. Since I do not monitor or have access to any analytics for the vanity URLs, is there a way to tell how others are discovering those vanity URLs? As stated above, they were used on direct mail flyers two years ago and never appeared online. However, I still get a good deal of traffic from them and cannot believe people have hung onto those flyers in such volume. I have used Open Site Explorer for the vanity URLs, which show no links existing anywhere online. I am wondering if the vanity URLs may exist in pdf lists of local businesses that match my category, etc. Is there any way to tell how traffic finds those vanity URLs without analytics or discovered links through link profiling tools?
Reporting & Analytics | | dsinger0 -
Unexplained Crawl Diagnostic Errors & Opencart
Hi, I've been looking at the crawl diagnostics for my site and trying to fix the errors that are showing up but Seomoz is producing some strange results. It's saying pages are duplicated upto 16 times but those pages dont exist. It's adding "page=3", "page=4" to the end of the product URL but I don't see how it's finding those pages, nothing on the site(as far as I can tell) is linking to them. There is no "page=3", just the one product page. Again on the duplicate content it's saying under the "other URLs" there's URLs like "http:///product-a" but again I don't see where it's finding these URLs and obviously those URL's dont work. Those three slashes aren't a typo either. So far I've reduced the amount of errors from 2,005 to 543 but the rest of them I can't make sense of. Also, what does one do when you have two products, eg: "product-a-white" and "product-a-black" to prevent Seomoz from seeing duplicates? Canonical links wont work because there's no parent item, just those two. Google Webmaster tools doesn't seem to have a problem though. Using Opencart 1.5, if it helps. Cheers,
Reporting & Analytics | | AsOneDesign0 -
Is the link data from Open Site Explorer in real time or an average?
I just started using Open Site Explorer to track internal and external link data. Is this information given in real time or is it an average over a specified period of time?
Reporting & Analytics | | mequoda0 -
Links are not found in OSE or am I missing ?
Hi.. All of our targeted pages have gone through serious Blog Posting and Social Bookmarking. All the sites were more than PR1 or 2 and all of them had a dofollow link with the perfect anchor texts. But when I trying to check our pages on OSE, it doesnt any of them. Why is that ? Im getting worried now. We have spent our last 7 months on this .
Reporting & Analytics | | qubesys0 -
Linking domain dropped followed by significant decrease in Domain Authority
A relatively high ranking site disappeared from the linking root domains of a client site, and the site dropped significantly in domain rank, authority, and trust. Oddly enough, traffic is almost double the previous week, and search results for keywords remain strong. I checked OSE and the linking domain is not listed. There are also several other linking root domains that have never shown in the domain analysis, but we'll leave those for a later topic. I visited the linking root domain that disappeared from the analysis and found everything to be in order. The site is a major local forum with a relatively high domain authority. All links are still there, and there hasn't been any change to the site as far as links are treated according to the owner. The only thing that has changed since the last site analysis is an event campaign ran with unpaid ads posted throughout the forum. The site being analyzed is a catering company that is cooperating with the forum owner and local bar to host a local celebratory event. Of course I'll wait for the next indexing to see if things change, but what could be going on here? This is more of an SEOmoz question, as we're talking over 10 point deduction in domain authority, although search results are still strong.
Reporting & Analytics | | kwoolf0