Can we add sites to the crawl queue for OSE?
-
Is it possible to request that Open Site Explorer crawls a new URL on its next run?
This tool is the first place I go to when working on a new site, and when there is "No Data Available" this is a little frustrating.
I fully appreciate that this lack of data is usually a signal that the website is either very new or of low quality, however that if often the reason that I am brought in and would very much like to benchmark and provide initial analysis using this tool.
It would make sense that OSE crawls the sites that Moz members are working on wouldnt it?
Scott.
-
Hi Scott,
Derek and Brad are correct in that there's no real way to tell Roger to index a specific page.
Your best bet to get your page index in OSE is to work at building quality backlinks and making sure your site is configured properly so we can reach your pages.
Also remember you can set up your campaign for your site and crawl it that way if you want to evaluate the rest of your page information.
I hope that helps.
Cheers,
Joel. -
You "ask it what it knows" when you type in a URL and hit Search. Think of it like walking up to a really smart person and asking them to list off every book every written that includes the word "rhinoceros." They can only list the books they've read. A person only has so much time for reading, and OSE only has so many resources for crawling the web. It will never get everything.
If OSE doesn't show any backlinks for a URL, it means it didn't see any backlinks in the portion of the web that it crawled. That might mean there are no backlinks at all, or it might mean the backlinks are yet undiscovered (probably because they're in an obscure corner of the web). The best suggestion is to use multiple tools, as Derek says below.
-
SeoMoz members have no control over what Mozbot crawls. Your best bet is to use a variety of site explorers to find links. Ex. Majestic, Google Webmaster tools, Link:www.example.com, Bing Link Explorer
-
Yes I know what you mean here, but if there is any way to "point" the crawl engine at the site and ask it what it knows?
I hadnt thought about it from that perspective, just asking the question to see what is possible to try and get more data if it can be got?!
-
As far as I know, crawling your website wouldn't accomplish anything. OSE would need to crawl the pages that link to your website. And if you knew what those were, you wouldn't need OSE, right?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
OSE for Facebook
Hi, I recall being able to use OSE for Facebook. Take https://www.facebook.com/VICE/ which we know as a URL would have many backlinks. It's need registering any. Has this always been the case?
Moz Pro | | wearehappymedia0 -
Have a Campaign, but only states 1 page has been crawled by SEOmoz bots. What needs to be done to have all the pages crawled?
We have a campaign running for a client in SEOmoz and only 1 page has been crawled per SEOmoz' data. There are many pages in the site and a new blog with more and more articles posted each month, yet Moz is not crawling anything, aside from maybe the Home page. The odd thing is, Moz is reporting more data on all the other inner pages though for errors, duplicate content, etc... What should we do so all the pages get crawled by Moz? I don't want to delete and start over as we followed all the steps properly when setting up. Thank you for any tips here.
Moz Pro | | WhiteboardCreations0 -
How Old is OSE link data?
I ran an anchor text report for my client today, which shows that their site has some incoming comment spam links using totally unrelated phrases (pharma products). However, when looking for the live link, the linking page no longer contains the link to them. Maybe the webmasters removed these, but I can't track down a single one... how old is this data? thanks
Moz Pro | | JMagary0 -
Can I base a SEO campaign on a specific MLS listing inteads of full broker web site?
I have been ask to do a SEO campaign for one MLS listing: house for sale in the rosedale area in Toronto, and I am having difficulty to target one specific MLS listing . I don't want to do the SEO campaign on http://jimmymolloy.com but rather one specific listing http://jimmymolloy.com/feature/?mls=A_C2337646&listing=for-sale Google Analytic does not allow me to have a URL with a query.
Moz Pro | | pd20120 -
Campaign Crawl Report
Hello, Just a quicky, is there anyway I can do a crawl report for something in a campaign so I can compare the changes? I know you can do a separate crawl test, but it wont show the differences,and the next crawl date isnt untill the 28th.
Moz Pro | | Prestige-SEO0 -
Issue in number of pages crawled
i wanted to figure out how our friend Roger Bot works. On the first crawl of one of my large sites, the number of pages crawled stopped at 10000 (due to the restriction on the pro account). However after a few weeks, the number of pages crawled went down to about 5500. This number seemed to be a more accurate count of the pages on our site. Today, it seems that Roger Bot has completed another crawl and the number is up to 10000 again. I know there has been no downtime on our site, and the items that we fixed on our site did not reduce or increase the number of pages we had. Just making sure there are no known issues with Roger Bot before I look deeper into our site to see if there is an issue. Thanks!
Moz Pro | | cchhita0 -
Can I exclude pages from my Crawl Diagnostics?
Right now my crawl diagnostic information is being skewed because it's including the onsite search from my website. Is there a way to remove certain pages like search from the errors and warnings of the crawl diagnostic? My search pages are coming up as: Long URL Title Element Too Long Missing Meta Description Blocked by meta-robots (Which is how I want it) Rel Canonical Here is what the crawl diagnostic thinks my page URL looks like: website.com/search/gutter%25252525252525252525252525252525252525252525252525252525 252525252525252525252525252525252525252525252525252525252525252 525252525252525252525252525252525252525252525252525252525252525 252525252525252525252525252525252525252525252525252525252525252 52525252525252525252525252525252525252525252525252Bcleaning/ Thank you, Jonathan
Moz Pro | | JonathanGoodman0 -
Only Crawling 1 page?
Hi Guys, Any advice much appreciated on this! Recently set up a new campaign on my dashboard with just 5 keywords. The domain is brammer.co.uk and a quick Google site:brammer.co.uk shows a good amount of indexed pages. However - first seomoz tool crawl has only crawled 1 url!! "Last Crawl Completed: Apr. 12th, 2011 Next Crawl Starts: Apr. 17th, 2011" Any ideas what's stopping the tool crawl anymore of the site?? Cheers in advance.. J
Moz Pro | | lovealbatross0