Can we add sites to the crawl queue for OSE?
-
Is it possible to request that Open Site Explorer crawls a new URL on its next run?
This tool is the first place I go to when working on a new site, and when there is "No Data Available" this is a little frustrating.
I fully appreciate that this lack of data is usually a signal that the website is either very new or of low quality, however that if often the reason that I am brought in and would very much like to benchmark and provide initial analysis using this tool.
It would make sense that OSE crawls the sites that Moz members are working on wouldnt it?
Scott.
-
Hi Scott,
Derek and Brad are correct in that there's no real way to tell Roger to index a specific page.
Your best bet to get your page index in OSE is to work at building quality backlinks and making sure your site is configured properly so we can reach your pages.
Also remember you can set up your campaign for your site and crawl it that way if you want to evaluate the rest of your page information.
I hope that helps.
Cheers,
Joel. -
You "ask it what it knows" when you type in a URL and hit Search. Think of it like walking up to a really smart person and asking them to list off every book every written that includes the word "rhinoceros." They can only list the books they've read. A person only has so much time for reading, and OSE only has so many resources for crawling the web. It will never get everything.
If OSE doesn't show any backlinks for a URL, it means it didn't see any backlinks in the portion of the web that it crawled. That might mean there are no backlinks at all, or it might mean the backlinks are yet undiscovered (probably because they're in an obscure corner of the web). The best suggestion is to use multiple tools, as Derek says below.
-
SeoMoz members have no control over what Mozbot crawls. Your best bet is to use a variety of site explorers to find links. Ex. Majestic, Google Webmaster tools, Link:www.example.com, Bing Link Explorer
-
Yes I know what you mean here, but if there is any way to "point" the crawl engine at the site and ask it what it knows?
I hadnt thought about it from that perspective, just asking the question to see what is possible to try and get more data if it can be got?!
-
As far as I know, crawling your website wouldn't accomplish anything. OSE would need to crawl the pages that link to your website. And if you knew what those were, you wouldn't need OSE, right?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
WEbsite cannot be crawled
I have received the following message from MOZ on a few of our websites now Our crawler was not able to access the robots.txt file on your site. This often occurs because of a server error from the robots.txt. Although this may have been caused by a temporary outage, we recommend making sure your robots.txt file is accessible and that your network and server are working correctly. Typically errors like this should be investigated and fixed by the site webmaster. I have spoken with our webmaster and they have advised the below: The Robots.txt file is definitely there on all pages and Google is able to crawl for these files. Moz however is having some difficulty with finding the files when there is a particular redirect in place. For example, the page currently redirects from threecounties.co.uk/ to https://www.threecounties.co.uk/ and when this happens, the Moz crawler cannot find the robots.txt on the first URL and this generates the reports you have been receiving. From what I understand, this is a flaw with the Moz software and not something that we could fix form our end. _Going forward, something we could do is remove these rewrite rules to www., but these are useful redirects and removing them would likely have SEO implications. _ Has anyone else had this issue and is there anything we can do to rectify, or should we leave as is?
Moz Pro | | threecounties0 -
Get into Google : New Sites
I have a brand new website. It was created 10 days ago. How long would it take for it to show up in search results? I understand that since the site is new, there are no sites sending it backlinks. Also, i have optimized the page for my keyword "xyz" and it received an A grade. The site does not figure even in the top 50 results. Please help me out. It is a one page web application that needs to drive traffic to survive.
Moz Pro | | dl_s0 -
Data from Open Site Explorer to Excel
Hello, Im having a problem with the data I pull off from Open Site Explorer. Everytime I download a report in CSV, when I open it in Excel 2007, all the information is like this: http://i.imgur.com/rwMxO.png What can I do to extract the information exactly as it appears in the Open Site Explorer with all the fields in the right place? Tks guys, Regards, Pedro Pereira [
Moz Pro | | PedroM
](<a href=)" target="_blank">a> rwMxO.png
0 -
Why are we not ranking? We have done all we can think of. Can you help?
We have an old domain that we have had for years now. And recently. December, we completely redesigned the site, I mean a complete overhaul, design, architecture, asp to php for example. Resubmitted to google via webmaster with a new site map etc. Our website for reference is www.completeoffice.co.uk. We are trying to rank in UK google, www.google.co.uk, for the term office refurbishment kent (we are on page 2) and office refurbishment (we are on page 17). This is confusing me, because thats 7 pages lower than we were originally, before the site design, its better seo wise, duplicate content is none, title tage meta tags all correct, domain authority has doubled from 12 to 24 in a few months all in an effort to rank better. our homepage and office refurb pages both got an A on the on page SEOmoz tool test. So we dont understand why we are not ranking highly. It seems sites, much uglier and less seo friendly than our with same or lower domain authority are ranking at least 10 pages higher than us. Why is this? Can anyone help or give us a few suggestions. All help greatly appreciated. SEOmozzers.
Moz Pro | | CompleteOffice0 -
How to add a simple page to a campaing.
Hello, My domain is www.artes-plasticas-pollock.com. this domain must be positioned by one keyword, but, inside this domain there are more pages to be posicioned with another keywords. As example, inside the domain there is a page http://www.artes-plasticas-pollock.com/index-rafael-navarro.php that must be positioned by the keyword "Rafael Navarro" ¿ How can I configure it ? May I create a new campaing ? Is it possible to create this page inside the existing campaing related to the main url www.artes-plasticas-pollock ? Please ... any information will be pleased. thanks pilar.
Moz Pro | | OkTuWeb0 -
Crawl Issues
My website - qtmoving.com - has 26 articles and when the SEOmoz did a crawl it only found 13 articles. Can someone please give me some insight as to why not all pages are being crawled.
Moz Pro | | CohesiveMarketing0 -
What is the difference between the link count on OSE and Linkscape?
We notice a huge difference in the number of inbound links . Site explore shows 1,200 as an example and linkscape shows over 100,000 for the same site.
Moz Pro | | JamesBarry0
