'Domain Does Not Respond To Web Requests'
-
Hi everyone,
This seems to be a fairly common query on the Q&A section, but I haven't been able to find a solution by reading through previous threads.
When I try to set up an SEOmoz campaign for spryz.co.nz, I get that ol' favourite error message:
'We have detected that the domain spryz.co.nz does not respond to web requests. Using this domain, we will be unable to crawl your site or present accurate SERP information.'
The problem isn't being caused by a Robots.txt file, and the site has experienced 99% uptime since it was launched.Traffic stats show that visits are coming through to the site via the search engines, suggesting that it may not be all crawlers that fail to access the site.
I've tried to set up this campaign several times throughout the day, since I've read that sometimes Roger goes on the blink, but I've still not been successful.
Any suggestions as to why Roger might be unable to crawl my website would be great.
-
Having the same issue - would prefer not to post the domain name though. It's my first time using SEOMoz, new PRO user, and I'm literally blocked from my first campaign. The site is responsive, hosted on a great server, there's a robots.txt that's intact and is only disallowing access to one particular directory, and the site is currently indexed in Google because I can see the pages.
Methinks there's something wrong on your end - this question has 120 views which means it's more than a few isolated cases. Should I just go ahead and set up the campaign anyway, ignoring the error?
-
Hi there,
Thanks for reaching out! That is interesting indeed that you aren't able to create a campaign for that particular domain. I do notice that I had trouble at first, but I tried again immediately and was able to go past the no response screen and create a campaign. http://screencast.com/t/H2apiZb0tvB. I have checked with our team and we are not seeing an issue with the crawler so far, however in the past our crawler have had trouble with pages that is missing a Robots.txt (404 when I checked it last) http://screencast.com/t/oacVSiHM3Nyf. However as I have shown, our crawler will now crawl sites in the absence of a robot.txt.
There's a few reason why it could be not responding to our web request:
1. The uptime was sporadic when we made the attempt.
2. Something on the server level is blocking our web service, which is provided by Amazon.Unfortunately the problem is too sporadic, I will start a ticket for you on my end and let's see how my campaign comes back before we decide anything. Hope that helps!
Best,
Peter SEOmoz Help Team.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What is 'domain authority'?
IN seomoz, it mentions domain authority but it doesnt define it. what does domain authority mean?
Getting Started | | torbett0 -
Trying to connect Google Analytics account But it's not listed in the drop down box
I'm trying to connect a Google Analytics account but when I go through the drop down box with our list of clients it doesn't show up. It is listed in our agency Analytics account. We have 30+ accounts on GA. Any reason why the account wouldn't make it into the list? Thanks, ~Anjolie
Getting Started | | RKD_Group0 -
How to authenticate Moz crawler so that others don't use Rogerbot useragent to scrape data from our site?
Is there any way to authenticate genuine Moz crawler. Because, our website keeps getting scrapping attacks and if there is no way to authenticate Moz crawler, then, any scraper can just set user agent as Rogerbot and scrape all our pages. Is there a fixed IP that can be used or any other customization that will help us authenticate and allow only Moz crawler to crawl our site. Looking forward to a solution to this problem. We haven't been able to use Moz crawler due to this issue.
Getting Started | | longclimber0 -
Hi, I'm looking to find out why a google+ account that was rarely used has 10,000 views. I want to discover what sites it is linked to. I entered the page url but no joy. can anyone help?
I would like to find out where all this traffic is coming from. It is most likely from an out of date sales site etc, but it's important to find out as it could be the result of hacking etc. It appears the page is linked to another site and I would like to find out which one(s) Entering the page url is not getting results, can anyone help?
Getting Started | | cyganswenia0 -
Moz Analytics - How can I turn a whole report to 'Monthly'?
Hello, After spending 3 days setting up 14 clients on this system, by this, I mean I went through each client and re-made the same report 14 times as there is no alternative..I have noticed on my reports that: 'Dashboard' is set to 'Weekly' Social is set to 'Daily' Branding is set to 'Monthly' First questions, when I try to run a report, I go to Add Module and change 'Weekly' to 'Monthly', but when I press next, it changes itself back again, am I doing something wrong? UPDATE: I have found that I have go back in the reports, change the heading to 'Monthly' and then re-add this section again. How can I just run a whole report with monthly values? Surely, I do not have to re-do the entire report AGAIN, just to get a different value? I am mortified at how unfinished this product is, if this is the case.
Getting Started | | Paul_Tovey0 -
Moz's official stance on Subdomain vs Subfolder - does it need updating?
Hi, I am drawing your attention to Moz's Domain basics here: http://moz.com/learn/seo/domain It reads: "Since search engines keep different metrics for domains than they do subdomains, it is recommended that webmasters place link-worthy content like blogs in subfolders rather than subdomains. (i.e. www.example.com/blog/ rather than blog.example.com) The notable exceptions to this are language-specific websites. (i.e., en.example.com for the English version of the website)." I am wondering if this is still Moz's current recommendation on the subfolders vs subdomains debate, given that the above (sort of) implies that SE's may not combine ranking factors to the domain as a whole if subdomains are used - which (sort of) contradicts Matt Cutts last video on the matter ( http://www.youtube.com/watch?v=_MswMYk05tk ) which implies that this is not the case and there is so little difference that their recommendation is to use whatever is easiest. It would also seem to me that if you were looking through the eyes of Google, it would be silly to treat them differently if there were no difference at all other than subdomain vs subfolder as one of the main reasons a user would use a sud-domain is a technical on for which it would not make sense for Google to treat differently in terms of its algorithm. I notice that in terms of Moz, while most of the site uses subfolders, you do have http://devblog.moz.com/ - and I was wondering if this is due to a technical reason or conscious decision, as it would seem to me that the content within this section is indeed linkworthy (as it has external links pointing to it from external sources), therefore it would seem to not be following the initial advice that is posted in Moz's basics on domains. Therefore I am assuming it is due to a technical reason - or that Moz's adive is out of date with current Moz thinking, and is indeed in line with Matt C in that it doesn't matter. Cheers
Getting Started | | James773 -
I can't export reports in pdf
Hi,I try to export pdf report for my client but this function doesn't work. I tried in different browsers and on different machines - still doesn't work. What am I doing wrong? Thanks in advance,JJ
Getting Started | | jjtech0 -
I'm new to Moz, where can I find something that explains the basics of everything?
For instance how do I use the Rank Tracker or Keyword Tool. I want to get the most out of Moz and I need a resource that explains it. Thanks
Getting Started | | AliciaMarie1