"Does not respond to web requests" error
-
When trying to set up a new campaign I get the following message:
"Roger has detected a problem: We have detected that the domain www.chicagofinancialadvisers.com does not respond to web requests. Using this domain, we will be unable to crawl your site or present accurate SERP information."Can someone please tell me what I need to do on my site to make this work? I haven't seen this before and have done many other campaigns. Thanks a lot!
-
Thanks Ryan. That worked!
-
Hello Brien,
I noticed your robots.txt file currently shows as
**User-agent: ***
That is not a properly formatted robots.txt file. You need to add an additional line. Try adjusting as follows:
**User-agent: ***
** Disallow:**
This change MAY resolve the issue. If not, I would suggest checking your server firewall settings to discover if any ports or crawlers are blocked.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Crawler errors or Page Load Time? What affect more to SEO
Hello, I have a page with a forum and at this moment the moz report says that have 15.1k of issues like url too long, meta noindex, title too long etc. But this page have a load time realy sloooow with 11 seconds. I know i need fix all that errors (i'm working on this) but... What is more important for SEO? The page load or that type of error like duplicate titles etc. Thank you!
Moz Pro | | DanielExposito1 -
503 Error or 200 OK??
So, in a Moz crawl and a Screaming From crawl, I'm getting some 503 Service Unavailable responses on the some pages. So I go to the pages in question, and the Moz bar is showing a 200 OK. The SEOBook http status checker (http://tools.seobook.com/server-header-checker/) also shows a 200 OK. What gives? The only reason I'm looking at this is because rankings plummeted a couple of weeks ago. Thanks! UPDATE So, I decided to use the mozbar to set the user agent as Googlebot and when I tried to access the pages in question I receive this message. I don't think this is an issue... anyone else have much experience here? Your access to this site has been limited Your access to this service has been temporarily limited. Please try again in a few minutes. (HTTP response code 503) Reason: Fake Google crawler automatically blocked Important note for site admins: If you are the administrator of this website note that your access has been limited because you broke one of the Wordfence firewall rules. The reason you access was limited is: "Fake Google crawler automatically blocked". If this is a false positive, meaning that your access to your own site has been limited incorrectly, then you will need to regain access to your site, go to the Wordfence "options" page, go to the section for Firewall Rules and disable the rule that caused you to be blocked. For example, if you were blocked because it was detected that you are a fake Google crawler, then disable the rule that blocks fake google crawlers. Or if you were blocked because you were accessing your site too quickly, then increase the number of accesses allowed per minute. If you're still having trouble, then simply disable the Wordfence firewall and you will still benefit from the other security features that Wordfence provides. If you are a site administrator and have been accidentally locked out, please enter your email in the box below and click "Send". If the email address you enter belongs to a known site administrator or someone set to receive Wordfence alerts, we will send you an email to help you regain access. Please read our FAQ if this does not work.
Moz Pro | | wiredseo0 -
Rogerbot does not catch all existing 4XX Errors
Hi I experienced that Rogerbot after a new Crawl presents me new 4XX Errors, so why doesn't he tell me all at once? I have a small static site and had 9 crawls ago 10 4XX Errors, so I tried to fix them all.
Moz Pro | | inlinear
The next crawl Rogerbot fount still 5 Errors so I thought that I did not fix them all... but this happened now many times so that I checked before the latest crawl if I really fixed all the errors 101%. Today, although I really corrected 5 Errors, Rogerbot digs out 2 "new" Errors. So does Rogerbot not catch all the errors that have been on my site many weeks before? Pls see the screenshot how I was chasing the errors 😉 404.png0 -
I need to get a page in the top 3 Google results for my keyword "teaching jobs" but am struggling to do so! Can anyone help?
I'm trying to get this page http://www.eteach.com/teaching-jobs to rank as the top search result on Google with the keyword "teaching jobs" but it seems to be number 5 in the results! My competitors are totally kicking my arse on getting this page to be above my website. I've got the keywords in there, I have the right content and I have links, what more can I do to make it rank as number 1! Help please!! If anyone has an SEO check list of things I need to make sure I do on my pages for them to rank in the top 3 results then that would be really handy!
Moz Pro | | Eteach_Marketing0 -
"Issue: Duplicate Page Content " in Crawl Diagnostics - but sample pages are not related to page indicated with duplicate content
In the crawl diagnostics for my campaign, the duplicate content warnings have been increasing, but when I look at the sample pages that SEOMoz says have duplicate content, they are completely different pages from the page identified. They have different Titles, Meta Descriptions and HTML content and often are different types of pages, i.e. product page appearing as having duplicate content vs. a category page. Anyone know what could be causing this?
Moz Pro | | EBCeller0 -
What percentage of total backlinks are shown by "Open site explorer" tool?
What percentage of total good quality back links are shown in the "Open site" or other toolbar. I was checking backlinks for a spammy sites which ranks in #3-5 for a competitive keyword.But could not find nothing more than 10-15 PR4-5 in content backlinks and rest are just forum/profile signatures. 24% Trust/Authority of the Host Domain
Moz Pro | | RyanSat
22% Link Popularity of the Specific Page
20% Anchor Text of External Links
15% On-Page Keyword Usage
7% Traffic and Click-Through Data
6% Social Graph Metrics
5% Registration and Hosting DataI0 -
For the "On Page Report Card", Can I see a list of all of the recommendations?
On the "On Page Repord Card", in the "Page Analysis Detail" section, I can click "Read More" for each factor and see a "Recommendation" on how to fix the page to meet that factors criteria. Are these recommendations the same everytime or will they vary depending on the page itself? Can get a copy of each recommendation that would appear if none of the criteria were met? Thanks, Daniel.
Moz Pro | | iSenseWebSolutions1 -
Why is blocking the SEOmoz crawler considered a red "error?"
Why is blocking the SEOmoz crawler considered a red "error?" Please see attached image... Y3Vay.png
Moz Pro | | vkernel0