"Does not respond to web requests" error
-
When trying to set up a new campaign I get the following message:
"Roger has detected a problem: We have detected that the domain www.chicagofinancialadvisers.com does not respond to web requests. Using this domain, we will be unable to crawl your site or present accurate SERP information."Can someone please tell me what I need to do on my site to make this work? I haven't seen this before and have done many other campaigns. Thanks a lot!
-
Thanks Ryan. That worked!
-
Hello Brien,
I noticed your robots.txt file currently shows as
**User-agent: ***
That is not a properly formatted robots.txt file. You need to add an additional line. Try adjusting as follows:
**User-agent: ***
** Disallow:**
This change MAY resolve the issue. If not, I would suggest checking your server firewall settings to discover if any ports or crawlers are blocked.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Web Stie STUCK on page 2 for 2years
Our website is a dating site for adults (not porn) (http://www.swingersocial.com) It’s been STUCK in the middle of page 2 for 1 ½ years. Moving up or down 2 ranking, that’s it. The way we determine our location is by entering the most common phrases used by people who search for our type of site. We surveyed 50+ people and the phrases they most frequently entered when using Google were “swingers websites” “swingers sites” and “free swingers sites”. In addition to our competitors ranking above us, they show “movement” in their rankings up & or down. Some even dropped to page 2 but they recovered – we are not seeing the same movement or recovery. Pages crawled on have remained virtually unchanged for months. The last 1 ½ years we have hired SEO companies, and as a result a great deal of effort has been put into the site. We are not naïve and realize some were better than others. We’ve invested a great deal of effort into content, key words, modifying tags, and throughout all of this we’ve seen no change. We had a detailed audit suggesting improvements (no surprise), the report overall was positive stating “There are a few things to do to optimize the site but looking at the keywords, SwingerSocial.com should be able to rank high.” We also implemented a blog area with over 70 blog posts. Our competitors on the other hand have only two blog posts and are continually ranking higher than us. Additionally, our competitors do not have sites on Pinterest, G+, FB, or Twitter – we have a presence on some of these. I do know that the number and quality of links are terrible. I’m looking for a partner to help us make strategic changes to the site that will allow us to rank higher.
Moz Pro | | sailor35700 -
Help - Analyzing Web Traffic Across Multiple Websites
Hi Moz Community, Hope you can help. Is there any way to discover the most visited pages for a particular website, one that I do not administer? I wouldn't need exact numbers, just a relative breakdown of the "Most Visited" pages/sections. For example, if I was reviewing www.jcrew.com, I'd be interested in determining the 10/20/50 most visited pages/products. And just to provide another example, I would be interested in the 10/20/50 most visited pages/stories on www.buzzfeed.com. Any and all help is greatly appreciated. Thank you!
Moz Pro | | MountArashi0 -
Can someone explain the attached on page error message?
Two weeks ago, I took down our mobile site and switched to one on dudamobile. However, for some reason, I keep getting this error/improvement message on my on page report. I do not see how I can have a cannocial problem with a page that is no longer there. Can someone explain that and what I can do about it? Thanks! eQTNRD6.png
Moz Pro | | KempRugeLawGroup0 -
Seo moz has only crawled 2 pages of my site. Ive been notified of a 403 error and need an answer as to why my pages are not being crawled?
SEO Moz has only crawled 2 pages of my clients site. I have noticed the following. A 403 error message screaming frog also cannot crawl the site but IIS can. Due to the lack of crawling ability, im getting no feed back on my on page optimization rankings or crawl diagnostics summary, so my competitive analysis and optimization is suffering Anybody have any idea as to what needs to be done to rectify this issue as access to the coding or cms platform is out of my hands. Thank you
Moz Pro | | nitro-digital0 -
Has any on else experienced a spike in crawl errors?
Hi, Since the last time our sites were crawled in SEOmoz they are all showing a spike in Errors. (Mainly duplicate page titles and duplicate content). We haven't changed anything to the structure of the sites but they are all using the same content management system. The image is an example of what we are witnessing for all our sites based on the same system. Is anyone else experiencing anything similar? or does anyone know of any changes that SEOmoz has implemented which may be affecting this? Thanks in advance, Anthony. WzdQV WzdQV WzdQV.jpg WzdQV.jpg
Moz Pro | | BallyhooLtd1 -
Tool Request - What keywords does a site rank for?
Hi folks, Something I've never had to do before so I'm not sure which tool to use, but is there a way to determine the keywords that a website currently ranks for? Hope someone can assist 🙂
Moz Pro | | ChristopherM1 -
What does this error mean when starting a new campaign
hi just about to start a new campaign to i can monitor my website and i get this message up but not sure what this means. Roger has detected a problem: We have detected that the domain www.headlineplus.com and the domain headlineplus.com both respond to web requests and do not redirect. Having two "twin" domains that both resolve forces them to battle for SERP positions, making your SEO efforts less effective. We suggest redirecting one, then entering the other here. can anyone please let me know why this message comes up please
Moz Pro | | ClaireH-1848860 -
Campaign 4XX error gives duplicate page URL
I ran the report for my site and had many more 4xx errors than I've had in the past month. I updated my .htaccess to include 301 statements based on Google Webmaster Tools Crawl Errors. Google has been reporting a positive downward trend in my errors, but my SEOmoz campaign has shown a dramatic increase in the 4xx pages. Here is an example of an 4xx URL page: http://www.maximphotostudio.net/engagements/266/inniswood_park_engagements/http:%2F%2Fwww.maximphotostudio.net%2Fengagements%2F266%2Finniswood_park_engagements%2F This is strange because URL: http://www.maximphotostudio.net/engagements/266/inniswood_park_engagements/ is valid and works great, but then there is a duplicate entry with %2F representing forward slashes and 2 http statements in each link. What is the reason for this?
Moz Pro | | maximphotostudio1