Website Server Issue?
-
I'm getting error messages that a website cannot be crawled and it might be due to the following issues:
Couldn't access the webpage because the server either timed out or refused/closed the connection before our crawler could receive a response. How to fix: Please contact your web hosting technical support team and ask them to fix the issue
Could Possibly Be:
1. DDoS protection system.
OR
2. Overloaded or misconfigured serverThey asked me to talk to my hosting company about this issue and he's at a loss (I don't think he knows everything he needs to know potentially). Have you seen these issues before? Where is the best spot to start troubleshooting this issue?
-
Hi,
Try using any other crawler to see if they provide you with the same message (ScreamingFrog), in some cases it could mean that the site is overloaded and will start blocking the crawler. We're doing something similar for ours as we have a lot of scrapers but exclude the 'good' sites.
Martijn.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Weird Layout on Initial Website Load?
Whenever I open my site from an uncached source, like google incognito, for a split second it displays purple links and a white background while it loads the rest of the content. I've included a screenshot. Is there any way to fix that.? The site is www.kemprugegreen.com. u8P9q
Web Design | | KempRugeLawGroup1 -
Website organic traffic unchanged, impressions took a 98% drop in the last week.
Hi all, I have a very curious predicament and I'd be grateful if someone could shed some light on the situation. As mentioned in the title, organic traffic to our website has remained unchanged, but organic impressions have taken a 98% drop in the last week. This happened suddenly over one day; on October 22, impressions were 700, on October 23, they were 500, and on October 24 they drastically dropped to 50. The next two days they were at 22 and then up to 35. Organic traffic, however, showed the normal "weekend drop" as of October 24, and is still showing normal level (even increased a bit) continuing into this week. These are organic impressions according to Google Analytics and Google Webmaster tools. We did perform a complete site redesign a month ago. Could this be an effect from the redesign? We also noticed drop in Domain Authority, but our competitors suffered a similar (if not greater) drop as well, so we wondered if it could be due in part to the algorithm update. If anyone could shed some light on the situation I would be so appreciative! Thanks!
Web Design | | Joanne_Pendon0 -
Managing website content/keywords for wordpress site
We are in the midst of redesigning our website and have been working with freelance blog/content writers to increase the unique content on our site. We are finding it increasingly difficult to manage the topics/keywords as we continue to expand. Googledrive and google spreadsheets have been our primary tools thus far. Can anyone recommend a good tool that would allow us to manage content and blog posts for our site?
Web Design | | Tom_Carc0 -
SEO Consulting for HUGE Website. How Big Is TOO Big Of A Change?
SEO Consulting for a HUGE Website. Their h1 tags have instagram/twitter, h2 have their menu/what's trending and h3 is the article title. Here's what I want to do... MY MAIN QUESTION: This site has tens of thousands of pages, all articles beyond the few dozen category/tag pages they have. If I change the structure to the following, will it be too much of a system shock to Google? Will this actually HURT them? Currently on the site: - h1 tags point to Twitter/Instagram sidebar widgets
Web Design | | BrianGilmore
h2 tags point to the menu/what’s trending section (which is the same on every page)
h3 points to the Title of the Article I want to change it to this: - h1 tags should delineate the article's name. That's all they should really be used for.
h2-4 should be reserved for article subheadings to be used by the editorial staff. EDIT: 30% of their >11 million monthly uniques come from search. I don't want to eff with that, but the way that NONE of their pages have optimized words, they have no sitemap, webmaster tools and are still doing this well makes me think that even putting in minimal changes to tidy things up will help them bring it to 70% organic search.0 -
Website Design Structure
When having a website designed, is it best to have a Responsive Design with Media Queries or is it best to have a Website designed for laptops andDesktops and a second mobil website? What are the SEO benefits of each strategy and Which is best for the user experience?
Web Design | | bronxpad1 -
Websites with only one "html file" and page href # is good for SEO?
I bought one website from templatemonster that contains only one HTML and the pages are generated by links (PROGRAMACAO) My website: www.nextformaturas.com.br This is good in term of SEO? or it is better an website with deveral pages with diferent contents? What are the pros and cons? I really lost on this.
Web Design | | Naghirniac0 -
Website using javascript to serve up content - SEO Friendly?
I'm checking out a dentist website http://www.sagedentalnj.com/ I was referred by a friend so just taking a little peek at it. When you click on the menu items, the url at the top doesn't change. When you view source, the page titles are all the same. when I do site:http://www.sagedentalnj.com/ none of his pages are indexed by google. What can be done with his site so that google sees his pages? Maybe submit sitemap?
Web Design | | Czubmeister0 -
How to make AJAX/javascript website more seo friendly?
I have a website that is heavy on AJAX. I need recommendations on how to add content as well as other on page ompitizations. The website is a luxury brand for 6 resorts, each with their own subfolder. The website is http://me.graficode.com/preprod/.
Web Design | | Melia0