Google haveing problems accessing part of my site
-
hi my site is, www.in2town.co.uk and for a few weeks now google has had trouble accessing part of my site.
Today googlewebmaster tools tells me that google is having major problems it shows, 123 pages where access were denied.
i have spoken to my hosting company who could not find a problem, so not sure what to do now. can anyone please give me advice on what the problem may be.
any help would be great
-
That's correct. I don't know the specifics, however in this case, I would assume the reduced frequency/volume would mean less trouble dealing with the hosting provider's internal challenges.
-
i have read it time and time again and do not understand how it can help. if you reduce the crawl rate then that means you reduce the number of pages and the number of times google visits your site or am i wrong
-
Fascinating how some problems can come up like this - it's not anything I'd ever seen - curious to see whether reducing Google's crawl rate will help.
-
i have asked about this problem on the google webmaster forum
http://productforums.google.com/forum/#!mydiscussions/webmasters/BogC5OZqdyM
and people are saying that it is a crawler issue with my hosting company,
they are suggesting a number of things that could be wrong which includes
Many hosting providers need to have mechanisms to throttle/limit either the number of requests or bandwidth to their customers sites.
However in this instance, they are not allowing for search engine crawlers which typically make numerous requests in quick succession. Significantly, their server's response is technically incorrect (RFC 2616), it should either not respond, or respond with a 503 (service unavailable). Google is aware of such incorrect 403s and will eventually try to crawl them again.To mitigate the issue somewhat, you can reduce your site's crawl rate in Google's Webmaster Tools
-
This confirms the need to get an expert involved - one experienced in these types of issues - a programmer or systems administrator type expert, not necessarily an SEO expert.
-
the site has been built from a template but this problem has only started to happen in the past month and the site is a couple of years old
-
unfortunately I don't know exactly what else to suggest. Who built the site? Is it a custom site, or was it automatically built using a hosting company automated site builder?
If it's a custom site or you have a developer you've worked with to build it, speak with them. if it's a site automatically built using a hosting site building system, only they can help resolve it. If not, you'll need to get a different site built. The problem you are describing should not be happening.
-
hi, when i put in http://www.in2town.co.uk on the tool you told me about it brings up some pages that are fine and some that are 403, and the 14th one down is the url as above. that comes up with a 403 however, when i just put the url on its own, it comes up as a 200, which is strange.
Have you got any ideas on what i should say to the hosting company. i have spoken to them a number of times and all i get is, everything is fine but it is not
-
When I put that URL in, I get a "200" "OK" response. If you get a "403 forbidden, you need to speak with the site systems (server) administrator to find out if they can pinpoint the problem. Something is broken on the site at the server level to cause that. Even if it's only intermittently broken.
-
hi, thanks for this. i have used that tool and it has come up with as an example the following
http://www.in2town.co.uk/news/parenting/parents-need-to-act-against-child-obesity 403 forbidden
i am not sure why this has happened and this is the same result as i am being shown in google webmaster tools.
-
One step I would suggest is that I would suggest using a tool like Screaming Frog. It can crawl your site and report on problems or broken internal links on the site.
I would also go into Google Analytics and go into the Content/Speed/Page Speed data. I would then expand the date range to cover from before they started listing problems until the day before you run the report (the current day can lag in updating in Google Analytics).
From there, look to see if there are any specific days where Google shows the page speed average (in the timeline chart) as being unusually high. There may be days that are more problematic than others.
Finally, with that same date range in place, I would filter down the list of pages in the speed report down to show me one of the pages Google lists as being unreachable. Then do the same with some others. See if Google shows any odd page speed times on those.
I would also then go to URIValet.com and run a check on some of those pages as well to see if URIValet shows the pages reachable and returning a proper "200" "okay" header message.
This is not a guarantee fix, however it can help bring answers.
-
the site is www.in2town.co.uk www.in2town.co.uk can you try it with the link and without the link, not sure why you cannot get on the site but will be interested to find out.
this problem of google have access errors has been going on for some weeks now and i do not understand what is going on
-
Hi Claire,
I was able to view your robots.txt file but nothing else. I can't seem to get to the site. Is the URL correct?
Dana
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Removed Subdomain Sites Still in Google Index
Hey guys, I've got kind of a strange situation going on and I can't seem to find it addressed anywhere. I have a site that at one point had several development sites set up at subdomains. Those sites have since launched on their own domains, but the subdomain sites are still showing up in the Google index. However, if you look at the cached version of pages on these non-existent subdomains, it lists the NEW url, not the dev one in the little blurb that says "This is Google's cached version of www.correcturl.com." Clearly Google recognizes that the content resides at the new location, so how come the old pages are still in the index? Attempting to visit one of them gives a "Server Not Found" error, so they are definitely gone. This is happening to a couple of sites, one that was launched over a year ago so it doesn't appear to be a "wait and see" solution. Any suggestions would be a huge help. Thanks!!
Technical SEO | | SarahLK0 -
Does my "spam" site affect my other sites on the same IP?
I have a link directory called Liberty Resource Directory. It's the main site on my dedicated IP, all my other sites are Addon domains on top of it. While exploring the new MOZ spam ranking I saw that LRD (Liberty Resource Directory) has a spam score of 9/17 and that Google penalizes 71% of sites with a similar score. Fair enough, thin content, bunch of follow links (there's over 2,000 links by now), no problem. That site isn't for Google, it's for me. Question, does that site (and linking to my own sites on it) negatively affect my other sites on the same IP? If so, by how much? Does a simple noindex fix that potential issues? Bonus: How does one go about going through hundreds of pages with thousands of links, built with raw, plain text HTML to change things to nofollow? =/
Technical SEO | | eglove0 -
My sites "pages indexed by Google" have gone up more than qten-fold.
Prior to doing a little work cleaning up broken links and keyword stuffing Google only indexed 23/333 pages. I realize it may not be because of the work but now we have around 300/333. My question is is this a big deal? cheers,
Technical SEO | | Billboard20120 -
What can i do to get google to visit my site more often
Hi, i am having serious problems since i upgraded my website from joomla 1.5 to 3.0 We have dropped down the rankings from page one for the word lifestyle magazine, and we have dropped down in rankings for other very important words including gastric band hypnotherapy and i am starting to regret having the site upgraded. i am finding the google is taking its time visiting my site, i know this for two reasons, one i have checked the cache and it is showing the 2nd july and i have checked articles that we have written and they are still not showing. example if i put this article name in word for word it does not come up, Carnival Divert Ships In The Caribbean Due To bad Weather this was an article that was done yesterday. in the old days before the upgrade that would have been in google now. these problems are costing us the loss of a great deal of traffic, we are losing around 70% of our traffic since the upgrade and would be grateful if people could give me advice on how to turn things around. we add articles all the time. each day we add a number of articles, i was considering changing the front page in the middle and having a few paragraphs of the latest story to get google to visit more often. i know this would look messy but i am running out of ideas. any help would be great
Technical SEO | | ClaireH-1848860 -
Any one worked on a sites.google.com/ website ?
Hi I have a client that has a sites.google.com/ website, Has anyone ever used one ? or had to do SEO on one ? Any help would be very much appreciated Thanks
Technical SEO | | tempowebdesign0 -
Google has not been visiting my site
Hi I am working on a site at the moment http://www.cheapflightsgatwick.com and i had the site using a different template and in the search engines for the search term cheap flights gatwick we were fourth and for the term holiday magazine we were 12th in google but now we are not even in google on the first page for the search terms. But now after changing the template in joomla our rankings have gone out of the window. It took me about a day to sort out the site with the new template so i was not expecting any problems with the search engines but for some reason there is. If you put into the search engine www.cheapflightsgatwick.com then you will see that google has not visited the site for four days and also it is not showing the description and instead it is showing details about joomla. Can anyone let me know if there is anything i need to do to sort this out and why google is taking so long to visit my site
Technical SEO | | ClaireH-1848860 -
Site 'filtered' by Google in early July.... and still filtered!
Hi, Our site got demoted by Google all of a sudden back in early July. You can view the site here: http://alturl.com/4pfrj and you may read the discussions I posted in Google's forums here: http://www.google.com/support/forum/p/Webmasters/thread?tid=6e8f9aab7e384d88&hl=en http://www.google.com/support/forum/p/Webmasters/thread?tid=276dc6687317641b&hl=en Those discussions chronicle what happened, and what we've done since. I don't want to make this a long post by retyping it all here, hence the links. However, we've made various changes (as detailed), such as getting rid of duplicate content (use of noindex on various pages etc), and ensuring there is no hidden text (we made an unintentional blunder there through use of a 3rd party control which used CSS hidden text to store certain data). We have also filed reconsideration requests with Google and been told that no manual penalty has been applied. So the problem is down to algorithmic filters which are being applied. So... my reason for posting here is simply to see if anyone here can help us discover if there is anything we have missed? I'd hope that we've addressed the main issues and that eventually our Google ranking will recover (ie. filter removed.... it isn't that we 'rank' poorly, but that a filter is bumping us down, to, for example, page 50).... but after three months it sure is taking a while! It appears that a 30 day penalty was originally applied, as our ranking recovered in early August. But a few days later it dived down again (so presumably Google analysed the site again, found a problem and applied another penalty/filter). I'd hope that might have been 30 or 60 days, but 60 days have now passed.... so perhaps we have a 90 day penalty now. OR.... perhaps there is no time frame this time, simply the need to 'fix' whatever is constantly triggering the filter (that said, I 'feel' like a time frame is there, especially given what happened after 30 days). Of course the other aspect that can always be worked on (and oft-mentioned) is the need for more and more original content. However, we've done a lot to increase this and think our Guide pages are pretty useful now. I've looked at many competitive sites which list in Google and they really don't offer anything more than we do..... so if that is the issue it sure is puzzling if we're filtered and they aren't. Anyway, I'm getting wordy now, so I'll pause. I'm just asking if anyone would like to have a quick look at the site and see what they can deduce? We have of course run it through SEOMoz's tools and made use of the suggestions. Our target pages generally rate as an A for SEO in the reports. Thanks!
Technical SEO | | Go2Holidays0 -
Google +1 Button on Flash sites
One of my customers is willing to add Google +1 button on their Flash websites. Is it possible? How can we add Google +1 button on a Flash site? Thanks in advance!
Technical SEO | | merkal20050