Google cant read my robots.txt from past 10 days
-
http://awesomescreenshot.com/08d1s6aybc
hi, my robots.txt is
http://wallpaperzoo.com/robots.txt
google says it cant read and has postponed the crawl.. its been 10 days and no crawl.. please help me in solving this issue.. this is save with
-
Hi sir Google started fetching again
thank you for your help
-
What you can do is from within Google Webmaster Tools is to ask Google to fetch the URL of the sitemap and then 'submit it to the index'. That's one technique I've seen working before.
Google Webmaster Tools or Googlebot doesn't seem to crawl robots.txt files very often. Have seen it a couple of times where it could take days for them to crawl the file again after I've changed something.
-
hi sir, there was a problem on 23rd september in robots.txt which i fixed on 25th after then google bot dint come.. how much time it will take to come back again?
-
There's nothing wrong with your robots.txt file as far as I can see. It is likely that Google tried accessing the robots.txt file when your server was slow/down for a split second and that's why you are seeing the error. Are you sure that the site hasn't been crawled in 10 days? Have you checked your server log files to verify that? Regardless, once Googlebot does eventually come back, it should read the robots.txt file correctly and then you should be good to go.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site appearing and disappearing from google serps.
Hi, My website is normally on page 2-3 on google consistently. Over the past month it has been appearing and then completely disappearing from the serps. One day it will be on page 2, then the next day completely missing from the serps. When i check the index it seems to be indexed correctly when doing site:mysite.com. I don't understand why this keeps happening, any experience with this issue? It doesn't seem to be a google dance as far as I can tell. When my other sites dance they typically just go up or down a few ranks for a couple weeks until they stabilize. Not completely fall off the search engine.
Algorithm Updates | | Chris_www0 -
Does Google use dateModified or date Published in its SERPs?
I was curious as to the prioritization of dateCreated / datePublished and dateModified in our microdata and how it affects google search results. I have read some entries online that say Google prioritizes dateModified in SERPs, but others that claim they prioritize datePublished or dateCreated. Do you know (or could you point me to some resources) as to whether Google uses dateModified or date Published in its SERPs? Thanks!
Algorithm Updates | | Parse.ly0 -
Googlebot soon to be executing javascript - Should I change my robots.txt?
This question came to mind as I was pursuing an unrelated issue and reviewing a site's robots/txt file. Currently this is a line item in the file: Disallow: https://* According to a recent post in the Google Webmasters Central Blog: [http://googlewebmastercentral.blogspot.com/2014/05/understanding-web-pages-better.html](http://googlewebmastercentral.blogspot.com/2014/05/understanding-web-pages-better.html "Understanding Web Pages Better") Googlebot is getting much closer to being able to properly render javascript. Pardon some ignorance on my part because I am not a developer, but wouldn't this require Googlebot be able to execute javascript? If so, I am concerned that disallowing Googlebot from the https:// versions of our pages could interfere with crawling and indexation because as soon as an end-user clicks the "checkout" button on our view cart page, everything on the site flips to https:// - If this were disallowed then would Googlebot stop crawling at that point and simply leave because all pages were now https:// ??? Or am I just waaayyyy over thinking it?...wouldn't be the first time! Thanks all! [](http://googlewebmastercentral.blogspot.com/2014/05/understanding-web-pages-better.html "Understanding Web Pages Better")
Algorithm Updates | | danatanseo0 -
Does Link Exchanges and Reciprocal Links Is Dead - Now Days ?
Hello, As we know randfish Rand discusses the egress of old link building practices and the ingress of new (old) link _earning _strategies, Rand has also discussed on Link Exchanges and Reciprocal Links, I have few questions which r related to Link Exchanges and Reciprocal Links. Few Question **1) Does Now Days Reciprocal Links Are Important Or Not For Link Building Strategies. ** ** 2) Webmaster Has To Perform Reciprocal Links Or Not.** 3) Can Reciprocal Links Boost Search Engine Ranking. 4) Does Reciprocal Links Has Negative Impact On Search Engine. Regards,
Algorithm Updates | | sumit60
Sumit0 -
When did Google include display results per page into their ranking algorithm?
It looks like the change took place approx. 1-2 weeks ago. Example: A search for "business credit cards" with search settings at "never show instant results" and "50 results per page", the SERP has a total of 5 different domains in the top 10 (4 domains have multiple results). With the slider set at "10 results per page", there are 9 different domains with only 1 having multiple results. I haven't seen any mention of this change, did I just miss it? Are they becoming that blatant about forcing as many page views as possible for the sake of serving more ads?
Algorithm Updates | | BrianCC0 -
Has there been a Google change in the last 24 hours?
We have come in this morning to find our site (paydayuk.co.uk) has suddenly disappeared from their SERPs, we have consistently been ranking in the top 5 for a wide range of search terms but now do not even appear for our brand name of Payday UK where we have been first for many months. Our site is still indexed and we have made no changes for a while as any SEO work is waiting on completion of a CMS system. Looking in https://groups.google.com/a/googleproductforums.com/forum/#!categories/webmasters/crawling-indexing--ranking and there seem to be a lot of people having the same issues but as of yet no answers. I'd also like to add we don’t use black hat techniques so we really don’t understand why we have been penalised. Can anyone help please?
Algorithm Updates | | Sarbs0 -
Problems with Google results
Hi Everybody, I ve been dealing with this issue for a while now. i have a multilingual website: www.vallnord.com When a search for Vallnord in Google it always shows the result in Catalan, but it does not show what I specified in the meta description, it displays what it crawls from the home page. I have 2 problems here: It is not showing my meta description. What can I do? It is not showing the language from which the search was made. Example: if you search from Google.com and your default language is english it should been displayed the result from the english HTML. www.vallnord.com/en but it is not like this. It is always the catalan (default language of the site) the one that is displayed. I have tried several things already: Inserting the Hreflang function Changing the descriptions Resubmitting the sitemap via Google Webmaster I can not figure out what is going on because if you search: "Vallnord Castellano" it will display the spanish URL but still not the proper description. Moreover if you search: "www.vallnord.com/es" on google , it will display the proper URL and description. FYI, I am using 301 redirects for the languages: es.vallnord.com it is the sames as www.vallnord.com/es In addition to this, If using Yahoo search engine there is no problem. it will show the proper language. from yahoo.com the first result is in english and from yahoo.es the first result Spanish. So any idea what would be the problem?And furthermore, any Idea which would be the solution? Thanks in advance, Guido.
Algorithm Updates | | SilbertAd0 -
Google results on an Ipad 2
Has anyone else seen different google organic results for a site when viewing on an Ipad compared to computer browser ? I've just checked a site and were no1 on google when searched on the Ipad 2 but when searched on my Macbook we are page 2 ? Could this just be different data centers or do google serve up different results to the 2 devices ? Would be really interested to know if anyone else has seen this. JP
Algorithm Updates | | Prongo0