Google cant read my robots.txt from past 10 days
-
http://awesomescreenshot.com/08d1s6aybc
hi, my robots.txt is
http://wallpaperzoo.com/robots.txt
google says it cant read and has postponed the crawl.. its been 10 days and no crawl.. please help me in solving this issue.. this is save with
-
Hi sir Google started fetching again
thank you for your help
-
What you can do is from within Google Webmaster Tools is to ask Google to fetch the URL of the sitemap and then 'submit it to the index'. That's one technique I've seen working before.
Google Webmaster Tools or Googlebot doesn't seem to crawl robots.txt files very often. Have seen it a couple of times where it could take days for them to crawl the file again after I've changed something.
-
hi sir, there was a problem on 23rd september in robots.txt which i fixed on 25th after then google bot dint come.. how much time it will take to come back again?
-
There's nothing wrong with your robots.txt file as far as I can see. It is likely that Google tried accessing the robots.txt file when your server was slow/down for a split second and that's why you are seeing the error. Are you sure that the site hasn't been crawled in 10 days? Have you checked your server log files to verify that? Regardless, once Googlebot does eventually come back, it should read the robots.txt file correctly and then you should be good to go.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Traffic cut-off since Google core update
Hi all, I am the webmaster of www.chepicap.com/en (Cryptocurrency news), and since the 3rd of june (Google core algorithm update) we got the hammer from Google. Organic traffic dropped with 90%+ overnight. We are still in the dark whether we can do to improve the current situation. Does someone have suggestions regarding this issue?
Algorithm Updates | | NielsDE0 -
Google inaccurate results: Common or error?
Hi all, While searching for our primary keyword, I can see 2 websites on second page results which are non-related to the keyword or industry but their company name is this keyword. Like if I want to rank and searching for "SEO", there are 2 websites which called "seo trucks" and "seo paints". I wonder how Google is ranking these websites for high competition keyword with 1 million searches per month. So the keyword in URL and this keyword mentioned across the website being their brand name taking over the other potential ranking factors like backlinks, relevant content, user clicks, etc..... Thanks
Algorithm Updates | | vtmoz0 -
Does Google’s Algorithm Populate Answer Boxes with Its Own Independent Research?
If you search 'best games to play for youtube' you get an answer box with answers pulled independently from the article at hand. Here's an image: http://imgur.com/a/S0j9B Here are all the games from my article, in the order in which they appear. Google's chosen games for Answer Box are bolded: Battlefield 1 Bloodborne GTA V FiFA 16 TrackMania Turbo Garry’s Mod League of Legends Call of Duty: Black Ops III Tom Clancy’s The Division Overwatch Just Cause 3 Counter-Strike: Global Offensive Brawlhalla Rocket League Dark Souls III Unravel Firewatch GoldenEye 007 (this was put in as a joke, but coded as an H2 nonetheless) Destiny Dead by Daylight Fallout 4 Undertale No Man’s Sky Minecraft As you can see, Google is choosing which games to display to its searchers. My Crazy Egg data shows that these were not picked by click volume (each of these H2s are hyperlinked), which means Google must be using some other popularity metric, such as its own search volume data or external sales data. I wrote this up in a post on my site, for anybody who's curious.
Algorithm Updates | | Edward_Sturm1 -
Using Google to find a discontinued product.
Hi Guys. I mostly use this forum for business questions, but now it's a personal one! I'm trying to find a supplier that might still have discontinued product. It's the Behritone C5A speaker monitor. All my searches bring up a plethora of pages that appear to sell the product... but they have no stock. (Wouldn't removing these pages make for a better internet?) No 2nd hand ones on eBay 😞 Do you have any suggestion about how I can get more relevant results... i.e find supplier that might still have stock? Any tips or trick I may be able to use to help me with this? Many thanks in advance to an awesome community 🙂 Isaac.
Algorithm Updates | | isaac6631 -
Best and easiest Google Depersonalization method
Hello, Moz hasn't written anything about depersonalization for years. This article has methods, but I don't know if they are valid anymore. What's an easy, effective way to depersonalize Google search these days? I would just log out of Google, but that shows different ranking results than Moz's rank tracker for one of our main keywords, so I don't know if that method is correct. Thanks
Algorithm Updates | | BobGW0 -
Same Meta description is being shown on Google?
Not sure why this is happening but when you this command into Google site:"mywebsite": + "key phrase" It brings up pages from my website which have the key phrase but I have noticed that Google is using the wrong meta description for all of them even though these pages all have their own unique meta description Does anyone know why this would be happening? Thanks
Algorithm Updates | | webguru20140 -
Google visits falling at the expense of Bing
Has anyone else noticed their percentage of search visits from Google slipping in the last few weeks at the expense of Bing? We've seen a 4% swing in the last month. Obviously Google is still the dominant presence (acconuting for 88.4% of all organic visits to our site kenwoodtravel.co.uk) but still it would be interesting to know if this is just a blip or more of a trend?
Algorithm Updates | | BrettCollins0 -
New Google "Knowledge Graph"
So according to CNN an hour ago regarding new Google update: "With Knowledge Graph, which will begin rolling out to some users immediately, results will be arranged according to categories with which the search term has been associated" http://www.cnn.com/2012/05/16/tech/web/google-search-knowledge-graph/index.html?hpt=hp_t3 Does this mean we need to start optimizing for Categories as well as Keywords?
Algorithm Updates | | JFritton0