Google cant read my robots.txt from past 10 days
-
http://awesomescreenshot.com/08d1s6aybc
hi, my robots.txt is
http://wallpaperzoo.com/robots.txt
google says it cant read and has postponed the crawl.. its been 10 days and no crawl.. please help me in solving this issue.. this is save with
-
Hi sir Google started fetching again
thank you for your help -
What you can do is from within Google Webmaster Tools is to ask Google to fetch the URL of the sitemap and then 'submit it to the index'. That's one technique I've seen working before.
Google Webmaster Tools or Googlebot doesn't seem to crawl robots.txt files very often. Have seen it a couple of times where it could take days for them to crawl the file again after I've changed something.
-
hi sir, there was a problem on 23rd september in robots.txt which i fixed on 25th after then google bot dint come.. how much time it will take to come back again?
-
There's nothing wrong with your robots.txt file as far as I can see. It is likely that Google tried accessing the robots.txt file when your server was slow/down for a split second and that's why you are seeing the error. Are you sure that the site hasn't been crawled in 10 days? Have you checked your server log files to verify that? Regardless, once Googlebot does eventually come back, it should read the robots.txt file correctly and then you should be good to go.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does Google considers the direct traffic on the pages with rel canonical tags?
Hi community, Let's say there is a duplicate page (A) pointing to original page (B) using rel canonical tag. Pagerank will be passed from Page A to B as the content is very similar and Google honours it hopefully. I wonder how Google treats the direct traffic on the duplicate Page A. We know that direct traffic is also an important ranking factor (correct me if I'm wrong). If the direct traffic is high on the duplicate page A, then how Google considers it? Will there be any score given to original page B? Thanks
Algorithm Updates | | vtmoz0 -
Google Algo Update
Hi I'm trying to look into how the update in Oct - Nov affected our site. It's positive with us ranking for more keywords, but I wondered if anyone knew what had changed. For example, one of our pages ranked for 291 keywords in October, now it ranks for 406 keywords. We've done nothing to this page in that time. I just want to try and understand what's happening with the algorithm a little more (although I'm sure everyone does :). Does anyone have any insights? Thanks!
Algorithm Updates | | BeckyKey0 -
Google traffic drops (October 24th 2014) - Penguin 3.0?
Hi All My client's site http://www.carismaautodesign.com took a bit of a knock around 24th October 2014. Google organic traffic down by ~32%. I am trying to understand if site has been hit by a Penguin refresh (3.0) - and if so why?
Algorithm Updates | | seowoody
If it was, it would appear to be a false negative as the site and backlink profile is clean. The content is perhaps the only area in question... as it's more of a brochure site, therefore content is relatively thin and promotional rather than in-depth/editorial. For example, the gallery pages are very similar in structure, with the images and specification text being the only variation, click on any of the vehicle interiors to see what I mean - http://www.carismaautodesign.com/gallery/mercedes/viano/ You will see the specification text is unique per vehicle interior, but not hugely; Then how could it be? The interiors all contain the same elements just a variation in seating, leather colour, stiching, wood finish etc. Question: Do you think... a.) This IS NOT a Penguin issue but something else (please ellaborate)
b.) This IS a Penguin issue but a false-negative, so do nothing, this will bounce back with next Penguin refresh
c.) This IS a Penguin issue related to content. Merge all gallery pages into one page per vehicle (i.e. one Mercedes Viano Interior page, with all 19 interior galleries as part of the page - thus building one stronger page with more intro text and simple bulleted specification per gallery)
d.) This IS a Penguin issue related to something else (please ellaborate) Thanks,0 -
New Google SERPs page title lengths, 60 characters?
It seems that the new Google SERPs have a shorter page title character length? From what I can gather they are 60 characters in length. Does this mean we all need to now optimise our page titles to 60 characters? Has anyone else noticed this and made any changes to page title lengths?
Algorithm Updates | | Adam_SEO_Learning0 -
Will Google discount new gTLDs?
I have the opportunity now to acquire a very desirable generic domain name with either the .org or .pro TLD. Obviously, the .pro version I can get for a far better price. The .org will probably end up being about 10x the price of the .pro. I feel like the .org would give me instant clout while the .pro might raise eyebrows. Also, I'm concerned that Google might also discount these new gTLDs. What do you guys think? Is the perceived authority of the old-time TLDs something worth investing in? Or will this fizzle away over time as the new gTLDs flood the market? THanks! Ira
Algorithm Updates | | iraweissman0 -
Best practice for cleaning up multiple Google Places listings and multiple Google accounts when logins were lost.
We are an inbound marketing agency, most of our clients are not relying on local seo. I have a pretty good understanding of it when starting fresh but not so much in joining a "movie in progress" kind of scenario. Recently we've brought on two clients who have had their websites in place for awhile, have made small attempts at marketing themselves online over the years and its resulted in multiple Google places listings, variations of the company names (one of them changed their name), worried there are yet more accounts out there they aren't aware of, etc (analytics, and others from well intentioned employees and past service providers - no internal leadership at the company level). In reading Google help forums I'm seeing some recently having their accounts suspended when they try to clean things up - in one case a person setup a new Google account thinking he would start fresh and in trying to claim listings, get rid of duplicates, etc. his account was suspended. What is the CURRENT recommended course of action in situations like these? With all the changes going on with Google, I don't know which route to take and have combed the Internet reading articles about this (including Google's resources) - would like some current real world advise.
Algorithm Updates | | rhgraves651 -
How can I tell Google two sites are non-competing?
We have two sites, both English language. One is a .ca and the other is a .com, I am worried that they are hurting one another in the search results. I'd like to obviously direct google.ca towards the .ca domain and .com towards the .com domain and let Google know they are connected sites, non-competing.
Algorithm Updates | | absoauto0 -
Profile Picture Display Next to Map on Google Search
On a recent Google search, I noticed that the top result also displayed the Google map of the location next to it but aside from that, it also displayed that business' Google+ Profile picture. I'm wondering how that is done. We have a Google+ account as well with an image associated with our business. At this point, the map doesn't even display for our business, but how does one get the map and the profile picture to appear on search results? Here's a link to the search result in question:
Algorithm Updates | | atuomala
http://www.google.com/search?q=tech+squad&rlz=1C1CHFX_enUS511US511&aq=f&oq=tech+squad&sourceid=chrome&ie=UTF-8#hl=en&rlz=1C1CHFX_enUS511US511&sclient=psy-ab&q=tech+squad+wi&oq=tech+squad+wi&gs_l=serp.3..0l2j0i30l2.6041.6590.0.6592.3.3.0.0.0.0.79.158.2.2.0.les%3B..0.0...1c.1.4.psy-ab.DoA-PSlyt-A&pbx=1&bav=on.2,or.r_gc.r_pw.r_qf.&bvm=bv.42768644,d.cGE&fp=b5d841bb1ab219dc&biw=1440&bih=785 tech_squad_serp.jpg0