What is the appropriate Robot.txt to unblock if Google cannot get all the resources from my homepage?
-
Hello everyone. I did some research as to why my website has decreased in the Google search rankings recently. After reading this Yoast article I believe it's because the robot.txt files I have set up on my wordpress website. The following is a screen shot of the results of a fetch & render query of my webpage.Googlebot couldn't get all resources for this page. Here's a list:
My current robot.txt settings are as follows;
User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: */xmlrpc.php Disallow: */wp-*.php Disallow: */trackback/ Disallow: *?wptheme= Disallow: *?comments= Disallow: *?replytocom Disallow: */comment-page- Disallow: *?s= Disallow: */wp-content/ Allow: */wp-content/uploads/ ```What to I need to allow/disallow to allow Google spiders to properly read my website?
-
Remove the following line:
Disallow: */wp-content/ And add: Allow: /wp-includes/js/
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Reviews - Google & Third Party
Hi We have reviews on our product pages & service reviews on Feefo, but how important is it to also drive customers to review your company on Google? I'm guessing we should be doing both, but it proves difficult when you already ask them to review your company through a third party? Any tips moz?
Algorithm Updates | | BeckyKey0 -
Anyone Notice Google's Latest Change Seems to Favor Google Books?
I've noticed a change in the search results lately. As I search around I notice a lot of results from books.google.com Seems a little (ok a lot) self serving... JMHO
Algorithm Updates | | get4it1 -
Ranking Well in Google But Not Bing - Why?
Hello Moz Community, I'm ranking well in Google (#2-#6 for various keywords) but on the second page of Bing. Are there certain differences that I should be aware of? Thanks, Cole
Algorithm Updates | | ColeLusby0 -
Is there a we to get Google to index our site quicker?
I have updated some pages on a website, is there a way to get Google to index the page quicker?
Algorithm Updates | | webguru20140 -
Does anyone have an idea of the benefits of Google Analytics Premium?
We've been having a discussion about the GA Premium service here in our office, trying to weigh up the pro's and con's... For the majority all it seems you gain access to is more support from google. We're trying to find out if that is the case or if you gain extra information, such as and insight into the search terms who must not be named. Of course i'm talking about the (Not Set) data... This section of data is ever increasing, yes i know we can access certain terms through webmasters but it was so much easier (in the good ol' days) when all the data was under one roof! Any thought opinions or even more questions would be greatly appreciated, i look forward to your responses. Anthony
Algorithm Updates | | Kal-SEO0 -
Images not getting indexed in google image search :( " site: hdwallpaperzones.com " )
hi as i have mentioned in title.. my website images are not getting indexed in google image search engine.. out of 360 images only 5 got indexed from 3 days.. please help me out.. thanks
Algorithm Updates | | toxicpls0 -
Did anyone else get a massive drop off in Bing recently?
Only for one set of my keywords. But I fell from first page to off the map in one fell swoop. Did this happen to anyone else? Should I be worried?
Algorithm Updates | | CleanEdisonInc0 -
How to do SEO for Google places.New trends and tips
How to do SEO for Google places.New trends and tips .Most clients wants their biz in Google places in First page .
Algorithm Updates | | innofidelity0