Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Robots.txt allows wp-admin/admin-ajax.php

Technical SEO

1266

AndyKubrin last edited by
Hello, Mozzers!
I noticed something peculiar in the robots.txt used by one of my clients:
```
Allow: /wp-admin/admin-ajax.php
```
What would be the purpose of allowing a search engine to crawl this file?
Is it OK? Should I do something about it?
Everything else on /wp-admin/ is disallowed.
Thanks in advance for your help.
-AK:
1 Reply Last reply
Reply Quote 2

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Can I Block https URLs using Host directive in robots.txt?

Hello Moz Community, Recently, I have found that Google bots has started crawling HTTPs urls of my website which is increasing the number of duplicate pages at our website. Instead of creating a separate robots.txt file for https version of my website, can I use Host directive in the robots.txt to suggest Google bots which is the original version of the website. Host: http://www.example.com I was wondering if this method will work and suggest Google bots that HTTPs URLs are the mirror of this website. Thanks for all of the great responses! Regards,
Ramendra
Technical SEO | | TJC.co.uk

0
Accessibility / display none

Hello, Does anyone ever had a problem with display none portion of a page made for accessibility? (Jaws reader/ NVDA) Thank You.
Technical SEO | | Vale7

0
Robots.txt blocking Addon Domains

I have this site as my primary domain: http://www.libertyresourcedirectory.com/ I don't want to give spiders access to the site at all so I tried to do a simple Disallow: / in the robots.txt. As a test I tried to crawl it with Screaming Frog afterwards and it didn't do anything. (Excellent.) However, there's a problem. In GWT, I got an alert that Google couldn't crawl ANY of my sites because of robots.txt issues. Changing the robots.txt on my primary domain, changed it for ALL my addon domains. (Ex. http://ethanglover.biz/ ) From a directory point of view, this makes sense, from a spider point of view, it doesn't. As a solution, I changed the robots.txt file back and added a robots meta tag to the primary domain. (noindex, nofollow). But this doesn't seem to be having any effect. As I understand it, the robots.txt takes priority. How can I separate all this out to allow domains to have different rules? I've tried uploading a separate robots.txt to the addon domain folders, but it's completely ignored. Even going to ethanglover.biz/robots.txt gave me the primary domain version of the file. (SERIOUSLY! I've tested this 100 times in many ways.) Has anyone experienced this? Am I in the twilight zone? Any known fixes? Thanks. Proof I'm not crazy in attached video. robotstxt_addon_domain.mp4
Technical SEO | | eglove

0
Why is robots.txt blocking URL's in sitemap?

Hi Folks, Any ideas why Google Webmaster Tools is indicating that my robots.txt is blocking URL's linked in my sitemap.xml, when in fact it isn't? I have checked the current robots.txt declarations and they are fine and I've also tested it in the 'robots.txt Tester' tool, which indicates for the URL's it's suggesting are blocked in the sitemap, in fact work fine. Is this a temporary issue that will be resolved over a few days or should I be concerned. I have recently removed the declaration from the robots.txt that would have been blocking them and then uploaded a new updated sitemap.xml. I'm assuming this issue is due to some sort of crossover. Thanks Gaz
Technical SEO | | PurpleGriffon

0
Domain Mapping - International Config - WP Multisite - Problems/issues experienced ?

Hi My new clients devs are setting up new client according to below domain mapping type config Is anyone aware of any probs this could cause ? or other way should be setting up ? Also will setting up an MA campaign for the root domain 'catch all' given the domain mapping aspect etc ? Cheers Dan Set up now using WP Multisite. The root domain for the network of websites we are going to roll out, is set up as "network.domain.com. This is a "parent" domain from which all languages variants will be children of this root domain name. so: "network.domain.com/uk/" - English Language
"network.domain.com/tr/" - Turkish Langauge
"network.domain.com/za/" - South African
etc I then will domain map domain names to each of these children, once I get DNS control of each language's settings. I have already mapped "www.domain.com" to "network.domain.com/uk/", so the english website is set up (and launched - as you know). I fully expect that "www.domain.tr" will be mapped to "network.domain.com/tr/" and so on, but depending on domain availability at the time of purchase. Any domain name can be mapped to each of these children, and the system doesn't care, or mind! I can also map one (or more) domain names to each child, and make ONE of them the primary domain name.
Technical SEO | | Dan-Lawrence

0
Meta-robots Nofollow

I don't understand Meta-robots Nofollow. Wordpress has my homepage set to this according to SEOMoz tool. Is this really bad?
Technical SEO | | hopkinspat

1
Removing robots.txt on WordPress site problem

Hi..am a little confused since I ticked the box in WordPress to allow search engines to now crawl my site (previously asked for them not to) but Google webmaster tools is telling me I still have robots.txt blocking them so am unable to submit the sitemap. Checked source code and the robots instruction has gone so a little lost. Any ideas please?
Technical SEO | | Wallander

0
Microdata / Rich snippet

How to change "votes" to "reviews" in the search result while implementing microdata / rich snippet?
Technical SEO | | gmk1567

0