What are your thoughts on security of placing CMS-related folders in a robots.txt file?
-
So I was just about to add a whole heap of CMS-related folders to my robots.txt file to exclude them from search, and thought "hey, I'm publicly telling people where my admin folders are"...surely that's not right?!
Should I leave them out of the robots.txt file, and hope for the best that they never get indexed? Should I use noindex meta data on every page?
What are people's thoughts?
Thanks,
James
PS. I know this is similar to lots of other discussions around meta noindex vs. robots.txt, but I'm after specific thoughts around the security aspect of listing your admin folders in a robots.txt file...
-
surly your admin folders are secured?, it would not matter if someone knows where they are.
-
As a rule, you want to avoid using robots.txt files whenever possible. It does not consistently protect you from crawlers and when it does block crawlers it kills any PR on those pages.
If you can block those pages with a noindex tag, it would be a preferable solution.
With respect to security for a CMS site, it really needs to be a comprehensive effort. Many site owners take a couple steps and then have a false-sense of security. Here are a few thoughts:
-
try the site address with /administrator after it to access Joomla and other sites
-
try the site address or blog with /wp-admin/ after it to access Joomla sites
-
make up a webpage and try accessing it to view the site's 404 page
-
right-click on a page and choose View Page Source. Often you will see the name of the CMS clearly listed. Other times you will see clear clues such as /wp/ in folder names. Other times you will find unique extensions such as Yoast SEO which will give you an idea of the CMS
Once a bad guy knows which CMS is in use, they know the default folder structure and more. The point is it requires a lot more effort then most people realize to hide the CMS in use. I applaud your effort, but be very thorough about it. There is a lot more involved then simply covering your robots.txt file.
-
-
I found three options for you: http://www.techiecorner.com/106/how-to-disable-directory-browsing-using-htaccess-apache-web-server/
I think if you do it with.htacces that is a folder specific file than nobody will be able to detect where admin contet is located.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Temporary redirect from 302 to 301 for PNG File?
#302HTTP #temporaryredirect
Technical SEO | | Damian_Ed 0
Hi everyone, Recently I have faced a crawl issue with my media images on website. For example this page url https://intreface.com/wp-content/uploads/2022/12/Horion-screen-side-2.png has 302 HTTP Status and the recommendation is to change it 301. I have read the article on temporary redirections here:
https://moz.com/learn/seo/redirection?_ga=2.45324708.1293586627.1702571936-916254120.1702571936
but its not written here how to redirect in my HTML 1 image url not the landing page.
Screenshot 2023-12-15 at 11.02.40.png
I have messaged to MOZ Support but they recommended to go for the MOZ Community!
Screenshot 2023-12-15 at 11.06.02.png Could you assist me wit this issue please? I can reach HTTML of the necessary page and change what I need for permanent redirection but firstly I need to understand how to do that correctly.0 -
Utilizing one robots.txt for two sites
I have two sites that are facilitated hosting in similar CMS. Maybe than having two separate robots.txt records (one for every space), my web office has made one which records the sitemaps for the two sites, similar to this:
Technical SEO | | eulabrant0 -
Payday Loan or Panda? Any thoughts?
So this is a very old problem, but I'm finally getting around to trying to figure it out. My site experienced a dramatic organic traffic drop from Google (-40%) on May 17th 2014. It then drops another 30% on May 19th 2014. See graphs. According to Moz these two dates correlate with Payday Loan 2.0 and Panda 4.0. Panda makes complete sense, as this site (www.ausedcar.com) has a large amount of content that is syndicated across other sites (used car inventory is essentially the same everywhere on the Internet). Payday Loan on the other hand, which seems to be the primary traffic drop doesn't make any sense at all. Is it possible I started getting hit by Panda on the 17th and then it completed on the 19th? I know the dates for algorithm changes are not perfect. Next, assuming it is Panda, what are some things you guys have done to help with this? As I mentioned this content is duplicated all over the Internet, so it seems like Google arbitrarily picks winners and losers (my site is twenty years old!). I know I need unique content, but not sure how exactly to do that besides rewriting words so it doesn't appear duplicate.
Technical SEO | | Catbelly0 -
Is it important to include image files in your sitemap?
I run an ecommerce business that has over 4000 product pages which, as you can imagine, branches off into thousands of image files. Is it necessary to include those in my sitemap for faster indexing? Thanks for you help! -Reed
Technical SEO | | IceIcebaby0 -
Would posting content into these sites be a good boost related to authority?
Hi, Would posting content into these sites be a good boost related to authority? Press releases PRWebPRLeapArticlesthetechscoop.netthecampussocialite.comtechi.combusiness2community.commediaite.comexaminer.commakezine.comhuffingtonpost.comAll these site charge to post is it worth?Thanks
Technical SEO | | mtthompsons1 -
Can you have the same brand name for two distinct Google Places Pages
Hello Mozers I'd like to know if two existing GPP can have the same name 'spabycar'. At present 'mobile nail technician' based in W1 and 'spabycar 'WD' are ranking (ironically the one that has the old brand name is very high!). Both need to be 'spabycar' If this is feasible, is there a down side, and is there a good tutorial source for an aspirant to follow? Thanks Catherine
Technical SEO | | catherine-2793880 -
Summarize your question.Google places listing has gone AWOL :-(
<cite>Bonjour from sunny wetherby UK :-)</cite> <cite>Ive got a rogue Google places listing. I want the listing to sit under http://www.barrettsteel.com/ not under www.barrettonline.co.uk</cite> <cite>Here is the problem illustrated:</cite> <cite>http://i216.photobucket.com/albums/cc53/zymurgy_bucket/local-listing-attached-badly.jpg</cite> <cite>So my question is please. How do move the Google Pla ces lisrting from under www.barrettonline.co.uk to underwww.barrettsteel.com</cite> <cite>Thanks in advance,</cite> <cite>David</cite>
Technical SEO | | Nightwing0 -
Is having a sitemap.xml file still beneficial?
Hi, I'm pretty new to SEO and something I've noticed is that a lot of things become relevant and irrelevant like the weather. I was just wondering if having a sitemap.xml file for Google's use is still a good idea and beneficial? Logically thinking, my websites would get crawled faster by having one. Cheers.
Technical SEO | | davieshussein0