Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Cgi-bin folder

Technical SEO

1241

Alick300 last edited by

Is there cg-bin folder of any site related to SEO in anyways and why we always block Cgi-bin folder in robots.txt.
1 Reply Last reply
Reply Quote 0
matbennett last edited by

As eyepaq says - it's for holding executable "program" files. search engiens don't need access so it gets blocked. If it's empty you can even remove it.
1 Reply Last reply
Reply Quote 1
eyepaq last edited by

No. No correlation with SEO.

However the same way you block your temp or admin folder, there is no need in letting any bot crawling your cgi-bin folder.

Cgi-bin folder holds executable file - you don't want external sources to see it anyway - it's "very" private from security reasons. You need more then knowing what's inside the folder to do harm but keeping it hidden is the wise call. Better safe then sorry.
1 Reply Last reply
Reply Quote 1

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Cross Domain Canonicalization for Site Folder

Hello colleagues! I have a client who decided to launch a separate domain so they could offer their content translated for other countries. Each country (except the US/English) content lives in its own country folder as follows: client.com/01/02/zh
client.com/01/02/tw etc. The problem is that they post the content in US/English on this domain too. It does NOT have its own folder, but exists righth after the date (as in the above example) Oh, and the content is the same as on their "main" domain so google likes to index that sometimes vs. the original client on the domain where we want the traffic to go. SO, is there a way to say "hey google, please index the US content only on the main domain, but continue to index the translated content in these folders on this totally separate domain?" Thank you so much in advance.
Technical SEO | | SimpleSearch

0
CGI Redirects

Trying to 301 old legacy files like oldsite.com/green/red.cgi/blue/ using this htaccess code: Redirect 301 /green/red.cgi/blue/ http://www.newsite.com/summary-page/
Instead it's redirecting to: newsite.com/red.cgi/blue/ FYI oldsite.com's htaccess file does not have any global 301 rules that would conflict Does anyone know if cgi files require a different 301 syntax? Thanks!
Technical SEO | | SoulSurfer8

0
Should Sitemaps be placed in the sub folder they reference?

I have a sitemap-index.xml file in the root. I then have several sitemaps linked to from the index in example.com/sitemaps/sitemap1.xml, example.com/sitemaps/sitemap2.xml, etc. I have seen on other sites that for example a sitemap containing blogs where the blogs are located at example.com/blog/blog1/ would be located at example.com/blog/sitemap.xml. Is it necessary to have the sitemap located in the same folder like this? I would like to have all sitemaps in a single sitemap folder for convenience but not if it will confuse search engines. My index count for URLs in some sitemaps has dropped dramatically in Google Webmaster Tools over the past month or so and I'm not sure if this is having an effect. If it matters, I have all sitemap files, including the index, listed in the robots.txt file.
Technical SEO | | Giovatto

0
Use of Location Folders

I'd like to understand the pro's and con's of using a location subfolder as an SEO strategy (example: http://sqmedia.us/Dallas/content-marketing.html), where the /Dallas folder is holding all of my keyword rich page titles. The strategy is to get local-SEO benefits from the use of the folder titled /Dallas (a folder which is unnecessary in the over all structure of this site), but how much is this strategy taking away from the page-title keyword effectiveness?
Technical SEO | | sqmedia

0
File name same as folder name, ok?

Is it ok to have a folder and file name to be both the same e.g domain.com/xyz-products/ domain.com/xyz-products.php File name would be a page that lists a number of products and then within the folder there would be x-product.php, y-product.php etc
Technical SEO | | NeilD

0
What are your thoughts on security of placing CMS-related folders in a robots.txt file?

So I was just about to add a whole heap of CMS-related folders to my robots.txt file to exclude them from search, and thought "hey, I'm publicly telling people where my admin folders are"...surely that's not right?! Should I leave them out of the robots.txt file, and hope for the best that they never get indexed? Should I use noindex meta data on every page? What are people's thoughts? Thanks, James PS. I know this is similar to lots of other discussions around meta noindex vs. robots.txt, but I'm after specific thoughts around the security aspect of listing your admin folders in a robots.txt file...
Technical SEO | | James-Distinction

0
Should I repeat keywords at each folder level of my urls?

Hey Guys, I'm wondering which URL is preferable when targeting the keyword phrase "ski goggles" a) http://www.evo.com/shop/ski/ski-goggles.aspx or b) http://www.evo.com/shop/ski/goggles.aspx URL a includes the keyword phrase exactly with a dash but also repeats the word "ski" and feels redundant. Any research/ testing to support either case? Thanks a bunch. Will
Technical SEO | | evoNick

0
CGI Parameters: should we worry about duplicate content?

Hi, My question is directed to CGI Parameters. I was able to dig up a bit of content on this but I want to make sure I understand the concept of CGI parameters and how they can affect indexing pages. Here are two pages: No CGI parameter appended to end of the URL: http://www.nytimes.com/2011/04/13/world/asia/13japan.html CGI parameter appended to the end of the URL: http://www.nytimes.com/2011/04/13/world/asia/13japan.html?pagewanted=2&ref=homepage&src=mv Questions: Can we safely say that CGI parameters = URL parameters that append to the end of a URL? Or are they different? And given that you have rel canonical implemented correctly on your pages, search engines will move ahead and index only the URL that is specified in that tag? Thanks in advance for giving your insights. Look forward to your response. Best regards, Jackson
Technical SEO | | jackson_lo

0