Cgi-bin folder
-
Is there cg-bin folder of any site related to SEO in anyways and why we always block Cgi-bin folder in robots.txt.
-
As eyepaq says - it's for holding executable "program" files. search engiens don't need access so it gets blocked. If it's empty you can even remove it.
-
No. No correlation with SEO.
However the same way you block your temp or admin folder, there is no need in letting any bot crawling your cgi-bin folder.
Cgi-bin folder holds executable file - you don't want external sources to see it anyway - it's "very" private from security reasons. You need more then knowing what's inside the folder to do harm but keeping it hidden is the wise call. Better safe then sorry.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should Sitemaps be placed in the sub folder they reference?
I have a sitemap-index.xml file in the root. I then have several sitemaps linked to from the index in example.com/sitemaps/sitemap1.xml, example.com/sitemaps/sitemap2.xml, etc. I have seen on other sites that for example a sitemap containing blogs where the blogs are located at example.com/blog/blog1/ would be located at example.com/blog/sitemap.xml. Is it necessary to have the sitemap located in the same folder like this? I would like to have all sitemaps in a single sitemap folder for convenience but not if it will confuse search engines. My index count for URLs in some sitemaps has dropped dramatically in Google Webmaster Tools over the past month or so and I'm not sure if this is having an effect. If it matters, I have all sitemap files, including the index, listed in the robots.txt file.
Technical SEO | | Giovatto0 -
URL Structure & SEO - Should we be using sub-folders?
Hi all, As part of our content marketing efforts we have run a number of initiatives in the past and created pages on the website to go along with them (also where the links for these particular projects point to). However, the URL structure isn't actually a reflection of where the pages sit on the site. Unfortunately I'm unable to provide a URL for reasons I won't bore you with, but here's an example: We recently ran a competition that was very successful in generating links. The URL for this is www.domain.co.uk/competition. However, the page actually sits within the About Us section - which is where all of our news and content marketing pages go - and uses a URL override. How much of an issue is this in regards to A) Our SEO in general?; and B) Ensuring we receive as much equity from the links we earn as possible? A brief explanation of what URL overrides actually are would also be useful! (We have a digital marketing agency who handle most of our SEO) Thanks in advance guys! John
Technical SEO | | NAHL-14300 -
Folder Hierarchy Structure Theory
Hi, I was wondering if search engines, in particular, Google, actually use folder hierarchy to determine how important a particular page on a website might be for ranking purposes, or is on-site page inter-linking only taken into consideration. I know that external and internal links help to support the authority or 'page rank' of a particular webpage on a website. In a typical Wordpress installation, for example, it is easy to create a page and assign child-pages to support it. These sub-pages would naturally link to their parent pages via menu and/or body links, so they would theoretically 'support' the authority of the parent folder/page. My question is... would search engines see the parent folder page as more authoritative than a child-page, even without a lot of on-site interlinking of child and parent pages, just because it is higher up in the folder structure? For example, I have a client who has a Wordpress website, but is using a plugin to make all pages have a .htm ending. The site is fairly 'flat', hierarchally speaking and does not use any /folders/, but the pages are inter-linked. In the following scenario, there are 4 testimonial pages... 1 main one and 3 supporting pages. The 3 supporting pages are linked to from the parent page and vice versa. /testimonials.htm /testimonials-quality.htm /testimonials-price.htm /testimonials-ease.htm I was wondering if it is worth suggesting to my client that we remove that plugin so that we can more easily employ the natural folder hierarchy functions of Wordpress, such as this scenario: /testimonials/ /testimonials/quality/ /testimonials/price/ /testimonials/ease/ Would the loss of 'link juice' due to redirects and the work that would be involved would be worth the possible ranking increases of potentially structuring the website better... or are we fine just relying on the existing page interlinking to show the search engines what are the important parent pages?
Technical SEO | | OrionGroup0 -
Use of Location Folders
I'd like to understand the pro's and con's of using a location subfolder as an SEO strategy (example: http://sqmedia.us/Dallas/content-marketing.html), where the /Dallas folder is holding all of my keyword rich page titles. The strategy is to get local-SEO benefits from the use of the folder titled /Dallas (a folder which is unnecessary in the over all structure of this site), but how much is this strategy taking away from the page-title keyword effectiveness?
Technical SEO | | sqmedia0 -
File name same as folder name, ok?
Is it ok to have a folder and file name to be both the same e.g domain.com/xyz-products/ domain.com/xyz-products.php File name would be a page that lists a number of products and then within the folder there would be x-product.php, y-product.php etc
Technical SEO | | NeilD0 -
Our UE team has presented me with a site structure where the content (folders) does not match the hierarchical directory structure (in the CME)
Our UE team has presented me with a new site structure where the content (folders) does not match the hierarchical directory structure (in the CME). I.E Sub-sectors, sectors and product pages are ALL just 1 directory off the root. example.com/sector example.com/sub-sector example.com/productpage FYI 'normal' folder hierarchy would be; example.com/sector/ example.com/sector/sub-sector example.com/sector/sub-sector/productpage I cannot find any SEO disadvantages re; crawl, if anything the SE's will crawl more efficeitly with clearly less depth... higher 'deep content', and a better nav - which is technically a sound solution with link consistency throughout - 1 to 2 clicks to all pages. Only disadvantage might be a user confusion... which can be off-set with contextual breadcrumbs. Are there any PURE SEO disadvantages to a structure this illogical? Note - This does not abuse any Search Engine guidelines. Thanks for reading, Rich
Technical SEO | | richcowley0 -
Which is better? Remove folder or fix the links and wait?
Dear Mozers, Recently I added a new language to one of my sites in a new folder (www.site.com/es/) but for some reasons many links got broken or were simply sending to a wrong page. This caused a bad indexing and it also showed a lot of duplicate content. I know how to fix it but my question is this: Is it better to remove the second language folder, fix it and then put it back up after a few months or just fix it now as it is and wait for G to come back and index the new links?
Technical SEO | | Silviu0 -
What is the best way to change your sites folder structure?
Hi, Our site was originally created with a very flat folder structure - most of the pages are at the top level. Because we will adding more content I want to tidy up the structure first. I just wanted to check what the best way to go about this was. Is it best to: First configure all the new 301 redirects to point to the new pages, while leaving the actual links on our site pointing to the old pages. Then changing the links on the site after a few weeks. Configure the redirects and change the actual links on my website at the same time to point to the new locations. My thinking that if I go with option 1 route then I will give Google a chance to process all the redirects and change the locations in their index before I start pointing them to the new locations. But does it make any difference? What is the best wat to go about making this sort of change to minimize any loss in rankings, page rank etc? Thanks for the help.
Technical SEO | | Maximise0