Irrelevant Subfolders?
-
I have my root domain and as I am a web designer, I use subfolders as sandboxes for client websites. I would not like these to be searched though because they are just demos, ideas and such. Would this be a robots.txt matter or can I have SEOMoz wok with a campaign where I can exclude subfolders? Thank you in advance! : )
-
Hi Wil,
If you want to exclude all robots from crawling these pages you can exclude them from Robots.txt.
If you want to exclude Rogertbot you can find the info on how to do it on: http://www.seomoz.org/dp/rogerbot
I hope that helped,
Istvan
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
404s on subfolder - how to redirect?
Hi all,
Technical SEO | | MFSMarketing
we have a lot of 404s to subfolders. Eg
www.website.com/blog-post-title/imagename/
www.website.com/blog-post-title/author/ We don't have these subfolders or blog posts anymore.
How do i redirect them? These links (404s) don't seem to have any value or backlinks. Thanks,
Stef0 -
Moving a subdomain to subfolder with CNAME
Hi all, We have our docs in a subdomain on GitHub and looking to move it into a subfolder. Github offers the ability to redirect via CNAME https://gitbookio.gitbooks.io/documentation/platform/domains.html I am getting conflicting information on if this will work or cause duplicate content and hurt our SEO.
Technical SEO | | kate.hassey0 -
How to deal with high authority but irrelevant external links
Hi, My client has an online ecommerce site where he sells wedding items. His previous SEO company added his URL to many websites that are related to porn. Those are high authority websites but are irrelevant to my client's business. Should I disavow those links?
Technical SEO | | alexkatalkin0 -
International architecture: Country specific subfolders > domain mapping to tld
Hi Ive got a clients dev saying they are setting up with country/language specific subfolders (as i recommended) BUT now they are saying they want to set up on network.domain.com (for example) and then each language will have its own sub-folder BUT will be domain mapped to the TLD as and when they get them. I have asked them to clarify since sounds a bit strange since thought best to have domain.com then /uk and /us etc etc and sure ok to forward country specific TLD's to these subfolders. Its this new subdomain (network.) thats concerning me and mapping rather than forwarding (or is it the same thing) but anyone know off hand if above sounds ok or also thinks a bit strange or know issues with such a set up ? many thanks dan
Technical SEO | | Dan-Lawrence0 -
Reverse proxy a successful blog from subdomain to subfolder?
I have an ecommerce site that we'll call confusedseo.com. I created a WordPress blog and CNAME'd it to blog.confusedseo.com. Since then, the blog has earned a PageRank of 3 and a decent amount of organic traffic. I am considering a reverse proxy to forward blog.confusedseo.com to confusedseo.com/blog/. As I understand it, this will greatly help the "link juice" of the root domain. However, I'm concerned about any potential harm done to the existing SEO value of the blog. What, if anything, should I be doing to ensure that the reverse proxy doesn't hurt my "juice" rather than help it?
Technical SEO | | bedbugsupply0 -
Subdomain hosted on a different server VS Subfolder on main server
We have a website developed in ColdFusion on a server does not support PHP. We have a blog for the site using WordPress (PHP), hosted on a different server, with a subdomain as the URL. (example: blog.website.com) I've heard that search engines treat subdomains as completely different websites from the main domain, so they could actually be in competition for rankings in the search engines - is that correct? I am under the impression that the traffic to the blog will not show as traffic to the main website, because it is hosted on a different server - is that right? If I am correct, I assume the best solution would be to install PHP on our main server, and put the blog in a subfolder ... or would the subdomain be OK as long as the blog is hosted on the main server? Thanks!
Technical SEO | | vermont0 -
Can SEOMoz crawl a single page as oppose to an entire subfolder?
I would like the following page to be crawled: http://www.ob.org/_programs/water/water_index.asp Instead, SEOMoz changes the page to the following subfolder which is an invalid url: http://www.ob.org/_programs/water/
Technical SEO | | OBIAnalytics0 -
Removing irrelevant items from Google News?
A client wants to know if it's possible to get Google to remove stories from Google News feeds if those stories have nothing to do with the client? Any advice would be greatly appreciated. Thank you.
Technical SEO | | JamesAMartin0