Irrelevant Subfolders?
-
I have my root domain and as I am a web designer, I use subfolders as sandboxes for client websites. I would not like these to be searched though because they are just demos, ideas and such. Would this be a robots.txt matter or can I have SEOMoz wok with a campaign where I can exclude subfolders? Thank you in advance! : )
-
Hi Wil,
If you want to exclude all robots from crawling these pages you can exclude them from Robots.txt.
If you want to exclude Rogertbot you can find the info on how to do it on: http://www.seomoz.org/dp/rogerbot
I hope that helped,
Istvan
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
404s on subfolder - how to redirect?
Hi all,
Technical SEO | | MFSMarketing
we have a lot of 404s to subfolders. Eg
www.website.com/blog-post-title/imagename/
www.website.com/blog-post-title/author/ We don't have these subfolders or blog posts anymore.
How do i redirect them? These links (404s) don't seem to have any value or backlinks. Thanks,
Stef0 -
Subdomain/subfolder question
Hi community, Let's say I have a men's/women's clothing website. Would it be better to do clothing.com/mens and clothing.com/womens OR mens.clothing.com and womens.clothing.com? I understand Moz's stance on blogs that it should be clothing.com/blog, but wanted to ask for this different circumstance. Thanks for your help!
Technical SEO | | IceIcebaby0 -
How to deal with high authority but irrelevant external links
Hi, My client has an online ecommerce site where he sells wedding items. His previous SEO company added his URL to many websites that are related to porn. Those are high authority websites but are irrelevant to my client's business. Should I disavow those links?
Technical SEO | | alexkatalkin0 -
301 Redirects in subfolders
Hi, we're making our site into a static site but I would like to transfer the Google juice. Most of the links and database exist on subfolders though. Could I simply do 301 redirects on the subfolders and retain the value or does it have to be on the full domain?
Technical SEO | | Therealmattyd0 -
Changing a blog url from subdomain to subfolder
I am abou to change my company blog from a subdomain (blog.mydomain.com) to a subfolder (mydomain.com/blog), from suggestions from this awesome community! Not only that though, because the current blog is on another server than the main site I have to move my blog between servers as well. This will be a big hassle for me, and means a big risk for errors as I don't have a clue what I am doing on the development part. Hint: I'm no developer. My blog is fairly new, having posted 18 blog posts so far. There is no major linking to or from the blog as it has been basically no activity on the blog. It has been fairly good optimized for SEO, with custom plugin settings for Wordpress SEO plugin and similar. Also followed advice from Rand regarding wordpress SEO. So I guess my question is: Would it be a big loss for me to just start over with a new blog on the subfolder domain? And move content over from the old blog manually (and then deleting the old one). Or would It be plain stupid taking that route? Thankfull for all help I can get!
Technical SEO | | danielpett0 -
Blog for a Prestashop: WP at subfolder or blog with dedicated keyword based domain?
Hi Scenario 1: site running with prestashop at domain whatever.com There's a blog at a subfolder of domain, running with Wordpress whatever.com/blog Scenario 2: site running with prestashop at whatever.com. The blog is at a different server and domain with keywords, e.g. cloudtechnology.com What's better for the blog posts to have better SERP results? Scenario 1 or 2? From the point of view of a user, I have seen for friends and colleagues that scenario 2 works for the blog, and might work for the shop, since blog posts contain links to shop etc. However, all examples I've seen for scenario 1 are still struggling...it's hard for them to make the blog visible at SERPs...is it a coincidence or is there a reason? Thanks for your answers!
Technical SEO | | pupila0 -
Set base-href to subfolders - problems?
A customer is using the <base>-tag in an odd way: <base href="http://domain.com/1.0.0/1/1/"> My own theory is that the subfolders are added as the root because of revision control. CSS, images and internal links are used like this:
Technical SEO | | Vivamedia
internal link I ran a test with Xenu Link Sleuth and found many broken links on the site, but I can't say if it is due to the base-tag. I have read that the base-tag may cause problems in some browsers, but is this usage of base-tag bad in some SEO-perspective? I have a lot of problems with this customer and I want to know if the base-tag is a part of it.0 -
Including spatial location in URL structure. Does subfolder number and keyword order actually matter?
The SEOMoz On-Page report for my site brings up one warning (among others) that I find interesting: Minimal Subfolders in the URL My site deals with trails and courses for both races and general running. The structure for a trail is, for example: /trails/Canada/British-Columbia/Greater-Vancouver-Regional-District/Baden--Powell-Trail/trail/2 The structure for courses is: /course/28 In both cases, the id at the end is used for a database lookup. I'm considering an URL structure that would be: /trail/Baden-Powell-Trail/ca-bc-vancouver This would use the country code (CA) and sub-country code (BC) along with the short name for the region. This could be good because: it puts the main keyword first the URL is much shorter there are only 3 levels in the URL structure However, there is evidence, from Google's Matt Cutts, that the keyword order and URL structure don't matter in that way: See this post: http://www.seomoz.org/q/all-page-files-in-root-or-to-use-directories If Matt Cutts says they aren't so important then why are they listed in the SEOMoz On-Page Report? I'd prefer to use /trail/ca-bc-vancouver/Baden-Powell-Trail. I'll probably do a similar thing for courses. Is this a good idea? Thoughts? Many thanks, in advance, for your help. Cheers, Edward watch?v=l_A1iRY6XTM watch?v=gRzMhlFZz9I
Technical SEO | | esarge0