Confirming Robots.txt code deep Directories
-
Just want to make sure I understand exactly what I am doing
If I place this in my Robots.txt
Disallow: /root/this/that
By doing this I want to make sure that I am ONLY blocking the directory /that/ and anything in front of that. I want to make sure that /root/this/ still stays in the index, its just the that directory I want gone.
Am I correct in understanding this?
-
that's right!
Disallow: /root/this/ will bock the complete directly whereas,
Disallow: /root/this/that will only block the "that" within "this"
hope this helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is my knowledge graph code wrong?
I inserted the Knowledge Graph code on our site last week and am still not seeing the knowledge graph in our search results. Is something incorrect with my code? <script type="<a class="attribute-value">application/ld+json</a>"> { "@context" : "http://schema.org", "@type" : "Organization", "name" : "IssueTrak", "url" : "http://www.issuetrak.com/", "sameAs" : [ "http://www.facebook.com/issuetrak", "http://www.twitter.com/issuetrak", "http://plus.google.com/google.com/+Issuetrak"] } script>head> I suspect it is the alignment of the "{" and "}" but others in the company say that doesn't matter. Any other explanations for why the KG isn't showing in the results? Thanks I did test it with Google's Structured Data Testing Tool and got the "all's good."
Technical SEO | | Nobody15969167212220 -
How to use rel="alternate" properly for mobile directory.
Hey everyone, For the URL - http://www.absoluteautomation.ca/dakota-alert-dcpa-p/dkdcpa2500.htm - I have the following tags in the header: rel="canonical" href="http://www.absoluteautomation.ca/dakota-alert-dcpa-p/dkdcpa2500.htm" /> rel="alternate" media="only screen and (max-width: 640px)" href="http://www.absoluteautomation.ca/mobile/Product.aspx?id=37564" /> Yes Google WMT is reading these as duplicate pages with duplicate titles, meta descriptions etc. How can I fix this? Thanks!
Technical SEO | | absoauto0 -
No confirmation page on Google's Disavow links tool?
I've been going through and doing some spring cleaning on some spammy links to my site. I used Google's Disavow links tool, but after I submit my text file, nothing happens. Should I be getting some sort of confirmation page? After I upload my file, I don't get any notifications telling me Google has received my file or anything like that. It just takes me back to this page: http://cl.ly/image/0S320q46321R/Image 2013-04-26 at 11.15.25 AM.png Am I doing something wrong or is this what everyone else is seeing too?
Technical SEO | | shawn810 -
Does PR/linkjuice get transfered from directory->subdirectory?
If website.com links to website.com/utilities Does website.com/utilities/business.html pagerank get benefited from this? The reason I ask is, I have the option to have: 1) website.com/utilities/business.html or 2) website.com/utilities-business My boss says would like to keep the first example and link directory to subdirectory --> File. I prefer the second example because the potential traffic in it is a source of revenue and I want it to get linkpower from homepage directly. So, unless linkjuice is transfered horizontally across subdirectories, 2) would be better SEO-wise, is this correct? Thanks a lot.
Technical SEO | | JorgeGarcia1 -
How to start more deep seo.
Okay, lets say that I already fixed all my technical seo stuff. All my crawl errors and warnings are gone. Now I really want to start seo so I have a few questions. 1. I know I should work based on a page and not on a domain, so it means I should choose which keyword to optimize each page for? only one keyword per page? or it might be more? 2. Some keywords are very competitive for me, example for my site www.theprinterdepo.com, the best keyword to rank would be printers, but it would take years a beginner to do that, what should I do? 3. If highly competitive keywords are not an option, which keywords shuold I use? the ones that have more global monthly searches? Thanks
Technical SEO | | levalencia10 -
How to allow one directory in robots.txt
Hello, is there a way to allow a certain child directory in robots.txt but keep all others blocked? For instance, we've got external links pointing to /user/password/, but we're blocking everything under /user/. And there are too many /user/somethings/ to just block every one BUT /user/password/. I hope that makes sense... Thanks!
Technical SEO | | poolguy0 -
Robots.txt question
Hello, What does the following command mean - User-agent: * Allow: / Does it mean that we are blocking all spiders ? Is Allow supported in robots.txt ? Thanks
Technical SEO | | seoug_20050 -
Robots.txt and canonical tag
In the SEOmoz post - http://www.seomoz.org/blog/robot-access-indexation-restriction-techniques-avoiding-conflicts, it's being said - If you have a robots.txt disallow in place for a page, the canonical tag will never be seen. Does it so happen that if a page is disallowed by robots.txt, spiders DO NOT read the html code ?
Technical SEO | | seoug_20050