Adding non-important folders to disallow in robots.txt file
-
Hi all,
If we have many non-important folders like /category/ in blog.....these will multiply the links. These are strictly for users who access very rarely but not for bots. Can we add such to disallow list in robots to stop link juice passing from them, so internal linking will me minimised to an extent. Can we add any such paths or pages in disallow list? Is this going to work pure technical or any penalty?
Thanks,
Satish
-
But as per the current SEO buzz, internal nofollow leads of waste of link juice and we cannot preserve it. Moreover some suggests not to use nofollow internally.
-
This is a great resource for all things robots.txt related: [http://www.robotstxt.org/robotstxt.html](http://www.robotstxt.org/robotstxt.html) -
Hi,
Yes you can block in robots.txt. You can also use rel="nofollow" if you don't want to pass link juice.
[No Link Juice](https://www.example.com) Hope this helps. Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Need only tens of pages to be indexed out of hundreds: Robots.txt is Okay for Google to proceed with?
Hi all, We 2 sub domains with hundreds of pages where we need only 50 pages to get indexed which are important. Unfortunately the CMS of these sub domains is very old and not supporting "noindex" tag to be deployed on page level. So we are planning to block the entire sites from robots.txt and allow the 50 pages needed. But we are not sure if this is the right approach as Google been suggesting to depend mostly on "noindex" than robots.txt. Please suggest whether we can proceed with robots.txt file. Thanks
Algorithm Updates | | vtmoz0 -
Best place to employ "branded" related keywords to gain SEO benefits and rank for "non branded" keywords?
Hi all, I want to put this question straight with an example rather than confusing with a scenario. If there is company called "vertigo", a tiles manufacturer. There are many search queries with thousands of searches like "vertigo tiles life", "vertigo tiles for garden", "vertigo tiles dealers", "vertigo tiles for kitchen", etc....These kind of pages will eventually have tendency to rank for non-branded keywords like "tiles for garden", "tiles for kitchen", etc. So where to employ these kind of help/info pages? Main website or sub-domain? Is it Okay to have these pages on sub-domain and traffic getting diverted to sub domain? What if the same pages are on main website? Will main website have ranking improvement for non branded keywords because of employing the landing pages with related topics? Thanks
Algorithm Updates | | vtmoz0 -
Homepage title tag: "Keywords for robots" vs "Phrases for users"
Hi all, We keep on listening and going through the articles that "Google is all about user" and people suggesting to just think about users but not search engine bots. I have gone through the title tags of all our competitors websites. Almost everybody directly targeted primary and secondary keywords and few more even. We have written a very good phrase as definite title tag for users beginning with keyword. But we are not getting ranked well comparing to the less optimised or backlinked websites. Two things here to mention is our title tag is almost 2 years old. Title tag begins with secondary keyword with primary keyword like "seo google" is secondary keyword and "seo" is primary keyword". Do I need to completely focus on only primary keyword to rank for it? Thanks
Algorithm Updates | | vtmoz0 -
Technical Argument to Prefer non-www to www?
I've been recommending using non-www vs. www as a preferable set up if a client is starting a site from scratch, and there aren't any pre-existing links to consider. I'm wondering if this recommendation still holds? I’ve been looking on the interwebs and I’m seeing far fewer articles arguing for the non-www version. In the two courts, I’m seeing highlighted: Pro www: (ex: www.domain.com) Works better with CDN networks, where a domain needs to be specified (though that argument is 3 years old) Ability to restrict cookies to one hostname (www) or subdomain (info. blog. promo.) if using multiple subdomains IT people generally prefer it Pro non-www (ex: domain.com) If you ever want to support or add https://, you don’t have to support 2 sets of urls/domains Mindset: fewer and fewer people think in terms of typing in www before a site url, the future is heading towards dropping that anyway. Though that is a bit of a cosmetic argument…. Is there a trend going back to www? Is there a technical argument to recommend non-www over www? Thanks!
Algorithm Updates | | Allie_Williams0 -
Adding the link masking directory to robots.txt?
Hey guys, Just want to know if you have any experience with this. Is it worthwhile blocking search engines from following the link masking directory.. (what i mean by this is the directory that holds the link redirectors to an affiliate site: example:
Algorithm Updates | | irdeto
mydomain.com/go/thislink goes to
amazon.com/affiliatelink I want to know if blocking the 'go' directory from getting crawled in robots.txt is a good idea or a bad idea? I am not using wordpress but rather a custom built php site where i need to manually decide on these things. i want to specifically know if this in any way violates guidelines for google. it doesn't change the custom experience because they know exactly where they will end up if they click on the link. any advice would be much appreciated.0 -
How important is Social Media for building domain authority / Google rankings? Are there any cases?
I really would like to know if someone tested the importance of Social Media for Google rankings.
Algorithm Updates | | Seeders
Are there some sites who build authority only by doing good social media?
Ofcourse, I know it is all about the mix (content, linkbuilding, social media, etc.) but how important is it?
I know many sites who rank good without any form of social media, but I do not know any sites who do only social media and rank high. I hope there are some good cases which give good insight. ps. I know it becomes more and more important...0 -
Importance of Links for Local Search
**According to an article about the "no no's for local SEO" links are not very important. Here is an excerpt: "**Local SEO is very different when compared to traditional SEO. The importance of backlinks in local SEO isn’t as important. In other words, links simply don’t matter as much when it comes to local SEO. Googles’ local search algorithm treats links completely differently than its standard algorithm." How accurate is this statement? Wouldn't more links help your local pages rank better in non-local organic results such as the results outside of the new carousel?
Algorithm Updates | | pbhatt0 -
Major Slipping of highly important keywords in a few days
Realized some major slipping on some of my most important keywords- mn realtors and minnesota homes for sale- my site is joeandcindy.com After months of working on traffic, blogs etc, wondering what could slap me 10-25 spots in just a few days. There were no errors showing up on the report that I can see. Just looking for some quick advice on what I might be missing and some quick action steps I might take to reverse the trend. Thanks!
Algorithm Updates | | jjwelu0
