Adding the link masking directory to robots.txt?
-
Hey guys,
Just want to know if you have any experience with this.
Is it worthwhile blocking search engines from following the link masking directory.. (what i mean by this is the directory that holds the link redirectors to an affiliate site:
example:
mydomain.com/go/thislinkgoes to
amazon.com/affiliatelinkI want to know if blocking the 'go' directory from getting crawled in robots.txt is a good idea or a bad idea?
I am not using wordpress but rather a custom built php site where i need to manually decide on these things.
i want to specifically know if this in any way violates guidelines for google. it doesn't change the custom experience because they know exactly where they will end up if they click on the link.
any advice would be much appreciated.
-
Iredeto,
I do this on a few of my sites and it works out well. Saves on crawl budget, keeps google from accessing my affiliate links, and keeps any pagerank from passing through the links, which keeps me in-line with Google's webmaster policy on links.
One thing to keep in mind is that Google may show those URLs in the search results with a message that they can't show the content because it has been blocked. If you want to keep this from happening you may have to remove that directory GWT using the URL Removal tool. If they get re-indexed in 90 days (or whatever the reset time frame is) you will have to do it again. Hopefully that won't be an issue once you get them all removed the first time and block the folder.
Using rel = "nofollow" tags on the hyperlinks going to that directory wouldn't hurt either.
-
Hi!
I know cases where you can hide the contents of a folder to Google to not affect the rest of the website.
Note that you are facing a machine that knows how to crawl and index content.
Also be sure to block with noindex nofollow meta tags and removing URLs from Google index if at some point you had not locked.
Best regards!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google's Better Ads Chrome update: will it affect email pop-ups?
Chrome will be blocking ads on websites that are not compliant with the Better Ads standards as of Feb 15. Could not find any clues whether a pop-up (not an ad) asking for your email is included in this? We currently have a pop-up that appears on exit intent only (which is not penalized). Any ideas?
Algorithm Updates | | ati10 -
Ranking drop after sub domain to sub directory migration. Usual?
Hi all, We had our help articles on sub-domain help.website.com. Then we moved it to sub directory website.com/help/. We expected ranking improvement of website.com as there is a wide saying of benefiting from sub domain to sub directory migration. We have noticed that ranking improvement of new sub directory pages (website.com/help/) but not for any main website pages (website.com). I presume that link juice from main website has benefited new sub directory pages but main website lost ranking due to the page rank dilution. Do you agree? Any ideas? Thanks
Algorithm Updates | | vtmoz0 -
Best way of seeing how many links come from individual root domain domains.
Just wondering how best to see this - which tool to use. I'm dealing with a website with several thousand inbound links from around 100 root domains. Thanks in advance, Luke
Algorithm Updates | | McTaggart0 -
While doing directory submission, We should submit unique description and title ?
Hello Moz Members, I just want to clarify that, We do directory submission in 50 of sites. For Example: I have to target 10 keyword's, and i am doing directory submission. I have 10 Unique titles and 10 unique description. I just need to submit these 10 keywords in 50 directory's 10 keywords * 50 directory = 500 submission. I will just submit the same 10 Unique titles and 10 unique description to these 500 directory. So it wont be count as duplicate content and duplicate title in every directory. Or Every time i do directory submission i have to submit unique description and unique title. Please help me with these question, I am really confused how shall i proceed to directory submission. If any one have fast approval directory sites list then please share the information with me. Regards & Thanks, Chhatarpal Singh
Algorithm Updates | | chhatarpal0 -
Since authorship markup requires a domain email, how can a community website allow users to link their Google+ profile?
It seems that Google now requires authors to have a valid email on the domain. This is easy for the traditional web publication. But what about community websites like SEOmoz? How can a community website allow users to link their Google+ profile? Will community websites like SEOmoz be required to 1. Give all users a domain email 2. Ask users to validate the email address with Google? Seems overly complicated.
Algorithm Updates | | designquotes0 -
Sub-Links of Organic SERP
I would like to know if you can modify (or suggest) the sub-links under an organic listing. For Example: Main Link/Title = COMPANY NAME - What We Do.... Sub-Links (popular pages within site) currently include links like: Locations / Catalog Request / Bestsellers Is it possible to suggest other pages as sub-links or do the search engines determine these? Please advise, and thanks in advance....
Algorithm Updates | | WhiteCap0 -
Is Yahoo Directory worth the money?
Our listing is up for renewal in Yahoo Directory. Is there a way to measure the impact the listing had vs. the year before without the listing beyond direct referrals? Thanks!
Algorithm Updates | | leighw0 -
Any way to tell if a link has been devalued?
I have some listings in lawyer directories some of which have very hig PR , links, traffic, etc. For example, www.nolo.com, I know that Google has more or less recently devalued a lot of directory links. I would assume that a monster site like nolo would not be one of those, but does anyone know any way to tell? Paul
Algorithm Updates | | diogenes0