Adding the link masking directory to robots.txt?
-
Hey guys,
Just want to know if you have any experience with this.
Is it worthwhile blocking search engines from following the link masking directory.. (what i mean by this is the directory that holds the link redirectors to an affiliate site:
example:
mydomain.com/go/thislinkgoes to
amazon.com/affiliatelinkI want to know if blocking the 'go' directory from getting crawled in robots.txt is a good idea or a bad idea?
I am not using wordpress but rather a custom built php site where i need to manually decide on these things.
i want to specifically know if this in any way violates guidelines for google. it doesn't change the custom experience because they know exactly where they will end up if they click on the link.
any advice would be much appreciated.
-
Iredeto,
I do this on a few of my sites and it works out well. Saves on crawl budget, keeps google from accessing my affiliate links, and keeps any pagerank from passing through the links, which keeps me in-line with Google's webmaster policy on links.
One thing to keep in mind is that Google may show those URLs in the search results with a message that they can't show the content because it has been blocked. If you want to keep this from happening you may have to remove that directory GWT using the URL Removal tool. If they get re-indexed in 90 days (or whatever the reset time frame is) you will have to do it again. Hopefully that won't be an issue once you get them all removed the first time and block the folder.
Using rel = "nofollow" tags on the hyperlinks going to that directory wouldn't hurt either.
-
Hi!
I know cases where you can hide the contents of a folder to Google to not affect the rest of the website.
Note that you are facing a machine that knows how to crawl and index content.
Also be sure to block with noindex nofollow meta tags and removing URLs from Google index if at some point you had not locked.
Best regards!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Domain has been redirected our site; but many incoming links from sub domain. Will this hurts?
Hi all, This is the scenario: Our website is newwebsite.com. Our old website is oldwebsite.com which has been redirected to newwebsite.com (years back). But one of the old website's sub domain has a lot of back links to our current website like: seo.oldwebsite.com to newwebsite.com. Will this scenario hurts with any wrong linking? Thanks
Algorithm Updates | | vtmoz0 -
A page will not be indexed if published without linking from anywhere?
Hi all, I have noticed one page from our competitors' website which has been hardly linked from one internal page. I just would like to know if the page not linked anywhere get indexed by Google or not? Will it be found by Google? What if a page not linked internally but go some backlinks from other websites? Thanks
Algorithm Updates | | vtmoz0 -
Do the referring domains matter a lot in back-links? Google's stand?
Hi, It's a known fact about quality of back-links than quantity. Still domains are heavily different from links. Multiple domains are huge comparing to multiple links. Taking an average, how much does 'number of referring domains" boost website authority? I am not speaking about low quality domains, just number of domains including which are irrelevant to the topic or industry. Thanks
Algorithm Updates | | vtmoz0 -
Do back-links to non indexed sub-domains / sub-directories considered by Google as website backlinks and pass Pagerank to website?
Hi, If some noindexed links on our website or sub-domain got some backlinks, will that backlinks pass Pagerank / linkjuice to website? Will they be considered as backlinks to website by Google? Here is a statement from Matt cutts for the question. My question is same as below with answer? Eric Enge: Can a NoIndex page accumulate PageRank? Matt Cutts: A NoIndex page can accumulate PageRank, because the links are still followed outwards from a NoIndex page. Thanks
Algorithm Updates | | vtmoz0 -
How can a site with two questionable inbound links outperform sites with 500-1000 links good PR?
Our site for years was performing at #1 for but in the last 6 months been pushed down to about the #5 spot. Some of the domains above us have a handful of links and they aren't from good sources. We don't have a Google penalty. We try to only have links from quality domains but have been pushed down the SERP's? Any suggestions?
Algorithm Updates | | northerncs0 -
Guest Blog Post or Article Content Should be Do-Follow or No-Follow Link ? Help Plz !
Many SEO writers and blogs after Google Matt Cutt said, You should not allow no-follow link in Guest Post. What should we do. ? I am allowing Guest post - what they ask in return a do-follow link to their site or blog. other articles or post i wrote about inspiration collected from different source or single portfolio site - i credit them(as blogger - we should respect them). What i am doing right or wrong ? Please advise and help me on this ! http://searchengineland.com/google-guest-blogging-for-links-you-better-nofollow-those-links-166218
Algorithm Updates | | Esaky0 -
Excessive internal links. Should I remove the footer links?
Hi guys, I have an ecommerce site selling eco-friendly items online. I ran some on-page optimisation reports from SEOMoz PRO and discovered that I have at least 120 internal links per page. 32 of these are in the footer, designed in part to aid user navigation but perhaps also to have a positive impact on SERPs and SEO in general for the ecommerce site. Will removing these links be beneficial to my search engine rankings, as I will have less than 100 internal links per page? Or is it a major change which may be dangerous for my site rankings? Please help as I'm not sure about this! I've attached an image of the footer links below. I won't be removing the Facebook/Twitter links, just the 3 columns on the left. Thank you, Pravin MAvLe.jpg
Algorithm Updates | | goforgreen0 -
Is Yahoo Directory worth the money?
Our listing is up for renewal in Yahoo Directory. Is there a way to measure the impact the listing had vs. the year before without the listing beyond direct referrals? Thanks!
Algorithm Updates | | leighw0