Blocking external links in Robots.txt - need advice on Best Practice
-
I look after an affiliate site that is doing quite well in the search rankings. We've been doing a review of our practices and one thing that has cropped up is our robots.txt.
In it, we disallow Google from crawling external links. This used to be best practice in the affiliate industry a couple of years ago, but I wanted to know if this is still the case, and what the implications are if we were to:
a/ leave it as is?
b/ allow crawling?
Thanks in advance.
-
I'm not sure how you're blocking Google from crawling external links in the robots.txt file--typically you only block them from crawling internal pages on your site.
If you're using a script, though, to track the clicks on external links and that script is running your site (and you're blocking that script in robot.txt), then that still should be fine. You may want to add a "nofollow" tag on those links, though, so you don't end up passing link "credit" or "link juice" to those affiliates (unless you want to do that?).
As far as external links go, though, it's typically okay (and expected) that you link out to other sites.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Looking for a way to crawl and test validity of affiliate links at scale. Ideas?
Hey all, I'm on the hunt for a service that will crawl our affiliate links and let us know when they return an error. I need to know that the last URL in the chain is returning a 200 over thousands of pages and links on a continual basis. The hitch is that most crawlers like Screaming Frog will return all of our links as working because it's only testing the first step, and this really requires a cloud solution anyway. Anyone happen to know of something? Edit for clarity's sake: I need something to check entire redirect chains in bulk that isn't a Wordpress plugin, isn't a website where you plug in a URL and it cuts you off after the first 100 results, and has the ability to crawl the site and provide reporting on a continual basis. Bi75jku
Affiliate Marketing | | BradsDeals0 -
Choosing an Affiliate Software & Link SEO-Value
Hello everyone. I'm looking for recommendations for affiliate software. I'm also curious if there is any truth in the "SEO-value" that some softwares are touting as a benefit. Choosing a Software: This Moz Blog Article says to build it yourself. However, I'm inclined to use an out-of-the-box software for advanced features such as real-time analytics for the affiliate, w-9/1099 processing, etc. MozPerks promotes hasoffers.com which seems overly pricey ($230/month with perk discount) and OMNISTAR who just left me on hold for 10 minutes (and still no anwer from their sales department). So I'm still looking. Any recommendations? SEO-value of Links from Affiliates: Here's an example of what OMNISTAR claims about the SEO value of the way they structure the links from affiliate sites to my site. I hear a variation on this pitch from each software and none of them seem to agree. Does Google want me to clearly tag promotional vs. editorial links? Is there SEO-value in promotional links on affiliate sites or is that a load of bologna? Thanks a bunch!
Affiliate Marketing | | Harbor_Compliance0 -
Affiliate program links
Hello, I've question regarding affiliate partner generated links.Company www.example.com has a affiliate program. Partner www.example2.com links to the company site with affiliate link www.example.com/?affid=xxxx in banner, article etc. As I understand these type of links could hurt our Company site, so I asked partners to use rel="nofollow" in all links where they link to us usign ?affid=xxxx in url. To be 100% sure that I did the right thing I decided to post here my question. Could these type of links hurt our Company site? Regards,
Affiliate Marketing | | juris_l
Juris0 -
What is the right way to link to your main site?
Hi, we have a system for tracking the leads that comes from a specific affiliate website. each affiliate has a unique tracking code. not only for affiliates that work with us but also for the SEO team has an affiliate tracking code so our bosses can track the leads and traffic that come from the activity of the SEO team. this means, links towards our website look like: www.mydomain.com/?t_src=campaign&t=AFF&t_cre=links&A=371 www.mydomain.com/?A=371 I have few questions about this: 1. This is the right way to link to my company site? How dose google crawl those link? It's can harm the link value? 2. What is my option to show my boss all the traffic / leads that our seo team brings to the site trough Google Analytics? Hope to get your support. Thanks in advance.
Affiliate Marketing | | JonsonSwartz0 -
How many affiliate links is considered too many?
Hi, Let's say you have great reviews for 50 products and some of these products do have affiliate links on review pages. And then you have user scores and you come up with a top 20 product list sorted by user scores. Now if you have the list of top 20 products on one page and all these products have an affiliate link (with nofollow relation and a 301 redirect) on the same page with only a couple of images and a summary of the review linked to the review pages, would this still be considered as what Google calls a "bridge page"? Would it be better to still generate the top 20 list but rather link to review pages only? (to avoid too many affiliate links on one single page).
Affiliate Marketing | | Gamer070 -
Using follow links on client and resellers site
Hi, i have a backup software product that I sell to business. I would like to create a badge that says something like "powered by x backups". This would be an image with a href url pointing back to my company site. I would intend to give the client a discount if they were to place the small logo in the footer section of their website. I would also like to offer this to resellers. For example "Authorised reseller of x backups" badge that could be placed on the resellers website. I would like to do this for two reasons: To promote my company to the visitors of the clients website To build link juice Would this be effective in building quality links? It this type of link building allowed by google? If not, how strict are they? Is there any variation of my intended promotion strategy that would be more effective? Thanks
Affiliate Marketing | | blakadz0 -
Affiliate links and Panda
Hello all, my question relates to the recent panda changes and the subsequent impact on pages with affiliate links. I have a hub of pages on my website talking about parcel delivery. Each page has unique, quality content and up until about 3 weeks ago they were starting to achieve strong rankings. We then decided to add some affiliate links (just 3) on each of the 5 parcel related pages we have. The rankings for each of these pages have tanked completely; rather than top 20, we're now outside of the top 100 for all of them. When we noticed this we masked the link, changing it to domain.com/index/affiliate; we also no followed the link. Alas this has had minimal effect in restoring the rankings. What i'm now thinking of doing is removing the affiliate links from the parcel landing pages altogether and instead providing users with a no followed link to another page within the site upon which we would have the affiliate links. This page would also be no indexed as well. My question is: Is this a good idea and if so will it restore my previous rankings over time, or have those pages (as a result of the affiliate links) been permanently down graded?? Any assistance would be gratefully received!
Affiliate Marketing | | danielparry0 -
Link exchange
Folks I just thought to myself that this forum must be filled with lovely SEOs who are busy working on multiple clients.... I'm very new to the industry myself... about a month into it 🙂 Any-hoots I was just wondering if anybody is working in a category that may be suitable for a link exchange with http://bluetea.com.au/ .... Please let me know
Affiliate Marketing | | PHDAustralia680