Blocking external links in Robots.txt - need advice on Best Practice
-
I look after an affiliate site that is doing quite well in the search rankings. We've been doing a review of our practices and one thing that has cropped up is our robots.txt.
In it, we disallow Google from crawling external links. This used to be best practice in the affiliate industry a couple of years ago, but I wanted to know if this is still the case, and what the implications are if we were to:
a/ leave it as is?
b/ allow crawling?
Thanks in advance.
-
I'm not sure how you're blocking Google from crawling external links in the robots.txt file--typically you only block them from crawling internal pages on your site.
If you're using a script, though, to track the clicks on external links and that script is running your site (and you're blocking that script in robot.txt), then that still should be fine. You may want to add a "nofollow" tag on those links, though, so you don't end up passing link "credit" or "link juice" to those affiliates (unless you want to do that?).
As far as external links go, though, it's typically okay (and expected) that you link out to other sites.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What's the best way to go about making Duplicate Content Pages on my domain for affiliates?
Hello, I would like to make a bunch of duplicate pages of my site's Home Page, that way affiliates of mine can have a page of their own with links specialized to themselves littered throughout the page. What's the best practice going about this without jeopardizing domain authority from tons of duplicate content signals firing off?
Affiliate Marketing | | Benavest0 -
Affiliates website, I need webpages linsk to study the topic
Kindly Provide me the best links where I can study about Affiliates websites, How to create them? there SEO? Content etc.
Affiliate Marketing | | bondhoward0 -
What is the best way to boost no. of quality sites linking in?
Analysing my site ranking compared to competitors, I see that our weakness is getting others to share our content and link into it via their own blogs and social media - although I would say our social media activity is pretty good. We have also worked on getting listed free on relevant sites related to our product area (learn spanish in spain) in order to increase sites linking but this is still minute compared to our 2 main competitors. (As a note: paid listings are not totally justifiable for us at this point in time but perhaps in the future we will expand on this). Am I missing something here?
Affiliate Marketing | | sparkit0 -
Links from Blogs and Forums
Hello Everyone, We are a small, local business in a city in Canada. We just got a new website and I am responsible for basically all of it, including the SEO. I was researching the back-links from our top-ranking competitor (another small, local business in our city). Here is their ranking: PA 40, with 441 Links from 111 RDs DA 30, with 6,864 Links from 116 RDs Here is the second-highest guy: PA 29, with 101 Links from 10 RDs DA 15, with 1,682 Links from 11 RDs As you can see, the top-ranking competitor's PA & DA are just about double his closest competitor. (Don't ask where we currently stand!) But, on closer examination, most of the links from our top-ranking competitor come from a local blog, where our competitor's ad appears in an advertisement on the sidebar. Each time they make a post, which is just about every day, our competitor gets a link. They also have a lot of links from a forum on a rock band's website. The original post was something about finding music, but the responses all have links to unrelated websites. This seems really, really slimy to me. What's going on?
Affiliate Marketing | | StillLearning0 -
Linking from High Authority Sites
Dear Mozzes, We're running a property portal for real estate classifieds which receives around 300.000 visits monthly. PA 47 DA 37 Mozrank 5,3 MozTrust 5,8 PR 4 Apart from our real estate classifieds we are producing proprietary trends and statistic, a rare commodity in the country we operate. Luckily we are about to enter into a partnership with one of the leading newspaper portals in the Country, appreciating PAs in the 80-90s. In this coming partnership we will provide: 1. Market statistics, which we can imagine as a chart followed by our logo and a "Powered By", which shall link to our site. 2. An in-framed part of our site in their portal where we list properties for sale, that will carry affiliate strings. In order to for us to get the maximum link value of this high authority site, how should we build the integration and linking? We've looked at similar examples here http://realestate.money.cnn.com/ here http://realestate.money.cnn.com/NY/New_York/ or maybe like the Simply Hired logo here: http://money.cnn.com/ Advices here would be super appreciated. Thanks Eric
Affiliate Marketing | | PropertyPortal0 -
Affiliate code in urls? affiliate link landing pages?
Is it considered a good or bad practice to have the affiliate code present in your affiliate link url's ? (as opposed to disguising them with something like phpbay) Does the general public not scour the hover-over url's of links they click? Also, is there any way of keeping customers on site when they click affiliate links? (via frames or something more modern) Or is that frowned upon by advertisers?
Affiliate Marketing | | Mozzin0 -
Link exchange
Folks I just thought to myself that this forum must be filled with lovely SEOs who are busy working on multiple clients.... I'm very new to the industry myself... about a month into it 🙂 Any-hoots I was just wondering if anybody is working in a category that may be suitable for a link exchange with http://bluetea.com.au/ .... Please let me know
Affiliate Marketing | | PHDAustralia680 -
Big Affiliate Site vs. Small In House Software For Link Value
Hi, I have a client who sells consumer products and is interested in affiliate advertising. Of course, they can get set up with something like CJ.com. They are also interested in possibly using some third party software to take their affiliate effort in house and be able to offer publishers simple urls. The idea is that this might help the link profile for SEO as well. Here are my questions: Does anybody have any experience with the complication level associated with third-party software for link value path? So, relative pluses and minuses. Can anyone recommend an affiiate software that is reliable and easy to use? Any opinion on one big site to go with... CJ, GAN, someone else? Thanks!
Affiliate Marketing | | 945010