Deny visitors by referrer in .htaccess to clean up spammy links?
-
I want to lead off by saying that I do not recommend trying this. My gut tells me that this is a bad idea, but I want to start a conversation about why.
Since penguin a few weeks ago, one of the most common topics of conversation in almost every SEO/Webmaster forum is "how to remove spammy links". As Ryan Kent pointed out, it is almost impossible to remove all of these links, as these webmasters and previous link builders rarely respond. This is particularly concerning given that he also points out that Google is very adamant that ALL of these links are removed.
After a handful of sleepless nights and some research, I found out that you can block traffic from specific referring sites using your.htaccess file. My thinking is that by blocking traffic from the domains with the spammy links, you could prevent Google from crawling from those sites to yours, thus indicating that you do not want to take credit for the link.
I think there are two parts to the conversation...
-
Would this work? Google would still see the link on the offending domain, but by blocking that domain are you preventing any strength or penalty associated with that domain from impacting your site?
-
If for whatever reason this would nto work, would a tweak in the algorithm by Google to allow this practice be beneficial to both Google and the SEO community? This would certainly save those of us tasked with cleaning up previous work by shoddy link builders a lot of time and allow us to focus on what Google wants in creating high quality sites.
Thoughts?
-
-
I've thought about this idea to get rid of bad links from specific referrers.
I agree with the comments Ian has made but when it is the homepage it's not always possible to follow those steps. Has anyone else had any experience/information?
-
Hi Ian,
Thanks for the response. I agree with you that this feels wrong and I don't recommend anyone try this (unless you have a site that you don't mind using as a guinea pig).
The problem is that, if this is your home page or a strong category page with many legitimate links, rankings, ect., it will be tough to abandon the page altogether.
To follow up about the original idea, I have done some additional research and found a few mentions, though none of them exceptionally credible, of the fact that the Googlebot does not pass referrer data to the server. Can anyone confirm?
-
Ooooh, that feels all wrong.
Seems to me .htaccess would block visitors that you may want. And while Google may obey the directive, they may also misinterpret a referrer-specific directive as cloaking.
If you want to get the same effect, here's what I'd do:
- Change the page to which the offending links point. Have it say "This page is gone, but you can get the information you want here" and make that a link to a new page.
- Set up your server to return a 410 code when folks visit that page.
- Set up a new page with the old page's content, so folks can click from the old to the new, but visiting bots and browsers get a 410 code and dump the page.
That will get Google to de-index the page in a hurry.
Another option: Simply add noindex, nofollow to the targeted page.
But I like the 410 option, because that should break the authority flow and has the best chance of giving Google what it wants, short of removing the link.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEO Links in Footer?
Hi, One of my clients uses a pretty powerful SEO tool, won't mention the name. They now have a "link equity" tool, which they are using on a lot of their client's sites, which include tons of fortune 500 companies. It involves add footer links to your site that change based on the content of the page they are on. The machine learning tries to figure out the most related pages and links to them with the heading tag of that page as the anchor text. Initially this sounds very spammy to me. But then, it seems a lot like "related products" tools that many companies use. The goal for this tool is to build up internal linking, especially for deeper pages on their site. They have over 10,000 currently. What are everyone's thoughts on this strategy?
White Hat / Black Hat SEO | | vetofunk2 -
Unnatural inbound links message from Google Webmaster Tools!
Hi Everyone, I just got this message from GWT(image below) This is probably a penguin Penalty. What is clear is I have to find the best and most efficient way to tackle this issue. We will probably lose tons of traffic in the next couple of weeks so I would like to get the best suggestions and maybe a guideline on how to do this in the most effective way! Thank you! 1a0X2M2a1h0A
White Hat / Black Hat SEO | | Ideas-Money-Art0 -
Benefits of having outbound links
Are there any strengths (benefits) in having outbound links within the site regarding SEO? If linking to reputable sites, would that help increase our SEO strength or does that only work if they links back to us?
White Hat / Black Hat SEO | | WebRiverGroup1 -
Is linking out to different websites with the same C-Block IP bad for SEO?
Many SEOs state that getting (too many) links from the same C-Block IP is bad practice and should be avoided. Is this also applicable if one website links out to different websites with the same C-Block IP? Thus, website A, B and C (on the same server) link to website D (different server) could be seen as spam but is this the same when website D links to website A, B and C?
White Hat / Black Hat SEO | | TT_Vakantiehuizen0 -
Links and how they count?
We managed to get ourselves out of a penalty 6 months ago and 100 days later after the message of penalty removable we finally felt that we were moving back on track (not a lot of movement before and 50% down due to links being taken away), we have around 120 really high quality links but 95% of them are urls or the business name. Anyway we still have a couple of pages that I feel are fairly down on rankings and most of the links as mentioned above are high quality but they are either anchor text of the website name or url my main question is that when looking at my competitors I see that they have the same or less links and from much less powerful places (most I would not touch) but they seem to have a ratio of 5 - 10 % of the links are the keywords they are trying to rank for. My question is if you have 50 links from better places but they are unrelated terms such as the web site name or just urls and you have 50 links from average places but 5 - 10% are on related terms to what you are trying to rank for which ones would win out.
White Hat / Black Hat SEO | | BobAnderson0 -
Duplicate content or not? If you're using abstracts from external sources you link to
I was wondering if a page (a blog post, for example) that offers links to external web pages along with abstracts from these pages would be considered duplicate content page and therefore penalized by Google. For example, I have a page that has very little original content (just two or three sentences that summarize or sometimes frame the topic) followed by five references to different external sources. Each reference contains a title, which is a link, and a short abstract, which basically is the first few sentences copied from the page it links to. So, except from a few sentences in the beginning everything is copied from other pages. Such a page would be very helpful for people interested in the topic as the sources it links to had been analyzed before, handpicked and were placed there to enhance user experience. But will this format be considered duplicate or near-duplicate content?
White Hat / Black Hat SEO | | romanbond0 -
Advice on using the disavow tool to remove hacked website links
Hey Everyone, Back in December, our website suffered an attack which created links to other hacked webistes which anchor text such as "This is an excellent time to discuss symptoms, fa" "Open to members of the nursing/paramedical profes" "The organs in the female reproductive system incl" The links were only visible when looking at the Cache of the page. We got these links removed and removed all traces of the attack such as pages which were created in their own directory on our server 3 months later I'm finding websites linking to us with similar anchor text to the ones above, however they're linking to the pages that were created on our server when we were attacked and they've been removed. So one of my questions is does this effect our site? We've seen some of our best performing keywords drop over the last few months and I have a feeling it's due to these spammy links. Here's a website that links to us <colgroup><col width="751"></colgroup>
White Hat / Black Hat SEO | | blagger
| http://www.fashion-game.com/extreme/blog/page-9 | If you do view source or look at the cached version then you'll find a link right at the bottom left corner. We have 268 of these links from 200 domains. Contacting these sites to have these links removed would be a very long process as most of them probably have no idea that those links even exist and I don't have the time to explain to each one how to remove the hacked files etc. I've been looking at using the Google Disavow tool to solve this problem but I'm not sure if it's a good idea or not. We haven't had any warnings from Google about our site being spam or having too many spam links, so do we need to use the tool? Any advice would be very much appreciated. Let me know if you require more details about our problem. <colgroup><col width="355"></colgroup>
| | | |0 -
Link quality warning from GWT and drop in keyword ranking.
So last December we saw our hard work pay off as our Panda penalty was lifted and our traffic shot back up to pre-Panda levels. Then in February we received this note: We've reviewed your site and we still see links to your site that violate our quality guidelines. Specifically, look for possibly artificial or unnatural links pointing to your site that could be intended to manipulate PageRank. Examples of unnatural linking could include buying links to pass PageRank or participating in link schemes. Since December we've lost position on 80% of our top 100 keywords. I've gone through our links and can't figure out what the problem may be. Maybe I'm not using OSE properly. We don't buy links so I'm not sure what the problem is. If someone can walk me through using OSE to see what the problem may be I would appreciate it. Our domain is http://bit.ly/rbkYkp
White Hat / Black Hat SEO | | IanTheScot0