Changing Links to Spans with Robots.txt Blocked Redirects using Linkify/jQuery
-
Hi,
I was recently penalized most likely because Google started following javascript links to bad neighborhoods that were not no-followed. The first thing I did was remove the Linkify plugin from my site so that all those links would disappear, but now I think I have a solution that works with Linkify without creating crawlable links. I did the following:
-
I blocked access to the Linkify scripts using robots.txt so that Google won't execute the scripts that create the links. This has worked for me in the past with banner ads linking to other sites of mine. At least it appears to work because those sites did not get links from pages running those banners in search console.
-
I created a /redirect/ directory that redirects all offsite URLs. I put a robots.txt block on this directory.
-
I configured the Linkify plugin to parse URLs into span elements instead of a elements and add no follow attributes. They still have an href attribute, but the URLs in the href now point to the redirect directory and the span onclick event redirects the user.
I have implemented this solution on another site of mine and I am hoping this will make it impossible for Google to categorize my pages as liking to any neighborhoods good or bad. Most of the content is UGC, so this should discourage link spam while giving users clickable URLs and still letting people post complaints about people that have profiles on adult websites. Here is a page where the solution has been implemented https://cyberbullyingreport.com/bully/predators-watch-owner-scott-breitenstein-of-dayton-ohio-5463.aspx, the Linkify plugin can be found at https://soapbox.github.io/linkifyjs/, and the custom jQuery is as follows:
jQuery(document).ready(function ($) { 2 $('p').linkify({ tagName: 'span', attributes: { rel: 'nofollow' }, formatHref: function (href) { href = 'https://cyberbullyingreport.com/redirect/?url=' + href; return href; }, events:{ click: function (e) { var href = $(this).attr('href'); window.location.href = href; } } }); 3 });
-
-
I know I was penalized because search console recorded an overnight drop in impressions/clicks of 75%. Never was a problem before. I didn't need to run Screaming Frog or anything to find links that explain the problem. I just browsed some of the user generated content and then once I realized that Google was crawling javascript generated links it all made sense.
Some of the sites people were liking to included: CheaterLand.com, PredatorsWatch.com, DirtyHomeWreckers.com, EscortBabylon.net, SugarDaddyforMe.com, GFEMonkey.com, EscortBabylon.com, CityXGuide.com, and AdultLook.com.
I think as long as Google respects the robots.txt directives I should be just fine. The redirect page itself is blocked by robots.txt so even if Google finds one of those URLs (ex: https://cyberbullyingreport.com/redirect/?url=https://moz.com/community/q/changing-links-to-spans-with-robots-txt-blocked-redirects-using-linkify-jquery) it shouldn't even follow the 302 since the robots.txt file (https://cyberbullyingreport.com/robots.txt) disallows that directory.
-
301 or 302 redirecting well if you’re going to do one you should do 302 but it’s not going to help you much unless you’re going to send the URL to a 410 when something is no followed it is the same thing as what robots.txt will do.
https://support.google.com/webmasters/forum/AAAA2Jdx3sUEbHp0yjgT6c?hl=sv
do you have a report from Google knowing that you have been penalized?
is there anyway you could run Screaming Frog and show some of these URLs that you’re talking about?
respectfully,
Tom
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Subtle line of asking links for money/service/benefits
Hello here, I am putting down a link building strategy according to the latest "good practices" and Google recommendations, but I find myself often confused. For example, I'd like to implement the technique suggested by Rand on his article below: https://moz.com/blog/headsmacking-tip-1-link-requests-in-order-confirmation-emails But if you look at the comments, a user suggests to "ask for links in exchange of discounts", and everyone there applaud him for the idea (Rand included). But, wait a second... am I the only one realizing that now days Google discourage to ask for links for "money, services, or any other kind of 'offered' benefit"? So.. where to draw the line here? Here are other examples that I am not sure are "safe" in link building: 1. Ask for links in exchange of a free Membership on a site (where usually a Membership is sold for a price) 2. Ask for links in exchange of exposure (isn't this a sort of "link exchange"?) 3. Ask for link in exchange of "anything else you can think of", even if necessarily doesn't involve money (i.e. for a "certified site badge", for a free e-book, or anything else) I'd really like to know your thoughts on this very sensitive issue. Thank you in advance to anyone for helping me to understand.
White Hat / Black Hat SEO | | fablau1 -
What sources do you use to keep on top of SEO news?
I want to try building an RSS feed of SEO news... but not wanting to find myself drowning in materials As such, looking for a short list of recommendations for keeping on top of SEO developments – the impetus is that I'm still discovering changes that happened 2, 3, even 5 years ago, and I want to try and catch these things as they happen. Thinking something actually from Google may be on the list, but some of these sources are pretty on top of things! Seroundtable.com also comes to mind. But what do you use to keep informed? Thanks 🙂
White Hat / Black Hat SEO | | ntcma1 -
Benefits of having outbound links
Are there any strengths (benefits) in having outbound links within the site regarding SEO? If linking to reputable sites, would that help increase our SEO strength or does that only work if they links back to us?
White Hat / Black Hat SEO | | WebRiverGroup1 -
Do inbound links from forums hurt our traffic?
We have a manual action against us on Google webmaster tools for unnatural links. While evaluating our back links, I noticed that forums with low page rank/domain authority are linking to us. Is this hurting us?
White Hat / Black Hat SEO | | imlovinseo0 -
Links and how they count?
We managed to get ourselves out of a penalty 6 months ago and 100 days later after the message of penalty removable we finally felt that we were moving back on track (not a lot of movement before and 50% down due to links being taken away), we have around 120 really high quality links but 95% of them are urls or the business name. Anyway we still have a couple of pages that I feel are fairly down on rankings and most of the links as mentioned above are high quality but they are either anchor text of the website name or url my main question is that when looking at my competitors I see that they have the same or less links and from much less powerful places (most I would not touch) but they seem to have a ratio of 5 - 10 % of the links are the keywords they are trying to rank for. My question is if you have 50 links from better places but they are unrelated terms such as the web site name or just urls and you have 50 links from average places but 5 - 10% are on related terms to what you are trying to rank for which ones would win out.
White Hat / Black Hat SEO | | BobAnderson0 -
What are some of the worst links that you have come across?
I'm talking the least relevant and incredibly spammy. We've all done site audits and stumbled across some ridiculous ones. The funnier the better. I'm compiling a list of hilarious links that sites have gotten. Any input would be great!
White Hat / Black Hat SEO | | KevinBloom0 -
How to Remove Unwanted Links
I dropped like a rock in Google rankings on the 24<sup>th</sup>
White Hat / Black Hat SEO | | rdominey
of April. After having to become familiar with Google webmaster tools and doing
allot of investigating I discovered that there is a website www.siteloki.com that has 6,742 links to my website. I have
tried to contact siteloki with no response. I tracked them on Whois to an
office suite in LA called the building to find that the suite listed is the
building management suite. I have had
the following sent to them via email, their contact page and posted on their website
forum and still no reply: Please take action to remove all links to this website
immediately! I have been notified by my client that your website has a
malicious attack using links from www.siteloki.com
against www.getyourphotosoncanvas.com. My client did not solicit these links, pay for these links or authorize any
third party to build links for them. They just appeared. The links are even
pointing to my client’s old website (same url). This is a big problem and I
don’t understand why these links exist. There are currently 6,471 links from
your domain. Please remove these links immediately or we will consider legal
action against your company. We have contacted Google on the behalf of our
client and informed them of this malicious act. I expect to see these links
removed immediately! Regards, I have submitted the site in the malware reporting section
of webmasters tools. I have searched but cannot find any documentation on how
to block this type of attack. It seems that Google failed to provide any means
for an honest website owner following the rules to block this type of attack and
as a result we have been unjustly penalized by Google with a drop to the bottom
in our page ranking. I would appreciate ANY HELP in removing these links and getting the Siteloki website blocked from linking to my website? Any Ideas?0 -
New website :301 redirection of a established domain
Hello , I am launching a new website which would host user generated content . Based on my brandname i have purchased a new domain . In order to improve SEO rankings i was considering to purchase a good quality domain (have gr8 link backs) and then perform 301 redirection of the domain to the new brandname.co.in domain . Does this work ? Is there any harm in doing this ? . Does the Link juice pass naturally ? Warm Rgd
White Hat / Black Hat SEO | | ShoutOut0