Changing Links to Spans with Robots.txt Blocked Redirects using Linkify/jQuery
-
Hi,
I was recently penalized most likely because Google started following javascript links to bad neighborhoods that were not no-followed. The first thing I did was remove the Linkify plugin from my site so that all those links would disappear, but now I think I have a solution that works with Linkify without creating crawlable links. I did the following:
-
I blocked access to the Linkify scripts using robots.txt so that Google won't execute the scripts that create the links. This has worked for me in the past with banner ads linking to other sites of mine. At least it appears to work because those sites did not get links from pages running those banners in search console.
-
I created a /redirect/ directory that redirects all offsite URLs. I put a robots.txt block on this directory.
-
I configured the Linkify plugin to parse URLs into span elements instead of a elements and add no follow attributes. They still have an href attribute, but the URLs in the href now point to the redirect directory and the span onclick event redirects the user.
I have implemented this solution on another site of mine and I am hoping this will make it impossible for Google to categorize my pages as liking to any neighborhoods good or bad. Most of the content is UGC, so this should discourage link spam while giving users clickable URLs and still letting people post complaints about people that have profiles on adult websites. Here is a page where the solution has been implemented https://cyberbullyingreport.com/bully/predators-watch-owner-scott-breitenstein-of-dayton-ohio-5463.aspx, the Linkify plugin can be found at https://soapbox.github.io/linkifyjs/, and the custom jQuery is as follows:
jQuery(document).ready(function ($) { 2 $('p').linkify({ tagName: 'span', attributes: { rel: 'nofollow' }, formatHref: function (href) { href = 'https://cyberbullyingreport.com/redirect/?url=' + href; return href; }, events:{ click: function (e) { var href = $(this).attr('href'); window.location.href = href; } } }); 3 });
-
-
I know I was penalized because search console recorded an overnight drop in impressions/clicks of 75%. Never was a problem before. I didn't need to run Screaming Frog or anything to find links that explain the problem. I just browsed some of the user generated content and then once I realized that Google was crawling javascript generated links it all made sense.
Some of the sites people were liking to included: CheaterLand.com, PredatorsWatch.com, DirtyHomeWreckers.com, EscortBabylon.net, SugarDaddyforMe.com, GFEMonkey.com, EscortBabylon.com, CityXGuide.com, and AdultLook.com.
I think as long as Google respects the robots.txt directives I should be just fine. The redirect page itself is blocked by robots.txt so even if Google finds one of those URLs (ex: https://cyberbullyingreport.com/redirect/?url=https://moz.com/community/q/changing-links-to-spans-with-robots-txt-blocked-redirects-using-linkify-jquery) it shouldn't even follow the 302 since the robots.txt file (https://cyberbullyingreport.com/robots.txt) disallows that directory.
-
301 or 302 redirecting well if you’re going to do one you should do 302 but it’s not going to help you much unless you’re going to send the URL to a 410 when something is no followed it is the same thing as what robots.txt will do.
https://support.google.com/webmasters/forum/AAAA2Jdx3sUEbHp0yjgT6c?hl=sv
do you have a report from Google knowing that you have been penalized?
is there anyway you could run Screaming Frog and show some of these URLs that you’re talking about?
respectfully,
Tom
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Buying a blog that already has links and authority
Hello, I hope all are well and keeping safe. I am about to launch a brand new website and am looking to get a slight boost of links to get started. I looked on Fiverr, but realise that this may not be the best option. I have found a blog for sale that matches with a section of my niche. SO have a couple of questions: 1. Is it still worth buying blogs and doing a 301? 2. If yes, would it still work if i 301 these to a category of my main blog? 3. According the moz tool, it has a spam score of 2%, I am attaching an image of the spam score from moz. I am a bit lost and would appreciate any feedback on best practice or if anyone has tried something similar. Many thanks Shehzad O92dqn7
White Hat / Black Hat SEO | | shehzad0 -
Inbound Links - Redirect, Leave Alone, etc
Hi, I recently download the inbound links report for my client to look for some opportunities. When they switched to our platform a couple years ago, the format of some of their webpages change, so a number of these inbound links are going to an error page and should be redirected. However, some of these are spammy. In that case, someone recommended to me to disavow them but still redirect anyway. In other cases, some were "last seen" a year or two ago, so when I try to go to the URL the link is coming from, I also get an error page. Should I bother to redirect in these cases? Should I disavow in both cases? Or leave them alone? Thanks for any input!
White Hat / Black Hat SEO | | AliMac261 -
What tools do you use to find scraped content?
This hasn’t been an issue for our company so far, but I like to be proactive. What tools do you use to find sites that may have scraped your content? Looking forward to your suggestions. Vic
White Hat / Black Hat SEO | | VicMarcusNWI0 -
Domain change after over optimisation
Hi Guys, Wanted to get some second opinions. I have a client who worked with someone on their SEO for about 15months, during which time they reaped the benefits of being top of Googles organic rankings for their main keyword. Unfortunately, the link building that was completed was terrible with no care taken for future updates, over optimised anchor text links on totally irrelevant websites. Contacting webmasters in order for links to be taken down was an impossible task, these websites were pure spam sites which I doubt anyone oversees with any care. So went the disavow route, it caused a slight increase, but the way Google indexes the website and its pages is still all over the place. We have a good new content marketing plan in place for the next 12 months, and already have some great in depth articles, videos and interactive infographics created, but the problem is my client really needs some quick wins and cannot risk the domain never recovering fully from the issues. They own the .com version of the .co.uk they have so my thoughts at this point are to move them to the .com, and the new content strategy can be worked on as if its a new project. For the keywords they target there are sites on the 1st and 2nd page with DA of 16-18, so it wont take long with the new content we have to get them back to a level playing field with the competition. The one worry I do have is Google seeing the .co.uk go down, and the .com appear with the same website on and looking down on the "start from scratch" method.... does anyone know of any examples when they have acted on this? We started an AdWords account (very small spend) and this has been going to the .co.uk, if we start a new one, with same billing info going to the .com could that also cause problems? So basically, because of my clients situation, and the level the competition is it, moving to the .com seems like the best move, its just whether there is any risk involved. I've never read of there being any, but thought I'd get some opinions before comitting to it. By the way, I'm only seeing this as a feasible option as they can still keep branded traffic due to buying both domains at the same time. Thanks in advance.
White Hat / Black Hat SEO | | gamnaking10 -
Unwanted link ?
Hello Working on my 404 pages, I've just found the following http://awesomescreenshot.com/08d22txtc9 This website http://basilurteaindia.com has a link mine as checked into Google. Link is presented with some of my content here http://basilurteaindia.com/images/19022012list.asp?type=2&file=C%3A%5CProgram+Files+(x86)%5ChMailServer%5CData%5Cace-egy.com%5Cm.kilany%5C9A%5C%7B9A532C2F-FB00-4C72-9403-7F26B7DC8E54%7D.eml Does someone know what the hell is that and how to remove it ?
White Hat / Black Hat SEO | | AymanH0 -
What are some of the worst links that you have come across?
I'm talking the least relevant and incredibly spammy. We've all done site audits and stumbled across some ridiculous ones. The funnier the better. I'm compiling a list of hilarious links that sites have gotten. Any input would be great!
White Hat / Black Hat SEO | | KevinBloom0 -
Buying a website and redirecting everything
We are considering purchasing an existing website in our industry with a domain authority of 52 and 20K inlinks and redirecting it to our new website with a domain authority of 26 and 1,000 inlinks. Would this be the best way to improve our new site's authority and inlinks? Would Google penalize us for doing that or would it effectively transfer the old sites authority to us?
White Hat / Black Hat SEO | | pbhatt0 -
Too many links with the same Anchor-text?
My first question at SeoMoz: Recently my gambling site has been experimenting a subtle yo-yo effect for our most sought-after keyword. A month ago we legitimately added a PR-6 inbound link with that keyword (tragamonedas) from an institutional site of our own development. We are worried that google might have regarded that move as an illegitimate link acquisition, since those apparent troubles with our keyword appear to have started right after that link was processed. Is it too late to change the anchor text, in case that action might deliver positive results? Also, we might have focused too much on the very same keyword in our link building campaign. Can a constant repetition of the same anchor harm our indexing reputation? Thank you in advance and good SEO luck, Andi.
White Hat / Black Hat SEO | | castano0