Changing Links to Spans with Robots.txt Blocked Redirects using Linkify/jQuery
-
Hi,
I was recently penalized most likely because Google started following javascript links to bad neighborhoods that were not no-followed. The first thing I did was remove the Linkify plugin from my site so that all those links would disappear, but now I think I have a solution that works with Linkify without creating crawlable links. I did the following:
-
I blocked access to the Linkify scripts using robots.txt so that Google won't execute the scripts that create the links. This has worked for me in the past with banner ads linking to other sites of mine. At least it appears to work because those sites did not get links from pages running those banners in search console.
-
I created a /redirect/ directory that redirects all offsite URLs. I put a robots.txt block on this directory.
-
I configured the Linkify plugin to parse URLs into span elements instead of a elements and add no follow attributes. They still have an href attribute, but the URLs in the href now point to the redirect directory and the span onclick event redirects the user.
I have implemented this solution on another site of mine and I am hoping this will make it impossible for Google to categorize my pages as liking to any neighborhoods good or bad. Most of the content is UGC, so this should discourage link spam while giving users clickable URLs and still letting people post complaints about people that have profiles on adult websites. Here is a page where the solution has been implemented https://cyberbullyingreport.com/bully/predators-watch-owner-scott-breitenstein-of-dayton-ohio-5463.aspx, the Linkify plugin can be found at https://soapbox.github.io/linkifyjs/, and the custom jQuery is as follows:
jQuery(document).ready(function ($) { 2 $('p').linkify({ tagName: 'span', attributes: { rel: 'nofollow' }, formatHref: function (href) { href = 'https://cyberbullyingreport.com/redirect/?url=' + href; return href; }, events:{ click: function (e) { var href = $(this).attr('href'); window.location.href = href; } } }); 3 });
-
-
I know I was penalized because search console recorded an overnight drop in impressions/clicks of 75%. Never was a problem before. I didn't need to run Screaming Frog or anything to find links that explain the problem. I just browsed some of the user generated content and then once I realized that Google was crawling javascript generated links it all made sense.
Some of the sites people were liking to included: CheaterLand.com, PredatorsWatch.com, DirtyHomeWreckers.com, EscortBabylon.net, SugarDaddyforMe.com, GFEMonkey.com, EscortBabylon.com, CityXGuide.com, and AdultLook.com.
I think as long as Google respects the robots.txt directives I should be just fine. The redirect page itself is blocked by robots.txt so even if Google finds one of those URLs (ex: https://cyberbullyingreport.com/redirect/?url=https://moz.com/community/q/changing-links-to-spans-with-robots-txt-blocked-redirects-using-linkify-jquery) it shouldn't even follow the 302 since the robots.txt file (https://cyberbullyingreport.com/robots.txt) disallows that directory.
-
301 or 302 redirecting well if you’re going to do one you should do 302 but it’s not going to help you much unless you’re going to send the URL to a 410 when something is no followed it is the same thing as what robots.txt will do.
https://support.google.com/webmasters/forum/AAAA2Jdx3sUEbHp0yjgT6c?hl=sv
do you have a report from Google knowing that you have been penalized?
is there anyway you could run Screaming Frog and show some of these URLs that you’re talking about?
respectfully,
Tom
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Back links issue and how to resolve it
Hi there! We have a client who has been generating back links from external sites over a period of two years with all the same anchor text which all link back to the home page. This anchor text is also their main search phrase they wish to score highly on. In total, they have roughly 300 domain names linking to their site. Over 50 of these domain names all have the same anchor text. These links have been generated through articles and blogs. So roughly 20% of the total number of links all have the same anchor text. Over the past 6 months the client has noticed a steady drop in their rankings for this term. From the back link analysis we have done, we believe it is this which is causing the problem. Does any one else agree? For the remedy, do we go in and see if we can change the anchor text or disavow them through Google webmaster tools? Suggestions? Thanks for your help! P 🙂
White Hat / Black Hat SEO | | Globalgraphics0 -
Mobile Redirect - Cloaking/Sneaky?
Question since Google is somewhat vague on what they consider mobile "equivalent" content. This is the hand we're dealt with due to budget, no m.dot, etc, responsive/dynamic is on the roadmap but still a couple quarters away but, for now, here's the situation. We have two sets of content and experiences, one for desktop and one for mobile. The problem is that desktop content does not = mobile content. The layout, user experience, images and copy aren't the same across both versions - they are not dramatically different but not identical. In many cases, no mobile equivalent exists. Dev wants to redirect visitors who find the desktop version in mobile search to the equivalent mobile experience, when it exists, when it doesn't they want to redirect to the mobile homepage - which really isn't a homepage it's an unfiltered view of the content. Yeah we have push state in place for the mobile version etc. My concern is that Google will look at this as cloaking, maybe not in the cases where there's a near equivalent piece of content, but definitely when we're redirecting to the "homepage". Not to mention this isn't a great user experience and will impact conversion/engagement metrics which are likely factors Google's algorithm considers. What's the MOZ Community say about this? Cloaking or Not and Why? Thanks!
White Hat / Black Hat SEO | | Jose_R0 -
Regular links may still
Good day: I understand guest articles are a good way to pass linkjuice and some authors have a link to their website on the "Author Bio" section of the article. These links are usually regular links. However, I noticed that some of these sites (using wordpress) have several SEO plugins with the following settings: Nofollow: Tell search engines not to spider links on this webpage. My question is: If the setting above was activated, I would assume the author's website link would look like a regular link but some other code could still be present in the page (ex, header) that would prevent this regular link from being followed. Therefore, the guest writer would not experience any linkjuice. Any idea if there's a way of being able to see if this scenario is happening? What code would we look for?
White Hat / Black Hat SEO | | Audreythenurse0 -
Google URL Shortener- Should I use one or multiple???
I have a client with a number of YouTube videos. I'm using Google URL Shortner to allow the link to show in the YouTube text (as its a long URL). Many of these links go to the same page ex .com/services-page Should I use a single short URL for each video linking to the .com/services-page or should they be unique each time? If unique, would Google possibly think I'm trying to manipulate results? Thanks in advance. I'm just not sure on this one and hope someone knows best practice on this. Thanks!
White Hat / Black Hat SEO | | mgordon1 -
Why should I reach out to webmasters before disavowing links?
Almost all the blogs, and Google themselves, tell us to reach out to webmasters and request the offending links be removed before using Google's Disavow tool. None of the blogs, nor Google, suggest why you "must" do this, it's time consuming and many webmasters don't care and don't act. Why is this a "required" thing to do?
White Hat / Black Hat SEO | | RealSelf0 -
Link Audit: How do I decide what is a good or bad link?
I am conducting a link audit for one of my formerly high-ranking pages. But despite reading quite a bit on the issue, I am still quite confused as to how to decide whether to keep or remove a link. Some links come from directories and social bookmarking sites. I know that generally speaking, you do not want to be on these types of sites, but what if their domain authorities, pageranks, and mozTrusts scores are good? For example, here is one of my links for "envelopes": http://www.folkd.com/detail/www.jampaper.com%2FEnvelopes The page itself has no MozRank, MozTrust, or links but the domain has an authority of 88, a MozRank of 6.41, a mozTrust of 6.31. Should I be looking on a page level or domain level basis? It also has over 5 million links, with over two million of those being external followed links. Is the high quantity of links a warning sign? I also used a free online tool (thesitevalue.com) to determine how much traffic the domain gets. Apparently it receives over 350,000 unique visits daily, so it must be useful to people. This, combined with the fact that we've received 5 visits from the link over the last year (not a lot, but something), makes me believe that the link's intent wasn't purely to "trick" Google. Despite this, I still have a feeling the link could be considered low-quality based on the domain's appearance. Similarly, some of our links are coming from domains named linkdirect.info, backlinks8.com, tolinkup.com, findyourlink.info, searchengineurl.com, websubmissionfree.com. Is it safe to assume these are harmful links strictly because of their names? Thank you!
White Hat / Black Hat SEO | | jampaper0 -
2 Questions about 301 Redirects
So I have a couple of questions about 301 redirects: Do Google penalties EVER pass through a 301? I've done 20+ domain 301s in the last year and have yet to see it happen, but the other day I read a an article (or maybe it was a QA post?) that suggested doing 302s to avoid transferring penalties. Has anyone seen any authoritative information regarding this? I 301'd a domain in February that another SEO firm had built a lot of spammy links and I began building contextual links for it at a very slow rate (like 10 or so a month). Within a month, my domain authority was a 26 on the new domain and my inbound links were non existent. By month 2, my links were 70k and domain authority was 34. By month 3, down to 25k inbound links and domain authority of 29, where it has settled for the last 3 months despite some really high quality links. My question (don't worry it's coming), is does anyone have any clue why my links shot up so quickly and then dropped? I'm assuming the 301 links kicked in and then only about 45% ended up 'sticking'?? Thanks in advance
White Hat / Black Hat SEO | | BrianJGomez0 -
Powered by/Credit backlinks and nofollow
Pseudo question: I have a website that has 100K pages. On about 50K of those pages I have information that is fed to me via an outside 3rd-party website. Now, I like to give credit where credit is due, so I add a backlink to the website that is feeding me this content. A simple backlink like so: Information provided by: Company ABC Now, this 3rd-party website wants me to remove the nofollow tags from the backlink, but I am very, very skeptical because to me, sending ~50K dofollow backlinks to a single site might make the Google monster upset with me. This 3rd-party site is being very hard-headed about this, to the point where I am thinking of terminating the relationship all together. I digress. Scoured the net before writing this, but couldn't really find anything directly related to my issue. Thoughts? Is a nofollow required here? We're not talking 1 or 2 links here; we're talking tens of thousands (50K is low; it will probably be upwards of 100K when all is said and done as my site has many, many pages). Thanks in advance.
White Hat / Black Hat SEO | | THB0