Adding the link masking directory to robots.txt?
-
Hey guys,
Just want to know if you have any experience with this.
Is it worthwhile blocking search engines from following the link masking directory.. (what i mean by this is the directory that holds the link redirectors to an affiliate site:
example:
mydomain.com/go/thislinkgoes to
amazon.com/affiliatelinkI want to know if blocking the 'go' directory from getting crawled in robots.txt is a good idea or a bad idea?
I am not using wordpress but rather a custom built php site where i need to manually decide on these things.
i want to specifically know if this in any way violates guidelines for google. it doesn't change the custom experience because they know exactly where they will end up if they click on the link.
any advice would be much appreciated.
-
Iredeto,
I do this on a few of my sites and it works out well. Saves on crawl budget, keeps google from accessing my affiliate links, and keeps any pagerank from passing through the links, which keeps me in-line with Google's webmaster policy on links.
One thing to keep in mind is that Google may show those URLs in the search results with a message that they can't show the content because it has been blocked. If you want to keep this from happening you may have to remove that directory GWT using the URL Removal tool. If they get re-indexed in 90 days (or whatever the reset time frame is) you will have to do it again. Hopefully that won't be an issue once you get them all removed the first time and block the folder.
Using rel = "nofollow" tags on the hyperlinks going to that directory wouldn't hurt either.
-
Hi!
I know cases where you can hide the contents of a folder to Google to not affect the rest of the website.
Note that you are facing a machine that knows how to crawl and index content.
Also be sure to block with noindex nofollow meta tags and removing URLs from Google index if at some point you had not locked.
Best regards!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to take down a sub domain which is receiving many spammy back-links?
Hi all, We have a sub domain which has less engagement for last few years. Eventually many spammy back links pointed to this sub domain. There are relevant back links too. We have deleted most of the pages which are employing spammy content or which have spammy back links. Still I'm confused whether to take this sub domain down or keep it. The confusion between "relevant backlinks might be helping our website" and "spammy backlinks are affecting to drop in rankings"? Thanks
Algorithm Updates | | vtmoz0 -
Domain location is a ranking factor? Back links & website?
If a website trying to rank in US and it has received many back-links from domains hosting from other countries; how it will impact website ranking? Can a website hosted in country will rank well in other country? How much the hosted location matters? Like....domain hosted in Germany but trying to rank in US?
Algorithm Updates | | vtmoz0 -
How to you get a company details to appear on the right side of google searches? For example is you type dr martens in google , the search page shows links and then a snippet of the company on the right. Thank you.
For example is you type dr martens in google , the search page shows links and then a snippet of the company on the right. Thank you. https://www.google.com/search?q=dr+martens&oq=dr+martens&aqs=chrome..69i57j0l5.2176j0j7&sourceid=chrome&espv=210&es_sm=122&ie=UTF-8
Algorithm Updates | | vijayvasu0 -
Images added to website automatically become URLs - is this an issue?
Hello Mozzers! I've just been trawling through a website and noticed all of the images had their own URLs. There's a bespoke CMS and that's how it works with images... So out of 1447 urls, 1314 are images. Firstly, is this an issue / problem from an SEO perspective. If it is, how should I deal with it? Thanks in advance, Luke
Algorithm Updates | | McTaggart0 -
Loss of 1,000 links has negatively affecting rankings
Hey there, One of the clients we're working with has lost about 1,000 or so backlinks over the last two or three months - mainly old article and directory links - and it has massively affected the site's search rankings. The site was ranking for pretty much all of its keywords in prominent positions on Google (mostly first page) but has now seen positions dive to 50, 90 and even outside of the top 100. Is there any immediate action we can put into place to help restore our rankings?
Algorithm Updates | | Webrevolve0 -
Considering the Panda algorithm updates, would you recommend reducing high amounts of inbound links from a single website?
My website has a significant number of inbound links (1,000+) from a single website, due to a sponsorship level contribution. Both my website and the other are authorities in the industry and in search results (PR of 5). Since even ethical websites can suffer a penalty from each iteration of Panda, I'm considering significantly removing the number of links from this website. Do you think that measurable change would be seen favorably by Google or would the drop in links be detrimental?
Algorithm Updates | | steelintheair0 -
How important are links after Panda
I have noticed that the sites in my niche that were at the top of the SERP's are still at the top of the SERP's after panda. I have also heard people theorizing that links are no longer important, its now all about bounce rates, time on site, etc. Is there any consensus about how important links are after Panda? thx Paul
Algorithm Updates | | diogenes1 -
Too many page links?`
Hi there This blog insert was flag suggesting there was too many page links? I cant identify the same problem? Can anyone explain?
Algorithm Updates | | footballfriends0