Need suggestion: Should the user profile link be disallowed in robots.txt
-
I maintain a myBB based forum here. The user profile links look something like this http://www.learnqtp.com/forums/User-Ankur
Now in my GWT, I can see many 404 errors for user profile links. This is primarily because we have tight control over spam and auto-profiles generated by bots. Either our moderators or our spam control software delete such spammy member profiles on a periodic basis but by then Google indexes those profiles.
I am wondering, would it be a good idea to disallow User profiles links using robots.txt?
Something like
Disallow: /forums/User-*
-
Maybe you should noindex user profile pages until they've been active on the site long enough that a) you're sure they're not a spam bot and b) they're actually active users, with some interesting information to share on their user page?
Like Shaliendra said, this is your call. It's really about your website goals, and what those user pages do for the site. Are you getting much organic traffic from user pages? If you aren't, it's probably easiest to just noindex all user pages. But if some profiles are bringing in organic traffic, I'd recommend that you add a noindex tag immediately, but remove it once someone has commented a few times.
Does that help?
-
I am only talking about user profile pages not the links inside them. I have already taken care of them.
Request others to please come up with suggestions.
-
If you are talking about user profile pages, it is your call.
If you are talking about links that users post in profile pages, use as signatures or use in threads, make sure to mark them nofollow.
Regards
-
Let me re-phrase my question a bit. Would it be a good idea to get the user profile links noindexed in the first place? or should we keep them indexed in Google?
-
Use meta="noindex" tag for the user profile pages. It is more effective.
Regards
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Header/Menu Links
Hey there, I seem to have heard mixed results on this. We have a LOT of pages that we are trying to tell google are important. Does it make that big of a difference if those pages are linked in the homepage/header menu? Or mega menu for example? We are essentially a review site, so we potentially have thousands of links that could be implemented into the menu. Thoughts are greatly appreciated!
On-Page Optimization | | HashtagHustler0 -
Automated checking for broken links within content pieces
Hi, I am wondering if anyone can send me in the right direction on a system suggestion. We have currently grown out amount of content pieces on our website and our manual checking if the links in the content pieces are still 200 status is becoming extremely time consuming. Does anyone have a recommendation of a system that will crawl your pages and check both the internal and external links within the content for a status code (404,200,etc)? Preferably something server side so it can just run on a schedule but really anything would be fine. I have tried things like Screaming frog, etc and it just doesn't seem to be the right tool.
On-Page Optimization | | GoAbroadKP0 -
Two Robots.txt files
Hi there Can somebody please help me that one of my client site have two robot.txt files (please see below). One txt file is blocked few folders and another one is blocked completely all the Search engines. Our tech team telling that due to some technical reasons they using second one which placed in inside the server and search engines unable to see this file. www.example.co.uk/robots.txt - Blocked few folderswww.example.co.uk/Robots.txt - Blocked all Search Engines I hope someone can give me the help I need in this one. Thanks in advance! Cheers,
On-Page Optimization | | TrulyTravel
Satla0 -
I have a client where every page has over 100 links
Some links are in the main navigation (it has a secondary and tertiary level) and some links are repeated in the left navigation. Every page has over 100 links if crawled. From a practical standpoint, would you (a) delete the 3rd-level links (or at least argue for that) or (b) rel='nofollow' them? From a usability standpoint, this setup works as they are almost one click from everything. From a crawl standpoint, I see some pages missed in google (the sitemap has over 200 links). Looking for the best on-page current SEO advice to set these guys on the road to success.
On-Page Optimization | | digimech0 -
Thoughts on these footer links
I have a site that has about 20 footer links. A main Category and 4-5 links under each. The site is very large, so I feel they do have some value for navigation, and they don't blend in with the background at all. I know penguin was cracking down on footer links, but I don't feel theses are "spammy" links. Will it hurt long-term to leave these links, or should we pull them?
On-Page Optimization | | netviper0 -
Seo Moz says there are 124 links on this page - do you see it?
I'm look at the report under Crawl Diagnostics Summary/Too Many On-Page Links It says there are 124 links on this page. I don't see it?! do you? http://www.kisswedding.com/blog/category/garden-wedding-ideas/
On-Page Optimization | | annasusmiles0 -
Do footer links apply too many on-page links?
We tend to put a a lot of links in the footers of some of our websites (e.g. www.AlohaWhistler.com). Our CAMPAIGNS report is showing that several pages on such sites have "too many on-page links". We understand the logic that having more than 100 links per page is "too much". Does this also apply to footer links?
On-Page Optimization | | RoyMcClean0 -
Randomly Generated Links & Passing PageRank
Hi all, If you were to have links that were randomly generated on each refresh, how would they be treated? I would imagine that they are treated normally and isolated to each crawl. So Google would just see the link structure changing from crawl to crawl and therefore give no long term value to these links. Any ideas? Thanks for your responses! Nick
On-Page Optimization | | NickPateman810