Need suggestion: Should the user profile link be disallowed in robots.txt
-
I maintain a myBB based forum here. The user profile links look something like this http://www.learnqtp.com/forums/User-Ankur
Now in my GWT, I can see many 404 errors for user profile links. This is primarily because we have tight control over spam and auto-profiles generated by bots. Either our moderators or our spam control software delete such spammy member profiles on a periodic basis but by then Google indexes those profiles.
I am wondering, would it be a good idea to disallow User profiles links using robots.txt?
Something like
Disallow: /forums/User-*
-
Maybe you should noindex user profile pages until they've been active on the site long enough that a) you're sure they're not a spam bot and b) they're actually active users, with some interesting information to share on their user page?
Like Shaliendra said, this is your call. It's really about your website goals, and what those user pages do for the site. Are you getting much organic traffic from user pages? If you aren't, it's probably easiest to just noindex all user pages. But if some profiles are bringing in organic traffic, I'd recommend that you add a noindex tag immediately, but remove it once someone has commented a few times.
Does that help?
-
I am only talking about user profile pages not the links inside them. I have already taken care of them.
Request others to please come up with suggestions.
-
If you are talking about user profile pages, it is your call.
If you are talking about links that users post in profile pages, use as signatures or use in threads, make sure to mark them nofollow.
Regards
-
Let me re-phrase my question a bit. Would it be a good idea to get the user profile links noindexed in the first place? or should we keep them indexed in Google?
-
Use meta="noindex" tag for the user profile pages. It is more effective.
Regards
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
ASP Canonical and Internal Linking
Hello - I'm working with a large ASP website and trying to troubleshoot issues I believe might be related to how the canonical element is used. On page - all internal links, including navigation links, use the following format (uppercase) - website.com**/F**older/Folder/Product . So, any page navigated to will always display the uppercase version of the URL. And, all of these pages have the canonical tag pointing to the lowercase version of the URL. The pages included in Google's index are all lowercase versions of the URL like this - website.com**/f**older/folder/product . My concern is that a lot of internal authority flow is being impacted/negated because all internal links point to the uppercase versions of URLs and all those pages reference the lowercase version URL in the canonical reference. Is this a valid concern?
On-Page Optimization | | LA_Steve0 -
Homepage canonical url with splash or not with splash? All other links are without but logo links with splash
Hello, There is so much contradicting information about the homepage canonical URL. Many websites have all the links without the trailing splash but their homepage URL still contains the splash. Now Moz is an example with this. Their urls don't have the splash, and their canonical does not have the splash. Why is it so and why so much different ways people have it?
On-Page Optimization | | advertisingcloud0 -
Query string parameters and canonical links
Hello everyone, My site uses query string parameters in a few places to manage tasks like pagination of lists. Eg: http://www.example.com/destinations/somewhere?page=2 I have set a canonical link with the href of the page without the query string but still getting thousands of duplicate title/meta description reports from these pages. Is there something I can do to change this? Do search engines actually penalise for use of query string parameters like this? They seem so commonplace, even for sites which use an absolute URI with no query string to serve content. Thanks 🙂
On-Page Optimization | | JHWXS0 -
Lists of Product Links: What is good, what is bad?
I am a web designer but a bit of an SEO noob (trying to get better at both). I am working with one particular client on a site I inherited with existing structure. This client has about 10 products on 2 pages. On every page there is a product list that is basically the same list sorted in 2 ways: 1st by product, 2nd by usage. These all link to internal anchors so this might be an example on www.site.com Cleaner X1 - links to www.site.com/cleaners.php#x1
On-Page Optimization | | mparry9
Cleaner X2 - links to www.site.com/cleaners.php#x2
Cleaner X3 - links to www.site.com/cleaners.php#x3
...
Cleaner For Brick - links to www.site.com/cleaners.php#x1
Cleaner For Marble - links to www.site.com/cleaners.php#x2
Cleaner For Stone - links to www.site.com/cleaners.php#x3 Obviously this adds about 20 links on every page on the site (including the actual pages these products are on). What are your thoughts on this? Good idea or bad to have on the site? Should I remove the redundant links on the actual page that product falls on...or is this bad and should be removed altogether?0 -
Why does the on page report reports a full path link as Cannibalize link?
On the seomoz on page report i get a cannibalize error. This is due to a link being full path. When i change the link to relative path then there is no Cannibalize error. Should i change the internal links of the site to relative path? I would appreciate your help.
On-Page Optimization | | pickaweb0 -
Too many on-page links
Hi, My website - www.thepartyhouse.com.au is showing as having too many on-page links for over 4,000 pages. Take for example the homepage which is showing as 188 links, but I don't understand this because I've used SEO tools to display the links and I am showing around 90 links on this page. How can I see what all the links are? Thanks
On-Page Optimization | | Spyre0 -
Optimise duplicate products or canonical link
We exist in a niche market with a good % of products that sell well at specific times of the year. Lets say for example a red cup can be sold as a christmas red cup and a valentine red cup or just a red cup. Would we be best to optimize each specific product specifically for those seasons/events on different pages or keep google pointed to just one page using a canonical link.
On-Page Optimization | | LadyApollo0 -
Max # of recommended links per page?
I've heard it said that Google may choose to stop following links after the first 100 on a page. The landing/category pages for my site's product catalog have earned quite a respectable PR and positioning in search results, and I'm currently paginating their product listings (about 200 products in a category) so that only a couple dozen products are shown on the first page, with links to "next page" and "previous page" being accomplished via query string (i.e. "?page=3"). An alternative option I have is to link to 100% of the contained products within the category's landing page (which would increase my on-page link count to ~300) and use CSS/Javascript to allow the user to simulate browsing between pages on the client side. My goal is to see as many of my product pages indexed as possible. Is this done better using my current scheme (where Googlebot would have to navigate to, say, Landing Page -> Page 6 -> Deeply Buried Product Page) or in the alternative method above, where all the links are in a single page? Since my landing pages are currently treated pretty well by search engines, would that "trust" cause them to follow more links than might normally be done? Thank you!
On-Page Optimization | | cadenzajon0