Need suggestion: Should the user profile link be disallowed in robots.txt
-
I maintain a myBB based forum here. The user profile links look something like this http://www.learnqtp.com/forums/User-Ankur
Now in my GWT, I can see many 404 errors for user profile links. This is primarily because we have tight control over spam and auto-profiles generated by bots. Either our moderators or our spam control software delete such spammy member profiles on a periodic basis but by then Google indexes those profiles.
I am wondering, would it be a good idea to disallow User profiles links using robots.txt?
Something like
Disallow: /forums/User-*
-
Maybe you should noindex user profile pages until they've been active on the site long enough that a) you're sure they're not a spam bot and b) they're actually active users, with some interesting information to share on their user page?
Like Shaliendra said, this is your call. It's really about your website goals, and what those user pages do for the site. Are you getting much organic traffic from user pages? If you aren't, it's probably easiest to just noindex all user pages. But if some profiles are bringing in organic traffic, I'd recommend that you add a noindex tag immediately, but remove it once someone has commented a few times.
Does that help?
-
I am only talking about user profile pages not the links inside them. I have already taken care of them.
Request others to please come up with suggestions.
-
If you are talking about user profile pages, it is your call.
If you are talking about links that users post in profile pages, use as signatures or use in threads, make sure to mark them nofollow.
Regards
-
Let me re-phrase my question a bit. Would it be a good idea to get the user profile links noindexed in the first place? or should we keep them indexed in Google?
-
Use meta="noindex" tag for the user profile pages. It is more effective.
Regards
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Are there detrimental effects of having multiple robot tags
Hi All, I came across some pages on our site that have multiple robot tags, but they have the same directives. Two are identical while one is for Google only. I know there aren't any real benefits from having it set up this way, but are there any detrimental effects such as slowing down the bots crawling these pages? name="googlebot" content="index, follow, noodp"/> Thanks!
On-Page Optimization | | STP_SEO0 -
How defined does the anchor text need to be for a domain url?
Ok, I'm looking to clean up my domain with irrelevant Anchor text linking to specific URL's. Whether, it's my root domain or a particular product URL. I'm finding a wide variety of terms (anchor text) pointing to my homepage, category and finally my product pages. Example, the Anchor text "Paragon Print Systems" is pointing to my homepage "barcodefactory.com" does this hurt my homepage enough to either have the link/anchor text removed from linking to my homepage? thanks much!, Warren
On-Page Optimization | | Warren.W0 -
To Many Links & Long Titles
One of our clients currently has a number of pages that "Too many on page links", Now her menu in itself has the majority. With it being an eComerce website it has quite a few categories. How harmfull is this to search rankings. The other side of thing's, Currently the client has a large number of pages where the title of the page is the product, However her products for example could be "The pink fluffy bear 2x4 with extendable arm and voice activation ( batteries not included )" Again, how Harmfull is having a large volume of pages named like this.
On-Page Optimization | | bmkdigital0 -
Need SEO Advice for different languages and How to optimize the page
Hello. Currently we have an ecommerce site that sells shoes online in France. The site is in English. Say the keyword is shoes and our page is example.com/shoes it;s perfectly fine to optimize that page for keyword of "shoes". Now, what about french keyword of shoes "chaussures" ? How do i optimize it? Should i create a different page or just edit the same page with keyword of "chaussures" in title tag etc? What is the best way to go about doing this? Thanks
On-Page Optimization | | WayneRooney0 -
When You Add a Robots.txt file to a website to block certain URLs, do they disappear from Google's index?
I have seen several websites recently that have have far too many webpages indexed by Google, because for each blog post they publish, Google might index the following: www.mywebsite.com/blog/title-of-post www.mywebsite.com/blog/tag/tag1 www.mywebsite.com/blog/tag/tag2 www.mywebsite.com/blog/category/categoryA etc My question is: if you add a robots.txt file that tells Google NOT to index pages in the "tag" and "category" folder, does that mean that the previously indexed pages will eventually disappear from Google's index? Or does it just mean that newly created pages won't get added to the index? Or does it mean nothing at all? thanks for any insight!
On-Page Optimization | | williammarlow0 -
Internal Linking
I am trying to figure out internal linking. Please help me. Your "root domain" (the top level example.com) is the easiest to rank on a SERP. When you build Page Rank on this page, you want to make sure the majority of the PR goes into internal pages that matter. To do this you determine what internal pages are most important and put them on the menu bar. You then link to these pages in the body text, or via side bars. This will ensure that the PR is flowing from the root domain into the internal pages multiple times. The second part is to link from these secondary pages back to the main page. Correct? When you build back links on the internal page, you want to pass the PR back to the main page... Please discuss this...
On-Page Optimization | | JML11790 -
Internal Linking
Okay, if I write 10 different articles on my site that are related to one of my products. Can I safely have a link in every article to my product?
On-Page Optimization | | azguy0
