Need suggestion: Should the user profile link be disallowed in robots.txt
-
I maintain a myBB based forum here. The user profile links look something like this http://www.learnqtp.com/forums/User-Ankur
Now in my GWT, I can see many 404 errors for user profile links. This is primarily because we have tight control over spam and auto-profiles generated by bots. Either our moderators or our spam control software delete such spammy member profiles on a periodic basis but by then Google indexes those profiles.
I am wondering, would it be a good idea to disallow User profiles links using robots.txt?
Something like
Disallow: /forums/User-*
-
Maybe you should noindex user profile pages until they've been active on the site long enough that a) you're sure they're not a spam bot and b) they're actually active users, with some interesting information to share on their user page?
Like Shaliendra said, this is your call. It's really about your website goals, and what those user pages do for the site. Are you getting much organic traffic from user pages? If you aren't, it's probably easiest to just noindex all user pages. But if some profiles are bringing in organic traffic, I'd recommend that you add a noindex tag immediately, but remove it once someone has commented a few times.
Does that help?
-
I am only talking about user profile pages not the links inside them. I have already taken care of them.
Request others to please come up with suggestions.
-
If you are talking about user profile pages, it is your call.
If you are talking about links that users post in profile pages, use as signatures or use in threads, make sure to mark them nofollow.
Regards
-
Let me re-phrase my question a bit. Would it be a good idea to get the user profile links noindexed in the first place? or should we keep them indexed in Google?
-
Use meta="noindex" tag for the user profile pages. It is more effective.
Regards
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Do links in footers or side bars count less than links in the center of the web page?
do links in footers or side bars count less than links in the center of the web page? How much less if so? I have some articles on my site. Would i get more of a boost in rankings to pages of my site by placing links in the text of my articles on my site to other pages on my site? Thanks mozzers!
On-Page Optimization | | Ron100 -
Too many on page links in sitemap.html
My crawl report is flagging an issue with too many links to one of my pages, this page is my sitemap.html. However, I have coded the page so that if required is specified it generates an .xml version of the page and if not then the html version is displayed. What is the best way to stop the crawl finding the html version whilst maintaining it on the site for clients navigation?
On-Page Optimization | | SamPenno0 -
Several Links in Some Pages
Dear all, Our main site is a bussiness directory, and following some SEO advices, we are creating landing pages for each category, in order to optimize them for the keywords. Those landing pages have links to the listings related to them. Using the same idea, we have created pages related to the regions, and those pages include links to the listings located in them. The only problem that I see with that, is the number of links that some categories or regions could have. Is there a limit of recomended number of links per page, from a SEO perspective? We also have a main category page, that includes a list of all categories, and this page could also have a relatively high number of links. The pages have around 300 to 500 words, some include also images, some include videos. Many thanks for your help, Daniel
On-Page Optimization | | te_c0 -
Do I need canonical link on target page?
I've placed in my head tag on duplicate content pages, but do i need to place it on the target page such as http://www.example.com/index.html too?
On-Page Optimization | | CaliB0 -
Blocked By Meta Robots
Hi I logged in the other day to find that over night I received 8347 notices saying certain pages are being kept out of the search engine indexes by meta-robots. I have not changed my robots.txt in years and I certainly didn't block Google from visiting those pages. Is this a fault on Roger Mozbot behalf? Or is there a bot preventing 8000+ pages being indexed? Is there a way to find out what meta-robot is doing this and where? And how I can get rid of it? I usually rank between #3 and #5 for the term 'sex toys' on google.com.au, but I now rank #7 to #9 so it would seem some of my pages/content is being blocked. My website is www.theloveshop.com THIS IS AN ADULT TOYS SITE. There is no porn videos or anything like that on it, but just in case you don't wish to look at sex toys or are around kids I thought I would mention it. Blake
On-Page Optimization | | wayne10 -
Canonical links
My website is relatively new, January. We climbed steadily to 6th for our search term then overnight rocketed to 1st. This only lasted a week and have been stuck at 9th ever since. When I use the SEO Moz tools our site should theoretically be top...I only joined today btw. Anyway in Google webmaster tools I noticed it said I had duplicate title tags, when I checked to see what the pages were- it was my home page! Google also seems to have cached two versions of our homepage, the root domain and the Default.aspx page. Now I have fixed this canonical linking issue today (using canonical link tag and 301s) so time will tell but has anyone got any first hand experience of this issue? Was it a big factor? Thanks!
On-Page Optimization | | SplashBacksNI0 -
Do footer links apply too many on-page links?
We tend to put a a lot of links in the footers of some of our websites (e.g. www.AlohaWhistler.com). Our CAMPAIGNS report is showing that several pages on such sites have "too many on-page links". We understand the logic that having more than 100 links per page is "too much". Does this also apply to footer links?
On-Page Optimization | | RoyMcClean0 -
How do you block development servers with robots.txt?
When we create client websites the urls are client.oursite.com. Google is indexing theses sites and attaching to our domain. How can we stop it with robots.txt? I've heard you need to have the robots file on both the main site and the dev sites... A code sample would be groovy. Thanks, TR
On-Page Optimization | | DisMedia0