Need suggestion: Should the user profile link be disallowed in robots.txt
-
I maintain a myBB based forum here. The user profile links look something like this http://www.learnqtp.com/forums/User-Ankur
Now in my GWT, I can see many 404 errors for user profile links. This is primarily because we have tight control over spam and auto-profiles generated by bots. Either our moderators or our spam control software delete such spammy member profiles on a periodic basis but by then Google indexes those profiles.
I am wondering, would it be a good idea to disallow User profiles links using robots.txt?
Something like
Disallow: /forums/User-*
-
Maybe you should noindex user profile pages until they've been active on the site long enough that a) you're sure they're not a spam bot and b) they're actually active users, with some interesting information to share on their user page?
Like Shaliendra said, this is your call. It's really about your website goals, and what those user pages do for the site. Are you getting much organic traffic from user pages? If you aren't, it's probably easiest to just noindex all user pages. But if some profiles are bringing in organic traffic, I'd recommend that you add a noindex tag immediately, but remove it once someone has commented a few times.
Does that help?
-
I am only talking about user profile pages not the links inside them. I have already taken care of them.
Request others to please come up with suggestions.
-
If you are talking about user profile pages, it is your call.
If you are talking about links that users post in profile pages, use as signatures or use in threads, make sure to mark them nofollow.
Regards
-
Let me re-phrase my question a bit. Would it be a good idea to get the user profile links noindexed in the first place? or should we keep them indexed in Google?
-
Use meta="noindex" tag for the user profile pages. It is more effective.
Regards
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Parsing jQuery Links as Real Links
While trying to diagnose a recent Google penalty I found out that links were being parsed by Google even though they were made using jQuery. I had the linkify plugin on my site and configured it to convert URLs to links on all of my pages. Today I found links to other sites of mine from sites that should not have been linking to them and found that the links came from pages whose links were generated via jQuery. This makes me wonder, how do I know if Google is counting javascript generated links? Is it possible that my native ad widgets are creating links that Google might count? Since I don't own any of the sites that advertise via the widgets I don't know how to tell if they are getting link juice or not. It used to be that Google didn't parse javascript, so you could add as many links to your site via javascript as you wanted without being seen by Google as linking to those sites. Does anyone know of a jQuery plugin that does turn URLs into clickable links that Google won't parse as real links?
On-Page Optimization | | STDCarriers0 -
How to block index of link and content
Hi, We have pages where articles are shown and in the sides we have small snippets of Articles which shows the title and close to 25 words and a image. When i search for something in Google the snippet image and content is shown and in Google when clicked it redirects to a page which is not meant to be shown for the keyword the visitor is querying Is there a way i can block all the links and content shown in the right and left side of the page so Google does not get confused with the page content thats not related to that page? thanks
On-Page Optimization | | AlexisWithers0 -
Need help ranking my site
Hi, Can anyone help me out? I am trying to get this site ranked for "Villa General Belgrano". It was on the first page of Google and then it disappeared. Did I over optimize the anchor text? http://www.opensiteexplorer.org/anchors?site=www.lawebdelvalle.com.ar
On-Page Optimization | | Carla_Dawson0 -
Can we listed URL on Website sitemap page which are blocked by Robots.txt
Hi, I need your help here. I have a website, and few pages are created for country specific. (www.example.com/uk). I have blocked many country specific pages from Robots.txt file. It is advisable to listed those urls (blocked by robots.txt) on my website sitemap. (html sitemap page) I really appreciate your help. Thanks, Nilay
On-Page Optimization | | Internet-Marketing-Profs0 -
Internal links
Hi, I am working on a project with very few really important pages - its now about 20 pages and this number will not increase to more than 30 by the end of the year. Today the majority of these Pages is 2 clicks away from the Homepage for navigational reasons. Now I am wondering: 1. If I should set links from the Homepage directly to these Pages to pass more juice?
On-Page Optimization | | Naturalmente
2 If you would suggest so - how would you recommend me to do this in Terms of usability and SEO? thanks in advance for your help.0 -
Too Many On-Page Links
Hi All, New to SEOMoz, so thanks in advance for any answers! Looking at our Crawl Diagnostics and "Too Many On-Page Links" is first on the list. The site was build with the intention of users being able to quickly get to where they want to go with drop down menus (sub nav), so we built the navigation using bullet points/css. Yes, agreed there are too many links on each page from our navigation, main nav cats are 4 with sub nav about 40, but what is the best way to resolve the problem other then removing most of the links (from the sub nav drop down)? Could we just use the attribute rel=nofollow for the sub nav links? TIA
On-Page Optimization | | bmmedia0 -
Dropped rankings due to too many links, help needed
We have just re-designed our site and added a big drop-down navigation menu to help users get straight to the category/sub-category they are looking for, take a look at http://www.discountfiresupplies.co.uk to see what I mean. Since doing so our rankings have dropped which we've been told may be because there are now so many links on (particularly) the home page diluting the rankings of that page. We've been advised that if we hide the drop-down menus using "style='display: none'" until they are required that the search engines will ignore them which we have now done but is this correct or will they still be indexed? And if so do you have any other suggestions? Thanks, Tariq
On-Page Optimization | | tjhossy0 -
Internal link structure for large site
I am working on a very large directory site which is undergoing a complete redesign. I am considering the internal link structure from first principles. When a site has over 100,000 pages, how do you ensure that each page is linked to from somewhere so that there are no orphans? Trying to get my head around the structure makes my brain hurt. Any tips?
On-Page Optimization | | mascotmike0