How deep should I let my forum be spidered
-
I run quote a niche website that's been running since late 1999 and over that time I've built up something like 4000 resources which consist of either text articles or image galleries and reviews along side another few thousand news stories relating to the niche interest. On top of the main site I also have a forum which isn't especially optimised for SEO and I was wondering, whilst was cleaning it up, whether anyone has any tips / suggestion / best practices for forum SEO.
Because it is all UGC the quality of the posts can be quite weak so I was wondering whether I should block robots completely from the forum, which seems a little harsh, whether I should let the whole forum be spidered (which seems a little excessive and potentially a bad thing) or whether I should restrict things to that only the main index and perhaps one page of topics and their posts be accessible to robots and then nofollow the rest?
Any thoughts?
-
Hey mate.
UGC can actually be very high quality content, and subsequently I would let Google run wild through all the content.
Forums can be tricky and you would want to make sure Google doesnt index any duplicate content.
I would use this tool, http://home.snafu.de/tilman/xenulink.html to crawl the forum. Export to Excel and then assess all the URL structures and see if it's picking up any strange parameters or duplicate content.
I would block the weird paramters in Google webmaster tools, and if technically possible 301 all the duplicate content issues. The problem with restricting things in robots.txt is that you might actually block link juice accidentally.
Hope that helps - my background is more in ecommerce but im sure we'll have similar site structure issues.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate content because of member only restrictions on a forum.
Our website's Community Forum links to the membership profile pages, which by default are blocked for non-members. https://www.foodbloggerpro.com/community/ https://www.foodbloggerpro.com/community/member/1301/ We're getting warnings in Moz for duplicate content (and errors) on these member profile pages. Any ideas for how we can creatively solve this problem? Should we redirect those pages or just beef them up with more content? Just ignore it and assume that search spiders will be smart enough to figure it out? See attached video for further explanation. Community_Area.mp4
On-Page Optimization | | Bjork0 -
Which page to rank for a Keyword? Home Page or Deep Page?
So, we have a situation where there is one particular keyword we want to rank for. We have been up and down over the years, at our best probably position 4-5, and now at 20ish. Thats for our home page of course, which the majority of our linking is probably pointing at. We also have a sub page which is optimised for that particular service. The term is "web design brisbane".
On-Page Optimization | | MauriceKintek
So as you can imagine, Web Design is in itself a service and we offer others. Should we optimise our home page for it and remove the sub page?
Keep the sub page because its one our services and optimise both?
Do some kind of canonical thing?
Change our interlinking? All our competitors home pages seem to be the ones that rank, and it feels and looks better in results if its the home page, but if switching up to our sub page is better im all ears. Also if our sub page is somehow hurting or leaking SEO from the home page, id like to know as well. Would prefer to not have to provide a link, due to competition but if someone wants to know we can always PM.0 -
Does Commented HTML Code Get Spidered?
I have a change, perhaps temporary, to my web store that I will be making. I am wondering if the code that I will comment out () gets spidered?
On-Page Optimization | | lbohen0 -
Advice with keywords - category - Forum
Hiya guys Everyone has been really good to me on here, just wanted a bit of advice with the keywords on my forum. my website is a nightlife forum for the UK, each city has its own section. Each section has a eg: _What's on in Birmingham? Club Nights, Upcoming Events, Promotions _ as the Title category, Should I drop the Club Nights, Upcoming Events, Promotions and put that in the description of the forum. So it'll just be What's on in Birmingham? with a description Find Club night information, Upcoming events and pr............. eg Just wondering if it was to stop searches been made, like, Club nights in Birmingham etc. from being targeted. Your thoughts please guys Thanks for reading Lukescotty
On-Page Optimization | | Lukescotty0 -
SeoMoz Pro - Keyword stuffing but its a forum
hiya guys Just going through the Seomoz Pro on page thing One of the flagged items is keyword stuffing, I have 22 keywords (stuffed) but they want it down to 15, the thing is, my website is a forum. and the keyword stuffing is from the title name of each categories eg its a nightlife forum so Birmingham nightlife forum Brighton nightlife forum Leeds nightlife forum Manchester nightlife forum u get the picture To avoid this keyword stuffing, should I change some of the forum names to eg leeds nightlife forum > leeds nightlife forums? just by adding the 's that will half the keywords, but its not solving the problem in the long run, if i was to add more cities and areas with the XXXX nightlife foru. What you reckon guys? Cheers guys Luke
On-Page Optimization | | Lukescotty0 -
Customer forum on an ecommerce site, good or bad idea?
Hi, We are an online furniture retailer in Ireland and have been going for about 4 years, there is about 1000 visitors onto the site everyday. We have been thinking of new ways to interact with the customer and build the sites online content and constantly working to improve rankings. Have been toying with the idea of a user/customer forum and was wondering what the general consensus was with that as an idea, I appreciate that there could be negative aspects for the brand and was wondering if anyone had experience of similar and how that was perceived by the user and in what way did people interact with the forum. I assume differently to how they would interact with an "independent" furniture forum. My hope would be that the forum would be used for discussing general home improvements, asking questions relating to the home for community feedback and assistance and other similar home related topics. All thoughts and feed back welcome. Cheers. Eunan.
On-Page Optimization | | eunaneunan0 -
Getting 403 error in forum
Hi all, I am getting 403 error for my site where it is throwing error for the following url http://www.topuniversityforum.in/members/member id/ignore and it is showing 7 similar url for 7 user ids. I want to know how can i resolve it and if it is going to have any negative effect on its ranking.
On-Page Optimization | | akhilendra0 -
Are Forum Links To Website Causing Rankings To Drop ?
For many years our top keywords hve ranked in top 5. Usually top 3 for a few in particular. The last few months those particular keywords have dropped to 25 +. Our other rankings are doing pretty decent and are remaiing the same. We are members of about 6 forums since they relate to our website and sometime we have contests on them. I do know that the forums are based on those particular keywords and are linking to our website, as well as other forum members linking to us. Each forum has roughly 300 links to our website. It does not seem as if our competitors are affected by this, as they are members of the forums as well. What could be causing this for these certain keywords? If those forums were affected by Google's new algorythm would this have an affect on our website?
On-Page Optimization | | hfranz0