XML Sitemaps for Message Boards / Forums - Best Practices?
-
I'm working with a message board that has been around for 10+ years and never taken SEO best practices into consideration. They recently started seeing mobile URLs show up in regular results, which they don't want. I'm recommending they implement multiple sitemaps to properly indicate to Google how to crawl the site and what to index. I've never dealt with a site this large so I'm not sure best practices. They have a HUGE community and new URLs are created every second. Doing a site: search returns "About 12,100,000" URLs. What are some best practices / the best way to approach sitemaps for a site of this size?
-
-
Very Helpful.
-
A lot depends on what the mobile forum is - a separate url or just a css change? Read this
In the end, you need to tell Google what version of the site to serve to regular visitors and which to serve to mobile users.
If it's such a big forum, the crawlers are probably constantly on the site anyways so sitemaps would do you less good than optimizing the actual forum.
However, if you are inclined to create a sitemap, consider making a sitemap index and splitting up the threads/sections/replies into separate sitemaps (at your discretion). The goal being that each sitemap is updated as often as the next the crawler can keep up with it (so if sitemap gets checked 3 times a day, make sure all the links that need to be crawled would probably appear during those 3 checks)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEO Best Practices for Customer Portals
We have a customer portal which is used to display customer's serial numbers, the knowledgebase, support ticketing information and forum. This information is behind a wall as a user must have support in order to view. The question is what are the best practices for SEO with the customer portal? Should we block these sections from bots? Is there a way to take advantage of the number of pages that are within the portal?
Intermediate & Advanced SEO | | ASCI-Marketing0 -
Splitting One Site Into Two Sites Best Practices Needed
Okay, working with a large site that, for business reasons beyond organic search, wants to split an existing site in two. So, the old domain name stays and a new one is born with some of the content from the old site, along with some new content of its own. The general idea, for more than just search reasons, is that it makes both the old site and new sites more purely about their respective subject matter. The existing content on the old site that is becoming part of the new site will be 301'd to the new site's domain. So, the old site will have a lot of 301s and links to the new site. No links coming back from the new site to the old site anticipated at this time. Would like any and all insights into any potential pitfalls and best practices for this to come off as well as it can under the circumstances. For instance, should all those links from the old site to the new site be nofollowed, kind of like a non-editorial link to an affiliate or advertiser? Is there weirdness for Google in 301ing to a new domain from some, but not all, content of the old site. Would you individually submit requests to remove from index for the hundreds and hundreds of old site pages moving to the new site or just figure that the 301 will eventually take care of that? Is there substantial organic search risk of any kind to the old site, beyond the obvious of just not having those pages to produce any more? Anything else? Any ideas about how long the new site can expect to wander the wilderness of no organic search traffic? The old site has a 45 domain authority. Thanks!
Intermediate & Advanced SEO | | 945010 -
Advanced: SEO best practice for a large forum to minimise risk...?
Hi Hope someone can offer some insight here. We have a site with an active forum. The transactional side of the site is about 300 pages totals, and the forum is well over 100,000 (and growing daily) meaning the 'important' pages account for less that 0.5% of all pages on the site. Rankings are pretty good and we're ticking lots of boxes with the main site, with good natural links, logical architecture, appropriate keyword targeting. I'm worried about the following: crawl budget PR flow Panda We actively moderate the forum for spam and generally the content is good (for a forum anyway), so I'm just looking for any best practice tips for minimising risk. I've contemplated moving the forum to a subdomain so there's that separation, or even noindexing the forum completely, although it does pull in traffic. Has anyone been in a similar situation? Thanks!
Intermediate & Advanced SEO | | iProspect_Manchester1 -
Across the board drops in DA
Hi, has anyone noticed across the board drops in DA in specific industries lately? If you're tracking 5 companies in one industry it seems odd that all five would drop 2-3 DA points all at once don't you think? Just peculiar I think and would love some insight if there is some to be had / shared.
Intermediate & Advanced SEO | | wearehappymedia0 -
Need help creating sitemap
Hello, The details of my question is sitemap related. Below is the background info: we are ecommerce site with around 4000 pages, and 20000 images. we dont have a sitemap implemented on our site yet. i have checked alot of sitemap tools out there, like g-sitecrawler, xml sitemap, a1 sitemap builder etc, and i tried to create sitemaps via them, but all them give different results. the major links are all there, but the results start to vary for level 2, level 3 links and so on. plus no matter how much i read up on sitemaps, the more i am getting confused. i read lots of seomoz articles on sitemaps, and due to my limited seo and technical knowledge, the extra information on these articles gets more confusing. i also just read an article on seomoz that instead of having one sitemap, having multiple smaller sitemaps is very good idea, specially if we are adding lots of new products (which we are). Now my question: My question is having understood the immense value of sitemap (and by having it very poorly implemented before), how can i make sure that i get a very good sitemap (both xml and html sitemap). i do not want to do something again and just repeat old mistakes by having a poorly implemented sitemap for our site. I am hoping that one of the professionals out there, can help me also make and implement the sitemap. If you can please point me to the right direction.
Intermediate & Advanced SEO | | kannu10 -
Canonical / 301 Redundancy
Suppose I have two dynamic URLs that lead to the identical page: www.example.com/product.php?x=1&y=1 and www.example.com/product.php?y=1 The x=1 parameter had some historical meaning, but is now unused. All references to the x=1 parameter have been removed from internal links and sitemaps. I have implemented two solutions: First, the header of www.example.com/product.php?x=1&y=1 includes: Second, the .htaccess file includes the following: Redirect permanent /product.php?x=1&y=1 http://www.example.com/product.php?y=1 Questions: 1. I assume that since canonical is still relatively new, it's best to play it safe and implement both solutions. Is this correct? 2. When I point my browser to www.example.com/product.php?x=1&y=1, it does NOT redirect to www.example.com/product.php?y=1. The address bar continues to show the non-canonical URL. Is this because the canonical tag somehow takes precedence over the 301 redirect? 3. How long will Google Webmaster Tools continue to show these as duplicates, even though I've implemeted BOTH canonical and 301? It's been a few weeks and I thought it would have rolled off by now. Thanks!
Intermediate & Advanced SEO | | ahirai0 -
Setting a 404, best practices
Is it enough to just delete a page, or is it necessary to do something else to 404 a page correctly? Is there a great link to explain how to set http status codes?
Intermediate & Advanced SEO | | nicole.healthline0 -
Xml sitemap advice for website with over 100,000 articles
Hi, I have read numerous articles that support submitting multiple XML sitemaps for websites that have thousands of articles... in our case we have over 100,000. So, I was thinking I should submit one sitemap for each news category. My question is how many page levels should each sitemap instruct the spiders to go? Would it not be enough to just submit the top level URL for each category and then let the spiders follow the rest of the links organically? So, if I have 12 categories the total number of URL´s will be 12??? If this is true, how do you suggest handling or home page, where the latest articles are displayed regardless of their category... so I.E. the spiders will find l links to a given article both on the home page and in the category it belongs to. We are using canonical tags. Thanks, Jarrett
Intermediate & Advanced SEO | | jarrett.mackay0