XML Sitemaps for Message Boards / Forums - Best Practices?
-
I'm working with a message board that has been around for 10+ years and never taken SEO best practices into consideration. They recently started seeing mobile URLs show up in regular results, which they don't want. I'm recommending they implement multiple sitemaps to properly indicate to Google how to crawl the site and what to index. I've never dealt with a site this large so I'm not sure best practices. They have a HUGE community and new URLs are created every second. Doing a site: search returns "About 12,100,000" URLs. What are some best practices / the best way to approach sitemaps for a site of this size?
-
-
Very Helpful.
-
A lot depends on what the mobile forum is - a separate url or just a css change? Read this
In the end, you need to tell Google what version of the site to serve to regular visitors and which to serve to mobile users.
If it's such a big forum, the crawlers are probably constantly on the site anyways so sitemaps would do you less good than optimizing the actual forum.
However, if you are inclined to create a sitemap, consider making a sitemap index and splitting up the threads/sections/replies into separate sitemaps (at your discretion). The goal being that each sitemap is updated as often as the next the crawler can keep up with it (so if sitemap gets checked 3 times a day, make sure all the links that need to be crawled would probably appear during those 3 checks)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google for Jobs best practice for Job Boards?
I head up SEO for a niche job board. We disallowed our job ad pages (/job/) in the robots.txt as this is user-generated content and really eating up our crawl budget, causing penalties etc. Now Google for Jobs has hit the UK (our strongest region for traffic), I'm torn about what to do next. Our jobs will only show in GfJ if we remove the jobs pages from the robots.txt and apply the directed structured data to every single jobs page and monitor this constantly. I will also have to constantly invest in our website developers no indexing / canonicalizing new job pages and paginations. Is GfJ worth it? I have spoken to one other job board who has seen more brand awareness from appearing in GfJ but almost no traffic / application increase. But are we missing a trick here? Any advice would be greatly appreciated.
Intermediate & Advanced SEO | | gracekimberley11 -
Sitemap: unique sitemap or different sitemaps by Country
Hi guys, i have a question about sitemaps. We are doing an international site, e.x. www.offers.com for landing page and www.offers.com/br for brazil, www.offers.com/it for italy, etc... i don't if we should do an unique sitemap for all countries or separate sitemaps by country, e.x.: unique sitemap: www.offers.com/sitemap.xml - including all sitemaps www.offers.com/br/sitemap.xml - sitemap for brazil market only. Thank you
Intermediate & Advanced SEO | | thekiller990 -
Best way to show content from articles I am published/featured in
Hi. I was wondering what was the best way to show my audience articles that my client is featured in. My client is specifically a surgeon, who has been referenced in many articles around his specific field of cosmetic surgery. An idea posed is to repost the entire article but just reference back to the original article. Is there an SEO friendly way of doing this? I have seen this done before, like search engine journal's author Larry Kim might repost something he wrote or published on wordstream onto search engine journal sometimes, but makes the reference that it was originally posted on wordstream. I know the standard thinking is to always just write new and unique content, but there is already a good amount written about our client and referencing his work, how can we use this to our advantage and give new or prospecting patients information regarding his credibility? Our client really does not want us to write articles for him, and he does not have the time to write them either. Again Question: How can we leverage articles and studies that have already been published online that is featuring our client and show them in full onto our own website?
Intermediate & Advanced SEO | | Ideas-Money-Art0 -
When does it make sense to make a meta description longer than what's considered best practice?
I've seen all the length recommendations and understand the reasoning is that they will be cut off when you search the time but I've also noticed that Google will "move" the meta description if the search term that the user is using is in the cached version of the page. S I have a case where Google is indexing the pages but not caching the content (at least not yet). So we see the meta description just fine on the Google results but we can't see the content cache when checking the Google cached version. **My question is: **In this case, why would it be a bad idea to make a slightly lengthier (but still relevant) meta description with the intent that one of the terms in that description could match the user's search terms and the description would "move" to highlight that term in the results.
Intermediate & Advanced SEO | | navidash0 -
Does anyone know of any tools that can help split up xml sitemap to make it more efficient and better for seo?
Hello All, We want to split up our Sitemap , currently it's almost 10K pages in one xml sitemap but we want to make it in smaller chunks splitting it by category or location or both. Ideally into 100 per sitemap is what I read is the best number to help improve indexation and seo ranking. Any thoughts on this ? Does anyone know or any good tools out there which can assist us in doing this ? Also another question I have is that should we put all of our products (1250) in one site map or should this also be split up in to say products for category etc etc ? thanks Pete
Intermediate & Advanced SEO | | PeteC120 -
Any Good XML Sitemaps Generator?
I was wondering if everyone could recommend what XML Sitemap generators they use. I've been using XML-Sitemap, and it's been a little hit and miss for me. Some sites it works great, other it really has serious problems indexing pages. I've also uses Google's, but unfortunately it's not very flexible to use. Any recommendation would be much appreciated.
Intermediate & Advanced SEO | | alrockn0 -
Automotive part / OEM / Manufacturer numbers
Hi All, What's the best way to optimise pages for OE / Manufacturer Part numbers? Disclaimer: All part numbers in this post are fictional. I dont want this post out ranking my client for real part numbers 🙂 Take this for Throttle Body for example: WOODYS S-AB-Q.123.53G This is the main part number from WOODYS (the manufacturer). However, these are all variations of exactly the same product: Woodys 2.78972.11.0 Woodys 2.78972.16.0 Woodys 2.78972.20.0 Woodys 2.78972.26.0 Oh, and car brands use OE numbers for these parts, such as: VWA 9808e40923G VWA 9808e40923L VWA 9808e40923M VWA 9808e40923P VWA 9808e40923Q These internal part numbers are vitally important as most of my clients customers are garages/mechanics so they're very likely to search on OE numbers. So, would you suggest: Optimising 10 different pages for the same product (using the part numbers in the URL, Title and H1). The problem is there's no unique content for these pages, only the part number varies, so this would likely get penalised for dupe content, or not enough unique content. Optimising one page for all terms. If so, how do you suggest doing this to ensure all part/OE numbers rank well and part numbers are prominent in the SERPS?
Intermediate & Advanced SEO | | seowoody
Could Schema.org help here by marking up these EO numbers with the isSimilarTo property of the Product type? I'm trying to ensure these part number get equal presence in the SERP snippet when searched for, even though I can't physically include all these numbers in the Title tag, URL and H1 of one page. 3. Something else? Thanks, Woody 🙂1 -
XML Sitemap Index Percentage (Large Sites)
Hi all I'm wanting to find out from those who have experience dealing with large sites (10s/100s of millions of pages). What's a typical (or highest) percentage of indexed pages vs. submitted pages you've seen? This information can be found in webmaster tools where Google shows you the pages submitted & indexed for each of your sitemap. I'm trying to figure out whether, The average index % out there There is a ceiling (i.e. will never reach 100%) It's possible to improve the indexing percentage further Just to give you some background, sitemap index files (according to schema.org) have been implemented to improve crawl efficiency and I'm wanting to find out other ways to improve this further. I've been thinking about looking at the URL parameters to exclude as there are hundreds (e-commerce site) to help Google improve crawl efficiency and utilise the daily crawl quote more effectively to discover pages that have not been discovered yet. However, I'm not sure yet whether this is the best path to take or I'm just flogging a dead horse if there is such a ceiling or if I'm already at the average ballpark for large sites. Any suggestions/insights would be appreciated. Thanks.
Intermediate & Advanced SEO | | danng0