XML Sitemaps for Message Boards / Forums - Best Practices?
-
I'm working with a message board that has been around for 10+ years and never taken SEO best practices into consideration. They recently started seeing mobile URLs show up in regular results, which they don't want. I'm recommending they implement multiple sitemaps to properly indicate to Google how to crawl the site and what to index. I've never dealt with a site this large so I'm not sure best practices. They have a HUGE community and new URLs are created every second. Doing a site: search returns "About 12,100,000" URLs. What are some best practices / the best way to approach sitemaps for a site of this size?
-
-
Very Helpful.
-
A lot depends on what the mobile forum is - a separate url or just a css change? Read this
In the end, you need to tell Google what version of the site to serve to regular visitors and which to serve to mobile users.
If it's such a big forum, the crawlers are probably constantly on the site anyways so sitemaps would do you less good than optimizing the actual forum.
However, if you are inclined to create a sitemap, consider making a sitemap index and splitting up the threads/sections/replies into separate sitemaps (at your discretion). The goal being that each sitemap is updated as often as the next the crawler can keep up with it (so if sitemap gets checked 3 times a day, make sure all the links that need to be crawled would probably appear during those 3 checks)
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unanswered questions in forums
What should be done with forum questions that go unanswered for a long time (i.e. year or longer)? Are these types of questions valuable content? Should we opt out of having these types of pages indexed? Since these pages are just one sentences doesn't seem like it is adding value to the site.
Intermediate & Advanced SEO | | nandaMesa0 -
Xml sitemap only shows up sometimes (magento)
Hi Moz community, I'm using Magento platform. I can generate a sitemap using their xml generator, but it will only pull up sometimes in web explorers, the rest of the time it will show a 404 page. GWT also tells me that I get a 404 error when testing the sitemap, but sometimes it will acknowledge that it's there. Anyone had this problem before or know how to help. sitemap= www.ice.com/sitemap.xml Let me know what other information I can provide to help. Thanks!
Intermediate & Advanced SEO | | IceIcebaby0 -
Sort term product pages and fast indexing - XML sitemaps be updated daily, weekly, etc?
Hi everyone, I am currently working on a website that the XML sitemap is set to update weekly. Our client has requested that this be changed to daily. The real issue is that the website creates short term product pages (10-20 days) and then the product page URL's go 404. So the real problem is quick indexing not daily vs weekly sitemap. I suspect that daily vs weekly sitemaps may help solve the indexing time but does not completely solve the problem. So my question for you is how can I improve indexing time on this project? The real problem is how to get the product pages indexed and ranking before the 404 page shows u?. . Here are some of my initial thoughts and background on the project. Product pages are only available for 10 to 20 days (Auction site).Once the auction on the product ends the URL goes 404. If the pages only exist for 10 to 20 days (404 shows up when the auction is over), this sucks for SEO for several reasons (BTW I was called onto the project as the SEO specialist after the project and site were completed). Reason 1 - It is highly unlikely that the product pages will rank (positions 1 -5) since the site has a very low Domain Authority) and by the time Google indexes the link the auction is over therefore the user sees a 404. Possible solution 1 - all products have authorship from a "trustworthy" author therefore the indexing time improves. Possible solution 2 - Incorporate G+ posts for each product to improve indexing time. There is still a ranking issue here since the site has a low DA. The product might appear but at the bottom of page 2 or 1..etc. Any other ideas? From what I understand, even though sitemaps are fed to Google on a weekly or daily basis this does not mean that Google indexes them right away (please confirm). Best case scenario - Google indexes the links every day (totally unrealistic in my opinion), URL shows up on page 1 or 2 of Google and slowly start to move up. By the time the product ranks in the first 5 positions the auction is over and therefore the user sees a 404. I do think that a sitemap updated daily is better for this project than weekly but I would like to hear the communities opinion. Thanks
Intermediate & Advanced SEO | | Carla_Dawson0 -
When migrating website platforms but keeping the domain name how best do we add the new site to google webmaster tools? Best redirect practices?
We are moving from BigCommerce to Shopify but maintaining our domain name and need to make sure that all links redirect to their corresponding links. We understand the nature of 301s and are fine with that, but when it comes to adding the site to google webmaster tools, not losing link juice and the change of address tool we are kind of lost. Any advice would be most welcome. Thank you so much in advance!
Intermediate & Advanced SEO | | WNL0 -
What is best practice to eliminate my IP addr content from showing in SERPs?
Our eCommerce platform provider has our site load balanced in a few data centers. Our site has two of our own exclusive IP addresses associated with it (one in each data center). Problem is Google is showing our IP addresses in the SERPs with what I would assume is bad duplicate content (our own at that). I brought this to the attention of our provider and they say they must keep the IP addresses open to allow their site monitoring software to work. Their solution was to add robots.txt files for both IP addresses with site wide/root disallows. As a side note, we just added canonical tags so the pages indexed within the IP addresses ultimately show the correct URL (non IP address) via the canonical. So here are my questions. Is there a better way? If not, is there anything else we need to do get Google to drop the several hundred thousand indexed pages at the IP address level? Or do we sit back and wait now?
Intermediate & Advanced SEO | | ovenbird0 -
How And/Or If To Prune Footer Links
Hi, I have a site with a site-wide footer that currently has 28 internal links.The footer terms are the terms the pages are focused on. This footer is on every page of the site (hundreds of pages). Some pages of my site have 10 or so additional links pointing to internal and external pages (besides the footer) and some pages (like the homepage) have about 50 links besides the footer. I'm going for a half dozen new terms with new pages that I would be adding to the site-wide footer. Do you think I should trim the existing footer before adding these new terms? I guess I would remove the terms that show no real hope of ever getting to page one... like pages stuck in the 40s. Or, pages I for whatever reason don't care much if they rank or not. Would trimming it to a smaller number do more to help the remaining linked pages/terms? What do you think? Thanks!
Intermediate & Advanced SEO | | 945010 -
What are best SEO practices for product pages of unique items when the item is no longer available?
Hello, my company sells used cars though a website. Each vehicle page contains photos and details of the unit, but once the vehicle is sold, all the contents are replaced by a simple text like "this vehicle is not available anymore".
Intermediate & Advanced SEO | | Darioz
Title of the page also change to a generic one.
URL remains the same. I doubt this is the correct way of doing, but I cannot understand what method would be better. The improvement I am considering for pages of no longer available vehicles is this: keep the page alive but with reduced vehicle details, a text like: this vehicles is not available anymore and automatic recommendations for similar items. What do you think? Is this a good practice or do you suggest anything different? Also, should I put a NOINDEX tag on the expired vehicles pages? Thank you in advance for your help.0 -
We are changing ?page= dynamic url's to /page/ static urls. Will this hurt the progress we have made with the pages using dynamic addresses?
Question about changing url from dynamic to static to improve SEO but concern about hurting progress made so far.
Intermediate & Advanced SEO | | h3counsel0