Sitemap for 170 K webpages
-
I have 170 K pages on my website which I want to be indexed. I have created a multiple HTML sitemaps (e.g. sitemap1.html, sitemap2.html,...etc) with each sitemap page having 3000 links.
Is this right approach or should i switch to xml based sitemaps and that too multiple one.
Please suggest.
-
Yes, you could do multiple XML Sitemaps and then do a sitemap-index.xml referring to each of the individual sitemaps. I would also then add the URL to the XML Sitemap Index file in the robots.txt Something like: Sitemap: http://www.yourdomain.com/sitemap-index.xml
-
There is no penalty for including multiple sitemaps. See this google blog post on the topic.
In addition to potentially getting more pages indexed as mentioned in the seomoz post link above, it will help keep you and the future site maintainers sane when its time to update or edit the sitemap.
-
Don't use html sitemaps, rather use XML sitemaps instead.
Here is a great post about utilizing XML sitemaps.
http://www.seomoz.org/blog/multiple-xml-sitemaps-increased-indexation-and-traffic
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can you keep you old HTTP xml sitemape when moving to HTTPS site wide?
Hi Mozers, I want to keep the HTTP xml sitemape live on my http site to keep track of indexation during the HTTPS migration. I'm not sure if this is doable since once our tech. team forces the redirects every http page will become https. Any ideas? Thanks
Technical SEO | | znotes0 -
"HTTP error: 404 not found" submitting YOAST SITEMAP
When I upload the YOAST site map to google webmaster i get "HTTP error: 404 not found" just for the portfolio tag and categories..For other things iI dont get any i kinf of errors Is it because i dont have any tags and categories of portfolio element? I have to say in my template I have the portfolio post option but im not using it. Tx
Technical SEO | | tourtravel0 -
How do you handle Wordpress sitemaps within your site?
I have a regular site map on my site and I also have a Wordpress site installed within it that we use for blog/news content. I currently have an auto-sitemap generator installed in Wordpress which automatically updates the sitemap and submits it to the search engines each time the blog is updated. The question I have (which I think I know the answer to but I just want to confirm) is do I have to include all of the articles within the blog in the main site's sitemap despite the Wordpress sitemap having them in there already? If I do include the articles in the main website's sitemap, they would also be in the Wordpress sitemap as well, which is redundant. Redundancy is not good, so I just want to make sure.
Technical SEO | | iresqkeith0 -
Google Webmaster Sitemap *pending*
Hey guys, I've noticed that my sitemap has been "pending" for quite some time in Google Webmaster tools. This leads me to believe that Google is not indexing my site. Could someone help me and point me to what I'm doing wrong? My site is The Tech Block
Technical SEO | | ttb0 -
Is it best to create multiple xml sitemaps for different sections of a site?
I have a client with a very big site that includes a blog, videos, photo gallery, etc. Is it best to create a separate xml file for each of these sections? It seems to me like that would be the best way to keep everything organized. Or at least separate the blog out from the main site. Does anybody have any tips or recommendations? I'm not finding any good information about this.
Technical SEO | | MichaelWeisbaum0 -
Best practices for migrating an html sitemap? Or just get rid of it all together?
We are migrating a very large site to a new CMS and I'm trying to determine the best way to handle all the links (~15k) in our html sitemap. The developers don't see the purpose of using an html sitemap anymore and I have yet to come up with a good reason why we should migrate rather than just get rid of the sitemap since it is not very useful to users. The html sitemap was created about 6 years ago when page rank sculpting was a high priority. Currently, since we already have an XML sitemap, I'm not sure that there's really a need for a html sitemap, other than to maintain all the internal links. How valuable are the internal links found in an html sitemap? And will it be a problem if we remove these from our link profile? 15,000 links sounds significant, but they only account for less than .5% of our internal links. What do all you think?
Technical SEO | | BostonWright0 -
301 technical issue / sitemap being found in search
Hi I recently did some 301 redirects. But now when I type into the search results 'Kyocera black and white printers' my site map is coming up in the listings - does any have any advise as to why this may be happening and how to fix it? Please see image attached, thanks. 1XqHW.jpg
Technical SEO | | Socialdude0