Sitemap Size effect SEO
-
So I've noticed that the sitemap I use has a capacity of 4500 URLs, but my website is much larger. Is it worth paying for a commercial sitemap that encompasses my entire site?
I also notice that of the 4500 URLs which have been submitted, only 104 are indexed. Is this normal, if not, why is the index rate so low?
-
As Thomas said, 50k is the max but you can also have an index file with multiple sitemaps within it so you can have lots more if needed. A couple of things I would add.
- Manual crawling is fine but remember, that also means that pages that can't be crawled will be excluded from the sitemap which defeats the purpose of the sitemap
- As for why the indexation levels are so low, the first thing I would check is if everything in the sitemap is a 200 response. Make sure there aren't any redirects or 404s otherwise Google may decide not to trust the sitemap.
I hope this helps
-
Sitemaps can go up to 50,000 urls, so 4500 isn't an issue for Google to handle.
https://support.google.com/webmasters/answer/183668?hl=en
Just to add on to Bernadette's post, this section "If your website's CMS can't do that, then I would recommend crawling the website yourself and updating the sitemap file. However, keep in mind that if you do it manually then you'll need to update it whenever you add or remove a page on the website." You can use something like Screaming Frog for this if you were wondering what software to use. It's a very common tool for an SEO to use, so hopefully you'll have it to save purchasing a license to something else.
-
Spacecollective, your website's sitemap file(s) don't have any direct impact on search engine rankings. If you have a large website with over 4500 URLs, then most likely you're using content management system (CMS) that "should" be able to create a sitemap file.
If your website's CMS can't do that, then I would recommend crawling the website yourself and updating the sitemap file. However, keep in mind that if you do it manually then you'll need to update it whenever you add or remove a page on the website.
Typically, if your navigational structure is set up in a way that all pages on the website can be crawled via links on the site, you generally shouldn't have anything to worry about.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Sitemap Rules
Hello there, I have some questions pertaining to sitemaps that I would appreciate some guidance on. 1. Can an XML sitemap contain URLs that are blocked by robots.txt? Logically, it makes sense to me to not include pages blocked by robots.txt but would like some clarity on the matter i.e. will having pages blocked by robots.txt in a sitemap, negatively impact the benefit of a sitemap? 2. Can a XML sitemap include URLs from multiple subdomains? For example: http://www.example.com/www-sitemap.xml would include the home page URL of two other subdomains i.e. http://blog.example.com/ & http://blog2.example.com/ Thanks
Technical SEO | | SEONOW1230 -
Remove sitemap, effect ranking?
We are considering to remove our sitemap because it doesn't display the right structure. Will it affect current rankings if we remove the sitemap en continuing without a sitemap? Thanks
Technical SEO | | rijwielcashencarry0400 -
Should Sitemaps be placed in the sub folder they reference?
I have a sitemap-index.xml file in the root. I then have several sitemaps linked to from the index in example.com/sitemaps/sitemap1.xml, example.com/sitemaps/sitemap2.xml, etc. I have seen on other sites that for example a sitemap containing blogs where the blogs are located at example.com/blog/blog1/ would be located at example.com/blog/sitemap.xml. Is it necessary to have the sitemap located in the same folder like this? I would like to have all sitemaps in a single sitemap folder for convenience but not if it will confuse search engines. My index count for URLs in some sitemaps has dropped dramatically in Google Webmaster Tools over the past month or so and I'm not sure if this is having an effect. If it matters, I have all sitemap files, including the index, listed in the robots.txt file.
Technical SEO | | Giovatto0 -
Minimising the effects of duplicate content
Hello, We realised that one of our clients, copied a large part of content from our website to his. The normal reaction would be to send a cease and desist letter. Nevertheless this would probably mean loosing a good client. The client dumped the text of several articles (for example:
Technical SEO | | Lvet
http://www.velascolawyers.com/en/property-law/136-the-ley-de-costas-coastal-law.html ) Into the same page:
http://www.freundlinger-partners.com/en/home/faqs-property-law/ I convinced the client to place our authorship tags on this page, but I am wondering if this is enough. What do you think? Cheers
Luca0 -
Multiple domain SEO strategy
Hi Mozzers I'm an AM at a web dev. We're building a new site for a client who sells paint to different markets: Paint for boats Paint for construction industry Paint for, well you get the idea! Would we be better off setting up separate domains - boatpaintxxx.com, housepaintxxx.com, etc - and treat each as a searate microsites for standalone SEO activity or have them as individual pages/sub doms from a single domain - paints4all.com or something? From what i've read today, including the excellent Beginners Guide - I'm guessing there's no definitive answer! Feedback appreciated! Thanks.
Technical SEO | | rikmon0 -
SEO for Wordpress
I am sure you have been asked this many times... What is the BEST Wordpress Plug-in for SEO - Easy to Title Page, Meta Tags, etc... ?? Thanks
Technical SEO | | tab540 -
Sitemap with links and images together
Hi there, my e-commerce platform (Magento with "Mageworx SEO Enterprise" plugin) is generating an sitemap.xml that mix text (links) and images, and the result is something like that: <url><loc>http://www.e-lustre.com.br/abajur/abajur-keops</loc>
Technical SEO | | e-Lustre
<lastmod>2011-04-10</lastmod>
<changefreq>daily</changefreq>
<priority>1.0</priority>
image:imageimage:lochttp://www.e-lustre.com.br/catalog/product/image/size/250x250/e/-/e- lustre_mantra_0030_grande.jpg</image:loc></image:image></url> WebmasterTools accepts it, but recognize it as an image sitemap. Have you seen that kind of sitemap, and most important, do you think that's a problem ? Full file: http://www.e-lustre.com.br/sitemap.xml0 -
Mobile SEO or Block Crawlers?
We're in the process of launching mobile versions of many of our brand sites and our ecommerce site and one of our partners suggested that we should block crawlers on the mobile view so it doesn't compete for the same keywords as the standard site (We will be automatically redirecting mobile handsets to the mobile site). Does this advice make sense? It seems counterintuitive to me.
Technical SEO | | BruceMillard0