Using Site Maps Correctly
-
Hello
I'm looking to submit a sitemap for a post driven site with over 5000 pages.
The site hasn't got a sitemap but it is indexed by google - will submitting a sitemap make a difference at this stage?
Also, most free sitemap tools only go up to 5000 pages, and I'm thinking I would try a sitemap using a free version of the tool before I buy one - If my site is 5500 pages but I only submit a sitemap for 5000 (I have no control of which pages get included in the sitemap) would this have a negative effect for the pages that didn't get included?
Thanks
-
Submitting a sitemap in Webmaster Console is always a good idea at any stage. If your website URLs are crawled and indexed in search engines than there will be no negative impact of it but in the longer run if you add more pages sitemap will defiantly a help.
If you are using CMS like WordPress, Joomla, Zencart or any other they all have extensions and plugins in their directory that will help you generate the sitemap of your current site and will add links as soon as you will add more pages.
Rest peter explains almost everything in detail like if you have URL issues and issues with crawling and indexing.
If you have a custom CMS, I think you should seriously consider the idea by Peter as this is something you need on regular basis anyways!
Hope this helps!
-
It's hard to tell without seeing your URL architecture.
First there are two specific terms and you never, never ever should forget them. They are - crawling and indexing. Once you prepare sitemap and submit there (or include in robots.txt) all bots get some map of your site and start crawling pages based on their crawling budget for your site. In crawling process they MAY find new pages that doesn't include in this map and will crawl them too. Again this is based on your crawling budget.
So when you submit sitemap - bot will get within seconds list of "non-crawled" 5000 pages and will start crawl them. Then he can find missed 500 pages and will crawl them too. Tricky is that when you update sitemap - he can detect quick changes there and start recrawling them again. But for missed 500 pages he can visit you again to check them for changes. And this will be also under your crawling budget. But if pages there isn't changed often - isn't big deal.
So you shouldn't hesitated about negative impact there. Only negative impact can happen if you have some serious URL architecture issues and messy URLs there. Then submitting partial sitemap can obfuscate this issues and some of your URLs to remain non-crawled.
Technically in SearchConsole you can see sitemap statistics like submitted and indexed. In perfect world numbers should be almost equal with little difference. But if you see huge difference between them - then you're in trouble. For example - on some site i have sitemap with submitted 44,950 pages and indexed of them was 29,643. This is pure example site crawling troubles or sitemap troubles. Because 1/3 of all pages isn't indexed at all.
PS: I forgot. You should use own CMS plugin for generating sitemap inside. Even if your CMS was custom made you should write (or hire someone) to create plugin inside. It's near 20-30 lines of write-here-your-favorite-language (PHP/Python/Perl/Ruby) and isn't big deal. This plugin will minimize crawling time from 3rd party sitemap generator tool because CMS already have all information inside and just need to be exported to XML.
-
It would definitely be better to submit a complete sitemap. If your site is built in Wordpress, Joomla, Magento, or many other standard CMS, it should have the ability to generate a full sitemap. Plugins like Yoast or Google Sitemaps help. Just depends on the site.
Otherwise you can probably get any pro SEO or agency to create a full 5500+ sitemap for you for $100 bucks or so.
PM me if you need more help.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Losing referrer data on http link that redirects to an https site when on an https site. Is this typical or is something else going on here?
I am trying to resolve a referral data issue. Our client noticed that their referrals from one of their sites to another had dropped to almost nothing from being their top referrer. The referring site SiteA which is an HTTPs site, held a link to SiteB, which is also an HTTPs site, so there should be no loss, however the link to SiteB on SiteA had the HTTP protocol. When we changed the link to the HTTPs protocol, the referrals started flowing in. Is this typical? If the 301 redirect is properly in place for SiteB, why would we lose the referral data?
Reporting & Analytics | | Velir0 -
.com site referral traffic to ccTLDs
We have 7 international domains set up along with our main .com site. All of the ccTLds are showing their main referral traffic as coming from the .com site in GA, and most of those being from mobile. Each site is set up correctly with geo-targeting and hreflang tags. Has anyone experienced this before?
Reporting & Analytics | | ggpaul5620 -
High Bounce Rate on traffic generating area of our site
Hi, Our eCommerce site currently includes a blog section known as Igloo which we have filled with unique and helpful content that is useful to a fair few people, not just customers of ours. It currently attracts a large number of visitors (more than the actual eCommerce side of the site in actual fact) organically who aren't currently customers of ours. Very few of these turn in to paying clients so it's not really a money spinner but it has worked quite well from a linkbait perspective / traffic generation perspective and undoubtedly a few of these people do end up making a purchase on the actual shopping end of our site. We're look at ways to encourage these people finding help on this free resource to take a look at our homepage and hopefully make an order but in the meantime I am worried that there may be a few downsides to us creating this content: Google may see us more as a help site than a shopping site. Since selling products is where we make our money this could ultimately be a bad thing. Our bounce rate is REALLY high (I'm talking around 94%) on the help site versus around 20% on the eCommerce site. I guess people land on the article they want, read it and then disappear. Would this bounce rate skew our entire site stats and ultimately result in decreased performance in the SERPS. I would appreciate your opinions and, in the event you do feel it may be hurting us overall perhaps some suggestions on how to mitigate the effects? Many thanks!
Reporting & Analytics | | ChrisHolgate0 -
SEO dealing with a CDN on a site.
This one is stumping me and I need some help. I have a client who's site is www.site.com and we have set them up a CDN through Max CDN at cdn.site.com which is basically a cname to the www.site.com site. The images in the GWT for www.site.com are de-indexing rapidly and the images on cdn.site.com are not indexing. In the Max CDN account I have the images from cdn.site.com sending a canonical header from www.site.com but that does not seem to help, they are all still de-indexing.
Reporting & Analytics | | LesleyPaone0 -
Anyone heard of or used Hadoop software for SEO/website analytics?
Hey Moz Community, We received an interesting email this morning from a client's marketing manager asking if we knew about or have ever used "Hadoop software" because "[Business owner name] says it’s supposed to help us see how people use our website and more." I've never heard of this software and upon looking around on the web, it says it can do a lot, but my client is a small law firm who average 800-1,000 visitors per month, mainly from their 2 local office cities/markets. Anyone have any thoughts on this or ever use this software? Thanks in advance. Patrick
Reporting & Analytics | | WhiteboardCreations0 -
Why we shouldn't use AWstats to measure marketing efforts?
and what are the disadvantages of awstats compared to Google Analytics?
Reporting & Analytics | | esiow20130 -
Google Analytics Customer filters & the correct syntax
Good afternoon from torential rain and thunder Wetherby UK 😞 Ive been delving into the world of custom filters in Google analytics and Ive hit a problem. Apart from Googles advice being out of date (wont get into that one) I wanted to set up clone a profile of an existing site and set up a filter that would exclde USA traffic. But a country can be called many things e.g. United Sates USA America so how do you know what is the correct name to put in the filter pattern box? Here is a screen shot of my efforts :http://i216.photobucket.com/albums/cc53/zymurgy_bucket/filter-toexclude-usatrafficcopy.jpg (I chaned it to United States) So my question is where is there a list of the correct syntax for the filter pattern box? Taker for example you wnated to set up a custom filter to include just Palm OS traffic apprently from reading this the correct filter patter is Palm OS, i only know that from this http://www.e-nor.com/blog/web-analytics/tracking-mobile-devices-in-google-analytics Whilst Google has lots of documentation about filed patterns as documented here http://support.google.com/analytics/bin/answer.py?hl=en&answer=1034380&topic=1034830&ctx=topic Where in the name of flying spacial jockstraps is there documentation for the correct filter pattern syntax. Help my head hurts 😞
Reporting & Analytics | | Nightwing0 -
Something strange going on with new client's site...
Please forgive my stupidity if there is something obvious here which I have missed (I keep assuming that must be the case), but any advice on this would be much appreciated. We've just acquired a new client. Despite having a site for plenty of time now they did not previously have analytics with their last company (I know, a crime!). They've been with us for about a month now and we've managed to get them some great rankings already. To be fair, the rankings weren't bad before us either. Anyway. They have multiple position one rankings for well searched terms both locally and nationally. One would assume therefore that a lot of their traffic would come from Google right? Not according to their analytics. In fact, very little of it does... instead, 70% of their average 3,000 visits per month comes from just one referring site. A framed version of their site which is through reachlocal, which itself doesn't rank for any of their terms. I don't get it... The URL of the site is: www.namgrass.co.uk (ignore there being a .com too, that's a portal as they cover other countries). The referring site causing me all this confusion is: http://namgrass.rtrk.co.uk/ (see source code at the bottom for the reachlocal thing). Now I know reach local certainly isn't sending them all that traffic, so why does GA say it is... and what is this reachlocal thing anyway?? I mean, I know what reachlocal is, but what gives here with regards to it? Any ideas, please??
Reporting & Analytics | | SteveOllington0
