Using Site Maps Correctly
-
Hello
I'm looking to submit a sitemap for a post driven site with over 5000 pages.
The site hasn't got a sitemap but it is indexed by google - will submitting a sitemap make a difference at this stage?
Also, most free sitemap tools only go up to 5000 pages, and I'm thinking I would try a sitemap using a free version of the tool before I buy one - If my site is 5500 pages but I only submit a sitemap for 5000 (I have no control of which pages get included in the sitemap) would this have a negative effect for the pages that didn't get included?
Thanks
-
Submitting a sitemap in Webmaster Console is always a good idea at any stage. If your website URLs are crawled and indexed in search engines than there will be no negative impact of it but in the longer run if you add more pages sitemap will defiantly a help.
If you are using CMS like WordPress, Joomla, Zencart or any other they all have extensions and plugins in their directory that will help you generate the sitemap of your current site and will add links as soon as you will add more pages.
Rest peter explains almost everything in detail like if you have URL issues and issues with crawling and indexing.
If you have a custom CMS, I think you should seriously consider the idea by Peter as this is something you need on regular basis anyways!
Hope this helps!
-
It's hard to tell without seeing your URL architecture.
First there are two specific terms and you never, never ever should forget them. They are - crawling and indexing. Once you prepare sitemap and submit there (or include in robots.txt) all bots get some map of your site and start crawling pages based on their crawling budget for your site. In crawling process they MAY find new pages that doesn't include in this map and will crawl them too. Again this is based on your crawling budget.
So when you submit sitemap - bot will get within seconds list of "non-crawled" 5000 pages and will start crawl them. Then he can find missed 500 pages and will crawl them too. Tricky is that when you update sitemap - he can detect quick changes there and start recrawling them again. But for missed 500 pages he can visit you again to check them for changes. And this will be also under your crawling budget. But if pages there isn't changed often - isn't big deal.
So you shouldn't hesitated about negative impact there. Only negative impact can happen if you have some serious URL architecture issues and messy URLs there. Then submitting partial sitemap can obfuscate this issues and some of your URLs to remain non-crawled.
Technically in SearchConsole you can see sitemap statistics like submitted and indexed. In perfect world numbers should be almost equal with little difference. But if you see huge difference between them - then you're in trouble. For example - on some site i have sitemap with submitted 44,950 pages and indexed of them was 29,643. This is pure example site crawling troubles or sitemap troubles. Because 1/3 of all pages isn't indexed at all.
PS: I forgot. You should use own CMS plugin for generating sitemap inside. Even if your CMS was custom made you should write (or hire someone) to create plugin inside. It's near 20-30 lines of write-here-your-favorite-language (PHP/Python/Perl/Ruby) and isn't big deal. This plugin will minimize crawling time from 3rd party sitemap generator tool because CMS already have all information inside and just need to be exported to XML.
-
It would definitely be better to submit a complete sitemap. If your site is built in Wordpress, Joomla, Magento, or many other standard CMS, it should have the ability to generate a full sitemap. Plugins like Yoast or Google Sitemaps help. Just depends on the site.
Otherwise you can probably get any pro SEO or agency to create a full 5500+ sitemap for you for $100 bucks or so.
PM me if you need more help.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How do I create a segment that shows me all pages using a certain keyword? But nothing that doesn't have that keyword?
There must be an easy answer to this, but I can't seem to find it. All I want to do is create a segment in Google Analytics that shows all pages and search strings with "orthopaedics" in the title, with pageviews, uniques etc. If I simply navigate to "All Pages" in Google Analytics and then click Advanced Filters and do an Include Page Contains "orthopaedics" it works just fine. (See attached Screen Shot) But when I try to recreate this as a segment, it pulls in all other pages the users visited before arriving on the orthopaedics page I want to include, which I don't want. I can manually exclude each URL I don't want, but this is tedious and I feel there must be a simpler method I'm just missing. At the end of the day, I'm trying to create a list of every page and dynamically created query string that includes the word "orthopaedics" to say doctor X, your orthopaedics section generated X views, and here's a list of the pages. Mm6YTKa
Reporting & Analytics | | Patrick_at_Nebraska_Medicine0 -
Multiple GA codes, one site.
Hi all, Is anyone running two GA codes on one website successfully? Â My organisation own a number of websites so we used to have one global GA code on all our sites to track global stats, and then we would also have site unique GA on each property to just track that one property. Â This worked fine, but of late we seem to be getting no data from the globally based code. Obviously, with the site-specific codes we can enter the name for that domain in GA but for the overall code, it is called 'all.com' I'm wondering if Google has now tied the GA domain to the code or if we are doing something wrong. All the codes are the same as they always were but have stopped working. As a stop gap, we have swapped to using Piwik as the all.com code. However, we are then comparing the stats in two different analytics programs so will get a different result. Also, it would be nice to be able to add the all.com to tools such as this to generate weekly reports. Anyone else having GA woe like this? Thanks. Carl
Reporting & Analytics | | WonkyDog0 -
Www.googleadservices.com/pagead/conversion_async.js what is this url doing on my site?
Hello Guys, I am using google tagmanager and i have configured adwords in tag manager now what i find is that this link -Â www.googleadservices.com/pagead/conversion_async.js showing on my homepage not in view source but when i do inspect element at that time it appears. So do you think after using google tag manager still i need to use the given link? Thanks, Raghu
Reporting & Analytics | | raghuvinder0 -
Stop getting info from Google analytics on purchases in our site
Hi guys, We have eCommerce.
Reporting & Analytics | | WayneRooney
We connected the site to the Google analytic eCommerce.
Everything was work fine until 3 weeks ago.  Suddenly we stooped getting purchases information in the analytic although i see purchases in the website. We didn't change anything in the website and i really don't know how to solve this problem.
If someone here can point me where i can get some info on how to fix it it can be great. Thanks a lot!0 -
How can I tell if I am using Universal Analytics?
On my Google Analytics page it says at the top "Upgrade to Universal Analytics Upgrade to get a deeper understanding of your users through new tools and more accurate data." Below that it talks about how to do it. I do not see what they are talking about for "Select an account and a property, then click Universal Analytics Upgrade." But if I can into my change history is shows Changed By-System Property "http://www123.com" started transfer to Universal Analytics processing." This was on May 22nd and it hasn't said anything since. Why does it keep telling me to do it if it already has? It doesn't say transfer complete anywhere so I don't know if it finished or not.
Reporting & Analytics | | EcommerceSite0 -
How to transfer 5 domains under one Brand domain without using 301 redirect with minimum SEO loss
My situation is rather complicated , I have 5 domains with different but identical services. Some with good traffic some without, but 2 main domains have manual penalty from Google (I have tried a lot to get rid of the penalty but no success) so finally I have decided to move all our websites under one anchor website 'a brand' and pass all the seo juice + blog articles under the same. Now the problem with the procedure is that we cannot redirect the penalized domain urls to new one as penalization will also be transferred and the whole motive of creating a new brand will be lost. So if somebody could guide me as of how I should proceed transferring SEO value with minimum traffic loss but without using 301) **One way I could figure out is using canonical (Am not confident over it) but say I create 5 identical pages on new domain and declare them as canonical of old domain pages (So is it safe? will it transfer the penalty of old domain to canonical new urls? )Â ** Rest i am bifurcating all the traffic sources like direct / organic / referral / Social etc and chalking out what we can control manually and in what ratio, gradually I will work on each section to transfer the traffic. Main Problem is of Organic and Not available. Some suggestions or blog urls would be appreciated.
Reporting & Analytics | | ngupta10 -
Redirects to external sites
Hi there, I have listings that all link to external sites. In order to track how many times each link was clicked I coded a redirect script that simply redirects to the respective url. www.mysite.com/go/123/ - redirects to another site www.mysite.com/go/541/ - redirects to a different site Is there anything I should or should not do from seo stand point such as blocking the /go directory in robots or adding a no-follow tag to links? I'd appreciate your advice. Thanks
Reporting & Analytics | | vmotuz0 -
Get Stats on Search Box Usage, Using Google Analytics?
Let me preface this by first saying I am NOT using the Google Adsense Search feature.   The question I am asking is pertaining to the standard search box that comes already coded in most .php blogs. In the Google Analytics, is there a way to tell how many people are searching using the search box?  I've looked through the navigation and pathway options in analytics and I don't seem to see anything that definitively points to the number of people using the search box but, perhaps, I am overlooking it. Any insight?
Reporting & Analytics | | ChristineCadena0