How would you create and then segment a large sitemap?
-
I have a site with around 17,000 pages and would like to create a sitemap and then segment it into product categories.
Is it best to create a map and then edit it in something like xmlSpy or is there a way to silo sitemap creation from the outset?
-
Thanks Saijo,
We are trying to silo product types/categories and break them into different sitemaps. I'm familiar with SF but I don't think it will create sitemaps with the granularity that we are looking for.
I'm using XMLSpy but I'm finding it hard to break out blocks of content.
-
To my knowledge, Screaming Frog doesn't allow you to create an XML sitemap. Perhaps Excel allows you to format the output from SF but I'm not sure. I did find a utility called XMLSpy which, though pricey, allows me to do some of the sorting I was looking for. Once sorted, I can manually pull out sections to segment my sitemap. It is a pain in the neck because I can determine a silo and do it automatically. That being said, I think I can develop a sitemap template and have our new web programmer to develop a way to auto generate a group of segmented sitemaps.
Anyone know if there is a canned solution that works with IIS?
-
If you site is structured such that the urls contain the categories you wish to sort , you can use something like Screaming Frog ( http://www.screamingfrog.co.uk/seo-spider/ ) and export all the urls and sort them out via excel in to categories and go that way
NOTE : the free version has a 500 url limit, so you might want to look at paid ( ask them if it can handle 17,00 urls before getting it ) or look at http://home.snafu.de/tilman/xenulink.html ( I haven't used it myself , so don't know if you can export stuff to excel from there )
Good luck mate , sounds like you have a big job ahead of you.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How does changing sitemaps affect SEO
Hi all, I have a question regarding changing the size of my sitemaps. Currently I generate sitemaps in batches of 50k. A situation has come up where I need to change that size to 15k in order to be crawled by one of our licensed services. I haven't been able to find any documentation on whether or not changing the size of my sitemaps(but not the pages included in them) will affect my rankings negatively or my SEO efforts in general. If anyone has any insights or has experienced this with their site please let me know!
Technical SEO | | Jason-Reid0 -
Http to https for large ecommerce - our steps taken (any others recommended?)
**Here is the message from our technical team for the http to https migration; are there any other http to https migration steps recommended? ** Http to https migration steps (for this large ecommerce site): We implemented HTTPS (HTTP over TLS) protocol today (5/4/2017). Applied a patch to ensure that HTTPS pages did not have NoIndex, NoFollow and tested before and after . Added new IIS HTTPS Redirect to enforce HTTPS from HTTP and changed others, including the WWW redirect Changed HTTPS only for Cookies as required as per new PCI vulnerabilities Changed the Basepage HTML template to use Relative Paths or Absolute URLs with HTTPS only (to prevent mixed content) Created and ran a SQL Script to cleanup 16 tables from HTTP to HTTPS (about 20,000 of them, including internal URL links, site settings, etc) Ran Google Sitemap Generator to create new sitemaps with HTTPS Added new HTTPS instance of the site into Webmaster Tools, then added verification code to master page, verified and then submitted the sitemaps to Search Console (QUESTION: will historical data in Google Console/ WMT be preserved for https?) **Follow up steps for http to https migration for large ecommerce: ** From this point forward, to avoid “mixed content”, the Marketing team must use either Relative Paths or Absolute Paths with HTTPS only in any customization (i.e. Basepage) or any new link, such as created in Content Management (i.e. Long Description). Any mixed content will make the website look not secure to customers and search engine spiders – so it is very important to be disciplined and diligent about this. Contact Salesforce to change the protocol to HTTPS only. Meanwhile, to prevent mixed content, we put in a temporary custom javascript change as workaround – but this should not be permanent especially as to the next upgrade will remove it – so we need Saleforce to make a change ASAP. We did not change Blog site (on sub domain), but we should even though it is only a Content site because it will not be signaled as Secure. This means we need to have someone make the changes to WordPress to enforce HTTPS and then change any links. In terms of impact to page ranking due to Google’s treatment of HTTPS over HTTP and due to some impact to page speed – we will need to monitor closely to see how indexing, organic traffic and page ranking goes and take any additional actions as necessary.
Technical SEO | | seo20170 -
301 Redirects, Sitemaps and Indexing - How to hide redirected urls from search engines?
We have several pages in our site like this one, http://www.spectralink.com/solutions, which redirect to deeper page, http://www.spectralink.com/solutions/work-smarter-not-harder. Both urls are listed in the sitemap and both pages are being indexed. Should we remove those redirecting pages from the site map? Should we prevent the redirecting url from being indexed? If so, what's the best way to do that?
Technical SEO | | HeroDesignStudio0 -
How to manually create sitemap with 301 redirrctions?
Hi there I've just redirected a few of my pages (created in Dreamweaver) using the PHP Redirect function, I am wondering how do I create a manual sitemap to submit to Google to reflect this or will Google pick it up automatically based on the below? Example:
Technical SEO | | IsaCleanse
URL Being redirected: http://industrytix.com.au/cuban-club-perth-tickets.php **New URL: **[http://industrytix.com.au/buy-tickets/cuban-club-perth-nyd/ Redirect code:](http://industrytix.com.au/buy-tickets/cuban-club-perth-nyd/) Header( "HTTP/1.1 301 Moved Permanently" );
Header( "Location: http://industrytix.com.au/buy-tickets/cuban-club-perth-nyd/" );
?>0 -
Google Webmaster tools Sitemap submitted vs indexed vs Index Status
I'm having an odd error I'm trying to diagnose. Our Index Status is growing and is now up to 1,115. However when I look at Sitemaps we have 763 submitted but only 134 indexed. The submitted and indexed were virtually the same around 750 until 15 days ago when the indexed dipped dramatically. Additionally when I look under HTML improvements I only find 3 duplicate pages, and I ran screaming frog on the site and got similar results, low duplicates. Our actual content should be around 950 pages counting all the category pages. What's going on here?
Technical SEO | | K-WINTER0 -
XML Sitemap Creation
I am looking for a tool where I can add a list of URL's and output an XML sitemap. Ideally this would be Web based or work on the mac? Extra bonus if it handles video sitemaps. My alternative is XLS and a bunch of concatenates, but I'd rather something cleaner. It doesn't need to crawl the site. Thanks.
Technical SEO | | Jeff_Lucas0 -
XML Sitemap without PHP
Is it possible to generate an XML sitemap for a site without PHP? If so, how?
Technical SEO | | jeffreytrull11 -
Could somebody suggest a GOOD Wordpress XML sitemap generator?
We have been putzing around with Google XML Sitemaps Generator (a plug-in on Wordpress) for our Wordpress blog and we cannot get it to write an XML sitemap! Could somebody suggest a viable alternative that actually works? Thank you for your help! Jay
Technical SEO | | theideapeople0