Tool for Generating Sitemap/ URL List
-
HI,
I'm looking for a tool that'll generate a URL list for a site. I looked at this thread here http://www.seomoz.org/q/online-sitemap-generator which came up when I searched for sitemap generator.
However, I don't need a sitemap per se, and I don't need to submit it to Google - just a list of pages is what I need.If it updated automatically, that would be useful as well.
Anyone know of a tool, on or offline? Or anyone used Xenu and know if it's what I'm looking for? Or is there a simple solution that I'm missing?
Thanks.
-
Try DYNO Mapper. If you are looking for a tool that will generate a URL list, I think it will work really well for you. It also will give you a full content inventory that you can filter, search, and export content types. This particular Sitemap Generator is also integrated with Google Analytics and has collaboration capability for multiple users.
-
Just spent the last couple hours checking out Xenu. Found this article here on SEOmoz:
http://www.seomoz.org/blog/xenu-link-sleuth-more-than-just-a-broken-links-finder.
Turns out Xenu is an awesome tool - can't wait to make more use of it.
I also took Joyce's advice which was more suited to the task I was trying to do. Thanks for the simple solution!
-
Yes Xenu will do it, it checks all the links and then offers you a sitemap as a report at the end
I like Xenu, it's quick too
-
If you're using SEOMoz Pro, you can enter the domain into Open Site Explorer and then click the Top Pages tab. This will give you an exportable list of up to 10,000 pages on the site. Alternatively, you can go to Google Analytics and click Content > Top Content. This will give you an even bigger list, though you may have to filter out any attributes that Google picks up.
Xenu can also be a good option, but you still have to do a lot of filtering.
I hope that helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
301 and the base URL
Hi, please bear with me as I'm pretty new to all this! I've my base URL but obviously want to add keywords to it for seo purposes. Should I redirect from the base URL to the URL with the keywords appended? So my landing page goes from say www.moz.com to www.moz.com/keywords-here. If I do that, should I replicate all the meta data (descriptions etc) on the original landing page? Or does it not matter? Thanks, Nick
On-Page Optimization | | nickwoodward0 -
Unused CSS Tool Recomendations
Hey moz, Im currently converting my site to mobile as its an older template based on tables it will make a lot of CSS redundant. I cant delete as i go incase a page uses it im unaware of so need to clean it up when its done. Is there a tool that crawls my entire website and highlights the unused CSS in the files? Ive found a few paying tools that claim to do so but am reluctant to just spend money on something without a recommendation. Cheers!
On-Page Optimization | | ATP1 -
Left Nav / SEO
We are in the middle of a redesign and would love to get rid of our left navigation. It's currently full of SEO rich categories and has been the same for 10+ years. Our programmers think removing the left navigation will have no impact on our SEO. As SEO's, do you agree with this?
On-Page Optimization | | EileenCleary0 -
Title tag and URL Optimization
Hello guys, Should the URL reflect the structure of the title of a webpage? This is the old title with the Url: 20mm O/D Black Polypropylene LSZH Flexible Conduit 100m Coil /Product/20mm-o-d-black-polypropylene-lszh-conduit-100m-coil/1352 I changed the keyword position and it looks like this: 20mm Flexible Conduit | O/D Black Polypropylene LSZH | 100m I kept the same Url for now, should I change that too? Thanks
On-Page Optimization | | PremioOscar0 -
How to deal with tracking numbers in URLs
I am working on a site at the minute that has links like this: Jobs in London URL looks like: domain.com/jobs-in-london/ However, my developers insist that they need to use tracking codes, so everytime someone clicks on the above link, they are redirected (301) to a new URL that looks like: domain.com/search/1234567(unique search id) This is killing me when I am trying to get internal pages, like /jobs-in-london/ ranked. What to do?
On-Page Optimization | | MirandaP0 -
How do I do a 301 Redirect in IIS 7 from http://www.freightmonster.com/index.html to http://freightmonster.com/index.html when I don't have a physical page to redirect?
I'm trying to get rid of my Rel Canonical links and use the 301 Redirect instead.
On-Page Optimization | | FreightBoy0 -
Is it worth changing urls with underscores?
A few pages on one of my sites have underscores linking keywords rather than hyphens (keywords_and_keyword rather than keyword-and-keyword). Possibly from a time before I knew hyphens were preferred... One of the pages ranks well, and drives a good amount of traffic. The others do not do so well, but are still within the top 10 landing pages for the site. Is it worth me changing the underscores to hyphens (setting up 301 redirects first of course) or doesn't it make that much difference?
On-Page Optimization | | Jingo010 -
Page speed tools
Working on reducing page load time, since that is one of the ranking factors that Google uses. I've been using Page Speed FireFox plugin (requires FireBug), which is free. Pretty happy with it but wondering if others have pointers to good tools for this task. Thanks...
On-Page Optimization | | scanlin0