Tool for Generating Sitemap/ URL List
-
HI,
I'm looking for a tool that'll generate a URL list for a site. I looked at this thread here http://www.seomoz.org/q/online-sitemap-generator which came up when I searched for sitemap generator.
However, I don't need a sitemap per se, and I don't need to submit it to Google - just a list of pages is what I need.If it updated automatically, that would be useful as well.
Anyone know of a tool, on or offline? Or anyone used Xenu and know if it's what I'm looking for? Or is there a simple solution that I'm missing?
Thanks.
-
Try DYNO Mapper. If you are looking for a tool that will generate a URL list, I think it will work really well for you. It also will give you a full content inventory that you can filter, search, and export content types. This particular Sitemap Generator is also integrated with Google Analytics and has collaboration capability for multiple users.
-
Just spent the last couple hours checking out Xenu. Found this article here on SEOmoz:
http://www.seomoz.org/blog/xenu-link-sleuth-more-than-just-a-broken-links-finder.
Turns out Xenu is an awesome tool - can't wait to make more use of it.
I also took Joyce's advice which was more suited to the task I was trying to do. Thanks for the simple solution!
-
Yes Xenu will do it, it checks all the links and then offers you a sitemap as a report at the end
I like Xenu, it's quick too
-
If you're using SEOMoz Pro, you can enter the domain into Open Site Explorer and then click the Top Pages tab. This will give you an exportable list of up to 10,000 pages on the site. Alternatively, you can go to Google Analytics and click Content > Top Content. This will give you an even bigger list, though you may have to filter out any attributes that Google picks up.
Xenu can also be a good option, but you still have to do a lot of filtering.
I hope that helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Placement of key words in URL
I notice that the MOZ Page Grader considers "/keyword1/keyword2-keyword3" in a URL string to be less effective than "/keyword1-keyword2-keyword3". Is this correct from Google's perspective? If I am trying to maximise my SEO for the page title "Business building tips", for example, does Google think my URL is more relevant if it's in the form: 1. www.website.com/business-building-tips
On-Page Optimization | | Gavin.Atkinson
2. www.website.com/business/building-tips or
3. www.website.com/business/business-building-tips My instinct tells me 3 is more powerful, but logic tells me if I have a whole section devoted to "business" and one of those pages is "business building tips" then 2 should work just as well, possibly better?0 -
Internal Duplicate Content/Canonical Issue/ or nothing to worry about
Unfortunately, my developer cannot give me an answer to this so I really do hope someone can help. The homepage of my website is http://www.laddersfree.co.uk however I also have a page http://www.laddersfree.co.uk/index.php that has a page rank and essentially duplicates the home page. Does someone know what this is? Do I need to get my developer to do a 404? It is worrying that he has not come back to me. Thanks Jason
On-Page Optimization | | gymmad0 -
Duplicate content shown in Google webmaster tools for 301 redirected URLs.
Why does Google webmaster tools shows 5 URLs that have been 301 redirected as having duplicate meta descriptions?
On-Page Optimization | | Madlena0 -
Which URL Structure is best for New Website?
Hello All, I have one client and he want to develop his website but the he want the URL structure for his website page like below: http://www.example.com/category/xxproduct.html But I have suggest him below URL Structure http://www.example.com/category/xx-product.html So Can you people please suggest me that from above which URL structure is better from SEO side as well as from Visual side..? Is using - is better to separate the words in URL?
On-Page Optimization | | jemindesai0 -
My report is showing duplicate titles.. for http://www.mysite.com/ & http://mysite.com
Is this because I setup my campaign wrong in seo moz or should I have my site setup differently to forward all traffic without the www to the www version? Or vise versa? Thanks for any advice!
On-Page Optimization | | Fridaythe15th0 -
Recommendation: Add a canonical URL tag referencing this URL to the header of the page.
Please clarify: In the page optimization tool, seomoz recommends using the canonical url tag on the unique page itself. Is it the same canonical url tag used when want juice to go to the original page? Although the canonical URL tag is generally thought of as a way to solve duplicate content problems, it can be extremely wise to use it on every (unique) page of a site to help prevent any query strings, session IDs, scraped versions, licensing deals or future developments to potentially create a secondary version and pull link juice or other metrics away from the original. We believe the canonical URL tag is a best practice to help prevent future problems, even if nothing is specifically duplicate/problematic today. Please give example.
On-Page Optimization | | AllIsWell0 -
Absolute URLs
Hi, this is a very basic question but I want to confirm, as I remembered it was consider a good practice to use the absolute version of your links when linking to other pages of your site, not for any issue related to passing authority or PageRank, but because if someone scraps your content then they would take the links as well (as if they didn't remove them). Have the practices for internal linking with absolute or realtive URLs changed in any way? Which is the best way? absolute or relative? is there any harm for using the relative version? Relative: Absolute: [](<strong><em>http://www.cheapdomain.com/myfolder/mypage.html)[](<strong><em>http://www.cheapdomain.com/myfolder/mypage.html) [Thanks!](<strong><em>http://www.cheapdomain.com/myfolder/mypage.html)
On-Page Optimization | | andresgmontero0