Tool for Generating Sitemap/ URL List
-
HI,
I'm looking for a tool that'll generate a URL list for a site. I looked at this thread here http://www.seomoz.org/q/online-sitemap-generator which came up when I searched for sitemap generator.
However, I don't need a sitemap per se, and I don't need to submit it to Google - just a list of pages is what I need.If it updated automatically, that would be useful as well.
Anyone know of a tool, on or offline? Or anyone used Xenu and know if it's what I'm looking for? Or is there a simple solution that I'm missing?
Thanks.
-
Try DYNO Mapper. If you are looking for a tool that will generate a URL list, I think it will work really well for you. It also will give you a full content inventory that you can filter, search, and export content types. This particular Sitemap Generator is also integrated with Google Analytics and has collaboration capability for multiple users.
-
Just spent the last couple hours checking out Xenu. Found this article here on SEOmoz:
http://www.seomoz.org/blog/xenu-link-sleuth-more-than-just-a-broken-links-finder.
Turns out Xenu is an awesome tool - can't wait to make more use of it.
I also took Joyce's advice which was more suited to the task I was trying to do. Thanks for the simple solution!
-
Yes Xenu will do it, it checks all the links and then offers you a sitemap as a report at the end
I like Xenu, it's quick too
-
If you're using SEOMoz Pro, you can enter the domain into Open Site Explorer and then click the Top Pages tab. This will give you an exportable list of up to 10,000 pages on the site. Alternatively, you can go to Google Analytics and click Content > Top Content. This will give you an even bigger list, though you may have to filter out any attributes that Google picks up.
Xenu can also be a good option, but you still have to do a lot of filtering.
I hope that helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is the URL Matching the Page Title Important?
Hello I have tried searching for an answer on this but I can't get a clear answer due to the results when searching for URL title. I have just launched our second Shopify site for one of our brands. My first site launched in 2014 but when I launched I didn't pay much heed to SEO for page titles, URLs, etc so have retrospectively fixed this over time. For my Shopify site just launching I want to get it as right as possible from the start (learning from mistakes). My question is regarding URLs and what my approach should be for better SEO. So, I have a page with a Title of Newton Leather Wallets, Purses, Card Holders & Glasses Cases and the URL is https://www.tumbleandhide.com/collections/newton-leather-wallets-card-holders It was my understanding that I should try and make the URL reflect the Page Title more accurately. The problem is that this takes the character count to 77. On other pages it can be in the 80s. Will the above link be better for SEO than say just https://www.tumbleandhide.com/collections/newton I am just wary of the URL's being too long as my Moz Site Crawl is returning a lot of URLs that are too long. Thanks in Advance.
On-Page Optimization | | lukegj0 -
Ecommerce URLs with numbers
Hi everybody! I have to optimize an ecommerce where somebody has previously done the SEO optimization, although the URLs have numbers before the product's name They have told me that these numbers are useful to find the products, so I think it shouldn't be really bad if I don't redirect them to "clear" ones. For example: /colesterol-sobrepeso/2217-hc-grass-capsulas-duras-15-capsulas.html > /colesterol-sobrepeso/hc-grass-capsulas-duras-15-capsulas.html Am I right? After all, they contain the keywords and the subfolders are also ok. Or it would be better if I redirect the whole site? Thanks!
On-Page Optimization | | Estherpuntu0 -
My home page URL http://seadwellers.com/ redirects to http://www.seadwellers.com/. Is this a problem?
"The URL http://seadwellers.com/ redirects to http://www.seadwellers.com/. Do you want to crawl http://www.seadwellers.com/ instead?" I was given this when I tried to crawl my home page using MOZ software. I was not aware of this, do not know if it could be a problem concerning any aspect of SEO, etc? :
On-Page Optimization | | sdwellers0 -
Webmaster Tools
Webmaster tools is throwing up an error for this page http://www.itwasntmyfault.com.au/testimonial/sharon-m/ HTTP Error: 404 - When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot due to an HTTP status error. All accessible URLs will still be submitted.The page is fine.... any advice?Thanks
On-Page Optimization | | RGOnline0 -
One or two keywords/pages
Hi, I have a question about good keyword practice. I have a page: http://www.holdnyt.dk/skader-karantaener/fodbold/england-premier-league It lists all injuries ("skader" in danish) and suspensions ("karantaener" in danish) for the english premier league in football/soccer. On the page one can choose to show only injuries OR suspensions, which have their own URLs: http://www.holdnyt.dk/skader-karantaener/fodbold/england-premier-league/skader http://www.holdnyt.dk/skader-karantaener/fodbold/england-premier-league/karantaener My question is - what is best. To optimize the first URL (the more general one) to fit both of the following keywords:
On-Page Optimization | | rasmusbang
"skader premier league" and "karantaener premier league" OR should i focus on optimizing the two latter URLs, the more specfiic to target the two keywords. Regards, Rasmus0 -
Recommendation: Add a canonical URL tag referencing this URL to the header of the page.
Please clarify: In the page optimization tool, seomoz recommends using the canonical url tag on the unique page itself. Is it the same canonical url tag used when want juice to go to the original page? Although the canonical URL tag is generally thought of as a way to solve duplicate content problems, it can be extremely wise to use it on every (unique) page of a site to help prevent any query strings, session IDs, scraped versions, licensing deals or future developments to potentially create a secondary version and pull link juice or other metrics away from the original. We believe the canonical URL tag is a best practice to help prevent future problems, even if nothing is specifically duplicate/problematic today. Please give example.
On-Page Optimization | | AllIsWell0 -
"Canonical URL Tag Usage" recommendation in SEOmoz "On-Page Optimization" Tool
Here comes another one related to SEOmoz "On-Page Optimization" Tool. The tool says the following about one of our pages: Canonical URL Tag Usage Explanation: Although the canonical URL tag is generally thought of as a way to solve duplicate content problems, it can be extremely wise to
On-Page Optimization | | gerardoH
use it on every (unique) page of a site to help prevent any query strings, session IDs, scraped versions, licensing deals or future
developments to potentially create a secondary version and pull link juice or other metrics away from the original. We believe
the canonical URL tag is a best practice to help prevent future problems, even if nothing is specifically duplicate/problematic
today. Recommendation: Add a canonical URL tag referencing this URL to the header of the page. Let's say our page is http://www.example.com/brands/abc-brand and on its header we'll place the following tag: Is this correct? I thought the canonical tag was meant for duplicates of the original page, for example: http://www.example.com/brands/print/abc-brand href="http://www.example.com/brands/abc-brand**?SESSID=123** Thanks in advance.0 -
Cononical URL problem
Hello, In the keywords difficulty tool, one of my client's sites has a different number for links to root domain and links to page even though they're both the home page. How do I tell if I have a cononical URL problem, and what do I do about it if I do? Thank you.
On-Page Optimization | | BobGW0