Tool for Generating Sitemap/ URL List
-
HI,
I'm looking for a tool that'll generate a URL list for a site. I looked at this thread here http://www.seomoz.org/q/online-sitemap-generator which came up when I searched for sitemap generator.
However, I don't need a sitemap per se, and I don't need to submit it to Google - just a list of pages is what I need.If it updated automatically, that would be useful as well.
Anyone know of a tool, on or offline? Or anyone used Xenu and know if it's what I'm looking for? Or is there a simple solution that I'm missing?
Thanks.
-
Try DYNO Mapper. If you are looking for a tool that will generate a URL list, I think it will work really well for you. It also will give you a full content inventory that you can filter, search, and export content types. This particular Sitemap Generator is also integrated with Google Analytics and has collaboration capability for multiple users.
-
Just spent the last couple hours checking out Xenu. Found this article here on SEOmoz:
http://www.seomoz.org/blog/xenu-link-sleuth-more-than-just-a-broken-links-finder.
Turns out Xenu is an awesome tool - can't wait to make more use of it.
I also took Joyce's advice which was more suited to the task I was trying to do. Thanks for the simple solution!
-
Yes Xenu will do it, it checks all the links and then offers you a sitemap as a report at the end
I like Xenu, it's quick too
-
If you're using SEOMoz Pro, you can enter the domain into Open Site Explorer and then click the Top Pages tab. This will give you an exportable list of up to 10,000 pages on the site. Alternatively, you can go to Google Analytics and click Content > Top Content. This will give you an even bigger list, though you may have to filter out any attributes that Google picks up.
Xenu can also be a good option, but you still have to do a lot of filtering.
I hope that helps!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Embedded twitter post is good for https://www.fitness-china.com/hip-thrust-machine seo
We have short video posts on twitter. Embedded twitter post is good for https://www.fitness-china.com/hip-thrust-machine SEO?
On-Page Optimization | | ahislop5740 -
301 and the base URL
Hi, please bear with me as I'm pretty new to all this! I've my base URL but obviously want to add keywords to it for seo purposes. Should I redirect from the base URL to the URL with the keywords appended? So my landing page goes from say www.moz.com to www.moz.com/keywords-here. If I do that, should I replicate all the meta data (descriptions etc) on the original landing page? Or does it not matter? Thanks, Nick
On-Page Optimization | | nickwoodward0 -
Correct .htaccess settings for canonical url?
I want to forward all urls to http:www.mysite.com but am a little confuse because I am getting duplicate content error: Pages with Duplicate Page Content as of Jan 15http://titanappliancerepair.com/ 1 duplicatehttp://titanappliancerepair.com 1 duplicatehttp://titanappliancerepair.com/index.html 1 duplicate*****************************************************************What should I put ion htaccess file so I can forwardhttp://titanappliancerepair.com/index.htmlhttp://titanappliancerepair.comhttp://titanappliancerepair.com/to http://www.titanappliancerepair.comor what is the correct way to do it?I'm confused because when I enter http://titanappliancerepair.com/ in browser it showshttp://titanappliancerepair.com so how can it be considered duplicate content?.Can someone help?I have godaddy and they have gave me this code to put RewriteEngine on
On-Page Optimization | | webbutler13
rewritecond %{http_host} ^coolexample.com [nc]
rewriterule ^(.)$ http://www.coolexample.com/$1 [r=301,nc]What is correct?0 -
Content with changing URL and duplicate content
Hi everyone, I have a question regarding content (user reviews), that are changing URL all the time. We get a lot of reviews from users that have been dining at our partner restaurants, which get posted on our site under (new) “reviews”. My worry however is that the URL for these reviews is changing all the time. The reason for this is that they start on page 1, and then get pushed down to page 2, and so on when new reviews come in. http://www.r2n.dk/restaurant-anmeldelser I’m guessing that this could cause for serious indexing problems? I can see in google that some reviews are indexed multiple times with different URLs, and some are not indexed at all. We further more have the specific reviews under each restaurant profile. I’m not sure if this could be considered duplicate content? Maybe we should tell google not to index the “new reviews section” by using robots.txt. We don’t get much traffic on these URLs anyways, and all reviews are still under each restaurant-profile. Or maybe the canonical tag can be used? I look forward to your input. Cheers, Christian
On-Page Optimization | | Christian_T2 -
Sitemap.xml parameters
Hello, I confused about the parameters when create sitemap.xml. I don'n know how to use these parameters efficiently. Some parameters are:
On-Page Optimization | | JohnHuynh
- **Page changing frequency: **what is a good value? Can I choose "None"?
- **Last modified date: **what is a good value? Can I choose "Don’t specify"?
- **Page priority: **what is a good value? Can I choose "Don’t specify"? Thanks,0 -
Sitemap for webshop?
Had a client today who contacted me regarding having no sitemap on his webshop (with about 2000 products). He told me that his design company, who had made his webshop, adviced him against putting up a site since it would be giving google to much information at once, hence making it stop visiting his webshop as much? Well my first through was -okay this must be some kind of joke.
On-Page Optimization | | zarkas
But with seo, there are always something, now and then, that still surprises you, So are there anyone else here, who could share some info on this?0 -
Localised content/pages for identical products
I've got a question about localising the website of a nationwide company. We're a small dance school with nationwide (40 cities) coverage for around 40 products. Currently, we have one page for each product (style of dance), and one page for each city; the product pages cover keywords like 'cheerleading dance class' while the city pages target the 'london dance classes'-type keywords. To make 'localised product pages', I feel like we should make a page for every city/product combo 'London cheerleading classes' - but that seems like a nightmare for both writing sexy & original content, and link building/social stats. The other thing I can think of (which I refuse to do because it would look stupid & flag the page as keyword stuffed) is filling the page with the keyword phrases which are appropriate for every city. Is there another way to let google know 'this page is appropriate for these cities...'? We do currently list the cities a product is available in, but it doesn't seem to help local rankings very much. Would this just be a link building job, using hyper-targeted anchor texts (inc. city names) for each product? How do the pro's tackle this problem?
On-Page Optimization | | AlecPR0 -
Is it worth changing urls with underscores?
A few pages on one of my sites have underscores linking keywords rather than hyphens (keywords_and_keyword rather than keyword-and-keyword). Possibly from a time before I knew hyphens were preferred... One of the pages ranks well, and drives a good amount of traffic. The others do not do so well, but are still within the top 10 landing pages for the site. Is it worth me changing the underscores to hyphens (setting up 301 redirects first of course) or doesn't it make that much difference?
On-Page Optimization | | Jingo010
