What tools produce a complete list of all URLs for 301 redirects?
-
I am project managing the rebuild of a major corporate website and need to set up 301 redirects from the old pages to the new ones. The problem is that the old site sits on multiple CMS platforms so there is no way I can get a list of pages from the old CMS. Is there a good tool out there that will crawl through all the sites and produce a nice spreadsheet with all the URLs on it? Somebody mentioned Xenu but I have never used it. Any recommendations? Thanks -Adrian
-
And the winner is... LinkExaminer! Pretty amazing tool, simple to use, but comprehensive - just the thing we need. Thanks for your help Marcin!
Adrian
-
Thanks Marcin and Robert. I will definitely give Xenu Link Sleuth a try first. The Search Engine Optimization Toolkit does look quite awesome though. Thanks for your help. - Adrian
-
There actually many tools which could do the job, both free and paid.
-
Xenu Link Sleuth - free, available at http://home.snafu.de/tilman/xenulink.html
-
Screaming Frog - free (limited)/paid, available at http://www.screamingfrog.co.uk/seo-spider/
-
A1 Website Analyzer - paid, but with a fully functional trial, available at http://www.pendragonexpress.com/hop/a1
-
LinkExaminer - free, available at http://www.analogx.com/contents/download/Network/lnkexam/Freeware.htm
-
Search Engine Optimization Toolkit - free, available at http://www.iis.net/download/seotoolkit
If you just need a crawl, I would recommend 1, 4 or 5. They are both free and work fast. If you want to optimize your content, 5 is a powerhouse.
Hope it helps.
Marcin
-
-
xenu linksleuth is the answer to your question. Good luck. Nothing better I am aware of.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Is 301 redirect the only way when using Vanity URLs?
We have been using vanity urls for some of our pages. Mostly the pages that have a vanity URL have a long URL length. But now the problem is, the vanity URL is getting displayed on the search engine when the particular keyword related to the page is entered. I checked the google search console, the vanity URL is indexed and the original URL remains unindexed. What should I do? Is adding 301 redirect to the vanity URLs are solution? Since some of vanity URLs are not redirecting to the original. Some of the original pages are not getting traffic. Also, can using canonical tag help?
Technical SEO | | tejasbansode0 -
301 redirect question
Hi Everyone When doing 301 redirects for a large site, if a page has 0 inbound links would you still redirect it or just leave it? Im just curious on the best practice for this Thanks in advance
Technical SEO | | TheZenAgency0 -
301 Redirect Attribute-Based Dynamic URL to Renamed Attributes
I work with a number of ecommerce sites that have dynamically-created urls based off of product attributes we've assigned in our cms. I am updating a handful of these attributes to more seo-friendly terms because they are outdated but am not certain how to go about redirecting all the urls that each attribute is in/could be in. For example: If I had the attributes hoagie and beanie and changed them to sandwich and headwear, a dynamic url might change like this: .com/hoagie/french-bread ---> .com/sandwich/french-bread .com/beanie/hoagie/novelty ----> .com/headwear/sandwich/novelty Since the urls are dynamically created, I am not sure how I go about redirecting all of them, or if I need to redirect all of them at all (instead just redirecting the urls indexed by Google, etc.) I also have a number of links within copy on each of the sites that contain linked anchor text using attributes that will be changing. I am assuming I will need to 301 each of these or update them manually to reflect the new attributes. I am new to the seo field and would appreciate any and all advice or direction to guides and tutorials that could aid me with this project. Thanks!
Technical SEO | | OfficeFurn0 -
Do I need both canonical meta tags AND 301 redirects?
I implemented a 301 redirect set to the "www" version in the .htaccess (apache server) file and my logs are DOWN 30-40%! I have to be doing something wrong! AddType application/x-httpd-php .html .htm RewriteCond %{HTTP_HOST} ^luckygemstones.com
Technical SEO | | spkcp111
RewriteRule (.*) http://www.luckygemstones.com/$1 [R=301,L] RewriteCond %{THE_REQUEST} ^./index.htm
RewriteRule ^(.)index.htm$ http://www.luckygemstones.com/$1 [R=301,L] IndexIgnore *
ErrorDocument 404 http://www.luckygemstones.com/page-not-found.htm
ErrorDocument 500 http://www.luckygemstones.com/internal-serv-error.htm
ErrorDocument 403 http://www.luckygemstones.com/forbidden-request.htm
ErrorDocument 401 http://www.luckygemstones.com/not-authorized.htm I've also started adding canoncial META's to EACH page: I'm using HMTL 4.0 loose still--1000's of pages--painful to convert to HTML5 so I left the / off the tag so it would validate. Am I doing something wrong? Thanks, Kathleen0 -
OSE says URL redirects to URL with trailing slash but it doesn't.
Site is www.example.com/folder/us and OSE says this URL redirects to www.example.com/folder/us/, but it does not. When I look at the OSE report for the latter version with the "/" it says "No Data Available For This URL". Why would that be? The original URL is www.example.com and it redirects to www.example.com/folder/us. Is this anything I need to worry about? I thought that the trailing / doesn't really mean much anymore but nonetheless, why does it think it redirects there?
Technical SEO | | rock220 -
Getting a complete list of all incoming links
I'm trying to clean up my spam incoming links and can't find where to download a complete list. Google Webmaster Tools has a button to download "Sample Links" or "Latest Links" but they don't show them all. Looking for wisdom from all the gurus out there. Thanks, Jean
Technical SEO | | JeanYates0 -
Does 301 redirect pass "freshness?"
Greetings! I work for an online retailer, and we recently launched a voting tool that allows customers to voice their opinion whether or not we should carry a new item. It's been a huge success and we've been generating thousands of comments. As a result, it's helped our SEO, and our products are showing up on the first page for some keywords without having any external links pointing to these pages. Our plan is to sell a product if it does well during the voting period. Unfortunately, we're not able to process the sale on the voting page, and need to redirect users to another page on our site. I understand that a 301 redirect transfers "linkjuice" to the new destination URL. But does it also transfer "freshness?" I ask because our new landing pages will not be updated as frequently as the voting pages. Example of our Voting Page:
Technical SEO | | znotes
http://www.uncommongoods.com/voting/product/50012/infant-fortune-cookie-booties Example of Redirected Item Page (where sale can be processed):
http://www.uncommongoods.com/product/baby-tube-socks-set-of-4 Any help/comments would be appreciated. Thank you!0
