What tools produce a complete list of all URLs for 301 redirects?
-
I am project managing the rebuild of a major corporate website and need to set up 301 redirects from the old pages to the new ones. The problem is that the old site sits on multiple CMS platforms so there is no way I can get a list of pages from the old CMS. Is there a good tool out there that will crawl through all the sites and produce a nice spreadsheet with all the URLs on it? Somebody mentioned Xenu but I have never used it. Any recommendations? Thanks -Adrian
-
And the winner is... LinkExaminer! Pretty amazing tool, simple to use, but comprehensive - just the thing we need. Thanks for your help Marcin!
Adrian
-
Thanks Marcin and Robert. I will definitely give Xenu Link Sleuth a try first. The Search Engine Optimization Toolkit does look quite awesome though. Thanks for your help. - Adrian
-
There actually many tools which could do the job, both free and paid.
-
Xenu Link Sleuth - free, available at http://home.snafu.de/tilman/xenulink.html
-
Screaming Frog - free (limited)/paid, available at http://www.screamingfrog.co.uk/seo-spider/
-
A1 Website Analyzer - paid, but with a fully functional trial, available at http://www.pendragonexpress.com/hop/a1
-
LinkExaminer - free, available at http://www.analogx.com/contents/download/Network/lnkexam/Freeware.htm
-
Search Engine Optimization Toolkit - free, available at http://www.iis.net/download/seotoolkit
If you just need a crawl, I would recommend 1, 4 or 5. They are both free and work fast. If you want to optimize your content, 5 is a powerhouse.
Hope it helps.
Marcin
-
-
xenu linksleuth is the answer to your question. Good luck. Nothing better I am aware of.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Weird Cigarette URLs showing up in Google Webmaster Tools
Hi there, I'm noticing a bunch of URLs showing up in my google webmaster tools that are all cigarette related (they are appearing as 404s in the crawl error report). They are throwing 404 errors which is why they are listed here... Anyone have any idea of what this could be? I recently switched from Wordpress to Shopify and these weird URLs just started appearing on my webmaster tools in the last week. Kinda bizarre / a little alarming! Thanks,
Technical SEO | | TheBatesMillStore
Bianca0 -
Product Level 301 Redirects Best Practice
When creating a 301 mapping file for product pages, what is best practice
Technical SEO | | Bucktown
for which version of the URL to redirect to? Base directory or one
subdirectory/category path. Example Old URL: www.example.com/clothing/pants/blue-pants-123 Which of the following should be the new target URL: www.example.com/apparel/pants/blue-pants-123 www.example.com/apparel/blue-apparel/blue-pants-123 www.example.com/apparel/collections/spring-collection/blue-pants-123 www.example.com/blue-pants-123 This is assuming the canonical tag will be www.example.com/blue-pants-123. Also, if www.example.com/blue-pants-123 cannot be reached via site
navigation would it be detrimental to make that the target URL if Google
cannot crawl that naturally? Thanks0 -
301 redirect new site design
Hi I'm just setting up some 301 redirects for a new site design about to go live. The old site structure had some 'overview' pages in the urls (without any content) that just 302'd to a sub page. Do i need 301 redirect these overview page urls or since they had no content theres no need and I probably shouldn't or should i ? Also for pages that have no direct equivalent replacement is it still best to 301 to nearest relevant page or just leave it. For example a thank you page that currently shows after user submits email form wont be on new site (since message shows on form page after submission rather than new page). Should i 301 to form page or just leave it ? Cheers Dan
Technical SEO | | Dan-Lawrence0 -
301 Redirect / cross-domain canonical to a URL w/ Ampersand
I have a question regarding ampersands, we are needing to redirect to a URL w/ an ampersand in the URL: http://local.sfgate.com/b18915250/Sam-&-Associates-Insurance-Agency Will Google pass page authority/juice despite the fact that there is an ampersand in the URL, if we were to 301 redirect or cross-domain canonical to the url? Should we 301 redirect to http://local.sfgate.com/b18915250/Sam-%26-Associates-Insurance-Agency instead of http://local.sfgate.com/b18915250/Sam-&-Associates-Insurance-Agency? I don't have the option of removing the ampersand Thank you for your time!
Technical SEO | | Gatelist0 -
301 Redirect on a PDF, DOCX files?
Hi, I have to rename many pdf and docx files. How can I implement 301 redirect on them as they are linked from 'n' number of places? Regards, Shailendra Sial
Technical SEO | | IM_Learner1 -
Delete 301 redirected pages from server after redirect is in place?
Should I remove the redirected old pages from my site after the redirects are in place? Google is hating the redirects and we have tanked. I did over 50 redirects this week, consolidating content and making one great page our of 3-10 pages with very little content per page. But the old pages are still visible to google's bot. Also, I have not put a rel canonical to itself on the new pages. Is that necessary? Thanks! Jean
Technical SEO | | JeanYates0 -
Does it really matter to maintain 301 redirect after de-indexing of old URLs?
Today, I was reading latest blog post on SEOmoz blog about. Uncrawled 301s - A Quick Fix for When Relaunches Go Too Well This is very interesting study about 301 & How it useful to maintain traffic. I'm working on eCommerce website and I have done similar stuff on my website. I have big confusion to manage 301 redirect. My website generates new URLs due to following actions. Re-write dynamic URLs. Re-launch entire website on different eCommerce platform. [osCommerce to Magento Commerce] Re-name category. Trasfer one product from one category to another category. I'm managing my 301 redirect with old practice. Excel sheet data from Google webmaster tools and set specific new URLs for redirect. Hoooo... Now, I have 8.5K redirect in htaccess... And, I'm thinking it's too much. Can we remove old 301 redirect from htaccess or not? This is big question for me. Because, all pages are not hyperlink on external website. Google have just de-indexed old URLs and indexed new URLs. So, Is it require to maintain 301 redirect after Google process?
Technical SEO | | CommercePundit0 -
Looking for some help adding a 301 redirect for my Site
Hi there, I am trying to eliminate the 'www' using a 301 redirect script as I have duplicate page titles for both versions (with and without the 'www') I checked the page authority and found the pages without the 'www' to be ranked higher. For this reason I believe it would be wise to go for this option. I have an .htaccess file, all I need is the code and I should be ok 🙂 Thanks!
Technical SEO | | debeenus0