What tools produce a complete list of all URLs for 301 redirects?
-
I am project managing the rebuild of a major corporate website and need to set up 301 redirects from the old pages to the new ones. The problem is that the old site sits on multiple CMS platforms so there is no way I can get a list of pages from the old CMS. Is there a good tool out there that will crawl through all the sites and produce a nice spreadsheet with all the URLs on it? Somebody mentioned Xenu but I have never used it. Any recommendations? Thanks -Adrian
-
And the winner is... LinkExaminer! Pretty amazing tool, simple to use, but comprehensive - just the thing we need. Thanks for your help Marcin!
Adrian
-
Thanks Marcin and Robert. I will definitely give Xenu Link Sleuth a try first. The Search Engine Optimization Toolkit does look quite awesome though. Thanks for your help. - Adrian
-
There actually many tools which could do the job, both free and paid.
-
Xenu Link Sleuth - free, available at http://home.snafu.de/tilman/xenulink.html
-
Screaming Frog - free (limited)/paid, available at http://www.screamingfrog.co.uk/seo-spider/
-
A1 Website Analyzer - paid, but with a fully functional trial, available at http://www.pendragonexpress.com/hop/a1
-
LinkExaminer - free, available at http://www.analogx.com/contents/download/Network/lnkexam/Freeware.htm
-
Search Engine Optimization Toolkit - free, available at http://www.iis.net/download/seotoolkit
If you just need a crawl, I would recommend 1, 4 or 5. They are both free and work fast. If you want to optimize your content, 5 is a powerhouse.
Hope it helps.
Marcin
-
-
xenu linksleuth is the answer to your question. Good luck. Nothing better I am aware of.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Forced Redirects/HTTP<>HTTPS 301 Question
Hi All, Sorry for what's about to be a long-ish question, but tl;dr: Has anyone else had experience with a 301 redirect at the server level between HTTP and HTTPS versions of a site in order to maintain accurate social media share counts? This is new to me and I'm wondering how common it is. I'm having issues with this forced redirect between HTTP/HTTPS as outlined below and am struggling to find any information that will help me to troubleshoot this or better understand the situation. If anyone has any recommendations for things to try or sources to read up on, I'd appreciate it. I'm especially concerned about any issues that this may be causing at the SEO level and the known-unknowns. A magazine I work for recently relaunched after switching platforms from Atavist to Newspack (which is run via WordPress). Since then, we've been having some issues with 301s, but they relate to new stories that are native to our new platform/CMS and have had zero URL changes. We've always used HTTPS. Basically, the preview for any post we make linking to the new site, including these new (non-migrated pages) on Facebook previews as a 301 in the title and with no image. This also overrides the social media metadata we set through Yoast Premium. I ran some of the links through the Facebook debugger and it appears that Facebook is reading these links to our site (using https) as redirects to http that then redirect to https. I was told by our tech support person on Newspack's team that this is intentional, so that Facebook will maintain accurate share counts versus separate share counts for http/https, however this forced redirect seems to be failing if we can't post our links with any metadata. (The only way to reliably fix is by adding a query parameter to each URL which, obviously, still gives us inaccurate share counts.) This is the first time I've encountered this intentional redirect thing and I've asked a few times for more information about how it's set up just for my own edification, but all I can get is that it’s something managed at the server level and is designed to prevent separate share counts for HTTP and HTTPS. Has anyone encountered this method before, and can anyone either explain it to me or point me in the direction of a resource where I can learn more about how it's configured as well as the pros and cons? I'm especially concerned about our SEO with this and how this may impact the way search engines read our site. So far, nothing's come up on scans, but I'd like to stay one step ahead of this. Thanks in advance!
Technical SEO | | ogiovetti0 -
Can i do 301 redirect
So this is what im doing, 301 redirect to my site/allen-webdesign points to main domain Allen is the city i have a page called local-webdesign with all the cities. Will this improve my ranking or should i stop?
Technical SEO | | jsdfw0 -
I have multiple URLs that redirect to the same website. Is this an issue?
I have multiple URLs that all lead to the same website. Years ago they were purchased and were sitting dormant. Currently they are 301 redirects and each of the URLs feed to different areas of my website. Should I be worried about losing authority? And if so, is there a better way to do this?
Technical SEO | | undrdog990 -
Is there a tool to see all redirects?
I'm thinking this is a silly question, but I've never had to deal with it I thought I'd ask. Ok is there a tool out there that will show all the redirects to a domain. I'm working on a project that I keep stumbling on urls that redirect to the site I'm studying. They don't show up in Open Site or ahrefs as linking domains, but they keep popping up on me. Any thoughts?
Technical SEO | | BCutrer0 -
Creating a Target URL For 301 Redirect in Wordpress
I am confused as to what to put in as the target URL. Is this just a new URL that I must create a name for? I am having trouble finding any answer for this on the internet, just more people asking the same question. I am finally realizing that all the information is found right here at SEOmoz to both learn and ask questions about.
Technical SEO | | lartinos0 -
Redirected Subdomain Development URLs Showing In SERPs?
I develop client websites within a subdomain of another website (with noindex, nofollow so that incomplete websites on the wrong domains aren't ever seen by web users). Then, when we launch a client's site on their own domain, we redirect all of the development URLS to the appropriate page on the new live site. (meaning at site launch, all pages on http://client-site.developersite.com would be set to 301 redirect to identical pages pages on http://www.client-site.com). This system has always seemed to work fine, but today I discovered 94,700 pages indexed by Google on my root domain and found that these were mostly old URLs of sites in development that redirect to the actual client sites. Many are several years old. Any idea why Google would be indexing these pages? Thanks in advance!
Technical SEO | | VTDesignWorks0 -
Not sure which URL to use for 301 redirect
A client has new website design completed by another developer, was launched in April of this year. No 301 redirect was set up so duplicate content is an issue. Client has had a website with same domain name for about 10 years, but has not had any SEO work completed before or since his new site design. For non-www there are 6 referring links - 1 considered to have authority, for www there are also 6 but 3 considered to have authority. More links seem to coming from www than non-www. But for one of the clients keywords they are ranked #1 for their area and that links to their non-www address. And even though no redirects set up by developer, non-www has had far more visits according to Google Analytics. So many basics that still need to be done for site: no meta-descriptions on any page, H1 and page titles could use keywords, call to action moved above fold, etc. Considering this is a new site, and new SEO work and many more inbound links needed, does it matter which address I redirect to? _Cindy Barnard
Technical SEO | | CeCeBar0 -
Crawl Tool Producing Random URL's
For some reason SEOmoz's crawl tool is returning duplicate content URL's that don't exist on my website. It is returning pages like "mydomain.com/pages/pages/pages/pages/pages/pricing" Nothing like that exists as a URL on my website. Has anyone experienced something similar to this, know what's causing it, or know how I can fix it?
Technical SEO | | MyNet0