Hi Ewan,
This is a question that probably deserves a blog post at some point, along with a number of related questions about link-based penalties. I've been gathering info for some time, and have seem many instances of redirecting sites that have been penalized. I wish I could collect data on penalized sites more scientifically, but we work with what's available.
Manual Penalties
Manual penalties appear to carry through to new domains almost instantly when redirected to pages housing the same content. Google appears to use a number of signals to make sure that the redirect is to the same site and not to a competitor.
Some Googlers have claimed that if you received an "unnatural link" warning in WMT, it's manual. I'm not entirely convinced of this, but it's now harder than ever to differentiate between manual link-based penalties and Penguin algorithmic adjustment. That brings us to...
Refreshing Adjustments (Penalties)
Panda, Penguin, and a few other updates are a little different. We've seen instances where a user makes a big change (a complete redesign for Panda, or redirecting the entire site); the trend seems to be a brief recovery followed by a drop once the algorithm refreshes.
The obvious up-side here is that if you were going to recover from the penalty anyway, you may start to recover a bit sooner and don't have to wait for the next refresh. The downside is that it's a lot of work to do correctly, and it might be a very short-lived change.
--
Generally, I'd say it's best to clean up the site and keep going on the same domain. If you have a lot of bad links pointing to a specific page, you may want to 410 that page and start a new one, then mention this in your reconsideration request. Otherwise, it's the old process of removal (keeping notes) and using disavow if reconsideration and clean up prove insufficient.