Certainly sounds dodgy, but suddenly removing all of those backlinks might cause you some SEO issues.
Depending on how Google is currently reading your site it may improve as your site would seem less spammy without them or it may really hurt the site (at least to start with) loosing that many back links would maybe make Google think something is up with your site?
I would take the bullet and remove the duplicate content but warn your clients that it may take a while for the natural benefits to come through. Because if your site isn't penalised yet for having that many dodgy backlinks and duplicate content it soon will be!