Thanks Cian,
It is an algorithmic penalty (likley primarily Panda). Significant recovery work has happened since over a year ago but we are not seeing any recoveries despite the recent Panda refreshes.
What I hear you saying is try to avoid any cross connections (including GA id etc) and start fresh and maybe repoint some valuable links?