There are so many questions here really before it's clear what the cause could be, some of these include:-
- Are these from other sites you own? One would presume so if you access to remove the links?
- If so, are these sites hosted on the same server as the domain they were pointing to?
- Did the sitewide links contain keywords as the anchor text or were they branded?
- Why did you choose to remove the sitewide links in the first place?
- Did any of the other websites containing these sitewide links suffer recently as a result of algorithm updates or decline in rankings.
- The page(s) that the sitewide links were pointing to, does this have other existing links still pointing to it? What % were they were reduced by after the removal of the sitewide links? And of those existing, are they a mix of nofollow/dofollow?
Often, the removal of mass of backlinks will cause a drop as you've experienced. Normally because it's a major shift in the link profile and doesn't look 'natural', providing there was legitimate reasons and there are no other issues that can be raised against the domain in question, often, ranking positions are pretty much reinstated again.
Incidentally, how we often get round this scenario is to keep the sitewide link in place but write a script to only make the link on the homepage followed - and all the others add the nofollow link relationship.