Hi
According to Matt Cutts a 404 means page is gone (but not necessarily permantent) - permanent would be 410 (although he also indicates that there is very little difference between these two from SEO perspective.
What to do with these 404 depends a bit on the situation
-
if these pages have external links pointing to them - I would try to redirect (even better ask the one who's linking to update the links although maybe difficult in practice)
-
if these are old url's which haven't been used for a while & don't generate traffic - just leave them - they will disappear
-
where do these 404 come from - if they are just listed in WMT - you can ignore them. If it's actual people trying to visit your site on these pages I would try to redirect them to the appropriate new page (if not for the SEO than for the user experience)
-
check that no internal links exist to these old 404 pages (Screaming Frog is made for this)
You say you 301'd the pages of the old site - did you check your landing page report in Analytics before migration - you should make sure that these top 5000 url's are properly redirected - normally Google will figure it out after a while, but it can have a negative impact on your results if a lot of these landingpages generate a 404.
As additional resource - probably a bit too late now you could check the different steps in this guide: http://moz.com/blog/web-site-migration-guide-tips-for-seos to be sure that you didn't miss something important.
Hope this helps
Dirk