At what point is there diminishing returns from getting multiple links from one website?

Edit: To clarify, when doing competitor research I come across lots of sites that have say 500 linking domains with 5000 total links. When setting goals and calculating the types of links I should go after, I'm not certain which number is the one actually helping the site to rank.