Hi Paul,
I'm not 100% on the actual "size" of the image having any negative effects. In my experience it's directly related to how well the image is optimized. Yes, load time ABSOLUTLEY has an effect on rankings, and while some will say that it's a small effect, I contend that it's an important consideration.
While Google may not give it primary consideration in their algorithm, it can drive your users away as they wait for a page to load. People are going to wait a mere second or two before they back out of a page that is not loading. So bounce rate is the factor you need to weigh with regard to image size.
Other tips to optimize your image properly include, always add height and width to the image for faster loading, always add an Alt-attribute to the IMG-tag, always add a Title-attribute to the IMG-tag, etc.
Best of luck,
W