You are dead on with this and now I feel like a idiot
I want to point out that below the infographic on my page is a whole bunch of text that I wrote to go along with it.
I also did something really stupid a few weeks ago: I updated all my links on Ezine Articles. So, to Google, it looks like my website got a bunch of links all at once - very unnatural and I'm kicking myself for doing that. I just deleted all my article from there because they were shitty and I don't want my name on them anymore. The content I create is far superior now than it was 4 years ago.
I am working on cleaning up my link profile for 2013 and i will take your advice about the infographics. They get me a lot of links, but way to fast and all with the same anchor text. I like the idea of just asking people to give me credit if they want to use my infographic.
Thanks for the help!