Right, you seem to be asking two questions here - responsive or not? And if you head for responsive then could it impact your SEO.
Responsive or Not As with any website question the issue is going to come down to what's best for your users or your target users. The same question could be 'app or mobile website' for example.
The more I've worked in web design the more I am seeing that when a user is searching on their phone they want the same answers as if they were searching on their laptop or desktop. The relevancy of what they are delivered should not be changed however the format must be changed to suit their device.
The above point being said about 'best for your users', I fall very much into the line of thought that you should be providing exactly the same content to mobile and non-mobile users, it will simply be the design or layout which changes.
A responsive site takes time to code and test - but once the wireframe is sorted and responds well to different devices, then you're sorted as each page should flow across the devices without an issue.
If you have a separate mobile website then you are suddenly coding and managing two websites and, to be honest, a mobile website will need testing across devices and tweaking accordingly so you're almost duplicating your work (you're doing responsive web design but on a second site), something I just don't see the point of, if your whole website fits the majority of devices accessing it.
So for me: responsive
Can Responsive Affect SEO? You need to make sure that it is done well and that you're not deluding the search engines or users in any way. Personally, I don't see the point in 'hiding a sidebar' when responsive web design and CSS permits you to reformat it and display it in a mobile-friendly way. Why reduce the mobile user's experience if you, with a bit more work, can give them an appropriate and rich experience?
So if you do it properly, you're providing the same content to mobile users but just showing it differently. If you keep that in mind then there should be no negative SEO implications and you never know, your conversions from mobile users and referrals/shares from mobile users may increase above your competitors because you've taken time to give them a great experience.
Hope this helps - you're not dealing with small issues - we're in the middle of recoding our website for responsive web design, so all the best as you make these decisions.