I'm reading this response and this is happening on my site as well.  How did this happen in the first place?  I have duplicate content because of https and http copies of all my web pages.  If I type https://www.mywebsite.com I can't get to my site.  Could this be coming from my hosting company?  I've set up my site to simply be http://www.mywebsite.com.  I'm a little worried to change my robots.txt and I would love to know how this happened in the first place.