Just implemented something similar to this, and used canonicals. Also, if you're able to add more than just canonicals, possibly worth looking at microdata? We used schema.org isVariantOf for colors and size variants, not sure how much this influences googles understanding / search display, but it's widely recommended and seems unlikely to hurt. Implementing took a little trial and error, this helped as did google's schema testing tool.
