Google is pretty "smart" and will understand that it is an e-commerce site with thousands of products and it won't penalize your site for keyword stuffing, but make sure you follow Google's guidelines, such as: category and product pages have unique page titles, H1 headings, image ALT tags, meta descriptions and all the other basic on-site elements make sense.
Also, it's recommended to use structured data for breadcrumbs, so the search robot will understand the information architecture of your website.
Sometimes it's difficult, but it's also a good practice to add a few sentences to at least the major category and sub-category pages. E.g. some intro text about the wide range of shirt, brands, etc, so this on-page content will make the category pages more unique and more relevant.