Extreme high number of pages found on webshop

[J05B]

Hi,

Im working for the first time on a magento webshop.  But i run into a problem where crawlers find then thousands of pages while there are a few hunderd products.

I expect is has something to do with filters that generate dynamic URL's.  I can't find any setting in Magento to prevent this and i think this will hurt SEO performance because  of duplicate content and high amount of pages that need to be crawled while the site has no authority.

What would my approach be to solve this? Do i need to ad certain tags to the pages or are these settings in my robots file.

[PaddyM556]

Answers above is spot on,

A comprehensive robots file should help with this, if your in a hurry to clean the indexation up, you a can remove URLs witihin the search console.

One other thing to consider is, your xml sitemap. look at this in detail & what your asking the search engine to crawl.  End of life products, legacy catalogues should not be included.

[WebQuest]

This could possibly be because of the different filters and combinations you have you your category and sub-category pages. I would recommend doing the following :

• Block access to unwanted pages through robots.txt

• Setup the canonicals properly so that all different types of filtering combinations will always point to the main category/sub-category pages

• Make sure your Pagination is properly setup with each page having its canonical pointing to  itself and the "rel=next/prev" included (although Google has mentioned that it has stopped supporting those, I would still recommend implementing them)

Here's a good guide on Magento SEO:

https://paulnrogers.com/magento-technical-seo-issues-to-look-out-for/

Hope that helps!