Listing of all Google Indexed Pages

[dhands17]

I started managing a site that has about 391,000 indexed pages.  I want to get to the bottom of why there are so many in preparation for a ecommerce Migration and improving SEO.  Anyone know of a tool?  Many tools I have came across can only take 100 at a time.  I would love to get them in excel or a database.  I look forward to the suggestions.

[KeriMorgret]

Using site:yourdomain.com in Google, and then going to the end of the results and telling it to show you all of the results, is a good first start. It should get you enough to get an idea of why there are duplicated pages.

The Moz crawl can also help you figure it out, as often with ecommerce you'll have URLs for sorting products by price, name, pagination parameters, etc. We'll throw up a flag when we see a bunch of duplicate content or duplicate titles.

Also look for the easy stuff, such as non-www doesn't direct to www. Fix that, and you've cut your pages in half.

[CommT]

I may not be answering this correctly...

Are you looking for a list of URLs? If so, easy peasy to use screaming frog.

If it's all the pages Google has indexed, I don't really know and I'm sorry! However, I will come back to this thread to see if someone else has the answer for you, because I'm quite interested in it myself!!!

Best of luck,

Amelia