Another option is if the site uses a CMS. If so, then you can create a sitemap for content pages/posts etc,.

Personally, I'm with Krzysztof Furtak  on SF. Screaming Frog rocks. It'll find most pages, except perhaps Orphan pages as it wouldn't be able to find a link to crawl to discover the page.

If it's really important to get as many pages as possible, I'd do the following (I've put an Astrix (*) next to ones that some people may think are a tad extreme)

• Run a Screaming Frog crawl
• Grab a sitemap from your CMS
• Check any server-based analytics (AWSTATS etc)
• Check your access_log file & parse out URLs in there**(*)**
• site: queries, with & without www, and also using * as a subdomain (use something like Moz's toolbar to export)
• As Krzysztof suggests, Scrapebox would extract data too, but be careful scraping, you may get an IP slap.(*)
• Export crawl data from Moz & a tool such as Deep Crawl
• Throw the pages from all into Excel and de-dupe.
• Once you have a de-duped list, as an optional last step, go back to Screaming Frog and enter list mode (I have the paid version, not sure if it's possible with the free one) and run a crawl over all the de-duped URLs to get status codes etc

If you're going to do this sort of thing a fair bit - buy a Screaming Frog license, it's an awesome tool and can be useful in a multitude of situations.

The site: command is handy for asking Google what pages it knows about, however if Muzzmoz wants to know the number of pages on a site, you'd need more than this.

Also, re: your different ways or querying, I like to use:

site:*.domain.com - This can show other subdomains too, that may otherwise be missed

Good answer Dirk.

I like your idea of adding valuable, relevant content to the pages Dirk, good thinking.

Personally, I'd rather Iet Google know these pages are removed intentionally and not due to errors, so 410 rather than leaving as 40.

One thing to be mindful of, though, is how much crawl budget you're willing to give to these pages. If we're talking about a lot of pages in bulk, I'd be worried how much crawl budget they'd eat up over time. As you point out, they'd likely drop in rank anyway due to loss of internal links too, so might be the cost to the crawl budget isn't worth it?.

Another solution (using your idea Dirk), would be to somehow automate the process of, when a listing is marked as sold, the listing is removed, other properties in the same area are added (as you suggest),  then some time later (month or two?), a 410 header set.

The other option would be to 301 the old pages back to the area page for the properties (perhaps with something like a bootstrap message saying the property is sold but others in the area are available). This would pass juice etc back to that page. but, of course, you'd be telling G that the page had permanently moved, which isn't quite the case.