We 410'ed URLs to decrease URLs submitted and increase crawl rate, but dynamically generated sub URLs from pagination are showing as 404s. Should we 410 these sub URLs?

[jeffchen]

Hi everyone!

We recently 410'ed some URLs to decrease the URLs submitted and hopefully increase our crawl rate.

We had some dynamically generated sub-URLs for pagination that are shown as 404s in google. These sub-URLs were canonical to the main URLs and not included in our sitemap.

Ex: We assumed that if we 410'ed example.com/url, then the dynamically generated example.com/url/page1 would also 410, but instead it 404’ed.

Does it make sense to go through and 410 these dynamically generated sub-URLs or is it not worth it?

Thanks in advice for your help!

Jeff

[jeffchen]

Awesome - thanks for your help Mike! I really appreciate it!

Jeff

[MikeRoberts]

You could but its not completely necessary to go through all those sub-pages to 410 them. While a 410 Gone response is a stronger signal, those pages serving 404s will eventually be removed from the crawl and/or SERPs by the bots anyway. So if those pages are just dynamically-generated flak, and don't provide anything of benefit, then leave them as 404s and don't worry about it.