That's a good idea - you could even display a banner ad that pops up anytime a user accesses the site "Shipping on all products delayed x weeks due to global pandemic". I think people would be pretty understanding.

Depending on what CMS you're using you should be able to add a meta no-index tag sitewide fairly easily. I know with WordPress there's an option in the backend under "Setting", "Reading" Discourage search engines from indexing this site that should allow you to apply a meta no index tag site wide.

If you're not on WordPress you should be able to edit your code and apply a no index sitewide. You might need a tool like FileZilla so you can set up some sort of FTP access and edit your Header file directly.

Cheers

I would leave all the disallows out except for the /wp-admin/ section. For example, I'd rewrite the robots.txt file to read:

User-agent: *
Disallow: /wp-admin/

Also, you kind of want Google to index your cached content. In the event your servers go down it will still be able to make your content available.

I hope that helps. Let me know how that works out for you!

Hey there --

Blocking resources with the robots.txt file prevents search engines from crawling content the no-index tag would be better suited for preventing content from being indexed.

However, previous best practice would dictate blocking access to /wp-includes/ and /wp-content/ directories, etc but that's no longer necessary.

Today, Google will fetch all your styling and JavaScript files so they can render your pages completely. Search engines now try to understand your page's layout and presentation as a key part of how they evaluate quality.

So, yeah this might have some impact on your SEO.

Also, if you're using a plugin to cache content you should want Google to crawl your cache content. And in my experience, Googlebot does a good job of not indexing /wp-content/ sections.

So, for your example page, https://example.com/wp-content/cache/minify/df983.js it shouldn't end up in their index.

Hope this helps some.