Block web archieve/way back machine
-
Hi i want to block web archive/wayback machine from indexing my site and creating a record of it on their database.
Any ideas on how to do this?
Cheers,
Superpak -
You can block Wayback Machine from crawling and creating a record of your site by adding the following to your Robots.txt file:
User-agent: ia_archiver
Disallow: /This will not only stop new records from being created but also stop people viewing what had previously been indexed by Wayback Machine.
More information about this can be found here: https://archive.org/about/exclude.php
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to get back links with higher rank ?
Hi All , These days I am finding new ways of creating back links. Could any one tell me how to get backlinks with higher DA ?
Intermediate & Advanced SEO | | mozentution2 -
Best way to get love from expired domain
Hi so i have bought a couple of expired domains (which when they were live had good PA , link profile etc) my question is is it better to just 301 the domains to my main site or to set up a "landing page" for the expired domain and then put a number of links on this page to my main site which would give me the most benefit google-wise ? obviously 301ing would be the easiest !
Intermediate & Advanced SEO | | odysseytravels0 -
Risks of moving a site back to an old domain
Hi Nearly 2 years ago our site was moved from a .co.uk domain to .media. Because this TLD isn't supported for DA, it's hard to measure whether linkbuilding campaigns (for example) are having a positive effect. The old site has a DA of 38 (even after 2 years of inactivity) and the new one is 1, but the new one has better Trust and Citation flow, for example. I'm now investigating whether it's worth moving back to .co.uk and I want to fully understand the risks involved. So far I know of the following potential risks: It's a lot of work so human error is a real risk Could create a redirect loop as the old site has 301 redirects in place to the new one It will take several months for metrics to recover Any thoughts on more risks, how these challenges can be overcome etc will be welcome. Or do I just set fire to the lot and create a new site with yet another 301 redirect from the .media site? What would you do?
Intermediate & Advanced SEO | | AxonnMedia0 -
Rank Without any back linking ?
Hello, I am thinking , I will rank my site without SEO . I will add quality content on my site . Will not do any backlinking ! Can anyone suggest me if it is really possible. I know some guys or some websites those are ranked without any SEO attempts. Thanks
Intermediate & Advanced SEO | | businessowner0 -
What is the recommended way to save Image Files in WP?
Hi, Is there a recommended setting (or even plugin) to use when saving image files on my Wordpress blog?
Intermediate & Advanced SEO | | BeytzNet
Which folders etc. Thanks0 -
Penguin/Panda/Domain Purchase
If I move forward with the acquisition: 1. Should I, if there is a way, just acquire the domain and then attempt to unlink existing links? 2. Can I just buy the domain, completely kill the site, and then build again from scratch? Even if I do that, the links to the domain will still be out there. 3. Should I even move forward with the purchase if I know these tactics have been used? Thanks!
Intermediate & Advanced SEO | | dbuckles0 -
Best way to clean up a nasty backlink profile?
A new client of mine sadly has a TON of terrible links (3800 links from 1500 domains) which are pointing to landing pages that have been created specifically for manipulating engines. Besides contacting these sites and asking to have the links removed the only solution I can think of it to delete these pages and let them 404. Obviously I am not thrilled about that but I'm not sure what else to do. Does anyone have any other ideas for how to clean up this backlink profile? Thanks
Intermediate & Advanced SEO | | LukeMontgomery0 -
Web fonts & SEO
Hi everyone ! My question is regarding web fonts. We are currently working on a new design for our website and we're thinking about using web fonts instead of images containing the fonts we'd like to have. I'd like to know if web fonts can affect SEO as they need to be downloaded on the visitor's computers and consequently can slow down the load time of our web pages. If anyone has used web fonts in the past, do you have some specific tips to share ? Thank you in advance for your answers! Jeremie
Intermediate & Advanced SEO | | Maxxum0