Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Block web archieve/way back machine

Intermediate & Advanced SEO

930

Mikey008 last edited by

Hi i want to block web archive/wayback machine from indexing my site and creating a record of it on their database.

Any ideas on how to do this?

Cheers,
Superpak
1 Reply Last reply
Reply Quote 2
davebuts last edited by

You can block Wayback Machine from crawling and creating a record of your site by adding the following to your Robots.txt file:

User-agent: ia_archiver
Disallow: /

This will not only stop new records from being created but also stop people viewing what had previously been indexed by Wayback Machine.

More information about this can be found here: https://archive.org/about/exclude.php
1 Reply Last reply
Reply Quote 9

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Page / Domain Authority Question

If my website were to purchase a sponsored article on a site with a powerful domain authority that contained a do-follow link, and the link would be "domain.com/articles/new-article" ... obviously new-article would have 0 page authority, being new... is that still considered a valuable link and why or why not?
Intermediate & Advanced SEO | | cat5com

0
Domain.com/postname vs. Domain.com/blog/postname

I am wondering what is the best practice regarding blogs? I read that it would be best to structure a website like a pyramide instead of a flat panckage But I have seen many blogs where the post shows right after the domain name. Domain.com/postname instead of Domains/blog/postname My point is that if a website has many post then the structure will get very flat and this will maybe make your most optimized and important pages less important to google domain.com/page a) What do you think about this, which one of the two blog solutions do you prefer and why? b) in context to blog If for instance you had a keyword like Copenhagen property would you then consider renaming your blog to realetateagent.com/Copenhagen-property-news/post-name c) Would write a little intro like 200 words for the page 1 of your blog and add in some keywords.
Intermediate & Advanced SEO | | nm1977

0
Keyword Targeting / Cannibalisation

Hi Guys We're about to launch a very large website for a flooring company and would like to find out more about _key word _cannibalisation - to put my mind at rest. I know Rand posted a Whiteboard Friday early last year about this topic and mentioned using part of the same keyword was ok to use. All our keywords are specifically geared for "user intent" meaning each keyword has relevance and the content to back up the keyword. We've ensured the keywords are located within each url, placed at the start of the page title, h1 etc.
Intermediate & Advanced SEO | | GaryVictory

1
Block a country, will affect my ranking?

Dear Mozzers, I intend to block some certain countries from viewing my website (including proxy), will it affect my Google ranking? Thank you for your help. BR/Tran
Intermediate & Advanced SEO | | SteveTran2013

0
Need to know alternate ways to promote a Weekly Campaign

Hi Everyone, I first thank for all people who read this post and offer to help me.. 1. My client is going to start a weekly campaign starting from tomorrow its a online flower delivery company 2. They have already setup the Google ad words for it but he wanted to know is there any other way he can bring visitors to this campaign. 3. I have setup email blasting apart from this anything you people can suggest and little explanation too.
Intermediate & Advanced SEO | | dineshsearch

0
Term steady, then jumps up for a few hours then back, why?

Hi, So I have a page which is steady for months, sits at say position 15 then every few weeks it will jump to position 7 for maybe 24 hours then back to position 15 again. I assume it's trying to work out if we should be higher up the results by putting us in a higher position and monitoring the CTR and other metrics. Anyone got any other ideas? And when it does this can you do anything to signal this position is better? Maybe when it moves, ramp up social activity, shares etc. Thanks.
Intermediate & Advanced SEO | | Bondara

0
Best way to migrate to a new URL structure

Hello everyone, We’re changing our URL structure from something like this: example.com/index.php?language=English To something like this: example.com**/english/**index.php The change is implemented with mod_rewrite so all the old URLs can still work We have hundreds of thousands of pages that are currently indexed with the old URL structure What’s the best way to get Google to rapidly update its index and to maintain as much ranking as possible? 301 redirect all the old URLs to the new equivalent format? If we detect that the URL is in an old format, render the page with a canonical tag pointing to the new equivalent format as well as adding a noindex, nofollow tag? Something else? Thanks for your input!
Intermediate & Advanced SEO | | anthematic

0
What are the best way to get a new subdomain ranked properly

Our main site (blog with 700 high quality articles) ranks pretty well and we recently launced a rapidly growing forum (55.000 posts in the first 11 weeks) on a subdomain. What would be a good strategy for ranking the forum quickly
Intermediate & Advanced SEO | | xpd

1