Block web archieve/way back machine
-
Hi i want to block web archive/wayback machine from indexing my site and creating a record of it on their database.
Any ideas on how to do this?
Cheers,
Superpak -
You can block Wayback Machine from crawling and creating a record of your site by adding the following to your Robots.txt file:
User-agent: ia_archiver
Disallow: /This will not only stop new records from being created but also stop people viewing what had previously been indexed by Wayback Machine.
More information about this can be found here: https://archive.org/about/exclude.php
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to get back links with higher rank ?
Hi All , These days I am finding new ways of creating back links. Could any one tell me how to get backlinks with higher DA ?
Intermediate & Advanced SEO | | mozentution2 -
Google Bot / SEO and Cookies
Hi, I'm trying to monetise my website via a paid subscription / paid content strategy. The plan is, after the user is on the website for 1min 30secs or clicks 3 map markers or visits 3+ pages A popup will appear asking for a signup + payment Cookie will be set for 6 months, if a users returns and cookie is detected (php) they will be redirected to sign up page My site relies heavily on organic SEO, so my question is: Will google bot be presented with this sign up stuff? Does google bot set cookies? will everything be indexed properly... And what affects on SEO?
Intermediate & Advanced SEO | | thinkLukeSEO0 -
Product/service keyword in business title????
Is this Ok to add a keyword with business name in the google places (without having in the domain name)? While we create google listing for new business? Please view attached image. Source - http://moz.com/blog/top-20-local-search-ranking-factors-an-illustrated-guide (13. Product/service keyword in business title) dS7FWJL.png
Intermediate & Advanced SEO | | Dan_Brown10 -
Block a country, will affect my ranking?
Dear Mozzers, I intend to block some certain countries from viewing my website (including proxy), will it affect my Google ranking? Thank you for your help. BR/Tran
Intermediate & Advanced SEO | | SteveTran20130 -
How important is the HTML structure for on-page/on-site SEO?
To be more specific, say a page layout has Header, Body, Left Sidebar, Footer sections. Which layout from the following options is more SEO-friendly? Header > Body > Right Sidebar > Footer Body > Header > Right Sidebar > Footer Does it make a big difference to code HTML so that the the copy of the body appears in front of all other sections when spiders crawl a website? Is it worth taking extra steps to make this happen? I am asking this question because our site has a header navigation with a lot of dropdown menus. So I assume that this is "noise" for spiders as it pushes the main content of the page down. Please bear in mind that the question is more geared towards how search engine see the page rather than how it appears to the end user as layout can be controlled by CSS.This question also assumes that all other on-site SEO best practices are followed for both options.
Intermediate & Advanced SEO | | Saugar0 -
We will be switching our shopping cart platform from volusion to magento and really cautious / nervous about our rankings / seo stuff. any advice for anyone that has migrated stores, etc. these urls are years old, etc.
shopping cart platform switch and SEO. What do you suggest? What's the best way to ensure we keep rankings.
Intermediate & Advanced SEO | | PaulDylan0 -
Redirecting, then redirecting back
Hey, mozzers! My first question ever... I have a client who has (fictitionally) WickerPatioHomeStore.com, which features wicker home decor. Not too long ago, they wanted a shorter, easier URL, so they redirected to another domain they own, WickerPatio.com (again, fictional). They saw somewhat of a drop in traffic, and wonder if there's a correlation with the words "home store" not being in their domain any more. When considering the two, I figure that relevant factors would be age of domains, history of content of the domains, and inbound links to each domain. Any thoughts on other things to consider? Thanks very much!! ~ Scott
Intermediate & Advanced SEO | | GRIP-SEO0 -
Why does this show up in my browser/index2.php
i type is a clients site and BOOM...it suddenly ends with "index2.php" who will link to that? I can't seem to make it not appear...according the template creator... Is this "index2.php" affecting my link juice?
Intermediate & Advanced SEO | | SEObleu.com0