Block web archieve/way back machine
-
Hi i want to block web archive/wayback machine from indexing my site and creating a record of it on their database.
Any ideas on how to do this?
Cheers,
Superpak -
You can block Wayback Machine from crawling and creating a record of your site by adding the following to your Robots.txt file:
User-agent: ia_archiver
Disallow: /This will not only stop new records from being created but also stop people viewing what had previously been indexed by Wayback Machine.
More information about this can be found here: https://archive.org/about/exclude.php
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
SEO Strategy - Content/Outreach/Links
Hi everyone I'm trying to prioritise my tasks for 2018 & wondered if anyone had any useful templates they use? In terms of SEO tasks, my priority was going to be content/outreach/links - Focusing on user guides/blogs onsite Then outreach articles/some PR that doesn't go against Google guidelines offsite. My struggle with the onsite content/blogs we produce is we have no real social media plan/manager so my content outreach always seems hampered by this. I've tried taking on some of the social stuff, but this ends up being too much for just me to do. I wondered if there were any other SEOs who face this issue and who have found some good solutions? I'm stuck in a bit of a rut and can't seem to effectively push forward with outreach/content writing. Thank you Becky
Intermediate & Advanced SEO | | BeckyKey1 -
HELP!!! Steep Drop in Organic Traffic Starting 11/1/16
Starting November 1st, organic web traffic from Google dropped from an average of about 60 visits a day to about 5 per day. So we are more than 90% off!!!! At the end of September, we modified the header of the site to simplify it. We also added a snippet of code to each page to enable Zoho "Sales IQ" to work. Sales IQ enables us to track visitors and engage in chat sessions with them. Apart from that no changes have been made from the site. Any ideas as to what could have caused this drop in traffic? Was there a Google update at that time that could have caused the drop? Or could the recent site changes have caused this? I have attached a Google Webmasters Tool report showing the drop in traffic. I would very much appreciate some insight into this, as all organic traffic to our site has ceased. Thanks,
Intermediate & Advanced SEO | | Kingalan1
Alan 9VNB1O50 -
Reverting back to old domain name.
I've recently been asked by a client if I can foresee any issues with reverting back to their original domain name. With the original domain name they had a pretty decent DA for their sector which they have now lost. Although I do appreciate that over time this might come back, the CEO is very keen to switch back to the old domain. They do currently have 301 redirects from the old domain to the new and have implemented rel canonical. As yet they have not notified Google of the change of address using Webmaster Tools. Can anyone forsee any issues with returning back to the old domain name? They have only been using the new domain name for a couple of months which currently has a DA for 1.
Intermediate & Advanced SEO | | Macrofireball0 -
Block lightbox content
I'm working on a new website with aggregator of content.
Intermediate & Advanced SEO | | JohnPalmer
i'll show to my users content from another website in my website in LIGHTBOX windows when they'll click on the title of the items. ** I don't have specific url for these items.
What is the best way to say for SE "Don't index these pages"?0 -
Google v bing/yahoo
Had a penguin unnatural link building penalty was lifted last June. Since then no significant recovery in rankings. Saying that why do we still rank on page 1 on Bing and Yahoo but rank nowhere in Google. Any suggestions. Thanks David
Intermediate & Advanced SEO | | Archers0 -
Doing large scale visual link/content analysis
Hi i currently have a list of about 5000 URLs i want to visually check quickly, to identify decent content. I'm currently opening 200 at a time with firefox, more than 200 it gets really choppy and slow as you would expect. I was wondering if anyone knew any other ways of opening a large amount of web pages. It would be sweet if there was a tool which can scan a list, add the webpages to a pdf/powerpoint and send them back to you for analysis. Kind Regards, Chris
Intermediate & Advanced SEO | | Mikey0080 -
Link Audit - Sponsor/Partners Images Links
Hi everyone, 1. I'm conducting a link audit and read that if you are a sponsor or partner of a company, links should be nofollowed. I always no follow them if they are money keywords, but branded I leave alone. is that a good strategy? Or do i nofollow my brand name as well? 2. What if I'm a sponsor and have my company logo on their website that links to my website? How would i know if that link should be nofollowed? a. Does it depend on the "alt" of the image? b. Does it depend on the landing page of the link of the image? c. Do i just no follow image links from sponsor pages and partner pages as a whole? Please keep in mind that I'm sponsoring websites that are relevant to my niche. PLEASE HELP!
Intermediate & Advanced SEO | | Shawn1240 -
Any advice for my website http://cvcsports.com?
I run the website http://cvcsports.com for myself and my parents. We offer custom varsity jackets for athletes/companies/etc. We rank first in Google for "letterman jackets" and near the top for "varsity jackets". I really want to reach #1 for "varsity jackets" (we were briefly #1 a few days ago but didn't stay there). Does anyone have any advice on what I can do to achieve that? Thanks in advance for the tips!
Intermediate & Advanced SEO | | BrandonDoyle0