Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Block web archieve/way back machine

Intermediate & Advanced SEO

930

Mikey008 last edited by

Hi i want to block web archive/wayback machine from indexing my site and creating a record of it on their database.

Any ideas on how to do this?

Cheers,
Superpak
1 Reply Last reply
Reply Quote 2
davebuts last edited by

You can block Wayback Machine from crawling and creating a record of your site by adding the following to your Robots.txt file:

User-agent: ia_archiver
Disallow: /

This will not only stop new records from being created but also stop people viewing what had previously been indexed by Wayback Machine.

More information about this can be found here: https://archive.org/about/exclude.php
1 Reply Last reply
Reply Quote 9

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

How to hold a variable constant for an A/B test

For example, let's say you want to A/B test a title tag change. You are hoping to identify whether a title tag change increases CTR. But, position is always fluctuating a bit and that affects CTR, too. So, I'm interested in how you could hold position constant in order to isolate the change in CTR that is due to the title tag change. Does anyone know of resources/tools/tutorials for how to do this? It's been... a very long time since I took statistics (-: I have access to Excel, MS Access, and R studio.
Intermediate & Advanced SEO | | LivDetrick

0
Easiest Way to Balance Links Across Site?

I'm struggling to reach the last few spots for my client's main keyword, hovering around mid-page on the first SERP. I have continuously built more links to this page but have not seen a correlation in movement, until I finally realised that I have too high a ratio of links pointing to the home page relative to those pointing to other pages on the site, which doesn't look natural (stupidly, for the last year we have mainly only been trying to rank the home page). I already have links on most UK directories - since the links I need are really just safe links (they don't need to have power), can anyone suggest the best/cheapest source of link-building that I could use to point more links to other pages on the site, to balance the site's overall profile? A press release, perhaps? Thanks in advance!
Intermediate & Advanced SEO | | zakkyg

0
Fast/Easy Way to Implement Canonical tags in Bulk in Magento CMS?

Hello Amazing SEO Community! Quick Q for a client with a TON of duplicate content. (yikes!) My client is currently undertaking a large SEO project around canonical tagging for their thousands of duplicate pages. Currently, one product sits on multiple URLs and they are being indexed as different pages (with the same content). The issue is found across all products and other pages, and across their international sites as well. One core challenge they face now is lack of time/resources from their developer side. The solution we see to the duplicate content is to manually add a canonical tag to each of our tens of thousands of pages. Their content management system is Magento. Has anyone ever tackled canonicalization for a large site that uses Magento? Any more efficient solutions to manual tagging is ideal. Thanks in advance for your input. -Bonnie
Intermediate & Advanced SEO | | accpar

0
How do you know if SEO factors are holding you back in rankings?

Hi there! I have been working on improving a site for almost a year now, and though we have made great strides in ranking for many relevant keywords our site is hovering at the bottom of page 2 and fluctuates from position 14 to 18 for almost a year. I am pretty prompt at addressing HTML improvements suggestions in WMT, but don't know where I should focus my limited time to get the most results. Competing websites have more backlinks than we do, but content is very thin and I don't think they update or add new content every week like we do. Please help! Am I missing something obvious?? Thanks in advance 🙂
Intermediate & Advanced SEO | | candiceone

0
SEO within the URL /

If I were optimizing for 'marketing success' and my URL structure was domain.com/marketing/success would that count? I'm not sure if the '/' affects the keyword term. My assumption is that it does, but I wasn't 100% sure. Thanks!
Intermediate & Advanced SEO | | KristinaWitmer

0
Are these Bad Internal Links/Anchor Text?

Hi my site www.over50choices.co.uk is 4 months old and I wondered whether my "Quick Links" section (right hand column) on 95% of my pages with the same/similar anchor text was not best practice ie should I vary the anchor text & the target locations more? ( they tend to point to my top 6 pages) They were set up originally to make the customer experience easy to find things but from what i have read Google doesnt like too many links looking the same ! I also have 3 Graphics (cross sales messages) just above the foot of most (not the home page) pages, linking to my 3 key value pages, all with similar Alt Text tags, again should i vary the alt text or is not a good idea to have this type of link on every page? What is best practice, as i am trying to balance the visual/customer experience whilst optimising for search? Thanks
Ash
Intermediate & Advanced SEO | | AshShep1

0
Anchor Tag around Table / Block

Our homepage (here) has four large promotional sections taking up most of the real estate. Each promo section has an image and styled text. We want each promo section to link to the appropriate page, so we created the promo sections as and wrapped each in an anchor. That works fine for users but I tried viewing our site in a text-only browser (Lynx) and couldn't follow those links! My fear is that GoogleBot can't follow them either and doesn't know what anchor text to pull. So, my question: What's the best way to make this entire block clickable, but still have it crawlable by robots? Or is our current implementation ok? For reference, here's a simplified version of the relevant code block: | | All Diamonds Extra 20% Off | [| | Jessica Simspon Extra 20% Off |](http://jessicasimpson.jewelry.com/shop/)
Intermediate & Advanced SEO | | Richline_Digital

0
Should I robots block this directory?

There's about 43k pages indexed in this directory, and while helpful to end users, I don't see it being a great source of unique content for search engines. Would you robots block or meta noindex nofollow these pages in the /blissindex/ directory? ie. http://www.careerbliss.com/blissindex/petsmart-index-980481/ http://www.careerbliss.com/blissindex/att-index-1043730/ http://www.careerbliss.com/blissindex/facebook-index-996632/
Intermediate & Advanced SEO | | CareerBliss

0