DEFCON 19 Talk Transcript

From Archiveteam
Jump to: navigation, search

Transcript of DEFCON 19 Talk from [[1]]



So again thank you all so much for coming to this, and for enjoying, I hope, DEFCON.

"Excellent, yeah," says one guy!

OK, so, the name of this talk is "Archive Team: A distributed preservation of service Attack." A hilarious title meant to bring you in, and it worked, apparently.

My name is Jason Scott, I am the mascot of Archive Team, which is a rogue band of archivists, preservationists and jerks dedicated to saving online, and in some cases offline, history.

And this project has been going on for a little while, and I thought, well, maybe it's time to kind of make people understand what we're up to here.

So, before I get started though, I want to dedicate this talk to Tim Recher, a very old friend of mine who unfortunately passed on this year. It had been one of his dreams to bring his family back to DEFCON, so they are here in the house tonight to enjoy DEFCON.

And I must say, you know, as time goes on, I'm currently forty years old, you know you have experiences of losing friends you didn't know you were going to keep around for such a short time, so it's always worthwhile, even though I'm in a talk, to skip a talk, if there's a friend who you haven't seen in awhile, spend maybe an extra four or five minutes with them to remember some things with them. It's just a benefit for that aspect of it. Huh.

Since we're all about saving websites let me in fact talk about soy sauce.

So, soy sauce... Soy sauce, as some of you might know, is basically fermented soybeans, wheat, it's a process in which these things are brewed just like beers, or other kinds of fermented dishes.

There's an experience to it, there's an idea behind it, it's a long process to learn, we've been doing it for hundreds of years, but different groups have different approaches, and it's extremely important that the essence of them are maintained. In fact this is part of the marketing of a lot of beers and crafts and everything else that, you know, it's important that all the components stay absolutely the same.

This is the Yamasa Soy Company, a company that's been around for two hundred and eight years, since the Edo period. They've had eight presidents, and they've been basically producing soy sauce in their local town for all this time.

And they are beloved enough that, for instance, here's a tourist who has just gone and taken a beautiful drawing of this soy sauce factory to kind of show, you know, they've been around forever and they do this kind of work.

So they were hit by the tsunami.

This is that same factory, and that's the creator the current ninth president of this soy sauce company. He was given it by his father after the flood, because his father felt he was too old to figure out what to do next - because what are you going to do about a company in which everything is absolutely obliterated?

Obliterated to the point, for instance, that this is the company safe from which he was able to extract their incorporation papers from two centuries ago so he could prove that the company was still around, and still existent. And again, I'm crediting Robert Gilhooly, this is the man who walked around with it.

Now bear in mind, when they got hit with the tsunami, one of the first things that they discover, or one of the first things that happened was that their head of sales ran to one of the dams, to save the dam, and was killed by the ensuing rush of water.

This man as well, this is the ninth president, was absolutely convinced his family have died and that he'd lost his children because their house was just a few blocks from this place. As it turned out his children had actually been led up a hill by some teachers and elderly residents who were nearby, most of which then proceeded to die.

So this was a family, this is a family businesses that's just ensconced in tragedy right now. And, so this is him kind of standing on the hill that he was able to run up to with all the employees who didn't die, overlooking their factory, losing this thing.

So, what, what I want to explain, though, is that this is a -- you know, a person might say "Well, who cares, it's a company, what does the company matter against human lives?"

Well, this is a company that was so entwined in the identity of this town that even with a 70% death rate, people were coming to this soy sauce company and giving them money, to say "When you make soy sauce again, I'll wait for my soy sauce delivery." Because I want to be able to, you know, keep this important thing around.

Now in the process of making soy sauce there's this whole process here, and one of the most important ones is the adding of the moromi, the fermentation where they add specific yeast, the specific fermenting agent that will be able to make more of the soy sauce, and it's got a specific flavor, and it's cooked in a certain way.

So one of the things that the president discovered was that -- and again, he's thirty-seven, right, and he's been saddled with this idea to rebuild the company. So he looks for the barrels - there's about thirty barrels that they keep this agent in.

And he comes and he finds most of them completely crushed. Gone, missing. He finds one or two that are intact, but to his horror, he tells people "Okay, we're good, we've got the moromi" but it turns out that too much sea water has gotten in, and it has killed it. So there's none.

So what ends up happening is, when everything looks bleak, they recall an experiment that had happened four months earlier, and what it was, there was a local marine biology laboratory that was doing experimentation and asked for some of the moromi. So what they have done was asked for a small segment, a barrel's worth or whatever, of moromi to test with. They had given it to them.

That lab had also been hit with the tsunami, and had its first floor destroyed. But among the first floor was a one kilogram bag of the moromi still in the plastic wrapping that they had never gotten around to. And from that small bit, this company is rebuilding itself into this very beloved brand once again.

This is the two new sales girls who were hired by this company, that's their new digs while they're working it out, and this is a piece of their old factory, and the girls saying, you know, "Thank you, thank you for your patience. We will have your soy sauce soon."

So why am I saying all this?

Well, what I'm trying to say is that, first of all, backups are important! Multiple backups apparently even better!

But even beyond that, this object, this yeast, was an emotional meaningful human item that had relevance to a culture and a world that was, you know, basically something that people considered part of their identity, really. Really, if you think about it.

And, even though it's just an object, it's got meaning that way. So, what I'm saying here is, that objects that maintain memory, objects that are part of us, have relevance to us even after their initial use may be initially gone.

In other words you look at these items and you say "I remember I was with this person" or "This proves that we were part of this" or, you know, "This is the proof that I invented this first" or, more accurately, "This is a friend who I have lost." "This is somebody who I can't speak to anymore, but I have their work."

And I think that that's something that can sometimes be lost when I start to tell you about some websites, because one of the things people say is "Well, who gives a crap?"

These are really old websites, and I say these websites are collections of memories that have been gathered up through people online. That is the driving heart and force of what I'm talking about here.

And there's a wide variety of old media I might mention, and old websites and things that are currently sustaining our memories magnetically, in forms that are kind of strange, and each year it becomes harder and harder to extract them. But beyond that, they contain things that their exterior may not really reveal.

So for instance, you might have writings that you might not remember doing, letters your family might have done, basically stored on very very old media.

Additionally, you get weird shit!

For instance, this is eBay: the home game. This speaks to a lot of things because it indicates that people thought that eBay...

Well, first of all you have this belief that someone thinks eBay is something you'd want to do at home with your family, uh... with "No money down!"

But it also indicates that eBay had some sort of cultural meaning to us in 2000 when this came out, that it was strong enough of a feeling to feel that this is an experience that you should share elsewhere. That idea of getting completely fucked on shipping costs. Very critical, right? All right.

And again I'm keeping a lot of things myself that are part of that, this for instance there are some very old issues of 2600, I have three complete runs of 2600 magazine. Actually, I've got a lot of stuff.

If you don't know me, that's fine, that's awesome, I don't have anything, I live a free life, how about that, but if you actually know me then you know about the shipping container, and you know about the various pieces of old media that I'm sent, I'm sent lots of floppy disks, old tapes, tape drives, you know, basically all sorts of collections of items that are things that I transfer out.

So I've been able to get my hands on some very old things, and I and I constantly make myself available for this because I believe that all of these old memories have meaning, and it's pretty easy with something like 2600. Very prominent, lots of copies, very easy. I'm to tell you that there's a hacker calendar now from 2600, in case you want to support them, but the 2600 magazine is just one of just many. But we also have these electronic artifacts. Right?

So, for some of you this has -- can I just get a clap if this has any emotional meaning to you whatsoever?

And I DO do error correction for the person going (MIMES CLAPPING), that's cool. I'll just quickly go over what we're looking at here.

These are two different items. The first one is a "Netscape Now" button. This is a period of time when browsers were just starting out, and you had a number of browsers, there were about twenty or thirty, but there were a few that were trying to build themselves up and one of the original investors in SGI went out and scooped up all the creators of the Mosaic browser, started a new company and named it Netscape.

And, as part of that, wanted to let you know that if you want to see a really good site, and you want to watch it properly, you should get Netscape right now. So there was a button that they produced, that was animated, that said "Go to get Netscape right now so you can see my website the way it was meant to be seen". And so, that's the Netscape Now button.

Underneath that is the "Under Construction" GIF. I'm not going to get into the "Gif" / "Jif" argument right now! So, the "Under Construction" GIF is basically a, uh, indicator that you're not finished with your website.

Now obviously we in the future with our incredible abilities, websites are actually never finished so it's redundant! We've factored that thing out on both sides of the equation. We're like, you know what "always building", because if you're not always building it's now currently a period of shame, right. Lack of dynamism is a shameful trade of your website, indication of your failure, and lack of interest, so you would not want to say "Oh, I'm under construction", of course you're under construction.

Alright? So, yeah, there's a very emotional reaction to that, but I find there's even a bigger one to this. Which is...

This is a collection that I have of all of the Netscape Now buttons! As you can see there's a whole variety of stories being told here, because some of them indicate what... Uh, some of them are obviously made by hand, some of them go for different versions, some of them re-jigger themselves, these are all actually MD5 different.

What I am discovering, for instance, here's a little story you might not know, is for instance some of them you look at and you say "Well, why is this different from the one next to it?" It is because they've removed the extra frames to save some space. They took out a K or two, and that way they got a little more space for their website. So the thing looks a little crappier, but "Thank god I've got more space!" We don't think about things that way. Also somebody there seems to be really against Netscape Now, so... screw you!

Similarly on the "under construction" thing, also a lot of emotional reaction. I got ten thousand of these things. And as you can see, there's all variety of things put under construction, and I think what I'm trying to say is... And again, if you go to textfiles.com/underconstruction it says "This page is under construction", and then puts all of them underneath. Which will crash some browsers!

And then it says "if there's a problem, mail me". That goes to one with mail me GIFs that DOES crash all browsers. So you can be a historian, and also be an asshole! It works out, actually.

So anyway, so what you have here is again like I said, a wide variety of interesting cultural artifacts, and I've found the people who go to this just automatically get a massive amount of reaction from it.

Well what we're experiencing right now is a bunch of websites that were started earlier, and "earlier" now could be anything from mid-1990s up through to even maybe a year ago or further, where they reach a point where somebody decides they're not going to stay up any more.

And it's usually done with, like a post-it note on the outside of a restaurant that's been shut down for health code violations. It is just simply something saying "By the way, we're gone."

Now, normally I would not care, right? I mean, if you've created, you know, hatsforcats.com, and suddenly no-one wants to buy your cat hats, and you say "Sorry, thank you for, you know, thank you for four months of wonderful business." And away you go, that's fine.

But what we have right now from the mid-1990s on to now is this whole period where we're taking user generated content and a large amount of marketing is being made to make it as easy and quick and frictionless to put as much of yourself online as quickly as possible into something... with a huge lack of any interest in telling you what that something is. It's just there. You get an IT department, and you don't even know how to reach it.

What ends up happening is you get this, right: AOL Hometown, which was a whole bunch of really interesting websites from the early 1990s, and in 2008 they said "You know what, we're out of here." And that was it. Hometown was gone. Same thing up there with Kickstart. You don't know what Kickstart is, I didn't know what Kickstart is, but I like the button. The indication there is like "See this light bulb? Going out."

And then you get these kind of surreal shutdowns, right, like Free Pro Hosting, which offers you more! And the next thing it says is "We're going to be discontinuing our free hosting service at the end of the year." And look at that smiling girl! "Guess what, we're closed!" "We're out of business!" So, hey, welcome to Free Pro Hosting where nothing is now free. That is a tough, tough sell. "What are you called?" "Free cars." "What do you sell?" "Cars." "Free?" "No." "No, Bob Free, pleased to meet you." Yeah.

So we started Archive Team. OK? ArchiveTeam.org. We are gonna rescue your shit. We are the A-Team. We are the team that will come in, and we will rescue things that need to be rescued. Help the helpless, go after the site, sight the sightless.

We're going to go after places that look like they're being shut down. And we download them, and then we figure out what to do next. We know, you know, so much in history, if you go ahead and look at a lot of things, how we have it with housing and things, that you know, basically, uh...when, when you evict somebody from a home, it is a huge-ass painful process that sucks. Right?

Yes, right there, you're looking outside, you can see in the window, you're the landlord, you can see them fucking up your apartment. You're like "I'm gonna get rid of them. It's going to take six weeks. But I'm going to get rid of them." And you have to apply in front of a judge, you have to show things, and you have to do all these things.

Well with web hosting, we don't have to do any of that. And some people think that's beautiful, and, yes, the wild west was fucking awesome until you died of dysentery.

And I'm saying that it's 2011, and this is DEFCON. This is one of these places which goes, like, "By the way, this idea is stupid, we don't do this anymore", well the idea of completely, uh... uncontrolled, non-transparent hosting of user content really needs to come to an end. But until then, we're duping stuff because the conversation otherwise ends.

Like if you go to AOL Hometown now, and go, like, "I need my old stuff" they go "That's a shame. That you still need it. Are you sure you need it? Would you like to buy a new account with more space?" Because right now it's OK.

So Archive Team set out on its mission, and we've started to download things. We've been having a great old time.

And then GeoCities went down. So, we were like, people came to us and they were like, "Hey, Archive Team. GeoCities? You gonna download it?"

How many people here know GeoCities? Ah, right! See, they don't want to make noise and call attention to themselves.

The thing about GeoCities, and I think GeoCities falls into this right now, right, GeoCities is the moromi. GeoCities is this place that started in 1994 as Beverly Hills Internet. Got turned into this very strange hosting company, gets bought by AOL -- not AOL -- by Yahoo for an enormous amount of cash. I mean an astounding amount of cash, billions of dollars, to become hosting.

Now at the time of its purchase by Yahoo, it is the second or third, depending on the month, most-browsed site on the internet. This is the most popular of popular sites. It is huge.

And one day, one day, they announced they were shutting it down -- oh, but I don't mean they really ANNOUNCED they were shutting it down. I meant that buried in one of the help files, which somebody brought to our attention, it said "I'm having trouble getting this done", and the answer was "Yes, because of the shutdown that functionality is currently not here." That was it! We're like "Wow, that is burying the fucking lead!"

And what they did was, they were shutting down "sometime in the summer" And then all of this site was going to go down. Bear in mind that when GeoCities finally went down it was the 218th most browsed site on the net. It'd only gone down a little bit. Yahoo made no attempt to get rid of it, you know, they didn't try to sell it off or anything like that, they just simply said "OK, let's turn this off. Hooray for us."

Now granted, you'll go look at one of the sites that was on there... And you'll be like "Well, of course. Of course." I mean look at this, then, this is the Rogue Cowboy. "Hey y'all, military couple, been here for a while." I'm reading this for you because you cannot possibly see it over the bucking bronco background.

Also, I want to point out that there's a little gold item there and it says "HTML Writers Guild." Another thing that's kind of gone by the wayside is HTML guilds. Now it's just stock options.

And the thing is, you know, you look at a site like that and you're like "Well this is... these guys are awful!" and I want to point out something I've really come to understand, which is how do we do this - how do we get rid of all of something?

In fact, how do we destroy cultures, how do we destroy lives, how do we do this? And the answer is: disenfranchise, demean, delete.

Disenfranchise: remove their ability to have any control over something. Like I said, with FaceBook, good luck calling them up to get something fixed. Good luck calling them up because something's not working like you expect. Go ahead and tell me that doesn't cost any money, and I get what I pay for, fine, but I'm telling you that's what the case is.

Second, demean: tell people that this thing is useless. Look at this thing, it's ugly, by our design standards this thing fails our test. We the board of Vogue, we've decided that this thing is not to our liking. And then, delete. Then say "Who gives a shit about these people? These are nothing. Whatever."

But we have to realize that for these people, this presentation, this website may be the widest audience that this genetic line has ever reached. And you can't turn away from that kind of power, even if that was never your hope. Printing a color photocopy was $1.50 a page at this time. To be able to do full color, occasionally with musical background, websites, that would have all the things you wanted to say? And it's interesting what people pull, for instance.

Welcome to space! Now it's interesting that the projector really gets rid of the beauty of the star field. I feel like it's not really there. But bear in mind there's a beautiful star field there. It's not animated, but it's something that's there and this person obviously likes space. And there were areas in GeoCities for you to store, based on Hollywood space, gay queer, western, and so on. And you were able to declare what your kind of interests were and go down there.

So, this particular person was in Area 51, the space thing. And there's a part in there called personal experiences which I just love, because you read it and their personal experiences are like "Was watching television. Felt outside of myself for twelve minutes. Continued watching television." Yeah. OK, great, hilarious, but also this person wanted to kind of express this, and obviously this leads to interesting conspiracy theorists and the paranormal network, and all of that.

Let's go with this one. Welcome. Patrick Joel Mielke, born on April 16th, 1981, entered heaven April 17th, 1983. Page lovingly dedicated to Patrick Joel, child of God, uh... loaned to us for a very short time. It's a celebration of his life and the love and joy he so enriched his lives with."

Now this is a woman, and this is I think what's sometimes not noticed here, the child died in 1983. This website was created in 1996. This is a woman who has enough pain at that time that when she sees GeoCities, where other people say "I'm gonna talk about watching TV" and "I'm gonna talk about my bucking bronco background and join the HTML Writers Guild", here's somebody who's saying "No, the world needs to know about my baby. I want to let everyone know how much I loved him." And she has pages after pages in the ten megabyte space, about how much her baby meant to her.

So here's a case -- and by the way of course there was a web ring, you know what a web ring was, of, uh, what was it... It's an Angels web ring, so it's a bunch of parents who lost children who are under two, to talk about, you know, they touched an angel for short period of time. This is real stuff. This is as real to save as anything else, I think. So, dig on it.

(AUDIENCE LAUGHTER AT NEW IMAGE)

Gets better the longer you look at it.

So, Jason, a question that now some of you who've never heard of before now will ask, "What the fuck is up with that, Jason?" It's a general question I get about everything.

Alright, here's the deal. He's an Under Construction GIF. And he got wrapped up in the trawl, basically I went through a bunch of GeoCities stuff and found a bunch of Under Construction GIFs, and he was one of them.

And I was like "What the hell is that? What's the fucking story about Bulgy McFish-Hat guy?" so I go look it up, and it's this guy in the Hollywood Hills section, and he is gay, and he has a page that he talks about his dream guy. Uh, it's from 1998. And he talks about what he wants in a man, what he will do with the man, where they'll go, the places he'll do, the dreams he'll live, it's from basically, like I said, 1998, and at the bottom it says "This is always under construction." And there is this guy at the bottom.

In 2005, it's updated. And it says "No need to keep looking. I've found him." And it's just a story that turned out, even with the bulge, to be pretty heartwarming. All of this buried in the little tiny graphics interchange format. Which I believe just got out of copyright. Oh, I'm sorry, patent.

OK. So when they closed it, right, Yahoo just decide to do this twerpy frigging goddamn thing. This to me is the embodiment of the problem. "Why did GeoCities close?" which by the way should really be said in like kind of a scream with a rending of documents, because that's usually when you "Why did GeoCities close???!!!?!?"

"We have decided to focus on helping our customers explore and build relationships online in other ways." That's like shooting somebody and saying "I have plans for your car." All right. It's this sort of corporate douchebaggery that ensures that I will never work within a corporate environment again.

(IMAGE ON SCREEN - 'FUCK YOU')

I don't know, that's my visceral reaction, what do you think? So that's what Archive Team said, so we said "You know what, Let's download it."

So... Downloading was very interesting. Downloading GeoCities was somewhat complicated. It took us about a hundred people to download over the course of about six months. We had no idea when the shutdown date was, right, so we just went at it.

Now it turns out that GeoCities had a very interesting thing. You got a gigabyte of bandwidth a month. But! Only about twelve megabytes of it could come out every hour. Bait and switch. So we would try to do it, it would go "Sorry. Error. 999 error. Content limit has been reached." It didn't take long, by putting our heads together, by having all these assembled people on our IRC channel to have someone go, "Do you think they're locking out Google?" So we go look, and, nope, they're not locking out Google. So we changed all of our user-agents to "Not The GoogleBot". Free!

At that point, we aimed a couple people at them, we had a hundred virtual machines that downloaded basically all of the -- GeoCities has an old neighborhood and the old neighborhood system which basically would be GeoCities slash WestHollywood slash... you know, Hills slash 2252. These are all pre-1999.

When Yahoo got their nutsack on it, they just reapplied it across to the Yahoo section. So basically what they did was, you could be "GeoCities.com/~toolbag" and be whatever. So it was going to be harder to find them. But, man, we sent people after them, and we did, and we downloaded as much as we could, which turned out to be a little bit over a terabyte, of GeoCities.

So, then what do you do? Well, first, bear in mind that this is GeoCities in 1999. This was a 9 terabyte array of theirs. Just to give you an idea of just how pathetic it is now, when people are like "Oh God, what are you going to do, where are you going to keep all that?" I can make a stack of nine terabytes right now that are barely functional. We have to keep in mind that this is a whole cage at Exodus dedicated to GeoCities.

So we ended up with it, and we're sitting on it and then GeoCities went down and it was the usual like "Who cares?" and I put up those animated GIFs by basically going through this terabyte of data, and coming up with a collection of interesting GIFs.

But then a year went by, and I thought, you know, we've got to get attention. We've got to remind people that GeoCities went down for no fucking reason.

So we did what anyone would do, we torrented it. So we put ourselves up on The Pirate Bay, we have a 641GB - because it compressed well - torrent, with 7,854 files that were basically 7zs, and we put that sucker up. And we torrented it. We were until recently, I think it's changed, but we were until recently the second largest torrent that ever appeared. The number one was high-definition versions of all of the World Cup games. So, nice counterpoint, huh? World Cup games, GeoCities.

Because we knew, right, that by warezing GeoCities, this would bring this massive amount of embarrassment back, and it did. We got all these great interviews and I'd put up this thing saying "Yahoo found a way--" I was quoted by Time for this -- "Yahoo found the way to destroy the most amount of history in the shortest amount of time." Alright, excellent.

Then Yahoo Video announced it was going down. We got that! Well we were helped, of course because, Yahoo Video sucks. But it was 10 terabytes. We just downloaded all of the video. Everything, all of it. Luckily they used numeric IDs, very easy to go through. We ended up downloading it, we're in the process of getting it all back up again somewhere.

And yes a lot of it is spam, some of it is really terrible. It seems to be really popular with people in countries that are not America, who were using it as a way that have stuff that needed bandwidth that they didn't have to pay for, where the bandwidth was expensive. But we've got this thing, and we were able to, you know, basically do this through the gift of volunteers who all work together very hard.

These are all the things that Yahoo! has shut down in the last four years. Just so you understand. Yahoo briefcase, where you were able to store 10 megs, whenever you wanted, and get it from anywhere via FTP. They shut down. Why? No spare USB drive? Content Mash, some of these you won't know.

Yahoo Pets was funded by Purina for a five year contract and on the day that the contract ran out they shut it down and redirected it to Yahoo Women. I don't know why, but they did.

But it was a case of there was this secret contract, and when I say they shut down, I mean with no warning. One day it was there, one day it was gone. It had pet pictures, it had forums in it, everything, gone. Totally gone. So in other words I'm saying Yahoo blows, OK? It is a fucking clown car. I wouldn't trust them with like a backup of my nutsack, because these guys... This is a case where a company went speculatively into user generated content and when they decided it wasn't worth it any more, they got out of it.

Like getting into a library and deciding "Oh, library business isn't working for us" and burning it to the ground. OK? And I've got people, I've got people who come to me and say "Yahoo was great to work at" - yeah, everywhere is great to work at if you're working for an arsonist company it's awesome! "We were trying to change the world", well, you sorta did. Awesome. Now you're using Bing as your search engine and you suck.

Friendster went down this year. Friendster we only got 12 million of the 112 million accounts because it turns out that digital cameras really came into prominence in 2005, but we basically got most of the larger earlier sites from it, a lot of people, you know it's funny 'cause if you talk to people about Friendster, they know Friendster, they're like "Yeah I remember that, it was like a social network, I think I was on it" and we feel that like collecting this material -- and believe me it was a javascript nightmare, we had to write customized scripts to go through the javascript, negotiate it, we all had to create accounts on Friendster which was still allowed, up to its death.

And all of us were like "Hobbies? Downloading Friendster." With some really funny giving the finger profile. "I'm downloading Friendster, that's why I'm here, what are you up to? Oh, you like cats, that's great."

Not everyone likes what we're doing. This is Lulu Poetry - poetry.com. This was exciting. They gave everyone two weeks to get off. Fourteen million poems. And as you can see their suggestion was "Well, be sure to copy and paste your poems before we go down." So you can always remember them.

"We're unable to save any customer information or poetry." Actually you don't hear that line a whole lot, do you? "I'm sorry, your poetry is unavailable." So we were like, okay, well we can do this. So we did. Within a short time we start getting banned. Locked out. So we switch IPs. They block out more. We have someone switch IPs, and we watch as an entire range is blocked out. We realized that there is a person or persons there, stopping us.

So we switch to S3. We switched to Amazon instances and we start doing it that way. And they run out of ways to block us out. They threaten one of my members, who is in Australia, and fifteen, with legal whatever, and I'm trying to explain to him that a cease and desist is not a lawsuit. He's fifteen years old, he's in Australia, he's probably not going to be flown to America for downloading poetry without a license. Interpol is not going to get in on this shit.

But, you know, whatever, make the kid nervous, but basically they were like, "No, no, you don't understand, we're actually going to be bought out, so this will survive, so we know what you're doing, it's OK" and I was like, "It's great you said that - fuck you!" and just kept going at it. So it turned out, as far as we can determine, they were on one shared server in some space, that's what their problem was, we were essentially, like I said, doing a distributed preservation of service attack. We took these guys out. We were taking them out, making a duplicate of them.

So we got lots of millions and millions of poems, which we're holding on to, because they did go down, by the way at 12.01 of the day they announced they were going down. I could tell that there was like one guy, we were watching cases on a couple of nights where we would actually watch the blocking slow down, because we figured out people got tired and went home. So by three in the morning our Australia guys are like "Oh wow, free and clear!"

And so, let me tell you nothing is greater than when you give somebody a goal that has blanketed on it some sort of moral righteousness. It does lead to some awesome shit, and fire. Anyway, so basically they, you know, they might come back, they might not. So sometimes you've got to be a little rough. We try not to be.

So Google Video announced it was going down. Now we were scared. Because Google Videos is huge. So we did it anyway. We started downloading, we were at somewhere like the ninth or tenth terabyte.

Downloading GeoCities, we had a distributed system that would download from these things, and we were like "Yup, gonna save it, what an embarrassment. Google Video, what an embarrassment, what YouTube, why don't you just transfer stuff to YouTube. What the hell's wrong with you? Why are we doing this? What's wrong with you, Yahoo?" -- Uh, sorry, Yahoo 2: Google! -- And basically, a week or two in, they give us an update. And the update says "OK, we're not going down. We're going to add a 'Migrate to YouTube' button, we're going to do what we need to do."

So basically, what they-- what I found out later is that internally in Google they went "Look what Archive Team is saying, this is really embarrassing. We have to stop this." And so they went "Yeah, got it", so they went, basically, "Okay, we give up." And so we won. One of the few times we won. It's nice to win.

So what's this? Guy with nine track tapes. Guy's got Usenet. That is Usenet from 1981 to 1991, all right. So here's what happened. So basically, this stuff is what became Dejan, went into the Google purchase of Deja News, which then became Google Groups, and Google then proceed to ruin it. Okay, if you know anything about Usenet, they ruined it. Unequivocally.

And we made a very important discovery. A lot of people are starting to think of Google as some sort of archive or library. That they're storing all this data, they're running ads, they're really storing all this data. But Google is a library or an archive in the same way that a supermarket is a food museum. These guys are basically gonna do whatever they gotta do.

So we took it, we took back, we found the original archives that Google had taken, we put them up on Archive.org. The UTZOO tapes. And people are doing with them - an Archive Team member did this. Not really associated with us, but did it and is part of us. Olduse.net. He is doing a real time posting of Usenet with a thirty-year lag. So you can go on, connect with the newsreader, and go experience what it was like.

This particular one says "Perhaps you're not aware of it, but a new Star Trek movie is in the making this summer. While that is all well and good, there is a problem with it. It seems that Leonard Nimoy will no longer be available for the role of Spock and thus they're killing him off. Loyal Trekkies here have taken great offense to this, as well they should! There are better ways to remove the necessity of having the character present." And anyway, so that's good. And then we never saw Leonard Nimoy again. So you can connect to this thing and be able to use it right now. That's living history.

Telehack.com, I don't have time to go into it. Telehack.com. OK, it looks like a command line, it is an entire world. Years of command-line history at that site. Spend a little time on it, get an account, it's unbelievable. All resuscitated from old archives.

And I don't mind being made fun of, all of this whole thing. This is the stuff for my teammates who make fun of me, we believe in lots of great, crazy humor.

So where else do we go from here? Well, we've got a group called wikiteam. They've written a thing from the outside, downloads tons and tons of wikis and we'll grab it and then we've been putting them up. So we've got wikiteam, if your wiki is like dying, we're gonna grab it, and we make the tools available so you can grab any wiki from anywhere else.

URL Team, because URL shortening was a fucking awful idea. URL shortening is like DNS retarded. You're gonna let some third party generally decide what everything you do directs to. You are stupid. I understand use of URL shorteners on a per-site basis, making a Flickr, that's fl.kr, whatever, that makes sense, but this is awful because if these things go down now anyone looking at the history, it's like people are talking cryptographic code. "Here's this awesome site... thing you can't figure out."

So we have been taking them on, this group over here has been basically taking all these old URL shorteners and turning them into archives that we then torrent. So, I also want to point out, this is Chronomex, Jeroenz0r, Soultcer, Swebb, Underscor, not me. This is not just a Jason Scott project. These guys, I just, I'm planning a fire, but these guys are going somewhere with it and they don't always need me, that's very important.

What else? What's left to save? I don't know if many of you know Len Sassaman. He was a wonderful cryptographer, wonderful human being who took his own life just a very short period of time ago. His wake was just last week, actually, and he was a big presenter at DEFCON. If you start going through the archives, you will find him there, he's a brilliant person who left a lot of friends and a lot of memories. He's a wonderful guy, and his widow said to me "Can you archive him?"

So I started a project called "Away From Keyboard". This is on Archive.org. And what we're doing is we are collecting artifacts from various people who have passed on to turn them into collections of files that at least we can get some piece of these people who are gone, and we can remember them and be able to build from them. So it doesn't always have to be about websites, it doesn't always have to be discs that I'm trying to save here. It's everything, and I think that's just critical.

What did we learn here?

We learned I'm really loud. We learned a lot of profanity, but hopefully you'll look at a web site that's going down, into something that's dying, and you'll say to yourself "Okay, that's not just a piece of crap, that's something that's meaningful to people. That's something that matters to people."

And I hope that that, you know, piece, sticks with you, if nothing else I did. So my final question for you is, "OK, is anyone here from Archive Team?" No, fuck you. You are all in Archive Team. I officially deputize you. You are allowed to be in Archive Team. Go where you need to, keep backups, store them somewhere, throw them over to someplace you don't remember, give me copies later, or give my successor copies later, about these things, because it turns out what you walk in is history, because the hardest part of history is to be there when it happens. That's the hardest part of any historian's job. And by being what you are right now doing, is you are in companies, you are with people, you are visiting things, and you are with history. So please, save it for the future because the future will wonder why the fuck we all thought Under Construction GIFs were so important.

Sometimes I put that as my user profile when I'm downloading a site. It hits the message, doesn't it? I'm just saying, if you take your site down, I'll see you there.

Jason Scott, Archive Team. Thank you for coming. And please, one more bit. Dedicated to Tim Recher who unfortunately died before really giving a big presentation here, so I'm just proud to say, my secret co-presenter Tim Recher. Thank you so much.


v · t · e         Archive Team
Current events

Alive... OR ARE THEY · Deathwatch · Projects

Archiveteam.jpg
Archiving projects

APKMirror · Archive.is · BetaArchive · Government Backup (#datarefuge · ftp-gov· Gmane · Internet Archive · It Died · Megalodon.jp · OldApps.com · OldVersion.com · OSBetaArchive · TEXTFILES.COM · The Dead, the Dying & The Damned · The Mail Archive · UK Web Archive · WebCite · Vaporwave.me

Blogging

Blog.pl · Blogger · Blogster · Blogter.hu · Freeblog.hu · Fuelmyblog · Jux · LiveJournal · My Opera · Nolblog.hu · Open Diary · ownlog.com · Posterous · Powerblogs · Proust · Roon · Splinder · Tumblr · Vox · Weblog.nl · Windows Live Spaces · Wordpress.com · Xanga · Yahoo! Blog · Zapd

Cloud hosting/file sharing

aDrive · AnyHub · Box · Dropbox · Docstoc · Google Drive · Google Groups Files · iCloud · Fileplanet · LayerVault · MediaCrush · MediaFire · Mega · MegaUpload · MobileMe · OneDrive · Pomf.se · RapidShare · Ubuntu One · Yahoo! Briefcase

Corporations

Apple · IBM · Google · Loblaw · Lycos Europe · Microsoft · Yahoo!

Events

Arab Spring · Great Ape-Snake War · Spanish Revolution

Font Repos

DaFont · Google Web Fonts · GNU FreeFont · Fontspace

Forums/Message boards

4chan · Captain Luffy Forums · College Confidential · DSLReports · ESPN Forums · forums.starwars.com · HeavenGames · Invisionfree · NeoGAF · The Classic Horror Film Board · Yahoo! Messages · Yahoo! Neighbors · Yuku.com

Gaming

Atomicgamer · Bazaar.tf · City of Heroes · Club Nintendo · Counter-Strike: Global Offensive · CS:GO Lounge · Desura · Dota 2 · Dota 2 Lounge · Emulation Zone · ESEA · GameBanana · GameMaker Sandbox · GameTrailers · Halo · HLTV.org · Infinite Crisis · joinDOTA · League of Legends · Liquipedia · Minecraft.net · Player.me · Playfire · Raptr · Steam · SteamDB · Team Fortress 2 · TF2 Outpost · Warhammer · Xfire

Image hosting

500px · AOL Pictures · Blipfoto · Blingee · Canv.as · Camera+ · Cameroid · DailyBooth · Degree Confluence Project · deviantART · Demotivalo.net · Flickr · Fotoalbum.hu · Fotolog.com · Fotopedia · Frontback · Geograph Britain and Ireland · GTF Képhost · ImageShack · Imgh.us · Imgur · Inkblazers · Instagram · Kepfeltoltes.hu · Kephost.com · Kephost.hu · Kepkezelo.com · Keptarad.hu · Madden GIFERATOR · MLKSHK · Microsoft Clip Art · Microsoft Photosynth · Nokia Memories · noob.hu · Odysee · Panoramio · Photobucket · Picasa · Picplz · Pixiv · Portalgraphics.net · PSharing · Ptch · puu.sh · Rawporter · Relay.im · ScreenshotsDatabase.com · Snapjoy · Streetfiles · Tabblo · Tinypic · Trovebox · TwitPic · Wallbase · Wallhaven · Webshots · Wikimedia Commons

Knowledge/Wikis

arXiv · Citizendium · Clipboard.com · Deletionpedia · EditThis · Encyclopedia Dramatica · Etherpad · Everything2 · infoAnarchy · GeoNames · GNUPedia · Google Books (Google Books Ngram· Horror Movie Database · Insurgency Wiki · Knol · Lost Media Wiki · Neoseeker.com · Notepad.cc · Nupedia · OpenCourseWare · OpenStreetMap · Orain · Pastebin · Patch.com · Project Gutenberg · Puella Magi · Referata · Resedagboken · SongMeanings · ShoutWiki · The Internet Movie Database · TropicalWikis · Uncyclopedia · Urban Dictionary · Urban Exploration Resource · Webmonkey · Wikia · Wikidot · WikiHow · Wikkii · WikiLeaks · Wikipedia (Simple English Wikipedia· Wikispaces · Wikispot · Wik.is · Wiki-Site · WikiTravel · Word Count Journal

Magazines/Blogs/News

Cyberpunkreview.com · Game Developer Magazine · Gigaom · Hardware Canucks · Helium · JPG Magazine · Polygamia.pl · San Fransisco Bay Guardian · Scoop · Regretsy · Yahoo! Voices

Microblogging

Heello · Identi.ca · Jaiku · Mommo.hu · Plurk · Sina Weibo · Twitter · TwitLonger

Music/Audio

AOL Music · Audimated.com · Cinch · digCCmixter · Dogmazic.net · Earbits · exfm · Free Music Archive · Gogoyoko · Indaba Music · Instacast · Jamendo · Last.fm · Music Unlimited · MOG · PureVolume · Reverbnation · ShareTheMusic · SoundCloud · Soundpedia · This Is My Jam · TuneWiki · Twaud.io · WinAmp

People

Aaron Swartz · Michael S. Hart · Steve Jobs · Mark Pilgrim · Dennis Ritchie · Len Sassaman Project

Protocols/Infrastructure

FTP · Gopher · IRC · Usenet · World Wide Web
BitTorrent DHT

Q&A

Askville · Answerbag · Answers.com · Ask.com · Askalo · Baidu Knows · Blurtit · ChaCha · Experts Exchange · Formspring · GirlsAskGuys · Google Answers · Google Baraza · JustAnswer · MetaFilter · Quora · Retrospring · StackExchange · The AnswerBank · The Internet Oracle · Uclue · WikiAnswers · Yahoo! Answers

Recipes/Food

Allrecipes · Epicurious · Food.com · Foodily · Food Network · Punchfork · ZipList

Social bookmarking

Addinto · Backflip · Balatarin · BibSonomy · Bkmrx · Blinklist · BlogMarks · BookmarkSync · CiteULike · Connotea · Delicious · Designer News · Digg · Diigo · Dir.eccion.es · Evernote · Excite Bookmark · Faves · Favilous · folkd · Freelish · Getboo · GiveALink.org · Gnolia · Google Bookmarks · Hacker News · HeyStaks · IndianPad · Kippt · Knowledge Plaza · Licorize · Linkwad · Menéame · Microsoft Developer Network · myVIP · Mister Wong · My Web · Mylink Vault · Newsvine · Oneview · Pearltrees · Pinboard · Pocket · Propeller.com · Reddit · sabros.us · Scloog · Scuttle · Simpy · SiteBar · Slashdot · Squidoo · StumbleUpon · Twine · Vizited · Yummymarks · Xmarks · Yahoo! Buzz · Zootool · Zotero

Social networks

Bebo · BlackPlanet · Classmates.com · Cyworld · Dogster · Dopplr · douban · Ello · Facebook · Flixster · FriendFeed · Friendster · Friends Reunited · Gaia Online · Google+ · Habbo · hi5 · Hyves · iWiW · LinkedIn · Miiverse · mixi · MyHeritage · MyLife · Myspace · myVIP · Netlog · Odnoklassniki · Orkut · Plaxo · Qzone · Renren · Skyrock · Sonico.com · Storylane · Tagged · tvtag · Upcoming · Viadeo · Vine · Vkontakte · WeeWorld · Weibo · Wretch · Yahoo! Groups · Yahoo! Stars India · Yahoo! Upcoming · more sites...

Shopping/Retail

Alibaba · AliExpress · Amazon · Apple Store · Barnes & Noble · DirectCanada · eBay · Kmart · NCIX · Printfection · RadioShack · Sears · Sears Canada · Target · The Book Depository · ThinkGeek · Toys "R" Us · Walmart

Software/code hosting

Android Development · Alioth · Assembla · BerliOS · Betavine · Bitbucket · BountySource · Codecademy · CodePlex · Freepository · Free Software Foundation · GNU Savannah · GitHost  · GitHub · GitHub Downloads · Gitorious · Gna! · Google Code · ibiblio · java.net · JavaForge · KnowledgeForge · Launchpad · LuaForge · Maemo · mozdev · OSOR.eu · OW2 Consortium · Openmoko · OpenSolaris · Ourproject.org · Ovi Store · Project Kenai · RubyForge · SEUL.org · SourceForge · Stypi · TestFlight · tigris.org · Transifex · TuxFamily · Yahoo! Downloads

Television/Radio

ABC · Austin City Limits · BBC · CBC · CBS · Computer Chronicles · CTV · Fox · G4 · Global TV · Jeopardy! · NBC · NHK · PBS · Penn & Teller: Bullshit! · The Howard Stern Show · TV News Archive (Understanding 9/11)

Torrenting/Piracy

ExtraTorrent · EZTV · isoHunt · KickassTorrents · The Pirate Bay · Torrentz · Library Genesis

Video hosting

Academic Earth · Bambuser · Blip.tv · Epic · Google Video · Justin.tv · Niconico · Nokia Trailers · Oddshot.tv · Plays.tv · Qwiki · Skillfeed · Stickam · TED Talks · Ticker.tv · Twitch.tv · Ustream · Videoplayer.hu · Viddler · Viddy · Vidme · Vimeo · Vine · Vstreamers · Yahoo! Video · YouTube · Famous Internet videos (Me at the zoo)

Web hosting

Angelfire · Brace.io · BT Internet · CableAmerica Personal Web Space · Claranet Netherlands Personal Web Pages · Comcast Personal Web Pages · Extra.hu · FortuneCity · Free ProHosting · GeoCities (patch· Google Business Sitebuilder · Google Sites · Internet Centrum · MBinternet · MSN TV · Nifty · Nwnyet · Parodius Networking · Prodigy.net · Saunalahti Iso G · Swipnet · Telenor · Tripod · University of Michigan personal webpages · Verizon Mysite · Verizon Personal Web Space · Webzdarma · Virgin Media

Web applications

Mailman · MediaWiki · phpBB · Simple Machines Forum · vBulletin

Information

A Million Ways to Die on the Web · Backup Tips · Cheap storage · Collecting items randomly · Data compression algorithms and tools · Dev · Discovery Data · DOS Floppies · Fortress of Solitude · Keywords · Naughty List · Nightmare Projects · Rescuing floppy disks · Rescuing optical media · Site exploration · The WARC Ecosystem · Working with ARCHIVE.ORG

Projects

ArchiveCorps · Audit2014 · Emularity · Faceoff · FlickrFckr · Froogle · INTERNETARCHIVE.BAK (Internet Archive Census· IRC Quotes · JSMESS · JSVLC · Just Solve the Problem · NewsGrabber · Project Newsletter · Valhalla · Web Roasting (ISP Hosting · University Web Hosting· Woohoo

Tools

ArchiveBot · ArchiveTeam Warrior (Tracker· Google Takeout · HTTrack · Video downloaders · Wget (Lua · WARC)

Teams

Bibliotheca Anonoma · LibreTeam · URLTeam · Yahoo Video Warroom · WikiTeam

Other

800notes · AOL · Akoha · Ancestry.com · April Fools' Day · Amplicate · AutoAdmit · Bre.ad · Circavie · Cobook · Co.mments · Countdown · Distill · Dmoz · Easel · Eircode · Electronic Frontier Foundation · FanFiction.Net · Feedly · Ficlets · Forrst · FunnyExam.com · FurAffinity · Google Helpouts · Google Moderator · Google Reader · ICQmail · IFTTT · Jajah · JuniorNet · Lulu Poetry · Mobile Phone Applications · Mochi Media · Mozilla Firefox · MyBlogLog · NBII · Neopets · Quantcast · Quizilla · Salon Table Talk · Shutdownify · Slidecast · SOPA blackout pages · starwars.yahoo.com · TechNet · Toshiba Support · USA-Gov · Volán · Widgetbox · Windows Technical Preview · Wunderlist · YTMND · Zoocasa

About Archive Team

Introduction · Philosophy · Who We Are · Our stance on robots.txt · Why Back Up? · Software · Formats · Storage Media · Recommended Reading · Films and documentaries about archiving · Talks · In The Media · FAQ