YouTube

From Archiveteam
Jump to: navigation, search
YouTube
YouTube logo
YouTube2018.png
URL https://www.youtube.com/[IAWcite.isMemWeb]
Project status Online! but possibly Endangered, see Vital signs
Archiving status Not saved yet
Project source Unknown
Project tracker Unknown
IRC channel #youtubearchive
Project lead Unknown
YouTube Annotations
YouTube logo
Youtube-annotations-example-fukkireta.jpg
URL https://www.youtube.com/[IAWcite.isMemWeb]
Project status Offline
Archiving status Partially saved
Project source Unknown
Project tracker Unknown
IRC channel #archiveteam
Project lead /u/omarroth
YouTube homepage screenshot as of 2011-04-22.

YouTube is a video sharing website currently owned by Google. YouTube is currently the most popular video hosting website on Earth.

Archiving tools

Several free FLV downloaders and video-to-URL converters exist on the web. AT rescue projects usually use youtube-dl.
YouTube annotations (speech bubbles and notes) are available as XML

http://www.youtube.com/api/reviews/y/read2?feat=TCS&video_id=

To transform this XML to SRT, use ann2srt

(Automatic) tubeup.py - Youtube Video IA Archiver

Note: When uploading to the Internet Archive, please avoid exposing the site to legal risk by adhering to their terms of service for blatantly copyrighted content. Unfortunately, they are subject to similar threats of DMCA takedowns as YouTube, so do use discretion.
Note: Be very careful dumping channels over 100 videos with this script. Let an admin know what you're doing, dump 50 videos, and have a collection created. Work is being started on adding a flag to specify a collection name instead of "Community Video" which is what it defaults to. Always try to create an item. For the time being the script will have to be hand edited to specify a different collection.

tubeup.py is an automated archival script that uses youtube-dl to download a Youtube video (or any other provider supported by youtube-dl), and then uploads it with all metadata to the Internet Archive.

This way, all metadata from the video, such as title, tags, categories, and description, are preserved in the corresponding Internet Archive item, without having to manually enter it.

It also creates a standardized Internet Archive item name format that makes it easy to find the video using the Youtube ID, and reduces duplication: https://archive.org/details/youtube-v9sGhNoSG3o

Youtube-dl also works with many other video sites.

(Manual) Recommended way to archive YouTube videos

First, download the video/playlist/channel/user using youtube-dl:

youtube-dl --continue --retries 4 --write-info-json --write-description --write-thumbnail --write-annotations --all-subs --ignore-errors -f bestvideo+bestaudio URL

This can be simplified by running the script by emijrp and others, which also handles upload.

You need a recent (2014) ffmpeg or avconv for the bestvideo+bestaudio muxing to work. On Windows, you also need to run youtube-dl with Python 3.3/3.4 instead of Python 2.7, otherwise non-ASCII filenames will fail to mux.

Also, make sure you're using the most recent version of youtube-dl. Previous versions didn't work if the highest quality video+audio was webm+m4a. New versions should automagically merge incompatible formats into a .mkv file.[1]

Then, upload it to https://archive.org/upload/ Make sure to upload not only the video itself (.mp4 and/or .mkv files), but also the metadata files created along with it (.info.json, .jpg, .annotations.xml and .description).

kyan likes this method:

Youtube sucker (look out it leaves some incompletes in the directory afterward. Can clean up w/ rm -v ./*.mp4 ./*.webm then ls | grep \.part$ and get the video IDs out of that and redownload them and repeat etc etc). Can upload the WARCs only e.g. using ia (Python Internet Archive client) or warcdealer (automated uploader I hacked together) — or if you want, can upload the other stuff too, but that's kind of wasteful of storage space. In my opinion, getting stuff without a WARC is a great crime, given the ready availability of tools to create WARCs. Note that this method also works for other Web sites supported by youtube-dl too, although it maybe would need different cleanup commands afterward. Depends on youtube-dl and warcprox running on localhost:8000.

youtube-dl --continue --retries 100 --write-info-json --write-description --write-thumbnail --proxy="localhost:8000" --write-annotations --all-subs --no-check-certificate --ignore-errors -k -f bestvideo+bestaudio/best (stick the video/channel/playlist/whatever URL in here)

Annotations removal

Annotations[2] were notes that could be added to videos after the upload, they provided plenty customization options such as color, size and internal & external links. People used them to correct mistakes in videos, created mini-games within YouTube and abused them too: "Please like and subscribe", "Watch in HD!!!"

On May 2nd, 2017 YouTube disabled editing and creating annotations for videos. On January 16th, 2019 they were removed completely from the site and API responses (~15:00 UTC, same time when they disabled editing).

An archiving project "YouTube Annotation Archive" organized by u/omarroth that achieved to discover ~1.4 billion videos and download the annotations, if they had any. The project did not completely finish due to the deadline. The annotation archive will be released at a later point (Update).

16GB of just video IDs that were encompassed by the project can be downloaded here.

Site reconnaissance

Little is known about its database, but according to data from 2006, it was 45TB and doubling every 4 months. At this rate it would be 660 Petabytes (Oct 2014) by now.

According to Leo Leung's calculations based on available information, an often updated Google spreadsheet estimates that in early 2015 YouTube's content reached 500 petabytes in size.

FYI, all of Google Video was about 45TB, and the Archive Team's previously biggest project, MobileMe was 200TB. The Internet Archive's total capacity is 50PB as of August 2014. So let's hope YouTube stays healthy, because the Archive Team may have finally met its match.

Vital signs

Will be living off Google for a long time if nothing changes.

Advertising policies

Around early 2017, numerous content creators have expressed concerns about recent changes with YouTube's advertising policies, and many have also noticed sharp drops in ad revenue as a result, with some creators like Casey Neistat and h3h3Productions expressing existential fears. While not necessarily a cause for imminent alarm, the situation should be watched closely in the event that a positive feedback loop was to begin with a creator exodus.

Annotations

On November 27th, 2018, YouTube updated its help page[IAWcite.isMemWeb] to include that all annotations (which had been disabled for new videos and replaced with "cards" early May 2017, but old annotations remained visible) will be removed from videos hosted on the platform on 15 January 2019.

Discussions (Channel Comments)

In addition, YouTube decided that their new “Community Post” feature can not co-exist with the “Discussion” feature[3], earlier known as “Channel Comments”. Therefore, all channels that reach a certain subscriber threshold (formerly 10000, last known threshold: 1500), all channel comments will be permanently erased instead of being merged or co-existing.

The owner from the YouTube channel is still able to access comments from the discussion tab for 30 days after their channel has received the community tab[3] feature. For channel visitors, the “Discussion” tab immediately becomes inaccessible after it becomes replaced with the “Community” tab. Attempting to access the discussion tab via URL get redirected to the main page of the channel.

Removal of AutoShare

AutoShare was a YouTube feature, introduced in 2009[4], that allowed users to automatically share the following actions on YouTube to Twitter, Google Plus and formerly Facebook[4] automatically:

  • Liking a video.
  • Publishing a video.
  • Adding video to playlist (AutoShare did not share name and URL of the target playlist.)
  • Favourising a video (removed YouTube feature, existing favourite videos automatically got moved to “Favourites” playlist.)


The tweet or Google+ post posted by AutoShare included the title and the URL to the target video.
An AutoShared tweet's metadata source tag[5] indicates “Google”.

Example tweet:[IAWcite.isMemWeb]

I liked a @YouTube video https://youtu.be/By_Cn5ixYLg?a YouTube Rewind 2018 but it's actually good

At some point, YouTube AutoShare tweets also included the Twitter handle[IAWcite.isMemWeb] of the YouTube channel the video was shared from, if the owner of the YouTube channel had also linked his Twitter account to his YouTube channel and publicly listed it on the channel.


Twitter's tweet search feature allowed finding out the URL or the title of a deleted video from each other, or the original video URL and full title from knowing the approximate title.
It could also help to trace back changes of a video's title.

This information was useful to find out more about lost videos, e.g. what caused them to be removed, captures by Wayback/Archive.Today, online discussions and potential re-uploads.


On 20190110, YouTube announced[6][7][8] that they would get rid of that feature on 20190131.

After the removal of AutoShare, people still tweeted “I liked a @YouTube video […]” manually.

Video Statistics

Without any clear warning in advance, YouTube has also entirely removed the public “video statistics” feature. The only warning sign was that their new “Polymer” website layout lacked the statistics feature, which could originally be found in the “More” menu below a video.

Only their “One” website layout that they introduced in 2013 and revised in late 2014[9], then barely changed for years, co-existing with “Polymer” since late 2017, included the statistics feature.
When YouTube released their “One” website layout, the number of parameters displayed by the video statistics was also reduced.[10]

The video statistics were also inaccessible from the “One” website since late 2018, and the API endpoints were removed as well.[11].

Parameters (legacy)[12]

  • Total view count + graph.
  • Total comments count + graph.
  • Total favourited count + graph (removed YouTube feature).
  • Total rating counts (no graph)
  • Like and dislike count (no graph)
  • Referral sources with date of initial referral (exact dates available as of 20071130).
    • View counts from each referral source.
      • First featured video view
      • First referral from related videos (separately mentioned)
      • First view from embededd video
      • First view from embededd video (specified website)
      • First referral from YouTube search (for separate search terms)
  • Most popular audiences (age and gender)
  • World map that highlights countries in which the video is more popular.
    • Countries with higher popularity are highlighted in darker green.
  • Honour badges
    • Total honour counts
    • List of honours
      • Rank: Most viewed of all time
      • This list is incomplete yet.

Parameters (recent)

  • Views
  • Total Watch time
    • Average watch time per user.
  • Subscribers gained from said video
  • Share counter: How often the video was shared.

Each of these parameters could be viewed as total cumulative count or counts per day, of which the latter was the default setting.

“One” channel layout

In March 2013, YouTube enforced all channels to change to the previously optional new channel layout called “One” channel layout. [13]

Good bye, unified page.

“One” channel layout crippled the Wayback Machine's ability to crawl channel pages.
The Wayback Machine often just captures the home page of a YouTube channel and misses the channel subpage tabs.

Since “One” channel layout was introduced in March 2013[13], the information of YouTube channels is no longer unified onto a single page, but has been split up onto different, separate subpages:

  • /home
  • /feed
    • (Includes “shared thoughts”[IAWcite.isMemWeb]).
    • Thoughts shared via the “Share your thoughts” feature (not to be confused with discussions or community) can only be written with the legacy channel layout (accessible via ?disable_polymer=1 URL parameter) but read via the new polymer channel layout too.
    • This feature is potentially endangered due to the lack of support from polymer layout, and depreciation of purpose. Existing thoughts shared via this feature could be deleted.
  • /videos
  • /playlists
  • /discussion
  • /community
  • /about
    • Does include channel views, total view count, subscriber count and channel creation date.
    • Includes user-added information such as E-Mail address (ReCaptcha needed to access due to spam prevention), channel description, user-specified country, link farm.
  • /search

Some channels have the custom layout[14] feature turned off. In that case, the channel description shows on the channel's home page[15], but the discussion page (usually not enabled by default in the customized channel layout mode) is enabled and accessible via the URL.

Because all this information is no longer on a single page, if the Wayback Machine captures a channel page, it often only captures the main (home) page of the channel, dismissing the information from the other channel tabs because they are on different pages instead of all information on one page.

Custom channel layouts

Due to the lack of customizability of the “One Channel Layout”, as of March 2013, all custom creative channel designs have been deleted. Also other information such as channel view counts are no longer shown, but channel view counts can still be accessed via the YouTube API.[16]

Video counter

Prior to that layout change, when YouTube video title stood above instead of below the video[17], there was a menu where one could quickly see other videos uploaded by the channel.[18] If this feature existed today, it could have helped chromebot save information about the 20 most recent uploads of a channel the video was archived from.

From March 2013 to August 2014[9], the video counter was still visible next the channel name,[19] although clicking on it just refers to the /videos page of the uploaders channel instead of directly listing the recent uploads.[18]

YouTube's August 2014 redesign[9], which has slightly changed throughout the years, still co-exists with the polymer design and is still accessible as of May 2019 by appending the &disable_polymer=1 or &disable_polymer=true parameter to the URL.
Since that August 2014 redesign, the number of publicly available videos is also no longer visible directly from the watch page[9]. And also not from the channel's “about” page, but only from the search results.

However, SocialBlade[IAWcite.isMemWeb] tracks the statistics of as many YouTube channels as possible, storing historical data.

Comment loading

Since YouTube's “One Channel Layout” redesign, comments “extraload” using AJAX, which means that they do not get loaded within the page itself, but only start loading after scrolling down towards the comments. This made the comment section inaccessible to the Wayback Machine. However, there was a page called youtube.com/all_comments?v=<video ID>, which actually loaded the first few comments without AJAX (included in HTML source code), but loading more comments required AJAX. But that one was discontinued and started redirecting to the main /watch?v= page since January of 2016. However, archive.today still was able to archive YouTube comments until late 2017. As of April 2019, archiving YouTube comments using http://archive.today/ is still possible by linking directly to a comment using YouTube's lc URL parameter.


Chromebot, operated via #ArchiveBot IRC, can still be used to archive YouTube comments thanks to it's bottomless page scrolling capabilities.

Comments on YouTube can be sorted by Top Comments or by Newest Comments. The uploader of a video can specify which way of sorting is used for the video by default, but it can be adjusted manually by the user. The default preset is Top Comments. However, no known URL parameter is able to select the sorting methods for the comments so far. Therefore, crawlers that can access the comments can only crawl them in the preselected way of sorting.

Trivia:

  • The Top Comments are not necessarily the comments with the highest number of upvotes, probably because YouTube does not want always the same comments to stay on top for too long. Older comments get pushed down despite having a high rating.
  • At some point, YouTube started hiding negative comment ratings and only shows how often a comment has been rated positively. Rating a comment negatively is still possible however. It's effect is pushing comments further down from the Top Comments.

Video Responses

“Video responses” was a feature where users could respond to videos with new or existing videos that were listed above the comment sections.

After YouTube removed that feature, the existing videos that were posted as response were not removed. But the links between the original video and the video used as response were removed.

Video reactions

YouTube's “Video Reactions” feature (2011) is similar to Facebook's extended liking feature[IAWcite.isMemWeb][20] since 2016.

The possible reactions were: “LOL, OMG, EPIC, CUTE, WTF or FAIL”.[21][22][23]

To view reaction counts, one needed to be logged in.

But the feature got removed within just months after it's initial release, erasing all existing reactions.

Reasons for video deletions

When a video gets removed for any reason (e.g. manually removed by uploader; uploader closed channel; guideline issues; channel terminated) or not available in a specific region, YouTube usually displays the reason for why the video can not be played.

However, at some point (estimatedly 2013 or 2014), YouTube started purging unavailability reasons for manually deleted videos and videos and channel pages of closed channels, meaning that YouTube shows “This video does not exist.” or “This video is unavailable.”, as if the video never existed in first place.

Since then, reasons for videos that have been permanently erased will only be displayed for a very limited time, after which the video reads as unavailable instead of removed by user or closed channel.

Example:

YouTube does not differentiate between these videos after the deletion reason gets purged.


Since 2017, a private video will also indicate as “not available” (same as a video that never existed in first place) immediately after privated, while playlists that contain the unavailable videos still indicate the distinction between private and deleted videos.

YouTu.be

YouTu.be used to be an image hoster back in 2006. In late 2015, they created a robots.txt file that disallows all crawlers (or “User-agent: * Disallow:/”).


Bulletins

YouTube used to have a feature called “Bulletins” in it's earlier years, likely introduced in 2006.[24]

After co-existing with “Channel Comments” that would later become “Discussions”, that feature got removed at some point, erasing all existing Bulletin posts.


das_captcha

From approximately 2010 to 2012, a whole generation of Wayback Machine captures of YouTube pages were redirected to /das_captcha (example)[IAWcite.isMemWeb] by HTTP 303 request, rendering these Wayback captures next to useless.



Stories

“Stories” is a feature of YouTube[25][26][27][28] which works in a similar way to , and . It can exclusively be accessed from the YouTube application for mobile devices (not YouTube's mobile website or desktop website). It allows users, momentarily exclusively with channels that have over 10000 subscribers, to publish vertical videos, usually filmed from their mobile devices, and optionally edited with e.g. text, using their basic editing features.

Volatility

“Story” videos automatically get deleted in 7 days (= 1 week) after being posted.

Accessability

“Stories” can be accessed from the “Stories” tab on YouTube channels when viewed in the mobile application.

Removed or blocked channels

Trivia

  • All ytimg servers (where YouTube saved images, stylesheets and the .swf file of the flash-based YouTube video player) used to be intoxicated by robots.txt (“user-agent:* disallow:/ noindex:/”). When browsing YouTube layouts starting circa 2008 using the Wayback Machine, the website could only be viewed as black text on white background, just with different sizes (e.g. <h1> video title), due to the missing images and stylesheet information. Only information visible in the HTML source code of the page itself could be rendered. On February 29th of 2012, the robots.txt file vanished off the ytimg servers, lifting the restrictions and making YouTube more properly browse-able through the Wayback Machine.
  • Yahoo and Bing Video results might contain titles, upload dates and thumbnails of unavailable YouTube videos (i.e. after privated, deleted or channel terminated). Please help saving them to Archive.Today and ArchiveBot/ChromeBot if found, before they become purged from the results.

References

  1. https://github.com/rg3/youtube-dl/pull/5456
  2. https://youtube.googleblog.com/2008/06/new-beta-feature-video-annotations.html
  3. 3.0 3.1 YouTube Help article: “Engage with creators on Community posts” mentions Note: As creators get the Community tab, it will replace the Discussion tab. You can access or delete any comments you left on the Discussion tab for 30 days after creators receive the Community tab. Follow the instructions below.”[IAWcite.isMemWeb]
  4. 4.0 4.1 Article “Youtube Adds AutoSharing – YouTwitFace Is Now Real” by TubularInsights.com, formerly “ReelSEO”, written by Mark R Robertson.[IAWcite.isMemWeb]
  5. Twitter tweet object metadata documentation.[IAWcite.isMemWeb]
  6. “YouTube is apparently removing the ability to auto-share videos to Twitter” – influencerupdate.biz[IAWcite.isMemWeb]
  7. YouTube to Remove Automatic Sharing to Twitter – Here is How to Fix It” article by TechWiser.com[IAWcite.isMemWeb]
  8. Tweet by @TeamYouTube: “After Jan 31st, we're saying goodbye 👋 to automated tweets like the one below. You can still share your YouTube activity with your followers in more customized posts via the Share button. Full update →[IAWcite.isMemWeb] https://goo.gl/ef8Vc3”
  9. 9.0 9.1 9.2 9.3 About YouTube's August 2014 layout that is still optionally accessible as of May 2019[IAWcite.isMemWeb]
  10. What public YouTube video statistics looked like in 2008[IAWcite.isMemWeb], 2010[IAWcite.isMemWeb] and shortly before discontinued[IAWcite.isMemWeb].
  11. Tweet by @Omarroth1: “YouTube actually removed any publicly available analytics maybe a couple months ago (the "analytics" tab that used to be available in the "report" and "transcription" tab. The endpoint is also no longer available. I'll see if I can write something 1/2”[IAWcite.isMemWeb] “Unfortunately the internal endpoint has been completely removed. It used to be `/insights_ajax`, now returns 404.”[IAWcite.isMemWeb]
  12. YouTube legacy video statistics (removed feature) as of June 2010[IAWcite.isMemWeb]
  13. 13.0 13.1 YouTube “One Channel” layout propaganda. 📅 Uploaded on 2013-03-08 01:06:18 UTC, 👁 309346 views, 👍 1125 likes, 👎 8648 dislikes, 💬4139 comments, mostly negative comments.[IAWcite.isMemWeb]
  14. YouTube Help article: “Customize channel layout”[IAWcite.isMemWeb]
  15. Channel with “customization” turned off, found via [1]. Original URL: https://www.youtube.com/channel/UCUQe-1KNzrCyJG9Lld07DZQ[IAWcite.isMemWeb]
  16. YouTube API: channel statistics[IAWcite.isMemWeb].
  17. YouTube 2012 watch page screenshot[IAWcite.isMemWeb]
  18. 18.0 18.1 (Couldn't find a screenshot of this former feature in action (quickly see recent videos of uploader without needing to leave watch page, by clicking on video counter). Whoever finds a screenshot of the feature in action should please add it to this reference.)
  19. PSY's “Gangnam Style” hits 2 Billion views. YouTube UI below video as of 2013-2014.[IAWcite.isMemWeb]
  20. TechCrunch article from 20160224: “Facebook Enhances Everyone’s Like With Love, Haha, Wow, Sad, Angry Buttons”.[IAWcite.isMemWeb]
  21. AdWeek article from 20110805: “YouTube Trades In LOL, OMG, WTF Buttons For Drop Down Reactions Menu”.[IAWcite.isMemWeb]
  22. MarketingHits.com article “YouTube Testing New “Reaction” Buttons: OMG, Epic, LOL, Fail, WTF, & Cute”,[IAWcite.isMemWeb] originally from ReelSEO[IAWcite.isMemWeb].
  23. 20110602 article by “Google Operating System” (unofficial blog): “YouTube Reactions”.[IAWcite.isMemWeb]
  24. Video: “How To Use YouTube's New Post Bulletin Feature”[IAWcite.isMemWeb]
  25. 201811291200 TheVerge article “YouTube is rolling out its Instagram-like Stories feature to more creators”[IAWcite.isMemWeb]
  26. YouTube Help article “Watch YouTube Stories”[IAWcite.isMemWeb]
  27. YouTube Help article: “YouTube Stories for creators”[IAWcite.isMemWeb]
  28. YouTube Creator Academy: Lessons: Stories.[IAWcite.isMemWeb]

See also

External links

v · t · e         YouTube

GLAM · Governments · Local TV News · VHS


v · t · e         Archive Team
Current events

Alive... OR ARE THEY · Deathwatch · Projects

Archiveteam.jpg
Archiving projects

APKMirror · Archive.is · BetaArchive · Government Backup (#datarefuge · ftp-gov· Gmane · Internet Archive · It Died · Megalodon.jp · OldApps.com · OldVersion.com · OSBetaArchive · TEXTFILES.COM · The Dead, the Dying & The Damned · The Mail Archive · UK Web Archive · WebCite · Vaporwave.me

Blogging

Blog.pl · Blogger · Blogster · Blogter.hu · Freeblog.hu · Fuelmyblog · Jux · LiveJournal · My Opera · Nolblog.hu · Open Diary · ownlog.com · Posterous · Powerblogs · Proust · Roon · Splinder · Tumblr · Vox · Weblog.nl · Windows Live Spaces · Wordpress.com · Xanga · Yahoo! Blog · Zapd

Cloud hosting/file sharing

aDrive · AnyHub · Box · Dropbox · Docstoc · Google Drive · Google Groups Files · iCloud · Fileplanet · LayerVault · MediaCrush · MediaFire · Mega · MegaUpload · MobileMe · OneDrive · Pomf.se · RapidShare · Ubuntu One · Yahoo! Briefcase

Corporations

Apple · IBM · Google · Loblaw · Lycos Europe · Microsoft · Yahoo!

Events

Arab Spring · Great Ape-Snake War · Spanish Revolution

Font Repos

DaFont · Google Web Fonts · GNU FreeFont · Fontspace

Forums/Message boards

4chan · Captain Luffy Forums · College Confidential · DSLReports · ESPN Forums · forums.starwars.com · HeavenGames · Invisionfree · NeoGAF · The Classic Horror Film Board · Yahoo! Messages · Yahoo! Neighbors · Yuku.com

Gaming

Atomicgamer · Bazaar.tf · City of Heroes · Club Nintendo · Counter-Strike: Global Offensive · CS:GO Lounge · Desura · Dota 2 · Dota 2 Lounge · Emulation Zone · ESEA · GameBanana · GameMaker Sandbox · GameTrailers · Halo · HLTV.org · Infinite Crisis · joinDOTA · League of Legends · Liquipedia · Minecraft.net · Player.me · Playfire · Raptr · Steam · SteamDB · Team Fortress 2 · TF2 Outpost · Warhammer · Xfire

Image hosting

500px · AOL Pictures · Blipfoto · Blingee · Canv.as · Camera+ · Cameroid · DailyBooth · Degree Confluence Project · deviantART · Demotivalo.net · Flickr · Fotoalbum.hu · Fotolog.com · Fotopedia · Frontback · Geograph Britain and Ireland · GTF Képhost · ImageShack · Imgh.us · Imgur · Inkblazers · Instagram · Kepfeltoltes.hu · Kephost.com · Kephost.hu · Kepkezelo.com · Keptarad.hu · Madden GIFERATOR · MLKSHK · Microsoft Clip Art · Microsoft Photosynth · Nokia Memories · noob.hu · Odysee · Panoramio · Photobucket · Picasa · Picplz · Pixiv · Portalgraphics.net · PSharing · Ptch · puu.sh · Rawporter · Relay.im · ScreenshotsDatabase.com · Snapjoy · Streetfiles · Tabblo · Tinypic · Trovebox · TwitPic · Wallbase · Wallhaven · Webshots · Wikimedia Commons

Knowledge/Wikis

arXiv · Citizendium · Clipboard.com · Deletionpedia · EditThis · Encyclopedia Dramatica · Etherpad · Everything2 · infoAnarchy · GeoNames · GNUPedia · Google Books (Google Books Ngram· Horror Movie Database · Insurgency Wiki · Knol · Lost Media Wiki · Neoseeker.com · Notepad.cc · Nupedia · OpenCourseWare · OpenStreetMap · Orain · Pastebin · Patch.com · Project Gutenberg · Puella Magi · Referata · Resedagboken · SongMeanings · ShoutWiki · The Internet Movie Database · TropicalWikis · Uncyclopedia · Urban Dictionary · Urban Exploration Resource · Webmonkey · Wikia · Wikidot · WikiHow · Wikkii · WikiLeaks · Wikipedia (Simple English Wikipedia· Wikispaces · Wikispot · Wik.is · Wiki-Site · WikiTravel · Word Count Journal

Magazines/Blogs/News

Cyberpunkreview.com · Game Developer Magazine · Gigaom · Hardware Canucks · Helium · JPG Magazine · Polygamia.pl · San Fransisco Bay Guardian · Scoop · Regretsy · Yahoo! Voices

Microblogging

Heello · Identi.ca · Jaiku · Mommo.hu · Plurk · Sina Weibo · Twitter · TwitLonger

Music/Audio

AOL Music · Audimated.com · Cinch · digCCmixter · Dogmazic.net · Earbits · exfm · Free Music Archive · Gogoyoko · Indaba Music · Instacast · Jamendo · Last.fm · Music Unlimited · MOG · PureVolume · Reverbnation · ShareTheMusic · SoundCloud · Soundpedia · This Is My Jam · TuneWiki · Twaud.io · WinAmp

People

Aaron Swartz · Michael S. Hart · Steve Jobs · Mark Pilgrim · Dennis Ritchie · Len Sassaman Project

Protocols/Infrastructure

FTP · Gopher · IRC · Usenet · World Wide Web
BitTorrent DHT

Q&A

Askville · Answerbag · Answers.com · Ask.com · Askalo · Baidu Knows · Blurtit · ChaCha · Experts Exchange · Formspring · GirlsAskGuys · Google Answers · Google Baraza · JustAnswer · MetaFilter · Quora · Retrospring · StackExchange · The AnswerBank · The Internet Oracle · Uclue · WikiAnswers · Yahoo! Answers

Recipes/Food

Allrecipes · Epicurious · Food.com · Foodily · Food Network · Punchfork · ZipList

Social bookmarking

Addinto · Backflip · Balatarin · BibSonomy · Bkmrx · Blinklist · BlogMarks · BookmarkSync · CiteULike · Connotea · Delicious · Designer News · Digg · Diigo · Dir.eccion.es · Evernote · Excite Bookmark · Faves · Favilous · folkd · Freelish · Getboo · GiveALink.org · Gnolia · Google Bookmarks · Hacker News · HeyStaks · IndianPad · Kippt · Knowledge Plaza · Licorize · Linkwad · Menéame · Microsoft Developer Network · myVIP · Mister Wong · My Web · Mylink Vault · Newsvine · Oneview · Pearltrees · Pinboard · Pocket · Propeller.com · Reddit · sabros.us · Scloog · Scuttle · Simpy · SiteBar · Slashdot · Squidoo · StumbleUpon · Twine · Vizited · Yummymarks · Xmarks · Yahoo! Buzz · Zootool · Zotero

Social networks

Bebo · BlackPlanet · Classmates.com · Cyworld · Dogster · Dopplr · douban · Ello · Facebook · Flixster · FriendFeed · Friendster · Friends Reunited · Gaia Online · Google+ · Habbo · hi5 · Hyves · iWiW · LinkedIn · Miiverse · mixi · MyHeritage · MyLife · Myspace · myVIP · Netlog · Odnoklassniki · Orkut · Plaxo · Qzone · Renren · Skyrock · Sonico.com · Storylane · Tagged · tvtag · Upcoming · Viadeo · Vine · Vkontakte · WeeWorld · Weibo · Wretch · Yahoo! Groups · Yahoo! Stars India · Yahoo! Upcoming · more sites...

Shopping/Retail

Alibaba · AliExpress · Amazon · Apple Store · Barnes & Noble · DirectCanada · eBay · Kmart · NCIX · Printfection · RadioShack · Sears · Sears Canada · Target · The Book Depository · ThinkGeek · Toys "R" Us · Walmart

Software/code hosting

Android Development · Alioth · Assembla · BerliOS · Betavine · Bitbucket · BountySource · Codecademy · CodePlex · Freepository · Free Software Foundation · GNU Savannah · GitHost  · GitHub · GitHub Downloads · Gitorious · Gna! · Google Code · ibiblio · java.net · JavaForge · KnowledgeForge · Launchpad · LuaForge · Maemo · mozdev · OSOR.eu · OW2 Consortium · Openmoko · OpenSolaris · Ourproject.org · Ovi Store · Project Kenai · RubyForge · SEUL.org · SourceForge · Stypi · TestFlight · tigris.org · Transifex · TuxFamily · Yahoo! Downloads

Television/Radio

ABC · Austin City Limits · BBC · CBC · CBS · Computer Chronicles · CTV · Fox · G4 · Global TV · Jeopardy! · NBC · NHK · PBS · Penn & Teller: Bullshit! · The Howard Stern Show · TV News Archive (Understanding 9/11)

Torrenting/Piracy

ExtraTorrent · EZTV · isoHunt · KickassTorrents · The Pirate Bay · Torrentz · Library Genesis

Video hosting

Academic Earth · Bambuser · Blip.tv · Epic · Google Video · Justin.tv · Niconico · Nokia Trailers · Oddshot.tv · Plays.tv · Qwiki · Skillfeed · Stickam · TED Talks · Ticker.tv · Twitch.tv · Ustream · Videoplayer.hu · Viddler · Viddy · Vidme · Vimeo · Vine · Vstreamers · Yahoo! Video · YouTube · Famous Internet videos (Me at the zoo)

Web hosting

Angelfire · Brace.io · BT Internet · CableAmerica Personal Web Space · Claranet Netherlands Personal Web Pages · Comcast Personal Web Pages · Extra.hu · FortuneCity · Free ProHosting · GeoCities (patch· Google Business Sitebuilder · Google Sites · Internet Centrum · MBinternet · MSN TV · Nifty · Nwnyet · Parodius Networking · Prodigy.net · Saunalahti Iso G · Swipnet · Telenor · Tripod · University of Michigan personal webpages · Verizon Mysite · Verizon Personal Web Space · Webzdarma · Virgin Media

Web applications

Mailman · MediaWiki · phpBB · Simple Machines Forum · vBulletin

Information

A Million Ways to Die on the Web · Backup Tips · Cheap storage · Collecting items randomly · Data compression algorithms and tools · Dev · Discovery Data · DOS Floppies · Fortress of Solitude · Keywords · Naughty List · Nightmare Projects · Rescuing floppy disks · Rescuing optical media · Site exploration · The WARC Ecosystem · Working with ARCHIVE.ORG

Projects

ArchiveCorps · Audit2014 · Emularity · Faceoff · FlickrFckr · Froogle · INTERNETARCHIVE.BAK (Internet Archive Census· IRC Quotes · JSMESS · JSVLC · Just Solve the Problem · NewsGrabber · Project Newsletter · Valhalla · Web Roasting (ISP Hosting · University Web Hosting· Woohoo

Tools

ArchiveBot · ArchiveTeam Warrior (Tracker· Google Takeout · HTTrack · Video downloaders · Wget (Lua · WARC)

Teams

Bibliotheca Anonoma · LibreTeam · URLTeam · Yahoo Video Warroom · WikiTeam

Other

800notes · AOL · Akoha · Ancestry.com · April Fools' Day · Amplicate · AutoAdmit · Bre.ad · Circavie · Cobook · Co.mments · Countdown · Distill · Dmoz · Easel · Eircode · Electronic Frontier Foundation · FanFiction.Net · Feedly · Ficlets · Forrst · FunnyExam.com · FurAffinity · Google Helpouts · Google Moderator · Google Reader · ICQmail · IFTTT · Jajah · JuniorNet · Lulu Poetry · Mobile Phone Applications · Mochi Media · Mozilla Firefox · MyBlogLog · NBII · Neopets · Quantcast · Quizilla · Salon Table Talk · Shutdownify · Slidecast · SOPA blackout pages · starwars.yahoo.com · TechNet · Toshiba Support · USA-Gov · Volán · Widgetbox · Windows Technical Preview · Wunderlist · YTMND · Zoocasa

About Archive Team

Introduction · Philosophy · Who We Are · Our stance on robots.txt · Why Back Up? · Software · Formats · Storage Media · Recommended Reading · Films and documentaries about archiving · Talks · In The Media · FAQ