Get in your comfort zone
MSN TV, formerly known as WebTV, is a bundled thin client and online service which is closing on September 30, 2013. Microsoft offers a bunch of migration options for MSN TV users, though it does not appear that an easy transition route will be offered for home pages. Many of these date back to the early 2000s, and all will be lost.
- Service Closure Announcement
- The MSN TV service will be closing on September 30, 2013. If you are an MSN TV subscriber, click here to read the Closure FAQ for details on what you should do before the service ends. You will also receive an email and letter providing more information.
URLs of MSN TV web pages take the form of either
community-X.webtv.net where X is a number from 1 to 4.
Sometimes random junk gets added into the start of the path part of the URL, in the form of
/@HH!XX!XX!XXXXXXXXXXXX/ for example, http://community-2.webtv.net/@HH!17!BF!62DA2CCF370F/TvFoutreach/COUNTDOWNTO666/ which is the same as http://community-2.webtv.net/TvFoutreach/COUNTDOWNTO666/. This will have to be sed'd out to remove duplicates.
Sometimes @Lookup will appear in the same place in the URL. This, too, will need to be sed'd out.
Pages are created with predictable filenames. So if we have:
Then chances are very good that page2.html, page3.html, page4.html, etc etc in the same directory, as well as a index.html, will exist as well. It'll also be worth checking for higher numbers as well. This pageX.html pattern seems to be a default name for MSN TV's page creator; a quick scan of the lists shows that there are no files on the community-* sites that end in ".html" which are not index.html or pageX.html
The below lists are raw, incomplete and overlapping. When the time comes to crawl MSN TV, the below lists will need to be merged, de-duplicated, and further processed to find more URLs.
- List of WebTV community sites, scraped from Wikimedia wikis
- List of URLs from a 2009-ish dump of the Open Directory Project, some may no longer work
- List of URLs scraped from a Twitter search. Original URLs before filtering
- List of URLS from Bing API searches on site:webtv.net (12,000+). Older version of this list.
- List of URLs in the Common Crawl index. (1746)
- *.webtv.net DNS names, might be for internal use, may or may not be anything public-facing here
- List of URLs from URLTeam, thanks to soultcer. Original unfiltered list
Archives of what was able to be grabbed are in webtv_fire_grab.