The Mail Archive
|The Mail Archive|
|Archiving status||Not saved yet|
The Mail Archive is what it sounds like; it's an ad-supported mailing list archive that users can add arbitrary mailing lists to. Started in 1998, it currently holds 121,034,946 archived postings, on 4,314 mailing lists as of October 2015.
We could use this as a starting point by parsing the OPML to get a sitemap for future web-scraping.
Single line Bash to scrape OPML file for mailing list URLs:
wget -qO- https://www.mail-archive.com/feeds/feeds.opml | \ egrep -o "^\s*htmlUrl=\"([^\"]*)\"$" | sed 's/^[^"]*"//' | \ sed 's/"$//'