On February 22, 02011, Jeremy Keith made a prediction that he hoped would be proven wrong.
On February 22, 02022, he was proven wrong, and the internet — and, specifically, the Internet Archive — is richer for it.
Keith’s prediction, made via Long Now’s Long Bets service, was that the URL associated with the record of the prediction (http://longbets.org/601/, if you’d like to check it yourself) would no longer be available in eleven years’ time. Keith, a web developer and writer, explained his prediction by noting that “link rot” — the tendency of hyperlinks to stop leading to where they were intended to go — “is the entropy of the web.” Even a site associated with an organization like Long Now that is committed to long-term archiving and planning in the face of a possible digital dark age could fall victim to this plague given even a relatively short interval between prediction and endpoint.
The next year, following a talk given by Keith at Webstock, a New Zealand-based technology conference, the prediction found its challenger in Matthew Haughey, a programmer and developer known for founding the long-running weblog Metafilter. Haughey’s argument hinged on the idea of the web as a relatively mature medium, one where “keeping a site going with a stable URI system is within reach of anyone with moderate technological knowledge.”
For the past 11 years the two sides have watched the state of the web intently, with $500 dollars from each side remaining in Long Now’s escrow account until the resolution of the bet. Fittingly, the two predictors picked internet and computing-related non-profits as the beneficiaries of the wager: for Keith, the Bletchley Park Trust; for Haughey, the Internet Archive.
Reflecting on the bet, both Keith and Haughey noted that, while the wager itself has clearly been decided in Haughey’s favor, the actual state of the impermanence of the web is more murky.
Keith writes:
I’m very happy to lose this bet. When I made the original prediction eleven years ago that a URL on the longbets.org site would no longer be available, I did so in a spirit of mischief—it was a deliberately meta move. But it was also informed by a genuine feeling of pessimism around the longevity of links on the web. While that pessimism was misplaced in this case, it was informed by data. The lifetime of a URL on the web remains shockingly short. What I think has changed in the intervening years is that people may have become more accustomed to the situation. People used to say “once something is online it’s there forever!”, which infuriated me because the real problem is the exact opposite: if you put something online, you have to put in real effort to keep it online. After all, we don’t really buy domain names; we just rent them. And if you publish on somebody else’s domain, you’re at their mercy: Geocities, MySpace, Facebook, Medium, Twitter.
Haughey’s own musing on the bet focuses on its limited scope: “The way the bet was written and agreed upon isn’t so much about the longevity of the web directly, instead it’s more about the longevity of the Long Bets project and website, which kind of misses the mark in my mind all these years later.”
The Internet Archive’s Jason Scott, a Long Now Speaker, also wrote a reflection on the conclusion of the bet, calling its resolution in the Internet Archive’s favor “at best a bittersweet victory, and at its darkest interpretation, a small oasis in a desert.”
The durability of one URL does not determine the broader state of the web. As both predictors noted, an untold number of personal web pages, social media profiles, and other pieces of the broader web go dark with every passing day, leaving behind only the residue of now-broken links. One study published in 02021 found that 25% of “deep links” — that is, links to specific pages within a broader site — found in New York Times between 01998 and 02019 were completely inaccessible. Even the accessible links frequently did not link to their original context, with one-eighth of a smaller sample of intact links leading to pages with a significant amount of “content drift” relative to their original content.
Thankfully, the Long Bets site has not yet succumbed to the forces of web entropy that have claimed so many others. The broader trends of the web do not paint quite so rosy a picture, though — and it is fitting that the resolution of the bet has resulted in a donation to the Internet Archive, an organization that has spent the last 25 years pushing back against the ephemerality of the web and the forces that would use it for ill.
