Long Bets has arrived at a decision for Long Bet #2 between blogger Dave Winer and Martin Nisenholtz of the NY Times. At stake is US$2000.00 plus half the interest that has accrued over the last 5 years in the Farsight Fund, all of which will go to the charity of the winner’s choice.
In the bet Winer asserts, “In a Google search of five keywords or phrases representing the top five news stories of 02007, weblogs will rank higher than the ‘New York Times‘ Web site”. The premise of this bet is excellent, but unfortunately the arguments were quite vague on how to adjudicate the bet. Long Bets encourages bettors to construct arguments that involve the least amount of interpretation possible. Once this bet came up for adjudication we urged both parties to come to their own decision, but they asked Long Bets to be the final arbiter. We have done our best with the information and resources available to us, but this process should be a good instructor both to future bettors and ourselves…
The major questions that affect the interpretation of this bet:
Q: Which list of “2007 top stories” to use?
A: We chose the Associated Press list, as it was the only one suggested by one of the bettors (Nisenholtz), and it was in effect at the time of the bet origination. We found many others, (some listed in the notes below), that may actually be better indices of what a “top story” is, but we felt that the AP list was our best choice for this bet.
Q: What is a weblog? Does Wikipedia count? What about the NY Times blog or other commercial blogs? Does it include any non-commercial user submitted web site?
A: We decided that a weblog had to be something that would have been recognized as a blog in 02002. This includes ad supported blogs and commercial blogs like those of the NY Times. While the bettors argument in this case discusses why non-commercial content will beat out commercial content, Winer never provides a definition of a weblog. As it turns out, including major news source blogs like those of the NY Times or sources like Wikipedia do not affect the ultimate outcome in the case of this bet, but they certainly could have.
Q: What is the NY Times? Does the International Herald Tribune count (which is owned by the NY Times and its content comes from there)?
A: We determined that it had to be on the nytimes.com web site to count. If the bettor wanted subsidiaries or other associated derivative content to count, they should have specified it in their argument. This did affect the outcome of one of the searches where the IHT.com result came in at 9 and blogs came in at 10. This result would not have affected the ultimate decision however.
Some other notes: The bettors also never defined what the search semantics should be, and or what date the searches should occur on. Both of which affect the data a fair amount. We tried the searches in a number of ways and a number of times since AP released their list of stories in December to arrive at our decision. We disregarded any search results that were dated after 12/31/02007 when calculating search rank.
Here are 02007’s top stories, as voted by AP Journalists with search rankings (lower is better). We also include results of the highest non-commercial/user submitted content and highest ranked commercial content as a reference.
“VIRGINIA TECH KILLINGS” (NYT score 26, blog 10) winner Blogs
Highest user contributed result: Wikipedia 1
Highest commercial news outlet result: USA Today 2
“MORTGAGE CRISIS” (NYT score 2, blog 10) winner NYT
Highest user contributed result: Wikipedia 1
Highest commercial news outlet result: NYT 2
“IRAQ WAR” (NYT score 24, blog 5,) winner Blogs
Highest user contributed result: Wikipedia 1
Highest commercial news outlet result: CNN 3
“OIL PRICES” (NYT score 172, blog 38) winner Blogs
Highest user contributed result: Monga Bay Blog 38
Highest commercial news outlet result: Bloomberg 1
“CHINESE EXPORTS” (NYT score 57, blog 3) winner Blogs
Highest user contributed result: Blogging Stocks 3
Highest commercial news outlet result: China Today 1
- Adding up page rank winners blogs win 4 to 1.
- Adding up page rank winners of user submitted content vs. commercial content, user submitted content wins 3-2.
- If you average page ranks of the NYT (avg rank 56.2) vs. blogs (avg. rank 13.2) Blogs win.
- If you use an average rank of user submitted content (avg. rank 8.8) vs. commercial (avg. rank 1.8) Commercial news outlets win.
The Long Bets decision on this bet is in favor of Winer’s side, weblog page ranks came out ahead of the NY Times. We will be calculating interest and sending a check on to Dave Winer’s charity of choice the World Wide Web Consortium in the next month.
Notes:
Aside from the observation that Wikipedia often ranks very high and was not really considered at the time of this bet in 02002, another interesting note was how well government sites ranked in subjects like oil prices, Chinese exports, and others. The government sites are often listed in the top ten of these types of subjects showing that people are also turning to the government websites for authority.
The other interesting thing to us was how much the bettors own definitions (or lack there of in this case) affected the bet. For instance had the bet been structured around commercial vs non-commercial content, and they had chosen an average ranking system (which actually seems to answer the question being asked more clearly), commercial content would have won by a factor of more than four.
Also of note is that with a slightly different analysis Rogers Cadenhead did come up with the same winning results based on page rank over at his blog Work Bench.
For reference here are some other “Top Stories of 2007” lists that could have been considered. Testing the first two of these lists yielded results similar to the AP list.
Pew’s Project for Excellence in Journalism’s News Coverage and Interest Indexes.
Foreign Policy, top 10 stories missed in 2007
CNN (not ranked – chronological)
MSNBC graph showing top story of the day, for the year (most clicked)
Telegraph UK Top read stories of 2007, by category
Doctors Without Borders (top *underreported* humanitarian stories):
BBC News (most popular)
Yahoo! News (most emailed)
