Great writeup, very interesting results! Congrats on completing it, and congrats to the winners.
I work at Manifold and helped run the tournament. Was surprised that political leaning was so important to the results!
The final calibration was definitely underwhelming, but I think it's not all due to the effect of people trying to take risky bets to win the top prizes. Another factor was a limited supply of currency with which to bet.
One condition of the tournament was that everyone began with a fixed amount of currency: S$ 1000. Each market has an AMM (automated market maker) which was also started with about S$ 2000 liquidity. As a player, you might not want to spend all your balance moving the probability from 85% to 95% if you could make more profit in another market. On the main Manifold site, you can buy more currency, and also there are automatic loans that return you your currency to bet again, which both help.
What's your explanation for why, aside from the midterms, conservatives did better? Was there some event where Republicans did surprisingly well which you should also exclude?
As someone who actually participated in the contest and thus saw how things worked first-hand, your analysis in the first section is *really bad*.
First off, your list of "biggest misses" makes no sense. The actual biggest miss was the Debt Ceiling market, which peaked at 93%, but there were tons of other misses larger than the ones you listed (off the top of my head, China GDP, all the major midterm markets, China COVID, Biden Approval, Erdogan, and Trump Indictment, and probably others as well.)
Additionally, your analysis of the tail probability effect is *way* off the mark. While it is true that there were a few cases of people gambling on longshot odds, that wasn't a significant factor. The main reason that even sure-thing markets would not go to 0/100 is because the participants never had enough liquidity to push them up that much.
The YES/NO asymmetry is probably due to differences in resolution timing affecting the opportunity costs, because most of the questions were posed as "Will X happen before Y time?". This means that betting on NO involves locking up your money until the given date, which was usually months in the future, and thus there is a huge opportunity cost to betting NO. Betting YES meanwhile had no such opportunity cost.
It was a fun contest, though I realized I stood no chance of winning once the china COVID market closed, I really missed the ball on that one. I'm surprised I ended up top 10 even after quitting 2/3rds of the way through.
Great writeup, very interesting results! Congrats on completing it, and congrats to the winners.
I work at Manifold and helped run the tournament. Was surprised that political leaning was so important to the results!
The final calibration was definitely underwhelming, but I think it's not all due to the effect of people trying to take risky bets to win the top prizes. Another factor was a limited supply of currency with which to bet.
One condition of the tournament was that everyone began with a fixed amount of currency: S$ 1000. Each market has an AMM (automated market maker) which was also started with about S$ 2000 liquidity. As a player, you might not want to spend all your balance moving the probability from 85% to 95% if you could make more profit in another market. On the main Manifold site, you can buy more currency, and also there are automatic loans that return you your currency to bet again, which both help.
See also "Opportunity costs are critical" in this writeup: https://blog.polybdenum.com/2023/08/01/how-i-came-second-out-of-999-in-the-salem-center-prediction-market-tournament-without-knowing-anything-about-prediction-markets-and-what-i-learned-along-the-way-part-1.html
The effect of limited currency meant users did not bet probabilities as far as they would have, which matches the worse calibration at the extremes.
What's your explanation for why, aside from the midterms, conservatives did better? Was there some event where Republicans did surprisingly well which you should also exclude?
As someone who actually participated in the contest and thus saw how things worked first-hand, your analysis in the first section is *really bad*.
First off, your list of "biggest misses" makes no sense. The actual biggest miss was the Debt Ceiling market, which peaked at 93%, but there were tons of other misses larger than the ones you listed (off the top of my head, China GDP, all the major midterm markets, China COVID, Biden Approval, Erdogan, and Trump Indictment, and probably others as well.)
Additionally, your analysis of the tail probability effect is *way* off the mark. While it is true that there were a few cases of people gambling on longshot odds, that wasn't a significant factor. The main reason that even sure-thing markets would not go to 0/100 is because the participants never had enough liquidity to push them up that much.
The YES/NO asymmetry is probably due to differences in resolution timing affecting the opportunity costs, because most of the questions were posed as "Will X happen before Y time?". This means that betting on NO involves locking up your money until the given date, which was usually months in the future, and thus there is a huge opportunity cost to betting NO. Betting YES meanwhile had no such opportunity cost.
Which occupations were the most successful at forecasting?
It was a fun contest, though I realized I stood no chance of winning once the china COVID market closed, I really missed the ball on that one. I'm surprised I ended up top 10 even after quitting 2/3rds of the way through.
Maybe a shorter timespan contest next time?