This doesn't work as well as you think - financial markets tend to break when participants are optimizing for the probability of finishing in the top N, especially when N is small. If I were trying to maximize my probability of winning this tournament in a blended pool of rational and irrational agents, I would be:
1) taking way more risk than I would if my goal were simply to maximize my portfolio's returns
2) taking risks that give me different risk exposures from other participants, even if the arithmetic expectation isn't in my favor.
For example, if the true probability of an event happening is 90%, and various market participants bid it up from 50% to 88%, well, I am probably more likely to sell it than buy it at those prices. The intuition is that if the event resolves YES, I will have bought it at the worst price among all the people who bought YES, but if the event resolves NO then I will have sold it at the best price among the people who sold YES.
Requiring people to trade a variety of markets also doesn't do much - it's pretty easy for people to buy YES at 99% on a variety of already-basically-resolved markets to meet this requirement, especially when those markets will formally resolve quickly.
All in all I expect that this competition tests the skill of "how good are you at gaming formal systems" a lot more than it tests the skill of "how good are you at making well calibrated predictions about the world".
Source: I was a quantitative trader for 2 years and they made us play a bunch of games market trading games with solutions like these during training.
I dislike the $50 on 20 markets constraint. It doesn't really solve what you are trying to solve anyway. Someone could just get lucky on a single market that resolves early - lets say there is a fatality between Taiwan and China tomorrow, and someone gets a 10x return on it. They can then just spread $50 around to 19 other markets without really thinking about it, and they will still be up big regardless of how those 19 turn out.
Meanwhile if you spend your energies trying to spread the money around on different markets early, you may be right in your probability assessments on many of them, but will get very limited return on each, since only putting $50 apiece.
This is a clever idea for identifying individuals who have real expertise rather than just credentials.
There also needs to be a system for identifying institutions that foster (or at least identify and collect) individuals with real expertise, rather than foisting frauds on the world. We need an entirely new system for identifying elite institutions.
This was recently driven home when Yale Law students shut down a speaker they disagreed with by shouting and banging on walls, and the Dean took no action against them. Yale Law School does not appear to cultivate elite skills of reasoned debate. There are far less famous, scrappier schools that do a much better job.
Hello! I placed 13th in the tourney, but the email that's its linked to I've lost access to for the past two months (layoffs..) let me know how to reach out to link into the subsequent things from this
I agree with some of the other commenters that using Brier scores would be better. At the moment, I think my strategy to place highly should be something like bet everything on a low probability event. If I lose, spend no more time on the contest. If I win I'm way ahead of the crowd and can buy a bunch of almost certain bets to meet the minimum bet threshold. My only hesitation is that I'm not sure how many contestants there will be.
If I think there are a ton of contestants, I want to go with the above strategy. But if I think it's a smaller group, then I would bet more sincerely since I could possibly win without such a high-variance strategy.
Probably not gonna happen but if you had enough input you would have no more degrees of freedom since every possible state in the system would have been defined and the house would just lose money, or at least reach a profitability limit. Would be neat though, the power to have all inputs.
I'm not too happy with the predictions you can bet on, a lot of them are just election outcomes and other things that are probably inherently unpredictable and you'll just and up selecting for survivorship bias.
Introducing the Salem/CSPI Forecasting Tournament
This doesn't work as well as you think - financial markets tend to break when participants are optimizing for the probability of finishing in the top N, especially when N is small. If I were trying to maximize my probability of winning this tournament in a blended pool of rational and irrational agents, I would be:
1) taking way more risk than I would if my goal were simply to maximize my portfolio's returns
2) taking risks that give me different risk exposures from other participants, even if the arithmetic expectation isn't in my favor.
For example, if the true probability of an event happening is 90%, and various market participants bid it up from 50% to 88%, well, I am probably more likely to sell it than buy it at those prices. The intuition is that if the event resolves YES, I will have bought it at the worst price among all the people who bought YES, but if the event resolves NO then I will have sold it at the best price among the people who sold YES.
Requiring people to trade a variety of markets also doesn't do much - it's pretty easy for people to buy YES at 99% on a variety of already-basically-resolved markets to meet this requirement, especially when those markets will formally resolve quickly.
All in all I expect that this competition tests the skill of "how good are you at gaming formal systems" a lot more than it tests the skill of "how good are you at making well calibrated predictions about the world".
Source: I was a quantitative trader for 2 years and they made us play a bunch of games market trading games with solutions like these during training.
I dislike the $50 on 20 markets constraint. It doesn't really solve what you are trying to solve anyway. Someone could just get lucky on a single market that resolves early - lets say there is a fatality between Taiwan and China tomorrow, and someone gets a 10x return on it. They can then just spread $50 around to 19 other markets without really thinking about it, and they will still be up big regardless of how those 19 turn out.
Meanwhile if you spend your energies trying to spread the money around on different markets early, you may be right in your probability assessments on many of them, but will get very limited return on each, since only putting $50 apiece.
This is a clever idea for identifying individuals who have real expertise rather than just credentials.
There also needs to be a system for identifying institutions that foster (or at least identify and collect) individuals with real expertise, rather than foisting frauds on the world. We need an entirely new system for identifying elite institutions.
This was recently driven home when Yale Law students shut down a speaker they disagreed with by shouting and banging on walls, and the Dean took no action against them. Yale Law School does not appear to cultivate elite skills of reasoned debate. There are far less famous, scrappier schools that do a much better job.
Any way to do this without creating a Google account?
This is awesome!
Hello! I placed 13th in the tourney, but the email that's its linked to I've lost access to for the past two months (layoffs..) let me know how to reach out to link into the subsequent things from this
Is this open to entrants from all over the world, or just the US?
I agree with some of the other commenters that using Brier scores would be better. At the moment, I think my strategy to place highly should be something like bet everything on a low probability event. If I lose, spend no more time on the contest. If I win I'm way ahead of the crowd and can buy a bunch of almost certain bets to meet the minimum bet threshold. My only hesitation is that I'm not sure how many contestants there will be.
If I think there are a ton of contestants, I want to go with the above strategy. But if I think it's a smaller group, then I would bet more sincerely since I could possibly win without such a high-variance strategy.
Probably not gonna happen but if you had enough input you would have no more degrees of freedom since every possible state in the system would have been defined and the house would just lose money, or at least reach a profitability limit. Would be neat though, the power to have all inputs.
I'm not too happy with the predictions you can bet on, a lot of them are just election outcomes and other things that are probably inherently unpredictable and you'll just and up selecting for survivorship bias.