This doesn't work as well as you think - financial markets tend to break when participants are optimizing for the probability of finishing in the top N, especially when N is small. If I were trying to maximize my probability of winning this tournament in a blended pool of rational and irrational agents, I would be:
1) taking way more risk than I would if my goal were simply to maximize my portfolio's returns
2) taking risks that give me different risk exposures from other participants, even if the arithmetic expectation isn't in my favor.
For example, if the true probability of an event happening is 90%, and various market participants bid it up from 50% to 88%, well, I am probably more likely to sell it than buy it at those prices. The intuition is that if the event resolves YES, I will have bought it at the worst price among all the people who bought YES, but if the event resolves NO then I will have sold it at the best price among the people who sold YES.
Requiring people to trade a variety of markets also doesn't do much - it's pretty easy for people to buy YES at 99% on a variety of already-basically-resolved markets to meet this requirement, especially when those markets will formally resolve quickly.
All in all I expect that this competition tests the skill of "how good are you at gaming formal systems" a lot more than it tests the skill of "how good are you at making well calibrated predictions about the world".
Source: I was a quantitative trader for 2 years and they made us play a bunch of games market trading games with solutions like these during training.
I will second this as someone with a few years experience in the derivatives trading industry, and a top forecaster on Good Judgment Open. Markets and forecasting are not the same, even though they are related. I would be very excited if this was a forecasting contest, but it is a manipulate-illiquid-markets contest.
One solution, by the way, is to force participants to buy in and trade with real money! You want people to at least have some incentive to respect the Kelly Criterion.
Could this be resolved if the tournament had say 10 rounds and the winner is the one who perhaps (1) has the best worst place finish, finishes in the bottom 75% the fewest times, highest finish for their 3rd worse finish, etc.
Points 1 and 2 seem like they could be somewhat overcome if the tournament requires a large number of small wagers (e.g., an upper limit on profits for a given market). Relatedly, the extent to which this is an issue depends on the size of the tournament, and I'm curious what the organizers envision.
I also think the organizer is trying to dissuade this type of strategy by offering those Gold/Silver/Bronze "awards" (i.e., encouraging participants to value positions other than just the top-5).
I dislike the $50 on 20 markets constraint. It doesn't really solve what you are trying to solve anyway. Someone could just get lucky on a single market that resolves early - lets say there is a fatality between Taiwan and China tomorrow, and someone gets a 10x return on it. They can then just spread $50 around to 19 other markets without really thinking about it, and they will still be up big regardless of how those 19 turn out.
Meanwhile if you spend your energies trying to spread the money around on different markets early, you may be right in your probability assessments on many of them, but will get very limited return on each, since only putting $50 apiece.
This seems difficult to overcome... maybe participants could be limited in $X profit for a given market (e.g., for a 10x market, if the profit limit is $100, the wager limit would be $10). If this is done, I don't think any limit in the amount spread would be necessary. Participants may be able to get around this somewhat using correlated markets, but I don't think this issue is too severe.
It would be best if this $100 limit was hardcoded into the site, and the judges didn't judge post hoc sum capped winnings to calculate scores. Under such post-hoc capping, someone could still take a longshot bet to win $10k. Then, even if just the first $100 is counted for their score, the participant stills has a ton of bonus money to play around with. Nonetheless, I imagine there are still pretty reasonable ways of doing these calculations post hoc if hardcoding the $100 limit isn't possible (e.g., by also subtracting capped losses, so if a player balloons to $10k then makes 100 bets that go even money, they aren't considered to be successful). Such a participant would still have an advantage, but it seems manageable.
This is a clever idea for identifying individuals who have real expertise rather than just credentials.
There also needs to be a system for identifying institutions that foster (or at least identify and collect) individuals with real expertise, rather than foisting frauds on the world. We need an entirely new system for identifying elite institutions.
This was recently driven home when Yale Law students shut down a speaker they disagreed with by shouting and banging on walls, and the Dean took no action against them. Yale Law School does not appear to cultivate elite skills of reasoned debate. There are far less famous, scrappier schools that do a much better job.
Hello! I placed 13th in the tourney, but the email that's its linked to I've lost access to for the past two months (layoffs..) let me know how to reach out to link into the subsequent things from this
I agree with some of the other commenters that using Brier scores would be better. At the moment, I think my strategy to place highly should be something like bet everything on a low probability event. If I lose, spend no more time on the contest. If I win I'm way ahead of the crowd and can buy a bunch of almost certain bets to meet the minimum bet threshold. My only hesitation is that I'm not sure how many contestants there will be.
If I think there are a ton of contestants, I want to go with the above strategy. But if I think it's a smaller group, then I would bet more sincerely since I could possibly win without such a high-variance strategy.
Probably not gonna happen but if you had enough input you would have no more degrees of freedom since every possible state in the system would have been defined and the house would just lose money, or at least reach a profitability limit. Would be neat though, the power to have all inputs.
I'm not too happy with the predictions you can bet on, a lot of them are just election outcomes and other things that are probably inherently unpredictable and you'll just and up selecting for survivorship bias.
This is better but still has problems! If I want to maximize my expected Brier score, I will give my good-faith probability forecasts for each event I consider. If I want to maximize my probability of having one of the N highest Brier scores, I am never in a million years going to say that something is "only 95%" to occur. 100% or bust!
This doesn't work as well as you think - financial markets tend to break when participants are optimizing for the probability of finishing in the top N, especially when N is small. If I were trying to maximize my probability of winning this tournament in a blended pool of rational and irrational agents, I would be:
1) taking way more risk than I would if my goal were simply to maximize my portfolio's returns
2) taking risks that give me different risk exposures from other participants, even if the arithmetic expectation isn't in my favor.
For example, if the true probability of an event happening is 90%, and various market participants bid it up from 50% to 88%, well, I am probably more likely to sell it than buy it at those prices. The intuition is that if the event resolves YES, I will have bought it at the worst price among all the people who bought YES, but if the event resolves NO then I will have sold it at the best price among the people who sold YES.
Requiring people to trade a variety of markets also doesn't do much - it's pretty easy for people to buy YES at 99% on a variety of already-basically-resolved markets to meet this requirement, especially when those markets will formally resolve quickly.
All in all I expect that this competition tests the skill of "how good are you at gaming formal systems" a lot more than it tests the skill of "how good are you at making well calibrated predictions about the world".
Source: I was a quantitative trader for 2 years and they made us play a bunch of games market trading games with solutions like these during training.
You can see some quantification of this here: https://arxiv.org/pdf/2106.11248.pdf (appendix A)
I will second this as someone with a few years experience in the derivatives trading industry, and a top forecaster on Good Judgment Open. Markets and forecasting are not the same, even though they are related. I would be very excited if this was a forecasting contest, but it is a manipulate-illiquid-markets contest.
One solution, by the way, is to force participants to buy in and trade with real money! You want people to at least have some incentive to respect the Kelly Criterion.
They should just make the top 5 polymarket traders the finalists
You want everyone to have the same starting bankroll but yeah
Could this be resolved if the tournament had say 10 rounds and the winner is the one who perhaps (1) has the best worst place finish, finishes in the bottom 75% the fewest times, highest finish for their 3rd worse finish, etc.
Gotta be a metric be quite so easily exploitable?
Points 1 and 2 seem like they could be somewhat overcome if the tournament requires a large number of small wagers (e.g., an upper limit on profits for a given market). Relatedly, the extent to which this is an issue depends on the size of the tournament, and I'm curious what the organizers envision.
I also think the organizer is trying to dissuade this type of strategy by offering those Gold/Silver/Bronze "awards" (i.e., encouraging participants to value positions other than just the top-5).
I dislike the $50 on 20 markets constraint. It doesn't really solve what you are trying to solve anyway. Someone could just get lucky on a single market that resolves early - lets say there is a fatality between Taiwan and China tomorrow, and someone gets a 10x return on it. They can then just spread $50 around to 19 other markets without really thinking about it, and they will still be up big regardless of how those 19 turn out.
Meanwhile if you spend your energies trying to spread the money around on different markets early, you may be right in your probability assessments on many of them, but will get very limited return on each, since only putting $50 apiece.
Going against conventional wisdom is supposed to be rewarding in this game.
This seems difficult to overcome... maybe participants could be limited in $X profit for a given market (e.g., for a 10x market, if the profit limit is $100, the wager limit would be $10). If this is done, I don't think any limit in the amount spread would be necessary. Participants may be able to get around this somewhat using correlated markets, but I don't think this issue is too severe.
It would be best if this $100 limit was hardcoded into the site, and the judges didn't judge post hoc sum capped winnings to calculate scores. Under such post-hoc capping, someone could still take a longshot bet to win $10k. Then, even if just the first $100 is counted for their score, the participant stills has a ton of bonus money to play around with. Nonetheless, I imagine there are still pretty reasonable ways of doing these calculations post hoc if hardcoding the $100 limit isn't possible (e.g., by also subtracting capped losses, so if a player balloons to $10k then makes 100 bets that go even money, they aren't considered to be successful). Such a participant would still have an advantage, but it seems manageable.
This is a clever idea for identifying individuals who have real expertise rather than just credentials.
There also needs to be a system for identifying institutions that foster (or at least identify and collect) individuals with real expertise, rather than foisting frauds on the world. We need an entirely new system for identifying elite institutions.
This was recently driven home when Yale Law students shut down a speaker they disagreed with by shouting and banging on walls, and the Dean took no action against them. Yale Law School does not appear to cultivate elite skills of reasoned debate. There are far less famous, scrappier schools that do a much better job.
Any way to do this without creating a Google account?
No, that’s just the way Manifold Markets is currently set up.
This is awesome!
Hello! I placed 13th in the tourney, but the email that's its linked to I've lost access to for the past two months (layoffs..) let me know how to reach out to link into the subsequent things from this
Is this open to entrants from all over the world, or just the US?
All over the world, unless you live in a country we’re not allowed to send money to for sanctions reasons.
I agree with some of the other commenters that using Brier scores would be better. At the moment, I think my strategy to place highly should be something like bet everything on a low probability event. If I lose, spend no more time on the contest. If I win I'm way ahead of the crowd and can buy a bunch of almost certain bets to meet the minimum bet threshold. My only hesitation is that I'm not sure how many contestants there will be.
If I think there are a ton of contestants, I want to go with the above strategy. But if I think it's a smaller group, then I would bet more sincerely since I could possibly win without such a high-variance strategy.
Probably not gonna happen but if you had enough input you would have no more degrees of freedom since every possible state in the system would have been defined and the house would just lose money, or at least reach a profitability limit. Would be neat though, the power to have all inputs.
I'm not too happy with the predictions you can bet on, a lot of them are just election outcomes and other things that are probably inherently unpredictable and you'll just and up selecting for survivorship bias.
This is better but still has problems! If I want to maximize my expected Brier score, I will give my good-faith probability forecasts for each event I consider. If I want to maximize my probability of having one of the N highest Brier scores, I am never in a million years going to say that something is "only 95%" to occur. 100% or bust!