The winner of the $25,000 prize, and the Salem Center fellowship for 2023-2024, is Jonathan Zubkoff, better known to other participants as zubbybadger. We also had a $5,000 award for the person who most improved their portfolio between February 15 and July 31, and the winner was Malte Schrödl, a college student studying in Aachen, Germany. Winners of the Gold, Silver, and Bronze prizes have all been notified, as have the top 20 finishers, who will be invited to a forecasting tournament taking place at the Salem Center in the fall.

The second place finisher in the tournament was at the end going by the handle “guyin2ndplace.” He’s written a very interesting account of his experience. See Part I and Part II. For other commentary on the tournament, see Astral Codex Ten. One can browse the history of all of the markets here, and get historical data related to activity on the site here.

This report is meant to provide a summary of what we have learned. Part I deals with the issue of calibration. How good were our betting markets at actually predicting the future? The rest of the report looks at the participants themselves. Part II gives summary statistics on their backgrounds and political views. Finally, Part III discusses what variables correlated with being a good or bad forecaster.

Data used for the calibration analysis is being made available at the bottom of this report. Most of the data related to the characteristics of bettors is being kept under wraps due to privacy concerns, but we are open to collaborating with those who would like to explore the results. All of the analysis below should be taken as preliminary. We invite others to look for further insights that can be found in the data.

I. How Good Was the Market?

I start by taking the beginning of day probability for each market and each day, and then finding an average for each market across the entire time period. From that, I calculated a Brier score for each individual market. The average across all markets was 0.133. The biggest misses were on the following questions, listed with their average probability and eventual outcome.

1. Republicans favored in Senate on Election Day?: 34.6% chance, settled YES

2. Paul Vallas mayor of Chicago?: 63.9%, settled NO

3. Biden cancels student debt in 2022?: 38.1%, settled YES

4. Will PredictIt survive?: 38.6%, settled YES

5. Michigan abortion rights landslide?: 57.1%, settled NO

I don’t see much of a pattern here. One might’ve thought that the market would be biased in the direction of the political leanings of participants, which are discussed below. But they are more conservative than liberal, and their biggest miss was not believing Republicans would be favored in the Senate on election day, when in fact they were. I also assume that participants wanted PredictIt to survive, given that they were presumably supporters of prediction markets. Overestimating Paul Vallas could’ve been a mistake based on their political and class orientation, but the upset in the Chicago mayoral race seems to have been something a lot of people missed.

For the next analysis, we take the first probability of each market every day across the whole tournament. This gives us 12,521 observations. Since there were 91 markets, that means each market existed for an average of just under 138 days.

I divide the observations into 99 bins, from 1% to 99%, and then calculate the average outcome. For example, the market on whether California would ban flavored tobacco was at 58% at the beginning of the day on August 23, and that market ended up settling as YES. Thus, the 58% likelihood was one observation and the dependent variable connected to it was 1. That observation was grouped in with all other observations that rounded to a 58% likelihood, and this was done for each observation in the dataset.

The results are in Figure 1 below. Dotted lines show 95% confidence intervals as calculated using the Adjusted Wald Technique developed for data with binomial outcomes.

Before the start of the tournament, it was known that by having prizes given at the end, individuals would have an incentive to bet on longshot probability events. Since there’s no advantage to being at, say, the 50th percentile over the 10th percentile, participants would have been smart to take large risks.

Figure 1 shows that this is exactly what they did. Of the 2,504 datapoints for which the market gave the outcome a 17% chance or less of happening, the market resolved YES exactly 0 times. For outcomes that the market gave a 20% chance of happening, the outcome was YES only 6% of the time.

The orange line in Figure 1 represents perfect calibration. Ideally, the blue line would hover around it. But it doesn’t at any point cross the perfect calibration (orange) line until the predicted probability of events is 47%. For high probability events, at every point on the x-axis beginning at 70% each bin had outcomes that were more likely to settle as YES than what the market predicted.

Basically, the market lets you divide events into four categories. Things that are given a 20% or less chance of happening have close to a 0% chance of happening in the real world. Between 20-45% the market becomes more useful but systematically underestimates the likelihood of events. Only between around 45%-70% does the market behave as one would wish. Anything that is given a higher probability than that is close to a sure thing. The problem with betting on longshots seems to be worse at the left side of the graph. In other words, people like the idea of a longshot YES bet over a longshot NO bet. There is probably a reason for this out there in some psychological literature. Or is this perhaps due to non-randomness in how we framed the questions, that is, which outcomes we chose to call YES and which we called NO?

This is pretty bad relative to real prediction markets. Here’s one study of PredictIt, based on prices one month before resolution.

One sees the same problem, with the calibration line being below the perfect calibration line at low prices and above it at higher prices. Nonetheless, this is clearly nowhere near as bad as our own chart. Markets that are based on prizes, like ours was, are clearly not useless. Since the errors are somewhat predictable, they provide good information. One can assume that things are correctly priced near the center of the probability distribution, overpriced at the low end, and underpriced at the high end. We are therefore left with data supporting the well-established but nonetheless important conclusion that prize markets are better than nothing, with markets based on money, or some adequate substitute that doesn’t over incentivize longshot bets, being much superior for predicting high and low probability events.

II. The Bettors

We had 548 individuals take the survey. Of those, 95.1% were male, compared to 4.3% female, and less than 1% other. It was a young crowd, with 48.8% aged 28-42, and another 35.7% aged 27 or younger. 88.9% were straight, with 5.5% bisexual.

Just under 74% were from the United States, and close to 85% identified as white. The sample was extremely secular compared to the general population, with 53% identifying as atheist or agnostic. 77.8% had graduated from college, and 39.7% had some kind of post-graduate degree.

The most common fields were tech (24.7%), academia (grad student or research fellow) (11.4%), government (7.7%), academia (professor) (6%), entrepreneur (non-tech) (6%), academia (other) (5.8%), nonprofit (5.6%), banking (5.2%), tech (business owner) (5.2%), politics (2.8%), and think tank (2.8%).

Individuals were asked to describe themselves politically on a five-point scale, from very liberal (1), to moderate (3), to very conservative (5). The survey takers rated themselves on average 3.26 on economic issues, 2.79 on social issues, and 3.07 overall. In other words, the sample can be described as broadly libertarian leaning.

More precisely, the field can be described as pro-democratic capitalism, of a techno-optimist flavor.

We know this because we asked respondents to rate 78 different ideas, people, and concepts, from Donald Trump to bringing back dinosaurs to embryo selection to banning pornography.

The four statements and ideas they rated most favorably were “I believe people are better off than at any other point in history” (4.07 on a 5-point scale); “I believe capitalism is a better economic system than all available alternatives” (4.07); “I believe anthropogenic climate change is real” (4.01); and “Democracy” (3.72). The least popular were “There should be no law against a boss pressuring an employee for sex” (1.82); “It is more likely than not that the Democrats stole the 2020 election” (1.85); “I believe humans have made contact with aliens” (1.85) (note this was before that topic started to be in the news); and “I believe people in developed countries should have fewer children” (1.89).

As found in previous studies of this kind, participants were more likely than chance to be first borns. 55.1% of respondents reported having no older siblings, compared to 34% who reported no younger siblings. With a sample size of 492 individuals answering both questions, this result passes all conventional thresholds of significance.

III. What Makes a Good Forecaster?

Figure 3 shows the twenty statements with the strongest correlation, positive or negative, with being a good forecaster, defined as the balance in S$ an individual ended up with. For this analysis, I only included bettors who placed at least S$50 on 5 different markets and answered all item questions, of which there were 192 individuals.

If there’s one takeaway from the chart above, it’s that the best forecasters liked left-wing ideas, people, and movements. The best predictor was agreeing with the statement that liberals are more honest than conservatives. Since I’ve written articles to this effect, this warmed my heart. But that result was just part of a general pattern of liberals being better forecasters than conservatives. A univariate regression likewise shows that those who identified as more conservative on the five-point scale ended up with lower balances, with the result just missing the conventional threshold of significance (p = .056).

There are two plausible theories here, other than the possibility that liberals are just superior at predicting the future. First, since this was a highly selected group, it may have been that liberals who participated in the Salem/CSPI tournament were of a higher quality, because they were familiar with and willing to participate in an event that was coded as being ideologically opposite of them. Recall that the crowd leaned more conservative overall, albeit slightly. For reference, participants rated BLM (2.46) higher than Biden (2.34) and Trump (2.1), but lower than libertarianism (3.21) or DeSantis (2.86). The Democratic Party (2.06) and the Republican Party (2.08) were rated about equally negatively.

The second possibility is that the liberal advantage is explained by the results of the midterms. Of our 91 markets, 18 had to do with the American elections held in November 2022. Imagine if each side tended to bet on how they wanted things to turn out. Then liberals would have ended up doing much better, because Democrats outperformed the conventional wisdom. PredictIt gave Republicans a 74% chance of winning the Senate on the day before the election, and the FiveThirtyEight model ended up at 59%.

This led me to wonder, what happens if you simply take out the 18 midterm markets, and see how people did in the other 73 markets? Figure 4 shows the results for the top 20 items most strongly related to forecasting outcomes not having to do with the midterms, otherwise using the same methodology as described for Figure 3.

Suddenly, conservatives are the better forecasters. If you want someone’s advice, you should look for an individual who thinks theology should dominate public life, incest definitely shouldn’t be legal, and Trump was robbed in 2020. Unless, that is, you want to predict the 2022 midterm results. The results in Figure 3 and Figure 4 are not driven by outliers. Removing the biggest winners of the tournament does not change the pattern of liberal respondents doing better overall and conservatives performing better in non-midterm elections.

For both Figure 3 and Figure 4, there is an obvious potential multiple comparisons issue. When a Benjamini-Hochberg FDR correction is applied to the p-values, 7 items remain significant in Figure 3 and 5 in Figure 4.

Is the conclusion that nobody is better than anyone else? Mostly, yes. The relationships found are quite weak, with very few items having a correlation of at least r =+/- 0.2 with forecasting results. Yet I think we do have to give the advantage to liberals. It would be unfair to discount their accomplishment just because much of it was driven by the 2022 midterms. If conservatives were too optimistic about their own side winning, then that is their fault. Maybe Democrats would’ve been too optimistic about their chances in a different year, but we can’t just assume that they would have been. So credit to the left leaning participants, but keep the two caveats in mind that they didn’t do as well in non-midterm markets, and the ones who participated in this tournament are a highly selected group. While the midterm story can explain the overall results, it doesn’t provide an obvious hypothesis as to why conservatives would be better in non-midterm markets.

Data

I’d like to encourage anyone who is interested to explore the data for themselves. Please reach out if you do at contact@cspicenter.org, as we would be interested in hearing about and publicizing your results. All of the analysis in this report was done in Stata or Excel. The spreadsheets to reproduce the data in Part I can be found below, along with the codebook, explaining what’s in each file. As mentioned above, we’re withholding most of the data relied upon in Part II and Part III, as it can be used to identify individuals and gain access to personal information. We are, however, making available the ratings of various items and political self-identification in the file Salemratings below. If you would like to test hypotheses about any other individualized data, contact us and we can possibly discuss some kind of collaboration or non-disclosure agreement.

All files listed below can be found here, in addition to a .json file with all the bets.

Salemcodebook

197KB ∙ PDF file

Download

Download

Salemratings

142KB ∙ XLSX file

Download

Download

Calibration

21.3KB ∙ XLSX file

Download

Download

Allobs

416KB ∙ XLSX file

Download

Download

Below, one can see our number of weekly users since the beginning of the tournament in August. Recently, we’ve gone from around 60-80 throughout much of the winter to about 40 now.

Given the drop-off, we thought it would be useful to find a way to incentive people to continue participating through the end of the tournament.

We’ve therefore decided that, in addition to the other prizes we plan to award, at the end of the tournament we will give $5,000 to the person who saw the greatest increase in their portfolio value between February 15, 2023, 9:00AM ET and the end of the tournament. One will need to have bet at least S$1,000 during that time period to qualify and ended up with at least S$2,000 by the end of the tournament.

By making it a percentage increase, we hope to incentivize more participation from new users, and old betters who’ve been unlucky so far. It will obviously be easier to increase a smaller amount of money by a given percentage than a larger amount.

Here are the top 20 traders as of now.

The rest of this post discusses some of the markets that remain, with probabilities current as of the afternoon of February 15. More will be added as time goes on. Remember, to get updates on new markets and when markets settle, follow CSPI on Twitter and make sure to get alerts for new Tweets in order not to miss anything.

The Ukraine War

Russian forces continue to pound away at Bakhmut. The market gives a 30% chance that they will take the city by the end of February.

There has been talk of a Ukrainian offensive to take back more territory. Any substantive attack should focus on the city of Melitopol. The community believes that the Russians have a 77% chance of holding the city through the end of March.

Analysts have speculated that Russia’s war goals are now to take the entirety of the provinces of Donetsk and Luhansk. Ukrainian officials claim that Putin has ordered his military to do so by the end of March. That will mean controlling Kramatorsk, one of the last remaining major cities in the region not under Russian control. The market says that their forces have a 34% chance of taking it by the end of May. This seems significantly more bullish than much of the expert community, which looks at the slow Russian advances around Bakhmut and suggests that major breakthroughs in its favor are highly unlikely in the near term.

Two new markets just went up that in effect ask whether Russia or Ukraine will have a successful offensive by the end of July.

Will Russia control all the Donbas?

Will Ukraine take either Donetsk, Luhansk, Melitopol, Sievierodonetsk, or Mariupol?

In other markets ending in July, the forecasting tournament gives a 44% chance of Ukraine getting ATACMS, a 14% chance of a nuclear weapon being used, and a 30% chance of a ceasefire. The odds of a stop to the fighting have decreased over time, being down from around 50%-55% throughout much of the fall.

US Politics

There is a 60% chance that Donald Trump will be the favorite to be the Republican nominee in July. This is up from winter, when the number hovered around 30%. The market is expecting Ron DeSantis to get into the race, giving him a 62% chance of doing so by June 1.

On the Democratic side, there is a 79% chance Biden will be the favorite to be his party’s nominee by summer. As recently as November 2, the probability was around 40%, with the president receiving a major boost after the midterms.

The debt limit standoff is one of the main issues in Washington, with both sides seemingly unwilling to compromise. The market seems optimistic about finding a solution, with 77% thinking that the debt ceiling will be raised by the end of July. There is also a non-substantial chance of chaos, with a 33% probability that the House will take a no-confidence vote on Kevin McCarthy.

With the recent news that Labor Secretary Marty Walsh is leaving to become head of the NHL players union, the question of whether a Biden cabinet official will be out by summer is at 89%.

World Politics and Economics

Here are the numbers for some other geopolitical events covered in the tournament:

Bolsonaro gets kicked out of the US: 31%

Any country recognizes the Taliban: 30%

Chinese take military action against Taiwan: 15%

Peter Obi elected president of Nigeria: 42%

And on economic forecasting:

US achieves at least 1% growth in 2023 Q1: 71%

US achieves at least 3% growth in 2023 Q1: 35%

World GDP Growth at least 3% in 2022: 51%

China at least 4% growth in 2022: 14%

We also have a new market on whether the US will achieve at least 1% growth in the second quarter of this year.

Other markets include whether Trump will be indicted for a crime, whether he’ll come back to Twitter, and whether Elon will pick a new CEO for the company.

As a reminder, in addition to the $5,000 going to the top performer from this point on, the winner of the tournament will get a $25,000 fellowship at the University of Texas, among other prizes for top forecasters. See here for details, including on the need to take our survey to qualify, and good luck.

Make sure to also follow the Salem Center account on Manifold Markets. This will give you an alert on the website whenever a new market is posted. We will also be posting new questions on the CSPI twitter account. Since we started, we’ve added two Trump indictment markets. We’ll also occasionally send out e-mails promoting recent additions.

Here are the markets that have drawn the most interest so far.

Despite our initial success, a few problems have popped up and we’ve decided to make some adjustments.

We dislike making changes to the rules, but it’s clear that this is going to have to be an iterative process, given the novelty of the tournament and what we are hoping to accomplish.

A commenter pointed to this helpful paper on some problems forecasting tournaments have as a general matter. Fiddling with the rules is a necessary evil, and we may have to do it again at some point. This is something we are going to keep being transparent about. Keep in mind that our new changes – and this will be true for any other adjustments going forward – have the goal of trying to make people reveal their true estimates of what they think is going to happen rather than strategize on how best to game the market. Those who simply bet what they truly think throughout the process will be at an advantage, and that advantage should grow over time as we figure things out.

For now, we are making the following four changes.

1. Subtracting first day winnings from the leaderboard. The first problem we saw was that there were some individuals who made a killing by taking advantage of those who did not know how the markets work (see discussion here). For example, someone came in and put all their money on China attacking Taiwan, driving it to 99%. Another participant came along and was able to take all their money. Something similar happened in the Harvard affirmative action market. The leaderboard thus came to be dominated by those who were lucky enough to have profited off of ignorance on the first day. To solve this problem, we’ve decided to subtract first day earnings from the leaderboard. So if you started out with S$1,000 and were at S$2,000 on midnight the day after launch, you’re getting S$1,000 subtracted from whatever your portfolio is now. You’re probably not hurt, and may still be at an advantage because you have more money to distribute how you see fit. It’s important to emphasize that those who made large profits on the first day did nothing wrong. For the sake of the integrity of the tournament, however, and to be fair to everyone else, we think that this is a necessary change. This problem is unlikely to happen again, as the first day was unique in bringing in a surge of inexperienced users into low liquidity markets. In the future, when a new market pops up, an alert will go out to those with experience (again, people who follow the Salem Center account on Manifold Markets and check the website), which means that we should avoid seeing wild fluctuations in price.

   

2. Removing profits from the leaderboard. Some comments noted that the market incentivizes people to take long shot bets. Since being at the median doesn’t get you anything, for example, someone at the middle of the distribution should rationally take big risks in order to try to move towards the top without any fear of sinking to the bottom. This problem is quite hard to completely solve given that people are not betting their own money. What we can do, however, is limit the incentivizes to engage in risky behavior among those who are in the running for prizes. Under the old leaderboard, you knew exactly how much profit you needed to make it into the top 5 or top 20. For someone ranked 6th near the end of the tournament, it matters whether he is slightly behind 5 or way behind 5, which would imply different strategies. We have tried to deal with this problem by removing profits from the leaderboard. In other words, you can still see who the top twenty traders are, but you can’t see how wealthy they are in Salem bucks. Individuals knowing exactly what they need to do to get in the top tier leads to strategic betting, whereas the goal is to get users to bet according to their true beliefs. People not having clear insight into whether they are in the top 1%, 5%, or 10% also helps for similar reasons. Hopefully, this means anyone with a positive balance will be encouraged to bet based on their true estimates of the likelihood of events happening. We understand that this makes the leaderboard a bit less fun, but it’s probably the best thing for the tournament. We could revert to the old format later if we think the new one takes too much away from the user experience, and just live with the consequences.

3. Change in the way liquidity works. Part of the reason markets could shift so quickly early on was because they started with the default Manifold Markets setting of S$100 in liquidity. From now on, we will begin each market with the same default liquidity amount at a price we set ourselves. After 24 hours, we will add $1,900 in liquidity to stop major fluctuations in market prices, based on the idea that one person should not be able to singlehandedly cause extreme movements. This is close to what we did with the initial questions.

4. Taking into account betting behavior at the end. At the start of the tournament, we said that the top 5 leaders at the end would be judged in terms of things like academic accomplishments and business success. We are now also going to be watching for how they ended up at the top of the leaderboard. Was it a wide range of prudent bets over time, or was their success the result of a few large gambles? This will matter. We may also divide the tournament into different time periods, and factor in consistency.

Going forward, we would like to emphasize that limit orders are your friend! By placing them, you help yourself and help the market. It’s good for you because you can take advantage of other people’s foolish mistakes, and good for the market because it prevents wild fluctuations that people can take advantage of.

So let’s say that there is currently a 10% chance China strikes Taiwan according to the market. In a world with no limit orders, one foolish individual can come into the market, place everything they have on YES, and single handedly shift the probability to, let’s say, 45%. The next person who happens to come along at the right time can by pure luck make a killing by starting to buy NOs at the bargain price of 55 cents before the market shifts back. This is what happened on the first day, and we don’t anticipate it being a major problem again, for reasons mentioned above. Yet it is still something of a problem, as lesser, though still significant, irrational shifts are still theoretically possible based on the behavior of one or two people who do not understand how the markets work.

With limit orders, however, you can have people who foresaw such a thing happening, and in the case of the Taiwan question they might have placed orders to buy NO when YES hits 80 cents. Thus, the one fool cannot singlehandedly shift the market in an extreme way, and YES tops off at 20. Some people are still able to take advantage of the fool, but it’s not enough to give anyone a jackpot payout that significantly affects the outcome of the tournament.

If you have any other complaints or concerns, please don’t hesitate to reach out to us or leave comments. The feedback we have received so far has helped improve the market, and we will keep trying to make it better as time goes on.

[UPDATE, 11/17/22: Please see another update here.]

Critiques of individual experts or schools of thought have always been with us. In recent years, however, we have seen more of a frontal attack on the concept of “expertise” itself. This trend has been accelerated by the experience of the covid-19 pandemic, in which smart people across the political spectrum – conservatives, liberals, and libertarians – started paying close attention to what the public health community was doing and came away shocked by what they saw.

The argument against expertise, particularly the kind of expertise validated and legitimized by university credentials, goes something like this. Individuals succeed in business because they provide goods and services people want at the prices they are willing to pay. In academia, however, people are hired with other people’s money. There is often no objective measure of whether one’s research is any good, and individuals that do the hiring suffer no direct consequences from making bad decisions. In sum, academics become academics by writing things that people who are already academics like and agree with. This is basically the definition of a self-licking ice cream cone. When Phil Tetlock did research on whether experts were any better at forecasting events than educated laymen, he found little to indicate that they were.

The question of why expertise has gone so far off the rails has gotten a lot of attention. Diagnosing the problem of fake expertise and understanding its causes, however, are easier than providing solutions.

CSPI, in partnership with Manifold Markets and the Salem Center, wants to begin the process of trying to find one. We are announcing a program that we hope will be the first step in putting expertise on a sounder footing, by creating incentives for individuals to be correct rather than to rely on credentials, defer to authority, or conform to established ideas and ways of thinking.

The Salem Center at the University of Texas is hiring a new research fellow for the 2023-2024 academic year. This position will pay $25,000 and not require teaching or in-person residency. Rather, it will provide an academic job and financial support for a researcher to do whatever they want with their time, in order to advance their career or work on other projects.

Unlike a typical fellowship, you will not apply to this one by sending us letters of recommendation and a CV listing all of your publications and awards, and then relying on our subjective judgements about other people’s subjective judgments about your work. Rather, you will participate in a forecasting tournament on economics and social and political issues. At the end of the process, we will interview the top five finalists and decide among them. There will be a presumption towards hiring the best forecaster, but that presumption can be overcome based on how close the runner ups are and other criteria including any previous writing and business success. That’s it. You can be a high school dropout, and if you can predict what’s going to happen better than PhDs in their field, we’ll gladly make you a finalist for the job. You don’t need to share our politics, our social values, or our idea of what a scholar should look or act like.

Here’s how it works. You go to this page on Manifold Markets. After signing up, go take this survey, which asks about your name, basic demographic information, and political views for research purposes. Make sure to list your real name and username on Manifold Markets, so we can connect them to the survey results. Your answers will be kept confidential, and used only for the purposes of future research on predictors of forecasting ability among individuals.

When you sign up to play, you will get 1,000 of what we call Salem bucks (S$1,000) to distribute across markets as you wish. For example, one question asks whether Republicans will take the House of Representatives in 2022. We’ve set the price at 88 cents, corresponding to an 88% chance. Buy “yes” or “no” based on what you think will happen. You can buy based on the current price or place limit orders that depend on future price movements. For example, if you think there is an 85% chance Republicans will take the House, you can order some shares at that price, and the system will automatically buy them for you if and when it falls to that level (alternatively, as of this writing you can just buy “no” at 12 cents, since you think “no” should be at 15 cents). Gains and losses will depend on the prices at which you buy and sell shares until each market settles, at which point there will be payouts to those who own winning shares, at S$1 per share. Earnings can be reinvested. Participants will be ranked by how many Salem bucks they have accumulated by the end of the tournament on July 31, 2023.

A public leaderboard, accessible through a tab on the Salem markets page, will show who the top twenty earners are at any particular moment. If you do not finish in the top 5 of the leaderboard, there is still a chance to get something out of the market. First of all, the top 20 winners will be invited to contribute an essay to our symposium on forecasting, explaining strategies that they use and any other thoughts on the topic. They will also receive an invitation to a virtual conference on forecasting to be held at UT-Austin, which will be attended by me, Robin Hanson, Bryan Caplan, and more names to be announced later. If there is enough interest, we may make it an in-person event.

We will also give the following awards.

1. Gold-Top 1% of forecasters

2. Silver-Top 5% of forecasters

3. Bronze-Top 10% of forecasters

If you are unfamiliar with prediction markets, here’s a test page filled with nonsense questions you can bet on with Hanania bucks ($H). Once you get the hang of it, come play on the real market.

We will publicly announce the award winners, and expect that the prizes will benefit each recipient’s CV or resumé. In order to be eligible for the fellowship or any of the other rewards, participants will have to spend at least S$50 on each of 20 different markets. This will prevent anyone from winning based on one or two lucky bets.

Eventually, we hope to use the data gathered from the tournament to learn about the science of forecasting as we build upon the work of Tetlock and others.

To summarize and make things as clear as possible, the rules are once again:

1. Everyone who signs up for the Salem Center/CSPI forecasting tournament will get 1,000 Salem Bucks (S$1,000). Click here for the market.

2. Anyone can create an account and bet, but you must take this Google survey within one week of signing up in order to be eligible for prizes. All other questions are optional, but you must leave your real name and Manifold Markets username. If you want to be anonymous regarding when we publicly disclose winners and the leaderboard, that’s fine, but we’ll need your name for security purposes. Survey results will be used for research purposes but otherwise kept completely confidential.

3. Rankings are determined by wealth in S$ as of the end of the night on July 31, 2023.

4. The prizes are as follows:

   A. The top 5 individuals will be the finalists for a UT-Austin fellowship for the 2023-2024 academic year, with a grant of $25,000. Final candidates will be judged based on factors such as traditional academic criteria, business success, and public writing, with an edge given to the top forecasters.

   B. The top 20 individuals will be invited to contribute to a symposium on forecasting and a conference on forecasting to take place during fall 2023.

   C. Public honors will be given to the top 1% (Gold), 5% (Silver), and 10% (Bronze) of forecasters.

5. In order to be eligible for prizes – the fellowship, top 20 status, or a gold, silver, or bronze award – a participant will have to spend at least S$50 on each of 20 different markets. Profiles will receive a checkmark when they have met that standard. You will be able to track your current and past bets over time, so there should not be any confusion regarding what you need to do to qualify.

6. You can practice here with Hanania bucks ($H). Nothing on that particular site counts, it is simply there for practice and so you can get an intuitive understanding of the Manifold Markets system. See here for the Manifold Markets FAQ. Once again, here is the real market on which the forecasting tournament will be based. Play there when you’re ready.

7. You have to have a Google account to participate on the Manifold Markets website, and also therefore to play in the CSPI/Salem Center tournament.

8. New questions will be added regularly. Sign up to the CSPI mailing list through Substack to get regular updates.

9. I reserve the right to disqualify anyone trying to “hack” the tournament or somehow play unfairly. I don’t anticipate this happening, but there may be ways to potentially game the system that we haven’t thought of.

If you have questions, please leave them in the comments, and I’ll try to answer. Good luck!