Mantic Monday: Judging April COVID Predictions

Feb 24, 2021

https://astralcodexten.substack.com/p/mantic-monday-judging-april-covid

 

Since this is getting broader than just Metaculus, I'm changing the name to Mantic Monday, after an obscure word for "oracular" (and changing the preview image to a mantis, since I don't know how else to visually represent "mantic". And posting it early Tuesday morning because I’m late).

In April 2020, I made my yearly predictions, and many of them were about the (then new) coronavirus pandemic.

Two other people on Less Wrong, Zvi and Bucky, decided to test themselves against me by trying to predict the same questions. Zvi saw my answers beforehand; Bucky didn’t. Here's how we did (except where otherwise stated, all predictions are for 12/31/20):

Black statements are those judged true, red statements false. The numbers on the left are our predictions, so for example I said there was a 60% chance that Bay Area lockdowns would extend beyond June 15.

You can see a list of the full questions and why I graded them the way I did in the appendix at the bottom.

I scored these using a logarthmic scoring rule, adjusted so that guessing 50-50 always gave zero points. It's not very intuitive. Getting everything maximally right gives a score of about 14; guessing 50-50 for everything gives a score of 0, getting everything maximally wrong gives a score of negative infinity.