The predictions were well-calibrated for the group stage. There were 48 games with three possible outcomes each. I took all 144 predictions and bucketed them as 0%-10%, 10%-20%, etc., and took the average prediction on the x-axis and the percentage that came true on the y-axis. The closer to the 45° line the better the calibration as that means the predictions occurred at the rate I predicted. And, this chart looks good! It is not surprising, because in aggregate, the average probability for favored team was 57% and they won 58% of games and the average probability for draw was 24% and they drew 19% of games. Thus, the draws were slightly over-forecasted and the underdogs winning was slightly under-forecasted.

I did not report binary outcomes for the group stage, but I did have predictions for which teams will get out of the group stage. This is not directly the same, but it is very close. And, they looked great as well. The highest likelihood team took the top seed in 6 of 8 groups. The second highest likelihood team took the second spot in 3 of 8 groups. Additionally, I had Netherlands as a second highest likelihood and they took the top spot in their group. Thus, 10 of 16 teams were favored with 9 of 16 teams favored in their position.*

As a comparison, Goldman Sachs had 4 of the top seeds and 1 of the second seeds. I did not capture all other predictions, but this record is higher than any of the other forecasts I followed.

Updating Predictions: likelihood of any game and likelihood of any team reaching any round. Details on method and full coverage.

*I updated my math: 6 + 3 + 1 = 10 …