When we say there is a 75% likelihood of an outcome occurring, we mean that if the event were to happen 4 times, 3 times the outcome would occur and 1 time it would not. Brier scores, mean square errors, mean absolute errors, all have their place. But, errors are one thing and calibration is another. We are extremely proud of how we our likelihoods are calibrated and this year's NCAA tournament is no exception.
Below I have placed all 52 games into buckets based on the likelihood of victory for the team we predicted was most likely to win: 50-59.9, 60-69.9, etc., The x-axis is the average prediction in the bucket and the y-axis is the percent of games where the most likely team won. What the chart illustrates is that the predictions are nearly perfectly calibrated. Overall, the average probability of victory for the most likely team has been 73% and, of those 52 teams, 38 have gone on to win or precisely 73%.