In ski-jumping competitions, athletes compete on their distance and their jumping style, where the latter is assessed by a jury consisting of 5 judges.

Professor Jan Bouwens (Accounting section, Amsterdam Business School), together with Christian Hofman  and  Christopher Lechner (both Munich School of Management) explored potential bias among judges in this sport using archival professional skiing data. Their paper Transparency and Biases in Subjective Performance Evaluation came up with interesting findings on favoritism in judging Olympic events. The authors also offer potential solutions to this problem.

Nationality bias

It was recently established that judges favor athletes from the same nation. This bias is amplified by the other judges, because even judges with a different nationality give an athlete exceptionally high style points when the athlete’s compatriot is a jury member. This nationality bias is even higher in the absence of spectators, as is common during the pandemic. In other words, spectators act as a deterrent to the nationality bias. While regulations discard the highest and lowest points awarded by the judges, nationality bias still affects the outcome of the contests. Roughly half of the athletes in a ski-jumping competition have an advantage due to the nationality bias. And in 6 out of 10 cases, the compatriot in the jury gives the highest style points. These findings beg the question whether ski-jumping competitions can be considered fair.

‘It appears that athletes don't have an equal chance of winning a ski-jumping competition’ says professor Hofmann. ‘Our analyses suggest that the nationality bias is at least 0.3 style points in favor of the athlete who has a compatriot in the jury. This bias can change almost 20% of all final rankings. It appears that judges are unable to suppress their nationality bias. This affects the chances of athletes winning or losing a contest. It shows favoritism has a big impact on the athlete’s chances of winning. Thus, fairness is not really present’.

Possible solution

One way to deal with this problem would be to prohibit a judge from giving points to athletes from their own country, since nationality bias cannot be prevented when they are present. An alternative solution would be to have a jury of more than 5 judges, a solution similar to the one made for figure skating competitions.

Publication

Transparency and Biases in Subjective Performance Evaluation, J. Bouwens, C. Hofmann, C. Lechner, January 1, 2022.
TRR 266 Accounting for Transparency Working Paper Series No. 72