Eurovision Sabermetrics: Critical Reaction

I'm fully aware that some may find this article a bit heavy. If there is anything you are unclear on please feel free to get in touch.


Over on EscInsight they have lanuched a considerable debate on their Eurovision Sabermetrics. The application of sabermetrics (normally seen in baseball) to Eurovision. This debate got going today with Elaine Dove's excellent article on ESCInsight. I aim to strengthen her argument by looking not at the emotion of Eurovision and its songs but by looking at how difficult to get Eurovision predictions right and discussing the limitations of how they went about figuring out Sabermetrics. We are not here to undermine their work but to give the health warning that comes with looking at these things. I would also state that I am discussing this under the principals that are learned in management, management science and technology and largely focus on arguments from a decision analytics. Different people may use different concepts and ideas to demonstrate similar things.

The first point is the limitations of computers. While it may seem all very well that we can input data into a programmable machine and come out with answers we cannot be completely sure that those answers are completely correct, particularly when we have to balance many factors as we do in Eurovision. In a documentary by the BBC All Watched Over By Machines Of Loving Grace (Episode 2), it discussed the way computers were perceived in the 1950's when it was thought that the whole world could be broken down into logical systems that could be analysed by computers, there by eliminating all of the randomness in the world. However they discovered that human behaviour itself was totally random and decided that computers could not do what they thought they could. Similarly in Eurovision we really don't know for sure how people will vote. As this is something that is a part of human behaviour we believe that it is ultimately random.

Smart people will point out that Sabermetrics work in baseball so why not Eurovision? Well, in my view Eurovision is to rare an event and too changeable/random (in terms of competing countries, song styles, combinations of countries and people musical tastes) that we can properly analyse it to the level that we can predict 10/10 songs. (I actually think that it might be possible but not without knowing all the songs.) Also Sabermetrics of a baseball team would be more constant over a shorter period of time as players stay the same. Countries in ESC don't actually send the same singer 3/4 times in a row and thereby comparisons are harder.

The second point is about how we make decisions. In something as complex as picking who will qualify for the Eurovision song contest we must weight up different aspects. As ESCInsight have not explained the weights used for their selections we assume that they are largely the same. The two areas the stated they focused on was voting history and relationships.

What are weights in decision making? Well imagine if you had just two pieces of data about a country qualifying. The running order position gave the song a 0.2 chance of qualification and the countries history gave it 0.4 chance. Put together it would give the country a 0.3 chance. However if you thought that running order was twice as important as history you would give them weights of 2 and 1 respectively which would actually only give the country a   0.26 chance. This is a VERY simiplistic model purely to demonstrate how this works. I believe that these weights are ever changing and cannot be fully determined. (I do accept we can get close to determining them).

There also is the problem about how we go about using historical voting patterns in preparing these statistics, some would say all historical data from, say, 2004 is useful. Others would say only after 2010 is useful because of the jury/public mix. Others could argue that data from more recent contests could be weighted more heavily than other contests, while still taking them into account.

Overall I think what is critically missing from ESCInsight's Sabermetrics is a description of how it was calculated and/or the resulting figures that came out of the program. I respect what they are trying to do but the primary aim of this piece is to raise the point that there are a HUGE number of factors (some of which I have pointed out) which determine how a song will do. There are a variety of ways with which these can be interrupted and how we come up with statistics for Eurovision is debateable. Most importantly: EUROVISION PREDICTIONS ARE AN ART NOT A SCIENCE!!!!