Overcoming the Biases: Who was Roma’s Most Cost Effective Player in 2017/18

“[Players] are overlooked for a variety of biased reasons and perceived flaws.” -Moneyball

It is rare that an idea, scene, or line in a movie hits the nail on the head so squarely that it gets remembered for years, influencing discussions of topics far outside the context of the original setting. For me, the scene where Jonah Hill’s character, Peter Brand, delivers the “breaking biases” monologue is one of those special cinematic moments. The scene is both logical and emotional, and it leaves a deep, lasting impression of its central point—that people are really bad at making unbiased performance evaluations using just their own two eyes.

People experience primacy and recency effects where the first and final performances are weighted more than those in the middle, availability bias effects where peak highs and rock-bottom lows are more influential than average performances, with more memorable events being most impactful (e.g. beautiful goals), and of course people experience the confirmation bias where we interpret new information as confirming our prior beliefs, even when they don’t.

These biases are especially problematic because they are invisible to us. That’s what makes them a bias. But, worse, even when the issue of biases is raised, people think that others are more affected by biases than themselves. People are both bad at being objective and also generally unaware of it.

As Roma fans, we saw these biases front and center when Roma was dismissed as “lucky” after deservedly advancing past a quality Barcelona side to get to the Champions League semi-finals. A few ESPN talk show hosts insisted that Roma basically drunkenly stumbled past the reigning English, Ukrainian, and soon-to-be Spanish champions on the way to semi’s. Dan Thomas and Alejandro whatever-the-hell-his-name-is—who are both impossible to see too little of—didn’t expect Roma to get as far as they did, so they came up with every rationalization they could to maintain their prior view of the team. Confirmation bias, indeed.

Of course, that is not to say that we can be sure data analytics solve all these problems. Especially today. Soccer data analytics is still in its infancy, so, the most effective strategy for understanding player performance tends to be combining data with the insight from a knowledgeable and experienced viewer.

Nevertheless, it can be interesting and insightful to strip away all human judgment and look at just what the numbers say. It’s not entirely possible because numbers are selected and interpreted by people, but I’ll do my best here to let the numbers tell us how effective and cost effective the players were this season.

So...why cost effectiveness? It is a sad truth that, despite the Champions League success this year and the subsequent influx of money to the club’s coffers, Roma cannot yet financially compete with other top clubs in Italy, let alone Europe. So, like in Moneyball, we need to find value that other teams are overlooking in order to field a stronger team with less money. Therefore, this analysis could give us some insight into how Monchi and Pallotta are thinking about the relative value of players going forward.

We start with the minutes, salary, and combined Serie A and Champions League player ratings from whoscored. Whoscored developed an objective rating system that weights the relative importance of certain actions for a team (e.g. pass completion percentage) and then uses that weight to automatically create a game-by-game performance score based on the actions of each player, using hundreds of metrics from each game.

What is cost effectiveness? This is how much a player contributed to the team’s performance divided by the cost of that player. But, how do we calculate the relative effectiveness of a player? In this case, we want a value that takes into account the overall performance of a player weighted by how much time they actually played. For example, while Skorupski was extremely effective for the 90 minutes he played, it wouldn’t make sense to treat his 90 minutes on the pitch as equal to Alisson’s 4410 minutes. Clearly, Alisson had a larger contribution to the team over the season. Therefore, the equation for cost effectiveness is (Player Rating * Minutes) / Salary.

One other issue had to be addressed: Because contribution is just Rating * Minutes, the results would be biased by the relative size of the scales for each variable. Because Rating only goes from 6.24 to 7.63, but minutes ranges from 82 to 4410, the “contribution” would be overwhelmed by the number of minutes, where a player with more time on the pitch would be almost guaranteed to have a higher contribution rating regardless of the performance rating. Therefore, to give each of the variables an equal weight, I standardized the performance rating, minutes, and salary using a method called “z-scoring” that puts all the information on the same scale. Finally, I dropped any player that didn’t play at least 180 minutes to keep things simpler.

What do we find?

Kolarov played the most minutes of any outfielder AND also had the highest rating of everyone except for Skorupski, who only played one match. Let’s just take a moment to reflect on how ridiculous those numbers are...

Hold on. A moment longer...

Okay, let’s continue.

With no natural, healthy left back behind him in the depth chart for most of the season, we couldn’t afford to rotate him, and, fortunately for us, his performances meant we didn’t have to. For once, Roma’s injury woes didn’t strike at the worst possible time, and we were actually very lucky that he was fit for basically the entire season. Combine all this with the fact that his salary is average for Roma, and you end up with a runaway leader for the most cost effective player of the season. Let’s hope he can maintain this form next season, but regardless, it would do us a world of good to find a suitable backup to ease the pressure on him.

Alisson was the best keeper in the league this year, according to whoscored, and yet his rating was only 7.0. This is because keepers cannot earn points as easily as players in other positions, where top performers averaged around 7.5 for the season. If the player rating was equal across positions and Alisson got a 7.5 performance rating, his cost effectiveness rating would be around 11.5, making him far and away the most cost effective player. Nevertheless, the ratings are what they are, and we can, ironically, only hope that Alisson is even less cost effective next year because it would mean (a) that he is still here and (b) that he signed a new, more lucrative contract that could keep him here for years to come.

Another interesting detail revealed here is the surprising weakness of Roma’s midfield this season. Only one midfielder, Pellegrini, was in the top ten most cost effective players for the team. Worse than that, Nainggolan was the only midfielder in the top ten highest rated players, period. Together, this suggests that our midfield is expensive relative to the rest of the team and at the same time generally underperforming. This is especially surprising because Roma has been boasting, arguably, the best midfield in the league for quite some time. If these whoscored ratings correlate with Roma’s internal ratings of player performances, this is particularly bad news for Strootman and Nainggolan. With their high salaries and declining performances this season, don’t be surprised if we see one of them or both on their way out this summer to clear space on the pitch and in the budget in order to give younger, cheaper players a chance.

While it would be painful to see one or two first-team players leave, maybe a shakeup of the midfield this offseason is what we need most.