Introducing the MOU Index: A new way of assessing managers

As data analytics advances in football, we can essentially analyse every aspect of a player's game. We can look at every touch, every decision, every moment and scrutinize over whether it was correct. We can aggregate these statistics to develop metrics of player efficiency, effectiveness, danger. We can look at telemetric data to gauge their speed, their effort, their physical ability. In truth, through data we are reaching a point where we can know players almost as well as they know themselves.

But we assume that these players exist in a vacuum. We look at everything they do, but still we struggle to appropriately examine their environments. Not only their connections with their fellow players, but the very systems in which they find themselves. And those systems and environments exist in a suit or a tracksuit or (more recently) a designer cardigan as a data blind spot; the football manager.

This is not to say, of course, that efforts are not made to look at football manager performance. We look at their points per game, their results, their pedigree, their philosophy, and the statistics generated by their players. But the analysis surrounding managers – how we compare them and how we judge their success – falls far too often into the realm of art rather than science. It’s about “fit”, “culture”, and “style” instead of the same level of scrutiny that we use for new player signings.

And when we do try to compare managers or look for a new manager for our clubs, we often fall into using statistics that don’t quite reflect the manager themselves. If we look at their points per game or results, how much of that is down to the manager versus the players that they find at their disposal? As we have seen this season, for example, how much can we attribute the success of a team like Liverpool to Arne Slot versus the squad that his predecessor Jurgen Klopp assembled?

To answer this question, I have developed a new metric for assessing managerial performance: The Manager Over and Underperformance (MOU) Index. What the MOU index seeks to do is to look at how managers are able to improve the teams that they have at their disposal. It leverages the wealth of available player data to make a prediction for how each player should perform in a given season. This allows me to control for the level of squad at a manager’s disposal. Is Pep really only successful because he spends the most money and has the best players? Are there any managers that are overlooked in world football because they’re at smaller teams?

I’m able to get a prediction with an R^2 of approximately 0.75, which means I’m explaining about 75% of the difference in player performance with my prediction. That includes things like their salaries, their minutes played, their position, the league they play in, their age; all characteristics of the player that the manager doesn’t really decide. So that means that the remaining 25% of performance that we see from players has to include everything else that isn’t contained in those variables. My contention is that those things, those soft variables like confidence, form, comfort, their connection with their teammates, their instructions, are down mainly to the manager. Of course, there are factors that are not down to the manager. If a player has just broken up with their partner their form may drop, and that is (hopefully) not the fault of the manager. But when we start aggregating these players at the position/squad level, those factors will come out in the wash. If we start to see that under manager A central defenders seem to consistently outperform their prediction, we can begin to conclude that manager A might be very good at improving central defenders.

So that’s how I set up the index. For the mathematically inclined the two equations that are used to find this exact score can be found in the paper itself (https://www.sloansportsconference.com/research-papers/manager-over-and-underachievement-mou-index-ranking-soccer-managers), but they aren’t necessary to understand what I’m doing here. It essentially describes a process as follows:

1.       Choose a statistic that you want to examine

2.       Predict how players in a team will perform at this statistic before a season starts

3.       Observe how they actually perform

4.       The percentage difference is your manager effect

For the main MOU index, I used the statistic of DAVIES developed by Mike Imburgio and American Soccer Analysis. It’s a personal favourite of mine that I use extensively in Zone 14, and I contend it’s the best existing statistic to judge player performance. It isn’t perfect, but it’s very good. To hear more about it I would recommend checking out ASA’s article introducing it:

So, using this equation we can develop a MOU Index. To date it has been made for every manager in the Top 5 European Men’s Leagues over the last 8 years (as far back as DAVIES goes). That means a ranking for 241 distinct managers, with some managers having just one season in the index while others have as many as the full eight. For more information on the specifics of how I make the index I would recommend reading the full paper, but here I’d like to focus a little bit on the more interesting question of what we can do with the index.

This boils down to 3 major things:

1.       Rank managers on their overall ability and their ability to improve certain position groups

2.       Directly compare managers

3.       Look multidimensionally at manager performance to find managers who excel in a certain style

Let’s look at examples of each of these 3.

Ranking Managers

This is the big question when it comes to managers: who’s the best, and who’s the worst? Well, here are the top and bottom 10 from my index:

	Manager	MOU		Manager	MOU
1	Jurgen Klopp	0.350	232	Frank Kramer	-0.050
2	Pep Guardiola	0.321	233	Steve Bruce	-0.050
3	Gian Piero Gasperini	0.286	234	Jan Siewert	-0.051
4	Simone Inzaghi	0.190	235	Stefan Leitl	-0.055
5	Stefano Pioli	0.165	236	Sergio	-0.057
6	Julian Nagelsmann	0.163	237	Olivier Pantaloni	-0.058
7	Roberto de Zerbi	0.084	238	Christophe Pelissier	-0.067
8	Bruno Genesio	0.069	239	Pepe Bordalas	-0.069
9	Adi Hutter	0.068	240	Christian Streich	-0.129
10	Juan Carlos Unzue	0.059	241	Sean Dyche	-0.190

At the top I doubt many people will be surprised to see the names Jurgen Klopp and Pep Guardiola. While it’s common knowledge that Pep and Klopp are the most successful managers in recent years, it’s interesting to see that Pep holds his advantage even when you control for the level of players at his disposal. Not only do those players perform well for him, but the evidence suggests that he makes them better. Rounding out our top 3 is Atalanta’s Gian Piero Gasperini, which appears surprising until you consider just how well he has made players play in his tenure. In the past two seasons that has been players like former Fulham and Everton forward Ademola Lookman, but before that he was able to get a fantastic tune from underrated players such as Josep Ilicic taking them to back to back 3rd place finishes in Serie A. It appears that the index does truly capture the managers who are best at improving their squad.

On the other hand, it is a bit of a surprise to see former Everton and Burnley manager Sean Dyche at the bottom of the list. This is partly due to the statistic we used in DAVIES favouring attacking play, and Sean Dyche being a traditionally defensive manager. But equally, upon further inspection, it makes sense that these players are not excelling statistically under Dyche. He focuses on cohesion and team results over the individual, so they may statistically underperform in his system but still achieve enough results in their defensive setup to keep themselves away from relegation. If you are looking to maximize the DAVIES of your players, Sean Dyche might not be your man. But if you pick an alternative statistic or metric, he may appear to be a much stronger option.

Comparing Two Managers

To illustrate how we can compare two managers directly, let’s take a closer look at the differences between Pep Guardiola and Jurgen Klopp by position.

In keeping with the values we see in the manager ranking, both of these managers are excellent at improving their players performance across the board. That being said, it’s clear to see that each manager has their position group strengths and weaknesses. Dribblers and, unsurprisingly, wide defenders appear to perform their best under Klopp as compared to Pep. Pep, however, dominates the midfield positions with much better MOU values for both deep midfielders and midfielders. This is in keeping with what we observe when watching Manchester City and Liverpool, respectively. City’s success has been rooted in their commitment to a possession based passing game, focusing on moving the ball through midfield players such as Kevin de Bruyne, Rodrigo, and Ilkay Gundogan over the years often at the expense of wide defenders altogether. Their game is focused not on taking players on with dribbling but instead beating them with incisive passing. Liverpool, on the other hand, have found success through their great passing full backs in Trent Alexander-Arnold and Andy Robertson, as well as through their mercurial and direct forward players such as Luis Diaz, Sadio Mane, and Mohamed Salah. Overall the choice between these two managers is close enough that the choice between them comes down to individual style. If you are looking to hire a manager to bring out the best in your midfield players, Pep would be the smart choice. If instead you look to take advantage of attacking players who travel with the ball at their feet, Klopp would make more sense for your squad.

Finding the best manager by attributes

Filtering managers on the basis of their strengths is another task that is easily accomplished by the MOU Index. If you are looking to build a squad, or have a squad already and are looking for the right manager to lead it, you might have specific position groups that are prioritized over others. Logically, this can be accomplished by creating a convex combination of any of these MOU values to generate your own ranking of managers. Creators might be especially important, so they receive a weight of 0.4, followed by central defenders with a 0.2 and so on with the coefficients summing to 1. This can be done for any combination of the position groups in the data, but is most simply shown using two variables. Let’s say you have a strong defence already, and are looking to improve your attack. You’re not too fussed about flair players, but want to make sure your manager will bring out the best in your creators and your finishers. We can plot that quite easily, and observe the outliers:

Yet again, the top 5 overperforms the herd here. Pep, Klopp, and Gasperini are exceptional in improving their creators and their finishers. As is true in most of these comparison plots, most managers appears to be pretty interchangeable, but some break away from the herd. This allows you to easily see who excels in both dimensions (or just one, in the case of someone like Nuno Espirito Santo) without needing to trawl through each manager one by one or watch hundreds of hours of film. All of this can be accomplished in a handful of statistics.

There is plenty of work still to be done on the MOU Index from expanding to new leagues, new statistics, relaxing assumptions, and adding in women’s football. As a first iteration, however, it represents a new and unique way of looking at football by allowing us to compare and contrast managers directly. It will allow teams and individuals to use a more analytical approach to manager hiring and firing, thus hopefully making manager markets that little bit more efficient and giving the teams that use it appropriately a leg up on their opponents.

If you want to have a go yourself and see all manager rankings, compare managers, and create scatterplots around position groups, feel free to explore this interactive tool: https://gferridge.shinyapps.io/mou_shiny/