Rating performance in soccer is a tricky task. No matter what position you want to evaluate, player performance is to some degree dependent upon team and opponent performance. If a striker scores three goals, but missed three other point blank chances and his team loses 4-3, how good a performance was it really? Say one goalkeeper makes five top notch saves but is eventually beaten by a howitzer strike and loses 1-0 while his counterpart across the field makes two routine saves and keeps a shutout – who had the better performance? The truth is there’s no one answer to any such question that is gospel, but there are answers that prove correct more often over time.
We at Sportify are creating a fantasy soccer game where our main goal is to make sure the results accurately reflect performance while maintaining transparency in scoring. That’s why we’ve included advanced metrics related to expected goals (xG). For those unfamiliar with xG, there’s a detailed explanation of our system here. In a sentence, xG and specifically Shot Quality xG (sQxG), which is most relevant in this context, is a measure of level of expected goals specific to finishing, a metric that correlates much stronger with winning games and leagues than shots, shots on goal or any similar such box score metric.
With MLS showing off its top 10 goalkeepers per the Audi Player Index, we thought it would be fun to take a look under the hood of our fantasy scoring engine and compare the two.
Four players are on both lists: Tim Howard, David Bingham, Josh Saunders and Clint Irwin. And while API rankings are not available for the remaining six goalkeepers in our Top 10, we can show where the remaining six of theirs fit on ours.
The most noticeable difference is Audi’s top keeper, Portland’s Jake Gleeson, who ranks 19 in our fantasy scoring. No disrespect to the Timbers’ shot stopper, but we’re talking about a GK who has kept only five clean sheets in 24 games and has conceded 1.4 goals per 90 (14th out of 25 GKs with a minimum of 900 minutes played), solid numbers – but best in the league? We think not.
Our #1, Travis Worra, may be a controversial pick as well as he isn’t even really the #1 on his own team, but when the former UNH Wildcat was deputizing for the injured Bill Hamid, he more than held his own between the sticks for D.C. United. Worra kept clean sheets in four out of 12 games, conceding a total of 14 goals (1.2 per 90) while never once conceding a penalty or making an error that led directly to a scoring chance.
Diving a bit deeper into the advanced stats is where Worra really stands out. Worra averaged a pretty modest 2.8 saves per 90, but the quality of those saves translated to 0.7 sQxG saved per 90 (6th among qualifying GKs), and the average shot quality of goals he conceded was 0.6 sQxG (highest among qualifying GKs). This tells us that Worra was making quality saves in net, and when he was beaten, it was by high quality attempts. When Bill Hamid returned to full fitness, he resumed the starting job and continued D.C. United’s strong play between the posts; he comes in as the 6th highest rated GK according to our fantasy scoring.
While Worra has proven he can play at the MLS level, he isn’t the only backup who has made a strong bid for a starting spot elsewhere in MLS. Zac MacMath was the Colorado Rapids’ #1 until some guy named Tim Howard (#2 in both Top 10s, ever heard of him?) came to town signing a contract that’ll keep him there through 2019. MacMath, like Worra is nowhere to be found in the Audi Top 10, and while his 2.3 saves per 90 are towards the bottom of the pile, MacMath is the only qualifying GK who has allowed less than 1 goal per 90 minutes (0.8). Yes, there are intricacies in measuring GK performance, but at the end of the day a GKs #1 job is to keep the ball out of the back of his team’s net. Zac MacMath has done that better than anyone this season. One shouldn’t fault him for his defense not allowing more shots, but it appears the Audi Player Index has done just that.
The Audi Player Index seems to reward keepers who are active at the back with little regard for the number of opportunities afforded them or how well they do the most important aspect of their job. If you look at saves per 90, the top 7 GKs are all in Audi’s Top 10. Yes, it’s important for GKs to make saves, but making more saves AND conceding more goals doesn’t make you a better GK. Thus far in the MLS season, eight GKs have allowed over 1.5 goals per 90, five of those eight are in Audi’s Top 10 including Joe Bendik who has conceded a league high 1.9 goals per 90, good for another league high 57 goals conceded!
The core pieces that go into our GK fantasy scoring engine are goals conceded, sQxG of those goals, sQxG saved, good keeper actions, bad keeper actions and serious errors. If you’re interested in a more detailed explanation, you can find one here. Keep in mind, while the pieces we’ve used to build this system are statistically significant when it comes to GKs keeping the ball out of their net, this is a fantasy scoring system not a comprehensive player rating system. That, however, doesn’t mean our fantasy scoring system doesn’t do a better job of rating players than some of the dedicated systems out there.