The Eye Test’s Most Adequate of 2020 – Why volume statistics aren’t always your friend

This article was originally posted as part of NRL Round 15 notes and trends, August 25, 2020.

This week should have been a fantastic matchup of halves, with Shaun Johnson of the Sharks facing up against the Panthers and Nathan Cleary. Cleary has been one of the standouts this season, whilst Johnson is having a great season, but you wouldn’t necessarily know it from the way he’s often covered by the mainstream media. I’d pointed it out previously with a radar chart comparison in early August:

Johnson ended up missing the game due to some minor injuries and the birth of his child (congratulations Shaun!) and Cleary had another strong showing. Despite this we can still have a look at their statistical output for the season and use it as a test case for counting stats and raw volume statistics not telling the full story.

If you’ve been following for me for any length of time you know that I’ve put together some advanced statistics for rugby league, as using raw numbers mean players who spend the whole 80 minutes on the field usually dominate. If you’ve not read them, I’d recommend checking out my articles on Run %, Tackle % and Involvement Rate on the website, as they’re all incredibly useful in identifying high

But back to the topic at hand. It’s lazy analysis to only use counting stats without context but is more palatable to the wider viewing audience so I’m not going to deride them for dishing up what the consumer wants and easier to digest in small doses.

Source: Fox Sports Stats

Comparison of raw numbers – Cleary seems ahead. More passes, more runs, more kicks, more attacking kicks, more tries, more try contributions, more line breaks and more line engagements. Johnson is only ahead in try assists (20 to 14) and weighted kicks (17 to 10).

Per game stats will give a slightly better comparison, although it paints a better picture for Cleary who has only played 12 games compared to Johnson’s 14.

When players are compared on the usual pregame shows, it’s assumed that all players in a certain position play a similar game or similar role within a team, leading to scorching hot takes like this on social media from mid-June:

And if you look at the raw volume or counting statistics at that time without any context you’d probably agree – Johnson isn’t impacting the game as much as Cleary is.

But there’s one variable that’s not usually discussed (although the wonderful Jason Oliver pointed it out in his Round 15 preview on SportsTechDaily, another must read each week), is possessions. The amount of times a player gets his hands on the ball plays a massive part in his statistical output. The basketball adage of “you can’t rebound the ball out of the basket” can be applied here with a twist, you can’t do more in attack in rugby league without the ball in your hands.

For the season Cleary has 964 possessions, compared to just 733 for Johnson, a difference of 231 possessions. On a per game basis, that’s roughly 73 possessions for Cleary and 52 for Johnson. Cleary has his hands on the ball nearly 30% more than Johnson on a per game basis.

Knowing this, what if we looked at the same statistics again for Cleary and Johnson on a per possession basis. Would it show anything?

Source: Fox Sports Stats

Not initially, as those numbers are essentially meaningless – 0.016 line break assists or 0.187 line breaks per possession isn’t really meaningful. You can’t create 20% of a line break.

Instead we’ll normalise it to take out any bias that having more possesions per game creates. I’m going to pick a set number of possessions in between both players to even things out. The number doesn’t matter so much as long as we use the same number for both, and for this exercise I’m going to use 60 per game, since it’s a nice round number that falls between both Cleary and Johnson’s season average.

Source: Fox Sports Stats

Now we can see that they’re not performing that differently. Cleary has an edge with kicking, especially long kicks, whilst Johnson leads on weighted kicks and try assists. Yet for all the calls that Johnson needs to run the ball more, his runs, passes and line engagements are very similar to Cleary’s. Does he really need to “do more”?

That leads into the other part of player this analysis their positioning and their role. As mentioned above it’s assumed that all #6s and all #7s should play identically but this is rarely the case.

This was shown in a great article by Jack Snape from the ABC showing the locations NRL halves are receiving the ball. If you apply the Eye Testtm during Sharks games you’d know that Johnson sticks primarily to the right side of the field, while Cleary tends to operate on both sides. The above article shows this, with the locations of Cleary’s touches coming evenly across the field whilst Johnson’s are mostly on the right. You could then argue that if Johnson had a similar level of freedom as Cleary, he would probably increase his raw statistics.

The other part of the role is not just what side they’re playing on but how often they’re involved and relied upon for their team. Cleary takes about 16% of the Panthers total possessions, makes 36% of their passes, 75% of their kicks, 66% of attacking kicks and 44% of line engagements. Johnson on the other hand, takes 12.6% of Sharks possessions, 29% of their kicks, only 52% of their kicks and 45% of attacking kicks.

This again ties back to role. Similar to a sport like Formula One, the Panthers play with a clearly defined lead half in Cleary, with Jarome Luai supporting him. The Sharks more often than not play with their halves on a closer to equal footing, a 1a/1b type scenario, with each either sticking to their side of the field or sharing in the playmaking duties.

Asking Johnson to “do more” or run the ball more won’t necessarily help his game. More stuff isn’t always better. Johnson will turn 30 in a few weeks and it could be that he doesn’t have the same explosiveness and has developed more as a player is now picking his spots and interjecting himself at the right time. Just because he’s not shredding teams anymore with one of his explosive runs and making defenders look as if their feet are stuck in cement doesn’t mean he’s not impacting games.

Whatever the reason, the outcome of this specific comparison of Cleary and Johnson is that they’re both having amazing seasons for their team, and the difference in their statistical output is simply down to the role they play for their side, which can be accounted for by looking at a per possession basis.

Claiming one is better based on one number or a group of counting statistics won’t prove anything other than teams and players are different and may play different styles. Hopefully we’ll see some more analysis based on possessions than just counting stats moving forward.

Cover image – “Nathan Cleary” by NAPARAZZI is licensed under CC BY-SA 2.0