Set restart margin: one of these things is not like the others – NRL Round 10 2020 stats and trends

Over the past few weeks on the Eye Test and on my personal twitter account, I’d been investigating that the fact there was a negative correlation between set restart differential and margin. It appeared that the top four teams were strategically giving away more restarts as a way of setting their defense.

On the weekend, the wonderful Jack Snape penned a tremendous article for the ABC expanding on that theory. If you’ve not read it, then please take some time, and have a read. It is very much worth your time.

It’s the type of quality analysis you don’t usually see in the mainstream media today and it deserves your attention. And I’m not just saying that because he was kind enough to link back to my post from Round 9.  

There are a few takeaways from the article, other than the initial theory that some clubs are strategically giving away set restarts. One is that the largest offenders, Penrith are giving away a significant number of their set restarts between their opponents 30-40m line, with Jarome Luai being a noted wrongdoer. The other is that the Eels and Roosters defend exceptionally well after a set restart, only conceding three tries in the 90 seconds after giving away a set restart.

One thing to note before we move ahead, especially for new readers is that there may be some minor differences in statistics between what I use and the ABC article. This is due to that article using data from the NRL, whilst I use Fox Sports statistics.

There are a few reasons I use Fox Sports over data, the first being that NRL Supercoach is my NRL fantasy drug of choice (you can find some of my ramblings at NRL Supecoach Talk) which uses Fox Sports stats, and they’re also the stats used during their broadcasts. The second is that they also have first and second half breakdowns on their site. And the third is that they are (currently) the only public source of set restart data available. We’re halfway through the season and the league’s official site still isn’t providing statistics around one of the biggest rule changes in years.

But I digress. Let’s have a look at the updated chart for Round 10.

Penrith still sit by themselves in the upper left quadrant with a -22 set restart differential and +84 margin. The Storm conceded a few more restarts in their big win over the Titans, whilst the Roosters picked one set restart up in loss win over Canberra. The other top four team, Parramatta moved from a -2 to a +2 in their loss to Manly.

The biggest change was the Tigers, who now have the second worst set restart differential in the NRL and have a better margin than the Eels since Round 3 thanks to their huge win over Brisbane. As mentioned last week, correlation isn’t causation and whilst teams are doing it strategically it’s not something that will win games alone. That said, there’s a need for context with this data, with two instances on the weekend highlighting why.

On Friday evening, the Tigers led the Broncos 26-0 at half time and held a 5-2 lead in set restarts awarded. This +3 set restart differential evaporated in the second half, with Brisbane given a number of late set restarts when the score was already 48-0. And the final tally for set restarts in that game? Tigers 5, Broncos 6. A similar situation happened in the second half of the Manly/Parramatta game with the Eels receiving a 3-1 set restart awarded count which aided their fightback before ultimately losing.

This will be recorded as a -1 set restart differential for the Tigers, with +48 for their margin. Yet by watching the game you will see that this was not a particular strategy by Wests and was mainly the result of the game being refereed by the most prolific caller of set restarts, Adam Gee (see below).

The other instance was the poster team for set restart differential winning a closer than anticipated contest against North Queensland 22-10. Regular readers will know that the Panthers have been on the negative end of set restart differential, and usually winning when they are. However, in this particular game, they came out ahead with a +1 for net set restarts, which seems unusual for them until you look and see that there was only one set restart awarded during the whole game. Again, this is not unusual considering that the official for the match was Ben Cummins, who is the stingiest when it comes to awarding set restarts (see below again).

The point here is that timing and volume is are extremely important context for set restart differential. If you look at the same chart above, but for total set restarts conceded plotted against net margin, you get a better picture of why volume matters. The further to the right, the more set restarts a team is conceding.

Penrith leads the NRL in set restarts conceded with 43, ahead of the Tigers in second place with 39. The Roosters, Eels and Storm all sit mid table with either 32 or 33 sets restarts conceded. Last week when all top four sides sat in the upper right quadrant, the Roosters, Eels and Storm combined had a combined differential of -10. No team had a differential lower than -5.

That means that those three teams faced a maximum of 60 extra tackles across seven games. Which is less than 10 extra play the balls per game to defend for all three teams. This would indicate they’re happy to concede when necessary, but it isn’t necessarily a large tactic in their arsenal.

Meanwhile Penrith, with -23, had faced a maximum additional 137 tackles across seven games. That equates to a maximum of 19.7 extra tackles per game that Penrith was defending. Giving away an extra three sets of six per game to less than one for each of the other top four teams is a significant difference and a sign that they are backing themselves to contain their opposition. The average NRL game has about 300 play the balls, and assuming that each side gets 150, an extra 19 tackles defended would be an increase of 13%.

I’d also like to note Wayne Bennett, the gentleman that he is, refusing to let his team be drawn into this gimmicked rule change.

Looping back, from the article on the ABC site last weekend, we now know where set restarts are being awarded, and whether or not teams are conceding a try shortly after. This leaves me with three wishes (nod if you’re reading Jack) to further prove or disprove this theory:

  1. Timing of set restarts. The Tigers/Broncos game is a perfect example of how looking at post match statistics only tells part of the story. With play by play logs we could see when set restarts were awarded, on which tackle and what the end result of those sets were.
  2. Margin when a set restart was awarded. One theory is that referees will subconsciously favour a side trailing significantly and award them more penalties and set restarts. By knowing the margin when a set restart is awarded, it will be easier to analyse whether they’re being used by teams to gain an advantage or maintain one.
  3. Average meters gained from a regular set compared with the average metres gained from a set restart. Based on my theory that teams are using set restarts to ensure their defensive line is set, you would expect that sets after a set restart is awarded average fewer metres than a normal set of six.

The last point was partially seen last week when I noted there was a small positive correlation with the number of runs under 7 metres with set restarts conceded, and a corresponding small negative correlation with the number of runs over 8 metres. This may be due to the fact that those giving away set restarts are the elite defensive teams (The Roosters and Eels) who have the confidence to endure the extra tackles to maintain field position.

That’s not too much to ask is it? It would easily prove that the consequence for a set restart is trivial for any team with a competent defense, which is already obvious to anyone watching the games and was theorised even before the rule was introduced. In the meantime, just remember that context is everything when using statistics.

A possible fix set restarts?

Given that we’ve established that the top teams are gaming the system, how do you fix this issue? The good news is that I think there’s a fairly simple solution.

Award a penalty.

Net penalties still have a positive correlation with margin since Round 3 (see below chart), which makes sense. They result in another set of six and also mean the defensive is giving up field position. Set restarts award another set of six tackles but allow the defensive side to fortify their line in favourable field position to slow momentum.

It would be unlikely to happen as it would undo the V’Landysball changes and prove there was zero to little thought put into said changes.

But rolling back the rules to award a penalty for consistent infringements would increase the consequences for defenders who at the moment are happy to give up another set of six to maintain field position. This would also be a great time to implement a five minute sin bin for repeated infringements, but let’s take some baby steps first before trying to ride up Mont Ventoux.

Penalty and referee trends

Round 10 is over and we’ve settled into a range for total penalties awarded during a game, although we did see the fewer set restarts awarded. It was the fewest number of set restarts called since Round 5, when coaches worked out how to use it to their advantage, with one being called every 22.6 play the balls. It was also highest number of penalties since Round 5. The breakdown is below

This drop in set restarts is likely attributed to Penrith, the team conceding the most set restarts, meeting Ben Cummins, who is the least likely to award a set restart. The last game of the round featuring the Panthers facing the Cowboys resulted in just one set restart, given to Penrith in the first half. Below is the average number of penalties and set restarts called per referee since Round 3.

Adam Gee awards was at it again in the Tigers v Broncos clash. He awarded seven set restarts in the first half, which sounds like a lot but was actually the fewest set restarts he’s awarded since Round 3. There have now been 14 games with seven or more set restarts since Round 3 (eight rounds), and Gee not only has six of them, but he has five of the top eight. The table of referees with seven or more set restarts since Round 3is below:

Set restarts by half, referee and round (Rounds 3-10, 2020)

The other thing to note is in every single one of those games, there were significantly fewer set restarts called in the second half. Grant Atkins had 9 in the first half on the weekend but zero in the second half of the Knights/Rabbitohs game. That’s a massive turn around in player behaviour…

On the other end of the scale is Cummins, who has called three or fewer set restarts in the first half five times and officiated four of the eight games with the least amount of set restarts. Which makes it no surprise that the Panthers/Cowboys game only had one set restart conceded in the entire game.

One of the reasons for bringing this up repeatedly is that games officiated by Gee are averaging over 49 points per game scored, whilst Cummins controlled games are only averaging 37.

This chart shows the average total points per game plotted against the average number of set restarts. There’s a very minor positive correlation with more set restarts in games with more scoring, but we’re looking at a low sample size with a maximum of eight games refereed by any one official.

But what about margin? Again, it’s a low sample size but there’s a negative correlation with margin of victory and number of set restarts given.

The negative correlation would have been even greater last week. Adam Gee had an average margin of 11.6 in games he controlled until this weekends 48 point Tigers/Broncos blowout, which pushed his average margin up to 16.1. Clearly one for the degenerate gamblers.

Error Rate

To finish off my look at Round 10, Error Rate is another statistic I’ve been working with this season to get an understanding of how often players are turning the ball over. What is Error Rate? I’m calculating it as the average number of possessions for every error committed by a player.

The average for the NRL this season is about 37 possessions per error, but that varies wildly by position. A hooker will make an error every 167 touches, while a winger will do so every 17. Here’s the top 20 for Round 10.

It’s probably not doing Craig Bellamy’s health any favours to see this list, as three Melbourne players sit in the top five, and four in the top eight. If you’ve watched any Storm games this season the names match up to the eye test (not the one you’re reading), especially Suliasi Vunivalu who always seems to manage a howler. As it turns out he commits an error every nine possessions.

Keegan Hipgrave’s disastrous game against the Storm on Friday has seen him rocket into the sixth spot, after not even placing inside the top 30 last round. There are 69 NRL players this season who have committed at least 9 errors. Of them, only two have handled the ball fewer than 100 times. One is Thomas Flegler (9 errors, 96 possessions), who also sits in the top 20. The other is Hipgrave, with 9 errors in just 82 possessions, which comes out to a rate of 9.1. Combined with his incredible ability to commit a penalty, he’s a walking liability for the Titans.

I’d also like to apologise to Jack Williams, who became something of a whipping boy on twitter on the previous times I’d posted this stat to twitter. It looked like Williams had been committing an error every seven possessions. It turns out I’d only been including the games where players committed an error, and his correct Error Rate is one every 9.3 possessions. Not significantly better but I’d like to be correct.