Introducing SIERA: Part 5

A helpful comment in one of the previous articles alerted us that our park-adjustment method was not quite correct, so we have updated the formula accordingly for this article and in the glossary. The formula only changes slightly and the tests from the previous article are very close to the same as well. However, for the sake of transparency, we are highlighting this at the beginning of the article. The new formula:

SIERA = 6.145 – 16.986*(SO/PA) + 11.434*(BB/PA) – 1.858((GB-FB-PU)/PA) + 7.653*((SO/PA)^2) +/- 6.664*(((GB-FB-PU)/PA)^2) + 10.130*(SO/PA)*((GB-FB-PU)/PA) – 5.195*(BB/PA)*((GB-FB-PU)/PA)
where +/- is as before such that it is a negative sign when (GB-FB-PU)/PA is positive and vice versa.

Over the last several days, we have covered plenty of introductory ground with regards to the ERA estimator SIERA, from explaining its evolution to demonstrating the creation of its formula. In Part 4, we tested the metric in several different areas against other estimators and found that the goals set forth at the outset of this process were attained: SIERA proved better than any other estimator that appropriately treats HR/FB rates as luck at predicting park-adjusted ERA in the same season, and it proved better in its predictive nature of park-adjusted ERA in the subsequent season than any estimator, regardless of its HR/FB treatment.

The final piece of this puzzle involves a bit of statistical exploratory surgery in order to explain why it fared better than its competition. While some may suggest that SIERA represents a mere five-percent improvement on QERA, the reality of the matter is that the metric vastly improves our understanding of certain types of pitchers consistently undervalued or overvalued by the current crop of estimators. This concept may feel largely abstract, so our time will be spent on exemplifying what we mean through archetypal personifications.

Recall that this method of estimation corrects a few problems inherent elsewhere, such as inconsistent denominators or the inclusion of insignificant regression terms. SIERA uses (GB-FB-PU)/PA to keep ground-ball rate normalized by PA like walk and strikeout rates are, so pitchers who allow an abnormal percentage of balls to be put into play can be better estimated. Similarly, we can also better estimate the ERAs of pitchers capable of inducing a surplus of worm beaters, who are therefore able to erase the inevitable ground-ball singles with double plays. SIERA also better estimates ERA for pitchers who are vulnerable to home runs but above average in the preventing base runners department, limiting dinger damage.

In the coming paragraphs, you will be visited by three pitchers, representative of the archetypes that other estimators consistently miss, but are now captured correctly viewed through SIERA.

Joel Pineiro

Pineiro had a 3.56 park-adjusted ERA last season to go along with a 3.1 percent BB-rate (1.14 BB/9), 12.1 percent K-rate (4.42 K/9), and somewhere around a 61.3 GB% depending on the source. Consider the various ERA estimators and their attempts at approximating his runs allowed per nine frames:

ERA = 3.56
SIERA = 3.60
FIP = 3.27
xFIP = 3.76
QERA = 3.96
tRA = 3.42

What should stand out initially is that Pineiro is precisely the kind of pitcher that exacerbates the denominator problem discussed earlier. He allows 84 percent of hitters to put the ball into fair territory, compared to the league average of 73 percent. Therefore, having a high ground-ball rate is particularly important and his well, well above-average rate is very helpful. However, QERA only considers the average effect of ground balls on ERA and underestimates the importance for someone in Pineiro’s situation.

For instance, if Pineiro maintained his 3.1% BB-rate and 12.1% K-rate, but had a 40%/20%/40% breakdown of GB/LD/(FB+PU), then his SIERA would be 4.56 and his QERA would be 4.55. Instead, going from 40 percent to 61 percent ground balls gives Pineiro an extra boost, as that 21-percent difference is part of a much larger set of balls in play.

Using xFIP leads to another problem in that Pineiro walks so few batters that he has fewer runners on base in the rare occurrences that he does give up home runs. FIP and xFIP multiply home runs and expected home runs, respectively, by 13 and divide by innings pitched. That treats the effect of blasts as constant for all pitchers, but the damage for someone like Pineiro is largely counteracted by his lack of ducks on the pond. Pineiro does allow plenty of ground balls, though, which can lead to singles, but he allows enough of them to the point that he frequently doubles runners off, as he induced 29 double plays last year.

This is the kind of adjustment that SIERA makes, as it allows for a quadratic term on ground balls. Both QERA and xFIP are clearly high for the reasons above, but FIP is nearly as close as SIERA with Pineiro and is on the low side, for the reason that Pineiro only allowed 6.5 percent home runs per fly ball, below the league average. This explains why tRA does a little better than SIERA for same-year ERA predictions, as it attributes this performance aspect, which is largely luck-laden due to its inconsistency, to skill in addition to crediting him for his low 15.8 percent line-drive rate, which we know is also largely luck-driven. Along the same line of reasoning, this is why SIERA bests all others at predicting park-adjusted ERA in the following year, as it expects such lucky marks to regress.

Johan Santana

Santana is another example of a pitcher who is much more accurately estimated through SIERA. From 2004-09, his average ERA (not weighted, just a quick average of his six ERAs) was very low, but estimators perpetually overestimated what his ERA should have been. The estimators produced the following in that same span:

ERA = 2.93
SIERA = 3.03
FIP = 3.55
xFIP = 3.61
QERA = 3.19
tRA = 3.15

The real issue is that FIP and xFIP are too bearish on home runs, neglecting to realize that his prowess when it comes to whiffing and walking mitigate the results even if he lacks the ground-balling tendencies as other star pitchers; as with Pineiro, fewer baserunners lead to fewer multi-run blasts. QERA is not specific enough about the interactions to properly nail down the ERA estimation, and by correcting for the shortcomings discussed in this paragraph, Santana exemplifies the impact on estimation SIERA brings to the table.

To see this, let’s look in more detail at Santana’s actual home runs from 2004-09. He gave up 146 blasts, but 72 percent of them were solo shots, while the league average was 57 percent. Santana allowed 1.33 runs on the average home run, 17 percent lower than the league’s 1.59 runs on the average homer! Had the coefficient on home runs in FIP been 17 percent lower, his FIP would have drop by 0.24, making up nearly half of the difference.

Brandon Webb

Webb is known for his extreme ground-balling ways, and since SIERA’s ground-ball percent terms have a negative quadratic on ERA, the more grounders induced, the greater the impact on run prevention. This is unlike others, which treat grounders as constant or offer diminishing returns as the rates get higher. The rationale is that ground balls lead to a lot of singles, but those singles can be erased by double plays. Consider the following chart, showing Webb’s 2004-08 numbers:

ERA = 3.01
SIERA = 3.18
FIP = 3.45
xFIP = 3.39
QERA = 3.63
tRA = 3.61

Webb’s SIERA is far closer to his ERA during 2004-2008. Those double plays were a major reason why this happened. Webb got 0.96 DP/9 IP, while the league averaged 0.81. This is more impressive, considering his WHIP was 1.24 to the league’s 1.40. His ratio of double plays-to-base-runners was .0860, well above the league’s .0646. Webb was somewhat vulnerable to singles due to his ground-balling ways. He allowed singles on 78.2 percent of hits in play compared to the league average of 74.7 percent, but he was less vulnerable to extra-base hits and was able to erase those runners back off the basepaths with double plays.

This last point is important, because SIERA adjusts for a common criticism of DIPS metrics, since it is based on a regression output. Many times you'll hear about DIPS for the first time and wonder if certain pitchers are more prone to doubles and triples on balls in play, even if they do not allow any more hits. As the pitchers who fit this criteria would presumably not be ground-ball pitchers (since ground balls have lower ISO on balls in play than fly balls do), a regression analysis will credit ground-ball pitchers for this double- and triple-prevention skill by giving them lower ERAs if fellow ground-ball pitchers have had lower ERAs for this reason, too.

Conclusion

Pineiro, Santana, and Webb are obvious examples of where the benefits are for using SIERA to estimate ERA. Each of them saw other estimators inaccurately estimate their ERAs. Pitchers like these demonstrate the reason why SIERA predicted next-year ERA better than any of the other available estimators. Developing better baseball statistics is not an academic exercise. It is a way to better value what happens on the field, allowing analysts to better understand and predict performance and front offices to build better teams.