OPS: Be Gone! - Part 2How accurate is OPS?By Tangotiger
Last week, I looked at 6 players of varying degrees of profile (high-walk, low-power to low-walk, high-power) who had an equal impact in run production when surrounded by typical teammates. We learned that to equalize these players using only OBA and SLG, we need to use the best-fit equation of 1.64*OBA+SLG.
After that article came I out, I received two very interesting questions.
Two great questions, and so, let's find out the answers.
Same OPS, widely differing OBA,SLG
The following table presents 6 players with the same OPS.
In order to construct a typical team, I have to adhere to the following constraints: each team will generate the same number of outs, and each player on the team will have the same number of PAs. Now, I'm counting outs as AB-H, though technically that is not true. You can also get outs on base. However, given that these teams are very similar to begin with, we'd expect the outs on base to even out. Here are the 6 teams that result from using the above players.
The first 5 players in that group are the most realistic looking of the players. We see that while they have the same OPS, they do not have the same run impact. In fact, there is an 12-run swing from one realistic end to the other. So, if you rely on OPS, realize that you will be off by up to +/- 6 runs. And this is for this kind of player. If you choose star players instead of average players, the swing will be even greater, perhaps double that. It's up to the reader to decide if this is an acceptable deviation within his objectives.
Now, let's look at other run measures as well. The following table presents our 6 players, with their expected PAs and outs to conform to the above guidelines (that is, this is how they would do if they each played in the same number of games).
In addition to the typical numbers, I present
Alright, so what do we have here? We know that BaseRuns is not a precise measure of run production for hitters (only for pitchers and teams). However, how accurate is BaseRuns? We know that static Linear Weights is not a precise measure of an individual hitter (nor for a pitcher nor a team). But, how accurate is it?
The above table shows that static Linear Weights is very accurate (a virtual match). This statement comes with a disclaimer. Because I chose 8 typical players to form my team, and because the static Linear Weights values are based on the historical typical team, we should expect it to be very accurate. If I would have chosen 8 typical 1960's hitters, we would not have achieved this level of accuracy using the static LWTS model. (We need custom LWTS values.)
Among the first 5 realistic hitters, we see that BaseRuns is accurate to within 1.5 runs. That is rather impressive, considering that the interaction effect occurs at the team level, and not the individual level.
Same OBA, same SLG, widely differing AVG
The following table presents 6 players with the same OBA, same SLG, but widely differing batting averages.
Essentially, as the walks and HR go up, I decrease the hits. In this way, I force the OBA and SLG to match, while varying the batting average.
We see that not considering the batting average in your OPS metric will have an effect of +/- 2 runs. We again see that Linear Weights, as expected, is an almost perfect match. BaseRuns comes to within 1 run of the true value
Conclusion
When using OPS, be aware of its accuracy and its limitations. As a back-of-the-envelope calculation, it works reasonably well. But, don't take it too far.
|