Thursday, January 01, 2015
Are Defensive Runs Saved predictive?
?John Dewan makes this startling claim:
One of the public perceptions has been that a player needs three full seasons before his defensive metrics provide a true indication of his defensive abilities. That has been my own personal rule of thumb, though I’ve known there is some reliability to sample sizes smaller than three years.
Based on the new research, BIS has found that Defensive Runs Saved based on as small a sample size as 350 innings in the field (about a quarter of the season) produces reliable results. This is a very significant finding.
And by reliable, it looks like he's after r=.50 or greater, which is reasonable enough (more signal than noise), and so, closer to 0% regression toward the mean than 100%. But, I'm not buying it, not for a second.
I did a quick check by grabbing the data from Fangraphs, 2013-2014 data. I have the DRS, DRS/9Innings, and Innings. John makes the claim that he gets r=.55 after using only 350 Innings. Not a minimum of 350 innings, but exactly 350 innings.
If I use a MINIMUM of 350 innings over each of 2013 and 2014, at a particular position, I get 210 fielders. The AVERAGE innings in 2013 is 933 and in 2014 is 938, which is the equivalent of 104 full games. My correlation, if I use DRS/9innings is r=.48. If I use just DRS, it's r=.46.
If r=.48 when I use an average of 104 full games, then I'd get to r=.50 with an average of around 113 games (a regression amount of roughly 1000 innings).
When I used to do this with UZR several years back, I'd use a rule of thumb of r=.50 at games=100 (though obviously less for SS and more for 1B). The results I see here from DRS data is in the same ballpark, and so, I see no reason to change my opinion on that.
I have no idea what John did, since he says the details are in his forthcoming book. Maybe he meant a MINIMUM of 350 innings, and he (probably) separated by position (meaning he ran one correlation per position as opposed to what I did here for a sniff test), and he did use more years. So, all of that would match what I've done. So, if he meant a minimum of 350 innings (implying an average of around 900-1000 innings), that's fine, and it's consistent with what we've always known.
But if you had EXACTLY 350 innings (or taking the first 350 innings or a random set of 350 innings), the correlation would come in at around r=.26.
Recent comments
Older comments
Page 1 of 150 pages 1 2 3 > Last ›Complete Archive – By Category
Complete Archive – By Date
FORUM TOPICS
Jul 12 15:22 MarcelsApr 16 14:31 Pitch Count Estimators
Mar 12 16:30 Appendix to THE BOOK - THE GORY DETAILS
Jan 29 09:41 NFL Overtime Idea
Jan 22 14:48 Weighting Years for NFL Player Projections
Jan 21 09:18 positional runs in pythagenpat
Oct 20 15:57 DRS: FG vs. BB-Ref
Apr 12 09:43 What if baseball was like survivor? You are eliminated ...
Nov 24 09:57 Win Attribution to offense, pitching, and fielding at the game level (prototype method)
Jul 13 10:20 How to watch great past games without spoilers