Last week, Sports Illustrated writer and Jason Parks man-crush Tom Verducci put out his annual column warning about a specific type of player: A young pitcher (25 or younger) who saw a significant increase in his workload in the previous season over the season before that (defined as an increase of at least 30 innings, including postseason and minor-league work). Verducci claims that this sort of pitcher is in danger of either a significant injury and/or a performance decline in 2013 because his 2012 was much busier than his 2011. It's a proposition that's become known as the Verducci Effect.
Let's start with the positive: This is a perfectly reasonable hypothesis. It makes sense intuitively. If you are training to run a marathon, you don't go from maxing out at one mile to maxing out at 10 in a week. You have to build up your endurance along the way. Rick Peterson, who has been a pitching coach for several teams, has given the Verducci Effect his imprimatur. There have been several cases fitting the criteria in which pitchers have gone on to suffer major injuries or major declines in performance. We have face validity through imperfect analogy, expert testimony, and case examples. It's not silly to believe that Verducci's hypothesis could be true.
But, as someone who has taught both research methods and statistics, I see a few problems. None of those forms of evidence (analogy, testimony, or example) are legal tender as proof of anything. That's not how science works. If it were, Jenny McCarthy would be Surgeon General. Verducci himself states that the “25 or under, 30 or more” rule of thumb is "is not a scientific, predictive system." He's right. There are plenty of problems with how Verducci justifies his hypothesis. It might be minimally decent science to add the disclaimer, but it's irresponsible journalism. If you don't have good evidence for a theory, why write an article at a highly visible outlet and name names?
What's even more frustrating about the Verducci Effect is that it's presented in the form of a testable hypothesis. And there have been people who have tested it. David Gassko at The Hardball Times looked at the group-level stats of Verducci identified pitchers with another group of under-25s who did not increase their workload. There was little evidence for the Verducci Effect. BP's own Derek Carty used a similar matched-pairs methodology and found comparable pitchers for each pitcher on Verducci's list, but who did not increase their workload by 30 IP or more. Again, no difference. Jeremy Greenhouse combined Gassko's approach with some regression-based analysis and found no evidence to support Verducci. J.C. Bradbury used a moderator model regression (and random effects!) to study whether the "Rule of 30" had support. Short answer: no.
So far, it's not looking good for Verducci, but it could be argued that each of the above studies had some small flaw in it. I propose to go a little bit deeper than some of these previous studies did and with somewhat more complete data. Let's give the Verducci Effect a full, impartial examination and then decide what to do about it.
Warning! Gory Mathematical Details Ahead!
I pulled together the following databases:
- Minor-league pitching data from 2006-2012.
- Historical regular and postseason pitching data.
- The BP injury database, as compiled by Corey Dawkins and Dan Turkenkopf. I am indebted to Ryan Lind for his work in converting the file for me.
Age was calculated as of April 1st (Opening Day) for all players. Innings pitched included minor-league, regular season MLB, and postseason innings. Per Verducci's criteria, I looked for all players who were 25 or younger and who had an increase in workload of more than 30 innings in the previous season.
Methodological Issues
There are some methodological issues to resolve. As a comparison group to the Verducci group, I selected all pitchers who were 25 or younger but did not have a 30 IP jump, but even this proved to have some hidden biases. Pitchers in the Verducci group were much more likely to work primarily as starters. It's not surprising, because it's a lot easier to get 30 extra innings when the baseline usage is 170 than it is when the baseline usage is 50. To that end, I considered only pitchers who started 80 percent of their games. (Later, I took off that filter. It didn't make a difference.)
I compared only major-league performance in the regular season and required that the pitcher have 50 major-league IP both in the year previous (when he amassed the 30 extra innings) and the "year after" in which the Verducci rule predicts doom. This allows a sample size that made it reasonably likely that I would be getting a good read on performance stats like strikeout rate, rather than making judgments on 20 MLB hitters faced in one year.
There were 75 "Verduccis" and 137 controls. To detect a medium-sized effect in a t-test (Cohen's d' = .5 for the initiated), you need a minimum of 64 per group. For a chi-square, you need about 58. With a bigger sample size, we could detect smaller and smaller differences between the groups, but there comes a point where, if the impact of being a Verducci Effect pitcher is that small, no one would bother writing a column about it. This sample size should be OK for our purposes.
Injuries
The BP Injury database is nice in that it goes back to 2002, lists injuries sustained in the minors, gives details on what happened, and tells how many games the player missed. I ran a series of codes for whether or not a pitcher suffered any injury at all (even a day-to-day one), whether he spent time on the DL, and whether he sustained an injury to several specific parts of the arm (e.g., shoulder, elbow, wrist), or to any part of his arm in general. I also calculated the total number of games a pitcher missed due to his injuries.
The results:
|
Control |
Verducci |
|
|
Shoulder |
12.4% |
17.3% |
|
Elbow |
10.2% |
13.3% |
|
Forearm |
6.6% |
4.0% |
|
Hand |
1.5% |
5.3% |
|
Fingers |
5.1% |
12.0% |
|
Thumb |
0.7% |
2.7% |
|
Upper Arm |
1.5% |
0.0% |
|
Wrist |
1.5% |
0.0% |
|
Any Arm Injury |
32.8% |
45.3% |
|
Any Injury |
54.0% |
68.0% |
I tested all of these by chi-square. Only one specific body part came in significant (fingers), but the "any arm injury" and "any injury" categories had p-values of .072 and .048, respectively. A Verducci victory? From the looks of it, those in the Verducci group really are more likely to have an injury.
Not so fast. One good reason to have an increase of more than 30 innings is to have ended your season two years ago with an injury and then come back to pitch a full season. It stands to reason that the Verducci group may include certain pitchers who have an injury history, and as such they would be more prone to injury.
To test this, I ran a logit regression with two binary predictors, Verducci group vs. control, and whether two years ago the pitcher had an injury to that body part. Results were pretty much the same. While Verducci status did not affect individual body parts, the findings around "any arm injury" were much the same, with a significance value of .088. So, there's still marginal evidence of increased arm injury risk, and it's not just previous history driving it. Verducci is vidicated!
Before you pop that champagne, hang on. I ran a t-test between the Verducci group and the control group to look at who missed more games as the result of injury. The Verducci group missed an average of 15.08 games over the course of a season, while the control group clocked in at 21.09 games. When I looked specifically at those who sustained an injury, the Verducci group missed an average of 22.17 games from those injuries while the control group missed an average of 39.04. Both differences are significant.
To say that pitchers on the Verducci list are more likely to experience an injury is correct. However, it's actually the control group that is more likely to land on the DL. In fact, 29.2 percent of controls spent time on the DL compared to 24.0 percent of the Verducci group (not significant, for the record). Perhaps we might interpret this in context. Teams probably treat their young pitchers gingerly, especially in the context of having pushed them a bit in the previous year. Maybe they are a little more careful with minor injuries and push back a start here and there. But there's no evidence that Verducci Effect pitchers are more likely to sustain serious injuries. In fact, what evidence is there runs in the opposite direction.
Even if these numbers were significant, we're talking about small absolute differences between groups. The real story is that pitchers in general tend to get hurt. If about a quarter of young pitchers spend time on the DL, then any list of young pitchers made before a season, even one drawn at random, will have several victims of injury. It's not being under 25 and overworked. It's being a pitcher that's the problem.
Performance
This one was easier than I thought it would be. Again, my control group were all pitchers 25 or younger who had not increased their workload by 30 IP in the previous year. Verducci status was not associated with differences year over year in strikeout rate (per batter faced), walk rate, home run rate, or ERA (i.e., his strikeout rate went from 12.1 percent to 13.2 percent for a difference of 1.1). This held after controlling for the previous season's number to account for possible ceiling or floor effects. I tried running predicting strikeout rate using last year's rate and Verducci status. Nothing.
I even looked at whether there would be an association between Verducci status and whether there would be a one-run spike in ERA. Verducci Effect pitchers (15.9 percent) were actually a little less likely than controls (19.4 percent, difference not significant) to have that happen. Again, there will be guys who fall apart in any list of pitchers. It doesn't seem to be associated with being young and extending yourself in the previous year.
This one wasn't even, "Well, it's a complicated story and I could see how at a glance, you might make that mistake…" Whether the analyses were simple or a little more sophisticated, there is very little evidence that Verducci status has any effect on performance.
Does it make a difference if you are big or little?
In his article, Tom argues that there two mitigating factors. One is that players who are bigger will be less likely to regress or face injuries. Bigger bodies are meant to handle bigger workloads. (This one gets thrown a lot… it makes sense, but has anyone ever done this study?) Additionally, Verducci suggests that the number of innings by which the pitcher exceeded the 30-inning threshold matters. (Guys like Chris Sale, who increased his 2012 workload by more than 120 innings, are at particularly great risk, according to Verducci.)
This is also testable. The number of innings criteria is quick to calculate. For size, I used the player's listed height and weight to generate a
I isolated only the Verducci pitchers, since they are the only ones affected, and tested whether
No.
Simmah Down Now
To all the fans of teams who had a guy or two on the Verducci list, you can calm down (for a moment… in a minute we'll talk about why all hope may still be lost). For all you fantasy owners preparing for your draft, please ignore the list.
The Verducci Effect is a case of speculation mixed with a really poor understanding of the scientific method, and that is a dangerous combination. It gives the illusion of knowledge, and that's more dangerous than simply not knowing something. It's tempting to want to grab onto the Verducci explanation, especially when a young pitcher with so much promise suffers such a large setback, because a wrong explanation feels better than no explanation.
With that said, my findings are not a license for teams to go out and Mark Prior-ize their pitchers. You can blow out a young (or old) arm from overuse. It's just that the Verducci formulation isn't a good guide to figure out who is at risk. I'll suggest that the reason has to do with how Verducci defines his sample.
Consider the ways in which a pitcher can pile on extra innings, and what effects we might expect that pathway to have for him with respect to injury risk and performance.
- He could show genuine improvement in his ability to pitch, and his manager will want him to be on the mound more. This may or may not increase his susceptibility to injury, but since the improvement is genuine, we'd expect performance to stay steady or even improve.
- He could hit a vein of good luck which is mistaken for genuine improvement and be left on the mound longer. Baseball is a game in which luck plays a bigger part than some people want to admit, and consequently, it’s often interpreted as skill. His performance will likely regress. Injury risk may or may not increase.
- He could have one of those years with his BABIP. Figure that a starter faces 800 batters in a year and roughly 600 balls in play. If his BABIP goes from .300 to .250, that's 30 extra outs. Even if he doesn't really face too many more hitters, that's a nice down payment on his extra 30 innings. You can't make the case that he's really over-extended himself, but his performance will probably regress.
- His team could make a deep playoff run, and he makes four extra starts in the postseason. Maybe he was pitching over his head to get them there. Maybe he really improved. Maybe the team got there in spite of his awful performance.
- He could suffer a season-ending injury two years ago, pitch a full season last year, and come into this season with a Verducci tag and an elbow that has already had surgery. Performance is anyone's guess, but injury risk is probably higher.
- He could have made mechanical adjustments that not only cleaned up his delivery and enable him to last longer in games, but also make him less likely to be injured. The improved mechanics might make him a better pitcher or just able to sustain the same performance deeper into a game. The effect on performance is an unknown, but he's probably less likely to suffer an injury.
I could keep going. What should be apparent is that Verducci Effect pitchers are a very mixed bunch. There are plenty of ways to get on the list, and different ones might bode well, poorly, or neutrally for a pitcher's future. The problem of the Verducci Effect formulation is that the sample is far too heterogeneous to expect coherent effects out of it. Maybe the real frontier here is in breaking players down into sub-groups based on how they got onto that list to begin with. It's much more complex, doesn't fit nicely onto the page of a magazine, and it's the way that real research is done.
So here's to hoping that I don't have to resurrect this column a year from now. It's time to just admit that the Verducci Effect doesn't hold water and move on.
Thank you for reading
This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.
Subscribe now
Wouldn't we expect mechanical adjustments to increase injury risk, especially in the short run? A pitcher with bad mechanics has still trained his body to support those mechanics. When he switches to a cleaner delivery, he's exposing joints to stress in new ways that they haven't been trained to endure.
Examples:
1) If the pitcher increases momentum, then his leg muscles would be more taxed but it could very well lesson the toll on the throwing arm, due to a more efficient use of the body to generate (and transfer) kinetic energy. So the legs would be at higher risk, but that is favorable when compared to the alternative.
2) If a pitcher increases balance, then the improved stabilization would most likely result in better mechanical repetition, thus lowering the variability with respect to recruiting associated muscle groups. Also, improvements of posture would allow the arm to function more efficiently during rotation, again helping to buffer against injury.
3) The danger zone is with torque, as often a pitcher will expose his joints to new loads, going against the buffer created through years of throwing, just as baseballATeam mentions. This is particularly true with pitchers who introduce scapular loading or who have timing issues with respect to trunk rotation, and the combination of more kinetic energy with ill-supported muscles and joints is a big risk factor.
And as lagniappe mentioned, conditioning is an underlying factor with all mechanical considerations.
Does it create a bias to exclude pitchers who don't pitch 50 innings in year 2? If I understand the methodology correctly, you would be excluding pitchers like Michael Pineda, who Verducci could fairly argue is exactly the kind of guy he's talking about.
"But there's no evidence that Verducci Effect pitchers are more likely to sustain serious injuries. In fact, what evidence is there runs in the opposite direction."
By setting that 50 IP threshold you have filtered out players that suffered the most serious injuries. Did the Verducci Effect miss them or just your filter?
I understand you would want the filter in place to make the performance comparisons, but you should not have used it for the injury portion.
Perhaps looking at a guy who did get hurt against one with comparable size/profile that didn't would be really interesting to see.
The other point I would make is perhaps we are looking at the wrong era for this. Keith Law mentioned the High School pitchers and termed the Pulsipher effect where a young, immature arm gets a very heavy workload and breaks down. Looking at an earlier era would give us more data but I am guessing we wouldn't have as accurate injury data then
1. The methodology behind the Verducci Effect is poor, and we should be very skeptical, if not outright dismissive, of the results it returns. But . . .
2. There might be something to the overall concept (pitchers under 25 with a greater increase in workload than their peers might get hurt more and/or maybe show worse performance than their peers), and so we should keep an open mind about this being a problem. And so . . .
3. We need a more comprehensive, less error-filled methodology to get us closer to finding an answer - i.e., more research required.
I can live with this. Thanks, Russell, for helping bring a bit of closure to this.
2. Are BP writers ever aloud to debunk work done earlier at BP?
3. More specifically about this study and Chris Sale, how do reliever to starter conversions fare compared to pitchers with a similar workload? Do mid season conversions do better or worse than off season conversions?
4. Following up on Shaun P.'s question no. 2 - I appreciate the need to do more research to get more specific, but in the aggregate: WHAT, if anything, DO WE KNOW? Do pitchers under 25 get hurt more than pitchers over 25? What are the healthiest ages for a pitcher? Do young pitchers who pitch deep into the post season have a worse-than-expected history of injury? . . .What types of pitch repertoires are most hurtful? least hurtful? (There must be a reason screwball pitchers have become extinct.) . . . Following up your own question - and something I've been wondering for over 43 years - ever since I heard on television Whitey Ford (who was shorter than average) speculate (and completely nail) that Denny McLain - who was on his way to his second consecutive Cy Young Award - might not have a long career because he was shorter than average - do shorter pitchers get hurt more (or not last as long) as taller pitchers?
2. I would debunk my own mother if she were wrong. (Mom, if you are reading this, you've never been wrong).
3 & 4. Good questions to which I have no answers. There's so much more to figure out.
Of the 5 cases mentioned 1 got hurt (Feliz), 1 was pretty bad (Bard), 2 stayed in relief (Chapman and Crow), and one was pretty awesome (Sale). Just sayin'
A few comments:
1) I agree that the 50 IP year II threshold is problematic, and I would not be comfortable drawing any conclusions without redoing the study while correcting this.
2) I looked at this as well a few years ago, and "concluded" that the Verducci Effect was merely regression toward the mean. Pitchers who had an increased workload tended to have very good seasons in year I. Very good seasons means they were "lucky" (as a group) and thus were going to to appear as if they got worse in year II due to regression. If one uses a control group, one must be sure that the control group was equally good in year I OR one must control for regression, perhaps by comparing actual and predicted (via Marcel or Pecota, etc.) performance in year II.
3) In testing this or any hypothesis, one must be careful to use "out of sample" (to the original hypothesis) data. In other words, if Verducci noticed his effect for players in, say, 2006 and 2007, one must remove that data (those years) in testing his hypothesis. Let's say that I "noticed" that in 2010, the HFA in baseball was much higher than usual, so I hypothesized that HFA was increasing in MLB (perhaps due to the decrease in greenie use). If I want to test this hypothesis, I cannot include the 2010 season, since obviously that will confirm it.