The M’s Bullpen Bet
Perhaps the most striking trend in baseball this off-season has been the creation of super-bullpens: teams that already have one or two great relievers doubling down and getting another one. The Yankees put together a remarkable bullpen last year, headed up the utterly dominant pair of Andrew Miller and Dellin Betances. Their big move of an otherwise-quiet offseason? Trading for closer Aroldis Chapman. The Dodgers tried to acquire Chapman, despite the fact they already employ a great closer in Kenley Jansen. The Red Sox traded four very good prospects for closer Craig Kimbrel, then traded a middle-of-the-rotation starter for the M’s Carson Smith, despite already having Koji Uehara and Junichi Tazawa as set-up men. The Orioles resigned set-up man Darren O’Day to a four-year deal to work in front of closer Zach Britton. The lowly Colorado Rockies, who may struggle to win 70 games, picked up ex-closer Jason Motte *and* traded for Jake McGee. You get the idea. This is something we haven’t really seen before, and something that runs counter to some pretty core sabermetric beliefs about bullpens and how to value them. Jerry Dipoto’s Mariners seem anomalous then, first for trading out Carson Smith and then for handing the closer job over to Steve Cishek, who lost his job last year and was quietly shipped out of Miami for a low-level prospect. What’s going on here? Are the M’s missing the boat, or is doing the opposite of what everyone else is doing the surest/only way to find market inefficiencies in baseball?
This is a gross oversimplification, but I tend to think baseball and baseball analysts have two primary views of relief pitching. The first is leverage: sure, they don’t pitch as many innings as some random 4th starter, but the innings they DO pitch are often critical to wins and losses. A great reliever pitches effectively in high leverage situations, and thus helps his team win close games. Winning close games can be the difference between “contending” and celebrating, so, by this view, it’s not a surprise to see teams (especially in the AL, where every team’s clustered around average) focus on adding depth in the bullpen. The second overarching view of relievers is volatility. Sure, they’re highly leveraged innings, but the pitchers are so inconsistent that it’s kind of insane to give four years and $30m plus to a set-up man. One year ago today, there were two things we thought we knew about the bullpen: Danny Farquhar was head and shoulders above everyone, and Mark Lowe signed a deal so he could retire as a Mariner or whatever. Sabermetrics has studied performance in one-run games, and can’t find much (if any) of a lasting effect. It’s noise. Just look at the 2012 Baltimore Orioles, who rode Jim Johnson to a ridiculous 29-9 record in one-run games and an unlikely playoff berth. But their great bullpen performance in 2015 is *totally* different and ultra-sustainable and basically sequencing and BABIP-luck free, right?
These views contradict each other in places, particularly whenever anyone tries to stick a value on a reliever, whether in trade or free agent contract terms. They’re not mutually exclusive: you can acknowledge that relievers are both really important and really volatile, and sort of shrug your shoulders and scan the waiver wire for live arms. That was plan the Oakland A’s implemented not so long ago, and it was ridiculously effective right up until it wasn’t (volatility!): from 2012-2014, the A’s edged out the vaunted Royals bullpen in ERA and strand rate. Their cumulative ERA was bested only by Craig Kimbrel’s Braves, and the A’s did it with guys like Dan Otero, Jesse Chavez, Jerry Blevins and converted 1B Sean Doolittle. But the current trend of building dominant ur-pens is really a response to two teams: Kansas City and Pittsburgh. As Jeff Sullivan noted in this post at Fangraphs, the Pirates and Royals rank 1st and 2nd, respectively, in cumulative wins above projected totals over the past three years. The Pirates have won 280 games in those three years, or an average of 93.3, while the Royals have won 270 (an average of 90), two pennants and a World Series. Neither team ranked in the top 10 in starting pitcher WAR, and the clubs rank 15th and 17th in runs scored over this time period.
Projection systems, relying on base runs and context-neutral stats, see flawed clubs. There are a number of reasons why the Royals and Pirates have won more games than their base runs would suggest, and many of these have nothing to do with pitching at all, but clearly their bullpens have been a part of the equation. By ERA, the Pirates and Royals have been in the top 10 in ERA in each of the last three years, finishing 2nd and 3rd in 2013 and 1st and 2nd in 2015. There’s all sorts of evidence for volatility (the M’s were first in ‘pen ERA in 2014! They were…not in 2015!), but these two clubs have sustained high-level, high-leverage performance from year to year. By win probability added, it’s even more stark. #1 in 2013? Pittsburgh. 2014? Kansas City. 2015? Pittsburgh. Over the past three years, the clubs are #1 and #2 in WPA, edging out New York, who’s also been surprisingly consistent. Pittsburgh and Kansas City (and maybe New York) have deep pens with elite set-up relievers who’ve succeeded in high-leverage innings before the 9th. For years, sabermetric dogma decried modern bullpen management that often left an elite closer to come in with the bases empty in the 9th while leaving a tie-game, 1-out, bases loaded situation to a much worse reliever. With depth, this problem is greatly reduced or even eliminated; with elite set-up relievers, you can play match-ups AND keep your closer for the 9th/extra innings.
Beyond the truism that three or four great relievers are better than one or two, this gets at *why* these clubs have been able to beat their projections. A team with a great bullpen can effectively *change* base runs/win probability by altering the run expectancy for batting events, and they can do so whenever the manager wants. Tie game in the 8th, and the tiring starter walks the lead-off man. BP’s run expectation data for 2015 shows 0.84 runs are expected to score through the rest of the inning. Now imagine that the manager has Dellin Betances, Andrew Miller and Aroldis Chapman available in the bullpen. Might that leadoff walk have a tougher time making it around? Of course, this literal arms race is already impacting everything in baseball. Scoring is down (the run expectancy of that leadoff walk was 0.86 in 2012 and .90 in 2008) and velocity is up in part – IN PART – because relievers pitch more innings, throw harder, and give up fewer runs than starters. For years, sabermetrics folks have debated how to value relievers because certain key assumptions that “worked” on pitchers in general worked less well on relievers. For example, HR/FB ratio is fairly stable for pitchers as a whole – that’s why xFIP exists – but relievers post consistently lower HR/FB numbers. Relievers strand more runners and thus, on average, post lower ERAs than their FIP would indicate (starters, on the other hand, have higher ERAs than their FIP). The more we learn about the times-through-the-order penalty, the more obvious it all gets, and the more it points to having a bullpen pitch more innings. If all of these trends push down scoring, and scoring IS down of course, then these advantages are multiplied. If every run counts, a deep bullpen is all the more vital.
But this obviousness cuts both ways. It demonstrates that successfully navigating high-leverage situations is crucial, but it doesn’t prove that teams can consistently succeed in such situations. Not only do you need great relievers, you need a manager who knows how to deploy them, AND you need some luck. By WAR – the context-neutral measure of value – the Cubs and Orioles had *better* bullpens than the Pirates and Royals last year. It’s only by WPA (and that great sabermetric bugbear, ERA) that the Pirates/Royals re-take the lead. One way of reading this seeming incongruity is that the Pirates/Royals had worse pitchers who saved their best innings for when it counted (read: got lucky). The fact that they’ve done so for a few years may be a statistical fluke: Oakland’s success from 2012-2014 did not help them in 2015. The Astros were brilliant in context-neutral stats, but their bullpen was terrible in the clutch, particularly during their second-half slide. They did everything the “leverage” folks asked for – they built a deep bullpen with quality arms to attack lefties and righties, and they watched it pitch brilliantly when up by 3, and poorly in tie games. Volatility, all is volatility.
Moreover, the fact that relievers throw harder, strand more runners, etc. as a whole would seem to *undercut* the case for valuing relievers more. The pool to which each reliever is measured against keeps getting better in a weird, journeyman form of the Willie Mays problem. As many have noted this year, the average bullpen arm in baseball is much, much better than you might think. Relievers – a group which includes Brendan Ryan, Tyler Olson, Mayckol Guaipe, and dozens of people you’ve never heard of – struck out 22.1% of the batters they faced. A reliever who strikes out a batter an inning is no longer the mark of a very good, bat-missing hurler – it’s pretty much league average.
Think of some of the elite relief performances we’ve seen in recent years – on paper, there was nothing flukey about them: Neal Cotts’ bizarre 2013, Farquhar’s 2014, Fernando Rodney’s 2012. Sure, the ERAs were low, or the strand rate high, but they struck out tons of batters and posted very good FIPs to go along with those freakishly low ERAs. And yet none of them was able to fashion the kind of sustained excellence we’ve seen – SO FAR – from O’Day, Miller, Chapman and the like. If velocity and fielding-independent stats are only partially correlated with future success in the pen, does it make sense to commit more years to a free agent reliever, or to swap elite prospects or solid starters for them? If you can get someone to K 22% of batters faced for a minimal amount, and if a certain percentage of non-roster invitees keeps coming up aces (Mark Lowe is good again? Didn’t Neal Cotts’ arm fall off? I thought Ryan Madson retired?), maybe it makes sense to avoid overpaying for Darren O’Day and let volatility work FOR you by acquiring a bunch of guys coming off poor results.
Now, the “leverage” partisans will say that it’s a baseball-specific form of nihilism to imagine that spinning the NRI wheel or bringing in Steve Cishek will produce a bullpen equivalent to Pittsburgh’s or New York’s. Yes, bullpens are volatile, but we’re talking about baseball: *Brandon Crawford* hits home runs now, Matt Lucroy sucks at pitch framing, that bad catcher with the below-league-average batting line you used to mock in 2011 just won the AL MVP award. EVERYTHING is volatile, and it doesn’t prevent teams from signing people or deploying them in ways they think will help them win. Still, the magnitude of the M’s bet here – trading for Joaquin Benoit aside – is pretty remarkable. In a tight division, in a pitcher’s park playing teams on the west coast, I could definitely see the M’s trying to shop for brand-name relievers as part of a retooling. Spend some money, still get value (reliever salaries have gone up sharply, but have you seen what bad starters get these days?), and squeeze more wins out of the moderate number of runs your club produces.
Instead, the M’s have largely shopped the bargain aisle, picking up the broken, the unlucky and those coming off bad years. It’s not that the M’s opted solely for the kind of NRI pile that Oakland used, or that the M’s have trawled successfully in the past – they’re spending (some) money and (mostly meh) prospects to acquire relievers. But they’re doing so in a very different way, and seemingly looking for very different things than the rest of baseball. Here’s hoping their ex-reliever GM sees something special in the M’s pen. If he doesn’t, and the Yankees and Red Sox take the wild-card slots, we’ll have another small-sample-size result to fight over.
Apropos of bullpens… maybe… the Mariners tweeted this a few days back. Dipoto pitching to Servais:
http://m.mlb.com/video/v542370483/chcnym-servais-singles-off-dipoto-in-the-7th
I’m a partisan of the “build a bigger pile of spring training invitees” approach to assembling a bullpen. GM Jerry has indicated he believes that a smaller number of players in the big league camp allows for a better opportunity to judge player performances and so he’s passed fewer invites out to bullpen prospects. Maybe his selectivity approach will produce better results but I harbor some doubts about the wisdom of that. He may need to consider using the 6th best rotation arm in the bullpen, rather than stashing them in Tacoma.
This is great analysis, Marc. I’ll admit, as a card-carrying member of the “throw a bunch of cheap live arms at the wall and see what sticks” camp, it’s been confounding to watch a lot of teams go the opposite direction and go all in on elite relievers. It certainly makes some sense when you’re a big-market team that is near contention like NYY/BOS/LAD. It makes a little less sense when you’re a team like the M’s, Baltimore or Rockies that has glaring holes and lacks the payroll space or farm system to absorb one of these top-shelf relievers blowing out their arms or regressing hardcore (which will happen to 90% of relievers eventually). I don’t think that Dipoto had the luxury of paying top dollar for a Chapman or Kimbrell – and as long as other teams were doing that, it makes some sense to cash our elite reliever in on a player like Miley that fits that park and the team’s 2-3 year budget window well.
I realize this isn’t the same as throwing huge amounts of money at a “proven closer”… but so many of those deals blew up in team’s faces, I really am not sold on the approach the Yankees and others are taking. Relievers just seem too volatile – even the good ones.
Past performance does inform you regarding a pitcher’s base talent level, but assuming they’ll be equally effective and available going forward just seems like a bad idea. It’s going to take more than one or two years’ worth of countervailing examples to change my mind… so, at present, Dipoto’s approach (and I guess Jack Z’s original approach) is preferable to me. Build a big pile o’ relievers on the cheap, then mix and match.