A few years ago, we introduced the concept of the secret sauce. It was designed to predict the odds of a team’s postseason success based upon the quality of their run prevention, using a team’s strikeout rate, the quality of their closer and the quality of their defense.
So how well has the sauce fared since its introduction? Prompted by some recent research, I took a look at how often the team with the best sauce won each series, from ’06 (the first season the Sauce was used) through ’09, and the result was only 54% success – not significantly better, statistically speaking, than flipping a coin.
The playoffs are notoriously noisy, of course – Billy Beane famously remarked, “My shit doesn’t work in the playoffs,” and it’s easy to see why. You’re talking about short series between teams that are very evenly matched – as mismatched as any two playoff teams might be, there is no team as bad as the 2010 Mariners in the playoffs. So compared to the regular season, you’re dealing with a real crapshoot.
It’s possible that the Secret Sauce ran up against a few years where its performance was flukily low. But what we have is a model based on historical data that has thus far been ineffective at predicting results out-of-sample, which doesn’t give us a lot of reason to be confident in it going forward. So for now, we’re retiring the Secret Sauce.
Of course, that doesn’t mean that we won’t be providing you analysis for the playoffs. This week, we’re going to start looking at the revamped PECOTAs, and one of the things we’re going to be doing is using PECOTA to give you an in-depth look at each team in the post season. Expect to see more about this as we unveil the changes in PECOTA and we get a clearer picture of the matchups we’ll be seeing in the playoffs.
Thank you for reading
This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.
Subscribe now
I would hope though that the recanting isn't just due to 4 years' worth of post-season series but also some suspicion that the original research was a bit suspect.