We recently went to great lengths to develop the new version of SS/SIM, the Scoresheet simulation value metric here at Baseball Prospectus. As fellow Scoresheet owners, we realize that optimally these valuations would have been complete several weeks ago. Unfortunately, the original formulation of SS/SIM was not available to us this year, and our replacement took more time to develop than we had anticipated. We set out to create an all-new run-based value metric for Scoresheet team owners to use in player valuation. This time consuming effort was spearheaded by BP staffers with Scoresheet experience, and included research into the Scoresheet engine, talking with Jeff and Dave Barton, and consulting with Scoresheet experts for feedback.
Development of SS/SIM for both hitters and pitchers starts with the Scoresheet definition of replacement level, namely the awful Triple-A players that are used when you run out of playing time in the Scoresheet simulation. From there we have captured the most important aspects of the Scoresheet simulation and built them into the SS/SIM methodology to produce a single metric for Scoresheet owners. These adjustments seek to raise the effective replacement level and account for the differences between MLB and Scoresheet. [edit: BJM 2011-03-28]
Hitters:
The offensive formulation builds a value estimate from the replacement level based on expected offensive output. The offensive valuation is built from a positionally adjusted perspective that also emphasizes overall offensive output, supplemented with a defensive adjustment that accounts for each player's Scoresheet defensive eligibility and range rating, as well as a penalty for expected errors.
The results of this process are valuations that use a very low replacement level–note that Player(AAA) is remarkably low in value compared to the replacement level used for metrics designed for evaluating MLB players. SS/SIM as a hitter metric provides a complete depiction of a player's value in Scoresheet because it uses replacement value specific to Scoresheet and incorporates the latest research on Scoresheet defensive range value and appropriate valuation of errors to form a comprehensive view of the player.
PItchers:
The pitching formulation is a bit more complicated. Pitcher valuation is based around projected IP and ERA because the foundation for value in Scoresheet for pitchers is their ERA, and because pitchers in Scoresheet are basically limited to their real life playing time. Because we expect Scoresheet owners to alter pitcher usage based on ERA, each pitcher is projected to throw a certain percentage of his real life innings for the Scoresheet fantasy team. The projection for how many of his real life innings will be utilized is separated by starters and relievers based on the expectation that Scoresheet managers will maximize usage of their relief pitchers because they have lower ERAs, which is the converse of real life MLB usage. As a result of this usage pattern, the ratio of SP IP to RP IP is lower in Scoresheet than MLB, and our methodology reflects that difference as well. We also put a premium on pitchers whose ERA is better than league average to account for the importance of ERA and the multiplicative nature of actual run production in the simulation. This adjustment is proportional to the extent that a pitcher is better than replacement level, so it impacts premium starters the most, but it also increases the importance of very good relievers as well. Given the expectation for how many innings a pitcher would pitch for his Scoresheet team, we can derive his SS/SIM by comparing his forecast for runs allowed over that time to what the Pitcher(AAA) would allow.
The results of this process are valuations that use a low replacement level, and also incorporate several differences between Scoresheet and MLB. Primarily, the significance of ERA within the simulation is because of how expected runs are scored, as a product of expected runs allowed by pitchers and expected runs scored by hitters, more or less. This multiplicative effect is captured in SS/SIM through the premium placed on elite pitchers. Additionally, a pitcher's ERA is the primary driver in our adjusted usage patterns, which capture the difference in usage between SP and RP from Scoresheet and MLB. SS/SIM as a pitcher metric provides a complete depiction of a pitcher's value in Scoresheet because it incorporates all of these factors to form a comprehensive view of each pitcher.
Usage:
We expect that as a run based metric, SS/SIM will have an approximate equivalency of 10 runs to 1 win. In practice, we hope that SS/SIM helps as a way to measure players of different positions against each other, and on a large scale, to help identify what value to expect from each player. On a more granular basis, because these are projected values, small differences in SS/SIM are probably worth prioritizing below other considerations, like team needs and draft context. Comparisons between pairs of players, especially when looking at draft picks after the middle of the draft, should largely be based on team composition to that point–whether you need 175 IP at 4.55 ERA or 65 IP at 3.60 ERA largely depends on who else you have on your roster.
SS/SIM is available on the player cards and in the Player Forecast Manager.
Thank you for reading
This is a free article. If you enjoyed it, consider subscribing to Baseball Prospectus. Subscriptions support ongoing public baseball research and analysis in an increasingly proprietary environment.
Subscribe now
With Scoresheet, the game is simmed based on current-year stats, and is at the whims of managerial decisions (such as the Napoli question in this thread). So PECOTA projections of batting stats and errors - as well as predicted playing time estimates - are part of the mix. What would be interesting would be a "preview" card rating for the following season's Strat-O-Matic set, but in many ways, that's just impossible, as the game company hasn't made decisions on the ratings for various players yet - most notably defensive ranges, which impact both position player values (directly) and pitcher values (indirectly).
http://pfm.sportsprospectus.com/index.php?raw=csv
I ran the PFM and overall I believe it is underestimating pitching, and not doing a great job identifying the most valuable pitchers in SS.
Most SS leagues are continuing leagues, so factoring upside would also be a nice option for the pfm.
" This IS the way it works. A couple posters repeatedly make the claim that *only* ERA matters as to what will happen in Scoresheet. But that is simply not true, nor has any studying of Scoresheet stats (either by us extensively over the years, and also by folks on this list) ever shown that ERA determines what happens in Scoresheet games regardless of what the other team's hitters did in real life. As far as whether ERA is too important compared to a pitcher's hit and walks allowed per inning pitched (WHIP), we do factor *both* ERA and WHIP into the algorithm that determines the odds of each at-bat's outcome. Basically the better a pitcher's WHIP and/or ERA the less the chance of a plate appearance leading to a hit, and the worse those numbers the greater the chance, all also influenced by how the hitter did in real life, and the fielders. But the sim is rather complex - there is no simple number I/we can give for the relative weighting of those two factors."
- see http://www.attheplate.com/notes/ss_pitch.htm
Thus I never ignore projected and actual WHIPS when drafting or trading for pitchers. If they've since radically changed the algorithm to seriously under-emphasize WHIP, I need someone to tell me.
http://sports.groups.yahoo.com/group/scoresheet-talk/
We will look into factoring UPSIDE into future iterations of SS/SIM. Thanks for your feedback.
We do always suggest that you run your league through PFM to account for the various crossovers or other oddities that may exist in your specific league.
1. Chris Snyder is ahead of Mike Napoli.
2. If I read this right, you're using player AAA as replacement level. Like the others, I view this as misguided.
3. Felix is expected to allow 66 total earned runs per the spreadsheet. Replacement AAA is 1.8 (league ERA) or roughly 7.30. 230 IP at 7.30 ERA is about 186 runs. That's a 120-run difference, but Felix is only listed at a value of 88. What have I calculated incorrectly?
4. Pujols is listed as 75 runs above Player AAA. That number is also quite a bit lower than I calculate.
--JRM
We found that experts feel like player AAA is too low of a replacement level to use for a metric to be effective. So while we started out with player AAA, we made an effort to adjust accordingly when appropriate so that both the process and the results seemed reasonable. I won't say that this process is perfect, and I'm open to suggestions to improve it.
1. This may be due to our projections having Napoli playing primarily away from catcher, even though he is catcher eligible in Scoresheet. I will investigate.
2. I understand the desire to have a higher replacement level (see above). The adjustments that we make for defense (compared to average not replacement level) and for overall offensive output (a pure runs above average view) have the effect of raising this replacement level for hitters.
3. The ERA that we're measuring Felix against is lower (better) than what you have there. Not because player AAA will perform better, but because in practice, the pitchers that you could find available will have ERAs a couple of runs lower than pitcher AAA would have. I think the effective replacement ERA for SP is around 5.50 or so.
4. I think this is resolved in the higher replacement level adjustments, but if not, I'd be interested in your methodology and how it compares to ours.
Thanks for your continued feedback. :-)
http://www.scoresheet.com/BB_FullSim.php
•Each pitcher's complete stats are used, including the number of strikeouts, walks, hits, earned runs and innings pitched he recorded that week in the majors - even wild pitches!
•Pitchers "pitch carefully" when first base is open and a runner is in scoring position, giving up more walks but fewer big hits
•Team-dependent stats such as wins and saves matter less than individual stats such as ERA
•Your closer will only pitch in save situations
-list this in the PECOTA sheet along w/ Scoresheet ID? Merging on strings is NOT fun
-Could you provide some explanation for how best to use PFM for a SS draft? Dollar values don't mean a lot in that kind of draft. Maybe make SS an option?
For now, the best way to use PFM is to put your players available and taken into the PFM and look at the relative value that the dollar values project. The minimum dollar value for the salary sets the PFM's replacement level for dollars, so every player projected to be worth that much or more should be drafted given the league and setup you put into PFM. The relative values in dollar values should be of some interest, but like you said, they don't correlate as well as the SS/SIM values should to overall value (they are also a second iteration of positional adjustment, which isn't really necessary).
http://www.baseballprospectus.com/article.php?articleid=13362
"The range of the player in center field for you is about 1.4 times as important as either the left or right fielder when figuring your overall team range. This means you should have at least one high range outfielder to play center field for you. The 'average' CFer has a range of about 2.16, while
the 'average' LFer and RFer have ranges of about 2.07. Since the range of your CFer matters 1.4 times as much as at other positions, it is better to have a 2.16 range player in center, along with two 2.07 range players in left and right, than it is to have three 2.10 players filling your three OF spots. (Most Scoresheet teams have a player of at least 2.11 range playing CF for them.)"
- http://www.scoresheet.com/PDFfiles/AL_rules_pckt.pdf
(page 5 has a whole lot more info on how defensive ratings affect results in SS (e.g., penalties for rostering a player at a position he's not qualified by SS to play.)
You're right that SS lumps all the OF together in one list when you're setting up your draft lists, and when viewing your league roster page. I used to find this a bit inconvenient (i.e. to have to go visit another SS webpage just to review a player's defensive stats), but then I learned that you can get the information on the undrafted players page by clicking on the player's stats line (which is hyperlinked) and when you click on it you see more information about that player at the bottom of the screen including his specific defensive number(s) at whatever position(s) he plays. Better yet, when your scorecards get published, it lumps the defensive numbers of all your rostered players together to give you a single number (positive or negative) which is the multiplier the SS algorithm applies whenever a ball is in play during a game, which has some small effect on whether a hit, out, error, or a run occurs.