Ahead in the Count: Methodology of The New MORP

This article will follow up on the new version of MORP that I introduced yesterday with a more thorough description of my methodology and my reasoning for it. Firstly, I will restate that the definition of MORP (Market value Over Replacement Player) is the marginal cost of acquiring a player’s contribution on the free-agent market. The basic structure that I am using includes adjusting for draft-pick compensation, which adds to the value of free agents by 10-20 percent. It also looks at all players with six years or more of major-league service time, all years of their free-agent contracts, and makes valuations of their performance based on actual performance rather than the projections, which are biased. I am also adjusting MORP so it is linear with respect to WARP. The discussion of linearity and of the decision to use actual rather than projected performance to evaluate contracts has been detailed in earlier articles, and I won’t reiterate them here in the interest of space. The basic reason why linearity is a fair assumption is that teams frequently have enough vacancies that they can add the number of wins they choose without filling them all. There are exceptions like the 2009 Yankees, who added three front-of-the-rotation starters and an elite first baseman in one offseason. However, even the Yankees do this infrequently enough that it does not regularly impact the market, and without two teams bidding for several superstars every offseason, this is not a large issue. The reason that using projection is so problematic was detailed last week, when I showed how free agents who reach the open market are a biased sample and regularly underperform their projections. For more details of these results, please see my previous work. Here are links to my three part series as well as my article on free agents underperforming their PECOTA projections. I will introduce some of the newer concepts in this article.

NORMATIVE VERSUS POSITIVE ANALYSIS

Since baseball is not a regular free market, the concept of MORP cuts between normative and positive analysis (i.e. what they should pay and what they do pay). On one hand, we don't have the ability to see the actual value of adding Johnny Damon to the Tigers because not only will we never experience the 2010 Tigers without Damon, but we only get to observe one of the infinitely many possible ways that the 2010 Tigers' season could play out with Damon in the mix. So we cannot properly analyze the actual dollar value added, but we can say that if most similar players cost $9 million and Damon cost $8 million, then the Tigers got a good deal. In a regular free market, if you pay $8 million for something worth $9 million, you can just trade it in right afterwards and make a free $1 million; this process removes these “arbitrage opportunities” because people start bidding up the instantly tradable Damons above $8 million. Therefore, prices of assets in pure free markets are more likely to represent value than the prices of baseball players. If the price of commercial real estate is too low, the market corrects this because people can buy up space and make money by reselling it or utilizing it; in fact, they will offer more rent to the owners if they can make more money from it (or if anyone could make more money from it, because they could rent it out themselves if anyone could).

We can't simply treat players' contracts like stocks or commercial real estate, though, because there can be market inefficiencies that the 30 people who happen to own baseball teams don't notice. If I notice a stock is undervalued, I buy it and sell it at a profit when it increases in price. On the other hand, when teams did not know how to value on-base percentage in 2003, it did not matter that Bill James knew that the market did not value OBP correctly—he couldn't make a 31st team like I could make an online trading account, so he couldn’t buy and sell players to beat the market. So we do need to focus on win values of a player to accurately assess the market value of those wins.

MORP is an attempt to get right at that value, and say how much it costs in real terms to acquire that player. Saying that it costs $7 million a year to acquire a player like Jose Valverde is not right, because it actually costs $7 million a year and a draft pick. So you need to net out the costs and the wins involved in acquiring Valverde, with arbitration and draft picks considered.

DRAFT PICKS

Due to the structure of baseball’s labor market, we know that the cost of surrendering draft picks when a team signs a player comes directly out of a player’s salary. The current rules for free agency require that if a team signs an Elias Sports Bureau-ranked “Type A” player (indicating they are among the elite players at their position) whose former team offered him arbitration, they must transfer their first-round pick to the other team (or second-round pick if the team had a top 15 pick) who will also get a compensation pick between rounds called a “sandwich pick.” If a team signs your Type B free agent by the Elias rankings (indicating they believe he is a second-tier player), you get a compensation pick as well, but it does not need to surrender a pick. This means that signing another team’s Type A player requires paying a “tax” in a way. Similarly, signing your own Type A player means foregoing that and the sandwich pick between rounds, which is a tax in terms of opportunity cost, because any team signing their own free agent would have the opportunity to get two picks otherwise. Of course, the player can also accept arbitration and allow an arbiter to give them what usually amounts to a modest raise.

Putting an arbitrary tax on most labor markets has a very different effect. For example, if you arbitrarily taxed IT professionals in one industry, they could switch to another industry. In those cases, the original industry would have no choice but to pay higher pre-tax wages so that after-tax wages would be equal to what the IT professionals received in every other industry. However, rare is the baseball player that has another industry that can employ him at the same salary. Thus, baseball players must effectively “pay” the whole tax themselves by receiving a lower salary. When the labor supply is fixed (and presumably, no one is going to quit baseball over the draft pick compensation rules), taxes fall on workers.

With this in mind, I went through each player with at least six years of service time—making him theoretically eligible for free agency if not already under contract—and figured out what draft picks were lost when teams signed Type A free agents or what draft picks would have been received had the player’s old team not re-signed him.

Adjusting for draft-pick compensation is the biggest addition to what I’m doing. The first thing I did was take all the draft picks actually surrendered by teams signing players, and of all the teams who signed their own players, I determined which of those players would have been offered arbitration had they reached free agency. Then I did my best to approximate which of these players would be Type A free agents after the contract. These decisions are actually pretty trivial in the vast majority of cases, and the few judgments I did make likely won't change the rounded estimates below. Doing Type B free agents was tricky and mostly could be ignored. I erred on the side of understating the effect of draft-pick compensation. In most cases, though, it was pretty clear who would have been offered arbitration, so the estimate should be close.

Then I used the same Sky Andrecheck’s draft pick value calculator which gives a nice formula for approximating career WAR value (using Sean Smith’s WAR numbers, which have a really high replacement level) as a function of specific draft pick. After simplifying this by ignoring player/pitcher and college/high school differences, I then converted that to WARP3 by noticing that there were about 865 WAR and 1,250 WARP3 distributed in a given year, so I multiplied by 1,250/865. Then I noticed that about one-third of all WARP3 was generated by players with over six years of service time. So I took the WAR number that had been multiplied by 1,250/865, and multiplied it by about 2/3 to approximate the percentage of the WARP the player generated in his first six years. Then I estimated that 70 percent of that was actually free (arbitration years do require some money to be paid), and so basically I got the number of WAR from Andrecheck’s draft pick calculator, and performed the following adjustments:

(0.7)*(2/3)*(1250/865)*expected(WAR) = 13.54*(pick number)^-0.49 = WARP3 value of a draft pick.

I then discounted this to represent these wins were in the future and subtracted to those from the WARP3 value provided that year. Looking through all players’ approximate debuts and draft years of those who played from 2007-09, I saw they were about three years apart, so I figured most players with over six years of service time are giving their average wins about six years after being drafted. I guessed that the discount rate for wins would be about 8 percent, and computed this from the first year of the contract. I also ballparked that most bonuses would be between $200,000 and $1.5 million and subtracted out the cost of paying this bonus if a team surrendered a draft pick. I valued the 16th pick at $1.5million and 80th pick at $200,000, and estimated a linear decline to signing bonuses in this range. Specifically, the cost is $(1.8 – 0.02*(pick number)) million.

ALL ELIGIBLE PLAYERS VERSUS ALL FREE AGENTS

Nate Silver’s version of MORP from 2005 and Fangraphs’ version from 2009 both involve projecting win values for players that have reached free agency. Sean Smith has developed a method for projecting wins for free agents similarly to Fangraphs’ method, but he recently found a problem with his methodology—his projection system was overestimating the win values for free agents, thus understating the dollar cost of a win on the free-agent market. Although he admits his system has over-projected win values historically, he still finds that the dollar value of a win is low even as he has lowered his projections. I checked this and PECOTA was doing the the same thing, similarly over-projecting free agents. The reason is selection bias as I detailed in that article.

The problem is also related to my finding from last month’s article that players who are re-signed by their old teams typically age better than players that sign a free-agent contract with a new team. Players that sign with new teams are often allowed to depart by their old teams due to reasons that are not observable to a projection system (propensity for injuries, character, etc.). So those players who are allowed to depart are bound to underperform projections on average and they do. This fact is very important, because it shows the dangers of using actual free agents to value the free-agent market. Actual free agents are not a random sample of the players eligible for free agency. They are a sample of the players eligible for free agency who did not agree on a contract with their old team. This is a relevant variable in approximating a player’s value, and any intelligent team should consider why a player reached free agency in the first place. On average, these players will underperform their projections, and so creating a dollar value of players using a projection system will incorrectly specify the price of talent.

Of course, the alternative is including players who were locked up to contracts before the free agency period began, and therefore involves using the dollar values for potential free agents signed before that period started. This does include contracts signed under different economic conditions, but there is a similar problem in evaluating contracts signed during the entire five- month offseason. The economy actually changed between the Placido Polanco and Damon signings this past offseason. Contracts are signed at different times and, therefore, under different conditions. The contracts are all set based on expected market conditions in the future, so there is always uncertainty. (In fact, each team faces different economic and baseball conditions as well, so we are never truly able to compare apples to apples.) However, looking only at contracts given out to free agents who change teams, or looking at the first year of deals, is not a good estimation of the market. If you want to figure out the strength of a labor market, you cannot only look at people who just switched jobs. You need to look at the whole labor market and what all workers are being paid. The Bureau of Labor Statistics might have quite a bit of explaining to do if it reported the average hourly earnings numbers only for workers who just received jobs, and claimed this represented the value of labor.

Because of the selection bias and the limits mentioned, I evaluated the cost of signing all players who had at least six years of service time rather than only those players that reached free agency. This eliminated the bias created when we only look at projections for players that are not a random sample. Fortunately, as I will explain below, I have a trick to still project 2010 MORP even without these win values.

DETAILS OF THE METHODOLOGY

I was very impressed by Sky Andrecheck’s article several months ago at Baseball Analysts, in which he used regression analysis to estimate the dollar value of a win looking at all players with at least six years of service time. As I demonstrated in my three-part series a couple months ago, teams have enough vacancies that the dollar value of a win should be linear, and my data shows this to be the case. Thus, Andrecheck’s use of linear regression seemed like an attractive way to get at the dollar value of a win. He showed the problem with regressing salaries on win values is that actual win values are not representative of what teams thought they were buying—there is some noise there. An important rule of regression analysis is that noise in the independent variable (i.e. measurement error that is not biased in any direction) causes the coefficients to be biased towards zero, while noise in the dependent variable does not cause this bias. The former is called “attenuation bias.” My feeling while reading this article was that Andrecheck had removed this bias by regressing with salary as the dependent variable. Money is money, but expected wins are not actual wins. The problem is that teams aren’t necessarily paying for the wins when they’re getting them. Some contracts are heavily back-loaded, while the win values are heavily frontloaded. The average annual value will be lower than the compensation received for early contract production, while it will be higher than the compensation received for early contract production. In other words, some of Joe Mauer’s 2018 salary will be payment for his 2011 production, and his 2011 salary will not be the full payment for that season's production. Thus, even regression with salary as the independent variable biases the coefficients towards zero, too. Thus, I believe Andrecheck did overestimate the dollar value of a win using his method slightly. Checking the regression analysis against the method I used, I find that the dollar value per win would have been $1-2 million higher than the amounts I found. This smells a lot like obvious attenuation bias.

Instead, since wins are priced linearly, I simply was able to get the dollar value of what was spent on free agents divided by the win value of what those players provided, while netting both values of the draft-pick compensation effects. To be specific, I used average annual value of contracts instead of the actual dollars spent in a given year, although this did not change the numbers much except for by about $200,000 per win in 2007 and almost nothing in 2008 or 2009.

However, I did make an exception for players with less than six years of service time who signed contracts that bought out free-agent years. For them, I simply used the fraction of the contract that occurred after the player reached six years of service time as a separate contract, and computed the average annual value of that subcontract.

2007-2009 MORP FORMULAS

The result gave me the following formulas, representing the MORP for players in 2007-09. Note that an adjustment needs to be made for players who did or would have led to draft-pick compensation by their old teams:

2007: MORP = $4.6 * WARP
2008: MORP = $5.1 * WARP
2009: MORP = $4.9 * WARP
(in millions)

As a source of validation, note that GDP decreased 2.4 percent in 2009 and the decrease in MORP was 3.9 percent. Since baseball tickets are a luxury good, and since the labor share of revenue is roughly constant over time, it makes sense that the baseball labor market saw a decline that was similar but slightly greater than the decrease in overall productivity in the United States.

Also note that had compensation been ignored, these formulas will be $4 million, $4.4 million and $4.4 million times WARP for these three years, meaning that free agency-eligible players would have gotten 16 percent, 15 percent and 12 percent more money over the last three years on average had there been no draft-pick compensation.

Also, note that these formulas are unique to WARP. There are 1,250 WARP3 allotted each year. A slightly higher replacement level like Fangraphs’ WAR has 1,000 WAR, meaning that each of these numbers would need to be 25 percent higher ($6.1 million per win). Similarly, the price of Sean Smith’s WAR would be (1,250/865) times these values ($7.1 million per win). Fangraphs estimates its own dollar per win at about $4.5 million, but this is because they have not added the approximate 15-percent due to draft pick compensation nor adjusted for the declining performance towards the end of deals which would add another 20 percent onto the contract cost. Using average annual value for production that is not expected to be the same in each year of the control is going to come up with biased results.

2010 FORMULA

After spending a lot of time thinking about to reconcile the fact that (A) I think player projections are a bad way for developing MORP, and (B) People are going to want estimates of how much players will be worth in 2010 or at least how to evaluate them at the end of the season, I found a neat solution.

While we really don’t know how individual players will do, we know that the total WARP provided by players with six years of service time is always about 30-35 percent of all WARP and that WARP always adds up to about 1,235-1,275 combined in a season So I took the fact that the last three years of WARP provided by these players is 427, 404, and 383, calculated a weighted average of these and got about 400 WARP3 as the aggregate amount that all these players will generate, and then used the same methods as before to approximate the amount of salary (now that everyone who is going to be making enough money to actually change these numbers is already signed) and amount of wins that need to be subtracted for draft picks’ WARP, and determined got the formula for 2010 as:

2010: MORP = $5.0*WARP (in millions)

The numbers at other websites using similarly methodology have suggested that the price of a win has gone down in 2010. However, this has a lot to do with the biggest free-agent contracts of the offseason going to players like Roy Halladay, Jason Bay, John Laockey and Chone Figgins, who are probably not as likely to age well as previous years’ big contract recipients like Alex Rodriguez, Teixeira, and Sabathia.

CONCLUSION

Working through these examples, it should be pretty clear that there is a lot of approximation involved in doing this analysis. Although this is a careful approximation of the market value of wins, there are other factors that cannot be captured in this framework. The Fernando Rodney signing by the Angels looks exceptionally foolish until you consider the fact that it decreases the chance that the Halos need to deal with Brian Fuentes’ 2011 option vesting if he continues to struggle. Those potential savings in that case make that signing look stronger. Bengie Molina might be a questionable sell to Giants; fans, but since they sent Buster Posey down to Triple-A, the Giants will likely get one additional year of service time from him before he becomes eligible for free agency. A discount on Posey’s 2016 season is a not- so- hidden benefit of that signing.

Still, there is a clear framework here that allows for a reasonable analysis of contracts. Including the draft-pick compensation gives a more accurate portrayal of how the market prices talent. Including all players who reached free agency instead of just those who were not re-signed by their old team raises the estimates of the dollar value of a win as well. Making each of these changes may seem small, but it changes the value drastically, and gives a far more accurate portrayal of value and a better starting point for evaluating transactions.

The most important thing is to make sure that free-agent draft pick compensation is part of any process to evaluate compensation. Glossing over this would be as dangerous as ignoring taxes when looking at labor markets, because that is exactly what it is. This would also provide a solid framework for studying any adjustment to compensation that might arise if the next collective bargaining agreement eliminates this kind of compensation. Free agent dollar compensation would likely rise by about 10-20 percent if this were to happen. This should also give a more accurate framework for evaluating trades. While it is important to make adjustments for in-season deals—because the value of a win is more important to teams in playoff races than to the average competitive team at the beginning of the season—using this framework will allow a more mathematical discussion of what happens when a trade takes place. The reality is that it is particularly limiting to look at trades without knowing how to value the contracts of the players, because baseball trades are really just trades of labor contracts. This is not the final answer on how to value players but it certainly should provide a reasonable framework for any discussion. Numbers will not explain everything there is to know about a transaction, but they sure help, this is at least a realistic baseline.