FanGraphs WAR and Positional Scarcity

Buster Olney caused a stir on ye olde Internets when tweeting his skepticism of the Wins Above Replacement (WAR) stat. Many responses were snarky and derisive, few were educational. Dave Cameron took it upon himself to do some teaching with a post at FanGraphs. If you have any confusion as to how WAR works, I cannot recommend enough that you click over and read Cameron’s article immediately. In fact, I’ll link it again -> here.

A passage I found insightful:

[Ben] Zobrist, on the other hand, is an excellent defensive second baseman and a terrific baserunner, and we’ve got him at 11.5 runs better than average between those two aspects of the game. While he doesn’t have Fielder’s power, he is much faster and more athletic, and is able to use those skills to more than close the gap once defense and base running are factored in to the overall package.

By solely focusing on offensive metrics – especially things like HRs and RBIs, which are heavily skewed towards power hitting first baseman – and not looking at the position averages at each position, the sport has had a long history of overvaluing Prince Fielder’s specific player type. The Ryan Howard extension is a perfect example – he’s something pretty close to a league average first baseman, but he’s getting paid like a superstar.

However, teams have begun to learn from their mistakes. Look at the relative salary difference that guys like Carl Crawford and Jayson Werth commanded last winter, compared to a bat-only guy like Adam Dunn. By nearly any measure you want to use, Dunn was the superior hitter (until this year, anyway), but he got nearly $100 million less than Crawford because teams realized there was more to the the game than standing at the plate and swinging for the fences.

FanGraphs WAR (fWAR) has not been proven to be the most accurate depiction of a player’s value. Along with Baseball Reference WAR (rWAR), the two have the most credence as they are backed up by (mostly) objective measurements and a transparent process. As Tango recently pointed out, “everyone has their own WAR”. The challenge is getting everyone to spell it out and be consistent with it.

Olney is not wrong for being skeptical and he should not have been lambasted the way he was minutes after his round of tweets. When we start blindly accepting Sabermetric tenets, the science stagnates. Sabermetrics cannot improve without skeptics who look at stats, ask questions, and seek improvement.

Leave a Reply

*

31 comments

  1. TH

    August 08, 2011 07:55 AM

    Yeah, that FanGraphs article is brilliant. I think I might keep that in my bookmarks forever to link to the SABR-haters, since it’s pitch perfect.

  2. SH

    August 08, 2011 10:20 AM

    Cameron writes:

    “Fielder is certainly a great hitter, but there are a lot of great hitting first basemen. He plays the same position as Albert Pujols, Joey Votto, Adrian Gonzalez, Miguel Cabrera, and Mark Teixeira, among others. Just those seven are all considered elite, franchise cornerstone type players”

    He lists six first basemen whom he then describes as “those seven.”

    If you you are trying to make a persuasive argument in support of your statistical analysis, demonstrating an inability to distinguish six from seven would seem to undermine your argument pretty badly.

    Not so brilliant, in my opinion.

  3. conshy matt

    August 08, 2011 10:29 AM

    i rarely post here, but i read your site frequently Bill (and even link to it often on other sites). well written article by DC, and while i don’t hold WAR to be gospel, this article should give pause to any sabre haters out there. of course, i doubt many of them will read it so it’s kinda like a tree falling in the woods…

  4. Richard

    August 08, 2011 11:02 AM

    hey, SH, I think that’s called a typo, or a mistake; settle down.

  5. KH

    August 08, 2011 11:03 AM

    I don’t know how you can put Tex in there with those other First baseman. Over his career he has proven imo to not be in those other first baseman’s class. I dont mind WAR as a stat but I have problems wrapping my head around the baserunning and defensive parts of the stat. They would seem to be the hardest things to quantify and for some players it is almost a ridiculous ammount of there value.

  6. SH

    August 08, 2011 11:15 AM

    Richard,

    So we’re taking it on faith that there are no similar mistakes in the math that is the whole basis of the article?

    It’s a good article, and I agree completely with the premise (though I’m extremely skeptical of the defensive component of WAR). But if someone were to read this and say “this guy can’t count to seven, why should I listen to anything he has to say about numbers”, they would be completely justified. Which is a shame.

  7. conshy matt

    August 08, 2011 11:46 AM

    that is funny, SH, and i had the same thought as i was reading the article. but i attributed it to either a simple mistake.

    agreed that the UZR component of WAR makes it inherently flawed. it tends to hit more often than it misses though.

    just browse through the WAR leaderboard for hitters, then cross reference that vs. the wOBA leaders. only 3 of the top 15 WAR players are missing in list 2, all 3 known defensive liabilities.

  8. Richard

    August 08, 2011 12:03 PM

    “But if someone were to read this and say “this guy can’t count to seven, why should I listen to anything he has to say about numbers”, they would be completely justified.”

    No, they wouldn’t, because that’s a moronic way to read that kind of mistake. A better way to respond to it (since I see you posted the same comment at Fangraphs) would be to say, “Dave, I like the article, but you made a mistake here”, etc. What’s so hard about that?

    “So we’re taking it on faith that there are no similar mistakes in the math that is the whole basis of the article?”

    This is just obnoxious. Nothing I said should imply that I think this. The one you highlighted is clearly just a mistake, typographical or otherwise. The rest of the article should be judged on its own merits.

  9. Phillie697

    August 08, 2011 03:13 PM

    @KH,

    Is your opinion on base running and defense being hard to measure based on any objective analysis or basically because you can’t do it? I’m not saying the method used in WAR for those two stats are perfect, but to say they are the hardest to quantify therefore we shouldn’t even bother is why players like Fielder and Howard HAVE been over-valued all these years.

    Just because something is harder to do doesn’t mean you put the blindfolds on and pretend they don’t exist. That’s worse than using imperfect statistical evaluation.

  10. VonHayes

    August 08, 2011 04:24 PM

    @Phillie697

    I got the impression @KH was just saying fielding and baserunning stats are more difficult than hitting to calculate, not that they aren’t worth quantifying. Pretty much anyone who knows sabermetrics would agree that hitting value is easier to quantify.

  11. Phillie697

    August 08, 2011 05:21 PM

    @Von Hayes,

    “I don’t know how you can put Tex in there with those other First baseman. Over his career he has proven imo to not be in those other first baseman’s class.”

    He sure does seem to suggest Tex’s superior defense means nothing when he compares Tex to the rest of the other elite first basemen. Based on WAR Tex has performed just as well as those other 1Bs.

  12. Rob

    August 08, 2011 08:14 PM

    I found Buster’s tweets interesting, not because he was off base but because of the basis of his argument – ask any GM who they’d want to have and they’d choose the power hitting 1B. That may be true, but certainly doesn’t make them right. Seems to me the key to getting a more universally accepted WAR (besides just changing people’s habits) is a reliable way to measure defense. So far good defense is kind of like art – you know it when you see it, but try defining it…

  13. Nick

    August 08, 2011 09:44 PM

    KH is talking about how he doesn’t understand (and I’m guessing Olney is in the same situation) how defensive statistics can be counted. It is almost the same as in football, I’d imagine, where placing yourself in the right spot means the QB will not throw to you (Nmadi Asomough, top free agent this offseason is a great example), and therefore you don’t get big statistics but are a valuable player.

    I disagree with that stance (in baseball at least), but that is how I think people like Buster Olney see it.

  14. Phillie697

    August 09, 2011 11:13 AM

    @Nick,

    Then my original post applies:

    “I’m not saying the method used in WAR for those two stats are perfect, but to say they are the hardest to quantify therefore we shouldn’t even bother is why players like Fielder and Howard HAVE been over-valued all these years.

    Just because something is harder to do doesn’t mean you put the blindfolds on and pretend they don’t exist. That’s worse than using imperfect statistical evaluation.”

  15. jonk

    August 09, 2011 12:57 PM

    I honestly do no like metrics that compare players to other players. I know there is little else we can do in analysis, but just because I won a race against other fatties doesn’t make me fast.

    What is league average anyway? If I put Victorino at first base, does that skew defensive stats (even slightly). What if I put Howard in CF (though hitting the cutoff man might be tough)?

    Just because a player plays at one postion doesn’t mean he couldn’t excel at another. Maybe he provides “less” value to his team, but I have always disliked the notion of combining analytical offensive measures with subjective and inconclusive defensive measures to get a “number” that represents a player’s value.

  16. Phillie697

    August 09, 2011 03:49 PM

    @jonk,

    Well, this is a competitive sport, where 25 guys from one team compete with 25 other guys from another team each game, and compete with 30 groups of 25 other guys overall. Comparison is the only way to do it; if every team has a Barry Bonds at OF and all you can buster is a Hunter Pence, your team probably will suck even tho Hunter Pence isn’t a bad player in today’s context. This is the reverse of your fatty example.

    Howard at CF would have way more problems than hitting the cutoff man. He would have trouble getting to many of the flyballs hit to CF, period. And while the defensive metrics out there are not perfect, I would not call them subjective nor inconclusive. Again, you’re just ignoring them for the sake of ignoring them, and end up over valuing Fielder and Howard.

    And to answer your question, yes, moving Vic to 1B will skew his defensive stats, because even if he can play 1B at an exceptional level the way he plays CF, the fact that there are just more players who are capable of playing 1B well makes him less valuable to begin with. Is that fair to him? I don’t know, but then again, why the hell would the Phillies management put him at 1B? There is obviously the assumption that teams will put the best players at the most difficult positions; if you have a player who can play exceptional SS AND 1B, why in the world would you put him at 1B?

  17. FC

    August 09, 2011 04:30 PM

    In the end all statistics have flaws, we use the tools that allow us to analyse and try to predict player performance according to other metrics. Even the metrics themselves can be flawed.

    But that’s fine, that’s as it should be: Baseball has some qualities that will never be quantifiable or written in a stat-sheet: heart, team chemistry, and those moments when a player rises to the occasion and defies his own stats. Who would have predicted Vogelsong would have been named an All-Star?

    Or even the opposite, when they lose it and can’t explain it. What did sabremetrics predict Adam Dunn was going to hit this year? Jayson Werth? I loved how Dave Shoenfields article used a lot of advanced metrics to predict the Phillies demise. He never accounted for the fact that other players could step in and perform: Stutes, Bastardo, Worley. I’m sure he had no idea Utley would be back in May and playing hard since. Or that Victorino would be having the best season of his career.

    That’s why I love baseball beyond statistics, it’s capacity to surprise you is boundless.

  18. Bill Baer

    August 09, 2011 05:24 PM

    FC,

    Projection systems have never promised 100% accuracy. Most will give you a confidence interval, which is an average +/- X 90%/95%/99% of the time, etc.

    If we assume the accuracy of projections is normally distributed, the projections will be accurate within one standard deviation 68% of the time, two standard deviations 95% of the time, and three standard deviations 99% of the time. The trick is reducing the variance in the projections, always an ongoing process.

    You can cherry pick a couple of players like Dunn and Werth, but they’re part of the process as well. And in the end, all you’re doing is making a very bad ad hominem argument.

  19. Andrew

    August 09, 2011 05:31 PM

    New to the “WAR” game and I am trying to understand how this works. Could someone please help with an explanation:

    Last year, Wilson Valdez served as a replacement utility player filling in for Chase Utley and Jimmy Rollins battling injuries, playing 111 games. Ryan Howard had an injury plagued year, his worst in the majors, but, still batted .276/.353/.505.

    Ryan Howard’s 2010 WAR: 1.3
    Wilson Valdez’s 2010 WAR: 1.0

    Position modifiers, fielding, base running, hitting has already been taken into account in their WAR. As I understand it, a replacement player is the same player playing any position (not a replacement first base man or replacement second baseman).

    Consider the following situation:
    Team A: 2010 Philadelphia Phillies with Valdez as a utility player (1.0) and Howard as a First Baseman (1.3) = 2.3 WAR

    Team B: 2010 Phillies with Wilson Valdez (1.0) as a utility player and a “replacement player” (0) at first base instead of Ryan Howard = 1.0 WAR

    Team C: 2010 Phillies with a “replacement player” (0) as a utility player and Ryan Howard at first base (1.3) = 1.3 WAR

    Is this statistic saying that the Team C (2010 Phillies w/ Ryan Howard w/o Valdez) would finish with nearly the same amount of wins as Team B (2010 Phillies w/ Valdez w/o Howard)?

  20. Bill Baer

    August 09, 2011 06:07 PM

    Yes, relative to his position, Howard was not much better than a replacement level player. It sounds odd, but consider that Howard’s .858 OPS was barely higher than the league average .813 OPS at first base. Then remember that Howard is at best an average fielder and a bad base runner.

    FanGraphs WAR uses UZR to create defensive value, and that is where the most skepticism should lie. UZR stabilizes between 2 and 3 seasons’ worth of data, but only one (or less, if you’re looking during a season in progress) is included in WAR.

    Additionally, FanGraphs docks Howard for playing a non-premium position and credits Valdez for playing a premium position. In 2010, Howard lost 10.8 “positional” runs while Valdez gained 3.0.

    If you’re interested in reading more, I wrote two posts on Howard’s WAR specifically last year.

    crashburnalley.com/2010/09/11/understanding-ryan-howards-war/

    crashburnalley.com/2010/09/16/more-on-ryan-howards-war/

  21. Phillie697

    August 10, 2011 12:03 AM

    @Bill,

    Well, while I do think one season’s worth of UZR probably isn’t enough to be indicative of a player’s true defensive capabilities, but as everyone who understands WAR knows, WAR is not a predicative stat; it’s a descriptive one. I don’t think it’s wrong to include whatever the in-season UZR is for that particular player, because that’s actually what happened on the field. The problem is when people want to use that UZR to predict future performance, and yes, that’s when it’s better to look at multiple season’s worth of data.

    Now, an argument could be made that UZR itself isn’t an entirely accurate measure of what truly happened on the field to begin with, and that to me would be a better argument to make when debating the accuracy of a player’s WAR as a measure of what he has in fact contributed.

    For example, I wonder just how entirely accurate UZR is on outfield defense. I mean, I’ve seen so many times where two OF get to the same fly ball, and most of the time of course the corner OF “yields” to the CF. How is that play scored in UZR? What if it’s just a REALLY high fly ball that even a terrible CF can get to, but nevertheless it’s outside the “normal zone” in UZR for CF, hence that CF would end up getting a + rating on that particular play? What if you have a particularly fast CF who can “intrude” into the “normal zone” of a corner OF, and the corner OF “yields” to the CF? The fact that CF got there really didn’t help the team; the corner OF would have easily made that play as well. But wouldn’t the CF be credited in UZR for incredible range even tho it didn’t really help his team much, if at all? How does UZR account for those situations?

  22. Bill Baer

    August 10, 2011 12:12 AM

    Well the way UZR works is the field is divvied up into 70 or so zones, and then each out and each chance in each zone is compared to the league average. When you divide the field up into 70 zones and include only a partial season’s worth of data, you are dealing with really small sample sizes. As such, the effect of good or bad luck, even on one or two plays, is magnified and can have a huge effect on the UZR result.

  23. FC

    August 10, 2011 12:26 AM

    Bill,

    Not one thing you have said actually counters what I said. You have simply decided to tell me that I cherry pick and that I make ad-hominems rather than acknowledge a simple truth: these wonderful stats, will still fail from time to time. That’s all I’m saying. And for that any single exception proves the rule.

    Stats are nice and they will help predict player performance most of the time, but I still like it when Baseball surprises you and defies statistics. I’m not saying they fail 100% of the time. As a matter of fact they are probably right 95% of the time. If the stats were right 100% of the time there would be no surprise, no hot streaks, no excitement, baseball would be predictable and boring, what would you prefer? Have sabremetric stats that predict game results with 100% accuracy? Or perhaps some nice tension, a close game, uncertainty? Maybe see that 0-25 guy hit the grand slam and be a hero. How about Dan Uggla having a horrible season and now a 30 hit streak?

    These are the little things stats can’t predict or quantify and why I watch the games and enjoy them.

    As for David’s article, you have to admit the way he went about it was laughably biased and bad. As a matter of fact you should have admonished him for giving stat-geeks a bad name.

  24. Phillie697

    August 10, 2011 12:27 AM

    @Bill,

    I agree, but like I mentioned, UZR does inherently favor good CFs in my opinion. I think while Brett Gardner is REALLY good defensively, I don’t think he’s as godly as UZR thinks he is because of the “yield” effect I described above. This probably applies to Vic too…

    Obviously overlapping can result in the in-field too, but I would venture to guess it happens way more often in the outfield than the infield; plus, most of the time, in the infield it’s whoever gets there first, whereas the outfield there is a sort of pecking order involved.

  25. Bill Baer

    August 10, 2011 12:30 AM

    Actually, FC, I basically agreed with you. I just illustrated generally how accurate projections are (which is, by the way, significantly more than by using your eyes or your gut).

    Not sure what your problem was with Schoenfield, especially since you’re advocating baseball is full of surprises and we can’t predict anything 100%.

  26. Bill Baer

    August 10, 2011 12:32 AM

    @ Phillie697

    That’s a good point, though I can’t add anything to that besides my own speculation. I’d like to see someone with database skills see if that is actually the case, and if so to what extent.

  27. jonk

    August 10, 2011 08:51 AM

    @Phillie697

    The point I am emphasizing is that position player’s positions are estimations and nothing more. Chase Utley is a good second baseman. Does that mean he wouldn’t have been a good CF or a good first baseman? Of course he may have, and accordingly, adjusted the metrics for that position. But he doesn’t play that position. So, because of someone’s estimation, Chase is playing 2nd and skewing the data at that position rather than somewhere else.

    My premise is that we put way too much value on these newfangled defensive stats that have so many variables, they piss me off. I understand that we “think” they pass the smell test, but I don’t buy the value of them.

    We need Voros McCrakin jr to come in and show that defensive plays have no bearing on the ability of the defender and it is all just luck.

  28. Phillie697

    August 10, 2011 12:10 PM

    @jonk,

    So your argument for the alternative is to IGNORE defensive abilities and just measure a player purely based on the easier to measure offensive numbers? And this is somehow more desirable to you? Really?

    Last time I checked, hitting the ball is only 1/3 of what a baseball player does. To suggest that we should measure their worth only based on 1/3 of what they do is, pardon my language, stupid.

    This is still a game of competition. We are not trying to measure how good of an athlete a player is; come up with your measurements if you are interested in that. We are just interested in coming up with a number that as accurately as possible measure how the player has contributed to the team’s win-loss record. So yeah, sorry, comparisons matter. When you compete, you WILL be compared to your competitors. It’s a competition after all.

  29. jonk

    August 10, 2011 01:03 PM

    @Phillie697

    You are making this much more simplistic than it really is and over complicating it at the same time.

    “So your argument for the alternative is to IGNORE defensive abilities and just measure a player purely based on the easier to measure offensive numbers? And this is somehow more desirable to you? Really?”

    I am not talking about ignoring defensive abilities. For the most part, I don’t think there is a HUGE difference between most defensive players. Even if you focus on replacement level defense and then figure out the incremental differences after, it is not nearly as large as we think. I’d say that the difference between a great SS and an average one is maybe like 1 play every 3 games. Most players can make most plays effectively. Defensive measures overvalue the differences.

    “Last time I checked, hitting the ball is only 1/3 of what a baseball player does. To suggest that we should measure their worth only based on 1/3 of what they do is, pardon my language, stupid.”

    Well, what does a baseball player do? Most of the time he stands around and does nothing, wether it is waiting for his turn at bat or waiting for the ball to come to him. 27 outs, 8 fielders (ignorning the catcher). Lets say between strikeouts, double plays and other outmakers there are 18 available outs for the fielders (ignoring making plays on hits and other activities in the field). So that is roughly just over 2 outs per fielder per game (also ingorning the 1st baseman’s catching). It’s not much. Maybe 4 plays a game if they are lucky of which most players can handle all 4.

    Then there is hitting. Hitting is certainly worth much more than 1/3rd of what a player does. Not all plays in the field are going to lead to runs and the pitcher has the most impact on what the defense needs to do. As a hitter, though, he can impact the score much more, possibly hitting up to 5 home runs a game. Obviously that is an extreme, but it makes my point.

    Is running the bases REALLY 1/3rd of what hitters do, or am I missing a part of the equation here.

    Think of this as a race. How fast do you need to be to win? Faster than the second fastest guy? What if you are much faster than him and win the race, but someone running a different race that came in second is faster than you. Just because he is running the 200 and you run the 100 doesn’t mean that he is faster, or better or couldn’t run the 100 as good. It is just silly to rate guys on the arbitrariness of where they play. You can only have 1 shortstop on your team. Is it anyone else’s fault that they can’t all play shortstop? The fact that Wilson Valdez’s WAR was 1.0 last year and Howard’s was 1.3 is absurd. There are way too many apples and oranges in this basket for me to think we can compare anything of value here. Yes, I know there are a lot smarter guys than me figuring this stuff out. And they all are looking at the trees and are ignoring the forrest.

  30. Phillie697

    August 10, 2011 04:03 PM

    @jonk,

    We’re going to have to agree to disagree. You are obviously one of those people who still think that offensive capabilities are more important than anything else to the point that it’s statistically insignificant to worry about base-running and defense. I won’t have the time to convince you otherwise, but here’s something to consider: one play every 3 games, even as you say, is 162/3 = 54 plays. Imagine ONE position out of nine on your team giving up 54 more hits over the course of the season… Yeah, that’s not important or anything.

    As for Howard v Valdez… I think you should worry about how Howard only accumulated 1.3 WAR over the course of one season than worry about Valdez’s 1.0 WAR. 2.0 WAR is traditionally considered a league average player. Valdez was a below-average player according to WAR, which is exactly what he is. It’s HOWARD you should be alarmed about. No offense, but I don’t think you realize just how much of a shell of his former self our $125 million 1B really is. And if you want to blame WAR again, may I remind you how the same system calculated the WARs for some of the other 1Bs in the league in 2010… Joey Votto, 7.3 WAR; Albert Pujols, 7.5 WAR; Adrian Gonzalez, 5.2 WAR; Miguel Cabrera, 6.3 WAR; Prince Fielder, 3.4 WAR; Mark Teixeira, 3.3 WAR. Ryan Howard is just not that good anymore, accept it.

Next ArticleShane Victorino's Ridiculous Season