The Intersection of Ignorance and Arrogance

In the great “stats vs. scouts” debates we’ve had over these many years, Saberists have often been labeled as arrogant know-it-alls. Some of it was warranted; some statheads were, and still are, too confident in fallible methods of objective measurement. However, it is typically grounded in the scientific method, by definition the least arrogant method of observation out there. For those, like me, who have long been separated from their last science class, the steps to the scientific method are:

  • Ask a question. E.g. “Are strikeouts strongly related to a pitcher’s success?”
  • Form a hypothesis. E.g. “Strikeouts are strongly correlated to a pitcher’s success in both the short- and long-term.”
  • Make a prediction. E.g. “Strikeouts will be strongly correlated to a pitcher’s success at a statistically significant level.”
  • Test your hypothesis. E.g. finding the strikeout rates and earned run averages of all pitchers who threw at least 150 innings in a season dating back to 2007 and determining the correlation between the two.
  • Analyze the results. E.g. “There is a statistically significant relationship between strikeouts and same-season ERA. r = .87, p < .05.”

Above examples and figures are completely made up.

Adhering to the scientific method is the least arrogant way one can attempt to answer any of life’s questions ranging from the trivial to those of the utmost importance. It is admitting your fallibility as a human being, realizing that your perception and your memory are too unreliable to take any untested claim as truth. Of course, humans are just as fallible using their memory to provide key details as they are running unbiased, methodologically-sound scientific tests, so one should take any single test’s results with a grain of salt as well.

I bring this up because Sam Donnellon wrote an article for the Daily News yesterday that smacked of old school arrogance.

RUBEN AMARO JR. is right. Moving Chase Utley to third base so Freddy Galvis can play second will improve the Phillies in 2013.

He just can’t prove it.

This is the exact opposite of the scientific method: make a claim, realize there is no evidence nor is there any objective way to validate your claim, and then declare it correct anyway. “Kim Kardashian is right. When it rains, it is really just God crying about your sins. She just can’t prove it.” Belief in the absence of facts is not only foolish but incredibly arrogant, to state matter-of-factly that your experience and perception not only should but does match that of everybody else.

Donnellon spends some time in the article talking about the lack of reliable defensive statistics, and he isn’t wrong. Sabermetric defensive statistics are far behind offensive statistics in terms of reliability. Colin Wyers (@cwyers) of Baseball Prospectus fame has done yeoman’s work over the years pointing out the fallacy of relying on subjective classification of batted balls. In other words, one scorer’s line drive is another’s fly ball. Of course Donnellon’s criticism is much less nuanced than Wyers’, but it is easy to see where he is going. Being skeptical of defensive statistics is completely warranted.

What many old school types have done in the past, when confronted with a new statistic that they don’t trust, is to find the biggest outlier and use that as a justification to discard the statistic entirely and return to the older, more familiar (and less illuminating) statistics of the past. For instance, Wins Above Replacement has only recently become an accepted part of the mainstream baseball lexicon. It wasn’t that long ago that writers dismissed it out of hand because Mark Reynolds earned a higher grade, according to the statistic, than Ryan Howard. WAR certainly has its problems (reliance on single-season samples, subjectivity of human scorers, etc.) but just because it isn’t a perfect statistic, you don’t revert back to the problematic methods that got us here in the first place. It’s like saying, “Chemotherapy isn’t guaranteed to cure my cancer, so I’m just going to go back to using mercury.”

Later in the article, Donnellon romanticizes the old school scout types like Pat Gillick. There is certainly nothing wrong with saying that the subjective has a legitimate place in baseball; objective measurements have not and will never come close to encapsulating all of the known variables. However, it is clearly wrong to defer, unquestioningly, to them on all subjects. Pat Gillick is one of the brightest baseball minds the game has ever seen, but I’m always going to be skeptical of anything that he — or anyone else — says until it is tested empirically. To uncritically take a statement as fact is ignorant; to pass that deference off as an admirable quality is arrogance, and a particularly offensive arrogance that some old school types have displayed when their preconceived notions have clashed with Sabermetric revelations.

Towards the end of his article, Donnellon writes:

These are plays not measured in any comprehensible statistics, although those scripting for something called “The Fielding Bible” have given it an honest effort. But all baserunners do not have the same speed, guts or baseball IQ. Strength of arm is not just about putouts, but by a lack of attempts to run on that arm. They have tried to measure range, but these are not dynamic or even easily understandable stats.

Essentially, Donnellon says in the above paragraph that some smart people have attempted to quantify defense, but it is imperfect because it fails to consider the several listed variables. Donnellon takes the absence of evidence as evidence of absence (of quality), and uses that to revert to his preconceived notion that the eye test is still the best method of defensive evaluation. What he should have done, instead of lazily clinging to old ideas, is talk to John Dewan or any of the other smart minds who work on The Fielding Bible. If they weren’t aware of Donnellon’s criticisms already, they likely would have welcomed the criticism and considered it deeply. Or he could have talked to Sean Forman of Baseball Reference, UZR creator Mitchel Lichtman, or any number of other smart minds in the baseball research arena. People like Donnellon are important to the progress of Sabermetrics because many prescribe to the doctrine and thus are biased and less likely to be aware of methodological flaws.

Instead, Donnellon firmly entrenches himself in the soil of old ideas, closing out his article:

The good news for you people is that Amaro, whose resume is still without a championship and thus incomplete, seems to get it, too.

I can’t tell you how many runs will be saved in moving Utley to third and having Galvis at second for a season.

But my eyes can tell he’s right for doing it.

Human perception and memory have been scientifically proven, time and time again, to be unreliable and often simply inaccurate. Furthermore, their knowledge of how the human brain processes and retains information is often lacking. As a result, they give far too much credit to our ability to pick out, process, and remember key information. For instance, in this study, all 16 experts disagreed with the statement “human memory works like a video camera, accurately recording the events we see and hear so that we can review and inspect them later.” 63 percent of the 1,500 regular people polled agreed with the statement.

Rather, statistics and “the eyes” should act as tools in a tool box, ready to be used at any particular moment for a job that they are specifically useful in completing. Statistics, compared to the eyes, will be the best choice when trying to determine exactly how much less impactful Chase Utley’s offensive contributions will be at third base than at second base. On the other hand, the eyes will be the best tool for the job when you’re watching Roy Halladay throw a bullpen and notice something isn’t quite right. Even if Pitch F/X was available for bullpens, we may not be able to notice anything if velocity, movement, and release point remain unchanged, but a trained professional like Rich Dubee will be able to see that Halladay is laboring and shouldn’t make his next start.

As it happens, the more accurate our objective measurements become, the less reliant we are on experts who utilize subjectivity. Thanks to amazing advances in medical knowledge and technology over many years, we can quickly and easily run a test to see if a woman is pregnant. No longer do we have to inject urine into female rabbits to reach a conclusion. The progress of and increased reliance on Sabermetrics to answer questions is emblematic of yet another area in which the scientific method has increased our awareness and knowledge where none existed previously. That progress should be celebrated and anyone that tries to tell you otherwise is attempting to sell you swampland in Florida.

Those who have firmly dug their heels into the ground may learn from Plato’s “Allegory of the Cave“. (Warning: The video below has some repetitive flashing lights, which may be bad for those of you with epilepsy.)

Leave a Reply

*

12 comments

  1. Gaël

    September 25, 2012 07:15 AM

    “I can’t tell you how many runs will be saved in moving Utley to third and having Galvis at second for a season.

    But my eyes can tell he’s right for doing it.”

    Wait, how does that even begin to make sense when no one’s even seen Utley play at third in a real game in like 8 years?

  2. Jordan

    September 25, 2012 07:27 AM

    Also confirmatory bias- we seek information that confirms the conclusions we’ve made. Changing our mind about something is generally a last resort, since we generally don’t want to think that the energy we’ve invested in a cause or belief are wasted.

    That being said, I’ve seen dogmatism on both sides.

    An important step between asking the question and forming a hypothesis is operationalizing your constructs. What is “successful pitching?” How is it measured? I’m not trying to be pedantic; this seems to be a crucial part of the debate, though admittedly it is probably more of an in-house discussion.

  3. tyler

    September 25, 2012 09:07 AM

    FanGraphs had a fairly decent post about the possibility of moving utley to 3b that didn’t mention any defensive stats at all. Donnellon would have done well to reference that a bit more. Neither stats nor scouting will tell us whether an Utley at 3b/Galvis at 2b experiment.

    donnellon’s article is flawed mostly in that it has no real thesis at all. He is absolutely correct that there are things stats can’t measure about defense– having a certain player in a certain position can change batter and baserunner strategy, and stats and scouting both have lots of room to grow in evaluating player psychology above and beyond tools and skillsets. But what is Donnellon actually saying? That stats don’t give us the whole picture on defense, so we might as well trust the front office? He could replace ‘stats’ with ‘scouting’ and come to the same conclusion.

    I do think there is a strong case for moving Utley to 3rd and keeping even Galvis at 2nd, or even putting Utley at 3rd a few games a week. A galvis/Rollins MI does give the team a fairly strong defense up the middle, which could impact the decision they make about who to acquire for CF. And maybe 3b would be easier on Utley’s knee, providing him more playing time.

    If the team moves forward with this experiment in the 2013 season, I’ll probably be one of the people thinking it will work. But I’m not going to pretend that’s because I feel like I understand something the defensive metrics miss, or that I understand scouting at all. I’m just a fan who wants to trust the decisions the team makes even when those decisions don’t immediately make sense to me.

  4. tyler

    September 25, 2012 09:10 AM

    i’m going to pretend that my poorly worded post above, where half my sentences are unfinished, is because I’m practicing becoming a sportswriter and not just because I’m lazy

  5. pedro3131

    September 25, 2012 09:58 AM

    I think the major problem is most people confuse correlation with causality. You used correlate in this article, but too often do I think we say there’s a causal relationship when often there are other non spurious factors at play.

  6. Richard

    September 25, 2012 11:26 AM

    Tyler, you say:

    “I do think there is a strong case for moving Utley to 3rd and keeping even Galvis at 2nd, or even putting Utley at 3rd a few games a week. A galvis/Rollins MI does give the team a fairly strong defense up the middle, which could impact the decision they make about who to acquire for CF. And maybe 3b would be easier on Utley’s knee, providing him more playing time.”

    Given that you add the “easier on Utley’s knee” at the end, I’m going to assume that’s not why you think there’s “a strong case for moving Utley to 3rd and keeping even Galvis at 2nd”. What, then, is that strong case, in your view?

    Keeping in mind that Utley has 7 years of being an elite defensive second baseman under his belt, and that this year Galvis only approximated Utley’s value for this year (which is to say, their defensive value was approximately equal, according to our available small samples).

  7. Joe

    September 25, 2012 11:42 AM

    My eyes tell me that Utley sometimes has trouble making a strong/accurate throw from 2nd base and that his arm would likely be a liability over at 3rd.

  8. Jesse

    September 25, 2012 12:46 PM

    Good post, though I’m not sure we should always assume that the scientific method is the least arrogant way to acquire knowledge. It depends on the field and question we’re talking about. Baseball lends itself immensely well to statistical study because of large sample sizes, discrete individual-performance measures (plate appearances, pitches, etc.), and ease of identifying positive results (winning, home runs, etc.).

    But not all problems work this way. Education is, to me, the most fascinating and important example. I think it’s immensely important that we improve the quality of teaching, and the lack of rigorous study of the subject until the past decade has been a huge problem. On the other hand, many statistical analyses try to wish away the problem by simplifying it. Showing which teachers get students to improve the most on standardized tests may or may not tell us very much about who the good teachers are. As with baseball defensive metrics, there are all sorts of methodological challenges — Are the tests well constructed? Is the sample big enough? — but there are also philosophical ones. What is the goal of teaching? To make students as good as possible at math and reading? To help make them better citizens? Both? Neither? Some of these questions are quite amenable to empirical analysis, but some are philosophical ones. And when we try to answer them with statistical analysis, we end up giving the appearance of knowledge without any. (This, it seems to me, is a huge problem in economics, where models often proceed from a starting point of rational behavior that may or may not be accurate).

    I guess what I’m trying to say is that there is a certain arrogance in the scientific method, too: a belief that objective analysis can help us solve a problem that humans have heretofore been unable to solve. Quite often that arrogance is justified. But sometimes precedent, collective wisdom and tradition — for all their warts and flaws — provide both the more prudent and humbler approach.

  9. LTG

    September 25, 2012 03:51 PM

    The scientific method is just a limit case of hermeneutics generally. It applies only to certain kinds of objects, namely, those objects that cannot interpret the results of the scientific method. For those that can interpret the results and incorporate them into their being (Heidegger calls them ‘Dasein’ but we usually just say ‘human beings’ although this term is ambiguous), the scientific method breaks down and must be supplemented by other forms of analysis and argumentation.

    Baseball is an interesting limit case between natural sciences and social sciences, and this is why it seems that the scientific method applies to it so well. On the one hand, the results of baseball studies will fluctuate as the results are applied to the game. On the other hand, since the play of the game is defined by a single value (runs), the game is less susceptible to the evaluative variability of the subject of social sciences, human beings.

  10. LTG

    September 25, 2012 04:05 PM

    I doubt referencing the Allegory of the Cave is going to help the “I’m not arrogant” argument. I don’t care about arrogance, but certainly implicitly comparing yourself to a persecuted wiseman is not helpful.

    Anyway, ironies:

    1. The Allegory is in part a diatribe against images. The cartoon is nothing but an image of the allegory.

    2. The sun in the Allegory represents the Good. According to the superficial Plato I can write up here, one did not know something unless one understood it as good, and bad things are not knowable because they are mere images. The writers of the scientific revolution explicitly rejected this epistemology.

    3. An allegory is itself an image to which the diatribe against images should apply. Should we take the Allegory at face value?

  11. Joey

    September 29, 2012 10:15 AM

    I’m sure that I’m not the first to have thought of this, but is it not possible to develop/employ some function of hang time given distance traveled that reduces the subjectivity (or at least makes it uniform) when classifying outfield hits?

  12. tyler

    October 01, 2012 01:44 PM

    richard– i guess i should have phrased my thought to say “i’m sure there’s a strong case to be made for utley to play 3rd.” for my part, the strongest case I can make is that Miguel Cabrera managed to survive at 3b without hurting the tigers too terribly.

    the other case for utley at 3b would be that he’s better than anyone else the team can get for that position (maybe not defensively, but in general). Although until the center field position has been filled, it’s pure speculation to say what it would do to the team’s win probability to move utley to the corner and put galvis (or anyone else who isn’t utley) at 2b.

Next ArticleChase Utley Back to Form in 2012