Bullpen Shutdowns and Meltdowns

The save — and the blown save — have received plenty of press in Philadelphia over the past two years. Brad Lidge, of course, converted all 48 of his save opportunities in 2008. Last year, he converted only 31 of 42 (74%). Ryan Madson has received some heat from Phillies fans and the media for a seeming inability to convert saves. He has converted four saves in six opportunities, but his last failed attempt resulted in his kicking a metal folding chair and breaking his toe, sidelining him for at least two months. Last year, Madson converted 10 of 16 opportunities (62.5%).

Of course, if you have read any Sabermetric takes on the save, you are aware of its flaws. For one, a reliever can be credited with a blown save prior to the ninth inning, but he can only be credited with a save if he finishes the game in the ninth inning or later. That means that non-closers (like Madson was prior to the last month of 2009) will have save conversion rates that aren’t representative of actual skill. There’s also “The Wes Littleton Save”. Take a look at the score of this game and then look at what is next to Littleton’s name in the box score. The Rangers, Littleton’s team, won 30-3 over the Orioles and Littleton was credited with a save. MLB’s official rules state, “Credit a pitcher with a save when he […] pitches effectively for at least three innings.”

Most Sabermetricians have scrapped the save statistic and for good reason. Many have worked to develop a way to accurately depict a reliever’s skills and contributions. I believe Jeff Zimmerman of Royals Review, inspired by Tom Tango, may be on the right track. For Royals relievers, he logged each pitcher’s “shutdowns” and “meltdowns”. A shutdown is when a reliever increases his team’s chances of winning by five percent and a meltdown is when a reliever decreases his team’s chances of winning by five percent. While the cut-off point of five percent is arbitrary, it is better than using saves (and holds) because it is blind to actual bullpen roles and focuses solely on performance.

There are still problems with S&M (ha, get it?), such as that a pitcher will still be penalized for events beyond his control. For example, if Jose Contreras does exactly what he’s supposed to do and induces a bunch of ground balls, but Wilson Valdez misplays a couple and allows several runs to score, Contreras will still bear the blame. It logs results, not performance. The most you can conclude from this is that, “Player A did or did not contribute to helping his team win in X games,” not “Player A is a good/bad pitcher”.  With that in mind, here are this year’s Shutdowns and Meltdowns. I also included INH% which is the percentage of inherited runners each reliever allowed to score (note that INH% is much less meaningful for relievers with defined roles, such as closer and set-up).

Before we jump to the numbers, you may be asking, “Five percent — what?” Take this chart of last night’s 4-0 win over the Cardinals as an example. Head over to the “play log” and look at the top of the ninth inning. Heading into the inning, the Phillies had a 98.4% chance to win the game. When Jose Contreras got David Freese to ground out to end the game the Phillies, of course, had a 100% chance, a difference of 1.6% or .016. That is Contreras’ Win Percent Added (WPA) to the game. It does not qualify for Shutdown or Meltdown consideration. That is an example of how a reliever increases or decreases his team’s chance of winning a game.

Without further ado, this year’s relief corps:

Pitcher (2010) APP Shutdowns Meltdowns DIFF INH%
Durbin 9 5 0 5 0%
Contreras 10 5 1 4 0%
Bastardo 7 1 0 1 0%
Lidge 3 1 0 1 0%
Madson 9 3 3 0 50%
Figueroa 5 2 2 0 50%
Romero 3 0 0 0 0%
Herndon 10 1 2 -1 33%
Baez 12 1 4 -3 0%

It is interesting to note how evenly-split the “opportunities” are: Contreras and Madson have six, Durbin and Baez have five, Figueroa has four, and Herndon has three. With Lidge and Romero having started the season on the DL, the Phillies were forced to make a bullpen out of a motley crew.

Last year’s relievers:

Pitcher (2009) APP Shutdowns Meltdowns DIFF INH%
Madson 79 35 12 23 0%
Park 38 16 4 12 32%
Eyre 42 13 4 9 23%
Durbin 59 18 12 6 23%
Lidge 67 20 15 5 17%
Romero 21 8 3 5 11%
Walker 32 8 3 5 55%
Moyer 5 4 0 4 0%
Condrey 45 10 7 3 29%
Myers 8 3 1 2 0%
Bastardo 1 0 0 0 0%
Carpenter 2 0 0 0 0%
Escalona 14 1 1 0 25%
Happ 12 2 2 0 20%
Kendrick 7 2 2 0 100%
Register 1 0 0 0 0%
Taschner 24 3 3 0 13%
Lopez 2 0 2 -2 100%

WFC relievers:

Pitcher (2008) APP Shutdowns Meltdowns DIFF INH%
Lidge 72 38 4 34 100%
Romero 81 32 13 19 28%
Madson 76 25 8 17 35%
Durbin 71 28 15 13 33%
Gordon 34 19 7 12 33%
Eyre 19 13 4 9 25%
Seanez 42 10 7 3 21%
Condrey 56 7 5 2 29%
Carpenter 1 0 0 0 0%
Happ 4 1 1 0 0%
Kendrick 1 0 0 0 0%
Eaton 2 0 1 -1 0%
Walrond 6 1 2 -1 25%
Swindle 3 0 2 -2 100%

A tip of the cap, as always, to Tom Tango and Jeff Zimmerman.

Leave a Reply


This site uses Akismet to reduce spam. Learn how your comment data is processed.


  1. Chris

    May 05, 2010 11:32 PM

    Something anyone can understand is how many swings and misses Madson generates from that changeup. I told my friend how it was the most swung on and missed pitch in ’08 and he almost didn’t believe me.

  2. Steve

    May 06, 2010 08:11 AM

    Herndon seems to be the “tough luck loser” in this equation, as he’s throw pretty well in his appearances, yet looks bad on the stat sheet. He’s done what he was supposed to do, which is generate a lot of ground balls, but has had tough luck with placement. A good example is his DP (Double Play) percentage, which is only 36%, according to baseball-reference.com. He’s induced 14DP opportunity balls, yet has had only 5 DPs turned. Other interesting tidbit is Baez. He’s got 12APP, but hasn’t pitched with inherited runners on since the season opener, (he stranded both), meaning his 4 meltdowns are a little more alarming than some other pitchers.

  3. Shooter-B

    May 06, 2010 02:50 PM

    I like the sound of meltdowns as a stat. But that could be too easily confused with “the Miltdown”, which is when a player goes completely Milton Bradley bonkers. Example, on the night of the infamous folding chair showdown…Madson gets credit for 1 meltdown & 1 Miltdown. Throw in blown saves, and that guy is a true triple-crown threat.

  4. Murgatroid

    May 06, 2010 11:05 PM

    Instead of setting a mark at 5%, why not just make a shutdown any time appearance where the pitcher increases his team’s chances of winning at all (and a meltdown the opposite)? It seems that the 5% mark would make it so low-leverage relievers (and probably LOOGYs) could really only accumulate meltdowns. If that assumption is true, you’d have the same problem as with saves/blown saves, just with a different set of relievers.

    Also, wouldn’t the percentage of shutdowns/meltdowns be more meaningful than the differential? And does it really matter that this stat would only evaluate results? Don’t we have xFIP/SIERA for performance evaluation?

  5. Bill Baer

    May 06, 2010 11:36 PM

    There’s also WXRL from Baseball Prospectus to measure relievers. While context-neutral stats like ERA estimators are awesome, I do think there are times when you need to consider context.

    For instance a 2.50 ERA is much more impressive from a set-up guy than from a mop-up guy.

    The problem with the save statistic is that it weighs a save with a 3-run lead the same as with a 1-run lead. And Tango/FanGraphs actually increased the threshold to 6% to make it so that truly influential relief performances are given proper credit.

    I don’t think differential is less useful than a percentage. Jose Contreras is going to get more shutdown/meltdown opportunities than Antonio Bastardo because he’s a better pitcher. The stat should reflect this.

  6. bfo_33

    May 07, 2010 06:26 AM

    I like the stat, far more meaningful than saves, and can be applied to the whole pen, vs just the closer (except for blown saves). One question – I’ve seen the win probability stat before, but don’t really know what goes into it. If you are up by 2 going into the 9th, facing the bottom of the Pirates order, is it treated the same way as being up by 2 in the ninth facing the top of the Rays order?

Next ArticleOn Punching Bags