The "Defensive" world of UZR

DWKB

ASFN Icon
Joined
May 15, 2002
Posts
18,224
Reaction score
7,490
Location
Annapolis, MD
WARNING STAT HEAD THREAD


For those interested in the discussion and analysis of defense and how it can be measured:

Baseball Primer has started its series on Ulitmate Zone Ratings (UZR).

I think, however flawed it may be, that this is the best defensive metric "out on the market" right now and I'm very thankful for Mitchel to show the in's and out's of it.

Since this isn't really a DBacks discussion I'll make a vein attempt to tie them into it:

Tony Womack was rated the worst defensive SS in the NL costing us around 22(!) runs more than a league average SS. The 2nd worst SS was Alex Gonzalez of CHN who cost his team 10 runs more than a league average SS.
 

unc84steve

Veteran
Joined
Oct 16, 2002
Posts
168
Reaction score
0
Location
Phoenix AZ
I'm happy to have a discussion about "zone ratings" with the stipulation that I know little about "zone ratings" except for some vague notions. Thus I'm willing to be educated, but I'm going to say right away I have a deep emotional dislike & distrust of this concept.

Why? Because I have no idea who is keeping track of these "zones" at different games. I have no idea who is judging when a ball is in the 3rd baseman's natural zone vs. the shortstop's natural zone (is there overlap?). I have no idea who issues these ratings that put Derek Jeter at the bottom consistently and why?

In my experience, all these issues tend to be subjective.

Look at the standard "retrosheet" zones diagram (link below)

Link to zones used by retrosheet.org

This looks nice for many, many plays. 8XD stands for "CF, extra deep", right? But is "D" = "deep" in relation to where a typical CF plays? Where Andruw Jones is playing that evening? Where the 375' line is?

Do different STATS recorders report this consistently?

These issues get more complicated when there are extreme players and extreme parks involved. We have several in our division named Barry, Randy, and Coors. It gets very confusing when Barry comes up and the manager puts the 2nd baseman where the RF used to be in the 1930's.

I notice that what is listed under "Where do we go from here?" is listed an analysis of various situations such as park factors, handedness of the pitcher, groundball, flyball tendencies, etc.

IMO, that is what is needed in baseball stats: more objective facts. I'd like to know how many times each defense (and pitcher BTW) faces a DP situation with the number of times a ground-ball out is made and the number of times a DP is completed. For goodness sakes, don't have some official scorer tell me whether or not a DP "should have been" made.

Similarly, let's count how often runners advance on wild pitches vs. catchers & passed balls vs pitchers. Knuckleball pitchers always seem to have lousy catchers who give up lots of passed balls (that's a joke ;) ).

I'll stop for now here :)
 

schillingfan

All Star
Joined
Sep 14, 2002
Posts
672
Reaction score
0
Location
Northeastern Pennsylvania
I always wonder what affect who is playing next to you has. If you have a 3B with terrific range, then he will get to more balls than the SS. I think this is a good thing because he would be moving towards the ball, whereas the SS would be moving away, and I would imagine that the 3B would have a much stronger throw than the SS.

Overemphasizing zone range undervalues sure hands and strong throws as well.
 

unc84steve

Veteran
Joined
Oct 16, 2002
Posts
168
Reaction score
0
Location
Phoenix AZ
Originally posted by schillingfan
I always wonder what affect who is playing next to you has. If you have a 3B with terrific range, then he will get to more balls than the SS. I think this is a good thing because he would be moving towards the ball, whereas the SS would be moving away, and I would imagine that the 3B would have a much stronger throw than the SS.

Overemphasizing zone range undervalues sure hands and strong throws as well.
We can engage in some BS'ing until DWKB (or someone else who knows more about zone ratings puts and end to it ;) )

Actually, with "range factors" it's perfectly possible to have 3B, SS neighbors with good RF's such as the Orioles B.Robbinson/Belanger combo or the Cards Pendleton/Ozzie Smith.

In fact, I think you'll find the same factors that help a given 3rd baseman's RF will help his neighboring SS's RF. Specifically a groundball staff, a staff with lefties (thus leading to RH by the opposition), a Green Monster behind you (ditto the previous reason), etc.

Ahhh, here's what I'm asking for:

We just got defensive innings. Heck we just got them to keep track of which OF position the man was standing in by inning (LF, CF or RF). Now I'm asking for defensive splits.

Splits by name of pitcher,
name of hitter,
who was on which base when the ball was hit,
what the count was when the ball was hit,

And when the ball is hit, I'm asking for some more facts:
who fielded the ball first,
who would have fielded the ball if he wouldn't have lost the ball in the sun,
to whom did he throw the ball
who else handled the ball until there was no play to be made
where did every runner end up
(perhaps whether the runners were in motion),
(perhaps who was covering on the hit & run)

When the ball is not hit (but something "happens" I'd like some more facts:
on that pitch that gets away and is called a "wild pitch" it would be nice to know
who the catcher is
who the runner is.
how many runners on
did a runner score?
who is the batter?

on that pitch that called a passed ball
who is the pitcher?
& several of the questions above

All these are "objective" facts that shouldn't be that much harder to catalogue than what we have now. However, I think they'd get to scoreboard issues of runs, bases & outs from another direction (I confess: the Bill James "Win Shares" direction) that would be quite helpful.

Heck, it would be nice someday soon to learn if the Yankees have less balks, WP's & PB's when Derek Jeter is on the field (because his intensity brings focus). OTOH, they may have more b/c he brings so many "Jeter girls" to the stands nobody on the team can keep their mind on the game. :rolleyes:
 
OP
OP
DWKB

DWKB

ASFN Icon
Joined
May 15, 2002
Posts
18,224
Reaction score
7,490
Location
Annapolis, MD
Originally posted by unc84steve
I'm happy to have a discussion about "zone ratings" with the stipulation that I know little about "zone ratings" except for some vague notions. Thus I'm willing to be educated, but I'm going to say right away I have a deep emotional dislike & distrust of this concept.

Why? Because I have no idea who is keeping track of these "zones" at different games. I have no idea who is judging when a ball is in the 3rd baseman's natural zone vs. the shortstop's natural zone (is there overlap?). I have no idea who issues these ratings that put Derek Jeter at the bottom consistently and why?

I believe there are "official" STATS Inc guys who track the PBP data (which includes the balls in zones).

Originally posted by schillingfan

Overemphasizing zone range undervalues sure hands and strong throws as well.

This is a toss out statement that gives me the feeling you didn't read the article and are offhandedly dismissing the work. If I'm incorrect about this please let me know, but that kind of comment really does nothing for the discussion. The reason I think you disn't read the article is because of these quotes:

The average SS committed 169 ROE errors in 5218 balls gotten to (outs plus ROE’s) in all zones. That is an error rate of 169 divided by 5218, or .032. Since Bordick got to a total of 277 balls in all zones, he should have committed .032 times 277, or 8.9 errors. Instead, Bordick committed only 1 error, for a net gain in errors of 7.9. Since an infield error is worth around .49 runs, the swing between an error and an out is .49 plus .29, or .78 runs. Therefore, Bordick saved another .78 times 7.9, or 6.2 runs, by virtue of his "good hands". So far, we have Bordick saving 6.2 runs with his range and another 6.2 runs with his sure hands.

There is one final thing to consider – Bordick’s non-ROE errors. Like ROE errors, that is easily done.

The average SS committed 45 non-ROE errors and Bordick none. If we do the same calculations as above, using .3 as the value of a non-ROE error, we come up with Bordick saving another .72 runs. So it looks like even at the ripe old age of 36, Mike Bordick saved his team last year a total of 13 runs by virtue of his outstanding play (range and hands) at SS!

Note: Before I begin explaining the UZR system, keep in mind that there are at least two components of defense that UZR does not address: One, an outfielder’s "arm", and two, an infielder’s skill at turning the double play. Of course, these skills can be measured (and they are in my Super-LWTS system). They are just not included in UZR. Like ZR, UZR is designed to measure and quantify only that skill which enables a fielder to turn batted balls into outs.
 

schillingfan

All Star
Joined
Sep 14, 2002
Posts
672
Reaction score
0
Location
Northeastern Pennsylvania
FWIW I did read a lot of the article, though I didn't understand a lot of it. But actually I wasn't referring to error rate, but precisely to those kinds of questions that Steve raised, like turning double plays. But actually I mentioned it because it rates Jimmy Rollins very low and watching him play 140 some games in the last two years and I can rarely recall situations where he made a poor relay throw or where I felt he made a poor throw or something where it cost the team an out.
 

unc84steve

Veteran
Joined
Oct 16, 2002
Posts
168
Reaction score
0
Location
Phoenix AZ
Okay, I can see with a little more digging that Retrosheet.org has event files that list play-by-play events where one could theoretically write computer programs to generate the very splits I ask for.

Link to Retrosheet Event Data page

I'm only seeing scattered years up until 1990, but that's cool.

This is the kind of database (along with the development & distribution of programs to harness it) that I'd find helpful.

I'll put it this way. Instead of complaining about how unfair it is that Cincinnati led the NL in passed ball the last 2 years while hard knuckleballer Jared Fernandez has been on the club, one could write a program to sort & count PB's by pitchers and see how often you have a clear leader.

You could see how often a runner in a SB situation leads to opposition balks--and how often they score.

HOWEVER, I'm still a bit skeptical about this ultimate zone rating because I feel I'm trusting something subjective compounded in 2 steps. I don't know who did the original zone ratings, other than people who work with STATS (are they at the games? do they check tapes after the game? how uniform is the quality of the work? is there room for influence from players re decisions--after all these might be used in salary arbitration cases). I also feel I'm trusting this guy offering his rating on various players.

BTW, Bill James' "Win Shares" method is subject to similar criticisms except he lays out more of the data & methods he uses (IMO). I don't know where James gets his home/road data, but that's the level of uncertainty I sense with that method.

Just continuing the discussion; awaiting to hear more :)
 
OP
OP
DWKB

DWKB

ASFN Icon
Joined
May 15, 2002
Posts
18,224
Reaction score
7,490
Location
Annapolis, MD
Originally posted by schillingfan
FWIW I did read a lot of the article, though I didn't understand a lot of it. But actually I wasn't referring to error rate, but precisely to those kinds of questions that Steve raised, like turning double plays. But actually I mentioned it because it rates Jimmy Rollins very low and watching him play 140 some games in the last two years and I can rarely recall situations where he made a poor relay throw or where I felt he made a poor throw or something where it cost the team an out.

Again, DPs aren't included in UZR but they are measured and rated in a seperate catagory.

Now I'm sure there is some value in your personal observations (I haven't seen Rollins play much myself), but I've gotta ask what your comparisons are based on? Who is Rollins better than who he isn't ranked above, that you know?

I've also gotta ask how you saw Rollins play these games. Did you go to all 140 Phils games live or was this on TV? TV gives us an incredibly distorted view of the plays. We aren't able to see the jump the player gets or his positioning. It's hard to say "so-and-so coulda reached that ball instead"

I'd like to point out that Rollins doesn't have any GGs either so even the flawed manager vote doesn't show different.

Clay Davenport's defensice stats show Rollins as being below average also. Ranging from -5 to -1 runs below average.
 
OP
OP
DWKB

DWKB

ASFN Icon
Joined
May 15, 2002
Posts
18,224
Reaction score
7,490
Location
Annapolis, MD
Originally posted by unc84steve
Okay, I can see with a little more digging that Retrosheet.org has event files that list play-by-play events where one could theoretically write computer programs to generate the very splits I ask for.

Link to Retrosheet Event Data page

I'm only seeing scattered years up until 1990, but that's cool.

This is the kind of database (along with the development & distribution of programs to harness it) that I'd find helpful.

I'll put it this way. Instead of complaining about how unfair it is that Cincinnati led the NL in passed ball the last 2 years while hard knuckleballer Jared Fernandez has been on the club, one could write a program to sort & count PB's by pitchers and see how often you have a clear leader.

You could see how often a runner in a SB situation leads to opposition balks--and how often they score.

HOWEVER, I'm still a bit skeptical about this ultimate zone rating because I feel I'm trusting something subjective compounded in 2 steps. I don't know who did the original zone ratings, other than people who work with STATS (are they at the games? do they check tapes after the game? how uniform is the quality of the work? is there room for influence from players re decisions--after all these might be used in salary arbitration cases). I also feel I'm trusting this guy offering his rating on various players.

BTW, Bill James' "Win Shares" method is subject to similar criticisms except he lays out more of the data & methods he uses (IMO). I don't know where James gets his home/road data, but that's the level of uncertainty I sense with that method.

Just continuing the discussion; awaiting to hear more :)

Retrosheet is free PBP data. STATS Inc is by purchase only. (just to let you know).

As far as I know, the players have no interaction regarding the STATS guys and I doubt anyone is savvy enough to use PBP data in an arbitration hearing. The arbitors rarely know what FLD% is much less anything like UZR.

MGL is laying out his methodology in the series, which is not complete, so I don't understand your concern regarding his "ratings". Are you saying you prefer the James model where he arbitrarily decides that DPs for SS and 2B is 40% of their perceived value? Why 40%, where does he get this? He doesn't tell us.
 

unc84steve

Veteran
Joined
Oct 16, 2002
Posts
168
Reaction score
0
Location
Phoenix AZ
Originally posted by DWKB
Retrosheet is free PBP data. STATS Inc is by purchase only. (just to let you know).

As far as I know, the players have no interaction regarding the STATS guys and I doubt anyone is savvy enough to use PBP data in an arbitration hearing. The arbitors rarely know what FLD% is much less anything like UZR.

MGL is laying out his methodology in the series, which is not complete, so I don't understand your concern regarding his "ratings". Are you saying you prefer the James model where he arbitrarily decides that DPs for SS and 2B is 40% of their perceived value? Why 40%, where does he get this? He doesn't tell us.
Okay, I'm trying to play nice, really I am. :rolleyes:

But I come across something like this:

At the same time, the computer keeps track of the total number of fielding errors for each fielding position, but not for each zone individually. Actually it compiles fielding errors in two separate categories: One, ROE errors, are fielding errors that result in an ROE. All other errors, such as on a hit, or a second error on an ROE, are called non-ROE errors.

Link to "Ultimate Zone Rating (UZR), Part 1 "
So what's an "ROE"?

Now I see (after doing a couple of Web searches) that it's guessed by you and a couple of others as "reached on error."

That's plain annoying & inconsiderate.

I'm serious & I'm joking. I'm already having my doubts about trying to follow this methodology being laid out, but when I see this and go "did I miss something or did he?" it only increases my doubts in the whole enterprise.

Now, the Bill James "Win Shares" is trying to do something completely different, but comes from objective foundations: wins, defensive runs allowed, DP's turned, etc.

Plus the James' method has the virtue that it's reproducible by me! LOL. (that's a virtue to me at least).

For example, when I use James' method to estimate the number of expected D'back DP's and the number of actual D'back DP's turned, I can see where new info can help & where missing data hurts. He tries to estimate the number of singles a team yields because most years don't count that directly--why shouldn't I use that if available?

I can easily make such adjustments because I can follow his whole process.

I'll be patient; I'm making the point that even if this is the as advertised "ultimate" zone rating, the ability to communicate what one is doing is key.
 
OP
OP
DWKB

DWKB

ASFN Icon
Joined
May 15, 2002
Posts
18,224
Reaction score
7,490
Location
Annapolis, MD
Originally posted by unc84steve
Okay, I'm trying to play nice, really I am. :rolleyes:

But I come across something like this:


So what's an "ROE"?

Now I see (after doing a couple of Web searches) that it's guessed by you and a couple of others as "reached on error."

That's plain annoying & inconsiderate.

I'm serious & I'm joking. I'm already having my doubts about trying to follow this methodology being laid out, but when I see this and go "did I miss something or did he?" it only increases my doubts in the whole enterprise.

Now, the Bill James "Win Shares" is trying to do something completely different, but comes from objective foundations: wins, defensive runs allowed, DP's turned, etc.

Plus the James' method has the virtue that it's reproducible by me! LOL. (that's a virtue to me at least).

For example, when I use James' method to estimate the number of expected D'back DP's and the number of actual D'back DP's turned, I can see where new info can help & where missing data hurts. He tries to estimate the number of singles a team yields because most years don't count that directly--why shouldn't I use that if available?

I can easily make such adjustments because I can follow his whole process.

I'll be patient; I'm making the point that even if this is the as advertised "ultimate" zone rating, the ability to communicate what one is doing is key.


I think it's a bit unreasonable to hold MGL (or anybody) to the standard that is Bill James when it comes to articulation.

I also don't see the big deal about ROE not being laid out as, for the most part, his audience knows the meaning of the analogy. My "guess" wasn't really a guess as much as it was a humbled statement. The forum below the article is for all the questions and issues to be asked. This is something you never get with James, and when you do, it's a horrible forum that he has no interest in participating in.

But it's not my job or position to defend UZR. It was my point to discuss thoughts on it. That obviously isn't what other people's intent were too. They "have their own 'people' and their 'people' are good" :rolleyes:
 

schillingfan

All Star
Joined
Sep 14, 2002
Posts
672
Reaction score
0
Location
Northeastern Pennsylvania
Originally posted by DWKB
But it's not my job or position to defend UZR. It was my point to discuss thoughts on it. That obviously isn't what other people's intent were too. They "have their own 'people' and their 'people' are good" :rolleyes:
Having a chip on your shoulder and being obnoxious doesn't foster discussion either.
 
OP
OP
DWKB

DWKB

ASFN Icon
Joined
May 15, 2002
Posts
18,224
Reaction score
7,490
Location
Annapolis, MD
Originally posted by schillingfan
Having a chip on your shoulder and being obnoxious doesn't foster discussion either.


I only have a chip on my shoulder about people who lie by sayiong they watched a guy play 140 games the last two years but then actually change that to they base their opinions 2nd hand on other peoples expertise. I guess I'd be much better off discussing it with them than wouldn't I?
 

schillingfan

All Star
Joined
Sep 14, 2002
Posts
672
Reaction score
0
Location
Northeastern Pennsylvania
Be my guest. I've told Steve several times that you are more than welcome on my Phillies board, but I've decided you are afraid they will show just how shallow your real understanding of statistics is.

You want people to talk with you, but then when you don't like what they say you insult them. Then you wonder why nobody will talk stats with you.

Think about it. I'm pretty favorably inclined to stat thingies, but not when they are presented to me with a haughty superior attitude.
 
Last edited:
OP
OP
DWKB

DWKB

ASFN Icon
Joined
May 15, 2002
Posts
18,224
Reaction score
7,490
Location
Annapolis, MD
Originally posted by schillingfan
Be my guest. I've told Steve several times that you are more than welcome on my Phillies board, but I've decided you are afraid they will show just how shallow your real understanding of statistics is.

Very classy and mature statement there. If I wanted to talk Phils I'd go to your Phillies board but to be honest I'm not interested in it. By your logic they would be just as afraid of me by not coming to this board to discuss things.

It was very sneaky of you to turn this discussion away from the subject and more into a pissing contest since you don't understand the material though. I've gotta congradulate you on that but I'm done pissing against you. I'd rather go back to the subject.
 
OP
OP
DWKB

DWKB

ASFN Icon
Joined
May 15, 2002
Posts
18,224
Reaction score
7,490
Location
Annapolis, MD
Originally posted by unc84steve
I'm happy to have a discussion about "zone ratings" with the stipulation that I know little about "zone ratings" except for some vague notions. Thus I'm willing to be educated, but I'm going to say right away I have a deep emotional dislike & distrust of this concept.

Why? Because I have no idea who is keeping track of these "zones" at different games. I have no idea who is judging when a ball is in the 3rd baseman's natural zone vs. the shortstop's natural zone (is there overlap?). I have no idea who issues these ratings that put Derek Jeter at the bottom consistently and why?

The 2nd part of UZR has been posted on Baseball Primer.

To answer Steve's questions (I hope):

By the way, all of my play-by-play data is courtesy of two independent sources. One is Gary Gillette and Pete Palmer and the other is STATS Inc. I believe that STATS, at least, uses three "stringers" for each game, and somehow combines their judgments, in order to reduce human error and bias.
 

Staff online

Forum statistics

Threads
534,873
Posts
5,246,975
Members
6,274
Latest member
G-PA
Top