Mythbusters: PFF Edition

BritCard

ASFN Icon
Joined
Jan 10, 2020
Posts
21,031
Reaction score
37,086
Location
UK
I wanted to write something about PFF as I feel it is often wrongly maligned by fans. I understand why, as players do the same thing and fans pick up on it but I wanted to address some of those issues. Or one issue really which is the big one often spouted,

"PFF doesn't know what the play is, so they can't possibly know if X did his job properly"

This one gets kicked around a lot. By pundits, coaches and players. Although much less so than it used to as people from all 3 groups have learned more about the process.

Let's see if we can put some of those doubts to rest.

Chip Kelly was a long time critique of PFF. From his days as a HC to when he was doing color commentary he was not a fan,

Reporter: “I’m just wondering why Tiller doesn’t play more (over Martin).”

Kelly: “Josh (Garnett) obviously earned his starting position. He came here, Josh was just a little bit behind because he missed the spring with us obviously because he wasn’t available for OTAs. But, we got through training camp and knew we thought we had a pretty good one. As he got going during the season, he just passed Andrew by and then obviously Marcus is playing center. Andrew Tiller is not a center. So, when Daniel Kilgore went down, Marcus went in at center.”

Reporter: The reason I ask is if you look at the statistics, Tiller seems to be better?

Kelly: “What statistics?”

Reporter: Quarterback hurries, sacks–?

Kelly: “Where does that come from?”

Reporter: Pro Football Focus.

Kelly: “OK. Stick with that.”

Reporter: Do you not put much stock in that?

Kelly: “No, I mean I’ve said it all along, how can they grade an offensive lineman when they don’t know what the play is? I’ve had it before, our left tackle gave up a sack. He didn’t give up a sack because the guy slanted in the ‘B’ gap. The guard had ‘B’ gap and that’s not the left tackle’s responsibility, but it’s written down as a sack on the left tackle because you don’t know what play was called. We called slide protection. We didn’t call man protection. If someone can look at a film and figure out what we call for a play and know what our scheme is and then give a guy a grade on it, I think there’s a lot of players and coaches that feel the same way. I don’t know. You can do whatever you want with it. It’s like me going into a bank and grading a teller because they gave me a lollipop. I gave them a 94.3.”

After the season, though, Kelly did what the company has invited its critics to do: He studied its process. He met some of its analysts — who watch every player on every snap — and watched them make evaluations. And then, he bought a share of the company.

“The coaches that have come in there cannot believe the process and how thorough it is,” Collinsworth said in a phone interview this week. “Is it perfect? I’m sure it’s not perfect. But it’s pretty darn close.”

Longtime Bengals offensive line coach Paul Alexander (23 years as Bengals O line coach) in 2017 reviewed about 600 plays where PFF had downgraded one of his blockers; he told company founder Neil Hornsby that he disagreed with perhaps 12, “which is pretty remarkable,” Alexander said.

So for all the talk of "PFF don't know what the play call is" only 2% of the grades that the Bengals O line coached watched of his own O line with PFF were called wrong.

xc_hide_links_from_guests_guests_error_hide_media


Every play is graded by 3 separate analysts, many of whom are former players or coaches at NFL and college levels. Those grades are then signed off by a supervisor. If there is any disagreement on a play and they can't come to an agreement a play is never marked down.

PFF say many coaches and players visit their HQ every year and not one has ever left the building still in disagreement.

I will say this, the way PFF grading works is play by play and it is not weighted. You could for example, have a running back with 3 TD's and 70 yards with a grade of 60. If those 60 yards all came on 4 plays and his other 20 touches didn't achieve much. A 60 yard TD run can be wiped out by missing a blitzer.

I think this is where a lot of agreement on PFF stems from. You can have players that make explosive plays that catch the eye, a PBU, an INT, a 70 yard TD etc but still have a poor game grade because they didn't perform well on other snaps.

I don't think anyone should worship at the alter of PFF alone, but I also don't think it's accurate to say PFF player grades are inaccurate.
 

unseenaz

ASFN Addict
Joined
Mar 6, 2013
Posts
6,528
Reaction score
4,936
Location
Gilbert
It's so interesting to me that Kelly was so against the advanced statistics and now is out of the league. I can see the argument from both sides, although I wouldn't die on either hill.
 
OP
OP
BritCard

BritCard

ASFN Icon
Joined
Jan 10, 2020
Posts
21,031
Reaction score
37,086
Location
UK
PFF grades are inaccurate.

There's no evidence they are inaccurate. When a guy like Kelly who vehemently hates them buys stock after a visit and the serving O line coach of the Bengals at the time could only find fault with 12 of 600 grades that's well within tolerance.

Its increasingly difficult to find players or coaches in the NFL who disagree with them. Every year the dissenting voices die off more.
 

slanidrac16

ASFN Icon
Supporting Member
Joined
Jul 11, 2002
Posts
15,002
Reaction score
14,417
Location
Plainfield, Il.
Every grade is just that...a grade. And if a player is graded out at 89.9 or whatever guarantees it will necessarily translate to the NFL

Let’s take an o-linemen. Who is the competition he’s playing against? Who is playing on the right or the left of him? What system is he playing in? How good is his Qb? How good are the other weapons on the team?

Bottom line is I want to draft a guy that has performed well against top notch opponents. Don’t overthink it.
 

MadCardDisease

Moderator
Moderator
Supporting Member
Joined
May 13, 2002
Posts
19,821
Reaction score
11,754
Location
Chandler, Az
There's no evidence they are inaccurate. When a guy like Kelly who vehemently hates them buys stock after a visit and the serving O line coach of the Bengals at the time could only find fault with 12 of 600 grades that's well within tolerance.

Its increasingly difficult to find players or coaches in the NFL who disagree with them. Every year the dissenting voices die off more.

And yet Bill Belichick (The Greatest NFL coach of the modern era and possibly all time) says it's impossible for PFF ot know what each player was supposed to do on a given play. Belichick even admits that he doesn't know for sure who is supposed to do what when watching game film of other teams.
 

GuernseyCard

ASFN Icon
Joined
Dec 29, 2012
Posts
10,123
Reaction score
5,680
Location
London UK
There's no evidence they are inaccurate. When a guy like Kelly who vehemently hates them buys stock after a visit and the serving O line coach of the Bengals at the time could only find fault with 12 of 600 grades that's well within tolerance.

Its increasingly difficult to find players or coaches in the NFL who disagree with them. Every year the dissenting voices die off more.

You offer anecdotal support of PFF grades.

Where's the actual empirical evidence in support of accuracy?

And by the way: any quantitative data that use - .5 or .9, etc... is as a rule - ******** stats!
 

Jetstream Green

Kool Aid with a touch of vodka
Joined
Feb 5, 2003
Posts
29,459
Reaction score
16,598
Location
San Antonio, Texas
PFF is not a franchise, not a professional league, and does not have any info which proceeds an actual game which they personally generate deciding an outcome. Who cares to really care if some people do not like them... watch the damn games!
 
OP
OP
BritCard

BritCard

ASFN Icon
Joined
Jan 10, 2020
Posts
21,031
Reaction score
37,086
Location
UK
Every grade is just that...a grade. And if a player is graded out at 89.9 or whatever guarantees it will necessarily translate to the NFL

Let’s take an o-linemen. Who is the competition he’s playing against? Who is playing on the right or the left of him? What system is he playing in? How good is his Qb? How good are the other weapons on the team?

Bottom line is I want to draft a guy that has performed well against top notch opponents. Don’t overthink it.

I'm thinking more at the NFL level. As you say, at the college level while the grades are accurate the context is more varied because of huge differences in opponent quality. A 90 in the SEC is worth twice a 90 in Mountain West.

The same can be said for the NFL to a small degree. There's no accounting for opponent. You aren't graded on a curve. Aaron Donald is the same as Rodney Gunter.

But the grades are generally right, but there is context outside the grades.
 
OP
OP
BritCard

BritCard

ASFN Icon
Joined
Jan 10, 2020
Posts
21,031
Reaction score
37,086
Location
UK
You offer anecdotal support of PFF grades.

Where's the actual empirical evidence in support of accuracy?

And by the way: any quantitative data that use - .5 or .9, etc... is as a rule - ******** stats!

Wide acceptance by the league is evidence enough.
 

GuernseyCard

ASFN Icon
Joined
Dec 29, 2012
Posts
10,123
Reaction score
5,680
Location
London UK
It's the Hump thing with BritCard.

I can feel fairly certain that the only grade that matters to Keim and MB comes from Kugler.
 

MadCardDisease

Moderator
Moderator
Supporting Member
Joined
May 13, 2002
Posts
19,821
Reaction score
11,754
Location
Chandler, Az
The League (teams) buys their game logs (stats) not their interpretation, which is what they market to fans like you.

Exactly!

Teams used to have Quality Control coaches who's job it was to watch game film and find tendencies (ie "how may times they pass vs run out of 21 personnel", etc). Now they can just get that info from one of these companies that does all the dirty work for them and thier QC coaches can now spend more time analyzing the data vs compiling it.
 
Last edited:

WildBB

Yogi n da Bear
Joined
Mar 20, 2004
Posts
14,295
Reaction score
1,239
Location
The Sonoran Jungle - West
PFF is not a franchise, not a professional league, and does not have any info which proceeds an actual game which they personally generate deciding an outcome. Who cares to really care if some people do not like them... watch the damn games!
Yeah, while players have penalties and OL have QB pressures and such, it doesn't always account for big plays and crucial situations, 3rd downs, 4th quarters...ect.

Also, does it account for QB influenced sacks due to coverage or poor reads?

Does it take into account DL (opponent) ratings?

How about # of passing attempts that they're needed to protect the QB in a game?

It all affects outcomes of grading.

They could get beaten a time or two in the first half yet have a solid game overall up until that point and thereafter.
 

Jetstream Green

Kool Aid with a touch of vodka
Joined
Feb 5, 2003
Posts
29,459
Reaction score
16,598
Location
San Antonio, Texas
Yeah, while players have penalties and OL have QB pressures and such, it doesn't always account for big plays and crucial situations, 3rd downs, 4th quarters...ect.

Also, does it account for QB influenced sacks due to coverage or poor reads?

Does it take into account DL (opponent) ratings?

How about # of passing attempts that they're needed to protect the QB in a game?

It all affects outcomes of grading.

They could get beaten a time or two in the first half yet have a solid game overall up until that point and thereafter.

Exactly, I am a big Kyler fan but there were a lot of sacks early on which were his fault for not maintaining the pocket the OL established for him, with him prematurely scrambling out or simply a matter of WRs not getting any separation for the prescribed duration of a high tempo play... so much of 11 guys working as one being codependent on the stats
 

slanidrac16

ASFN Icon
Supporting Member
Joined
Jul 11, 2002
Posts
15,002
Reaction score
14,417
Location
Plainfield, Il.
I'm thinking more at the NFL level. As you say, at the college level while the grades are accurate the context is more varied because of huge differences in opponent quality. A 90 in the SEC is worth twice a 90 in Mountain West.

The same can be said for the NFL to a small degree. There's no accounting for opponent. You aren't graded on a curve. Aaron Donald is the same as Rodney Gunter.

But the grades are generally right, but there is context outside the grades.
Exactly. So any grade is still just a barometer of performance. I was thinking more in the line of college linemen. There is definitely a difference with level of opponents. So I would lean forward a player with a lower grade playing in the SCC as opposed to a player with a higher grade from a player in the mountain west.

HEY! Maybe that’s Keims problem! Lol!
 

Solar7

Go Suns
Joined
May 18, 2002
Posts
10,973
Reaction score
11,582
Location
Las Vegas, NV
Exactly, I am a big Kyler fan but there were a lot of sacks early on which were his fault for not maintaining the pocket the OL established for him, with him prematurely scrambling out or simply a matter of WRs not getting any separation for the prescribed duration of a high tempo play... so much of 11 guys working as one being codependent on the stats
And there were a lot of sacks he saved them from allowing, so...
 

BurqueCardFan

Hall of Famer
Joined
Aug 17, 2006
Posts
1,742
Reaction score
1,682
Location
Albuquerque, NM
I'm thinking more at the NFL level. As you say, at the college level while the grades are accurate the context is more varied because of huge differences in opponent quality. A 90 in the SEC is worth twice a 90 in Mountain West.

The same can be said for the NFL to a small degree. There's no accounting for opponent. You aren't graded on a curve. Aaron Donald is the same as Rodney Gunter.

But the grades are generally right, but there is context outside the grades.

I disagree about the quality of opponent argument. It can easily be looked at in the opposite manner. Sometimes these big school players look better because of the qualify of talent around them. I think it is a big reason a lot of these big school high draft picks end up being busts once they go to a team drafting high. There's plenty of examples of Mountain West players (or other smaller conferences) that have out produced higher drafted big name players. You can't just make that statement in a vacuum.
 

slanidrac16

ASFN Icon
Supporting Member
Joined
Jul 11, 2002
Posts
15,002
Reaction score
14,417
Location
Plainfield, Il.
I disagree about the quality of opponent argument. It can easily be looked at in the opposite manner. Sometimes these big school players look better because of the qualify of talent around them. I think it is a big reason a lot of these big school high draft picks end up being busts once they go to a team drafting high. There's plenty of examples of Mountain West players (or other smaller conferences) that have out produced higher drafted big name players. You can't just make that statement in a vacuum.
Yep. The whole point is no matter who is grading and who they’re grading is subjective.
 

Staff online

Forum statistics

Threads
534,797
Posts
5,246,424
Members
6,273
Latest member
sarahmoose
Top