In Honor of the Facebook IPO: Hacking/Disrupting the NFL

In Game Plan I point out that while in the realm of football few decision makers are ever under the age of 40, the world's hottest tech company is Read more

Top Links 5-15-2012

The Link Leaderboard *The Essential Smart Football* | Smart Football (Source) RSP Football Writers Project: July 23rd | The Rookie Scouting Portfolio (Source) Jets QB coach says Tim Tebow has good mechanics Read more

Top Links 5-10-2012

The Link Leaderboard NFL Players Association: 'Punishment demands evidence' - USATODAY.com (Source) Adrian Peterson: I'll be surprised and disappointed if I miss Week 1 | ProFootballTalk (Source) Terrelle Pryor looking forward to Read more

What Does This Poker Bluff Have in Common With Sean Payton's Decision to Onside Kick During the Super Bowl?

In Game Plan, I spend a considerable amount of time comparing decision making in football to decision making in poker.  To me it's a natural connection. But I could Read more

Analysis

NFL Scouts as Doctors, Overconfidence in the NFL Draft, and RGIII

smoking

I’ve been thinking a lot lately about an analogy that could be made between the NFL’s player personnel executives (I’ll just say scouts from here on out) and doctors.  In the same way that doctors have to look at a patient’s symptoms and try to diagnose a disease, scouts have to look at a football player’s attributes and decide whether or not they will be a good pro.  Doctors use all of the education and experience that is available to them, they conduct a mental search, and then land on what they consider to be the most likely diagnosis.  Scouts do the same thing.

A potential problem with what doctors and scouts are doing is the mental search.  The human brain is limited in the amount of information it can store and it often makes for a poor computer.  Human bias makes its way into the work of doctors and I think it would be stupid to say that it doesn’t affect what scouts do as well.

One of the most reliable biases is overconfidence.  As humans we are poor judges of how confident we should be.  Doctors are no different.  One survey found that 94% of all doctors considered themselves to be “above average”.  But that’s impossible by definition.  Only ~49% of doctors can be above average.

Other studies have found that if doctors are asked to diagnose the cause of death for a patient, and then to rate how confident they are in their answers, they are often wrong (40% of cases) even when they are absolutely confident of their assessment.  One study asked dermatologists to diagnose potential cases of melanoma and then rate their level of confidence in their diagnosis.  The study group was confident more than 50% of the time, but they still got 30% of all of the melanoma cases wrong.

Despite misdiagnoses and overconfidence that can be easily documented, most doctors when asked, can’t remember a single instance of having gotten a diagnosis wrong.

It’s probably worth mentioning at this point that doctors go to school for at least seven years to do what they do.  In the study that asked doctors to diagnose the cause of death, that’s actually a difficult problem.  But the problem was that the doctors didn’t approach it as a difficult problem where they might say “I’m not sure, it’s too difficult to say”.  The problem was that there was no correlation between how sure they were, and the correctness of their diagnosis.

One of the potential ways to mitigate overconfidence is to engage in more regular feedback.  Doctors should be intent on studying the results of their actions.  Feedback is a way to confront yourself with your failures and try to avoid making them in the future, and at a minimum, to remove the issue of overconfidence from the equation.

This year’s NFL draft will put several teams in the difficult position of having to decide whether to pay a king’s ransom for RGIII.  On the surface the decision might look easy.  If you can get a guy like an Aaron Rodgers for instance, you pay whatever you have to.  But it’s also worth remembering that the NFL isn’t really that good at figuring out which college players will be good pros.  I’m not saying RGIII won’t be any good.  I frankly don’t have any idea how good he’ll be.  But overconfidence is a silent killer.

To illustrate how rampant overconfidence can be, look at the top 10 picks of the draft from 2005-2007.  Or if you want to make a more specific comparison, look at the top 2 picks each year and try to think about how many of those guys you would want to start a franchise with.  This is what I mean by engaging in regular feedback.  Teams thinking about trading for RGIII should think about how likely they are to make the correct decision, and then decide what they can pay for that chance (maybe 30-40%) of hitting a homerun.

*20 out of the 30 picks haven’t made a pro-bowl.

** The 2005 draft was particularly terrible.  None of the first 7 picks have delivered anything for the teams that drafted them.

*** Of the top 2 of each draft, only Mario Williams and Calvin Johnson have been legitimate successes.  So about 4 out of those guys haven’t provided much value to their teams.

Year Pick Player Pos Tm All Pro Pro Bowls Years Starter Games
2007 1 JaMarcus Russell QB OAK 0 0 2 31
2007 2 Calvin Johnson WR DET 1 1 5 76
2007 3 Joe Thomas T CLE 3 4 5 80
2007 4 Gaines Adams DE TAM 0 0 1 47
2007 5 Levi Brown T ARI 0 0 5 77
2007 6 LaRon Landry DB WAS 0 0 4 64
2007 7 Adrian Peterson RB MIN 2 4 5 73
2007 8 Jamaal Anderson DE ATL 0 0 3 75
2007 9 Ted Ginn Jr. WR MIA 0 0 3 75
2007 10 Amobi Okoye DT HOU 0 0 4 78
2006 1 Mario Williams DE HOU 0 2 5 82
2006 2 Reggie Bush RB NOR 0 0 5 75
2006 3 Vince Young QB TEN 0 2 4 61
2006 4 D’Brickashaw Ferguson T NYJ 0 2 6 96
2006 5 A.J. Hawk LB GNB 0 0 6 94
2006 6 Vernon Davis TE SFO 0 1 6 88
2006 7 Michael Huff DB OAK 0 0 5 91
2006 8 Donte Whitner DB BUF 0 0 5 84
2006 9 Ernie Sims LB DET 0 0 4 87
2006 10 Matt Leinart QB ARI 0 0 1 31
2005 1 Alex Smith QB SFO 0 0 6 70
2005 2 Ronnie Brown RB MIA 0 1 6 92
2005 3 Braylon Edwards WR CLE 0 1 5 99
2005 4 Cedric Benson RB CHI 0 0 5 91
2005 5 Cadillac Williams RB TAM 0 0 4 82
2005 6 Pacman Jones DB TEN 0 0 2 52
2005 7 Troy Williamson WR MIN 0 0 1 49
2005 8 Antrel Rolle DB ARI 0 2 5 100
2005 9 Carlos Rogers DB WAS 0 0 5 94
2005 10 Mike Williams WR DET 0 0 2 56
Posted on by FantasyDouche in Analysis, Methodology Leave a comment

Introducing Game Level Similarity Projections

One of the things I’ll be rolling out for subscribers when the season starts is Game Level Similarity Projections (GLSP, or since all of these projection systems have to have stupid names of obscure former players, we’ll call ours GILLESPIE after former Vikings and Buccs wide receiver Willie Gillespie)

The easiest thing we can do to get a sense as to what an individual player will do against a defense is to look at what similar players have done in the past against similar defenses.  Before I get into the theory of why we should look at similar players versus similar defenses (instead of running a regression), let’s look at an example.

Let’s take Ben Roethlisberger’s 2011 Week 2 matchup against the Seattle defense.  For purposes of picking out the similar games, we’ll use Ben’s 2010 stats and the Seattle defense’s 2010 stats (when the season starts we will slowly incorporate the 2011 data so as not to be using old data).

Here are their stats that we’ll be using to pick out similar games (the stats should be fairly easy to understand with the exception of Efficiency, which is basically an opponent adjustment):

Att/G Y/G TD/G INT/G Y/A Rush Yards Efficiency
Ben Roethlisberger 32.4 266.7 1.4 0.4 8.2 14.7 1.2
SEA Defense 36.4 264.1 1.9 0.8 7.2 6.0 1.1

We then take those averages and compare them to other players and other defenses, then find the most similar matchups that featured players and defenses with similar stats.  We get a table of similar games as follows:

Player Game Date Opp Comp Att Cmp% Yards TD Int Y/A Rushes Yds TD
K. Orton 19-Sep-10 vs SEA 25 35 71% 307 2 0 8.8 3 -5 0
J. Cutler 17-Oct-10 vs SEA 17 39 44% 290 0 0 7.4 2 19 0
P. Rivers 26-Sep-10 vs SEA 29 53 55% 455 2 2 8.6 1 2 0
A. Smith 12-Dec-10 vs SEA 17 27 63% 255 3 0 9.4 4 12 0
A. Smith 12-Sep-10 vs SEA 26 45 58% 225 0 2 5.0 0 0 0
B. Roethlisberger 14-Nov-10 vs NE 30 49 61% 387 3 1 7.9 1 12 0
G. Frerotte 23-Dec-00 vs SF 18 29 62% 205 1 0 7.1 3 23 1
J. Freeman 26-Dec-10 vs SEA 21 26 81% 237 5 0 9.1 4 23 0
E. Manning 07-Nov-10 vs SEA 21 32 66% 290 3 0 9.1 1 -1 0
M. Brunell 04-Nov-01 vs TEN 21 32 66% 261 1 1 8.2 2 21 0
M. Brunell 23-Sep-01 vs TEN 17 27 63% 235 0 0 8.7 6 19 0
M. Schaub 18-Nov-07 vs NO 21 33 64% 293 2 0 8.9 2 7 0
R. Gannon 09-Dec-99 vs TEN 20 28 71% 273 1 1 9.8 1 3 0
M. Ryan 19-Dec-10 vs SEA 20 35 57% 174 3 1 5.0 6 6 0
J. Kitna 19-Dec-10 vs WAS 25 37 68% 305 2 0 8.2 4 11 0
A. Rodgers 10-Oct-10 vs WAS 27 46 59% 293 1 1 6.4 4 30 0
J. Cutler 13-Dec-07 vs HOU 27 39 69% 254 1 0 6.5 2 11 0
M. Schaub 19-Sep-10 vs WAS 38 52 73% 497 3 1 9.6 1 2 0
M. Schaub 25-Nov-07 vs CLE 22 36 61% 256 2 2 7.1 1 7 0
C. Batch 21-Oct-01 vs TEN 25 42 60% 338 3 1 8.1 2 3 0
R. Gannon 22-Dec-01 vs TEN 29 50 58% 249 1 1 5.0 2 11 0
M. Hasselbeck 24-Nov-02 vs KC 25 36 69% 362 3 0 10.1 3 23 0
Comp Att Cmp% Yards TD Int Y/A Rushes Yds TD
Averages 23.7 37.6 63% 293 1.9 0.6 7.8 2.5 10.9 0.0

Let’s look over the table of similar games to see what it gives us.  First, the five most similar games all feature the SEA defense from 2010.  The next most similar game is actually our subject player Ben Roethlisberger against a defense, the Patriots, that was similar to SEA last year.  All of the games should feature QBs who are at least a rough approximation of Roethlisberger, and defenses who are at least rough approximations of the SEA defense.  Not every game will be an exact match, but they’ll all be close.

We can then use the averages of those 20 similar games to create a projection for Big Ben when he faces the SEA defense in Week 2.  In fact, we can create projections for the first 6 weeks of the season using this methodology.  To be clear, the error rates will be a little higher when we’re using last year’s data.  Schemes change and players change.  But over the course of the season we’ll be adding in more data from the current season and the error rates should go down.

Maybe a good discussion to have at this point is why we would use GILLESPIE, instead of using perhaps a simple regression.  First, GILLESPIE is going to overcome issues that you might have with a linear regression when not all of the variables have linear relationships.  Second, I find the use of regression analysis to border sometimes on being a black box.  GILLESPIE is just much more transparent because I can actually see each game that is going into the forecast.  This will become apparent during the season when I publish the similar games for some of the start/sit analysis.

There are a number of applications in using GILLESPIE.  We can create projections of the first 6 games of the season and stay away from bust candidates.  Once the season is in full swing we can forecast future weeks and project which guys will be rising and falling in value.  We can create the weekly rankings.  When the season is winding down we’ll be able to look at the playoff schedule and issue trade alerts where we tell you which of the guys that got you into the playoffs aren’t going to help you win the playoffs, along with some trade targets.

I’m excited about GILLESPIE.  I think it’s just one more tool in the arsenal to help you win your league this year.

Posted on by FantasyDouche in Analysis, Free Content, Methodology Leave a comment

Methodology: Advanced Red Zone Efficiency

Fantasy football scoring systems typically provide for outsized rewards for touchdowns.  For that reason, we look at advanced metrics for running back performance around the end zone.  We call it Advanced Red Zone Efficiency, but in reality we are scoring efficiency within the 10 yard line.  Runs inside the 10 yard line are the most valuable out of any that a running back receives.

Our Advanced Red Zone Efficiency scores take into account two basic measures.  How many of an RBs carries inside the 10 were converted into TDs, and what percent of yards needed for a TD did the RB average (yards per carry divided by yards to goal).

The TD conversion number is easy to understand.  However, the YPC/YTG percentage is not as intuitive.  A simple explanation is that running backs get the ball in different places.  A running back like Michael Turner is going to get a lot of goal line work.  A running back like Jamaal Charles is going to get the ball further away from the goal line.  So converting their YPC into a percentage of yards needed to score allows us to level the playing field a little.

We then create a standard score for each of these measures we’re looking at and then combine them.  That combined number is our Advanced Red Zone Efficiency.

Posted on by FantasyDouche in Methodology Leave a comment

Similarity Scores

Similarity Scores were introduced by Bill James, the father of sabremetrics.  To see how James applied the concept you should really consult his work.  My use of Similarity Scores would probably be considered a bastardization by the sabremetric purists (the only people more annoying than baseball purists are sabremetric purists).

However, I obviously give James all credit for coming up with the concept of Similarity Scores.

Here’s how I apply them.  I take the most basic box score statistics like yards per carry (or reception, or passing attempt), total yards, TDs, and age, and I measure each player’s distance from each other player on these simple stat lines.  Once I have done that, I can generate a list of the top 20 most similar player seasons to any individual player.  So I can tell you the 20 most similar seasons to Jamaal Charles 2010.

I then use that information to look at the following year production (let’s call this Y2).  This is a roundabout way of looking at career trajectory.  The theory goes that possession receivers will produce similarly to other possession receivers that came before them.  Speed receivers will perform similarly to other speed receivers who came before them. 

We’re primarily interested in what happens to players at the beginning and end of their careers.  What can we expect from older players, and what can we expect from younger players.

In our Draft Guide we use Similarity Scores to come up with a projection for the upcoming year.  There are all sorts of caveats that should be offered with the Similarity Projection, the simplest of which is that it is a single thing to consider.

Posted on by FantasyDouche in Methodology Leave a comment

Strength of Schedule

Planning your fantasy season around Strength of Schedule (SOS) is a little bit like counting cards at blackjack.  Card counters don’t win every hand.  They’re just trying to maximize their edge over the long run and get their money on the table when the odds are in their favor. 

Using SOS has the same benefits.  If you regularly and consistently pick players based on their seasonal SOS, and pick starts/sits based on weekly SOS, then you’ll be ahead over the long run.  You might lose some individual start/sits, but your goal is to get as many favorable matchups over the course of a season as possible

Our Methodology

Our system of using SOS relies primarily on fantasy points allowed by the opposing defense and then looking at the schedule in 3-6 game blocks. 

For purposes of visualization, we standardize our defensive ratings to make them simple to look at in bar graph form.  It’s easier to look at how many standard deviations a defense is above/below the mean, than it is to try to look at the difference between a defense that allows 15 points/game as opposed to one that allows 16 points/game.

image

For projecting individual weekly matchups we rely on both efficiency ratings for the defense, and efficiency ratings for players.  Basically how well did each do better than they were expected to.

((Player Efficiency * Defense Points Allowed) + (Defense Efficiency * Player Points Scored))/2 = Projection

After we have those projections we can look at players both from a start/sit standpoint, as well as finding players to target in trades.

Again, our SOS calculations aren’t going to be perfect.  For instance, it’s particularly challenging to forecast next season’s SOS based on last season’s defenses as defenses improve and deteriorate from year to year. 

However, our goal isn’t to be perfect, it’s just to get the maximum advantage we can.

Posted on by FantasyDouche in Methodology Leave a comment

Filtered Yards Per Carry

Filtered Yards Per Cary, or Filtered YPC, is one of the advanced metrics that we’ll use sometimes to try to uncover hidden talent, or to identify guys who were the beneficiaries of good blocking or a good system.

Filtered YPC basically throws out all runs less than 4 yards based on the theory that blocking is going to get a ball carrier four yards past the line of scrimmage.  What he does after that is usually more based on his talent.

Filtering carries might seem stupid in that guys usually stay in the same system from year to year.  So if they got good blocking last year, they’ll probably get it again next year.  But what we’re trying to uncover is what coaches might see in evaluating their depth chart.  Are they leaving “yards on the table” because they’re sticking with a guy who doesn’t do anything past four yards?  Are there two fantasy backs with other similar metrics but who are very different in terms of Filtered YPC.

So we’re trolling for clues that a change in usage would result in a breakout for a back.  If we know a guy has talent based on a great Filtered YPC number, and we suspect that he might get increased carries, then we can potentially get a big value pop.

Posted on by FantasyDouche in Methodology Leave a comment

The Adjusted 40 Score

The Adjusted 40 Score is credited to Football Outsiders’ Bill Barnwell, who explains it here:

The 40-yard dash matters: When compared with all three metrics, the 40 time was the single most indicative measure of a running back’s future success. That’s not to say that a player can succeed strictly on the basis of a fast 40, but that a 40 tends to be the biggest tip-off of future playing time and success with that playing time.

We need to adjust 40 times for weight: Raw 40-yard dash times have a minus .36 correlation with DPAR, carries and yards; remember, a negative relationship here would indicate that 40 times (which are better as they decrease) are strongly correlated with those rushing metrics (which are better as they increase).

The thing is, not all 40 times are created equal. Brandon Jacobs‘ 4.56 40 is incredible when you consider his 267-pound frame. On the other hand, Ahmad Bradshaw‘s 4.55 40 was disappointing for a player who weighed in nearly 70 pounds lighter. Adjusting 40 times for weight helps translate a raw metric into something that’s more indicative of a player’s NFL potential.

Ready for a math headache? The formula for the adjusted 40 score is (Weight * 200)/(40 Time4). The multipliers are as such in the formula to ensure both accuracy as well as simplicity — the scores that result revolve around a 100-point scale. The average adjusted 40 score of all running backs is 98.5; for all drafted running backs, it’s 102.4; for all running backs selected in the first round, it’s 112.1. Consider adjusted 40 score to be a sort of speed score — a higher number is better.

Posted on by FantasyDouche in Methodology Leave a comment