Offered for Consideration: Yet Another Theory of Draft Pick Value

Here’s my general outlook on conventional wisdom: It might be right… or it might not be.  But I don’t have a lot of reverence for conventional wisdom.  I just think there are too many examples of truths that we took for granted at one point, only to find out later that those truths weren’t actually… wait for it… true.  Probably my favorite example of the idiocy of conventional wisdom is the fact that Galileo was sentenced to house arrest for supporting the heliocentric view of the solar system and questioning the earth-centric view.

It strikes me that the current debate surrounding the value of NFL draft picks is a classic debate over the value of conventional wisdom.  The Harvard Sports Analysis Collective (HSAC) has advanced a theory of draft pick value (Harvard Chart) based on some analysis that disagrees with conventional wisdom.  The Harvard Chart is often the subject of criticism from adherents of the traditional system for valuing draft picks (which I will call the Jimmy Johnson Chart).

If you want to read a compelling criticism of the Harvard Chart, check out @SigmundBloom on Bleacher Report.  I posted a short response on Bloom’s article if you want to check that out.  But this post is going to go in another direction, so it wasn’t really appropriate as a comment.

I actually don’t think you can have just one theory of valuation for draft picks.  I think it depends on the team and what the team needs.  Good teams and bad teams aren’t going to have the same types of needs in the draft.  I’m always more interested in the shitty teams (the Patriots and the Packers don’t need any help) so let’s start there.  Here’s my general theory of how shitty football teams should think about their rosters:

  1. Football teams start 22 men (not counting special teams)
  2. Football is a violent sport which often results in losing players for games or even full seasons.
  3. Shitty teams are almost never just one player away.  If they were just one player away, they would be 8-8, not 2-14.
  4. Because of numbers 1 through 3, shitty teams should be looking to stockpile players.  They don’t just need one or two players.  They are often starting a number of players that couldn’t start on any other team.

Given those assumptions that I have about shitty teams and their draft needs, here is a chart that is based not on player value, but on how many “Games Started” you can expect to get from each pick in the draft.  image

But here’s something important.  When you’re talking about “expect”, it becomes pretty important how you define “expect”.  Should you define the expectation as being the average of games played for each draft spot?  That might not work. 

Consider for example 10 hypothetical picks from the 6th round.  Let’s say that 9 of the 10 players never start an NFL game.  But let’s say the other guy starts 120 games.  The average games started for the group is 12.  But 9 out of the 10 guys were well below the average.  This is what you could call skewness.  One outlier in the group is having an outsized impact on the average.

For that reason I used median games started as the measurement in the graph below.  That creates a more accurate picture of what you can “expect” to get out of each pick.  Half of the players picked in that slot will be better than the median and half will be worse.

If we’re talking about improving shitty teams, then what we really want is to pick a number of guys who can come in and contribute right away.  We want NFL starters.  That’s what the theory of draft value that I am proposing would give you.

Let’s compare this theory of draft value to the Jimmy Johnson Chart and the Harvard Chart.


My theory disagrees with the JJ Chart on the equivalent value of early picks and agrees with the JJ Chart on the worthlessness of the late picks.


My theory disagrees with the Harvard Chart on the value of late picks (HC says they’re worth something, I say they aren’t), but agrees more with the Harvard Chart on the value of the early picks.

I think my theory of value addresses a shortcoming in the Harvard Chart that @RumfordJohnny and I were discussing.  NFL teams have a limited ability to even evaluate all of the talent they have in camp.  Bringing in 20 guys from the 6th round isn’t that helpful when you have just a few weeks to get a look at them.  My theory of value would place very little value in those guys. 

There’s also another problem with the late round picks.  They aren’t much better than undrafted free agents, which have no draft value.  It just doesn’t make sense that you would assign value to a pick that might only be slightly better than a guy you could bring in off the street.

As to the Jimmy Johnson Chart, here’s a problem with it that its defenders would have a hard time explaining.  According to the JJ Chart, the 7th and 8th picks added together don’t equal the value of the number one overall pick.  If you’re a shitty team and you have a million holes to plug, wouldn’t you rather have two top 10 picks than just the one guy at the top of the draft?  I would.

To get back to a point I made in the beginning of this post, this doesn’t have to be a one size fits all theory of drafting.  After a team compiles enough starter-level players to dig themselves out of the cellar, then they can think about whether they might be one player away.  But until then, they should keep in mind that they are playing a violent sport that starts 22 players.

One more thing to add about being “one player away”.  Two of the greatest quarterbacks to ever put on a uniform, Peyton Manning and Dan Marino, have a combined one Super Bowl win.  Football is a team sport.  The Jimmy Johnson Chart ignores that reality.

  • TheFFGhost

    Interesting article and a great read. Question, your charts, how many years do they stretch? Are they over ten years or starts for all the players selected at pick “X”?

    • FantasyDouche

      The data covers 1990-2007 and is the median games started for each pick in the draft. So the median games played for the 1st pick is about 100 games. Then I do that for every single pick in the draft. By the time you get to the median for the 200th pick, it’s basically zero. More than half of the players picked in that slot start zero total games.

  • TheFFGhost

    Gotcha, followed your method with what you were doing with the data, just didn’t know what the range was. It’s really interesting to see the sharktooth variation between picks. I would have assumed such a behavior further down but that sharktooth look seems to persist throughout. Did you gather this data yourself or was this data provided somewhere else? Second question, how are the difference in career lengths account for? For instance say we look at Player A, Player B & Player C, all three are drafted 10th overall. Player A has been in the league for 10 years and started every game, while Player B has been in the league 9 years and started every game and Player C has been in the league 1 year and has played every game. Now all three are full time starters but produce a median of 9, obviously highly skewed. Have you considered essentially a window to make sure all the players are weighed equally? Like year 1-3 for all players or something where the contributions are all basically the same? I say that because your year one and two players would be throwing your median out of wack if I follow this correctly.

    • FantasyDouche

      The reason for the sharktooth variation is that each pick is essentially a small sample size. Each pick represents only about 17 observations, which is not very many. That’s why we would use the trend for all picks to forecast the value of each pick. It basically takes our small sample size problem and fixes it.

      As to the career length issue, that’s why I cut the data off in 2007. So that we could measure the total games started for the last year that we measure, and do it over a number of years (we can measure the 2007 class from 2007-2011).

      I wanted to measure total games started because I think measuring overall quality of career is important, rather than just measuring games started per year or maybe even just over the first three years.

      But while it’s possible that the period that I observe is going to affect the actual median in terms of games started, I don’t think it’s likely to affect the slope of the line. And really what we’re interested in here is each pick’s value relative to each other pick.

      • TheFFGhost

        True, fair enough reasoning there. Whatever the case, like I said…good read, very interesting.

  • evansj

    Does the Jimmy Johnson chart become a more realistic predictor of NFL teams behavior now that there is the rookie wage scale? Have to believe that the value of high picks escalated significantly when the scale went into place. Falcons go for Jones last year without the scale? Redskins go for Robert Griffin III if the risk included a 50 million dollar signing bonus? I remember thinking about the Raiders when they drafted J. Russel and feeling like the value of their pick was essentially “zero.” No one wanted to pay anyone available in the draft the money it would take if they were drafted first overall.

  • evansj

    Just read your old “Cheifs point shaving” article. You wrote that they went 7-5 against the spread. Haven’t looked, but I’ll bet they are significantly better against the spread in losses than they are in wins. Which, intuitively anyway, would seem to support your analysis of their suckitude. But maybe that’s what sucky teams have to do, avoid short term embarrassment in the hopes that no one notices the ineptitude that reveals itself over the long term. Starting to get excited about Haley destroying Pittsburgh’s offense. Mendenhall seems like the perfect guy to play the role of Thomas Jones.

  • James

    very interesting… but Jimmy Johnson has more rings than Marino and Manning combined. The reason JJ gives more value to #1 overall than say #7 & #8 combined is because a team gets any player they want. The 7 & 8 holes don’t offer that luxury…

  • thurl

    Is the Jimmy Johnson Chart you’re referring to the Draft Value Chart where the top picks are valued 3000,2600,2200, etc? If so, my understanding is that chart was compiled from average salaries each pick gets signed at. It should work for an average team. Advocating a different strategy for a “shitty” team might make sense, but if they’re shitty they probably haven’t been drafting well anyway. They

  • hindery

    I think the major problem with your analysis is that it doesn’t take into account positional value. A starting guard is simply not as valuable as a starting quarterback.

    I think this may explain some of the difference in value.

    I also completely disagree with your assertion that two top ten picks are worth more than the #1 pick most years. Andrew Luck is worth more than 2 picks in the 5-10 range. Same thing last year with Cam Newton. It might not be true every year, but I think most years the JJ chart is pretty accurate and the #1 has high value.

    • FantasyDouche

      Jamarcus Russell, Tim Couch, Alex Smith and David Carr might disagree with you.