product blog request demo
home
product
case studies
published research
training
schedule session
blog
about us
news
search
contact us
Call Us
March 28, 2016
The NFL Combine and Pro Day: 30 Years of Bad Data

The College of Business at the University of Louisville conducted a study in 2008 with the goal of determining what (if any) correlation there was between the players’ performance in the combine tests (both physical and psychological), and their performance in the NFL.  They found no correlation between the Combine performance and actual game performance, with the exception of sprint tests for running backs.

NFL teams recognize this problem, one GM told me that the medical exams are the only worthwhile information anymore, outside the most obvious criteria which is their ability to play football.  Yet still, every NFL hopeful runs through these series of tests that has gone relatively unchanged over the last 30 years.  So if everyone knows these combine tests aren’t valid measures of an athlete’s performance on the field, why do we still do them?

Bad Data; What is It?

Sparta_HQ_football-21The largest theory behind such resistance to evolve is that the league now has over 30 years of historical data.  Although the metrics are less than optimal, this data provides context to compare NFL hopefuls to players on current NFL rosters or even retired NFL greats.  Some of the most difficult conversations we have with organization executives is during the moment of realization when their historical data just cannot be used. If you have been committed to doing a certain process for so long, it is often hard to accept that those efforts and results may not be of any value.  More specifically, there is no technology that can “cleanse” or launder bad data, often described by software engineers as “Garbage In, Garbage Out.”

Bad data is either unreliable (inconsistent results), or lacks validity (not measuring what we intended to measure).  Reliability can certainly be improved by swapping hand timers with electronic timing systems, such as the ALGE system we use at our performance facility.  But the largest problem with current combine/pro-day assessments is the validity.

The Key is Validity

The questions we need to ask are; ‘are these tests measuring what we are intending to measure?’  ‘Are these assessments predictive of performance in the NFL?’  We continue to expect lineman to perform a 40 yard sprint, a distance which will never be seen by most NFL players in particular this larger group who operates in a confined space that resembles a closet. So the 40 yard becomes more of rite of passage; we watch to see if a rhino can actually perform the feat required of a cheetah without getting hurt.

The concern is not limited to the 40 yard dash, as such validity applies to the 225 bench test as well. This test generally lasts 20 seconds and beyond. We have had considerable success in developing combine athletes for this assessment by working on one of the most important facets for this test; breathing. Such an assessment requires sufficient lung capacity to keep a bridge in your back and subsequently less distance required for barbell to move. Such energy and biomechanical qualities make the assessment far more valid for big wave surfers than athletes who have an average play of 6-7 seconds.

The reality is that most combine tests are skills.  Skills that can be improved in a matter of months with good coaching and proper technique.  While we have had major success in preparing our athletes for these skills in order to improve their draft stock, we think everyone’s time could be better spent.

Sparta_HQ_football-35

The Solution; Settings to Create a Reliable Test

So while we cannot cancel these events, perhaps there are better alternatives. The solution cannot be just anything electronic, often times we confuse digital and scientific. Being on an app or software platform does not make the results reliable or valid, so lack of publications or even transparency of the results should be a key red flag that a technology was not an upgrade from the pencil or stopwatch.

The solution is called the Sparta Signature.  We have 3 years of NFL prospect data; senior bowl rosters, pro day collection, and team visits who use our software to gather objective data. The signature is reliable due to the variables assessed and the creation of settings, to ensure only good data is allowed to be collected. Most importantly, the assessment is valid because it is multifactorial to assess both performance capabilities and injury risk.

In addition to the signature, other assessments like sprinting or agility, can reveal an athlete’s ability to utilize their force production in a more skilled fashion. After all, a 40 yard sprint is a skill; the set-up, the first 2 steps, and top end speed mechanics. Yet, with all the time we spend on improving an athlete’s skill for a test, could we be neglecting more foundational needs to build better football players? Reducing the duration of such skill to 10 yards allows the assessment to be far more physical rather than skill.

The solution is an easy one; most of us know more reliable and valid assessments. The challenge is having the courage and conviction to realize historical garbage.

Posted
March 28, 2016
In
Related Posts

3 thoughts on “The NFL Combine and Pro Day: 30 Years of Bad Data”

  1. The Seahawks are very big on the SPARQ formula, which is a derivative of the combine measurements[1], and it’s been working very well for them. Seattle is possibly the most measurement-seeking team in the NFL, actually, and has a philosophical heritage that extends down from the notorious Al Davis. Al Davis had some good ideas even when he applied them incorrectly.

    When you look at players like Revis and Patrick Peterson and Adrian Peterson and Calvin Johnson, players who scored in the 95+ percentile in combine events and then went on to be stars, it’s hard to argue that combine events aren’t predictive indicators. You have to know which events apply to which positions and you have to know how to bin different tiers of the events for predictive purposes. Context matters. But players near the top of the draft tend to do very well in the combine and they also tend to hit more than players lower in the draft. This is probably not a coincidence. If Louisiana couldn’t find a connection, you have to question the usefulness of the methods they used. I would adjust my prior only very slightly based on their study.

    One problem with combine tests, and something Seattle adjusts for, is length. Wingspan is huge in most sports. It’d be a mistake to assume that people are naively using a 40 yard dash time without taking into account other factors, like reach.

    The fact is, there’s just not enough data to use traditional probability and stats methods to make predictions sometimes. Human brains are pretty good at making predictions with limited data, though, which is why betting markets and the stock market are so efficient.

    I love the ideas behind the Movement Signature, and would love to see it added to the combine. I’ve no doubt it’d be more predictive than the collection of tests they have now. It’d probably be hard to get branded vocabulary accepted there, though, I’d imagine. Nike owns SPARQ and even they don’t have their formula associated with the combine.

    [1] Or mostly. The exact incarnation of the current version is proprietary I think.

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe to our Blog