March 28, 2016
The College of Business at the University of Louisville conducted a study in 2008 with the goal of determining what (if any) correlation there was between the players’ performance in the combine tests (both physical and psychological), and their performance in the NFL. They found no correlation between the Combine performance and actual game performance, with the exception of sprint tests for running backs.
NFL teams recognize this problem, one GM told me that the medical exams are the only worthwhile information anymore, outside the most obvious criteria which is their ability to play football. Yet still, every NFL hopeful runs through these series of tests that has gone relatively unchanged over the last 30 years. So if everyone knows these combine tests aren’t valid measures of an athlete’s performance on the field, why do we still do them?
Bad Data: What is It?
The largest theory behind such resistance to evolve is that the league now has over 30 years of historical data. Although the metrics are less than optimal, this data provides context to compare NFL hopefuls to players on current NFL rosters or even retired NFL greats. Some of the most difficult conversations we have with organization executives is during the moment of realization when their historical data just cannot be used. If you have been committed to doing a certain process for so long, it is often hard to accept that those efforts and results may not be of any value. More specifically, there is no technology that can “cleanse” or launder bad data, often described by software engineers as “Garbage In, Garbage Out.”
Bad data is either unreliable (inconsistent results), or lacks validity (not measuring what we intended to measure). Reliability can certainly be improved by swapping hand timers with electronic timing systems, such as the ALGE system we use at our performance facility. But the largest problem with current combine/pro-day assessments is the validity.
The Key is Validity
The questions we need to ask are: ‘are these tests measuring what we are intending to measure?’ ‘Are these assessments predictive of performance in the NFL?’ We continue to expect lineman to perform a 40 yard sprint, a distance which will never be seen by most NFL players in particular this larger group who operates in a confined space that resembles a closet. So the 40 yard becomes more of rite of passage; we watch to see if a rhino can actually perform the feat required of a cheetah without getting hurt.
The concern is not limited to the 40 yard dash, as such validity applies to the 225 bench test as well. This test generally lasts 20 seconds and beyond. We have had considerable success in developing combine athletes for this assessment by working on one of the most important facets for this test; breathing. Such an assessment requires sufficient lung capacity to keep a bridge in your back and subsequently less distance required for barbell to move. Such energy and biomechanical qualities make the assessment far more valid for big wave surfers than athletes who have an average play of 6-7 seconds.
The reality is that most combine tests are skills. Skills that can be improved in a matter of months with good coaching and proper technique. While we have had major success in preparing our athletes for these skills in order to improve their draft stock, we think everyone’s time could be better spent.
The Solution: Settings to Create a Reliable Test
So while we cannot cancel these events, perhaps there are better alternatives. The solution cannot be just anything electronic, often times we confuse digital and scientific. Being on an app or software platform does not make the results reliable or valid, so lack of publications or even transparency of the results should be a key red flag that a technology was not an upgrade from the pencil or stopwatch.
The solution is called the Sparta Signature. We have 3 years of NFL prospect data; senior bowl rosters, pro day collection, and team visits who use our software to gather objective data. The signature is reliable due to the variables assessed and the creation of settings, to ensure only good data is allowed to be collected. Most importantly, the assessment is valid because it is multifactorial to assess both performance capabilities and injury risk.
In addition to the signature, other assessments like sprinting or agility, can reveal an athlete’s ability to utilize their force production in a more skilled fashion. After all, a 40 yard sprint is a skill; the set-up, the first 2 steps, and top end speed mechanics. Yet, with all the time we spend on improving an athlete’s skill for a test, could we be neglecting more foundational needs to build better football players? Reducing the duration of such skill to 10 yards allows the assessment to be far more physical rather than skill.
The solution is an easy one; most of us know more reliable and valid assessments. The challenge is having the courage and conviction to realize historical garbage.