Research Summary: Predictive Utility of Commercial Grade Technologies for Assessing Musculoskeletal Injury Risk in US Marine Corps Officer Candidates

Bird MB, Koltun KJ, Mi Q, Lovalekar M, Martin BJ, Doyle TLA and Nindl BC (2023), Predictive utility of commercial grade technologies for assessing musculoskeletal injury risk in US Marine Corps Officer candidates. Front. Physiol. 14:1088813. 

Key Takeaways

• Composite risk scores and other measures obtained from commercial technologies (Sparta Science) are significantly associated with risk of lower body and torso musculoskeletal injuries in US Marine Corps Officer Candidates.
• Sex, age, and 3-mile run time are associated with risk of lower body and body and torso musculoskeletal injuries in US Marine Corps Officer Candidates.
• In isolation, no single measure from traditional assessments or commercial scans is sufficiently predictive to determine which candidates will experience MSKI during a 10-week training course; larger populations and serial screening tests may increase the sensitivity of machine learning based MSKI prediction.

“When deciding what commercial-grade MSKI machine learning model and screening measures to implement it is important to understand (1) what population normative ranges the model was trained on and (2) if the screening measures are applicable to the population of interest."

Population: 689 individuals from four intake classes of the US Marine Corps Officer Candidates School (OCS) training course

Summary

The questions covered:

• Are composite risk scores from commercial technologies, specifically Sparta force plate measures and DARI markerless motion capture measures, predictive of musculoskeletal injury risk in Marine Corps Officer candidates?
• Do population-specific models trained on MCO candidates using common MSKI risk factors and component variables from the Sparta and DARI scans outperform the composite measures in predicting musculoskeletal injuries?

The authors conducted this study to evaluate the predictive value of commercial technologies for assessing musculoskeletal injury (MSKI) risk in US Marine Corps Officer candidates. They evaluated two types of commercial movement assessment technologies: Sparta Science force plate scans of countermovement jumps and DARI markerless motion capture for a movement screen consisting of a series of lunges, squats, and jumps. For both technologies, candidates were evaluated prior to the start of the 10-week training course. The outcomes of interest, MSKIs, were collected based on reports generated when candidates presented to the physical therapy department for treatment during the course of the training program. All lower body and torso MSK injuries were included for analysis. To fully evaluate the predictive capability of the technologies, the research team analyzed the composite risk scores developed by Sparta Science and DARI, respectively, as well as the component variables from each scan in population-specific machine learning models.

Evaluation of the primary analysis indicated that several traditional factors and composite scores from Sparta and Dari scans were significantly related to MSKI risk. Specifically, sex, age, 3-mile run time, Sparta MSKI Health Score, and DARI Readiness and Performance Scores were found to be predictors of lower body and torso musculoskeletal injuries in MCO training candidates. The area under the curve (AUC) values associated with these variables when conducting receiver operating characteristic assessments, however, indicated low predictive value when used as classifiers. Assessment of the population-specific models trained on the component measures from the Sparta and DARI scans, as well as traditional risk factors, revealed fair to excellent AUC performance in the training datasets. When the same models were deployed in test datasets, however, their AUC performance fell to chance or poor levels. This trend held for all models using traditional and technology-based measures, whether alone or in combination, but combining traditional risk factors with scan-based factors improved performance incrementally on average. Evaluation of variable importance across the population-specific models indicated that machine learning algorithm choice had a large impact on which variables were selected as most important to predicting MSKIs.

The results of this study indicate that Sparta and DARI scan-based measures are predictive of lower body and torso MSK injuries in MCO trainees beyond the predictive value of traditional MSKI risk factors alone. In the current analysis, however, this predictive value did not extend to clinical utility when used as classifiers to differentiate between trainees who would and would not experience an MSKI during the 10-week training course. The population-specific models trained on scan-based and traditional risk factors demonstrated promising initial predictive power, but their performance diminished when deployed in a novel testing dataset. Taken as a whole, these results suggest that both traditional and technology-enabled measures provide insight into MSKI risk in Marine Corps Officer trainees, but none are sufficiently predictive in isolation to serve as classifiers to differentiate between those who will and will not experience a lower body or torso MSKI.

Abstract

Recently, commercial-grade technologies have provided black box algorithms potentially relating to musculoskeletal injury (MSKI) risk and functional movement deficits, in which may add value to a high-performance model. Thus, the purpose of this manuscript was to evaluate composite and component scores from commercial-grade technologies associations to MSKI risk in Marine Officer Candidates. 689 candidates (Male candidates = 566, Female candidates = 123) performed counter movement jumps on SPARTA force plates and functional movements (squats, jumps, lunges) in DARI markerless motion capture at the start of Officer Candidates School (OCS). De-identified MSKI data was acquired from internal OCS reports for those who presented to the Physical Therapy department for MSKI treatment during the 10 weeks of training. Logistic regression analyses were conducted to validate the utility of the composite scores and supervised machine learning algorithms were deployed to create a population specific model on the normalized component variables in SPARTA and DARI. Common MSKI risk factors (cMSKI) such as older age, slower run times, and females were associated with greater MSKI risk. Composite scores were significantly associated with MSKI, although the area under the curve (AUC) demonstrated poor discrimination (AUC = .55–.57). When supervised machine learning algorithms were trained on the normalized component variables and cMSKI variables, the overall training models performed well, but when the training models were tested on the testing data the models classified MSKI “by chance” (testing AUC avg = .55–.57) across all models. Composite scores and component population-specific models were poor predictors of MSKI in candidates. While cMSKI, SPARTA, and DARI models performed similarly, this study does not dismiss the use of commercial technologies but questions the utility of a singular screening task to predict MSKI over 10 weeks. Further investigations should evaluate occupation-specific screening, serial measurements, and/or load exposure for creating MSKI risk models. 

doi: 10.3389/fphys.2023.1088813