Multi-site External Validation and Improvement of a Clinical Screening Tool for Future Firearm Violence

Abstract

Interventions in clinical settings, such as the emergency department (ED), are an opportunity for interpersonal firearm violence prevention, particularly among youth, whom interpersonal firearm violence disproportionately affects. A crucial prerequisite to successful clinical interventions is an accurate gauge of risk, to ensure the judicious allocation of scarce resources; providing that missing prerequisite is the primary goal of the proposed work. Machine learning methods, in contrast to traditional inferential statistical models, are distinguished by their emphasis on prospective prediction, and have enhanced clinical prediction in several fields, including heart disease, cancer diagnosis and outcomes, PTSD, suicide risk, and substance use, among others. Yet, with the exception of the SAFETY score—developed by the current investigative team—machine learning methods have not been leveraged to prospectively predict firearm violence.

In this proposed work, the research objectives are two-fold: 1) Externally validate the SAFETY score by determining its ability to predict firearm violence involvement within the next year on a new data set; and 2) Improve the SAFETY score by conducting a comparative analysis of four powerful machine learning methods: elastic net penalized logistic regression, random forests, support vector machines, and boosting (ensemble) methods. In this way, the project team is responding to Objective One: Research to help inform the development of innovative and promising opportunities to enhance safety and prevent firearm-related injuries, deaths, and crime. This approach is innovative because it builds upon the only work to apply machine learning methods to firearm violence prediction, and it is a promising opportunity to prevent firearm injuries because it will a) provide an explicit gauge of future firearm violence risk; and b) characterize risk factor effects in terms of their prospective prediction ability, unlike any prior research. Thus this research will both identify individuals in most need of intervention, and also point to potentially modifiable predictive factors. Properly addressing this research question in a generalizable way requires a contemporary data set with 1) a focus on a high-need, yet broad, study population; 2) comprehensive baseline measures that provide a broad basis for prediction; and 3) geographic variability (Midwest, West Coast, and East Coast) that enhances generalizability. Thus, 1,500 youth age 18-24 from urban EDs will be recruited in three broadly different locales—Flint, Philadelphia, and Seattle—and and a baseline survey will be administered covering several domains of potential risk factors for future violence, and follow up with those youth at 6- and 12-months to ascertain the primary outcome—firearm violence involvement (as victim or perpetrator)—as well as the secondary outcomes: high-risk firearm behaviors, non-firearm violence, and violent injury. Because this work requires a prospective longitudinal study, the project team has applied for Option B. This work will lay the ground for future research involving the development and testing of interventions for interpersonal firearm violence both by identifying potential high- leverage modifiable predictive factors, and by identifying youth most in need of intervention.