Variation in Performance of Commonly Used Statistical Methods for Estimating Effectiveness of State-Level Opioid Policies on Opioid-Related Mortality
Beth Ann Griffin, Megan S. Schuler, Elizabeth A. Stuart, Stephen Patrick, Elizabeth McNeer, Rosanna Smart, David Powell, Bradley Stein, Terry Schell, Rosalie Liccardo Pacula
Over the last two decades, there has been a surge of opioid-related overdose deaths resulting in a myriad of state policy responses. Researchers have evaluated the effectiveness of such policies using a wide-range of statistical models, each of which requires multiple design choices that can influence the accuracy and precision of the estimated policy effects. This simulation study used real-world data to compare model performance across a range of important statistical constructs to better understand which methods are appropriate for measuring the impacts of state-level opioid policies on opioid-related mortality. Our findings show that many commonly-used methods have very low statistical power to detect a significant policy effect (< 10%) when the policy effect size is small yet impactful (e.g., 5% reduction in opioid mortality). Many methods yielded high rates of Type I error, raising concerns of spurious conclusions about policy effectiveness. Finally, model performance was reduced when policy effectiveness had incremental, rather than instantaneous, onset. These findings highlight the limitations of existing statistical methods under scenarios that are likely to affect real-world policy studies. Given the necessity of identifying and implementing effective opioid-related policies, researchers and policymakers should be mindful of evaluation study statistical design
Year of publication: |
April 2020
|
---|---|
Authors: | Griffin, Beth Ann |
Other Persons: | Schuler, Megan S. (contributor) ; Stuart, Elizabeth (contributor) ; Patrick, Stephen (contributor) ; McNeer, Elizabeth (contributor) ; Smart, Rosanna (contributor) ; Powell, David (contributor) ; Stein, Bradley (contributor) ; Schell, Terry L. (contributor) ; Pacula, Rosalie Liccardo (contributor) |
Institutions: | National Bureau of Economic Research (contributor) |
Publisher: |
2020: Cambridge, Mass : National Bureau of Economic Research |
Subject: | Sterblichkeit | Mortality | Drogenkonsum | Drug consumption | Wirkungsanalyse | Impact assessment | Drogenpolitik | Drug policy | Empirische Methode | Empirical method | Simulation | Methodologie | Methodology | Statistische Methode | Statistical method |
Saved in:
Extent: | 1 Online-Ressource illustrations (black and white) |
---|---|
Series: | NBER working paper series ; no. w27029 |
Type of publication: | Book / Working Paper |
Language: | English |
Notes: | System requirements: Adobe [Acrobat] Reader required for PDF files Mode of access: World Wide Web Hardcopy version available to institutional subscribers |
Other identifiers: | 10.3386/w27029 [DOI] |
Source: | ECONIS - Online Catalogue of the ZBW |
Persistent link: https://ebvufind01.dmz1.zbw.eu/10012481986