Research Methods Brief: Attrition Happens (and What to Do About It)
Keywords:Attrition, Generalizability, Countering Violent Extremism, Preventing Violent Extremism, CVE, PVE, Evaluation, Research Method
Attrition (participant "dropout") is the loss of participants from a program/initiative or longitudinal (e.g., pre/post) data collection. If participants dropout for non-random, systematic reasons, those factors bias the sample and limit the study or evaluation’s generalizability. The importance of statistically diagnosing participant attrition can scarcely be overstated, given that P/CVE research and evaluations are commonly concerned, not merely with the results from a given sample of participants, but whether, how, or to what extent the results might generalize to other, perhaps much broader samples. Therefore, the threat to generalizability, posed by non-random participant attrition, threatens the very reason for conducting many, if not most, P/CVE-related research and evaluations.
Non-random attrition prevents research and evaluations from making valid claims or inferences about their target populations, and to know whether attrition likely threatens the validity of a project’s findings, one must test for it. The present article includes step-by-step guidance on how to diagnose participant attrition, including discussion of the implications: implications that potentially can salvage a P/CVE-related program from seemingly problematic participant attrition.
Bhaskaran, K., & Smeeth, L. (2014). What is the difference between missing completely at random and missing at random? International Journal of Epidemiology, 43(4), 1336–1339. https://doi.org/10.1093/ije/dyu080
Davis, L. L., Broome, M. E., & Cox, R. P. (2002). Maximizing retention in community‐based clinical trials. Journal of Nursing Scholarship, 34(1), 47–53.
Grace-Martin, K. (n.d.). How to Diagnose the Missing Data Mechanism. The Analysis Factor. Retrieved December 19, 2020, from https://www.theanalysisfactor.com/missing-data-mechanism/
Kazdin, A. E. (2003). Drawing valid inferences I: Internal and external validity. In Research design in clinical psychology. Allyn & Bacon.
Koehler, D. (2017). Structural quality standards for work to intervene with and counter violent extremism. Counter Extremism Network Coordination Unit (KPEBW).
Menard, S. (2002). Applied logistic regression analysis (Vol. 106). Sage.
Senaviratna, N. A. M. R., & Cooray, T. M. J. A. (2019). Diagnosing multicollinearity of logistic regression models. Asian Journal of Probability and Statistics, 5(2), 1–9. https://doi.org/10.9734/ajpas/2019/v5i230132
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Allyn & Bacon/Pearson Education.
West, S. G., Biesanz, J. C., & Kwok, O. M. (2004). Within-subject and longitudinal experiments: Design and analysis issues. In C. Sansone, C. C. Morf, & A. T. Panter (Eds.), The SAGE Handbook of Methods in Social Psychology (pp. 287–312). https://doi.org/10.4135/9781412976190.n13
West, S. G., Biesanz, J. C., & Pitts, S. C. (2000). Causal inference and generalization in field settings: Experimental and quasi-experimental designs. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (pp. 40–84). Cambridge University Press.
Williams, M. J. (2020). Preventing & countering violent extremism: Designing and evaluating evidence-based programs. Routledge. https://thescienceofpcve.org/book/
The JD Journal for Deradicalization uses a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND) Licence. You are free to share - copy and redistribute the material in any medium or format under the following conditions:
Attribution — You must give appropriate credit, provide a link to the license, andindicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
NonCommercial — You may not use the material for commercial purposes.
NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.