Research Methods Brief: Attrition Happens (and What to Do About It)

Michael J. Williams

Abstract


Attrition (participant "dropout") is the loss of participants from a program/initiative or longitudinal (e.g., pre/post) data collection.  If participants dropout for non-random, systematic reasons, those factors bias the sample and limit the study or evaluation’s generalizability.  The importance of statistically diagnosing participant attrition can scarcely be overstated, given that P/CVE research and evaluations are commonly concerned, not merely with the results from a given sample of participants, but whether, how, or to what extent the results might generalize to other, perhaps much broader samples.  Therefore, the threat to generalizability, posed by non-random participant attrition, threatens the very reason for conducting many, if not most, P/CVE-related research and evaluations.

Non-random attrition prevents research and evaluations from making valid claims or inferences about their target populations, and to know whether attrition likely threatens the validity of a project’s findings, one must test for it.  The present article includes step-by-step guidance on how to diagnose participant attrition, including discussion of the implications: implications that potentially can salvage a P/CVE-related program from seemingly problematic participant attrition.


Keywords


Attrition; Generalizability; Countering Violent Extremism; Preventing Violent Extremism; CVE; PVE; Evaluation; Research Method

Full Text:

PDF

References


Bhaskaran, K., & Smeeth, L. (2014). What is the difference between missing completely at random and missing at random? International Journal of Epidemiology, 43(4), 1336–1339. https://doi.org/10.1093/ije/dyu080

Davis, L. L., Broome, M. E., & Cox, R. P. (2002). Maximizing retention in community‐based clinical trials. Journal of Nursing Scholarship, 34(1), 47–53.

Grace-Martin, K. (n.d.). How to Diagnose the Missing Data Mechanism. The Analysis Factor. Retrieved December 19, 2020, from https://www.theanalysisfactor.com/missing-data-mechanism/

Kazdin, A. E. (2003). Drawing valid inferences I: Internal and external validity. In Research design in clinical psychology. Allyn & Bacon.

Koehler, D. (2017). Structural quality standards for work to intervene with and counter violent extremism. Counter Extremism Network Coordination Unit (KPEBW).

Menard, S. (2002). Applied logistic regression analysis (Vol. 106). Sage.

Senaviratna, N. A. M. R., & Cooray, T. M. J. A. (2019). Diagnosing multicollinearity of logistic regression models. Asian Journal of Probability and Statistics, 5(2), 1–9. https://doi.org/10.9734/ajpas/2019/v5i230132

Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Allyn & Bacon/Pearson Education.

West, S. G., Biesanz, J. C., & Kwok, O. M. (2004). Within-subject and longitudinal experiments: Design and analysis issues. In C. Sansone, C. C. Morf, & A. T. Panter (Eds.), The SAGE Handbook of Methods in Social Psychology (pp. 287–312). https://doi.org/10.4135/9781412976190.n13

West, S. G., Biesanz, J. C., & Pitts, S. C. (2000). Causal inference and generalization in field settings: Experimental and quasi-experimental designs. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (pp. 40–84). Cambridge University Press.

Williams, M. J. (2020). Preventing & countering violent extremism: Designing and evaluating evidence-based programs. Routledge. https://thescienceofpcve.org/book/


Refbacks

  • There are currently no refbacks.


Copyright (c) 2021 Michael J. Williams

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

ISSN: 2363-9849 

Proud Member of the Directory of Open Access Journals (DOAJ)