Sampling and the Curse of the Case Study



Preventing Violent Extremism, PVE, Countering Violent Extremism, CVE, P/CVE, Design, Evaluation, Sampling


Regardless of how the outcomes of a given P/CVE program are measured or evaluated, a fundamental, implicit (if not explicit) research question is: to what extent can obtained results apply to others within a given population.  In short, to what extent can the results apply to others, in general; what is the so-called generalizability of the findings?  In other words, the outcomes of a given P/CVE program are relatively useless unless they can be replicated, and the likelihood of replication is synonymous with generalizability.  Therefore, it is virtually impossible to overstate the importance of generalizability with respect to P/CVE research and evaluation, and generalizability is fundamentally a function of how well sampling is performed.  Therefore, it is also virtually impossible to overstate the importance of sampling with respect to an evidence-based approach to P/CVE.  This research methods brief describes fundamental issues (including potential pitfalls and means to avoid them) with respect to sampling in the context of P/CVE program design and evaluation: including issues related to sampling online “Big Data,” and “nested” (multi-level/hierarchical) program/research designs.


Baldwin, S. A., Bauer, D. J., Stice, E., & Rohde, P. (2011). Evaluating models for partially clustered designs. In Psychological Methods (Vol. 16, Issue 2, pp. 149–165). American Psychological Association.

Cialdini, R. B. (2013). Influence: Science and practice (5th ed.). Pearson.

Global Counter Terrorism Forum. (2013). Ankara memorandum on good practices for a multisectoral approach to countering violent extremism.

Groves, R. M., Fowler Jr, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey methodology. John Wiley & Sons.

Kaplan, R. M., Chambers, D. A., & Glasgow, R. E. (2014). Big data and large sample size: A cautionary note on the potential for bias. Clinical and Translational Science, 7(4), 342–346.

Kim, Y. J., Oh, Y., Park, S., Cho, S., & Park, H. (2013). Stratified sampling design based on data mining. Healthcare Informatics Research, 19(3), 186–195.

Leetaru, K. (2019). The Big Data revolution will be sampled: How “Big Data” has come to mean “small sampled data.” Forbes.

Palinkas, L. A., Horwitz, S. M., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2015). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 42(5), 533–544.

Pratkanis, A. R. (Ed.). (2011). The science of social influence: Advances and future progress. Psychology Press.

Ruggiero, G. (2020). How Big Data killed sampling. Altavia.

Sagan, C. (2011). The demon-haunted world: Science as a candle in the dark. Ballantine Books.

Sample. (n.d.). Better Evaluation. Retrieved June 15, 2019, from

Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer (Vol. 1). Sage.

West, S. G., Biesanz, J. C., & Kwok, O. M. (2004). Within-subject and longitudinal experiments: Design and analysis issues. In C. Sansone, C. C. Morf, & A. T. Panter (Eds.), The SAGE Handbook of Methods in Social Psychology (pp. 287–312).

Whittemore, A. S., & Halpern, J. (1997). Multi-stage sampling in genetic epidemiology. Statistics in Medicine, 16(1–3), 153–167.<153::aid-sim477>;2-7

Williams, M. J. (2022). Incentivizing Participants P/CVE Research, Evaluation, & Program Participants. Journal for Deradicalization, 31, 164–174.






Research Methods