Large trials of psychosocial interventions: examples from individual placement and support.

Abstract

Objectives: Large demonstration projects for health interventions often use randomized controlled trials (RCTs) to test the effectiveness of interventions implemented at larger scales, serving as crucial contributors to policy and funding decisions. Such trials are subject to limitations common to all RCTs, but their size and importance magnify the costs of failure to satisfy the assumptions for valid causal inference and generalizability. We examine common reasons for such threats to validity.

Methods: We examined large (N > 1,000) IPS RCTs for aspects of design and execution that undermine the validity of their results.

Results: We identified three large IPS RCTs and identified threats to validity associated with treatment adherence and attrition.

Conclusion: Large trials should rely on pilot studies to ensure that difficulties with recruitment, implementation of and participation in interventions, and follow-up measurement do not compromise study validity; intervention fidelity and participation should be measured to permit evaluation of study success. Funders should require and support the use of pilot studies and other prior research to justify the introduction of an intervention to a population and anticipate potential threats to validity.