4.5 KiB
| title | chunk | source | category | tags | date_saved | instance |
|---|---|---|---|---|---|---|
| Theory-driven evaluation | 2/2 | https://en.wikipedia.org/wiki/Theory-driven_evaluation | reference | science, encyclopedia | 2026-05-05T04:26:31.832962+00:00 | kb-cron |
The target population includes a specification of who participants are and how they are recruited. The implementing organization (for instance a clinic or school) and its staff of implementers (for instance therapists or teachers) are responsible for allocating resources, training, and delivering the interventions. Intervention and service delivery protocols would include therapy manuals or subject curricula. Associated organizations and community partners refers to organisations other than the implementing organisation. In the case of a psychotherapy intervention, this may include schools or general practitioners who advertise the program or refer beneficiaries to it. Ecological context refers to aspects of the environment, for instance family, friends, co-workers, other students, etc., that may moderate the effects of a program.
== Theory-driven methods == The full-range of research methods has been argued to apply. Chen (2015) provides examples using randomized experiments, quasi-experimental designs, process and outcome monitoring, and qualitative methods. Although proponents of theory-driven evaluation are critical of "black box" experiments, Chen and Rossi (1983, p. 292) argue that theory-driven experiments are possible and desirable:
[A]dvocates of the black box experimental paradigm often neglect the fact that after randomization exogenous variables are still correlated with outcome variables. Knowing how such exogenous factors affect outcomes makes it possible to construct more precise estimates of experimental effects by controlling for such exogenous variables. It has been argued that theory-driven evaluation focusses too much on statistical approaches, such as randomized experiments, quasi-experiments, and structural equation modelling; however, a case has also been made for the importance of qualitative methods, particularly when developing program theories and understanding implementation. There is also methodological debate concerning whether realist evaluations, considered a particular kind of theory-driven approach, may include randomized controlled trials in any form. Some evaluators think they may and conduct what they call "realist trials". Others argue that a realist trial is an "oxymoron", and recommend instead calling them "theory-oriented trials". A 2023 review of trials described as realist concluded that whether they are really realist depends on "ontological and epistemological" commitments of evaluators and that differences "cannot be resolved" by reviewing studies conducted.
== Examples == Examples discussed in a 2011 systematic review of 45 theory-driven evaluations include:
An evaluation of the Fort Bragg Child and Adolescent Mental Health Demonstration, a managed mental health care system with a single point of entry, which used individual interviews, focus groups, and document review to assist the development of a theory of change. The theory explained why it was thought that an integrated care system would be more cost-effective than a fragmented system. An evaluation of a board game created to help teach secondary school business education. This evaluation developed a theory of change and used it to select measures and design regression analyses of process and outcome. An evaluation of a garbage reduction program. The program attempted to encourage residents to reduce the volume of garbage they produce by reducing the frequency of collection; however, an unintended negative consequence identified by the evaluation was that residents produced the same volume as before, simply storing their garbage in their homes on non-collection days. This effect was identified using an comparative interrupted time series analysis with autoregressive integrated moving average (ARIMA). A 2014 review of theory-driven evaluation in school psychology highlighted two illustrative examples:
An evaluation of conjoint behavioral consultation, a "strength-based intervention focused on building behavioral and social competence in children". The evaluation tested a theory of change using a cluster-randomised controlled trial and mediation analysis. An evaluation of repeated reading and vocabulary previewing which tested causal theory using case study methodology, an adapted alternating treatments design with six students.
== References ==