Evaluation Review最新文献

The evidence-based policy movement assumes that rigorously evaluated interventions replicate across contexts to produce similar effects. This universalist assumption may be unwarranted for complex social programs, where context shapes effectiveness. This study examines generalizability using 15 cash transfer programs evaluated across 12 Middle Eastern and North African countries through four complementary approaches: leave-one-out cross-validation, permutation tests, equivalence testing, and fragility indices. Leave-one-out analysis demonstrates robust stability, with 100% of key relationships maintaining a magnitude within 25% of full-sample estimates. The correlation between transfer size and poverty reduction (r = 0.711, p < .001) shows strong consistency, and permutation tests yield p = .001, indicating genuine signals rather than chance patterns. Equivalence testing reveals that programs with nearly identical designs can produce statistically distinguishable outcomes, with confidence intervals excluding practical equivalence in 67% of matched comparisons. Fragility indices of 2 to 3 for most relationships indicate moderate robustness. These findings suggest that some design-outcome relationships show stability while context matters substantially for program effectiveness. This study offers methodological tools for assessing evidence robustness in small-sample comparative research and argues for epistemic humility in cross-context predictions.

By exploiting the panel data of 100 countries over the period 2006 to 2022, I show how climate change policies and green growth within a circular economy affect entrepreneurship. Higher energy intensity predicts a lower rate of new firm creation. In contrast, higher green growth correlates with a higher number of new businesses. The interaction between energy intensity and green growth indicates that environmental support can partially offset the adverse effects of energy-intensive economic structures. Using genetic distance to the United Kingdom as an instrumental variable, I establish a causal relationship. In particular, the findings suggest that well-designed environmental policies can act as a catalyst for new firm formation, especially in circular economies facing structural and developmental constraints.

The evidence base for parent support groups in child welfare systems is limited but growing. Evaluation of such programs requires flexible and context-specific designs that capture both outcomes and mechanisms of change. This exploratory mixed-methods evaluation examined a peer-led support group program for parents who experienced child removal in a large Florida county to better understand how these programs operate and benefit parents working toward reunification. A total of 22 parents participated in weekly sessions facilitated by peer specialists with prior child welfare involvement. Quantitative assessments measured changes in psychosocial functioning and engagement in child welfare services, while semi-structured interviews (n = 8) explored participant perceptions of group processes and benefits. Significant decreases were observed in depression, trauma, and loneliness, alongside increases in perceived emotional support. Thematic analysis identified four interrelated themes that informed an empirically grounded conceptual framework describing the functions and benefits of support groups.This evaluation study demonstrates how mixed-methods approaches can yield actionable evidence from small scale community-based interventions and offers insight into the mechanisms that may inform future implementation and evaluation of peer-led models.

Brokers serve as key connectors linking academic researchers who might otherwise remain unconnected, to co-authorship networks. This study examines whether more complex interdisciplinary co-authorships yield greater scholarly impact than ties with authors from the same discipline, which are facilitated by cognitive similarity. It also tests the extent to which such collaborations influence the academic performance of female researchers. The study analyzed 594 authors and 271 papers in the field of social learning. Despite their complexity, regression analyses confirm the benefits of novel sources and combinations of interdisciplinary knowledge. The higher coordination and communication costs of interdisciplinary collaboration are offset by the potential of the new knowledge generated. Interdisciplinary collaboration is associated with higher performance among female scholars, contributing to increased recognition and reputation. This pattern suggests that collaboration strategies and institutional support should reduce coordination barriers in interdisciplinary co-authorship and facilitate women's access to high-impact brokerage opportunities.

This paper aims to demonstrate how evaluation methods can inform the development of a program, using the example of a novel housing mobility program. The limited availability of Housing Choice Vouchers (HCVs) and other affordable housing assistance restricts the ability to address US national housing needs; millions more households qualify for HCVs than receive them. Those with HCVs encounter challenges in using vouchers in so-called high-opportunity areas. To address the dearth of HCVs and difficulty using them, a collaboration between faculty at Ohio State University and the community developed a new housing mobility program to increase the affordable housing supply, engaging stakeholders to understand the needs of low-income renters who are less likely to obtain an HCV. This paper outlines the Families Flourish program's development using both developmental and formative evaluations, examining the progress of the implementation and immediate outcomes using mixed methods. The developmental evaluation process required collaboration with a wide variety of stakeholders, partners, and the project team. According to the formative evaluation of the pilot program, participants' experiences suggested that enhanced housing security was associated with improvements in physical and mental health for parents and their children, and that a more stable and safe neighborhood environment was similarly associated with gains in employment, education, and income. Throughout the process of evaluation, particularly the developmental evaluation, the evaluator's communication and collaboration skills were essential to the methodology. The study concludes with lessons learned, as well as recommendations for policy and directions for future research.

This paper evaluates the impact of racial and low-income quotas on the academic performance of senior students in Brazilian colleges and universities. Using longitudinal data from Brazil's Higher Education Census and the National Examination of Student Performance (ENADE), and employing a fixed effects approach, the study examines the influence of these quotas on student outcomes. The results show that neither racial nor low-income quotas significantly impact the academic performance of either quota or non-quota students. This finding holds across different groups of majors, indicating that the inclusion of quota students does not detract from overall student achievement.

Policymakers are increasingly focused on promoting job quality as a means to enhance financial stability and well-being through employment. Thus, evaluations of public workforce inventions are adding employment outcome measures that go beyond wages in order to assess job quality more broadly. This study employs a mixed methods design to understand what aspects of job quality, beyond wages and benefits, are valued by most participants of a federally funded sectoral job training program and then assess the impact of the program using these grounded insights. We use secondary data from a randomized control trial of the Health Profession Opportunity Grant (HPOG) for our analyses. Themes from a qualitative analysis of 153 participant interviews informed the selection of three flexibility-related outcome measures for quantitative analysis using experimental survey data. Intention-to-treat estimates were generated using linear regression and ordered logit models, with subgroup analyses examining race/ethnicity, nativity, parental status, and education. Workplace flexibility emerged as a widely valued aspect of job quality, with participants emphasizing scheduling choice as a strategy to achieve work-life balance. Statistical models demonstrate that the program facilitates access to jobs with greater workplace flexibility. Heterogeneity analysis reveals differential impacts by ethnicity and education level, and notable positive results for individuals with children, a group more likely to value and benefit from flexibility. HPOG improved access to jobs offering workplace flexibility-an aspect of job quality prioritized by participants. Findings underscore the importance of incorporating non-wage metrics, especially work-life balance, in workforce program evaluations.

Knowing how to implement emergency material scheduling and transportation during emergency rescues, such as major and critical emergencies, has become a research hotspot in academia and industry in recent years. To leverage the speed and terrain-insensitive advantages of aviation, the weak limitations of geographical conditions must be addressed, and the material scheduling efficiency of aviation rescue centers in disaster-stricken areas needs to be improved. In this study, CRITIC and cloud model theory were integrated to evaluate the urgency of emergency material demands in different flood-stricken areas under catastrophic flood disaster risks. Furthermore, a mathematical model for a single aviation emergency rescue center to dispatch emergency materials to multiple disaster-stricken sites was designed based on the optimized ant colony algorithm. A penalty function was then incorporated to formulate a multi-objective aviation scheduling model, aiming to minimize both total rescue time and total cost. The model was solved using an improved genetic algorithm. Taking rainstorm-induced flood disasters in the megacity of Zhengzhou, China, in 2021 as the empirical research case, the operating paths for the aviation emergency rescue center to serve multiple demand points were optimized. The impact of material demand urgency on scheduling decisions was analyzed. Results revealed that when material demand urgency is considered, aircraft complete deliveries according to urgency rankings and return to the center. All tasks can be completed within the required time via four routes. Although total time increases, economic cost is significantly reduced, and disaster loss is mitigated. The findings obtained from this study provide a decision-making reference for improving the efficiency of aviation emergency rescue and enhancing urban risk management capabilities in response to major and critical emergencies.

The aim of this study was to investigate the impact of the shift in recent years from passive to active parental consent for youth' participation in research. A longitudinal study that took place during this change provided the unique opportunity to analyze which adolescent (demographic, alcohol use, norms) and parental factors predict the recipient of consent. The sample consisted of 691 adolescents between 12 and 17 years old (M_age = 14.22; SD = 1.03; 44.9% boys). Factors for each and across five domains (socio-demographic, alcohol use, individual, group/peers, and parental factors) were included in a multiple logistic regression analysis to predict non-consent at follow-up. Across domains, results showed that adolescents who are older (OR = 0.60, p < .001), female (OR = 0.60, p < .001) and those perceived less strict rules about alcohol (OR = 1.26, p = .05), have a higher odds of not having consent for participation. These findings indicate that a specific selection of adolescents were given permission to participate in research, yet this was not particularly an at-risk group. Implications of these findings, such as the balance between autonomy and protection, are discussed.

Negative menstruation experiences adversely affect schoolgirls' social participation, education, and overall health, yet comprehensive menstrual health interventions are limited in Tanzanian schools. This study evaluated the feasibility, acceptability, cost, and potential impact mechanisms of a comprehensive school-based menstrual, sexual, and reproductive health (MSRH) intervention. The intervention, piloted in four secondary schools, included education sessions for girls and boys, pain management, distribution of menstrual kits (reusable pads and menstrual cups), WASH improvements, and stakeholder engagement. A mixed-methods process evaluation assessed acceptability, fidelity, cost, context, and potential impact mechanisms using qualitative methods (in-depth interviews, focus group discussions, and WASH observations) and quantitative methods (survey questionnaires, structured observation of education sessions, costing, and monitoring data). The intervention was well received by students, teachers, and local government authorities; MSRH education reached 86% of schoolgirls and 72% of schoolboys, while over 93% of girls received menstrual kits. Total implementation cost across the four schools was 111,347,467 TZS (38,003 GBP), approximately 39.05 GBP per student, lower than comparable initiatives in East Africa. Findings indicate the intervention is feasible and acceptable in school settings and can inform future menstrual health and hygiene programs, though further research is needed to assess broader effectiveness and sustainability.