Increasing Effectiveness in Sustainability Research: Comparative Analysis
Scientific research is done to generate new knowledge. Research projects and programs also strive for excellence by means of impact. New transdisciplinary approaches are emerging that place research around the interests of stakeholders to produce more applicable and relevant knowledge. However, as these approaches are new and because they engage in complex systems, it has been difficult to know what works, how, why, and what opportunities exist to enhance the effectiveness of the research. Comparative analyses of applied research project case studies will generate empirical evidence and theoretical insights about whether and how transdisciplinary research contributes to learning, change and, ultimately, impact.
Through comparative analysis, we will synthesize what elements of project design and implementation are critical for enhancing research effectiveness and achieving outcomes.
Purpose: To contribute to improved and more effective inter- and transdisciplinary research.
Guiding question: How can we improve research effectiveness?
Method: This project examines and compares the components of project design and implementation with outcomes and score them for their effectiveness based on the Transdisciplinary Research Quality Assessment Framework, in order to analyze and compare the design, implementation, and outcomes of a series of applied research projects.
Phase 1. Evaluating policy-relevant research with a Theory of Change
Cases analyzed: Sustainable Wetlands Adaptation and Mitigation Program, Furniture Value Chains, Global Comparative Study REDD+, and Sustainable Forest Management in the Congo Basin
Results: This theory-based approach for research evaluation provides a valuable means of assessing research effectiveness (summative value) and supporting learning and adaptation (formative value) at the project or program scale. The approach is well suited to research-for-development projects represented by the case studies, but is also applicable to any research that aspires to have a social impact. However, the retrospective theory of change development proved challenging and resulted in simplistic theories of change. More work is needed to draw on theories of knowledge translation and policy processes to develop and further test more sophisticated and complex-aware theories of change.
Read the full peer-reviewed publication: Belcher, B., Suryadarma, D., and Halimanjaya, A. (2017) Evaluating Policy-Relevant Research: Lessons from a Series of Theory-Based Outcomes Assessments. Palgrave Communications 3: 1-16.
Phase 2. Assessing the effectiveness of research projects with theory of change and the Transdisciplinary Quality Assessment Framework (in progress).
Cases analyzed: Sustainability of Timber Harvesting in Brazil Nut Concessions, Political Economy of Fire and Haze, Global Comparative Study on Land Tenure, Implementation of Agroforestry Concessions, Sustainable Wetlands Adaptation and Mitigation Program.
Results: Projects with more transdisciplinary research design and implementation make more diverse contributions and can leverage more diverse mechanisms of change. By leveraging diverse contributions and mechanisms, projects have a greater potential for influence across more impact pathways.
Read the full peer-reviewed publication: Belcher, B. M., Claus, R., Davel, R., & Ramirez, L. F. (2019). Linking transdisciplinary research characteristics and quality to effectiveness: A comparative analysis of five research-for-development projects. Environmental Science & Policy, 101: 192-203. https://doi.org/10.1016/j.envsci.2019.08.013