Increasing Effectiveness in Sustainability Research: Comparative Analysis
Science generates new knowledge and research projects and programs also strive for excellence by means of social impact. Newly emerging transdisciplinary approaches place research around the interests of stakeholders, producing more applicable and relevant knowledge for societal change. However, as these approaches are new and because they engage in complex systems, it has been difficult to know what works, how, and why. How to enhance the effectiveness of such research has also been unclear. Comparative analyses of applied research project case studies generate empirical evidence and offer theoretical insights about whether and how transdisciplinary research contributes to learning, change, and ultimately impact.
Through comparative analysis, we will synthesize what elements of project design and implementation are critical for enhancing research effectiveness and realizing outcomes.
Purpose: To contribute to improved and more effective inter- and transdisciplinary research.
Guiding question: How can we improve research effectiveness?
Method: This project examines and compares the components of project design and implementation with outcomes and scores them for their effectiveness based on the Transdisciplinary Research Quality Assessment Framework in order to analyze and compare the design, implementation, and outcomes of a series of applied research projects.
Phase 1. Evaluating Policy-relevant Research with a Theory of Change
Cases analyzed: Sustainable Wetlands Adaptation and Mitigation Program, Furniture Value Chains, Global Comparative Study REDD+, and Sustainable Forest Management in the Congo Basin
Results: This theory-based approach for research evaluation works well to assess research effectiveness and support learning and adaptation at project/program scales. The approach is well suited to research-for-development projects represented by the case studies, but is also applicable to any research pursuing social impact. However, retrospective Theory of Change development proved challenging and resulted in simplistic Theories of Change. More work is needed to draw on theories of knowledge translation and policy processes to develop and further test more sophisticated and complex-aware Theories of Change.
Read the full peer-reviewed publication: Belcher, B., Suryadarma, D., and Halimanjaya, A. (2017). Evaluating policy-relevant research: Lessons from a series of theory-based outcomes assessments. Palgrave Communications, 3: 1-16. https://doi.org/10.1057/palcomms.2017.17
Phase 2. Assessing the Effectiveness of Research Projects with Theory of Change and the Transdisciplinary Quality Assessment Framework
Cases analyzed:
- Research-for-development context (CGIAR research): Sustainability of Timber Harvesting in Brazil Nut Concessions, Political Economy of Fire and Haze, Global Comparative Study on Land Tenure, Implementation of Agroforestry Concessions, Sustainable Wetlands Adaptation and Mitigation Program
- Higher education context (RRU DSocSci research): Truth-telling Project, Private Development Aid Project, Sanitation in the Niger Delta Project
Results: Projects with more transdisciplinary research design and implementation make more diverse contributions and can leverage more diverse mechanisms of change. By leveraging diverse contributions and mechanisms, projects have a greater potential for influence across more impact pathways.
Read the full peer-reviewed publication: Belcher, B. M., Claus, R., Davel, R., & Ramirez, L. F. (2019). Linking transdisciplinary research characteristics and quality to effectiveness: A comparative analysis of five research-for-development projects. Environmental Science & Policy, 101: 192-203. https://doi.org/10.1016/j.envsci.2019.08.013
Read the full peer-reviewed publication: Belcher, B. M., Claus, R., Davel, R., & Jones, S. M. (2021). Evaluating and improving the contributions of university research to social innovation. Social Enterprise Journal, 18(11): 51-120. https://doi.org/10.1108/SEJ-10-2020-0099