Characterizing Transdisciplinary Research Quality for Effective Design, Implementation, and Assessment
Transdisciplinary approaches transcend disciplinary and institutional boundaries. These new approaches aim to contextualize research around the interests of stakeholders to foster more socially robust knowledge. Effective evaluation of quality in research is critical for funding, management, ongoing development, knowledge mobilization, and the advancement of research methods, projects, and programs. Funders demand excellence from the organizations they support, and likewise, research managers guide their programs to achieve excellence. Scientists refer to their organizations’ definitions and measures of research excellence in their own planning and strategies. Nevertheless, how to define quality in research is an ongoing challenge, particularly in a transdisciplinary context because current approaches to research evaluation are largely discipline-specific. Research, by convention, is evaluated on the basis of publications, citations, and peer assessment. In the absence of a comprehensive guide, many evaluating bodies – including funding bodies – rely on conventional discipline-specific measures of quality which may disadvantage transdisciplinary and applied research. As such, conventional assessments of research effectiveness may not adequately capture important research outcomes and contributions to change. We therefore need to understand the components of transdisciplinary research quality for more robust and representative assessments.
Guiding question: What are appropriate criteria and indicators for defining and measuring transdisciplinary research?
Purpose: To define a framework for assessing and planning effective transdisciplinary research to contribute to the better use of science in solving social problems, particularly in natural resource management contexts.
Method: Systematic review of current knowledge of and approaches to the evaluation of research quality in an applied inter- and transdisciplinary context. Click to download the protocol.
Result: Four main principles for the evaluation of quality emerged: relevance, credibility, legitimacy, and effectiveness, which were further broken down into 22 quality criteria. This comprehensive transdisciplinary research quality assessment framework is specific enough for practical use, and flexible enough to be applied across research projects and contexts with varying levels of complexity.
Outcome: The transdisciplinary research quality assessment framework has been applied to supplement the final stage of the approval process for Royal Roads University to acquire an Ashoka U Changemaker Campus Designation. It has provided insight and guidance into the assessment of the social impact of Royal Roads University’s research programs.
We are continuing to apply lessons from other projects to refine the transdisciplinary research quality assessment framework for theoretical and empirical strengthening.
Read the full peer-reviewed publication: Belcher, B. M., Rasmussen, K. E., Kemshaw, M. R., & Zornes, D. A. (2016). Defining and assessing research quality in a transdisciplinary context. Research Evaluation, 25(1), 1-17. This paper provided the theoretical basis for an info brief published by the Independent Science and Partnership Council of the CGIAR.
Read CIFOR’s blog post