Characterizing Transdisciplinary Research Quality for Effective Design, Implementation, and Assessment
Transdisciplinary approaches transcend disciplinary and institutional boundaries. These new approaches aim to contextualize research around the interests of stakeholders to foster socially robust knowledge. Effective evaluation of research quality is critical for funding, management, ongoing development, knowledge mobilization, and the advancement of research methods.
Funders demand excellence from the organizations they support, and likewise, research managers guide their programs to achieve excellence. Scientists refer to their organizations’ definitions and measures of research excellence in their own planning and strategies. Nevertheless, how to define quality in research is an ongoing challenge, particularly in a transdisciplinary context because current standards and approaches to research evaluation are largely discipline-specific.
Research, by convention, is evaluated on the basis of publications, citations, and peer assessment. In the absence of a comprehensive guide, many evaluating bodies – including funders – rely on traditional quality standards. However, these standards tend to disadvantage transdisciplinary and applied research. Furthermore, traditional standards do not adequately capture important research outcomes and contributions to change. Therefore, we need to understand the components of transdisciplinary research quality for more robust and representative assessments and establish appropriate standards.
Guiding question: What are appropriate criteria and indicators for defining and measuring transdisciplinary research?
Purpose: To define a framework for assessing and planning effective transdisciplinary research to contribute to the better use of science in solving social problems, particularly in natural resource management contexts.
Method: Systematic review of current knowledge of and approaches to the evaluation of research quality in an applied inter- and transdisciplinary context. Click to download the protocol.
Result: Four main principles for the evaluation of quality emerged: relevance, credibility, legitimacy, and effectiveness, which were further broken down into 22 quality criteria. This comprehensive transdisciplinary research quality assessment framework is specific enough for practical use, and flexible enough to be applied across research projects and contexts with varying levels of complexity.
Outcome: The Transdisciplinary Research Quality Assessment Framework has been applied to supplement the final stage of the approval process for Royal Roads University to acquire an Ashoka U Changemaker Campus Designation. It has provided insight and guidance into the assessment of the social impact of Royal Roads University’s research programs.
We continue to apply lessons from other projects to refine the transdisciplinary research quality assessment framework for theoretical and empirical strengthening.
Read the full peer-reviewed publication: Belcher, B. M., Rasmussen, K. E., Kemshaw, M. R., & Zornes, D. A. (2016). Defining and assessing research quality in a transdisciplinary context. Research Evaluation, 25(1): 1-17.
The above paper provided the theoretical basis for an info brief published by the Independent Science and Partnership Council of the CGIAR. Read CIFOR’s blog post to learn more.
Check out the interactive resource below to learn more about the latest revisions made to the Quality Assessment Framework.