On 29 November 2022, Josefa Kny and Martina Schäfer (ZTG/TU Berlin) organised a digital reflection workshop on thematic line 1 "Societal Effects". The preliminary findings from the last two years of research were presented and discussed together with numerous participants who themselves conduct transdisciplinary research and are interested in the topic of strengthening and tracking societal impacts. The workshop was further enriched by comments from Arnim Wiek and Flurina Schneider.
The format:
Since the dialogue forum that took place in March 2022, in which the first results were presented and discussed with participants of the impact workshops, a lot has happened in topic line 1 "Societal Effects". The set of methods developed and improved over the past two years for reflecting on and strengthening the social effects of transdisciplinary research projects was tested in further impact workshops with several transdisciplinary research projects at different points in their courses, and existing findings were strengthened as well as new ones gained. More than 50 participants took advantage of the opportunity to learn about the current research results, to discuss them and to enter into dialogue with each other. While the focus of the workshop was primarily on the joint validation of previous findings and the exchange of experiences, there was also room for questions, comments and experiences between the inputs. In breakout sessions, the participants were able to exchange ideas in more detail. We thank all participants for their valuable input!
The preliminary results:
Martina Schäfer and Josefa Kny, responsible for topic line 1 "Societal effects", presented an update of their findings, which were additionally commented by two experts. The workshop was designed along three thematic sections. The first part dealt with the systematic classification of the variety of effect orders and forms. Martina Schäfer presented the effects heuristics* developed in the predecessor project TransImpact for analysing the social effects of transdisciplinary projects, which were used for systematisation. The preliminary conclusion was that the effects classifications proved to be suitable for conducting the workshops in order to classify effects according to their occurrence and controllability by the project. In addition, the effect categories were helpful for asking specific questions about certain potential forms of effects during the workshop or for categorising the listed effects afterwards. The results collected in the workshops with seven projects did not reveal a need for further effect categories. It has been shown that the forms of effects that were allocated to the first impact category in the heuristics (e.g. network formation and learning processes) remain relevant even after the end of the project. In the subsequent discussion, it was discussed to what extent the effects heuristics according to Schäfer et al. 2021 is considered as useful for categorising the effects occurring in transdisciplinary projects.
Fig. Categories for the analysis of social effects of transdisciplinary projects
Before the plenary discussion, the input was commented by Prof. Dr. Arnim Wiek (School of Sustainability, Arizona State University, Visiting Professorships at Albert Ludwigs University Freiburg & Leuphana University Lüneburg), who provided valuable input from his own work in the field of impact research. In his comment, he focused on the relevance of concrete attribution of research effects to specific research characteristics. The conclusion of his comment was that the heuristics is useful for strengthening an impact orientation from planning through the project process to the conclusion of transdisciplinary projects. In addition, he suggested that research features, such as the participation of relevant actors, should be included as a key element in the heuristics. Hereby the heuristics could help to counteract the previously observed mismatch between goals, research features and the intended effects. He also emphasised that more resources for consulting, project management, etc. are needed to enable impact analyses in transdisciplinary research projects. Furthermore, it is very important to agree on a common evaluation framework in the community of transdisciplinary researchers in order to be able to examine larger data sets and to enable generalisable approaches to impact.
These and other interesting points were then discussed in breakout sessions along the following questions: What advantages and disadvantages do you see with regard to the effects heuristics? Which forms of effects and, if applicable, other categories do you distinguish in your project or your work with effects?
In the second part, Josefa Kny presented preliminary results with regard to the identified impact pathways on the basis of the results from three projects that were accompanied in topic line 1: What connections can be traced between project activities, results and effects? In her input, she addressed the similarities and differences of the impact pathways that emerged from the work with the projects. In all three cases, rather "soft" effects such as learning processes and network building were seen as preconditions (mediators) for the development of more tangible effects in the 2nd and 3rd order. At the same time, it became clear that the controllability of effects tends to decrease with time and distance from the project context. In general, the visibility and dissemination of the projects' processes and results emerged as a core desired effect. All three projects also rely heavily on the initiation and development of networks that become permanent, i.e. that develop independently, become active and motivate other actors to participate. In all cases, however, the limits of this focus on inherent dynamism and the relevance of the project network to achieve impact were already evident, and, hence, require a stronger focus on collaboration and possible synergies. In all three cases, pilots or best practices that convincingly illustrate or make it possible to experience something as well as multipliers who pass on knowledge and practices from within the closer network were also used to ensure the continuity and dissemination of the project effects. While the former must be established at first, the latter must also accept their role. The questions of whether sufficient resources and competences are available for transfer and network activities, how these are distributed between science and practice and what is necessary for continuity therefore appear to be particularly relevant for achieving the desired effects.
Prof. Dr. Flurina Schneider (Scientific Director ISOE, Professor of Social Ecology and Transdisciplinarity at Goethe University Frankfurt) focused in her comment on the question of the relationship between transdisciplinary research processes and sustainability transformation. She shared her experience of working with impact pathways, in which she has gathered similar insights to those of the tdAcademy team. Schneider pleaded for a deeper analysis of what happens in collaboration, what potential for action arises in transdisciplinary projects and how this can be activated. Furthermore, like Arnim Wiek, she addressed the need for larger numbers of cases in order to gain valid insights.
In the last part of the workshop, the focus moved to the impact workshops as a tool. Based on the evaluation of the feedback questionnaires and feedback rounds with the workshop participants in the context of the impact workshops, it was discussed to what extent they proved to be a helpful tool for strategic impact orientation in transdisciplinary research projects and where challenges exist.
"In my opinion, a systematic approach like the one presented in the workshop is immensely valuable. Especially in transdisciplinary projects where diverse actors and topics play a role. I was able to take away a lot for my own research work." - feedback from a participant (translated from German)
The exchange:
The discussions focused primarily on the question of suitable approaches for tracing impact paths. It was also discussed how knowledge and learning processes can be initiated and how effects can be perpetuated.
The question of what resources are needed for systematic impact reflection and what advantages and disadvantages external monitoring has compared to a project-internal perspective for tracing effects was also discussed. Difficulties were raised in the area of funding policy, which has so far often not made it possible to apply for sufficient resources for impact reflection and evaluation.
It was also suggested that we reflect on our own understanding of our role as researchers within a transdisciplinary research approach: Are we in the classic role of researchers as observers? Or do researchers act more as change agents in certain phases? Subsequently, the questions of how to reflect on the different roles and what they mean for cooperation in transdisciplinary processes were also raised. How do you deal with the variously composed groups with diverse interests as well as different incentive mechanisms?
These and many other interesting questions were discussed during the reflection workshop. For us, the workshop provided many valuable suggestions and food for thought, and according to the feedback, the participants felt the same way:
"The workshop was very inspiring and showed me once again how important it is to take the time in research projects, especially in collaborative projects, to reflect together on the intended and unintended effects and to think about the question of resources and roles" - feedback from a participant (translated from German).
* Schäfer, Martina, Matthias Bergmann, and Lena Theiler. "Systematizing Societal Effects of Transdisciplinary Research." Research Evaluation 64 (2021): 215. doi:10.1093/reseval/rvab019.