Pilots Phase 2

As a research funding organization, VDI/VDE-IT works closely with the German Federal Ministry of Education and Research. The pilot of VDI/VDE-IT originated from this cooperation and was designed to accompany a funding call on technological solutions to support informal caregivers. The pilot set up a citizen advisory board (CAB), which participated in the proposal evaluation and selection of suitable projects. The members of the CAB were also involved as mentors during the implementation of the innovation projects. The CAB consisted of people who provide or have provided informal care to relatives, friends or acquaintances and are therefore regarded as lay experts in the field of informal care.

The main goal of pilot case 5 was to involve (public) representatives of the target group in several key steps of the funding lifecycle, from proposal evaluation to research and development. The pilot thus enabled informal caregivers to have a direct impact on the project selection and execution processes as well as the final product that is designed to support informal caregiving. In addition, their involvement would give CAB members an insight into the process of funding and developing technologies.

RCN is a Norwegian government agency funding both basic and applied research and innovation. The RCN pilot was carried out in close cooperation with RCL, which fulfils a similar role funding R&D in Lithuania. The Involve Hub developed in the context of pilot case 6 built on former and ongoing experiences with participatory activities at RCN, including some carried out in collaboration with RCL. While these were carried out without a special focus on research ethics or research integrity, the Involve Hub entailed a network of key stakeholders to share knowledge, discuss challenges and identify solutions related to ethical participation in R&I. The Hub was organized in three gatherings during the period August 2022 to January 2023, as well as two preliminary integrating workshops prior to the Hub-gatherings.

The Involve Hub was developed as a community of practice hub for mutual exchange on various aspects and ethical dilemmas of citizens participation in scientific knowledge production. Thematically, these activities 1) addressed potential ethical challenges of citizen involvement in research, 2) invited inputs for developing the Ethics Framework, and 3) collected feedback on how the Research Council can work better with regards to ethical citizen involvement.

FFG is the Austrian national funding agency for applied and industrial research and innovation and manages funding mostly provided by different ministries and other public institutions. For several years, the FFG R&I funding programs ‘benefit’ and ‘AAL’ have focused on combining pressing issues of demographic change with the development of useful ICT-based solutions. The programs are now evolving into a new funding topic called “Digital Solutions for People and Society”. Under the umbrella of this new theme, it was the aim of the FFG pilot to explore relevant subtopics that combine health topics, climate change and demographic change, while also taking into consideration the possibilities that information and communication technologies offer. As part of this pilot, FFG involved citizens in the definition of call topics through a four-step process: First, FFG team members conducted expert interviews to inform the approach FFG was taking. This was followed by a validation workshop with a wider group of stakeholders to deepen and validate the information received in the expert interviews. In the third phase, an online survey was shared through various channels with potentially interested citizens to collect their input and experiences with regard to the pilot topic. Finally, another validation workshop was planned to validate specific aspects of the survey.

The overall goal of the pilot was to elaborate topics for a funding call based on the input of experts and, most importantly, citizens in the broadest sense. The objective was to develop the topics of a call for research, development and innovation projects that are addressing what matters most to citizens. Since FFG does not usually perform broader citizen engagement processes when defining call topics, this was a new experience for the team. The separate activities had individual goals. The expert interviews in the beginning were used to get a deeper understanding of the topic. The first validation workshop had the goal to deepen and validate information from the expert interviews and to find out more about the interrelation between climate change, health, and technology so that the lives of citizens could be improved effectively. The expert interviews and first validation workshop built the base to design the online questionnaire that was subsequently made available online for self-selected citizens participation. This questionnaire was the key activity, collecting comments, preferences, and ideas from citizens. A first call for proposals has been launched based on these three steps. FFG launched a second call based on the results of the pilot at the end of 2023.

Innoviris is a public organization that funds and supports research and innovation in the Brussels Capital Region (BCR) and (co-)finances R&I projects in companies, research centers, and the non-profit and public sector. Pilot case 8 aimed to find a suitable design and method for engaging a broad cut of the population living in the Brussel Capital Region and identify challenges in the BCR that should be addressed through funding by Innoviris, via its policy-oriented research program Prospective Research. As a first step, the team at Innoviris conducted desk research to identify concrete challenges from the Regional Research and Innovation Strategies for Smart Specialisations (RIS3), which is distilled from political plans, megatrends, data analyses, and so on. The base layer for call topics at Innoviris is always the list of challenges identified through the participative RIS3 strategy. This was followed by a consultation with regional public actors and centers of expertise with knowledge on regional challenges, implemented through a semi-directive online questionnaire collecting inputs on the challenges identified through the desk research. Based on this, Innoviris developed and conducted an open online survey with residents of the BCR. Finally, two citizen ateliers were organized in person, where residents of the Brussels area discussed different urban challenges with experts and among themselves.

The pilot by Innoviris had two main goals: 1) To conduct a single and more comprehensive consultation to identify important urban challenges for future regional policies that should be addressed by researchers with direct feedback from individual citizens; and 2) To use the pilot to develop a set of guidelines for a more systematic and global approach for citizen participation in processes of agenda setting and scientific knowledge production. In addition, each activity also had individual goals. With the consultation of regional actors, Innoviris wanted to check, refine and focus current challenges as they were identified from documents such as RIS3. The online survey that presented citizens with five challenges identified in the previous activities provided a chance to add new and emerging challenges. For the citizen panel, which met in person twice, the goal was to allow a diverse audience to share opinions and knowledge, thus initiating a public dialogue between citizens, representatives of different RRI programs and experts.

UEFISCDI is a national Romanian research funding agency that has contributed to the development of the National Research, Innovation and Smart Specialisation Strategy 2022-2027. In the context of the pilot activities, UEFISCDI decided to employ a participatory approach to refine the 2022-2027 Romanian National Agenda for Research by engaging citizens in validating and enriching the proposed thematic priorities. The Strategic Research Agenda is a list of societal challenges and leading questions elaborated through a lengthy process with experts and public bodies. The two interactive workshops implemented by UEFISCDI were thus aimed at aligning the outputs experts and public authorities with the needs and experiences of citizens.

The goal of pilot case 9 was to test if such an ambitious endeavor, namely involving citizens in the development of national, strategic documents and instruments, was possible, especially considering that these kinds of initiatives are usually the prerogatives of authorities and experts. Another aim was to explore if citizens could engage and understand research topics well enough to meaningfully contribute to the process of refining, enriching or just validating the research questions, and thus whether this approach was a viable strategy to potentially repeat in the future. The citizen working groups were asked to add, rephrase or confirm what experts had elaborated based on their own views, beliefs, interests and needs.

As this was an initial experiment, the project team could not guarantee that participant inputs would be included in the final official document, as the findings of the workshop were presented to the official authorities merely as a suggestion. Nevertheless, some of inputs were indeed integrated in the final agenda.

CDTI is a Spanish public organization for technology development and innovation answering to the Ministry of Science and Innovation. CDTI manages the Neotec funding program, which provides grants for New Technology-Based Firms (NTBs) to carry out projects that require the use of technologies or knowledge developed from research activities as well as a viable business plan. The objective of Neotec is to foster the transfer of knowledge from research organisations or expert teams to industry, seeing NTBs as a viable instrument to take scientific and technological progress generated in laboratories into the market, and in turn into society. Due to its multiple, direct and clear social implications, the Neotec program has been selected as a pilot to test new open evaluation approaches based on the participation of program beneficiaries. To this end, CDTI implemented 5 focus groups and a total of 30 interviews with various stakeholders, followed by an in-depth analysis of the gathered material.

The main goal of the participatory activity was to develop an evaluation methodology including evaluation criteria for measuring the social impact of RDI projects funded by Neotec for the first time. Further on, CDTI aims to apply the outcome of the participatory exercise in evaluating other internal funding programs, as well as disseminating it within its sphere of influence – mainly, regional innovation agencies and other public bodies in charge of RDI policy in Spain and the European evaluation community.

The Open Innovation in Science Center (OISC) of the Ludwig Boltzmann Gesellschaft (LBG) is a research support office (RSO) and research training institution aimed at fostering participatory health research. Since 2022, it has been running the program “Inclusion Health”, which focuses specifically on developing methods, strategies and evaluation criteria for involving marginalized and vulnerable populations in research. Due to LBG’s focus on health research, this mainly meant people with lived experience of homelessness, since this population is both most severely affected by health inequities, and among the most difficult to reach for research. The program was rolled out in collaboration with the University of Edinburgh Centre for Homelessness and Inclusion Health.

This additional pilot aimed to develop staff capacities at LBG OISC, allowing them to 1) gain an understanding of ethical and methodological issues in involving disadvantaged groups, derived from research, practice and experience, and feedback learnings into the PRO-Ethics network; 2) acquire critical skills and knowledge in participatory research with disadvantaged groups; 3) gain novel expertise in evaluating funding proposals and contributed to PRO-Ethics materials and guidelines development; 4) establish an ongoing training module on participatory research with disadvantaged groups; and 5) make sustainable connections for national and international cooperation and knowledge exchange.

TACR was an initial partner in the PRO-Ethics project. The RFO is a Czech government agency founded in 2009 with the aim of encouraging cooperation between research organizations supported by the state and the business sector. TACR manages funds provided by the government, selected ministries (or ministry-like institutions) and international funds such as the Resilience and Recovery Fund (RRF). TACR had planned to implement a pilot in the context of the SSH funding program ETA. However, by autumn 2021 it became clear that TACR would not have sufficient funds for the planned calls in the ETA program, leaving the organization without a suitable program for carrying out the pilot. As a consequence, TACR had to terminate its further participation in the PRO-Ethics project.

Reflecting on the main challenges that lead to this unfortunate situation, TACR identified as a main barrier to the implementation of its pilot decisions taken at the political level of R&D, especially with regards to flow of financial resources and call approval. This was further exacerbated by time restraints, as political agreement by key stakeholders could not feasibly be achieved within the framework given by the PRO-Ethics project and the consensus at the time prioritized other sectoral priorities over social sciences and humanities. Furthermore, the existing legal and institutional framework presented a problem when an alternative pilot was sought, as without changes it was impossible to implement alternative participatory approaches as seen in other PRO-Ethics pilots. This, too, was exacerbated by time pressures, as the adaptation of legal frameworks required to implement the desired changes were impossible to achieve on short notice.

While not considering this a main hurdle for the pilot but more of an observation interesting for the project, there were reservations about the potential implementation of participatory activities in the context of TACR among some of its employees. The participation of experts was seen as quite welcome, while the participation of citizens was partly seen as something which would hinder processes at TACR. This would not present a problem for the implementation of the intended TACR pilot, as it was focused on the involvement of an academic group that were considered experts within the framework of the organization.

Finally, it is important to note that in the case of TACR, the organizational leeway allowed by the organization’s owner is quite low. TACR puts up programs for approval and can officially design them how it sees fit, and negotiations between TACR representatives and policymakers can try to steer things in the organization’s favor depending on the state of the (R&D) political landscape. Nevertheless, with the influence of the sectoral priorities and political negotiations, choices on topic prioritization have to be taken depending on the approved state budget allocated for R&D.

Key recommendations from pilot phase II:

- Before planning any participatory activity, carefully consider the resources and institutional support you have at your disposal, as well as the established structures of your organization.

- Involve key stakeholders such as politicians and other decision-makers from the very beginning and maybe even seek contractual agreements based on detailed plans with all parties.

- Be realistic and pragmatic regarding the scope of your participatory process – regarding both the organizational and the participant level. Consider what is feasible in terms of recruitment and engagement methodology, breadth of the target group, and process goals.

- Employing a professional recruitment company can be helpful in cases internal time and expertise is insufficient to implement this task. Providing compensation and other means of support (e.g. childcare) can help secure a wider representation. If you cannot reach your target group through your initial approach, be flexible and try other approaches.

- Identify and address potentially time-consuming administrative processes that may act as barriers to participation, such as payment processing procedures for compensation.

- Be aware of potential conditionalities that may impact participants’ willingness to continue their involvement – as in the childcare example named above. In the case of questionnaires, this might entail the option of skipping questions or not having one question be a precondition for accessing another one.

- Consider both the timing and type of participation format to make the activity more accessible to potential participants (e.g. physical vs. online events; events during holiday seasons; etc.).

- Be aware of potential differences in knowledge, interest, resources, or capacity within your group of participants that might lead to tensions or power differences. It is essential to have a good facilitator that can ensure balance in the conversation, address bias, anticipate potential conflicts of interest, and ensure that the activity remains focused on the goals of the process. Make sure to create a friendly atmosphere.

- Create clarity on and communicate transparently the roles and responsibilities of involved stakeholders. Ensure everyone has sufficient understanding of the process and their potential to contribute. This helps create trust in the process.

- Plan enough time for each participatory activity to allow participants to think and provide feedback. However, also consider that too long an activity might be strenuous and hinder certain groups from participating.

- Continuously adjust expectations and ambitions and manage resources proactively, anticipating bottlenecks, during all stages of the process.

- An online tool can be helpful in collecting additional feedback. However, consider selecting a user-friendly tool that is accessible to everyone without creating additional efforts.

- Try to make an ex-post reflection after each activity to share opinions and agree on core outputs. If this is not possible, try drafting a reporting template that includes input also by the moderators, to complete right after the activity. Reporting can be as important as the activity itself.

- Provide clear guidelines on quality criteria for participatory processes and make sure that staff members responsible for the evaluation have enough experience with impact measurement.