The vision of SINFONICA is to develop functional, efficient, and innovative strategies, methods and tools to engage CCAM stakeholders (i.e. citizens, vulnerable user groups, transport operators, public administrations, service providers, researchers, vehicle and technology suppliers) following a bottom-up approach, as to collect their main needs, desires, and concerns related to CCAM.

The participatory approach evaluated in the SINFONICA project relied on a robust methodological framework that integrated diverse engagement strategies tailored to the identified groups of people with mobility challenges, including interviews, focus groups, and workshops. These participatory methods were systematically implemented and iteratively refined across three rounds of data collection, being facilitated by the establishment of 4 Groups of Interest duly identified in Trikala (Greece), Hamburg (Germany), Noord-Brabant province (The Netherlands), and West Midlands region (United Kingdom). The design, implementation, and impact of the activities undertaken in these participatory rounds, along with the testing and verification of the methodology adopted across a wide community of actors across Europe, form the core object of the SINFONICA evaluation.

Why evaluate the SINFONICA participatory process?

In projects like SINFONICA, which operate at the intersection of technology and community, the participatory process serves as the bridge between innovation and user acceptance. Actively involving end-users and stakeholders throughout the process not only builds trust but also provides critical insights into public concerns, enhancing the relevance and inclusivity of project outcomes. Evaluating how well these processes are designed, implemented, and received is crucial to ensuring that they truly serve the community and support sustainable mobility innovation leaving no one behind. By providing valuable insight, it also supports informed decision-making of policy makers, public authorities and transport agencies who rely on the evaluation outcomes to shape regulations and guidelines that ensure the safe, ethical, and efficient deployment of CCAM solutions. It also improves the accuracy and impact of future assessments pertaining approaches similar or inspired to the SINFONICA one.

The SINFONICA participatory process evaluation framework

The SINFONICA framework evaluates the participatory process across three key phases:   1. Design and Planning, 2. Implementation, 3. Results and Impacts. Each phase includes specific criteria and variables that help assess the strengths, weaknesses, and areas for improvement in engaging stakeholders.

Phase 1: Design and Planning:

The first phase focuses on how well the participatory process is designed and planned to meet project goals. The framework evaluates through the following variables whether the SINFONICA process aligns with both the project’ and the participants’ needs.

  • The objectives need to be relevant and well-defined, setting a clear “rule of the game” and establishing expectations for stakeholder engagement and feedback return.
  • The process must be consistent: goals, resources, and timelines are aligned across project components.
  • Feasibility and design quality. The necessary resources must be foreseen and planned to ensure inclusivity and transparency. The SINFONICA Team identified several different factors that can influence the quality of design, among which there are leadership, transparency, diversity of participants, accessibility, communication, training.

Phase 2: Implementation

This phase evaluates the actual rollout of the participatory process, focusing on the quality of stakeholder engagement, the inclusivity of the process, and flexibility to adapt to participant needs.

  • Process Quality. To assess potential synergies with other participatory initiatives, as well as the adaptation to participants’ needs and agendas, the level of stakeholder empowerment, and the quality of the dynamization considering gender, diversity, language and emotional variables.
  • Evaluate the efficiency of resource allocation within the participatory process, including time, budget, and personnel.
  • Measure the capacity of adaptation of the process.

Phase 3: Results and Impacts

The final phase measures the effectiveness, efficiency, and long-term impact of the participatory process. This includes assessing whether project objectives were met and gauging the broader effects on trust-building and collaborative culture.

  • Efficacy: This variable assesses the degree to which the objectives and “rules of the game” were achieved. It also checks onto what extent the results have been communicated and are at free disposal of the participants (Devolution).
  • Efficiency: This considers whether resources were used effectively, aiming for maximum impact with optimal resource usage.
  • Impacts: It evaluates the extent to which the process fosters trust, empowers participants, and nurtures a culture of participation.

Iterative data collection and feedback mechanism

The evaluation itself, as the participatory process, was organized into three iterative rounds of data collection, each serving as a feedback loop to enhance the subsequent round:

  • First Round: The initial data collection activities—interviews, focus groups, and workshops—were guided by a preliminary set of standardized guidelines developed by the SINFONICA team. The evaluation of this first round provided valuable feedback that informed the refinement of these guidelines, ensuring they were better tailored to the diverse contexts of the four research sites.
  • Second Round: Building on the insights from the first round, the second round of data collection aimed to validate and test the revised engagement strategies. The key component of this round was a focus group with the leads of data collection activities from the four research sites. This session facilitated a cross-site exchange of experiences, challenges, and best practices, enabling the team to further refine the participatory guidelines for the final round.
  • Third Round: The final round of data collection, incorporating the lessons learned from the first two rounds, represented the culmination of the participatory process. This round aimed to validate the effectiveness of the refined methodologies and assess the overall impact of the engagement activities.

Preliminary insights of the SINFONICA participatory process evaluation

The SINFONICA project has already gathered valuable insights from evaluating its participatory process. A report on the evaluation process will be ready and made public as project’s deliverable in the next months. Nevertheless, here we anticipate some early results of the evaluation on some crucial areas for creating a participatory process that genuinely serves the community and gives significant insights on the needs or desires of the citizens.

  • Need for a clear communication plan: Establishing a robust communication plan ensures that participants are fully informed and made aware.
  • Transparency: Being transparent about objectives, methods, and expectations builds trust with stakeholders.
  • Flexibility: There is the need for a flexible approach that incorporates participants’ feedback and enables continuous adjustments, enhancing the relevance of the participatory process as it unfolds.
  • Strengthening synergies: To maximize the impact, it is essential to integrate the participatory process with other similar initiatives. It allows for a broader exchange of insights and a more comprehensive understanding of stakeholder needs across different research sites.

Building a pathway to better participatory processes

SINFONICA’s evaluation framework fills a significant gap in participatory research for mobility projects, providing a structured approach that others can follow. By assessing the participatory process across three detailed phases, SINFONICA not only measures effectiveness but also sets a foundation for continuous improvement. By focusing on inclusive design, dynamic implementation, and impactful results, SINFONICA sets a benchmark for participatory processes in European mobility projects, paving the way for more inclusive, transparent, and community-centered approaches in the development of CCAM technologies.

Author: Giacomo Cantini, Letizia Fontanesi (ICOOR/UNIMORE)