This summary presents key insights, backgrounds, and recommendations from an expert workshop on validating bicycling simulator studies, addressing critical aspects of transparency, standardisation, and real-world transferability in Vulnerable Road User (VRU) research.
First interactive workshop on bicycling simulator validation
On April 15th 2026, researchers in the field of simulator-based Vulnerable Road User (VRU) mobility research with gathered for a 60-minute interactive workshop focused on bicycling simulator validation. The session, moderated by Mathis Titgemeyer and featuring a presentation by Prof. Dr. Anja Huemer and Dr. Thomas Stemmler was part of the 5th Human-Centered VRU Simulation Workshop at the Technical University in Munich. The interactive workshop aimed to identify best practices for transparent and efficient validation of bicycling simulator studies. Participants were divided into eight groups, each tasked with prioritising critical validation aspects, brainstorming open resources, and proposing collaborative actions. This summary captures the key findings and recommendations from the workshop.
Backgrounds of Validating Bicycling Simulator Studies
Bicycling simulators are valuable tools in VRU research, enabling experiments that would be impractical or impossible to conduct in real-world settings. However, the transferability of findings from virtual to real-world contexts remains a central concern. The quality of simulator-based research hinges on its adherence to the core criteria of quantitative research: objectivity, reliability, and validity (Zöller, 2015; Bortz & Döring, 2006; Döring, 2023; Lienert & Raatz, 1998). While objectivity is inherently supported by the controlled nature of simulators, and reliability benefits from their repeatable conditions, validity is the most critical and debated criterion. In simulator research, validity refers to the method’s ability to accurately represent real-world phenomena, encompassing both physical validity the correspondence of the simulation to real-world environments and behavioural validity the alignment of human behaviour in the simulator with real-world actions (Blaauw, 1982; Himmels et al., 2023).
The technology underpinning any simulator directly influences the validity of studies conducted with it. Himmels et al. (2023) propose a framework that links physical and behavioural validity to simulator technology, emphasising how factors such as the realism of sensory cues and the extent of control actions impact perceived realism and behavioural responses. However, the relationship between technical complexity and validity is not straightforward, as validity is shaped by multiple, often fragmented factors (Wynne et al., 2019). This complexity is particularly pronounced in bicycling simulator research, where comprehensive, synthesised knowledge remains limited. The interactive workshop aimed to address this problem by bringing together experts of the field for a joint discussion and brainstorming.
Structure of the Workshop
The workshop was structured into three interactive steps, each designed to foster collaboration and actionable outcomes:
- Identifying Priorities (10 min):
Using tabletop posters summarising the aspects of validation identified through the Borsos et al.’s (2026) review, the groups discussed and selected the three most critical aspects for transparent validation, ensuring clarity and consensus on what matters most in bicycling simulator research. - Designing Solutions (10 min):
Participants brainstormed open resources—such as tools, templates, or platforms—that could streamline validation processes for the entire community. - Committing to Action (10 min):
Each group shared their top idea for a collaborative resource, along with a concrete plan for implementation beyond the workshop.
Results
Prioritised Aspects of Validation
The groups identified a range of critical aspects for transparent validation, with recurring themes across multiple teams:
- Behavioural Fidelity:
Recorded and analysed bicyclist behaviour, such as steering, head movement, and cognitive load, was consistently prioritised. Groups emphasised the need for standardised measurements and data collection protocols to ensure reproducibility. - Subjective Fidelity:
Measuring subjective realism and immersion (e.g., audio, wind, and visual fidelity) was highlighted as essential for validating the bicyclist experience. Standardised questionnaires and assessment tools were suggested to capture these dimensions. - Physical Fidelity:
Forces, steering angles, and tire interactions were noted as key components of physical fidelity, requiring precise measurement and reporting. - Scenographic Conditions:
Infrastructure details, road quality, and scenario realism were frequently mentioned. Groups proposed standardised scenarios and benchmarking tools to ensure consistency across studies. - Basic Data and Demographics:
Clarity on the purpose of the study, research design, and sample characteristics (e.g., gender, dropouts) was deemed vital for contextualising validation results.
Open Resources for Efficient Validation
Participants proposed a variety of open resources to support transparent and efficient validation:
- Standardised Scenarios:
Groups suggested developing benchmark scenarios based on real-world data, such as OpenCRG, OpenDrive, or ASAM Open Scenario formats. These scenarios could serve as a common reference for validating bicycling simulator studies. - Reporting Templates:
A checklist for journal editors and minimum reporting requirements (e.g., sample size, technology used) were proposed to improve transparency and reproducibility. - Open Databases:
Collaborative databases for bicyclist point-of-view videos, GPS data, and physiological measurements such as heart rate were recommended to provide real-world benchmarks for simulator validation. - Simulator Technology:
Open interfaces, hardware abstraction layers, and community-maintained software were identified as critical for technology standards for bicycling simulators. - Measurement Tools:
Standardised scales, reusable measurement devices, and open data sets for key indicators such as speed, torque, pedal power were highlighted as essential for consistent validation.
Opportunities for collaborative action
The workshop concluded with groups proposing actionable steps for post-workshop collaboration:
- Standardised Scenarios and Reporting:
Three Groups advocated for community-agreed templates and databases to standardise scenographic conditions and reporting practices. - Open Databases for Real-World Data:
Three Groups emphasised the need for open databases of cyclist behaviour, POV videos, and physiological data to enhance ecological validity and benchmarking. - Foundation Models and Open Firmware:
Two Groups proposed developing open firmware and foundation models for bicycling dynamics, ensuring realistic and reproducible simulations. - Collaborative Platforms:
All Groups suggested establishing community-maintained platforms (e.g., GitHub repositories) for sharing scenarios, measurement protocols, and analysis procedures.
Discussion & Conclusion
While the workshop provided valuable qualitative insights into the validation of bicycling simulator studies, limitations of its impact on international simulator research should be acknowledged:
- Representativeness:
The workshop included only a subset of the international research community, with participants primarily from Europe. This geographic limitation excludes perspectives of researchers from other regions. Future collaborative formats should aim for broader participation to capture a more global view. - Qualitative Nature:
The workshop focused on qualitative discussions and brainstorming, yielding actionable ideas but no statistical evidence. While qualitative approaches are valuable for generating hypotheses and identifying priorities, empirical studies might be required to filter for the most relevant and effective best practices and resources to be implemented.
Despite these limitations, the workshop succeeded in fostering collaboration and identifying key areas for improvement in bicycling simulator validation. The workshop underscored the importance of collaboration, standardisation, and transparency in validating bicycling simulator studies. By prioritising behavioural, subjective, and physical fidelity and leveraging open resources researchers can enhance the reliability and comparability of their findings. The proposed actions, from standardised scenarios to open databases, offer a roadmap for the research community to advance bicycling simulator validation collectively.
For further details and empirical results, please refer to the forthcoming publication on “What makes a bicycle simulator valid for experimental research? by Borsos et al. (2026).
Join the Community
Stay updated on the latest advancements in bicycling simulator validation and VRU research—subscribe to our newsletter for expert insights, collaborative opportunities, and upcoming events.
References
Borsos, A., Hammami, A., Pappalardo, G., Singler-Hack, O., Kuipers, R., Huemer, A. K., Stemmler, T., Käthner, D., D’Agostino, C., Llopis Castelló, D., Pérez-Zuriaga, A. M., Kiec, M., de Waard, D., Sporrel, B., & Happee, R. (2026). Bicycle simulator database [Data set]. Zenodo. https://doi.org/10.5281/zenodo.18734855
Huemer, A. K., & Stemmler, T. (2026, April 15). What makes a bicycle simulator valid for experimental research? – A state-of-the art review [Conference Presentation]. 5th Human-Centered VRU Simulation Workshop.