Toward explainable, robust and fair AI in automated and autonomous vehicles : challenges and opportunities for safety and security

In March 2022 the JRC (Units B.6, C.4, E.3) organized an Exploratory Workshop entitled "Toward explainable, robust, and fair AI in automated and autonomous vehicles", bringing together experts in fields such as Trustworthy AI, autonomous driving, and vehicle testing. This report summarizes the steps that followed the organization of the workshop, including the definition of the scientific objectives, the list of invited presenters and participants, and the conditions under which the workshop took place. The report also presents the main findings of each talk that occurred during the workshop and an analysis of the discussions that occurred during collaborative working sessions. Topics of interest included, among others, current regulations and standards regarding automated and autonomous road vehicles and analysis of their limitations; explainability of artificial intelligence ; accuracy, robustness, security, and fairness of AI systems. These insights are used to provide concluding remarks on the outlook of the Workshop, in particular how the findings of the Workshop can help to promote further research within and outside of the JRC on this topic, with the goal of making safer transport through innovative ecosystems and effective regulations. We identified gaps in the scientific literature on the relationship between AI and safety of Automated and Autonomous Vehicles (A&AVs) such as: establishment of reasoning vocabulary for acceptable factual and/or counterfactual interpretations, certification readiness matrix must be developed for each cyber scenario for different adversarial attacks and for naturally occurring perturbations, behavioural models are missing for motion prediction of different social agents and tests with standardized dummies lack the features of different social groups, currently there are not enough data to assess the fairness of A&AV vehicles and how fairness or bias influences safety. In our next report, we will focus on the above points by involving experts of the fields.