We’re excited to announce that Agata Ciabattoni, Emery Neufeld, and Radu Florin Tulcan have won the Distinguished Paper Award at the International Joint Conference on Artificial Intelligence (IJCAI) 2025 for their paper “Combining MORL with Restraining Bolts to Learn Normative Behaviour”!

Reinforcement Learning (RL) is a powerful Machine Learning technique for training autonomous systems, to achieve their goals through rewards and penalties. As these systems become more integrated into daily life, it is not enough for them to simply behave safely; they must also respect the legal, ethical, and social norms that guide human society. The authors of the paper present a framework combining Logic with RL to help autonomous agents follow such norms. First, the Restraining Bolt method—originally developed to enforce safety— is turned into Normative Restraining Bolts (NRBs). Unlike earlier approaches that reward agents for obeying rules, NRBs discourage violations by attaching penalties when rules are broken. While effective, NRBs have limitations: they require manual fine-tuning, struggle with hierarchies of norms, and need retraining when norms change. To address this, the authors reformulated the problem as multi-objective reinforcement learning (MORL), where each norm is treated as a separate objective. This leads to the proposal of Ordered Normative Restraining Bolts (ONRBs), which make it possible to prioritize among norms, adjust or change them without retraining, and algorithmically determine penalties instead of relying on trial and error. Through case studies, the paper shows that ONRBs provide a robust and principled way for RL agents to respect a wide range of norms while still achieving their goals.

IJCAI, held this year in August in Montréal, is one of the longest-standing and most respected conferences in the field of AI. Since its founding in 1969, IJCAI has provided a platform for presenting significant research contributions across all areas of AI, encouraging the exchange of ideas and collaboration within the global research community. The conference attracts a large number of submissions each year — in 2025, over 5,800 valid submissions were made for the main track of the conference. The paper Combining MORL with Restraining Bolts to Learn Normative Behaviour was selected as part of the top 0.05%, underscoring its significance and impact within the field.

Congratulations on this outstanding achievement!

Abstract

Normative Restraining Bolts (NRBs) adapt the restraining bolt technique (originally developed for safe reinforcement learning) to ensure compliance with social, legal, and ethical norms. While effective, NRBs rely on trial-and-error weight tuning, which hinders their ability to enforce hierarchical norms; moreover, norm updates require retraining. In this paper, we reformulate learning with NRBs as a multi-objective reinforcement learning (MORL) problem, where each norm is treated as a distinct objective. This enables the introduction of Ordered Normative Restraining Bolts (ONRBs), which support algorithmic weight selection, prioritized norms, norm updates, and provide formal guarantees on minimizing norm violations. Case studies show that ONRBs offer a robust and principled foundation for RL-agents to comply with a wide range of norms while achieving their goals.

About the authors

Agata Ciabattoni is Professor and Head of the Research Unit Theory and Logic at TU Wien Informatics. She serves as a board member of the cluster of excellence Bilateral AI, and as co-chair of the Vienna Center for Logic and Algorithms (VCLA). She is the principal investigator of the WWTF-funded projects Training and Guiding AI Agents with Ethical Rules (TAIGER) and Acquiring and explaining norms for AI systems (AXAIS), as well as the project Logical methods for Deontic Explanations (LoDEx), funded by the Austrian Science Fund FWF. She is also a recipient of the FWF START Prize 2011, the highest Austrian award for early-career researchers.

Emery Neufeld is a postdoctoral researcher at the Research Unit Theory and Logic at TU Wien Informatics. After studying mathematics and philosophy and spending some time in industry as a developer and data scientist, he began his doctoral studies at the Doctoral College for Resilient Embedded Systems at TU Wien under the supervision of Ezio Bartocci, Professor at the Research Unit Cyber-Physical Systems, and Agata Ciabattoni. Since defending his dissertation in 2023, he has been working on the WWTF-funded project Training and Guiding AI Agents with Ethical Rules (TAIGER).

Radu-Florin Tulcan is a student researcher and teaching assistant at the Research Unit Theory and Logic at TU Wien Informatics. Supervised by Agata Ciabattoni, he graduated with distinction from the bachelor’s program in Software and Information Engineering, and completed the Bachelor’s with Honors program at TU Wien Informatics. He is currently pursuing a master’s degree in Logic and Computation. Alongside his academic path, Radu has gained industry experience as a software developer and IT consultant and has actively contributed to the academic community through volunteering at conferences such as IJCAI-ECAI 2022 and the Conference on Digital Humanism 2025.

Curious about our other news? Subscribe to our news feed, calendar, or newsletter, or follow us on social media.