This website requires JavaScript.

Safe Model-Free Reinforcement Learning using Disturbance-Observer-Based Control Barrier Functions

Yikun ChengPan ZhaoNaira Hovakimyan
Nov 2022
摘要
Safe reinforcement learning (RL) with assured satisfaction of hard stateconstraints during training has recently received a lot of attention. Safetyfilters, e.g., based on control barrier functions (CBFs), provide a promisingway for safe RL via modifying the unsafe actions of an RL agent on the fly.Existing safety filter-based approaches typically involve learning of uncertaindynamics and quantifying the learned model error, which leads to conservativefilters before a large amount of data is collected to learn a good model,thereby preventing efficient exploration. This paper presents a method for safeand efficient model-free RL using disturbance observers (DOBs) and controlbarrier functions (CBFs). Unlike most existing safe RL methods that deal withhard state constraints, our method does not involve model learning, andleverages DOBs to accurately estimate the pointwise value of the uncertainty,which is then incorporated into a robust CBF condition to generate safeactions. The DOB-based CBF can be used as a safety filter with any model-freeRL algorithms by minimally modifying the actions of an RL agent whenevernecessary to ensure safety throughout the learning process. Simulation resultson a unicycle and a 2D quadrotor demonstrate that the proposed methodoutperforms a state-of-the-art safe RL algorithm using CBFs and Gaussianprocesses-based model learning, in terms of safety violation rate, and sampleand computational efficiency.
展开全部
图表提取

暂无人提供速读十问回答

论文十问由沈向洋博士提出,鼓励大家带着这十个问题去阅读论文,用有用的信息构建认知模型。写出自己的十问回答,还有机会在当前页面展示哦。

Q1论文试图解决什么问题?
Q2这是否是一个新的问题?
Q3这篇文章要验证一个什么科学假设?
0
被引用
笔记
问答