Quantized measurements in Q-learning based model-free optimal control

Abstract

Quantization noise is present in many real-time applications due to the resolution of analog-to-digital conversions. This can lead to error in policies that are learned by model-free Q-learning. A method for quantization error reduction for Q-learning algorithms is developed using the sample time and an exploration noise that is added to the control input. The method is illustrated with discrete-time policy and value iteration algorithms using both a simulated environment and a real-time physical system.