Self-ONNs in deep reinforcement learning

Abstract

Self-organized Operational Neural Networks (Self-ONNs) have been recently introduced as an alternative to Convolutional Neural Networks (CNNs), as a way to introduce non-linearity to the network. Self-ONNs use generative neurons which use some higher degree functions to estimate the data distributions or patterns in the input data. Comparably CNNs can only estimate these patterns with linear or first-degree functions which hinders the networks’ ability to learn more complex patterns. Self-ONNs have been used in multiple different use cases and they have been shown to perform better than equivalent CNN networks in tasks, such as image denoising and image transformation. In this paper I study the possibility of integrating Self-ONNs into deep reinforcement learning networks to see whether they increase the performance of said networks. An equal sized CNN and self-ONN network is placed as a feature extraction network into a deep reinforcement learning network, which uses advantage actor-critic (A2C) algorithm, which is then taught to play ten different Atari 2600 games. The performances of these two networks are then compared to each other. The test results seem to indicate that there is no benefit in using the self-ONNs in this case, though something that needs to be pointed out is the obviously too small number of frames used to teach the networks these games, as out of ten games only in two of them did the network both see substantial amount of learning as well as achieve average performance which went clearly over a random actor, which is the baseline model in this case.