REAM: A Reinforcement Learning-based Energy-Efficient and Adaptive Multi-Modal Routing Protocol for Underwater Acoustic Networks

Abstract

Enhancing reliability in dense underwater networks with hefty data traffic often raises energy consumption due to constant packet listening and retransmissions caused by packet loss. To address the challenging energy demand in Underwater Acoustic Sensor Networks (UASNs), we propose a Reinforce-ment learning (RL)-based Energy-efficient and Adaptive Multi-modal routing protocol abbreviated as REAM, that integrates Q-learning with multi-modal communication to enhance energy efficiency in underwater networks by adaptively selecting the optimal mode for packet transmission. We compare the performance of two variants of the proposed REAM protocol-REAM-MM, which uses two modems, and REAM-SM, which uses a single modem, against two other state-of-the-art protocols namely QELAR and MARLIN-Q. Our results demonstrate the effective-ness of using multiple modems instead of a single modem in reducing energy consumption and improving reliability. REAM-MM reduces energy consumption per bit by up to 81.2%, 79.6%, and 72.9% under low, medium, and high traffic scenarios, respec-tively, compared to the best-performing alternative, MARLIN-Q. REAM-MM achieves a consistently comparable Packet Delivery Ratio (PDR) to MARLIN-Q and a higher PDR than QELAR and REAM-SM. Additionally, it maintains the lowest energy consumption under all traffic conditions and dense networks.