Robust Warehouse AGV Navigation in Dynamic Environments Using Soft Actor-Critic Reinforcement Learning

Abstract

Autonomous guided vehicles (AGVs) operating in modern warehouse environments face the critical challenge of navigating safely and efficiently amid high-density dynamic obstacles, including human workers and peer vehicles. Classical reactive planners such as the Dynamic Window Approach (DWA) fail under these conditions due to their inability to predict obstacle motion, while conventional deep reinforcement learning methods often overfit to training layouts and lack the behavioral diversity needed for robust generalization. This paper presents a Soft Actor-Critic (SAC) based navigation framework that addresses these limitations through maximum-entropy reinforcement learning combined with a multicomponent reward design and domain-randomized training. The proposed method jointly optimizes goalreaching efficiency, collision avoidance, energy consumption, and trajectory smoothness within a unified learning objective. We evaluate the approach against three strong baselines—DDPG, TD3, and DWA— across four structurally distinct warehouse layouts, three of which are unseen during training. Experimental results demonstrate that SAC achieves the highest mean cumulative reward (99.32 vs. 82.18 for DDPG, 88.58 for TD3, and 55.41 for DWA), the lowest collision rate (1.5 TD3), and the shortest paths (15.54 m average vs. 22.17 m for DDPG and 25.54 m for TD3). Notably, SAC exhibits near-invariant path lengths across all layouts with a cross-layout standard deviation of only 0.22 m, providing compelling evidence of geometry-agnostic navigation. Energy consumption is approximately half that of DDPG and TD3, while trajectory smoothness is improved by a factor of two. These results establish that entropy-regularized deep reinforcement learning, coupled with principled reward shaping and domain randomization, produces warehouse navigation policies that are substantially more robust, efficient, and deployment-ready than both classical planners and deterministic actor-critic alternatives.