Solving the stochastic and dynamic capacitated vehicle routing problem utilizing advisory board reinforcement learning
Systems and Soft Computing, vol.9, 2026 (ESCI, Scopus)
- Publication Type: Article / Article
- Volume: 9
- Publication Date: 2026
- Doi Number: 10.1016/j.sasc.2026.200550
- Journal Name: Systems and Soft Computing
- Journal Indexes: Emerging Sources Citation Index (ESCI), Scopus
- Keywords: Advisory board mechanism, Double deep Q-learning, Neural network, Reinforcement learning, Stochastic demand, Vehicle routing problem
- Yıldız Technical University Affiliated: Yes
Abstract
The Capacitated Vehicle Routing Problem (CVRP) is a fundamental logistics optimization problem, yet conventional exact, heuristic, and metaheuristic approaches are instance-sensitive and solve each new problem separately. This study proposes an Advisory Board Reinforcement Learning framework (AdvB-RL) for stochastic and dynamic CVRP instances in which customer demands, active service nodes, fleet size, and vehicle capacities vary across episodes. The experiments are not tied to a specific geographical case study; instead, generated routing maps with different numbers of cities and vehicles are used to evaluate generalizable CVRP scenarios. Customer demands are stochastic variables, and 100 randomly generated test instances are used for robustness evaluation. AdvB-RL is implemented using a Double Deep Q-Learning structure supported by an advisory board of artificial neural networks. The state representation includes city demand status, vehicle capacity status, current city position, and initial city information. The neural architecture includes six hidden layers with 500 neurons each, and the model is trained for 100,000 episodes. The proposed method is compared with Google OR-Tools and Simulated Annealing using average travel distance and success rate. In the 20-city and 3-vehicle scenario, AdvB-RL achieved the lowest average distance of 286.46 and a 100% success rate, compared with 291/79% for Google OR-Tools and 292.91/79% for Simulated Annealing. In the 50-city and 3-vehicle scenario, AdvB-RL achieved 482 distance and 100% success, compared with 454/83% and 779/82%, respectively. The main contribution is a reusable advisory-board-based reinforcement learning framework that learns from stochastic core environments and provides reliable online/offline routing decisions for dynamic CVRP sub-problems.