Discounted cost exponential semi-Markov decision processes with unbounded transition rates: a service rate control problem with impatient customers
Probability in the Engineering and Informational Sciences, 2024 (SCI-Expanded, Scopus)
- Publication Type: Article / Article
- Publication Date: 2024
- Doi Number: 10.1017/s0269964824000044
- Journal Name: Probability in the Engineering and Informational Sciences
- Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, Applied Science & Technology Source, Compendex, Computer & Applied Sciences, INSPEC, zbMATH, Civil Engineering Abstracts
- Keywords: Customization, Semi-Markov decision processes, Service rate control, Unbounded transition rates
- Open Archive Collection: AVESIS Open Access Collection
- Yıldız Technical University Affiliated: Yes
Abstract
We focus on exponential semi-Markov decision processes with unbounded transition rates. We first provide several sufficient conditions under which the value iteration procedure converges to the optimal value function and optimal deterministic stationary policies exist. These conditions are also valid for general semi-Markov decision processes possibly with accumulation points. Then, we apply our results to a service rate control problem with impatient customers. The resulting exponential semi-Markov decision process has unbounded transition rates, which makes the well-known uniformization technique inapplicable. We analyze the structure of the optimal policy and the monotonicity of the optimal value function by using the customization technique that was introduced by the author in prior work.