Discounted cost exponential semi-Markov decision processes with unbounded transition rates: a service rate control problem with impatient customers


Creative Commons License

ÇEKYAY B.

Probability in the Engineering and Informational Sciences, 2024 (SCI-Expanded) identifier

  • Publication Type: Article / Article
  • Publication Date: 2024
  • Doi Number: 10.1017/s0269964824000044
  • Journal Name: Probability in the Engineering and Informational Sciences
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, Applied Science & Technology Source, Compendex, Computer & Applied Sciences, INSPEC, zbMATH, Civil Engineering Abstracts
  • Keywords: Customization, Semi-Markov decision processes, Service rate control, Unbounded transition rates
  • Yıldız Technical University Affiliated: Yes

Abstract

We focus on exponential semi-Markov decision processes with unbounded transition rates. We first provide several sufficient conditions under which the value iteration procedure converges to the optimal value function and optimal deterministic stationary policies exist. These conditions are also valid for general semi-Markov decision processes possibly with accumulation points. Then, we apply our results to a service rate control problem with impatient customers. The resulting exponential semi-Markov decision process has unbounded transition rates, which makes the well-known uniformization technique inapplicable. We analyze the structure of the optimal policy and the monotonicity of the optimal value function by using the customization technique that was introduced by the author in prior work.