Mor Harchol-Balter. Introduction to Probability for Computing, Cambridge University Press, 2024. Not for distribution. 422 24 Discrete-Time Markov Chains: Finite-State Definition 24.3The transition probability matrix associated with any DTMC is a matrix, P, whose ( , )th entry, , represents the probability of moving to state on the next transition, given that the current state is .