Copyright © 2009 Mario Lefebvre and Jean-Luc Guilbault. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

A Markov chain with state space {0,…,N} and transition probabilities depending on the current state is studied. The chain can be considered as a discrete Ornstein-Uhlenbeck process. The probability that the process hits N before 0 is computed explicitly. Similarly, the probability that the process hits N before −M is computed in the case when the state space is {−M,…,0,…,N} and the transition probabilities pi,i+1 are not necessarily the same when i is positive and i is negative.