Stochastic Modeling with Markov Chains

Dr. Thuan Nguyen

Information

This course aims to provide an introduction on Markov chains in discrete time. The main content includes:

• Markov chains; Transition probabilities; Chapman-Kolmogorov equations;
• Classification of states: Communicating and absorbing states; Recurrence and transience;
• Hitting probabilities and mean hitting times; Birth and death chains, in particular M/M/1 model;
• 1/2/3-dimensional random walks;
• Stationary distribution; Convergence to the equilibrium; Ergodic theorem;
• Markov Chain Monte Carlo methods; Metropolis-Hasting algorithm; Gibbs sampler.

References:

• R. Durrett, Essentials of Stochastic Processes, 3rd eds., Springer, 2016. (available online in the IP range of Saarland University)
• O. Häggström, Finite Markov Chains and Algorithmic Applications, Cambridge, 2002. (available online in the IP range of Saarland University)
• J. Norris, Markov Chains, Cambridge, 1997.

All further information and course material can be found on the learning management system Moodle. If you are interested to attend the course, please ask for the enrolment key at your earliest convenience via email to:

 nguyen [at] math.uni-sb.de

The course will be held in English.

Moodle enrolment

Lecture

Wednesday 14:15 - 15:45     Starting date: 12 April 2023     Place: Building E2 4, room 1.15, HS IV

Tutorial

Wednesday 9:00 - 10:00 am     Starting date: 19 April 2023     Place: Building E2 5, Seminarraum 1 (U.37).