Module 19:
Finite-state Markov chains and random walks

Module 19

What this module is for

Both. This belongs in the course, but as a focused introduction rather than an entire stochastic-process programme.

Lesson 19.1 · Core

States, transitions and the Markov property

Depth: Working proficiency

QT/QR relevance: Both; elementary dynamic models

Lesson 19.2 · Core

First-step analysis and gambler’s ruin

Depth: Interview mastery

QT/QR relevance: Both; threshold and stopping problems

Lesson 19.3 · Core

Stationary distributions versus limiting distributions

Depth: Working proficiency

QT/QR relevance: Both; long-run modelling

Lesson 19.4 · Core

Occupation counts and accumulated rewards

Depth: Interview mastery

QT/QR relevance: Both; activity and state-dependent outcomes

Lesson 19.5 · Core · QR Extension

Simulation and balance

Depth: Working proficiency; detailed-balance extension for QR

QT/QR relevance: Especially QR; model checking and later computational methods

Assessment

Module 19 assessment

Open assessment · No submitted assessment yet