Introduction

Markov Models for disease progression are common in medical decision making (see references below). The parameters in a Markov model can be estimated by observing the time it takes patients in any state i to make a transition to another state j (fully observed data). Frequently, however, what is recorded is the starting state and the end-state T years later, without information on whether other states were visited (partially observed data). Two issues then arise:

1. How to estimate Markov transition rate and probability parameters from such partially observed data.
2. How to synthesise evidence from several studies, some of which may have recorded fully observed data, and some of which may have recorded partially observed data.

Welton et al (2005) describes how these partially observed data can be used to inform a Markov rate matrix, and takes a Bayesian statistical approach using the WinBUGS software. The paper covers:

• How to map between transition rates and probabilities in multi-state models using Kolmorgorov’s forward equations. Estimating transition rates and probabilities for fully observed data.
• Estimating transition rates and probabilities for partially observed data.
• How to use the software WinBUGS Differential Interface (WBDiff) to solve the forward equations numerically for more complex models.
• Comparing the fit of different models and assessing the consistency of the evidence.
• How to synthesise information from different sources to estimate parameters in a Markov transition rate model.
• Discussion of how to apply the estimated model to a target population, including calibration to a particular study that is most representative of the target population (for example the most recent study).

Resources available on this site

• Supplementary material (PDF, 126kb) describes the relationship between rates and transition probabilities for 2, 3, and 4-state models

The files below cannot be opened directly;  they need to be saved to a local file before opening in WinBUGS. Right-click a link and then 'Save Link/Target As'

WinBUGS code to fit the models presented in Welton et al (2005) (ODC, 5kb). These include programs for:

1. Fully observed data
2. Partially observed data
   1. 3-state forward model
   2. Model comparison and evidence consistency
   3. 3-state general and 4-state forward models
   4. 3-state and 4-state forward models using WBDiff modules to provide numerical solutions to Kolmogorov’s forward equations
   5. Multiple source evidence synthesis

To run those programs that use WBDiff Modules, you will need to download the WBDiff modules ThreeStateForward (ODC, 3.2kb) and FourStateForward (ODC, 3.7kb) and also the WBDiff software. The file WBDiff_example.pdf that you will have downloaded with this software, gives instructions on how to compile WBDiff modules, including downloading the BlackBox software.  These ODC files are required to run the relevant WinBUGS code in 2.iv as described above.

While we have taken every effort to ensure the reliability of the code, we cannot guarantee that its use is appropriate in any specific case, nor can we take responsibility for interpretation of the results, for any inaccuracies, or for any consequences of its use.

References

1. Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Health Econ 1983; 13:322-338.
2. Briggs A, Sculpher M. An Introduction to Markov Modelling for Economic Evaluation. Pharmacoeconomics, PharmacoEcon 1998; 13(4):397-409.
3. Welton NJ, Ades AE. Estimation of Markov chain transition probabilities and rates from fully and partially observed data:  Uncertainty propagation, evidence synthesis and model calibration. Medical Decision Making, 2005;25:633-645.