Markov Chain Models

Suppose a population consists of k disjoint sub-populations labeled by the integers $1 \ldots k$. Let p_i be the fraction of the population in sub-population i. Form the vector $\vec{S} \in R^k$ called the state vector of the system:

$$\vec{S} = \left( \begin{array}{l} p_1 \\ p_2 \\ \cdots \\ p_k \end{array}\right)$$

The sum $\Sigma_{i=1}^k p_i$ of the population fractions is assumed to be 1.

The state vector $\vec{S}$ will depend on time t. In a discrete model, we might measure the population fractions p_i at times $\{t_1,t_2, \ldots t_n, \ldots \}$. Use $\vec{S}[t_n]$ and p_i[t_n] respectively to denote values at the end of the n'th time period.

In a Markov chain model one assumes that there are transition probabilities T_ij representing the proportion of the population in sub-population j during time period n which enter sub-population i for time period n+1. These transition probabilities are independent of time. Let T be the k x k matrix whose i,j entry is T_ij.

We also assume that the entire population in time period n+1 comes by this transition process from the population during time period n. Thus

$$p_i(t_{n+1}) = T_{i1} p_1(t_n) + T_{i2} p_2(t_n) + \cdots T_{ik} p_k(t_n) +$$

This is just the formula for the i'th entry of a matrix product of T and the n'th state vector $\vec{S}[t_n]$. Thus in matrix form, $\vec{S}[t_{n+1}] = T \circ \vec{S}[t_{n}]$.

More generally, we can keep track of the change in population over r time periods by computing the r'th power of T. Specifically $\vec{S}[t_{n+r}] = T^r \circ \vec{S}[t_{n}]$.

The sum of the entries in each column (e.g. the j'th) of the transition matrix T should be 1 because the entries of that column reflect all the possible destinations at time t_n+1 for the people in region j at time t_n.