In mean field theory one assumes that
correlations between the states of different cells are not
generated by application of a cellular automaton to a configuration.
Under this assumption,
the probability of a large block is estimated as the product of
the probabilities of the states of cells it contains. The mean field
theory will fail to accurately model a cellular automaton if
correlations are generated as the rule is iterated.
The Markov approximation
for 
takes spatial correlations into account explicitly.
In the Markov approximation,
the probability
of large blocks are estimated in terms of the probability of
smaller blocks which they contain. This leads to a systematic
generalization of the mean field theory. In this section,
the first step of this generalization is
examined.
In 2nd-order Markov approximation, correlations
are introduced in terms of the probabilities of
contiguous pairs of cells. Pair probabilities are used to define a
2-step Markov process. Longer block probabilities are estimated as follows.
Let 
0,1 be
the possible states of a cell in position i in a block.
Let 
be an n-block, and 
be the
probability of an n-block. If the probabilities of all 2-blocks are
known, the probability of an n-block, 
, is estimated by
where the 1-block probabilities are found by appropriate summation of the 2-block probabilities. That the extension of small block probabilities defined by equation (8) is a consistent assignment of probabilities to larger blocks is proven in [3]. That this extension is the extension of maximum entropy consistent with the given assignment of 2-block probabilities is proven in [12]. Blocks which always have the same probability according to equation (8) are said to be of the same 2nd-order type.
It is useful to refer to 2nd-order types by a code.
1st-order types are coded by a single index, i, which is the
power to which 
is raised in the 1st-order estimate for
the probability of a block (equation 5). Knowing the value
of this index, and the length of the block, the other relevant exponent,
that attached to 
, can be found.
The 2nd-order estimate (equation 8) involves a number of
different block probabilities raised to different, interrelated,
powers. A good code should specify the minimum number of exponent
values such that the rest can be found by appeal to the Kolmogorov
consistency conditions on block probabilities [2].
These state that
2nd-order types are coded here by a triple
where x is the total number of 10 and 01
sub-blocks counting overlaps,
y is the number of 11 sub-blocks again
counting overlaps, and z is the is the number of
cells in state 1 in the central n-2 region of the n-block.
The number of other 1- and 2-blocks in the n-block can be
found using the Kolmogorov consistency conditions.
As an example, 10010 and 10100 are of the same 2nd-order type, coded by
. The probability of this type is
.
The Kolmogorov consistency conditions
were just used to produce a compact code for 2nd-order types. In much the
same way, we again apply the consistency conditions to
find a compact parameterization of block probabilities themselves.
Here the probabilities of 1- and 2- blocks will be parameterized
by and 
. Any other pair of linearly independent
1- and/or 2-block probabilities could also
serve as parameters. As will become clear, however, it
is desirable to chose parameters in a hierarchical fashion. That is,
so that parameters for small-block probabilities are a subset of
the parameters for long-block probabilities.
The other 2-block probabilities can
be found from the parameters chosen
using the Kolmogorov consistency conditions,
e.g. 
.
The 2nd-order Markov approximation preserves the combinatorial information
contained in both the cellular automaton map from neighborhood
blocks to single cells and the map from (d+1)-length blocks
to 2-blocks.
The 2nd-order approximation is constructed by substitution of the probability
estimate given by equation (8) into equations of the form (2) for
the evolution of and .
Then, as was done in the derivation of the mean field equation (7),
the sum is rearranged so that blocks of the same type are collected
together.
A coefficient 
is associated to
each 2nd-order type of d-block, and a coefficient 
is associated to
each 2nd-order type of (d+1)-block.
The 
coefficients count the
number of d-blocks of the given 2nd-order type which lead to a 1 under
the cellular automaton, and the 
coefficients count the number of
(d+1)-blocks of a given 2nd-order type which lead to 11. Let
be the
probability at time t
of a block of 2nd-order type according to
equation (8).
The 2nd-order equations are then
where the sums run over the 2nd-order types of d- and (d+1)- blocks respectively.
As was the case for 0th- and 1st-order theories, many rules may lead
to the same 2nd-order coefficient values. Thus each allowed
set of 2nd-order coefficients defines a 2nd-order class of cellular
automata.
Second order classes
determined by and coefficient values are strict refinements of
mean field classes.
The 
coefficient values
of the mean field class containing a 2nd-order class
are the sum of
coefficient values which count blocks of the same
1st-order type.