List of figures
Prologue to the second edition, Peter W. Glynn
Preface to the second edition, Sean Meyn
Preface to the first edition
I COMMUNICATION and REGENERATION
1 Heuristics
1.1 A range of Markovian environments
1.2 Basic models in practice
1.3 Stochastic stability for Markov models
1.4 Commentary
2 Markov models
2.1 Markov models in time series
2.2 Nonlinear state space models*
2.3 Models in control and systems theory
2.4 Markov models with regeneration times
2.5 Commentary*
3 Transition probabilities
3.1 Defining a Markovian process
3.2 Foundations on a countable space
3.3 Specific transition matrices
3.4 Foundations for general state space chains
3.5 Building transition kernels for specific models
3.6 Commentary
4 Irreducibility
4.1 Communication and irreducibility: Countable spaces
4.2 ψ-Irreducibility
4.3 ψ-Irreducibility for random walk models
4.4 ψ-Irreducible linear models
4.5 Commentary
5 Pseudo-atoms
5.1 Splitting ψ-irreducible chains
5.2 Small sets
5.3 Small sets for specific models
5.4 Cyclic behavior
5.5 Petite sets and sampled chains
5.6 Commentary
6 Topology and continuity
6.1 Feller properties and forms of stability
6.2 T-chains
6.3 Continuous components for specific models
6.4 e-Chains
6.5 Commentary
7 The nonlinear state space model
7.1 Forward accessibility and continuous components
7.2 Minimal sets and irreducibility
7.3 Periodicity for nonlinear state space models
7.4 Forward accessible examples
7.5 Equicontinuity and the nonlinear state space model
7.6 Commentary*
II STABILITY STRUCTURES
8 Transience and recurrence
8.1 Classifying chains on countable spaces
8.2 Classifying C-irreducible chains
8.3 Recurrence and transience relationships
8.4 Classification using drift criteria
8.5 Classifying random walk on R+
8.6 Commentary*
9 Harris and topological recurrence
9.1 Harris recurrence
9.2 Non-evanescent and recurrent chains
9.3 Topologically recurrent and transient states
9.4 Criteria for stability on a topological space
9.5 Stochastic comparison and increment analysis
9.6 Commentary
10 The existence of π
10.1 Stationarity and invariance
10.2 The existence of π: chains with atoms
10.3 Invariant measures for countable space models*
10.4 The existence of π: C-irreducible chains
10.5 Invariant measures for general models
10.6 Commentary
11 Drift and regularity
11.1 Regular chains
11.2 Drift, hitting times and deterministic models
11.3 Drift criteria for regularity
11.4 Using the regularity criteria
11.5 Evaluating non-positivity
11.6 Commentary
12 Invariance and tightness
12.1 Chains bounded in probability
12.2 Generalized sampling and invariant measures
12.3 The existence of a a-finite invariant measure
12.4 Invariant measures for e-chains
12.5 Establishing boundedness in probability
12.6 Commentary
III CONVERGENCE
13 Ergodicity
13.1 Ergodic chains on countable spaces
13.2 Renewal and regeneration
13.3 Ergodicity of positive Harris chains
13.4 Sums of transition probabilities
13.5 Commentary*
14 f-Ergodicity and f-regularity
14.1 f-Properties: chains with atoms
14.2 f-Regularity and drift
14.3 f-Ergodicity for general chains
14.4 f-Ergodicity of specific models
14.5 A key renewal theorem
14.6 Commentary*
15 Geometric ergodicity
15.1 Geometric properties: chains with atoms
15.2 Kendall sets and drift criteria
15.3 f-Geometric regularity of Φ and its skeleton
15.4 f-Geometric ergodicity for general chains
15.5 Simple random walk and linear models
15.6 Commentary*
16 V-Uniform ergodicity
16.1 Operator norm convergence
16.2 Uniform ergodicity
16.3 Geometric ergodicity and increment analysis
16.4 Models from queueing theory
16.5 Autoregressive and state space models
16.6 Commentary*
17 Sample paths and limit theorems
17.1 Invariant a-fields and the LLN
17.2 Ergodic theorems for chains possessing an atom
17.3 General Harris chains
17.4 The functional CLT
17.5 Criteria for the CLT and the LIL
17.6 Applications
17.7 Commentary*
18 Positivity
18.1 Null recurrent chains
18.2 Characterizing positivity using pn
18.3 Positivity and T-chains
18.4 Positivity and e-chains
18.5 The LLN for e-chains
18.6 Commentary
19 Generalized classification criteria
19.1 State-dependent drifts
19.2 History-dependent drift criteria
19.3 Mixed drift conditions
19.4 Commentary*
20 Epilogue to the second edition
20.1 Geometric ergodicity and spectral theory
20.2 Simulation and MCMC
20.3 Continuous time models
IV APPENDICES
A Mud maps
A.1 Recurrence versus transience
A.2 Positivity versus nullity
A.3 Convergence properties
B Testing for stability
B.1 Glossary of drift conditions
B.2 The scalar SETAR model: a complete classification
C Glossary of model assumptions
C.1 Regenerative models
C.2 State space models
D Some mathematical background
D.1 Some measure theory
D.2 Some probability theory
D.3 Some topology
D.4 Some real analysis
D.5 Convergence concepts for measures
D.6 Some martingale theory
D.7 Some results on sequences and numbers
Bibliography
Indexes
General index
Symbols
