Markov chain

Definition

In probability theory and statistics, a Markov chain or Markov process is a stochastic process describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs now." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC). Markov processes are named in honor of the Russian mathematician Andrey Markov.

Related concepts

A Mathematical Theory of Communication Abstract Wiener space Actuarial mathematics Additive process Adjacency matrix Advances in Mathematics Agner Krarup Erlang Airy process Alexander Pushkin Algorithmic composition Andrei Kolmogorov Andrey Markov Arithmetic coding Asset pricing model AstroTurf Authoritarian Autoregressive conditional heteroskedasticity Autoregressive integrated moving average Autoregressive model Autoregressive moving-average model Base stealing Bayes' theorem Bayesian inference Bayesian statistics Bernoulli distribution Bernoulli process Bernoulli scheme Bernoulli trial Bessel process Biased random walk on a graph Binomial distribution Binomial options pricing model Bioinformatics Biology Birth process Birth–death process Black–Derman–Toy model Black–Karasinski model Black–Scholes model Block-coding system Blumenthal's zero–one law Boole's inequality Boolean network Borel–Cantelli lemma Branching process Brownian bridge Brownian excursion Brownian meander Brownian motion Bulk queue Bunt (baseball)Burkholder–Davis–Gundy inequalities Bühlmann model Cameron–Martin theorem Capitalism Cauchy process Central limit theorem Chacon system Chan–Karolyi–Longstaff–Sanders process Chapman–Kolmogorov equation Chemical species Chemistry Chen model Chinese restaurant process Chris Heyde Classical Wiener space Claude Shannon Closed manifold Collectively exhaustive events Compartmental models in epidemiology Complementary event Compound Poisson process Conditional independence Conditional probability Conditional probability distribution Constant elasticity of variance model Contact process (mathematics)Context-free system Continuous-time Continuous-time Markov chain Continuous-time random walk Continuous-time stochastic process Continuous or discrete variable Continuous stochastic process Convergence of random variables Copolymer Cosma Shalizi Countable set Countably infinite Cox process Cox–Ingersoll–Ross model Cramér–Lundberg model Credit rating agency Crystallization Csound Càdlàg D. G. Champernowne Das Kapital Data compression Defective matrix Democratic regime Determinantal point process Determinism Deterministic system Diagonal matrix Diameter (graph theory)Diffeomorphism Diffusion process Dirac measure Dirichlet process Discrete-time Markov chain Discrete-time stochastic process Dissociated press Doléans-Dade exponential Donsker's theorem Doob's martingale convergence theorems Doob's martingale inequality Doob's optional stopping theorem Doob's upcrossing inequality Doob decomposition theorem Doob–Meyer decomposition theorem Dot product Dynamical system Dynamics of Markovian particles Dynkin's formula Dyson Brownian motion Econometrics Economic development Economics Edmund F. Robertson Efficient-market hypothesis Eigendecomposition Eigenvalue Eigenvector Element (mathematics)Elementary event Empirical process Encyclopedia of Mathematics Engelbert–Schmidt zero–one law Entropy encoding Enzyme Ergodic Ergodic theory Ergodicity Eugene Borisovich Dynkin Eugene Dynkin Eugene Onegin Eugene Seneta European Mathematical Society Event (probability theory)Examples of Markov chains Exchangeable random variables Expected value Experiment (probability theory)Exponential distribution Extreme value theory Feller-continuous process Feller process Feynman–Kac formula Filtration (probability theory)Finance Finite graph Finite group Finite set First-order differential equation Fisher–Tippett–Gnedenko theorem Fizmatgiz Fleming–Viot process Fluid queue Forward equation Fractional Brownian motion Francis Galton G-network Galton–Watson process Gambler's ruin Gamma process Garman–Kohlhagen model Gaussian process Gaussian random field Gauss–Markov process General equilibrium Genome Geometric Brownian motion Geometric process Giandomenico Majone Gibbs measure Girsanov theorem Girth (graph theory)Google Greatest common divisor Harris chain Hawkes process Heath–Jarrow–Morton framework Henri Poincaré Henry William Watson Herbert A. Simon Hertz Heston model Hewitt–Savage zero–one law Hi Ho! Cherry-O Hidden Markov model Hopfield model Ho–Lee model Hull–White model Hunt process Identity matrix In silico Independence (probability theory)Independent and identically distributed random variables Indeterminism Infinitesimal generator (stochastic processes)Information entropy Information theory Integers Interacting particle system Irénée-Jules Bienaymé Ising model Itô's lemma Itô diffusion Itô integral Itô process J. L. Doob J. Laurie Snell James D. Hamilton James R. Norris John G. Kemeny Joint probability distribution Jordan normal form Joseph O'Rourke (professor)Journal of Econometrics Jump diffusion Jump process Karl Marx Kelly's lemma Kishor S. Trivedi Kolmogorov's criterion Kolmogorov's zero–one law Kolmogorov continuity theorem Kolmogorov extension theorem Korn–Kreer–Lenssen model Kosambi–Karhunen–Loève theorem Kronecker delta Kunita–Watanabe inequality LIBOR market model LZ77 and LZ78 Large deviation principle Large deviations theory Lattice QCD Laurent E. Calvet Law of large numbers Law of the iterated logarithm Law of total probability Lempel–Ziv–Markov chain algorithm List of inequalities List of stochastic processes topics Little-o notation Local martingale Local time (mathematics)Locally compact space Loop-erased random walk Louis Bachelier Lévy's zero–one law Lévy process Lévy–Prokhorov metric M/G/1 queue M/M/1 queue M/M/c queue MIDI MacTutor History of Mathematics Archive Machine learning Main diagonal Malliavin calculus Marcinkiewicz–Zygmund inequality Marginal distribution Mark V. Shaney Markov additive process Markov chain Monte Carlo Markov chain approximation method Markov chain central limit theorem Markov chain geostatistics Markov chain mixing time Markov chain tree theorem Markov chains on a measurable state space Markov decision process Markov information source Markov kernel Markov model Markov odometer Markov operator Markov process Markov property Markov random field Markov switching multifractal Markov text Martingale (probability theory)Martingale difference sequence Martingale representation theorem Master equation Mathematical finance Mathematical statistics Matrix (mathematics)Matrix population models Maurice Fréchet Max (software)Maximal entropy random walk Maximal ergodic theorem McKean–Vlasov process Measure-preserving dynamical system Measure (mathematics)Memorylessness Michaelis-Menten Michaelis-Menten kinetics Michaelis–Menten kinetics Middle class Mixing (mathematics)Models of DNA evolution Molecules Moran process Moving-average model Mutual exclusivity Natural language generation Natural numbers Neurobiology Non-homogeneous Poisson process Norbert Wiener Norm (mathematics)Normal distribution Nucleotide Number line Optional stopping theorem Ornstein isomorphism theorem Ornstein–Uhlenbeck process Outcome (probability)Oxford English Dictionary PageRank Pareto distribution Parody generator Partially observable Markov decision process Path-dependent Pattern recognition Paul Ehrenfest Pavel Nekrasov Percolation theory Perron–Frobenius theorem Phase-type distribution Phrase (music)Phylogenetics Physics Piecewise-deterministic Markov process Pitman–Yor process Point process Poisson distribution Poisson point process Poisson process Polish space Political Population dynamics Posterior distribution Potts model Predictable process Probability Probability axioms Probability distribution Probability measure Probability space Probability theory Probability vector Progressively measurable process Prokhorov's theorem Protein folding Quadratic variation Quantum Markov chain Quasimartingale Queueing model Queueing theory Random dynamical system Random field Random graph Random variable Random walk Randomness Realization (probability)Reflection principle (Wiener process)Regenerative process Regular conditional probability Reinforcement learning Rendleman–Bartter model Renewal process Renewal theory Right stochastic matrix Risk process Roland Dobrushin Ruin theory SABR volatility model Sample-continuous process Sample space Sanov's theorem Schramm–Loewner evolution Self-avoiding walk Self-similar process Semi-Markov process Semimartingale Sequence Sigma-martingale Signal processing Singleton (mathematics)Skorokhod's representation theorem Skorokhod integral Skorokhod space Snakes and Ladders Snell envelope Society for Industrial and Applied Mathematics Sofic system Software Sparre–Anderson model Speech processing Springer-Verlag Stable process Standard simplex State estimation State space Stationary probability distribution Stationary process Stationary stochastic process Statistical mechanics Statistical model Statistics Steric effects Steven Lalley Stochastic analysis Stochastic cellular automata Stochastic cellular automaton Stochastic chains with memory of variable length Stochastic control Stochastic differential equation Stochastic matrix Stochastic process Stopping time Stratonovich integral Submartingale Subshift of finite type SuperCollider Superlattice Supermartingale Superprocess Sydney Chapman (mathematician)System of linear equations Systems biology Tanaka equation Tao Te Ching Tatyana Afanasyeva-Ehrenfest Telegraph process Telescoping Markov chain Thermodynamics Thue–Morse sequence Time reversibility Time series Time series analysis Transition rate matrix Tree diagram (probability theory)Uniform integrability Unit vector University of St Andrews Usenet Usual hypotheses Variable-order Markov model Variance Variance gamma process Vasicek model Venn diagram Viterbi algorithm Weak law of large numbers White noise Wiener process Wiener sausage Wilkie investment model William Feller Yuval Peres Zero matrix Zero–one law

34 concepts already in your glossary