Minimum description length

Definition

Minimum Description Length (MDL) is a model selection principle where the shortest description of the data is the best model. MDL methods learn through a data compression perspective and are sometimes described as mathematical applications of Occam's razor. The MDL principle can be extended to other forms of inductive inference and learning, for example to estimation and sequential prediction, without explicitly identifying a single model of the data.

Related concepts

Accelerated failure time model Actuarial science Adaptive clinical trial Akaike information criterion Algorithmic Information Theory Algorithmic information theory Algorithmic probability Alphabet Analysis of covariance Analysis of variance Anderson–Darling test Anscombe transform Approximation theory Arithmetic mean Arithmetic–geometric mean Artificial general intelligence Asymptotic theory (statistics)Autocorrelation Automated theorem proving Autoregressive conditional heteroskedasticity Autoregressive model Autoregressive–moving-average model Average absolute deviation Bar chart Bayes estimator Bayes factor Bayesian Information Criterion Bayesian experimental design Bayesian inference Bayesian information criterion Bayesian linear regression Bayesian model averaging Bayesian probability Bias of an estimator Binary numeral system Binomial regression Bioinformatics Biostatistics Biplot Blocking (statistics)Bootstrapping (statistics)Box plot Box–Cox transformation Box–Jenkins method Breusch–Godfrey test Calibration curve Canonical correlation Cartography Categorical variable Census Central limit theorem Central tendency Chebyshev nodes Chebyshev polynomials Chemometrics Chi-squared test Clinical study design Clinical trial Cluster analysis Cluster sampling Cochran–Mantel–Haenszel statistics Coefficient of determination Coefficient of variation Cohen's kappa Cohort study Cointegration Completeness (statistics)Computational learning theory Computational statistics Confidence interval Confounding Contingency table Continuous probability distribution Contraharmonic mean Control chart Correlation and dependence Correlogram Count data Credible interval Crime statistics Cross-correlation Cross-sectional study Cross-validation (statistics)Cubic mean Curve fitting Data cleaning Data collection Data compression Data preprocessing Data transformation (statistics)Decomposition of time series Degrees of freedom (statistics)Demographic statistics Density estimation Descriptive statistics Design of experiments Detrending Dickey–Fuller test Differencing Dimensionality reduction Divergence (statistics)Durbin–Watson statistic Econometrics Effect size Efficiency (statistics)Elliptical distribution Empirical distribution function Engineering statistics Entropy (information theory)Environmental statistics Epidemiology Errors and residuals Errors and residuals in statistics Estimating equations Experiment Exponential family Exponential smoothing F-test Factor analysis Factorial experiment Failure rate Fan chart (statistics)Feature scaling First-hitting-time model Fisher transformation Forest plot Fourier analysis Frequency distribution Frequency domain Frequentist inference Friedman test Frisch–Waugh–Lovell theorem Function (mathematics)G-test Gaussian quadrature Gauss–Markov theorem General linear model Generalized least squares Generalized linear model Generalized mean Geographic information system Geometric mean Geostatistics Goodness of fit Granger causality Graphical model Grouped data Growth curve (statistics)Harmonic mean Heatmap Heinz mean Heronian mean Histogram Hodges–Lehmann estimator Homoscedasticity and heteroscedasticity Index of dispersion Inductive inference Inductive probability Information theory Interaction (statistics)Interquartile range Interval estimation Isotonic regression Iteratively reweighted least squares Jackknife resampling Jarque–Bera test Johansen test Jonckheere's trend test Jorma Rissanen Jurimetrics Kaplan–Meier estimator Kendall rank correlation coefficient Kendall tau rank correlation coefficient Kolmogorov complexity Kolmogorov structure function Kolmogorov–Smirnov test Kraft–McMillan theorem Kriging Kruskal–Wallis test Kurtosis L-moment Lasso (statistics)Least-squares spectral analysis Least squares Lehmann–Scheffé theorem Lehmer mean Lempel–Ziv complexity Likelihood-ratio test Likelihood function Likelihood interval Lilliefors test Line chart Linear discriminant analysis Linear least squares (mathematics)Linear regression List of fields of application of statistics List of statistical tests List of statistics articles Ljung–Box test Local regression Location parameter Location–scale family Log-rank test Log transformation Logistic regression Loss function Lossless compression Lp space M-estimator MIT Press Mallows's Cp Manifold hypothesis Mann–Whitney U test Marginal likelihood Marvin Minsky Maximum a posteriori estimation Maximum likelihood McNemar's test Mean Mean and predicted response Median Median-unbiased estimator Medical statistics Method of moments (statistics)Methods engineering Minimum-variance unbiased estimator Minimum distance estimation Minimum mean-square error Minimum message length Min–max normalization Missing data Mixed model Mode (statistics)Model selection Model specification Moment (mathematics)Monotone likelihood ratio Moving least squares Multiple comparisons Multivariate adaptive regression splines Multivariate analysis of variance Multivariate distribution Multivariate normal distribution Multivariate statistics National accounts Natural experiment Nelson–Aalen estimator Non-linear least squares Nonlinear regression Nonparametric regression Nonparametric statistics Normalization (statistics)Numerical analysis Numerical integration Numerical smoothing and differentiation Objective Bayesian probability Observational study Occam's razor Official statistics One- and two-tailed tests One-part code One-to-one correspondence Opinion poll Optimal decision Optimal design Order statistic Ordinary least squares Orthogonal polynomials Outlier Outline of statistics Parameter estimation Parametric statistics Partial autocorrelation function Partial correlation Partial least squares regression Partition of sums of squares Pearson correlation coefficient Pearson product-moment correlation coefficient Percentile Permutation test Pie chart Pivotal quantity Plug-in principle Point estimation Poisson regression Polynomial regression Population (statistics)Population statistics Posterior probability Power (statistics)Power transform Prediction interval Principal component analysis Prior probability Probabilistic design Probabilistic model Probability distribution Probability theory Proportional hazards model Psychometrics Quality control Quantile regression Quasi-experiment Questionnaire Q–Q plot Radar chart Random assignment Randomization test Randomized controlled trial Randomized experiment Range (statistics)Rank correlation Ranking (statistics)Rao–Blackwell theorem Regression analysis Regression validation Reliability engineering Replication (statistics)Resampling (statistics)Response surface methodology Ridge regression Robust regression Robust statistics Run chart Sample median Sample size determination Sampling (statistics)Sampling distribution Scale parameter Scatter plot Scientific control Score test Seasonal adjustment Segmented regression Self-extracting archive Semiparametric regression Shape of the distribution Shape parameter Shapiro–Wilk test Sign test Simple linear regression Simultaneous equations model Skewness Social statistics Solomonoff's theory of inductive inference Spatial analysis Spearman's rank correlation coefficient Spectral density estimation Standard deviation Standard error Standard score Stationary process Statistic Statistical classification Statistical dispersion Statistical distance Statistical graphics Statistical hypothesis test Statistical inference Statistical model Statistical parameter Statistical population Statistical power Statistical process control Statistical theory Statistics Stem-and-leaf display Stepwise regression Stochastic approximation Stratified sampling Structural break Structural equation modeling Student's t-test Studentized residual Sufficient statistic Survey methodology Survival analysis Survival function Symbols System identification Tikhonov regularization Time domain Time series Tolerance interval Total least squares Trend estimation Truncation (statistics)U-statistic Uniformly most powerful test Unit vector normalization Universal code (data compression)Up-and-Down Designs V-statistic Van der Waerden test Variance Variance-stabilizing transformation Vector autoregression Violin plot Wald test Wavelet Weighted least squares Whittle likelihood Wilcoxon signed-rank test Winsorizing Yeo–Johnson transformation YouTube Z-test

17 concepts already in your glossary