blanketglossary

Double descent

Definition

Double descent in statistics and machine learning is the phenomenon where a model's error rate on the test set initially decreases with the number of parameters, then peaks, then decreases again. This phenomenon has been considered surprising, as it contradicts assumptions about overfitting in classical machine learning.

Related concepts

15.ai AAAI Conference on Artificial Intelligence AI agent AI alignment AI anthropomorphism AI boom AI bubble AI data center AI effect AI literacy AI nationalism AI safety AI slop AI takeover AI veganism AI winter Accelerated failure time model Action selection Activation function Active learning (machine learning)Actuarial science Adaptive clinical trial Adobe Firefly Adversarial machine learning Agent2Agent Aidan Gomez Akaike information criterion Alan Turing AlexNet Alex Graves (computer scientist)Alex Krizhevsky Algorithmic probability Allen Newell AlphaFold AlphaGo AlphaZero Analysis of covariance Analysis of variance Anderson–Darling test Andrej Karpathy Andrew Ng Anomaly detection Anscombe transform Applications of artificial intelligence Apprenticeship learning Arithmetic mean Arithmetic–geometric mean Artificial Intelligence Act Artificial Intelligence Cold War Artificial general intelligence Artificial human companion Artificial intelligence Artificial intelligence and elections Artificial intelligence arms race Artificial intelligence in architecture Artificial intelligence in education Artificial intelligence in fiction Artificial intelligence in healthcare Artificial intelligence in mental health Artificial intelligence in video games Artificial intelligence visual art Artificial neural network Artificial superintelligence Ashish Vaswani Association rule learning Asymptotic theory (statistics)Attention (machine learning)Aurora (text-to-image model)AutoGPT Autocorrelation Autoencoder Automated machine learning Automated reasoning Automated theorem proving Autoregressive conditional heteroskedasticity Autoregressive model Autoregressive–moving-average model Average absolute deviation BERT (language model)BIRCH BLOOM (language model)Backpropagation Bar chart Batch learning Batch normalization Bayes estimator Bayes factor Bayesian inference Bayesian information criterion Bayesian linear regression Bayesian network Bayesian probability Bernard Widrow Bias of an estimator Bias–variance tradeoff Binomial regression Bioinformatics Biostatistics Biplot Blocking (statistics)Boltzmann machine Boosting (machine learning)Bootstrap aggregating Bootstrapping (statistics)Box plot Box–Cox transformation Box–Jenkins method Breusch–Godfrey test CURE algorithm Canonical correlation Cartography Categorical variable Census Central limit theorem Central tendency Chatbot psychosis Chemometrics Chi-squared test Chinchilla (language model)Christopher D. Manning Claude (language model)Claude Shannon Cliff Shaw Clinical study design Clinical trial Cluster analysis Cluster sampling Cochran–Mantel–Haenszel statistics Coefficient of determination Coefficient of variation Cohen's kappa Cohort study Cointegration Communications on Pure and Applied Mathematics Competition in artificial intelligence Completeness (statistics)Computational learning theory Computer vision Conditional random field Conference on Neural Information Processing Systems Confidence interval Confounding Confusion matrix Conjugate gradient method Contingency table Continuous probability distribution Contraharmonic mean Control chart Convolution Convolutional neural network Correlation and dependence Correlogram Count data Credible interval Crime statistics Cross-correlation Cross-sectional study Cross-validation (statistics)Crowdsourcing Cubic mean Curriculum learning DALL-E DBRX DBSCAN Daniel Kokotajlo (researcher)Data augmentation Data cleaning Data collection Data mining Data preprocessing Data transformation (statistics)David Silver (computer scientist)Decision tree learning Decomposition of time series DeepDream DeepSeek (chatbot)Deep learning Deep learning speech synthesis Degrees of freedom (statistics)Demis Hassabis Demographic statistics Density estimation Descriptive statistics Design of experiments Detrending Dickey–Fuller test Differencing Differentiable neural computer Diffusion model Diffusion process Dimensionality reduction Divergence (statistics)Dream Machine (text-to-video model)Durbin–Watson statistic ECML PKDD Echo state network Econometrics Effect size Effective dimension Efficiency (statistics)Electrochemical RAM ElevenLabs Elliptical distribution Empirical distribution function Empirical risk minimization Engineering statistics Ensemble learning Environmental impact of artificial intelligence Environmental statistics Epidemiology Errors and residuals Estimating equations Ethics of artificial intelligence Expectation–maximization algorithm Experiment Explainable artificial intelligence Exponential family Exponential smoothing F-test Facial recognition system Factor analysis Factorial experiment Failure rate Fan chart (statistics)Feature engineering Feature learning Feature scaling Feedforward neural network Fei-Fei Li First-hitting-time model Fisher transformation Flux (text-to-image model)Forest plot Fourier analysis Frank Rosenblatt François Chollet Frequency distribution Frequency domain Frequentist inference Friedman test Fuzzy clustering G-test GPT Image Gated recurrent unit Gating mechanism Gaussian process Gemini (chatbot)Gemini (language model)Gemma (language model)General linear model Generalized linear model Generalized mean Generative AI Generative adversarial network Generative engine optimization Generative model Generative pre-trained transformer Genie (world model)Geoffrey Hinton Geographic information system Geometric mean Geostatistics GloVe Glossary of artificial intelligence Goodness of fit Gradient descent Grammar induction Granger causality Graph neural network Graphical model Grok (chatbot)Grokking (machine learning)Grouped data Hallucination (artificial intelligence)Handwriting recognition Harmonic mean Heatmap Heinz mean Herbert A. Simon Heronian mean Hessian matrix Hidden Markov model Hierarchical clustering Highway network Histogram History of artificial intelligence Hodges–Lehmann estimator Homoscedasticity and heteroscedasticity Huawei PanGu Human-in-the-loop Human image synthesis Humanity's Last Exam Hyperparameter (machine learning)IBM Granite IBM Watson IBM Watsonx Ian Goodfellow Ideogram (text-to-image model)Ilya Sutskever Imagen (text-to-image model)Imitation learning Independent component analysis Index of dispersion Intelligent agent Interaction (statistics)International Conference on Learning Representations International Conference on Machine Learning International Joint Conference on Artificial Intelligence Interquartile range Interval estimation Isolation forest Isotonic regression Isotropy Jackknife resampling James Goodnight Jan Leike Jarque–Bera test Johansen test John Hopfield John McCarthy (computer scientist)John Schulman John von Neumann Jonckheere's trend test Joseph Weizenbaum Journal of Machine Learning Research Jurimetrics Jürgen Schmidhuber K-means clustering K-nearest neighbors algorithm Kaplan–Meier estimator Kendall rank correlation coefficient Kernel machines Kinship Kling AI Kolmogorov–Smirnov test Kriging Kruskal–Wallis test Kunihiko Fukushima Kurtosis L-moment LaMDA Language model Large language model Latent diffusion model LeNet Learning curve (machine learning)Learning to rank Least-squares spectral analysis Lehmann–Scheffé theorem Lehmer mean Lethal autonomous weapon Likelihood-ratio test Likelihood function Likelihood interval Lilliefors test Line chart Linear discriminant analysis Linear regression List of artificial intelligence companies List of artificial intelligence projects List of datasets for machine-learning research List of datasets in computer vision and image processing List of fields of application of statistics List of statistical tests List of statistics articles Ljung–Box test Llama (language model)Local outlier factor Location parameter Location–scale family Log-rank test Log transformation Logistic regression Long short-term memory Loss function Loss functions for classification Lotfi A. Zadeh Lp space M-estimator Machine Learning (journal)Machine learning Mamba (deep learning architecture)Mann–Whitney U test Marvin Minsky Maximum a posteriori estimation Maximum likelihood McNemar's test Mean Mean shift Mechanistic interpretability Median Median-unbiased estimator Medical statistics Memtransistor Meta-learning (computer science)Method of moments (statistics)Methods engineering Midjourney MiniMax (company)Minimum-variance unbiased estimator Minimum distance estimation Min–max normalization Missing data Mixed model Mode (statistics)Model Context Protocol Model selection Model specification Moment (mathematics)Monotone likelihood ratio MuZero Multi-agent reinforcement learning Multilayer perceptron Multimodal learning Multiple comparisons Multivariate adaptive regression splines Multivariate analysis of variance Multivariate distribution Multivariate normal distribution Multivariate statistics Music and artificial intelligence Mustafa Suleyman Naive Bayes classifier Nathaniel Rochester (computer scientist)National accounts Natural experiment Nelson–Aalen estimator Neural Turing machine Neural field Neural machine translation Neural network (machine learning)Neural radiance field Neuro-symbolic AI Neuromorphic engineering Noam Shazeer Non-negative matrix factorization Nonlinear regression Nonparametric regression Nonparametric statistics Normalization (machine learning)Normalization (statistics)OPTICS algorithm Oasis (Minecraft clone)Observational study Occam learning Official statistics Oliver Selfridge One- and two-tailed tests Online machine learning Ontology learning OpenAI Five Opinion poll Optical character recognition Optimal decision Optimal design Order statistic Ordinary least squares Oriol Vinyals Outlier Outline of machine learning Outline of statistics Overfitting PaLM Parameter Parametric statistics Partial autocorrelation function Partial correlation Partition of sums of squares Paul Werbos Pearson correlation coefficient Pearson product-moment correlation coefficient Percentile Perceptron Permutation test Physics-informed neural networks Pie chart Pivotal quantity Plug-in principle Point estimation Poisson regression Policy gradient method Population (statistics)Population statistics Posterior probability Power (statistics)Power transform Precautionary principle Prediction interval Principal component analysis Prior probability Probabilistic design Probability distribution Probably approximately correct learning Project Debater Prompt engineering Proper generalized decomposition Proportional hazards model Psychometrics Q-learning Quality control Quantum machine learning Quasi-Newton method Quasi-experiment Questionnaire Quoc V. Le Qwen Q–Q plot Radar chart Random assignment Random forest Random sample consensus Randomization test Randomized controlled trial Randomized experiment Range (statistics)Rank correlation Ranking (statistics)Rao–Blackwell theorem Reasoning model Receiver operating characteristic Recraft Rectifier (neural networks)Recurrent neural network Recursive self-improvement Reflection (artificial intelligence)Regression analysis Regression validation Regularization (mathematics)Regulation of artificial intelligence Regulation of artificial intelligence in the United States Reinforcement learning Reinforcement learning from human feedback Relevance vector machine Reliability engineering Replica trick Replication (statistics)Resampling (statistics)Reservoir computing Residual neural network Restricted Boltzmann machine Retrieval-augmented generation Riffusion Robot control Robust regression Robust statistics Rule-based machine learning Run chart Runway (company)SIAM Journal on Mathematics of Data Science Sample median Sample size determination Sampling (statistics)Sampling distribution Scale parameter Scatter plot Scientific control Score test Seasonal adjustment Seedance 2.0 Self-driving car Self-organizing map Self-play (reinforcement learning technique)Self-supervised learning Semantic analysis (machine learning)Semi-supervised learning Semiparametric regression Seppo Linnainmaa Seq2seq Seymour Papert Shape of the distribution Shape parameter Shapiro–Wilk test Shun'ichi Amari Sigmoid function Sign test Simple linear regression Simultaneous equations model Skewness Social statistics Softmax function Sora (text-to-video model)Sparse dictionary learning Spatial analysis Spearman's rank correlation coefficient Spectral density estimation Speech recognition Spiking neural network Stable Diffusion Standard deviation Standard error Standard score State–action–reward–state–action Stationary process Statistic Statistical classification Statistical dispersion Statistical distance Statistical graphics Statistical hypothesis test Statistical inference Statistical learning theory Statistical model Statistical parameter Statistical population Statistical power Statistical process control Statistical theory Statistics Stem-and-leaf display Stephen Grossberg Stochastic approximation Stochastic gradient descent Stratified sampling Structural break Structural equation modeling Structured prediction Stuart Geman Student's t-test Sufficient statistic Suno (platform)Supervised learning Support vector machine Survey methodology Survival analysis Survival function Symbolic artificial intelligence System identification T-distributed stochastic neighbor embedding T5 (language model)Takeo Kanade Temporal difference learning Test set Text-to-image model Text-to-video model Thermodynamic limit Time domain Time series Timeline of artificial intelligence Tolerance interval Topological deep learning Training, validation, and test data sets Transformer (deep learning)Transformer (deep learning architecture)Trend estimation Truncation (statistics)U-Net U-statistic Udio Uncanny valley Uniformly most powerful test Unit vector normalization Unsupervised learning Up-and-Down Designs V-statistic Van der Waerden test Vapnik–Chervonenkis theory Variance Variance-stabilizing transformation Variational autoencoder Vector autoregression Veo (text-to-video model)Vibe coding Violin plot Virtual politician Vision transformer Wald test Walter Pitts Warren Sturgis McCulloch WaveNet Wavelet Weak artificial intelligence Weight initialization Whisper (speech recognition system)Whittle likelihood Wilcoxon signed-rank test Winsorizing Word2vec Word embedding Workplace impact of artificial intelligence World model (artificial intelligence)Xiaomi MiMo Yann LeCun Yeo–Johnson transformation Yoshua Bengio Z-test

27 concepts already in your glossary