blanketglossary

Self-supervised learning

Definition

Self-supervised learning (SSL) is a paradigm in machine learning where a model is trained on a task using the data itself to generate supervisory signals, rather than relying on externally-provided labels. In the context of neural networks, self-supervised learning aims to leverage inherent structures or relationships within the input data to create meaningful training signals. SSL tasks are designed so that solving them requires capturing essential features or relationships in the data. The input data is typically augmented or transformed in a way that creates pairs of related samples, where one sample serves as the input, and the other is used to formulate the supervisory signal. This augmentation can involve introducing noise, cropping, rotation, or other transformations. Self-supervised learning more closely imitates the way humans learn to classify objects.

Related concepts

15.ai AAAI Conference on Artificial Intelligence AI agent AI alignment AI anthropomorphism AI boom AI bubble AI data center AI effect AI literacy AI nationalism AI safety AI slop AI takeover AI veganism AI winter Action selection Activation function Active learning (machine learning)Adaptive learning Adobe Firefly Adversarial machine learning Agent2Agent Aidan Gomez Alan Turing AlexNet Alex Graves (computer scientist)Alex Krizhevsky Allen Newell AlphaFold AlphaGo AlphaZero Andrej Karpathy Andrew Ng Anomaly detection Applications of artificial intelligence Apprenticeship learning Artificial Intelligence Act Artificial Intelligence Cold War Artificial general intelligence Artificial human companion Artificial intelligence Artificial intelligence and elections Artificial intelligence arms race Artificial intelligence in architecture Artificial intelligence in education Artificial intelligence in fiction Artificial intelligence in healthcare Artificial intelligence in mental health Artificial intelligence in video games Artificial intelligence visual art Artificial neural network Artificial superintelligence Ashish Vaswani Association rule learning Attention (machine learning)Audio signal processing Aurora (text-to-image model)AutoGPT Autoencoder Autoencoders Automated machine learning Automated reasoning Automated theorem proving Autoregressive model BERT (language model)BIRCH BLOOM (language model)Backpropagation Batch learning Batch normalization Bayesian network Bernard Widrow Bias–variance tradeoff Binary classification Boltzmann machine Boosting (machine learning)Bootstrap aggregating CURE algorithm Canonical correlation Chatbot psychosis Chinchilla (language model)Christopher D. Manning Claude (language model)Claude Shannon Cliff Shaw Cluster analysis Coefficient of determination Competition in artificial intelligence Computational learning theory Computer vision Concept drift Conditional random field Conference on Neural Information Processing Systems Confusion matrix Conjugate gradient method Contrastive Language-Image Pre-training Convolution Convolutional neural network Convolutional neural networks Cosine similarity Crowdsourcing Curriculum learning DALL-E DBRX DBSCAN Daniel Kokotajlo (researcher)Data augmentation Data cleaning Data mining David Silver (computer scientist)Decision tree learning DeepDream DeepSeek (chatbot)Deep learning Deep learning speech synthesis Demis Hassabis Density estimation Differentiable neural computer Diffusion model Diffusion process Dimensionality reduction Domain knowledge Double descent Dream Machine (text-to-video model)ECML PKDD Echo state network Electrochemical RAM ElevenLabs Empirical risk minimization Ensemble learning Environmental impact of artificial intelligence Ethics of artificial intelligence Expectation–maximization algorithm Explainable artificial intelligence Facebook Facial recognition system Factor analysis Feature (machine learning)Feature engineering Feature learning Feedforward neural network Fei-Fei Li Flux (text-to-image model)Frank Rosenblatt François Chollet Fuzzy clustering GPT-3 GPT Image Gated recurrent unit Gating mechanism Gemini (chatbot)Gemini (language model)Gemma (language model)Generative AI Generative adversarial network Generative engine optimization Generative model Generative pre-trained transformer Genie (world model)Geoffrey Hinton GloVe Glossary of artificial intelligence Google Gradient descent Grammar induction Graph neural network Graphical model Grok (chatbot)Hallucination (artificial intelligence)Handwriting recognition Herbert A. Simon Hidden Markov model Hierarchical clustering Highway network History of artificial intelligence Huawei PanGu Human-in-the-loop Human image synthesis Humanity's Last Exam Hyperparameter (machine learning)IBM Granite IBM Watson IBM Watsonx Ian Goodfellow Ideogram (text-to-image model)Ilya Sutskever ImageNet Imagen (text-to-image model)Imitation learning Independent component analysis Intelligent agent International Conference on Learning Representations International Conference on Machine Learning International Joint Conference on Artificial Intelligence Isolation forest James Goodnight Jan Leike John Hopfield John McCarthy (computer scientist)John Schulman John von Neumann Joseph Weizenbaum Journal of Machine Learning Research Jürgen Schmidhuber K-means clustering K-nearest neighbors algorithm Kernel machines Kling AI Kunihiko Fukushima LaMDA Labeled data Language model Large language model Latent diffusion model Latent space LeNet Learning curve (machine learning)Learning to rank Lethal autonomous weapon Linear discriminant analysis Linear regression List of artificial intelligence companies List of artificial intelligence projects List of datasets for machine-learning research List of datasets in computer vision and image processing Llama (language model)Local outlier factor Logistic regression Long short-term memory Loss function Loss functions for classification Lotfi A. Zadeh Machine Learning (journal)Machine learning Mamba (deep learning architecture)Marvin Minsky Mean shift Mean squared error Mechanistic interpretability Memtransistor Meta-learning (computer science)Midjourney MiniMax (company)Model Context Protocol MuZero Multi-agent reinforcement learning Multilayer perceptron Multimodal learning Music and artificial intelligence Mustafa Suleyman Naive Bayes classifier Nathaniel Rochester (computer scientist)Natural language processing Neural Turing machine Neural field Neural machine translation Neural network (machine learning)Neural radiance field Neuro-symbolic AI Neuromorphic engineering Noam Shazeer Non-negative matrix factorization Normalization (machine learning)OPTICS algorithm Oasis (Minecraft clone)Occam learning Oliver Selfridge Online machine learning Ontology learning OpenAI OpenAI Five Optical character recognition Oriol Vinyals Outline of machine learning Overfitting PaLM Parameter Paul Werbos Perceptron Physics-informed neural networks Policy gradient method Polysemy Precautionary principle Principal component analysis Probably approximately correct learning Project Debater Prompt engineering Proper generalized decomposition Q-learning Quantum machine learning Quasi-Newton method Quoc V. Le Qwen Random forest Random sample consensus Reasoning model Receiver operating characteristic Recraft Rectifier (neural networks)Recurrent neural network Recursive self-improvement Reflection (artificial intelligence)Regression analysis Regularization (mathematics)Regulation of artificial intelligence Regulation of artificial intelligence in the United States Reinforcement learning Reinforcement learning from human feedback Relevance vector machine Reservoir computing Residual neural network Restricted Boltzmann machine Retrieval-augmented generation Riffusion Robot control Rule-based machine learning Runway (company)Seedance 2.0 Self-driving car Self-organizing map Self-play (reinforcement learning technique)Semantic analysis (machine learning)Semi-supervised learning Seppo Linnainmaa Seq2seq Seymour Papert Shun'ichi Amari Sigmoid function Softmax function Sora (text-to-video model)Sparse dictionary learning Speech recognition Spiking neural network Stable Diffusion State–action–reward–state–action Statistical classification Statistical learning theory Stephen Grossberg Stochastic gradient descent Structured prediction Suno (platform)Supervised learning Support vector machine Symbolic artificial intelligence T-distributed stochastic neighbor embedding T5 (language model)Takeo Kanade Temporal difference learning Text-to-image model Text-to-video model Timeline of artificial intelligence Topological deep learning Training, validation, and test data sets Transfer learning Transformer (deep learning)Transformer (deep learning architecture)U-Net Udio Uncanny valley Unsupervised learning Vapnik–Chervonenkis theory Variational autoencoder Veo (text-to-video model)Vibe coding Virtual politician Vision transformer Walter Pitts Warren Sturgis McCulloch WaveNet Weak artificial intelligence Weight initialization Whisper (speech recognition system)Word2vec Word embedding Word sense disambiguation Workplace impact of artificial intelligence World model (artificial intelligence)Xiaomi MiMo Yann LeCun Yarowsky algorithm Yoshua Bengio

17 concepts already in your glossary