Information theory

Definition

Information theory is the mathematical study of the quantification, storage, and communication of a particular type of mathematically defined information. The field was established and formalized by Claude Shannon in the 1940s, though early contributions were made in the 1920s through the works of Harry Nyquist and Ralph Hartley.

Related concepts

842 (compression algorithm)A-law algorithm ACM Computing Classification System A Mathematical Theory of Communication Abstract algebra Acta Astronautica Active networking Adaptive Huffman coding Adaptive coding Adaptive differential pulse-code modulation Alan Turing Alexander Lerner Alexey Lyapunov Alfred Radcliffe-Brown Algebra Algebraic code-excited linear prediction Algebraic geometry Algebraic number theory Algebraic topology Algorithm Algorithm design Algorithmic efficiency Algorithmic information theory Algorithmic probability Algorithms Allenna Leonard Analysis of algorithms Analytic geometry Analytic number theory Andrey Kolmogorov Anomaly detection Anthony Wilden Application security Applied mathematics Arithmetic Arithmetic coding Arithmetic geometry Art Artificial brain Artificial intelligence Asymmetric numeral systems Asymptotic equipartition property Attention Audio codec Augmented reality Automata theory Automated planning and scheduling Average bitrate Ban (unit)Base-2 logarithm Bayesian brain Bayesian inference Behavior informatics Bell Labs Bell System Technical Journal Bernoulli trial Binary entropy function Binary erasure channel Binary logarithm Binary symmetric channel Binding problem Bio-inspired computing Biocybernetics Bioinformatics Biomedical cybernetics Biorobotics Biosemiotics Bit Bit rate Black hole Black hole information paradox Block cipher Boltzmann constant Brain–computer interface Brenda McCowan Broadcast channel Brotli Brute force attack Buckminster Fuller Burrows–Wheeler transform Byte Byte-pair encoding Bzip2 Calculus Canonical Huffman code Catastrophe theory Category theory Causality Cellular computing Chain code Channel capacity Channel code Channel resolvability Charles François (systems scientist)Charles Sanders Peirce Charles Seife Chroma subsampling Cipher Ciphertext Circuit (computer science)Claude Bernard Claude Elwood Shannon Claude Shannon Cliff Joslyn Code-excited linear prediction Code (cryptography)Coding theory Coding tree unit Cognitive informatics Cognitive neuroscience Cognitive science Color space Combinatorics Common logarithm Communication Communication channel Communication protocol Communication source Communication theory Commutative algebra Compact disc Companding Compiler construction Complex analysis Complex geometry Compressed data structure Compressed suffix array Compression artifact Computability theory Computational biology Computational chemistry Computational complexity Computational complexity theory Computational engineering Computational geometry Computational intelligence Computational mathematics Computational neuroscience Computational physics Computational problem Computational social science Computational theory of mind Computer accessibility Computer algebra Computer animation Computer architecture Computer data storage Computer graphics Computer hardware Computer network Computer science Computer security Computer vision Computing Computing platform Concurrency (computer science)Concurrent computing Conditional entropy Conditional mutual information Conditional probability Connectionism Constant bitrate Constructor theory Context mixing Context tree weighting Control flow Control theory Conversation theory Convolution Covert channel Cross-entropy Cross-validation (statistics)Cryptanalysis Cryptanalysis of the Enigma Cryptographic algorithm Cryptographically secure pseudorandom number generator Cryptography Csiszar, I Cyber-physical system Cyberneticist Cybernetics Cybernetics in the Soviet Union Cybersemiotics Cyberwarfare Data compression Data compression symmetry Data mining Data processing Data storage Database Daubechies wavelet David A. Huffman David J. C. MacKay Deblocking filter Deciban Decimal digit Decision-making Decision support system Decision theory Decoding the Universe Deflate Delta encoding Delta modulation Dependability Detection theory Dictionary coder Differential entropy Differential equation Differential geometry Differential pulse-code modulation Differential topology Digital art Digital library Digital marketing Digital signal processing Digital subscriber line Diophantine geometry Directed information Discrete cosine transform Discrete geometry Discrete mathematics Discrete random variable Discrete sine transform Discrete wavelet transform Display resolution Distributed artificial intelligence Distributed computing Document management system Doede Nauta Domain-specific language Dynamic Markov compression Dynamic range E-commerce E (mathematical constant)Educational technology Electrical engineering Electronic design automation Electronic publishing Electronic voting Elementary algebra Elias gamma coding Embedded system Embedded zerotrees of wavelet transforms Emergence Encoding Encyclopedia of Mathematics Engineering cybernetics Engineering mathematics Enigma machine Enterprise information system Enterprise software Entropy (information theory)Entropy coding Entropy in thermodynamics and information theory Entropy rate Epistemology Equiprobable Ergodic theory Erich von Holst Ernst von Glasersfeld Error-correcting code Error-correction Error detection and correction Estimation theory Euclidean geometry European Mathematical Society Event processing Expected value Exponential-Golomb coding Extractor (mathematics)FM-index Fast Fourier transform Fault tolerance Fazlollah Reza Ferruccio Rossi-Landi Fibonacci coding Film frame Finite geometry Fisher information Form factor (design)Formal language Formal methods Formal science Foundations of mathematics Fourier transform Fractal compression Frame rate Francis Heylighen Francisco Varela Frederic Vester Free energy principle Functional analysis Fungible information Future of mathematics Gallager, R Gambling Gambling and information theory Game theory Gaussian channel Gaussian noise General topology Genevieve Bell Genome informatics Geoffrey Vickers Geographic information system Geometric topology Geometry Gerald Edelman Giulio Tononi Glossary of mathematical symbols Golomb coding Gordon Pask Gordon S. Brown Grammar-based code Grammatical Man Graph theory Graphics processing unit Green computing Gregory Bateson Group theory Hamming distance Hardware acceleration Hardware security Harmonic analysis Harry Nyquist Hartley (unit)Health informatics Heinz von Foerster History of information theory History of mathematics Homeostasis Homological algebra Homotopy theory Hubert Yockey Huffman coding Human-centered computing Human–computer interaction Humberto Maturana Hutter Prize Hypercomplex analysis I. A. Richards IEEE Igor Aleksander Image compression Image resolution Imaging system Incremental encoding Independent and identically distributed Independent identically distributed random variables Inductive probability Industrial process control Info-metrics Inforg Informal mathematics Information Information algebra Information asymmetry Information content Information entropy Information field theory Information fluctuation complexity Information geometry Information overload Information processing theory Information rate Information retrieval Information science Information security Information storage Information structure Information system Information technology Information theoretic security Information theory and measure theory Infosphere Integrated circuit Integrated development environment Integrated information theory Intelligence Intelligence (Information Gathering)Intelligence (information gathering)Interaction design Interlaced video International Journal of Computer Mathematics Internet Interpreter (computing)Intrusion detection system J. Willard Gibbs Jacob Bekenstein Jacque Fresco Jakob Johann von Uexküll James Gleick James Massey Jason Jixuan Hu Jay Wright Forrester Jennifer Wilby John Horgan (journalist)John N. Warfield John R. Pierce Joint entropy Karhunen–Loève theorem Karl J. Friston Kevin Warwick Key (cryptography)Knot theory Knowledge representation and reasoning Kolmogorov complexity Kullback–Leibler divergence LHA (file format)LZ4 (compression algorithm)LZ77 and LZ78 LZFSE LZJB LZMA LZRW LZWL LZX Lapped transform Large numbers Latency (audio)Laurance Doyle Learning Lempel–Ziv–Oberhumer Lempel–Ziv–Stac Lempel–Ziv–Storer–Szymanski Lempel–Ziv–Welch Levenshtein coding Library (computing)Likelihood-ratio test Limiting density of discrete points Line spectral pairs Linear algebra Linear predictive coding Linguistics List of computer size categories List of films about mathematicians List of mathematicians List of unsolved problems in information theory Lists of mathematics topics Log area ratio Logic in computer science Logic of information Lossless compression Lossless data compression Lossy compression Lossy data compression Ludwig Boltzmann Ludwig von Bertalanffy Machine learning Macroblock Maleyka Abbaszadeh Management cybernetics Manfred Clynes Margaret Boden Margaret Mead Marian Mazur Mark Adler Mathematical analysis Mathematical and theoretical biology Mathematical chemistry Mathematical economics Mathematical finance Mathematical geology Mathematical linguistics Mathematical logic Mathematical optimization Mathematical physics Mathematical psychology Mathematical sociology Mathematical software Mathematical statistics Mathematicians Mathematics Mathematics and art Mathematics education Measure (mathematics)Medical cybernetics Memoryless Metric (mathematics)Middleware Min-entropy Mind Mind uploading Minimum description length Minimum message length Mixed reality Mobile computing Model of computation Modeling language Modified Huffman coding Modified discrete cosine transform Molecular dynamics Motion compensation Motion estimation Motion vector Move-to-front transform Multi-agent system Multi-task learning Multilinear algebra Multimedia database Multinomial distribution Multiprocessing Multithreading (computer architecture)Music Mutual information N. Katherine Hayles Nat (unit)Natalia Bekhtereva Natural computing Natural language processing Natural logarithm Network architecture Network coding Network performance Network scheduler Network security Network service Networking hardware Neural computation Neurobiology Neurocomputing Neuroinformatics Neuroscience Niklas Luhmann Noisy-channel coding theorem Norbert Wiener Number Number theory Numerical analysis Nyquist–Shannon sampling theorem One-time pad Open source Operating system Operations research Order theory Outer space Outline of computer science PAQ Parallel computing Pattern recognition Peak signal-to-noise ratio Pearson's chi-squared test Perception Peripheral Perplexity Petro Grigorenko Phil Katz Philosophy of artificial intelligence Philosophy of information Philosophy of mathematics Photograph manipulation Physical computing Physics Pixel Plagiarism detection Plaintext Pointwise mutual information Posterior probability Pragmatic information Prediction by partial matching Prefix code Printed circuit board Prior probability Probability Probability distribution Probability mass function Probability theory Process (computing)Processor (computing)Programming language Programming language theory Programming paradigm Programming team Programming tool Pseudorandom number generator Psychoacoustics Public-key cryptography Pyramid (image processing)Qian Xuesen Quantification (science)Quantities of information Quantization (image processing)Quantization (signal processing)Quantum computing Quantum information science Ralph Hartley Random seed Randomized algorithm Range coding Ranulph Glanville Rate–distortion theory Re-Pair Real-time computing Real-time control Real analysis Reason Receiver (information theory)Recreational mathematics Redundancy (information theory)Reflection seismology Reinforcement learning Relative entropy Relay channel Remote control Rendering (computer graphics)Requirements analysis Robert K. Logan Robert McEliece Robert Spekkens Robert Trappl Rolf Landauer Run-length encoding Rényi entropy Sampling (signal processing)Scientific American Search for extraterrestrial intelligence Second-order cybernetics Security hacker Security service (telecommunication)Self-information Semantics (computer science)Semiotics Sequence of symbols Sequitur algorithm Sergei P. Kurdyumov Set partitioning in hierarchical trees Set theory Shannon's source coding theorem Shannon (unit)Shannon coding Shannon–Fano coding Shannon–Fano–Elias coding Shannon–Hartley law Shannon–Hartley theorem Signal Signal (electrical engineering)Signal noise Signal processing Silence compression Smallest grammar problem Snappy (compression)Social computing Social influence Social information processing Social software Sociocybernetics Software configuration management Software construction Software deployment Software design Software development Software development process Software engineering Software framework Software maintenance Software quality Software repository Solid modeling Sound quality Source coding Source coding theorem Space Speech coding Sports gambling Stafford Beer Standard test image State (computer science)Stationary process Statistical independence Statistical inference Statistics Stochastic computing Stochastic process Stream processing Stuart Kauffman Stuart Umpleby Sub-band coding Supervised learning Symmetric-key algorithm Symmetric function Synergetics (Haken)System on a chip Systems biology Systems science Systems theory Talcott Parsons Te Sun Han Telecommunications Texture compression Theoretical computer science Theory of computation Thermal physics Thermodynamics Thomas M. Cover Timeline of information theory Timeline of mathematics Topology Transform coding Transistor Triangle inequality Tsallis entropy Tunstall coding Type theory Ubiquitous computing Ulla Mitzdorf Ultra (cryptography)Umberto Eco Unary coding Unicity distance Units of measurement Universal algebra Universal code (data compression)University of Illinois Press Unsupervised learning Urbana, Illinois Valentin Turchin Valentino Braitenberg Vannevar Bush Variable bitrate Variety (cybernetics)Venona project Very-large-scale integration Victory in Europe Day Video Video codec Video compression picture types Video game Video quality Virtual machine Virtual reality Virtual world Visualization (graphics)Voyager program W. Ross Ashby Walter Bradford Cannon Walter Pitts Warped linear predictive coding Warren Sturgis McCulloch Warren Weaver Wavelet transform Wiley-Interscience William Grey Walter Winfried Nöth Wireless sensor network Word processor World Scientific Publishing World Wide Web ZIP (file format)Zstd Μ-law algorithm

69 concepts already in your glossary