Probability distribution
A probability distribution is a mathematical function that assigns probabilities to the possible outcomes of a random variable, quantifying the likelihood of each outcome occurring in a probabilistic experiment.[1] For discrete random variables, which take on countable values such as integers, the distribution is described by a probability mass function p(x) where p(x) \geq 0 for all x and \sum p(x) = 1 over all possible values, ensuring the total probability sums to unity.[1] In contrast, for continuous random variables, which can take any value in a continuum, the distribution is given by a probability density function f(x) where f(x) \geq 0 and \int_{-\infty}^{\infty} f(x) \, dx = 1, with probabilities computed as integrals over intervals rather than at single points.[1] Probability distributions form the foundation of statistical inference and modeling uncertainty across diverse fields, enabling predictions and decision-making under randomness.[2] Common discrete distributions include the binomial distribution, which models the number of successes in a fixed number of independent Bernoulli trials with success probability p, having mean np and variance np(1-p), often applied to scenarios like quality control or voting outcomes.[3] The Poisson distribution describes the count of rare events in a fixed interval, with parameter \mu (mean rate), mean \mu, and variance \mu, widely used in queueing theory, reliability engineering, and modeling arrivals such as customer traffic or accidents.[3] Among continuous distributions, the normal distribution (or Gaussian) is paramount due to the central limit theorem, which states that the sum of many independent random variables approximates a normal distribution regardless of their original forms; it features a bell-shaped density f(x) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}, with mean \mu and variance \sigma^2, and applies to phenomena like measurement errors, stock returns, and biological traits.[3] The exponential distribution models the time between independent events in a Poisson process, with density f(x) = \lambda e^{-\lambda x} for x \geq 0, rate parameter \lambda, mean $1/\lambda, and variance $1/\lambda^2, commonly used for lifetimes, service times, and inter-arrival durations in telecommunications or manufacturing.[3][4] Other notable distributions, such as the uniform (equal probability over an interval) and gamma (generalizing exponential for sums of waiting times), further extend modeling capabilities in simulations, risk assessment, and scientific data analysis.[3]Fundamentals
Introduction
A probability distribution is a mathematical function that describes the possible outcomes of a random variable and assigns probabilities to those outcomes, providing a complete characterization of the uncertainty inherent in random processes.[1] This framework allows for the quantification of likelihoods, enabling predictions about the behavior of systems influenced by chance, from simple experiments to complex natural phenomena.[5] The origins of probability distributions trace back to the 17th century, when mathematicians Blaise Pascal and Pierre de Fermat exchanged correspondence in 1654 to resolve problems arising from games of chance, such as dividing stakes in interrupted dice games.[6] Their work laid the groundwork for systematic approaches to calculating odds and expectations in gambling scenarios, marking the birth of probability as a mathematical discipline.[6] The field was later formalized in a rigorous axiomatic framework by Andrey Kolmogorov in his 1933 monograph Foundations of the Theory of Probability, which defined probability measures on abstract spaces and unified disparate ideas into a coherent theory.[7] Probability distributions play a central role across diverse fields by modeling randomness in real-world data and processes. In statistics, they underpin inference, hypothesis testing, and estimation techniques essential for drawing conclusions from samples.[1] In physics, distributions describe particle behaviors and thermodynamic systems, such as the Maxwell-Boltzmann distribution for molecular speeds.[5] Finance relies on them for risk assessment and option pricing, as seen in models like the Black-Scholes framework that assume log-normal asset returns.[8] In machine learning, probabilistic distributions form the basis for algorithms in supervised and unsupervised learning, facilitating tasks like generative modeling and uncertainty quantification.[9] Distributions are broadly classified into discrete and continuous types, reflecting the nature of the random variable's possible values. Discrete distributions apply to scenarios with countable outcomes, such as the number of heads in a series of coin flips, where each specific count has a nonzero probability.[1] Continuous distributions, in contrast, handle uncountable outcomes over intervals, like human heights measured in real numbers, where probabilities are assigned to ranges rather than exact points.[1] The cumulative distribution function serves as a fundamental tool for unifying these cases, capturing the probability that the random variable falls below a given value.[1]Definition
In probability theory, a random variable is a measurable function X: \Omega \to \mathbb{R} defined on a probability space (\Omega, \mathcal{F}, P), where \Omega is the sample space, \mathcal{F} is a \sigma-algebra of events, and P is a probability measure, such that for every real number a, the set \{\omega \in \Omega : X(\omega) < a\} belongs to \mathcal{F}.[10] This measurability ensures that probabilities of events defined in terms of X can be consistently assigned. The probability distribution of a random variable X is the induced probability measure \mu on the Borel \sigma-algebra of \mathbb{R}, defined by \mu(B) = P(X^{-1}(B)) for every Borel set B \subseteq \mathbb{R}, which assigns probabilities to the possible outcomes or ranges of X.[10] This distribution satisfies Kolmogorov's axioms: non-negativity, meaning \mu(B) \geq 0 for all Borel sets B; additivity for disjoint countable unions, \mu\left(\bigcup_{n=1}^\infty B_n\right) = \sum_{n=1}^\infty \mu(B_n) if the B_n are disjoint; and normalization, \mu(\mathbb{R}) = 1.[10] In general, the probability distribution describes the law of X, where for discrete random variables it is given by the probabilities P(X = x) at each point x in the support, and for continuous random variables by a density function f such that probabilities are obtained via integration over intervals.[10] The total probability over the support satisfies \int P(X \in \, dx) = 1, ensuring the measure is normalized across all possible outcomes.[10]Terminology
A random variable is a function that assigns a real number to each outcome in a probability space, mapping the sample space to the real numbers.[11] Random variables are classified as discrete if their possible values form a countable set, such as the integers, or continuous if they can take any value in a continuous interval of the real numbers.[12][13] The support of a probability distribution is the smallest closed set of points such that the probability of the random variable taking a value outside this set is zero, representing the set where the distribution assigns positive probability.[14][15] Parameters of a probability distribution are numerical characteristics that define its shape and location, such as the mean and variance, which respectively indicate the central tendency and spread of the distribution.[16][17] For discrete random variables, the probability mass function (PMF) is the function that assigns to each possible value the probability that the random variable equals that value.[18][19] For continuous random variables, the probability density function (PDF) is a non-negative function whose integral over any interval gives the probability that the random variable falls within that interval; such distributions are absolutely continuous with respect to the Lebesgue measure, meaning the cumulative distribution function is the integral of the PDF.[20] The expectation, also known as the mean, of a random variable is the weighted average of its possible values, where the weights are the probabilities.[21][22] The variance measures the expected squared deviation of the random variable from its mean, quantifying the dispersion of the distribution.[23][24] Two random variables are independent if the occurrence of one does not affect the probability distribution of the other, formally meaning that the joint probability is the product of the marginal probabilities for all pairs of values.[25][26] The cumulative distribution function serves as a unifying concept that defines the probability that the random variable is less than or equal to a given value, applicable to both discrete and continuous cases.[13]Cumulative Distribution Function
Properties
The cumulative distribution function (CDF) of a random variable X, denoted F_X(x), is defined as F_X(x) = P(X \leq x) for x \in \mathbb{R}, mapping to the interval [0, 1].[27] This function encapsulates the probability that X takes a value less than or equal to x, providing a complete probabilistic description of the distribution.[28] Key properties of the CDF include non-decreasing monotonicity, right-continuity, and specific boundary behaviors. Specifically, F_X(x) is non-decreasing, meaning that if x_1 < x_2, then F_X(x_1) \leq F_X(x_2), reflecting the accumulation of probability as x increases.[27] It is right-continuous at every point, so \lim_{y \to x^+} F_X(y) = F_X(x).[28] The limits satisfy \lim_{x \to -\infty} F_X(x) = 0 and \lim_{x \to \infty} F_X(x) = 1, ensuring the total probability sums to 1 over the entire real line.[27] For discrete distributions, the CDF exhibits jumps at points where the random variable has positive probability mass, with the jump size equal to P(X = x); in contrast, for continuous distributions, the CDF is continuous everywhere.[28] The explicit form of the CDF depends on whether the distribution is discrete or continuous. For a discrete random variable with probability mass function p_k = P(X = k), the CDF is given by F_X(x) = \sum_{k \leq x} p_k, summing the probabilities up to x.[27] For a continuous random variable with probability density function f(t), it is F_X(x) = \int_{-\infty}^x f(t) \, dt, representing the integral of the density from negative infinity to x.[27] The CDF uniquely determines the probability distribution of X, as any two random variables with the same CDF induce identical probability measures.[29] This uniqueness theorem ensures that the CDF serves as the canonical representation for characterizing distributions in probability theory.[30]Relation to Other Functions
The cumulative distribution function (CDF) F_X(x) = P(X \leq x) serves as a foundational tool for deriving other key functions that characterize the distribution of a random variable X. For continuous random variables, the probability density function (PDF) f_X(x) is directly related to the CDF through differentiation, where f_X(x) = \frac{d}{dx} F_X(x), assuming the CDF is absolutely continuous and differentiable.[27] This relationship implies that the CDF can be recovered by integrating the PDF: F_X(x) = \int_{-\infty}^x f_X(t) \, dt.[31] For discrete random variables, the probability mass function (PMF) p_X(x) = P(X = x) is obtained via finite differences of the CDF, specifically p_X(x) = F_X(x) - \lim_{t \uparrow x} F_X(t), which for support on integers simplifies to p_X(x) = F_X(x) - F_X(x-1).[19] Conversely, the CDF is the cumulative sum of the PMF: F_X(x) = \sum_{t \leq x} p_X(t).[27] Another important function derived from the CDF is the quantile function, defined as the generalized inverse Q_X(p) = F_X^{-1}(p) = \inf \{ x : F_X(x) \geq p \} for p \in (0,1), which provides the value below which a proportion p of the distribution lies.[32] This function is particularly useful for computing percentiles and in simulation methods, such as inverse transform sampling, where uniform random variables are transformed to follow the target distribution. The quantile function inverts the CDF in the sense that F_X(Q_X(p)) \geq p and Q_X(F_X(x)) \leq x, with equality holding under continuity and strict monotonicity.[32] The survival function, often denoted S_X(x) = 1 - F_X(x), represents the probability P(X > x) and is widely used in reliability engineering and survival analysis to model the probability of an event not occurring by time x.[33] It complements the CDF by focusing on tail probabilities and is non-increasing, with S_X(x) approaching 0 as x goes to infinity. Additionally, the CDF enables straightforward computation of probabilities over intervals: for any real numbers a < b, P(a < X \leq b) = F_X(b) - F_X(a), which holds for both continuous and discrete cases due to the right-continuity of the CDF.[34] This property underscores the CDF's role in bounding and calculating distributional intervals efficiently.[27]Discrete Probability Distributions
Definition and Examples
A discrete probability distribution describes the probabilities associated with a random variable whose possible values form a countable set, such as the integers or a finite list. In this framework, the probability that the random variable X takes a specific value x is given by P(X = x) = p(x), where p is the probability mass function satisfying p(x) \geq 0 for all x and \sum p(x) = 1 over the support, ensuring the total probability sums to unity.[35][18] These distributions assign positive probability only to countable points, with zero probability for intervals between points. The cumulative distribution function is a step function, jumping at each point with positive probability.[36] Prominent examples include the Bernoulli distribution, which models a single trial with success probability p (e.g., coin flip, where P(X=1) = p and P(X=0) = 1-p), and the discrete uniform distribution on \{1, 2, \dots, n\}, which assigns equal probability $1/n to each integer (e.g., die roll).[37][38]Probability Mass Function
The probability mass function (PMF) of a discrete random variable X is defined as the function p(x) = P(X = x), which assigns to each possible value x in the support of X the probability that X equals x.[18] This function fully characterizes the distribution of X, providing the probabilities for all discrete outcomes.[39] The PMF satisfies two fundamental properties: p(x) \geq 0 for all x in the sample space, ensuring non-negative probabilities, and \sum_{x} p(x) = 1, guaranteeing that the total probability over all possible outcomes is unity.[18] The support of the PMF consists of the set of all x where p(x) > 0, which may be finite, countably infinite, or a subset of the integers.[19] Key properties of the PMF enable the computation of important distributional characteristics. The expected value, or mean, of X is given by E[X] = \sum_{x} x \, p(x), representing the long-run average value of the random variable. The variance, which measures the spread of the distribution, is \operatorname{Var}(X) = E[X^2] - (E[X])^2, where E[X^2] = \sum_{x} x^2 \, p(x).[23] To compute the PMF for specific distributions, standard formulas are applied. For the binomial distribution with parameters n (number of trials) and p (success probability), the PMF is p(k) = \binom{n}{k} p^k (1-p)^{n-k} for k = 0, 1, \dots, n. For the Poisson distribution with parameter \lambda (average rate), it is p(k) = e^{-\lambda} \frac{\lambda^k}{k!} for k = 0, 1, 2, \dots .[40] The PMF relates to the probability generating function (PGF) of X, defined as G(s) = \sum_{x} p(x) s^x = E[s^X], which encodes the probabilities as coefficients in its power series expansion and facilitates calculations for sums of independent random variables.[41]Continuous Probability Distributions
Definition and Examples
A continuous probability distribution, specifically an absolutely continuous one, describes the probabilities associated with a random variable whose possible values form an uncountable set, such as an interval on the real line. In this framework, the probability that the random variable X falls within an open interval (a, b) is computed as the integral \int_a^b f(x) \, dx, where f is the probability density function satisfying f(x) \geq 0 for all x and \int_{-\infty}^{\infty} f(x) \, dx = 1.[42][43] These distributions exhibit absolute continuity with respect to the Lebesgue measure, implying no point masses: the probability assigned to any single point is zero, P(X = x) = 0 for all x./03:_Distributions/3.13:_Absolute_Continuity_and_Density_Functions) The cumulative distribution function for such a distribution arises as the integral of the density function up to a given point.[43] Prominent examples include the uniform distribution on the interval [a, b], which assigns equal probability to every point within that bounded range, modeling scenarios like random selection from a continuous uniform source.[44] The exponential distribution, parameterized by a rate \lambda > 0, captures waiting times between independent events occurring at a constant average rate, such as inter-arrival times in a Poisson process.[45] The normal distribution, defined by mean \mu \in \mathbb{R} and standard deviation \sigma > 0, produces the characteristic bell-shaped curve and serves as a foundational model for phenomena where values cluster symmetrically around the center, underpinning much of inferential statistics.Probability Density Function
For a continuous random variable X with support over the real numbers, the probability density function (PDF), denoted f(x), is a non-negative function such that the probability that X falls within an interval [a, b] is given by the integral \int_a^b f(x) \, dx, rather than the value of the function at a specific point.[31] Unlike probabilities in discrete distributions, the PDF value f(x) at any point does not represent a probability and can exceed 1, as it measures density rather than likelihood at a point; the probability of X equaling exactly any single value is zero.[31] The interpretation of the PDF emphasizes that probabilities are determined by the area under the curve over an interval, providing a geometric understanding of continuous outcomes.[46] Key properties of the PDF include normalization, where \int_{-\infty}^{\infty} f(x) \, dx = 1, ensuring the total probability over the entire support is unity, and non-negativity, f(x) \geq 0 for all x.[46] The expected value (mean) \mu is computed as \mu = \int_{-\infty}^{\infty} x f(x) \, dx, and the variance \sigma^2 as \sigma^2 = \int_{-\infty}^{\infty} (x - \mu)^2 f(x) \, dx, which quantify central tendency and spread using weighted integrals over the density.[47] A classic example is the uniform distribution on the interval [a, b], where the PDF is constant: f(x) = \begin{cases} \frac{1}{b - a} & a \leq x \leq b, \\ 0 & \text{otherwise}. \end{cases} This reflects equal likelihood across the interval, with the height \frac{1}{b - a} ensuring the area integrates to 1.[31] Another fundamental case is the exponential distribution with rate parameter \lambda > 0, modeling waiting times or lifetimes, with PDF: f(x) = \begin{cases} \lambda e^{-\lambda x} & x \geq 0, \\ 0 & \text{otherwise}. \end{cases} Here, the density decays exponentially, capturing memoryless properties in processes like radioactive decay.[47]Other Types of Distributions
Singular Distributions
Singular distributions, also known as singular continuous distributions, are probability distributions that are neither discrete nor absolutely continuous with respect to the Lebesgue measure.[48] Their cumulative distribution function (CDF) is continuous and non-decreasing but lacks a probability density function (PDF), as the distribution assigns positive probability to sets of Lebesgue measure zero while having no point masses.[49] This contrasts with absolutely continuous distributions, where the CDF is the integral of a density function. A key property of singular distributions is that the derivative of their CDF is zero almost everywhere with respect to the Lebesgue measure, yet the CDF still increases over intervals, concentrating probability on "pathological" sets like fractals. These distributions are mutually singular with the Lebesgue measure, meaning there exists a set of measure zero that carries all the probability mass.[50] Unlike discrete distributions, they have no atoms, ensuring the CDF has no jumps. The Cantor distribution provides a canonical example of a singular continuous distribution. It is supported on the ternary Cantor set in [0,1], a compact set of Lebesgue measure zero constructed by iteratively removing middle-third intervals.[51] The CDF of the Cantor distribution, known as the Cantor-Lebesgue function or devil's staircase, is constant on the removed intervals and increases continuously from 0 to 1 over the Cantor set, resulting in a continuous but nowhere differentiable function except at countably many points. This function maps the unit interval onto [0,1] in a measure-preserving way, illustrating how probability can be distributed without density. In general, any probability distribution on the real line can be uniquely decomposed into a mixture of a discrete component (with point masses), an absolutely continuous component (with a PDF), and a singular continuous component, as per the Lebesgue decomposition theorem.[49] The singular part captures distributions that evade both atomic and density-based descriptions, highlighting the richness of measure-theoretic probability.[48]Multivariate Distributions
A multivariate probability distribution describes the joint behavior of multiple random variables, extending the univariate case to vector-valued outcomes. For a random vector \mathbf{X} = (X_1, \dots, X_n) taking values in \mathbb{R}^n, the joint cumulative distribution function (CDF) is defined as F(x_1, \dots, x_n) = P(X_1 \leq x_1, \dots, X_n \leq x_n), which fully characterizes the distribution and is non-decreasing in each argument with limits F(-\infty, \dots, x_i, \dots, -\infty) = 0 and approaching 1 as all arguments go to \infty.[52] For discrete random vectors, the joint probability mass function (PMF) p(x_1, \dots, x_n) = P(X_1 = x_1, \dots, X_n = x_n) specifies probabilities at each point in the support, summing to 1 over all possible outcomes. In the continuous case, the joint probability density function (PDF) f(x_1, \dots, x_n) satisfies \int_{-\infty}^{\infty} \cdots \int_{-\infty}^{\infty} f(x_1, \dots, x_n) \, dx_1 \cdots dx_n = 1, and the joint CDF relates to it via F(x_1, \dots, x_n) = \int_{-\infty}^{x_1} \cdots \int_{-\infty}^{x_n} f(u_1, \dots, u_n) \, du_1 \cdots du_n.[52] Marginal distributions are derived from the joint by eliminating variables not of interest, providing the univariate or lower-dimensional laws. For a continuous bivariate case with joint PDF f_{X,Y}(x,y), the marginal PDF of X is f_X(x) = \int_{-\infty}^{\infty} f_{X,Y}(x,y) \, dy, assuming the integral exists; similarly for discrete cases, summation replaces integration.[53] This process generalizes to higher dimensions by integrating or summing over the unwanted coordinates, yielding the marginal CDF or PMF for the retained variables. Marginals capture individual behaviors but lose information on dependencies among the variables. Independence between random variables implies no influence between their outcomes, formalized such that the joint distribution factors into marginals. Specifically, X_1, \dots, X_n are mutually independent if the joint CDF equals the product of marginal CDFs: F(x_1, \dots, x_n) = F_1(x_1) \cdots F_n(x_n), or equivalently for PMFs/PDFs: p(x_1, \dots, x_n) = p_1(x_1) \cdots p_n(x_n) or f(x_1, \dots, x_n) = f_1(x_1) \cdots f_n(x_n).[54] This property simplifies computations, as expectations and variances of functions separate additively. Prominent examples include the multivariate normal distribution, which generalizes the univariate normal to vectors with mean vector \boldsymbol{\mu} and covariance matrix \boldsymbol{\Sigma}, capturing linear correlations through its elliptical contours and central limit theorem applicability in high dimensions.[55] Copulas provide a flexible framework for modeling dependence separately from marginals, as per Sklar's theorem, which states that any joint CDF F can be expressed as F(x_1, \dots, x_n) = C(F_1(x_1), \dots, F_n(x_n)), where C is a copula—a multivariate CDF with uniform [0,1] marginals—allowing construction of diverse dependence structures like tail dependence in finance or risk assessment.[56]Advanced Characterizations
Kolmogorov Axioms
The Kolmogorov axioms form the rigorous mathematical foundation of modern probability theory, establishing it as a branch of measure theory and providing the basis for defining probability distributions. These axioms ensure that probabilities behave consistently as a countably additive measure on a structured space of events, allowing for the precise modeling of uncertainty in both discrete and continuous settings. By abstracting probability from empirical frequencies to an axiomatic system, they enable the derivation of all key properties of distributions without reliance on specific interpretations of probability. A probability space, the fundamental structure underlying this theory, consists of a triple (\Omega, \mathcal{F}, P), where \Omega is the sample space representing all possible outcomes, \mathcal{F} is a \sigma-algebra of subsets of \Omega (known as events), and P: \mathcal{F} \to [0, 1] is a probability measure that assigns a non-negative real number to each event, with P(\Omega) = 1.[57] The \sigma-algebra \mathcal{F} ensures closure under countable unions, intersections, and complements, providing a complete framework for defining events and their probabilities. Random variables are then introduced as measurable functions X: \Omega \to \mathbb{R}, meaning that for every Borel set B \subseteq \mathbb{R}, the preimage X^{-1}(B) \in \mathcal{F}.[57] The probability measure P satisfies three axioms:-
Non-negativity: For every event A \in \mathcal{F}, P(A) \geq 0.
This ensures probabilities represent non-negative extents of possibility.[58] -
Normalization: P(\Omega) = 1.
This normalizes the total probability of the entire sample space to unity.[58] - Countable additivity: If \{A_i\}_{i=1}^\infty \subseteq \mathcal{F} is a countable collection of pairwise disjoint events (i.e., A_i \cap A_j = \emptyset for i \neq j), then P\left( \bigcup_{i=1}^\infty A_i \right) = \sum_{i=1}^\infty P(A_i). This axiom extends finite additivity to countable collections, crucial for handling infinite sample spaces in continuous distributions.[58]