Fact-checked by Grok 2 weeks ago

Sampling distribution

In statistics, a sampling distribution refers to the probability distribution of a statistic, such as the sample mean or sample proportion, obtained from all possible random samples of a fixed size drawn from a given population.^[1] This distribution describes the variability and relative frequencies of the statistic across repeated sampling under a specified plan, enabling inferences about population parameters.^[2] The mean of a sampling distribution for the sample mean equals the population mean, denoted as \mu, while its standard deviation, known as the standard error, is the population standard deviation \sigma divided by the square root of the sample size n.^[3] This standard error quantifies the precision of the statistic as an estimate of the population parameter and decreases as sample size increases, reflecting reduced sampling variability.^[4] A fundamental property of sampling distributions is highlighted by the Central Limit Theorem (CLT), which states that, for sufficiently large sample sizes, the sampling distribution of the sample mean approaches a normal distribution regardless of the population's underlying distribution, provided the population has finite variance.^[5] This approximation facilitates the use of normal probabilities for hypothesis testing, confidence intervals, and other inferential procedures, even when the population distribution is unknown or non-normal.^[4] Sampling distributions are essential in statistical inference, as they underpin methods for estimating population characteristics and assessing the reliability of sample-based conclusions in fields such as quality control, survey sampling, and experimental design.^[2] For instance, in simple random sampling, the distribution allows evaluation of estimator bias and variance to optimize sample sizes and design choices.^[1]

Definition and Basics

Definition

In statistics, the sampling distribution refers to the probability distribution of a given statistic, such as the sample mean or sample variance, obtained from all possible random samples of a fixed size n drawn from a specific population.^[6] This distribution describes the possible values that the statistic can take and the probabilities associated with those values across the entirety of conceivable samples.^[7] The sampling distribution arises through the process of repeated random sampling, where multiple independent samples of size n are drawn from the population, and the chosen statistic is computed for each sample.^[6] Theoretically, this involves considering an infinite number of such samples to form the complete distribution, allowing statisticians to characterize the variability and behavior of the statistic without needing to enumerate every sample in practice.^[8] Unlike an empirical distribution, which is constructed from a finite set of observed samples and thus approximates the true distribution, the sampling distribution is a theoretical construct based on the infinite population of all possible samples.^[9] This theoretical nature enables precise probabilistic statements about the statistic's behavior relative to the underlying population distribution.^[6] Standard notation in this context uses \theta to denote a population parameter and \hat{\theta} to represent the corresponding sample statistic or estimator derived from a sample.^[10]

Key Components

The sampling distribution represents the probability distribution of a statistic computed from all possible random samples of a fixed size drawn from a population. For discrete statistics, it takes the form of a probability mass function, assigning probabilities to each possible value of the statistic; for continuous statistics, it is a probability density function describing the likelihood over a continuum of values. This probabilistic framework allows for quantifying the uncertainty in the statistic as an estimator of population parameters.^[7]^[11] A fundamental requirement for the validity of a sampling distribution is the assumption of random sampling, where observations are independent and identically distributed (i.i.d.). Independence ensures that the value of one observation does not influence others, while identical distribution means each is drawn from the same underlying population probability distribution. This i.i.d. condition holds under simple random sampling with replacement or when sampling without replacement from a sufficiently large population relative to the sample size. Violations, such as dependence between observations, can distort the distribution and lead to invalid inferences.^[12]^[13] The choice of statistic fundamentally shapes the sampling distribution, as the statistic is a function of the sample data that summarizes specific population characteristics. Common examples include the sample mean \bar{x}, which estimates the population mean and typically yields a symmetric distribution under i.i.d. assumptions; the sample proportion \hat{p}, used for binary outcomes and forming a binomial-based distribution; and the sample variance s^2, which captures spread and often follows a chi-squared distribution for normal populations. The form and properties of the resulting distribution depend directly on this functional choice, influencing its shape, center, and spread.^[14]^[15] The sample size n plays a pivotal role in determining the characteristics of the sampling distribution, particularly its variability and concentration around the expected value. For a fixed statistic, larger n reduces the spread of the distribution, as measured by its standard deviation or standard error, making the statistic more precise as an estimator. This narrowing effect arises because additional observations average out random fluctuations, with the standard error scaling inversely with \sqrt{n}. Conversely, smaller n results in greater variability, highlighting the trade-off between precision and practical feasibility in sampling.^[16]^[17]

Theoretical Foundations

Relation to Population Distribution

The sampling distribution of a statistic arises directly from the underlying probability distribution of the population, as it represents the distribution of that statistic across all possible random samples of a fixed size drawn from the population. For instance, if the population follows a normal distribution, the sampling distribution of the sample mean will also be normal, regardless of sample size. Similarly, for a population governed by a binomial distribution, the sampling distribution of the sample proportion inherits characteristics from the binomial process. This emergence underscores that the sampling distribution is not independent but is fundamentally shaped by the population's probabilistic structure.^[7] A key connection lies in parameter estimation, where the sampling distribution is centered on the true population parameter, ensuring that statistics like the sample mean are unbiased estimators of parameters such as the population mean μ. This centering property implies that, on average, the statistic equals the population parameter, providing a theoretical foundation for inference about unknown population values from sample data.^[18] The shape of the population distribution, including its skewness or kurtosis, directly affects the sampling distribution, especially for small sample sizes. A positively skewed population, for example, imparts skewness to the sampling distribution of the mean, leading to asymmetry that mirrors the population's tail behavior; higher kurtosis can similarly result in heavier tails in the sampling distribution. These effects diminish with larger samples, where the central limit theorem approximates normality, but they remain pronounced without such large-sample conditions.^[16]^[19] When sampling from a finite population without replacement, the sampling distribution requires adjustment via the finite population correction to account for reduced variability compared to infinite populations. This correction scales the variance downward by a factor that depends on the ratio of sample size to population size, becoming negligible when the population is large relative to the sample (e.g., when n is much smaller than N). In contrast, for infinite populations or sampling with replacement, no such adjustment is needed, and the sampling distribution reflects the full population variability.^[20]^[21]

Sampling Variability

Sampling variability arises primarily from the inherent randomness in the process of selecting samples from a population, where each sample represents only a subset of the total units and thus may differ due to chance alone.^[22] This randomness is further influenced by the heterogeneity within the population, as greater diversity in population characteristics leads to more pronounced differences across repeated samples.^[23] A key manifestation of this variability is sampling error, defined as the random difference between a sample statistic—such as the sample mean—and the corresponding population parameter.^[24] Unlike systematic biases, sampling error reflects unavoidable fluctuations inherent to the sampling mechanism, assuming random selection without other errors.^[25] Several factors modulate the degree of sampling variability. Increasing the sample size tends to diminish this variability, as larger samples capture more of the population's diversity and yield statistics closer to the true parameters.^[16] Conversely, higher population variance amplifies the variability, since samples drawn from more dispersed populations exhibit greater spread in their statistics..pdf) The spread in a sampling distribution qualitatively indicates the precision of estimates derived from samples; narrower spreads suggest more reliable inferences about the population, while wider spreads highlight greater uncertainty in those estimates.^[17] The standard error serves as a quantitative measure of this spread.^[16]

Core Properties

Central Limit Theorem

The Central Limit Theorem (CLT) states that, for a sufficiently large sample size n, the sampling distribution of the sample mean \bar{X} from any population with finite mean \mu and variance \sigma^2 will approximate a normal distribution with mean \mu and variance \sigma^2 / n, regardless of the underlying population distribution.^[26] This convergence holds under the assumption of independent and identically distributed (i.i.d.) random variables, enabling the use of normal-based inference even for non-normal populations.^[27] Mathematically, the CLT can be expressed in its standardized form, where the statistic Z = \frac{\bar{X} - \mu}{\sigma / \sqrt{n}} converges in distribution to a standard normal random variable N(0,1) as n \to \infty:

Z = \frac{\bar{X} - \mu}{\sigma / \sqrt{n}} \xrightarrow{d} N(0,1).

^[28] This asymptotic normality underpins much of statistical hypothesis testing and confidence interval construction.^[29] The theorem requires key conditions, including the independence of observations and the existence of a finite population variance \sigma^2 < \infty; without finite variance, such as in Cauchy distributions, the CLT does not apply.^[30] In practice, the approximation is often reasonable for n \geq 30, though this rule of thumb varies by population shape—symmetric distributions converge faster, while skewed ones may require larger n.^[31] Historically, the CLT traces its origins to Pierre-Simon Laplace's work in the early 19th century, where he extended Abraham de Moivre's 1733 approximation for the binomial distribution to more general cases, establishing the normal approximation for sums of i.i.d. variables around 1810–1812.^[32] Later refinements, such as Aleksandr Lyapunov's 1901 general proof under weaker moment conditions, solidified its foundational role in probability theory and statistical inference.^[33] Despite its robustness, the CLT has limitations: it fails for very small sample sizes where the normal approximation is poor, and for populations with heavy tails or infinite variance, where convergence is slow or nonexistent.^[30] In such cases, alternatives like the bootstrap method can approximate sampling distributions non-parametrically by resampling with replacement from the data.^[29]

Standard Error

The standard error (SE) of a statistic \hat{\theta} is defined as the standard deviation of its sampling distribution, which quantifies the precision of \hat{\theta} as an estimate of the population parameter \theta.^[34] In general, SE(\hat{\theta}) = \sqrt{\text{Var}(\hat{\theta})}, where \text{Var}(\hat{\theta}) is the variance of the sampling distribution of \hat{\theta}.^[35] For the sample mean \bar{X}, the standard error is given by SE(\bar{X}) = \sigma / \sqrt{n}, where \sigma is the population standard deviation and n is the sample size; when \sigma is unknown, it is estimated using the sample standard deviation s, yielding SE(\bar{X}) \approx s / \sqrt{n}.^[36] This formula arises from the derivation of the variance of the sample mean under the assumption of independent and identically distributed (i.i.d.) observations X_1, X_2, \dots, X_n from a population with mean \mu and variance \sigma^2:

\bar{X} = \frac{1}{n} \sum_{i=1}^n X_i

The variance is then

\text{Var}(\bar{X}) = \text{Var}\left( \frac{1}{n} \sum_{i=1}^n X_i \right) = \frac{1}{n^2} \sum_{i=1}^n \text{Var}(X_i) = \frac{1}{n^2} \cdot n \sigma^2 = \frac{\sigma^2}{n},

since the X_i are i.i.d. and uncorrelated.^[35] Thus, SE(\bar{X}) = \sqrt{\text{Var}(\bar{X})} = \sigma / \sqrt{n}.^[35] For other statistics, the standard error takes a similar form based on the variance of their sampling distribution. For instance, the standard error of the sample proportion \hat{p} is SE(\hat{p}) = \sqrt{p(1-p)/n}, where p is the population proportion; it is estimated as \sqrt{\hat{p}(1-\hat{p})/n}.^[37] A narrower standard error indicates greater precision in the estimate, as it reflects less variability in the sampling distribution across repeated samples of size n.^[36] The standard error plays a key role in constructing confidence intervals, where it scales the critical value to form the margin of error around the point estimate.^[36]

Applications and Examples

Sampling Distribution of the Sample Mean

The sampling distribution of the sample mean refers to the probability distribution of the means obtained from all possible random samples of size n drawn from a population.^[38] For independent and identically distributed (i.i.d.) random variables X_1, X_2, \dots, X_n with population mean \mu and variance \sigma^2, the sample mean \bar{X} = \frac{1}{n} \sum_{i=1}^n X_i has an expected value E(\bar{X}) = \mu and variance \text{Var}(\bar{X}) = \frac{\sigma^2}{n}.^[35] This follows from the linearity of expectation, where E(\bar{X}) = E\left(\frac{1}{n} \sum_{i=1}^n X_i\right) = \frac{1}{n} \sum_{i=1}^n E(X_i) = \frac{1}{n} \cdot n\mu = \mu, since each E(X_i) = \mu. For the variance, assuming independence, \text{Var}(\bar{X}) = \text{Var}\left(\frac{1}{n} \sum_{i=1}^n X_i\right) = \frac{1}{n^2} \sum_{i=1}^n \text{Var}(X_i) = \frac{1}{n^2} \cdot n\sigma^2 = \frac{\sigma^2}{n}.^[39] If the population distribution is normal, specifically X_i \sim N(\mu, \sigma^2), then the sampling distribution of \bar{X} is also normal: \bar{X} \sim N\left(\mu, \frac{\sigma^2}{n}\right). The probability density function is thus

f(\bar{x}) = \frac{1}{\sqrt{2\pi \frac{\sigma^2}{n}}} \exp\left( -\frac{(\bar{x} - \mu)^2}{2 \frac{\sigma^2}{n}} \right).

^[35] To illustrate the behavior for non-normal populations, consider simulations where repeated samples of size n are drawn from a skewed distribution, such as an exponential distribution with mean \mu = 1. Histograms of the resulting sample means show that for small n (e.g., n=5), the distribution remains somewhat skewed, but as n increases (e.g., to n=30), the histogram approaches a normal shape centered at \mu with reduced spread, consistent with the central limit theorem for large n.^[40] For sampling without replacement from a finite population of size N, the variance of the sample mean requires adjustment by the finite population correction factor \frac{N-n}{N-1}, yielding \text{Var}(\bar{X}) = \frac{\sigma^2}{n} \cdot \frac{N-n}{N-1}, which accounts for the reduced variability when the sample depletes a significant portion of the population.^[41]

Sampling Distribution of the Sample Proportion

The sampling distribution of the sample proportion refers to the probability distribution of the proportion of successes, denoted as \hat{p}, obtained from all possible random samples of fixed size n drawn from a population consisting of Bernoulli trials with success probability p. The sample proportion \hat{p} is calculated as the number of successes divided by the sample size n, where each trial is independent and binary.^[42] The mean of this sampling distribution is equal to the population proportion p, so E(\hat{p}) = p. The variance is given by \text{Var}(\hat{p}) = \frac{p(1-p)}{n}.^[42] These properties follow from the fact that the total number of successes X in the sample follows a binomial distribution with parameters n and p, so \hat{p} = X/n. Thus, E(\hat{p}) = E(X)/n = (np)/n = p, and \text{Var}(\hat{p}) = \text{Var}(X)/n^2 = [np(1-p)]/n^2 = p(1-p)/n.^[42] An alternative derivation uses indicator random variables: define I_j = 1 if the j-th trial is a success and $0otherwise, forj = 1, \dots, n. Then \hat{p} = \frac{1}{n} \sum_{j=1}^n I_j, where each I_jis Bernoulli with parameterp. The expected value is E(\hat{p}) = \frac{1}{n} \sum_{j=1}^n E(I_j) = \frac{1}{n} \cdot n p = p, since the I_jare independent. The variance is\text{Var}(\hat{p}) = \frac{1}{n^2} \sum_{j=1}^n \text{Var}(I_j) = \frac{1}{n^2} \cdot n \cdot p(1-p) = \frac{p(1-p)}{n}, as \text{Var}(I_j) = p(1-p)$.^[42] Under the central limit theorem, for sufficiently large n—specifically when np \geq 5 and n(1-p) \geq 5—the sampling distribution of \hat{p} is approximately normal with mean p and standard deviation \sqrt{p(1-p)/n}. For example, if p = 0.5 and n = 100, a histogram constructed from many simulated values of \hat{p} would closely resemble a normal curve centered at $0.5with spread decreasing asn$ increases.^[43] For sampling without replacement from a finite population of size N, the variance of the sample proportion requires adjustment by the finite population correction factor \frac{N-n}{N-1}, yielding \text{Var}(\hat{p}) = \frac{p(1-p)}{n} \cdot \frac{N-n}{N-1}, which accounts for the reduced variability when the sample depletes a significant portion of the population.^[44] This distribution is widely applied in survey sampling to estimate population proportions, such as voter preferences, and in quality control to assess defect rates in manufacturing processes.^[45]

References

[1]
https://stats.libretexts.org/Bookshelves/Introductory_Statistics/Introductory_Statistics_(Shafer_and_Zhang)/06%3A_Sampling_Distributions
[2]
None
### Definition of Sampling Distribution
[3]
[PDF] Statistical Interpretation of Key Comparison Reference Value and ...
A sampling distribution is a probability distribution that describes the relative fre- quencies of occurrence for all possible results of meas- urement when the ...
[4]
1.3.6.6.1. Normal Distribution - Information Technology Laboratory
The sampling distribution of the mean is centered at the population mean, μ, of the original variable. In addition, the standard deviation of the sampling ...
[5]
4 Sampling Distributions – STAT 500 | Applied Statistics
4.1 (Sampling Distribution) The sampling distribution of a statistic is a probability distribution based on a large number of samples of size n from a given ...
[6]
[PDF] Sampling Distributions - NJIT
The sampling distribution of a statistic is the distribution of all possible values taken by the statistic when all possible samples of a fixed size n are taken ...
[7]
[PDF] 4.1 Sampling Distributions
Definition. The sampling distribution of a statistic is the distribution of values taken by the statistic in all possible samples of the same size from the ...
[8]
Chapter 8: Sampling Distributions - Maricopa Open Digital Press
... sampling distribution is a theoretical distribution rather than an empirical distribution. The distribution is based on sample statistics (sample means) not ...
[9]
[PDF] Parameter Estimation
Aug 30, 2020 · Standard notation in statistics, where a “hat” (i.e.,ˆ) is placed on top of the parameter to denote that ˆθ is an estimate of θ.
[10]
[PDF] Chapter 4 Sufficient Statistics
A statistic is a function of the data that does not depend on any unknown parameters. The probability distribution of the statistic is called the sampling ...Missing: key | Show results with:key<|control11|><|separator|>
[11]
[PDF] Chapter 4 Sampling Distributions and Limits
A sampling distribution is the distribution of a random variable corresponding to a function of some i.i.d. sequence. Sampling distributions can sometimes be ...
[12]
[PDF] Data Acquisition, Sampling, and the IID Assumption - Rose-Hulman
IID means Independent and Identically Distributed. Simple random sampling is approximately IID when the sample size is small compared to the population size.
[13]
[PDF] Sampling Distributions What Is a Statistic? . It is a function of ...
A statistic is a function of observable random variables used to estimate a population parameter. For example, _k for μ and s² for σ².
[14]
[PDF] 7 Random samples and sampling distributions - Arizona Math
The sampling distrbution of a statistic depends on fY (y) and the func- tion that defines the statistic. If we know these, then in principle we can compute the ...Missing: key components
[15]
4.1.3 - Impact of Sample Size | STAT 200
Increasing sample size decreases standard error and makes the sampling distribution more normal, regardless of the population's shape.
[16]
Sampling Distribution Properties - Data Science Exploration
The variability of the sampling distributions decreases as the sample size increases; that is, the sample means generally are closer to the center as the sample ...
[17]
Sampling Distributions - Utah State University
A sampling distribution is the distribution of a statistic. Use the applet below to investigate the sampling distribution of the sample proportion. Sampling ...Missing: definition authoritative
[18]
[PDF] SAMPLING-SKEWED BIOLOGICAL POPULATIONS - VTechWorks
Positive skewness of a population affects confidence statements about the population total T for two reasons: (1) it imparts skew to the sampling distribution ...
[19]
[PDF] Sampling and Sampling Distributions - University of Richmond
Finite Population σ2. ¯p = N − n. N − 1 p(1 − p) n. Infinite Population σ2. ¯p = p(1 − p) n. ▷ The term N−n. N−1 is the finite population correction ...
[20]
[PDF] STAT 421 Finite Population Sampling with Application to the ...
We see that in the hypergeometric sampling case the variance of ¯Y has the additional finite pop- ulation correction factor (N − n)/(N − 1), which degenerates ...
[21]
From Samples to Populations - Stat 20
If the sample is drawn from the population with some amount of randomness, the sampling variability describes the variability from one sample to the next.
[22]
[PDF] A Simulated Sampling Distribution - UCLA Statistics & Data Science
The VARIABILITY of a statistic is determined by the spread (variability) of the sampling ... Source: Harvard School of Public Health. Lecture 5 - Ch. 4.1-4.2.
[23]
Random Sampling Error - Troy University Spectrum
The difference between the sample mean and the population mean is known as sampling error or sampling bias.
[24]
[PDF] Bias in Sampling - MATH 130, Elements of Statistics I
Definition. Sampling error is the error that results from using sampling to estimate the information regarding a population. This type of error occurs ...
[25]
[PDF] Central Limit Theorem - Chance
We will discuss the theorem in the case that the individual random vari- ables are identically distributed, but the theorem is true, under certain conditions,.
[26]
[PDF] The Central Limit Theorem - UMD MATH
The expression “Zn −→ N(0,1)” abbreviates “Zn converges in distribution to. N(0,1) as n → ∞”. Informally, this means that if n is large enough, then we ...
[27]
[PDF] The Central Limit Theorem - Arizona Math
e x2/2 dx = Φ(z). In practical terms the central limit theorem states that P{a<Zn  b} ⇡ P{a<Z  b} = Φ(b) Φ(a). This theorem is an enormously useful tool in ...
[28]
[PDF] Lecture 10 : Setup for the Central Limit Theorem
The setup for the Central Limit Theorem involves triangular arrays, the Lindeberg condition, and the Lyapounov condition, which is stronger than Lindeberg's.
[29]
[PDF] A PEDAGOGICAL NOTE ON THE CENTRAL LIMIT THEOREM
There are some limiting conditions for the CLT. The most obvious is that the standard deviations exist for the xi. There are probability distributions where the ...Missing: statement | Show results with:statement
[30]
[PDF] 6 Central Limit Theorem
The Central Limit Theorem (CLT) states that if n is large enough, the sample mean has an approximately normal distribution, even if the population is not ...
[31]
Pierre Simon de Laplace (1749-1827)
Laplace made major contributions to the development of probability and statistics ... Central Limit Theorem, furthering Bayes' work with his eponymous ...
[32]
[PDF] Studying “moments” of the Central Limit theorem
The journey encompassing central limit theorem includes reformations of definition, relaxing of important associated conditions, and numerous types of rigorous ...
[33]
Sampling Distributions
The standard deviation of the sampling distribution of sample means is called the standard error of the mean. It is a measure of how far, on average, sample ...Missing: definition derivation
[34]
24.4 - Mean and Variance of Sample Mean | STAT 414
We'll finally accomplish what we set out to do in this lesson, namely to determine the theoretical mean and variance of the continuous random variable.Missing: error | Show results with:error
[35]
[PDF] Standard errors and confidence intervals - Stanford University
In this lecture, we show how we can characterize the sampling distribution for three important, commonly used estimators: sample means; OLS linear.
[36]
Standard Error of a Proportion
The standard error of a proportion is a statistic indicating how greatly a particular sample proportion is likely to differ from the proportion in the ...
[37]
[PDF] Sampling Distributions - Rose-Hulman
Overview: Now we will derive the sampling distribution of the sample mean for IID samples. The key to doing this to recognize that the sample mean is a linear ...
[38]
6.2 The Sampling Distribution of the Sample Mean (σ Known)
The following images look at sampling distributions of the sample mean built from taking 1000 samples of different sample sizes from a non-normal Population (in ...
[39]
7.4 Finite Population Correction Factor - OpenStax
Dec 13, 2023 · ... proportions. The Finite Population Correction Factor for the variance of the means shown in the standardizing formula is: Z=ˉx−µσ√n ...
[40]
Sampling Distribution of Proportion - Stat Trek
This lesson describes the sampling distribution of a proportion. Explains how to compute standard error of a proportion. Includes problem with solution.