Ancillary statistic

In statistics, an ancillary statistic is a function of the sample data whose probability distribution does not depend on the unknown parameters of the model, meaning it remains invariant across all possible parameter values.^[1] This concept, introduced by Ronald A. Fisher in his 1925 paper "Theory of Statistical Estimation," allows for the identification of data aspects that provide no direct information about the parameters but can influence inference when combined with other statistics.^[2] Ancillary statistics play a crucial role in data reduction and conditional inference, enabling the separation of parameter-free variability from parameter-dependent information.^[3] One of the most significant theoretical advancements involving ancillary statistics is Basu's theorem, proved by Debabrata Basu in 1955, which states that any boundedly complete sufficient statistic is independent of any ancillary statistic.^[4] This independence result is pivotal for proving statistical properties without computing full joint distributions, such as in the analysis of normal distributions with known variance, where the sample mean is a complete sufficient statistic for the mean parameter and independent of the ancillary sample variance,^[5] or in a sample from a uniform distribution on (\theta, \theta + 1), the range (maximum minus minimum) is an ancillary statistic, as its distribution is free of \theta.^[6] Ancillary statistics also arise in location-scale families, where studentized residuals—ratios that normalize for scale and location—exhibit parameter-free distributions, facilitating robust inference procedures.^[5] Their utility extends to recovering ancillary information lost in sufficient statistics, a principle Fisher emphasized for improving estimation precision in complex models.^[2] Despite their lack of standalone informational value about parameters, ancillary statistics ensure that conditional distributions remain relevant for observed data, underpinning modern frequentist methods.^[3]

Definition and Properties

Definition

In statistics, the concept of an ancillary statistic was introduced by Ronald A. Fisher in the 1920s within his framework for conditional inference and estimation. Fisher used the term to describe quantities derived from the data that support inference without depending on the parameters.^[7] The idea was later formalized by Debabrata Basu in 1964, who emphasized its role in recovering information lost in data reduction.^[8] An ancillary statistic is a function of the observed data whose probability distribution remains invariant to the unknown parameter θ in the underlying statistical model.^[7] This parameter-free distribution allows ancillary statistics to serve as a foundational element in conditional approaches to inference, where the focus shifts to aspects of the data unrelated to θ. Formally, consider a random sample X drawn from a distribution parameterized by θ. A statistic A(X) is ancillary if its cumulative distribution function satisfies

P(A(X) \leq a \mid \theta) = P(A(X) \leq a)

for all values of θ in the parameter space and all a in the support of A(X).^[7] This condition ensures that the sampling variability of A(X) is identical regardless of the true parameter value. In contrast to sufficient statistics, which encapsulate all information about θ present in the sample, ancillary statistics convey none about the parameter.^[7]

Properties

An ancillary statistic A is defined such that its distribution does not depend on the unknown parameter \theta, meaning the probability measure P_\theta(A \in \cdot) is identical for all \theta \in \Theta and thus free of \theta. This parameter independence implies that the marginal distribution of A remains unchanged across the parameter space, ensuring that A alone conveys no direct evidence about \theta.^[9] This property extends to invariance under reparameterization: if A is ancillary for \theta, then A is also ancillary for any one-to-one transformation g(\theta), as the distribution of A continues to lack dependence on the transformed parameter. Ancillary statistics are non-informative regarding \theta, contributing zero Fisher information, since their likelihood does not vary with \theta and the expected value of the score function with respect to A is null.^[9] Ancillary statistics are not unique; any measurable function h(A) of an ancillary statistic A is also ancillary provided that h preserves the parameter-free distribution of A.^[9] In the context of independent and identically distributed (i.i.d.) samples, the \sigma-algebra generated by an ancillary statistic is independent of \theta, meaning events defined by the ancillary statistic have probabilities unaffected by the parameter value.^[10]

Examples

Normal Distribution

In the context of independent and identically distributed (i.i.d.) samples from a normal distribution N(\mu, 1) with known variance \sigma^2 = 1, the sample variance serves as a classic example of an ancillary statistic for the location parameter \mu. Specifically, the sample dispersion S^2 = \frac{1}{n-1} \sum_{i=1}^n (X_i - \bar{X})^2, where \bar{X} is the sample mean, has a sampling distribution that does not depend on \mu. This independence arises because the residuals X_i - \bar{X} are translation-invariant, shifting with \mu in a way that cancels out in the squared differences.^[7] The normalized form (n-1) S^2 / \sigma^2 follows a chi-squared distribution with n-1 degrees of freedom, \chi^2_{n-1}, which is free of \mu. This result stems from the quadratic form of the residuals under normality, where the sum of squared standardized deviations yields the chi-squared law independently of the mean. In contrast, the sample mean \bar{X} is not ancillary, as its distribution N(\mu, 1/n) explicitly depends on \mu, providing information about the location parameter rather than being parameter-free.^[11] For the full normal model N(\mu, \sigma^2) with both location and scale parameters unknown, a higher-dimensional ancillary statistic emerges in the form of the configuration statistic, defined as the vector of normalized residuals \mathbf{U} = \left( \frac{X_1 - \bar{X}}{S}, \dots, \frac{X_n - \bar{X}}{S} \right). This vector captures the shape or configuration of the sample, with a joint distribution independent of both \mu and \sigma^2, as standardization removes scale and centering eliminates location effects. The configuration statistic thus provides a parameter-free summary of the data's relative positions, useful for illustrating ancillarity in location-scale settings.^[12]

Location-Scale Families

In location families, where the density is of the form f(x \mid \theta) = \psi(x - \theta) for \theta \in \mathbb{R}, ancillary statistics arise from location-invariant functions of the data. For instance, differences between observations, such as X_i - X_j or the sample range X_{(n)} - X_{(1)}, have distributions that do not depend on \theta, as shifting all data by a constant leaves these differences unchanged.^[5]^[1] In scale families, with densities f(x \mid \theta) = \frac{1}{\theta} \psi\left( \frac{x}{\theta} \right) for \theta > 0, ancillary statistics are scale-invariant, such as ratios \frac{X_i}{X_j} or \frac{|X_i|}{\sum |X_j|}, whose distributions are free of \theta because multiplying all data by a positive constant preserves the ratios.^[5]^[1] For location-scale families, densities take the form f(x \mid \mu, \sigma) = \frac{1}{\sigma} \psi\left( \frac{x - \mu}{\sigma} \right) with \mu \in \mathbb{R} and \sigma > 0, and ancillary statistics are invariant under affine transformations ax + b with a > 0. Examples include studentized residuals \frac{X_i - \bar{X}}{S}, where \bar{X} is the sample mean and S is the sample standard deviation; the joint distribution of these residuals is independent of \mu and \sigma.^[9]^[1] More generally, the vector of standardized observations T = \frac{X - \mu}{\sigma} has a parameter-free distribution, and data-based approximations like \frac{X - \bar{X}}{S} serve as ancillary statistics capturing this invariance.^[9] A concrete example occurs in the uniform distribution on [\theta - 1/2, \theta + 1/2], a location family; here, the range X_{(n)} - X_{(1)} is ancillary for \theta, as its distribution does not depend on the location parameter.^[1] This illustrates how such statistics standardize away the parameter, facilitating inference in broader location-scale settings.^[9]

Applications

Basu's Theorem

Basu's theorem states that if T is a boundedly complete sufficient statistic for a parameter \theta and A is an ancillary statistic, then T and A are stochastically independent for every value of \theta. This result, proved by Debabrata Basu in 1955, builds on Ronald Fisher's earlier introduction of the concepts of sufficiency and ancillarity in the 1920s. The proof relies on the completeness property of T. Specifically, for any bounded measurable function f, the conditional expectation satisfies E[f(T) \mid A = a] = E[f(T)], which holds almost surely for all a in the support of A.^[4] This equality implies that the joint distribution factors as P(T \leq t, A \leq a) = P(T \leq t) P(A \leq a) for all t and a, establishing independence.^[4] A key implication of the theorem is that it facilitates exact conditional inference by separating the parameter-dependent component captured by the sufficient statistic from the ancillary component, which provides no information about \theta but can be used to refine inference without introducing bias.^[4] This separation aligns with the conditionality principle, allowing inference to be based solely on the conditional distribution of T given A, thereby achieving uniformity across ancillary values.^[4]

Recovery of Information

One key application of ancillary statistics arises in recovering information lost when using a non-sufficient statistic by pairing it with an appropriate ancillary complement, thereby achieving full sufficiency in the conditional distribution.^[9] This approach leverages the fact that the joint distribution of a sufficient statistic and its ancillary complement is minimal sufficient, allowing the conditional distribution of the non-sufficient part given the ancillary to encapsulate the complete parameter information.^[12] A classic example illustrates this recovery: consider two independent and identically distributed observations X_1, X_2 \sim N(\theta, 1), where \theta is the unknown mean. The single observation X_1 is not sufficient for \theta, carrying Fisher information of 1, but the difference D = X_1 - X_2 is ancillary with distribution N(0, 2), independent of \theta. Conditioning on D = d yields the full information from both observations, increasing the Fisher information to 2. The conditional density is given by

f(X_1 \mid X_1 - X_2 = d; \theta) = N\left(\theta + \frac{d}{2}, \frac{1}{2}\right),

demonstrating recovered precision equivalent to the variance of a single normal observation halved.^[9] In conditional inference, ancillaries are used to condition out parameter-free variation, enhancing the exactness of hypothesis tests and confidence intervals by focusing on the relevant data distribution. This method improves performance over unconditional approaches, particularly in small samples, by eliminating ancillary-induced variability.^[12] Basu's theorem facilitates this by establishing independence between sufficient and ancillary statistics in certain models, enabling such conditioning without information loss (detailed in the Basu's Theorem section). An advanced application appears in constructing prediction intervals, where ancillaries standardize future observations into parameter-free pivots. For instance, in normal models, conditioning on residuals or differences creates pivotal quantities whose distributions do not depend on unknown parameters, yielding exact predictive distributions that account for both estimation and prediction uncertainty.^[9]

Ancillary Complement

In statistical inference, an ancillary complement to a statistic T is an ancillary statistic U whose distribution does not depend on the unknown parameter \theta, such that the joint statistic (T, U) is sufficient for \theta even when T alone is not.^[2] This concept, introduced by Ronald Fisher, allows for the recovery of information lost in a non-sufficient reduction of the data by incorporating the ancillary component.^[2] Ancillary complements are not always unique; multiple such U may exist for a given T, and their selection depends on the structure of the statistical model.^[2] For instance, in models from exponential families, the existence and form of ancillary complements often relate to the dimensionality of the parameter space and the availability of maximal ancillaries, which can be used to construct optimal conditional inferences. A classic example arises in the context of binomial trials, analogous to estimating a baseball player's batting success probability p, where X denotes the number of hits observed in N at-bats. Here, N is ancillary because its distribution does not depend on p (e.g., fixed by the game schedule or design). The proportion X/N is not sufficient for p, as it loses information about the scale of observation, but the joint statistic (X, N) is minimal sufficient. In this model, conditional on N = n, X \sim \operatorname{Bin}(n, p), and the pair (X, N) fully captures the information about p from the data. This mechanism underpins the recovery of ancillary information in more general settings, where conditioning on the complement enhances inference precision.^[2]

Relation to Sufficiency

Ancillary statistics were introduced by Ronald A. Fisher in 1925 as part of his foundational work on statistical estimation, where he motivated their role in achieving exact inference by conditioning on relevant subsets of the sample space, tying them directly to the concept of sufficiency to avoid irrelevant variation in tests and estimates. Fisher argued that ancillaries, whose distributions do not depend on the parameters, complement sufficient statistics to define these subsets, enabling precise likelihood-based inference without approximation.^[13] In terms of decomposition, any statistic can be factored into a sufficient part that captures all information about the parameter and an ancillary part that carries none, extending the factorization theorem; specifically, for a sufficient statistic T (such as the maximum likelihood estimator), there often exists an ancillary statistic U such that the pair (T, U) is minimal sufficient, allowing the full data to be recovered up to the ancillary variation.^[13] This decomposition highlights how sufficiency reduces the data dimension while preserving inferential content, whereas ancillarity maintains the structural variation in the data for conditional inference, ensuring that inferences are tailored to the observed configuration without introducing parameter-dependent bias. Ancillaries are particularly useful when paired with complete sufficient statistics, as this combination facilitates the construction of unbiased estimators through independence properties, such as those established in Basu's theorem, where the ancillary is independent of the complete sufficient statistic. In this framework, conditioning on the ancillary refines unbiasedness by eliminating extraneous variability, leading to uniformly minimum variance unbiased estimators in many cases. In Bayesian statistics, ancillary statistics contribute to the development of reference priors by ensuring conditional posterior independence from the ancillary given the sufficient statistic, which helps in deriving objective priors that maximize expected information while respecting the model's ancillary structure. This application underscores the modern utility of ancillaries in non-informative Bayesian inference, bridging frequentist conditioning with posterior computation.^[13]