Fact-checked by Grok 2 weeks ago

Positive-definite function

In mathematics, particularly in harmonic analysis and functional analysis, a positive-definite function is a continuous complex-valued function \Phi: \mathbb{R}^s \to \mathbb{C} such that for any integer N \geq 1, any distinct points x_1, \dots, x_N \in \mathbb{R}^s, and any complex coefficients c_1, \dots, c_N \in \mathbb{C}, the quadratic form \sum_{j=1}^N \sum_{k=1}^N \overline{c_j} c_k \Phi(x_j - x_k) \geq 0.^[1] It is called strictly positive-definite if equality holds if and only if all c_j = 0.^[1] These functions satisfy basic properties such as \Phi(0) \geq 0, \Phi(-x) = \overline{\Phi(x)} (hence \Phi is Hermitian), and |\Phi(x)| \leq \Phi(0) for all x \in \mathbb{R}^s.^[1] A cornerstone result characterizing positive-definite functions is Bochner's theorem (1932), which states that a continuous function \Phi: \mathbb{R}^s \to \mathbb{C} is positive-definite if and only if it is the Fourier transform of a finite non-negative Borel measure \mu on \mathbb{R}^s, i.e., \Phi(x) = \int_{\mathbb{R}^s} e^{-i x \cdot y} \, d\mu(y) (up to normalization constants).^[1] For the one-dimensional case on \mathbb{R}, this takes the form h(t) = \frac{1}{2\pi} \int_{-\infty}^{\infty} e^{i \omega t} \, dK(\omega), where K is a non-decreasing function of bounded variation, generalizing the representation of characteristic functions in probability theory.^[2] The theorem extends to locally compact abelian groups, where positive-definite functions correspond to Fourier transforms of positive Radon measures.^[1] Positive-definite functions arise in diverse applications, including the construction of radial basis functions for scattered data interpolation in numerical analysis, where strict positive-definiteness ensures unique solutions to interpolation problems.^[1] They also play a key role in embedding metric spaces into Hilbert spaces via Schoenberg's theorem, which characterizes radial positive-definite functions on spheres and Euclidean spaces.^[1] In probability, normalized positive-definite functions with \Phi(0) = 1 are precisely the characteristic functions of probability distributions.^[2]

Kernel-Based Definition

Formal Definition

A continuous function f: \mathbb{R}^d \to \mathbb{C} is positive-definite if, for every finite set of points x_1, \dots, x_n \in \mathbb{R}^d and complex coefficients c_1, \dots, c_n \in \mathbb{C}, the quadratic form satisfies \sum_{i,j=1}^n \overline{c_i} c_j f(x_i - x_j) \geq 0.^[3] It is called strictly positive-definite if the inequality is strict (>0) whenever the coefficients are not all zero. This condition ensures that f defines a valid (semi-)inner product structure in associated reproducing kernel Hilbert spaces.^[3] From this definition, it follows that f(0) \geq 0 and |f(x)| \leq f(0) for all x \in \mathbb{R}^d.^[3] The value f(0) is non-negative by setting n=1 and c_1 = 1, yielding |c_1|^2 f(0) \geq 0. The boundedness |f(x)| \leq f(0) arises from considering n=2 with points $0 and x, where the associated $2 \times 2 matrix being positive semidefinite implies the determinant condition f(0)^2 - |f(x)|^2 \geq 0.^[3] Equivalently, the matrix (f(x_i - x_j))_{i,j=1}^n is Hermitian positive semi-definite (or positive definite in the strict case), meaning all its eigenvalues are non-negative (or positive).^[3] This matrix perspective links directly to the theory of positive semi-definite matrices in linear algebra.^[4]

Basic Properties

A positive-definite function f: \mathbb{R}^d \to \mathbb{C} satisfies the kernel condition that for any finite set of points x_1, \dots, x_n \in \mathbb{R}^d and coefficients c_1, \dots, c_n \in \mathbb{C}, the quadratic form \sum_{j,k=1}^n \overline{c_j} c_k f(x_j - x_k) \geq 0. This condition implies several fundamental analytic properties.^[5] One key property is hermiticity: f(-x) = \overline{f(x)} for all x \in \mathbb{R}^d. To see this, consider n=2 with points $0 and x; the associated $2 \times 2 matrix \begin{pmatrix} f(0) & f(x) \\ f(-x) & f(0) \end{pmatrix} must be Hermitian positive semidefinite, implying f(-x) = \overline{f(x)} and \operatorname{[Re](/page/Re)} f(x) \leq f(0).^[5] Boundedness follows directly: |f(x)| \leq f(0) for all x \in \mathbb{R}^d. For n=1, the condition gives f(0) \geq 0; for n=2 with points $0 and x, the determinant condition yields f(0)^2 - |f(x)|^2 \geq 0, so |f(x)| \leq f(0). If f is strictly positive-definite, then equality holds only when x=0.^[5] Regarding continuity, positive-definite functions are not necessarily continuous without assumptions, but continuity at $0 implies uniform continuity on all of \mathbb{R}^d. Specifically, if |f(x) - f(0)| < \epsilon for |x| < \delta, then for any s, t \in \mathbb{R}^d, the modulus of continuity satisfies |f(s) - f(t)|^2 \leq 4 f(0) |f(0) - f(s-t)|, bounding the difference by $2\sqrt{\epsilon} whenever |s-t| < \delta, hence uniform continuity.^[6] For radial positive-definite functions f(x) = \psi(\|x\|^2) with \psi: [0, \infty) \to \mathbb{R} (real-valued case), \psi is completely monotone, meaning it admits a representation \psi(t) = \int_0^\infty e^{-t u} d\mu(u) for a positive measure \mu. Completely monotone functions are nonnegative and decreasing on [0, \infty), so |\psi(r)| = \psi(r) decreases as r = \|x\| increases.

Examples

A prominent example of a positive-definite function is the Gaussian kernel, defined as f(x) = e^{-\|x\|^2 / 2} for x \in \mathbb{R}^d. This function is positive-definite because the associated Gram matrix for any finite set of points has eigenvalues that are positive, as it corresponds to the inner product in an infinite-dimensional feature space via the Fourier transform.^[7] Another common example is the exponential kernel, given by f(x) = e^{-a \|x\|} for a > 0 and x \in \mathbb{R}^d. Its positive-definiteness follows from the fact that the Gram matrix formed by evaluating f(x_i - x_j) is positive semi-definite, with strict positive-definiteness holding for distinct points due to the decay properties ensuring invertibility.^[8] The trigonometric function f(x) = \cos(b x) for real b and x \in \mathbb{R} provides a simple one-dimensional example. It is positive-definite as a nonnegative linear combination of the exponential characters e^{i b x} and e^{-i b x}, each of which generates a positive-definite Gram matrix equivalent to a Vandermonde structure with positive eigenvalues.^[1] More broadly, the characteristic function \phi(t) = \mathbb{E}[e^{i t \cdot X}] of any probability distribution on \mathbb{R}^d is positive-definite, with \phi(0) = 1, as the associated matrices satisfy the required nonnegativity condition for complex coefficients. This connection highlights the role of positive-definiteness in probability theory, though the full characterization relies on deeper results.^[9]

Bochner's Theorem

Statement

Bochner's theorem characterizes continuous positive-definite functions on Euclidean space in terms of Fourier transforms of positive measures. Specifically, a continuous function f: \mathbb{R}^d \to \mathbb{C} is positive-definite if and only if there exists a finite positive Borel measure \mu on \mathbb{R}^d such that

f(x) = \int_{\mathbb{R}^d} e^{-2\pi i \langle \xi, x \rangle} \, d\mu(\xi)

for all x \in \mathbb{R}^d. When the measure \mu is a probability measure (satisfying \mu(\mathbb{R}^d) = 1), the function f satisfies f(0) = 1 and serves as the characteristic function of the probability distribution induced by \mu. The proof proceeds in two directions. One direction establishes that positive-definiteness implies the existence of such a measure \mu via the Riesz representation theorem applied to the space of continuous functions vanishing at infinity, where the positive-definite condition yields a positive linear functional representable by integration against \mu. The converse direction shows that the Fourier transform of a positive finite measure is positive-definite, as the defining quadratic form expands to an integral of a squared modulus against \mu, which is nonnegative by the measure's positivity.

Applications in Analysis

Positive-definite functions play a pivotal role in radial basis function (RBF) interpolation, where their positive-definiteness guarantees that the interpolation matrix is invertible, ensuring a unique solution to scattered data approximation problems in multivariate settings.^[10] For instance, functions with positive Fourier transforms, as characterized by Bochner's theorem, yield stable and well-posed interpolation schemes, particularly for irregularly spaced data points in higher dimensions. This property is essential in numerical analysis for applications like surface reconstruction and geophysical modeling, where the Gaussian function serves as a classic positive-definite radial basis for smooth interpolants. In the context of reproducing kernel Hilbert spaces (RKHS), positive-definite functions act as kernels that define the inner product structure of the space, enabling pointwise evaluation as a continuous linear functional via the reproducing property. Bochner's theorem underpins this connection by ensuring that such kernels correspond to Fourier transforms of positive measures, which facilitates the construction of Hilbert spaces suitable for approximation and regularization in functional analysis.^[11] This framework is foundational for embedding infinite-dimensional problems into finite-dimensional computations while preserving norms and convergence properties. Bochner's theorem also aids in solving integral equations involving convolution operators through Fourier inversion, where the positive-definiteness of the kernel translates to non-negative spectral measures, simplifying the analysis of operator invertibility and boundedness.^[12] In harmonic analysis, this approach resolves certain Fredholm equations on locally compact groups by diagonalizing convolutions in the Fourier domain. The theorem's origins trace back to Salomon Bochner's 1932 work, which introduced the characterization of positive-definite functions to address moment problems in harmonic analysis, providing criteria for the existence and uniqueness of representing measures.

Applications in Statistics

In statistics, positive-definite functions are essential for defining valid covariance structures in probabilistic models, particularly in Gaussian processes (GPs). A covariance function k(\mathbf{x}, \mathbf{x}') for a GP must be positive semi-definite to ensure that the resulting covariance matrix for any finite collection of input points is positive semi-definite, guaranteeing non-negative variances and proper second-moment specifications for the process. This property allows GPs to model complex dependencies in data, such as in regression and forecasting tasks, where the kernel encodes assumptions about smoothness and correlation. Without positive-definiteness, the implied multivariate Gaussian distribution would be invalid, potentially leading to negative variances or non-degenerate correlations. In spatial statistics and geostatistics, positive-definite functions underpin kriging methods for interpolating spatial data. The Matérn covariance function, parameterized by a smoothness parameter \nu > 0 and range \ell, is widely used due to its flexibility in modeling varying degrees of spatial continuity; it is positive definite, as established by Bochner's theorem, which represents it as the Fourier transform of a positive spectral density measure. Similarly, the exponential covariance function k(h) = \sigma^2 \exp(-|h|/\ell), where h is the spatial lag, is positive definite and serves as a simple model for isotropic exponential decay in correlations, facilitating unbiased and minimum-variance predictions in kriging applications like environmental monitoring. These functions ensure the kriging equations yield positive-definite systems, avoiding computational instabilities. The central limit theorem (CLT) provides implications for positive-definite functions through their connection to characteristic functions. Since continuous positive-definite functions vanishing at infinity are characteristic functions of probability measures by Bochner's theorem, the CLT ensures that normalized sums of independent random variables converge in distribution to a Gaussian, with the corresponding characteristic functions—products of individual positive-definite functions—converging pointwise to the Gaussian characteristic function \exp(-t^2/2), which is itself positive definite. This validates the use of such sums in approximating limiting distributions in probabilistic models. A modern application post-2000 involves positive-definite autocorrelations in time series analysis for intrusion detection. In network security, kernel recursive least squares methods employ positive-definite kernels to model multivariate time series of traffic features, such as packet header entropies, enabling online anomaly detection for intrusions like denial-of-service attacks by identifying deviations from learned normal patterns. The positive-definiteness ensures stable kernel matrices for recursive updates, supporting real-time processing of streaming data.^[13]

Generalizations

To Abelian Groups

In the context of harmonic analysis on locally compact abelian groups, Pontryagin duality provides the foundational framework, associating each such group G with its dual group \hat{G}, the set of continuous homomorphisms from G to the circle group \mathbb{T}, equipped with the compact-open topology, which ensures \hat{G} is also a locally compact abelian group. This duality underpins the generalization of positive-definite functions beyond Euclidean spaces. A continuous function f: G \to \mathbb{C} on a locally compact abelian group G is positive-definite if, for every finite set of points x_1, \dots, x_n \in G and complex coefficients c_1, \dots, c_n \in \mathbb{C}, the inequality

\sum_{i=1}^n \sum_{j=1}^n c_i \overline{c_j} f(x_i^{-1} x_j) \geq 0

holds. This adapts the kernel-based definition to the group structure, where the argument x_i^{-1} x_j captures the relative positions via the group operation, extending the Euclidean case where the group is \mathbb{R}^d under addition. The generalized Bochner's theorem states that a continuous positive-definite function f on G is the Fourier transform of a unique positive finite Radon measure \mu on the dual group \hat{G}, given by

f(x) = \int_{\hat{G}} \langle x, \chi \rangle \, d\mu(\chi),

where \langle x, \chi \rangle = \chi(x) denotes the pairing between G and \hat{G}. This representation highlights the role of positive measures in characterizing positive-definiteness, mirroring the classical case but leveraging the dual group's structure for arbitrary locally compact abelian G. A concrete example arises on the circle group \mathbb{T} = \mathbb{R}/\mathbb{Z}, whose dual is the integer group \mathbb{Z}. Here, trigonometric polynomials of the form f(\theta) = \sum_{k=-N}^N a_k e^{2\pi i k \theta} with a_k \geq 0 for all k and \sum a_k = f(0) < \infty are positive-definite, as their Fourier coefficients correspond to a positive finite measure on \mathbb{Z}.

To Non-Abelian Settings

The extension of positive-definiteness to non-abelian groups, such as Lie groups, typically involves left-invariant kernels of the form f(g^{-1} h), where G is the group, g, h \in G, and f: G \to \mathbb{C} satisfies the condition that for any finite set \{g_1, \dots, g_n\} \subset G and complex coefficients c_1, \dots, c_n \in \mathbb{C},

\sum_{j,k=1}^n c_j \overline{c_k} f(g_j^{-1} g_k) \geq 0.

This formulation ensures the kernel induces a positive semidefinite Gram matrix, analogous to the abelian case but accounting for non-commutativity through the group multiplication.^[14] Partial analogs of Bochner's theorem exist for non-abelian groups, particularly compact ones, where a continuous positive definite function \phi on G admits a representation as the Fourier transform over the dual \hat{G}, the set of equivalence classes of irreducible unitary representations T:

\phi(x) = \sum_{[T] \in \hat{G}} d_T \operatorname{Tr} \left[ \hat{\phi}(T) T(x) \right],

with d_T = \dim H_T the dimension of the representation space H_T, and \hat{\phi}(T) a positive semidefinite operator on H_T for each T. This links positive definiteness to operator-valued measures on the dual, where the measure assigns positive operators rather than scalars, reflecting the higher-dimensional nature of non-abelian representations. For general locally compact groups, such characterizations hold under additional assumptions like coamenability, but fail otherwise due to the absence of a simple scalar dual structure.^[15] In quantum mechanics, positive-definite kernels arise in the construction of coherent states on non-abelian Lie groups, serving as reproducing kernels for quantization schemes. For semisimple Lie groups, coherent states are built from orbits of highest-weight vectors under group actions, yielding compact Kähler manifolds where the kernel K(z, \bar{w}) = \langle z | w \rangle is positive definite, ensuring an overcomplete basis with resolution of unity. These kernels facilitate the coherent state quantization method, bridging classical phase spaces (e.g., flag manifolds) to quantum Hilbert spaces, with applications in modeling non-abelian gauge theories like SU(3) in quantum chromodynamics.^[16]^[17] A key challenge in non-abelian settings is the lack of a full Pontryagin duality, as the dual involves infinite-dimensional or operator-valued structures rather than a simple abelian group of characters; this necessitates decomposing into irreducible representations for analysis, complicating extensions of locally defined positive definite functions to the entire group. Unlike abelian Lie groups, where Bochner-type integrals over the dual suffice, non-abelian cases often require additional conditions like complete strong positivity for global extendibility, and counterexamples exist for non-amenable groups where positive definiteness does not imply a representing measure.^[14]

Alternative Definitions

Local Positivity Condition

In analysis and optimization, an alternative notion of positive-definiteness for functions emphasizes local behavior near the origin. A real-valued function f: \mathbb{R}^n \to \mathbb{R} with f(0) = 0 is said to be positive-definite on a neighborhood D of the origin if f(x) > 0 for all x \in D \setminus \{0\}.^[18] This local positivity condition ensures the function takes strictly positive values in a small ball around zero, excluding the origin itself. For twice continuously differentiable functions, this local positivity is closely related to the properties of the Hessian matrix at the origin. Specifically, if the Hessian at zero is positive definite, then the function has a strict local minimum at zero, implying it is locally positive definite in this sense.^[19] For quadratic polynomials, this equivalence holds directly, as the Hessian is constant and determines the global behavior of the function. Classic examples include the squared Euclidean norm f(x) = \|x\|^2, which is positive definite everywhere, and the shifted exponential f(x) = e^{\|x\|^2} - 1, which satisfies the condition locally near zero due to its Taylor expansion starting with the quadratic term.^[18] These functions illustrate how the condition captures strict convexity or growth near the origin. In optimization, the local positivity condition is instrumental for identifying strict local minima of objective functions, particularly through second-order tests that confirm the Hessian is positive definite, thereby guaranteeing convergence properties in algorithms like Newton's method.^[20] This notion underpins stability analysis in dynamical systems as well, where locally positive definite Lyapunov functions prove asymptotic stability around equilibria.^[18]

Terminology Conflicts

The term "positive-definite function" encompasses multiple distinct concepts in mathematics, arising from different historical and contextual developments, which has occasionally led to ambiguities in usage across subfields.^[21] One primary definition originates in the theory of integral equations and functional analysis, tracing back to David Hilbert's foundational work in the early 1900s, where positive-definite kernels were employed to describe self-adjoint operators in Hilbert spaces.^[21] This kernel-based notion was rigorously advanced by Salomon Bochner in the 1930s, culminating in his 1932 theorem, which characterizes continuous positive-definite functions on Euclidean spaces as the Fourier transforms of finite positive Borel measures.^[21] Independently, a local definition emerged in the calculus of variations, where the positive-definiteness of the Hessian matrix at a point ensures a local minimum for the function. These parallel evolutions have sparked terminology conflicts, particularly in applied areas such as radial basis function interpolation, where the global kernel condition (per Bochner) must hold for well-posedness, yet discussions of the basis functions' local curvature properties invoke the Hessian sense, potentially blurring distinctions.^[22] Holger Wendland's 2005 monograph on scattered data approximation highlights this overlap by dedicating a chapter to positive-definite functions in the kernel context, using explicit characterizations to differentiate them from conditional or local variants in radial settings.^[22] Contemporary resolutions emphasize contextual precision, with authors favoring "positive-definite kernel" or "Bochner-positive definite function" for the global, translation-invariant case, while reserving "locally positive-definite function" or "strictly positive at a point" for Hessian-based local analyses.^[21] The term also intersects with operator theory post-1950s, linking to "completely positive functions" or maps on C*-algebras, as introduced by W. Forrest Stinespring in 1955, which extend positivity preservation to tensor products and finite matrix amplifications, distinct from the earlier scalar or kernel meanings.

References

[1]
[PDF] MATH 590: Meshfree Methods - Chapter 3: Positive Definite Functions
The function Φ is called strictly positive definite on Rs if the quadratic form (2) is zero only for c ≡ 0. Remark. For theoretical reasons the definition ...
[2]
[PDF] Derivation of the Fourier Inversion Formula, Bochner's Theorem, and ...
In Section 5, positive-semidefinite functions are introduced, and a weak version of Bochner's the- orem is proved. This result is then used to motivate ...
[3]
[PDF] Elements of Positive Definite Kernel and Reproducing Kernel Hilbert ...
The definition of "positive definite" kernel requires only the positive semidefiniteness (non-negative definiteness) of the. Gram matrix. This unmatched ...
[4]
INTEGRAL REPRESENTATIONS OF POSITIVE DEFINITE ...
A positive definite function shall be defined as a complex-valued function f(x, y), with domain a set E = SXG, where 5 is a semi-group and G is a group, such ...
[5]
METRIC SPACES AND POSITIVE DEFINITE FUNCTIONS*
Equivalent. Statement: 7//(x) is a positive function, then log/(x) differs by a constant from a positive definite function if and only iff(x) is bounded away.
[6]
https://soar.wichita.edu/bitstreams/cd5a2d81-4203-421b-856f-73622d4f5efc/download
[7]
[PDF] isotropic positive definite functions on spheres - SOAR
Thus the “ modulus of continuity” at each t is bounded by twice the square root of the that at 0, and so the continuity at 0 implies uniform continuity ...
[8]
[PDF] Learning with Kernels
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and ... Moreover, it is positive definite, since positive definiteness of k ...
[9]
[PDF] Kernel Design - Gaussian Process Summer Schools
white noise: K(x,y) = δx,y bias: K(x,y) = 1 exponential: K(x,y) = exp(−|x − y|). Brownian: K(x,y) = min(x,y). Gaussian: K(x,y) = exp −(x − y)2.Missing: characteristic | Show results with:characteristic
[10]
[PDF] 18.175: Lecture 10 Characteristic functions and central limit theorem
positive definite if this matrix is always positive semidefinite. Hermitian. ▷ Bochner's theorem: a continuous function from R to C with φ(0) = 1 is a ...
[11]
[PDF] Radial basis functions - UC Davis Math
has a positive Fourier transform and therefore, by Bochner's theorem, gives rise to positive definite interpolation matrices A with centres Ξ from Rn.
[12]
[PDF] 2. Positive Definite Kernel and Reproducing Kernel Hilbert Space
RKHS associated with a positive definite kernel k gives a desired feature space!! In kernel methods, the above feature map and feature space are always used. 14 ...
[13]
[PDF] On a theorem of Bochner - Numdam
Bochner's Theorem. We first recall that a mapping/of G into C is positive definite if, for any integer N,. N.
[14]
[PDF] Multivariate Online Anomaly Detection Using Kernel Recursive ...
The KOAD algorithm is also able to detect anomalies in distributions (such as arising from DoS attacks) which are evident in timeseries of header entropies [1],.
[15]
[PDF] Extension of Positive Definite Functions on Lie Groups
Seminar Sophus Lie. 1 (1991) 225–233. Extension of Positive Definite Functions on Lie Groups. Jürgen Friedrich. The aim of this paper is to give a survey of ...
[16]
[PDF] Characteristic Kernels on Groups and Semigroups
Bochner's theorem can be extended to compact groups as follows [11, Section 34.10]. Theorem 11. A continuous function φ on a compact group G is positive ...
[17]
http://www.numdam.org/item?id=AIHPA_1986__44_2_173_0
[18]
https://web.stanford.edu/class/ee363/lectures/loc-lyap.pdf
[19]
https://www.maths.tcd.ie/~dwilkins/Courses/MAU23203/MAU23203_Mich2020_CourseNotes/MAU23203_Mich2020_SecondOrderPartialDerivatives_Notes.pdf
[20]
[PDF] Locally positive definite function
Locally positive definite function function V : R n. → R is locally positive definite (LPD) if there exists an. R > 0 such that. • V (0) = 0. • V (z) > 0 for ...
[21]
[PDF] Second Order Partial Derivatives and the Hessian Matrix
Suppose that the Hessian matrix (Hi,j(p)) of the function f at the point p is positive definite. Then f has a local minimum at p. Proof The first order ...<|control11|><|separator|>
[22]
[PDF] Convexity and Second Derivatives, Gradient Descent and Acceleration
Let f : S → R be a function twice differentiable at a stationary point x ∈ S. If. Hf (x) is positive definite then x is a local minimum. Proof. Let us ...
[23]
[PDF] Positive Definite Kernels: Past, Present and Future
A positive definite kernel K can be viewed as an infinite-dimensional positive definite matrix. K. In fact, this can be done in two different ways. First, in ...
[24]
[PDF] Revisiting Arnold's topological proof of the Morse index theorem
Jul 2, 2023 · Marston Morse gave the first clear general statements in the 1920s and 1930s, leading to what is known today as Morse theory [17].
[25]
Positive definite functions (Chapter 6) - Scattered Data Approximation
6 - Positive definite functions. Published online by Cambridge University Press: 22 February 2010. Holger Wendland. Show author details. Holger Wendland ...