Orthogonal basis
In linear algebra, an orthogonal basis for a subspace of an inner product space is a basis consisting of nonzero vectors that are pairwise orthogonal, meaning the inner product of any two distinct vectors in the set is zero.[1] Such bases are particularly valuable because they simplify the representation of vectors within the subspace as linear combinations of the basis vectors, with coefficients directly computable via inner products.[2] Orthogonal bases extend naturally to orthonormal bases by normalizing the vectors to have unit length, which further streamlines computations such as orthogonal projections onto the subspace.[1] A key property is that any orthogonal set of nonzero vectors is linearly independent, ensuring that an orthogonal basis with n vectors in an n-dimensional space fully spans it without redundancy.[2] The Gram-Schmidt process provides a systematic method to construct an orthogonal basis from any given basis, making these structures accessible for practical applications in solving systems of equations and data analysis.[1] These concepts are foundational in areas like signal processing and quantum mechanics, where orthogonality reflects physical independence, such as mutually perpendicular directions or non-interfering wave functions.[3] In finite-dimensional Euclidean spaces, orthogonal bases facilitate efficient matrix diagonalization[4] and least-squares approximations, underscoring their role in numerical methods and optimization.[2]Fundamentals
Definition
In an inner product space, which is a vector space over the real numbers \mathbb{R} or complex numbers \mathbb{C} equipped with an inner product \langle \cdot, \cdot \rangle—a positive-definite sesquilinear form (bilinear over \mathbb{R}) that satisfies \langle v, v \rangle \geq 0 for all v with equality if and only if v = 0—the inner product generalizes the dot product and induces a norm \|v\| = \sqrt{\langle v, v \rangle}.[5] A basis for such a vector space V is a set of vectors \{v_1, \dots, v_n\} that is linearly independent (no nontrivial linear combination equals zero) and spans V (every vector in V is a unique linear combination of them).[6] An orthogonal basis for an inner product space V is a basis \{v_1, \dots, v_n\} such that the vectors are pairwise orthogonal, meaning \langle v_i, v_j \rangle = 0 for all i \neq j, with each v_i \neq 0.[7] This orthogonality simplifies representations in V, as it ensures no overlap in the directions of the basis vectors under the inner product. An orthogonal basis differs from an orthonormal basis, where in addition \|v_i\| = 1 (or \langle v_i, v_i \rangle = 1) for all i; to obtain an orthonormal basis from an orthogonal one, normalize each vector by dividing by its norm: \hat{v}_i = v_i / \|v_i\|./10:_Inner_Product_Spaces/10.02:_Orthogonal_Sets_of_Vectors) For example, consider \mathbb{R}^2 as an inner product space with the standard dot product \langle (x_1, y_1), (x_2, y_2) \rangle = x_1 x_2 + y_1 y_2. The standard basis \{(1,0), (0,1)\} is orthogonal, since \langle (1,0), (0,1) \rangle = 0, and in fact orthonormal, as each has norm $1$.[8] The concept of orthogonal bases originated in 19th-century developments on Euclidean spaces, with Augustin-Louis Cauchy establishing key properties via the Cauchy-Schwarz inequality around 1821 and Bernhard Riemann advancing orthogonal curvilinear coordinate systems in his 1854 work on geometry.[9][10]Properties
In an inner product space, a key property of an orthogonal basis \{v_1, v_2, \dots, v_n\} is that any vector v in the span can be uniquely expressed as v = \sum_{i=1}^n \frac{\langle v, v_i \rangle}{\langle v_i, v_i \rangle} v_i, where the coefficients \frac{\langle v, v_i \rangle}{\|v_i\|^2} are known as the Fourier coefficients relative to the basis.[11] This expansion arises because orthogonality ensures that the projection of v onto each basis vector v_i is independent of the others, simplifying the decomposition process.[12] To see the uniqueness of these coefficients, suppose v = \sum c_i v_i = \sum d_i v_i. Then \sum (c_i - d_i) v_i = 0. Taking the inner product with v_j yields (c_j - d_j) \|v_j\|^2 = 0, so c_j = d_j since \|v_j\|^2 > 0. This follows from the linearity of the inner product and the orthogonality condition \langle v_i, v_j \rangle = 0 for i \neq j.[13] A significant consequence is Parseval's identity, which states that for any v in the span, \|v\|^2 = \sum_{i=1}^n \frac{|\langle v, v_i \rangle|^2}{\|v_i\|^2}. This identity reflects the Pythagorean theorem generalized to orthogonal decompositions, quantifying how the energy or norm of v distributes across the basis vectors.[14] In finite-dimensional inner product spaces, any orthogonal set is linearly independent, as the coefficient uniqueness argument above implies no nontrivial linear combination sums to zero.[13] Thus, if an orthogonal set spans the space, it forms a basis; completeness in this context simply requires spanning the entire space.[12] Orthogonal bases are preserved under transformations that preserve the inner product: in real inner product spaces, if \{v_i\} is orthogonal and U is an orthogonal matrix (satisfying U^T U = I), then \{U v_i\} is also orthogonal because \langle U v_i, U v_j \rangle = \langle v_i, v_j \rangle.[3] In complex inner product spaces, the analogous transformations are unitary matrices satisfying U^* U = I, where U^* is the conjugate transpose, and \langle U v_i, U v_j \rangle = \langle v_i, U^* U v_j \rangle = \langle v_i, v_j \rangle.[15] This preservation stems from such matrices maintaining inner products. While orthogonal bases are not unique—any rescaling of the vectors or reordering yields another orthogonal basis—they provide canonical decompositions via the Fourier coefficients, offering a standardized way to represent vectors in the space.[12] Orthonormal bases, where each \|v_i\| = 1, represent a normalized special case that simplifies these coefficients to \langle v, v_i \rangle.[16]Finite-Dimensional Inner Product Spaces
Construction Methods
In finite-dimensional inner product spaces, an orthogonal basis can be constructed from a given linearly independent set of vectors through successive orthogonalization, where each subsequent vector is adjusted by subtracting its projection onto the span of the previous orthogonal vectors.[17] Specifically, for a linearly independent list \{u_1, u_2, \dots, u_n\}, define v_1 = u_1 and for k = 2, \dots, n, set v_k = u_k - \sum_{j=1}^{k-1} \proj_{v_j} u_k, yielding an orthogonal set \{v_1, v_2, \dots, v_n\} that spans the same subspace.[17] This process relies on the orthogonal projection formula \proj_v u = \frac{\langle u, v \rangle}{\langle v, v \rangle} v, which ensures that u - \proj_v u is orthogonal to v.[17] To illustrate, consider constructing an orthogonal basis for \mathbb{R}^3 with the standard dot product from the set \{(1,0,0), (1,1,0), (1,1,1)\}. Set v_1 = (1,0,0). Then v_2 = (1,1,0) - \proj_{v_1} (1,1,0) = (1,1,0) - \frac{(1,1,0) \cdot (1,0,0)}{(1,0,0) \cdot (1,0,0)} (1,0,0) = (1,1,0) - (1,0,0) = (0,1,0). Finally, v_3 = (1,1,1) - \proj_{v_1} (1,1,1) - \proj_{v_2} (1,1,1) = (1,1,1) - (1,0,0) - (0,1,0) = (0,0,1), resulting in the orthogonal basis \{(1,0,0), (0,1,0), (0,0,1)\}.[18] The existence of such an orthogonal basis is guaranteed for any finite-dimensional inner product space: given any basis, the successive orthogonalization procedure produces an orthogonal basis for the space.[17] This result follows from the linear independence of orthogonal sets of nonzero vectors and the spanning properties preserved in the construction.[17] While theoretically robust, numerical implementations of this construction, particularly the classical variant, can encounter stability issues due to rounding errors leading to loss of orthogonality, especially for ill-conditioned bases; modified approaches or reorthogonalization mitigate these in practice, though the focus here remains on the theoretical method.[19]Orthonormalization Processes
The Gram-Schmidt process is a fundamental algorithm for constructing an orthonormal basis from a given linearly independent set of vectors in a finite-dimensional inner product space. It achieves this by iteratively orthogonalizing each subsequent vector against the previously constructed orthogonal set and then normalizing. The process was initially developed by Jørgen Pedersen Gram in his 1883 paper on least squares approximations and further formalized by Erhard Schmidt in 1907, who presented it as a method for orthogonalizing systems of functions, establishing its classical form.[20] The classical Gram-Schmidt algorithm proceeds as follows for a linearly independent set \{u_1, u_2, \dots, u_n\}:- Set v_1 = u_1 / \|u_1\|, where \|\cdot\| denotes the norm induced by the inner product.
- For k = 2 to n, compute the orthogonal component by subtracting the projections onto the previous orthonormal vectors: first form the remainder w_k = u_k - \sum_{i=1}^{k-1} \langle u_k, v_i \rangle v_i, where \langle \cdot, \cdot \rangle is the inner product, and then normalize v_k = w_k / \|w_k\|.