Fact-checked by Grok 2 weeks ago

Linear algebra

Linear algebra is the branch of mathematics that studies systems of linear equations and linear transformations between vector spaces, including their geometric interpretations and representations using matrices.^[1] It provides tools for compactly representing and solving sets of linear equations of the form Ax = b, where A is a matrix, x is a vector of unknowns, and b is a constant vector.^[2] At its core, linear algebra focuses on vector spaces—sets of vectors that can be added and scaled—and linear maps that preserve these operations, enabling the analysis of phenomena ranging from geometric configurations to approximate linear behaviors in science and engineering.^[1] Key concepts in linear algebra include vectors, which are elements of a vector space (such as column or row matrices representing points or directions), and matrices, which are rectangular arrays used to encode linear transformations or systems of equations.^[2] For instance, a linear equation like a_1 x_1 + a_2 x_2 + \dots + a_n x_n = b combines coefficients and variables, and multiple such equations form a system whose solutions correspond to intersections of hyperplanes in higher dimensions.^[1] Fundamental operations involve matrix multiplication, determinants (which indicate invertibility), eigenvalues and eigenvectors (describing scaling behaviors under transformations), and methods like Gaussian elimination for solving systems.^[3] These elements unify algebraic manipulation with geometric intuition, such as how rotations or projections act on spaces.^[2] The history of linear algebra traces back over 4,000 years to ancient civilizations, with Babylonians around 2000 BC developing methods to solve 2×2 systems of linear equations for practical problems like land division.^[3] By 200 BC, Chinese mathematicians in Nine Chapters on the Mathematical Art extended this to 3×3 systems using similar elimination techniques.^[3] In the 18th and 19th centuries, advancements accelerated: Carl Friedrich Gauss formalized Gaussian elimination in 1809 for astronomical calculations, while Arthur Cayley introduced matrix multiplication and the Cayley-Hamilton theorem in 1858, establishing matrices as central objects.^[3] The term "linear algebra" emerged in the early 20th century, with post-World War II computing revolutionizing its applications in numerical methods.^[3] Linear algebra's applications span numerous fields, serving as a foundation for machine learning algorithms that process high-dimensional data, computer graphics for rendering transformations, and quantum mechanics via Schrödinger's equation.^[1] In engineering and physics, it enables least-squares approximations for data fitting, Fourier analysis for signal processing, and optimization in control systems.^[3] Economists use it for input-output models, while statisticians rely on it for principal component analysis to reduce dimensionality.^[2] Its versatility underscores its role as an essential tool in modern technology, from search engines to artificial intelligence.^[1]

Historical Development

Early Origins

The earliest known methods for solving systems of linear equations date back to ancient Babylon around 2000–1800 BCE, where clay tablets record solutions to 2×2 systems using a method equivalent to completing the square or substitution for practical applications such as land measurement and resource distribution.^[4] These techniques were later advanced in ancient China with the text Nine Chapters on the Mathematical Art (Jiuzhang suanshu), compiled around 200 BCE, which presented a method akin to Gaussian elimination for resolving practical problems such as resource allocation and taxation.^[5] This procedure involved successive substitutions to reduce equations, organized in tabular form, and represented a concrete tool for handling multiple unknowns without abstract notation.^[6] In the late 17th century, Gottfried Wilhelm Leibniz advanced the study of linear systems by introducing the concept of determinants in 1693, framing them as tools for elimination in solving equations, such as expressing the ratio of coefficients in a 3x3 array to find unknowns.^[7] Building on this, 18th-century mathematicians refined these ideas: Leonhard Euler explored determinants in the context of curve intersections around 1750, contributing to their theoretical properties, while Joseph-Louis Lagrange developed methods for linear equations in his 1754 letter to Euler, emphasizing permutations and substitutions to resolve polynomial systems.^[8]^[9] Concurrently, Carl Friedrich Gauss formalized the elimination technique in 1809 within his astronomical work Theoria Motus Corporum Coelestium, applying it to least-squares problems for planetary orbits and establishing a rigorous algorithm for positive definite systems.^[10] The Renaissance period saw foundational algebraic advancements, including Gerolamo Cardano's 1545 Ars Magna, which systematized solutions to higher-degree equations and implicitly used coefficient arrays that prefigured matrix-like structures in equation solving. By the mid-19th century, these threads converged in more explicit forms: Hermann Grassmann introduced vector concepts and line geometry in his 1844 Die lineale Ausdehnungslehre, treating directed segments as extensible elements with addition and multiplication rules to model spatial relations.^[11] Arthur Cayley then formalized matrix theory in his 1858 memoir "A Memoir on the Theory of Matrices," coining the term "matrix" for coefficient arrays and defining operations like addition, multiplication, and inversion to unify linear transformations.^[12] These developments provided the concrete foundations that later enabled abstract vector space formulations in the early 20th century.

Abstract Formulation

The abstract formulation of linear algebra marked a pivotal shift in the early 20th century, moving from concrete computational methods—such as 19th-century matrix techniques for solving systems—to a rigorous, axiomatic framework that emphasized structural properties independent of specific representations. This development positioned linear algebra as a cornerstone of modern mathematics, enabling its integration with broader algebraic and analytic theories. Italian mathematician Giuseppe Peano initiated this transition in 1888 with the first axiomatic definition of a vector space in his treatise Calcolo geometrico secondo l'Ausdehnungslehre di H. Grassmann preceduto dalle operazioni della logica deduttiva, where he outlined the basic operations and properties of vectors in an abstract setting over the real numbers.^[13] Peano's axioms, which included addition, scalar multiplication, and associativity, provided a foundation that abstracted away from geometric or coordinate-based intuitions, though they initially received limited attention.^[14] Subsequent refinements expanded Peano's ideas to more general contexts. In 1905, Georg Hamel advanced the theory by formalizing linear independence and the concept of a basis (now known as a Hamel basis) in his paper "Eine Basis aller Zahlen und die unstetigen Lösungen der Funktionalgleichung f(x+y)=f(x)+f(y)," published in Mathematische Annalen. Hamel's work demonstrated the existence of bases in infinite-dimensional spaces over the rationals, using the axiom of choice to construct pathological examples like discontinuous additive functions on the reals. Meanwhile, David Hilbert's Grundlagen der Geometrie (1899) exerted significant influence by promoting axiomatic rigor across mathematical disciplines, inspiring abstract treatments of spaces that decoupled linear structures from Euclidean geometry. Hilbert's emphasis on undefined primitives and incidence axioms encouraged similar abstraction in linear algebra. Key milestones in the early 1900s further solidified this abstract paradigm. E. H. Moore and H. L. Smith developed the Moore-Smith convergence theorem around 1922 (building on Moore's earlier general analysis from 1900–1910), which generalized limits and continuity to directed sets in abstract topological spaces, facilitating the study of convergence in non-metric linear environments.^[15] In the 1920s, Emmy Noether's groundbreaking contributions to abstract algebra, particularly her 1921 paper "Idealtheorie in Ringbereichen" and work on non-commutative rings and modules, provided tools to view vector spaces as modules over fields, unifying linear algebra with ring theory and influencing its structural depth. These ideas tied into emerging functional analysis, notably through the 1907 Riesz–Fischer theorem, independently proved by Frigyes Riesz and Ernst Fischer, which established the completeness of L² spaces and laid the groundwork for Hilbert spaces as infinite-dimensional analogs of Euclidean spaces.^[16] Later, Stefan Banach refined Peano's axiomatization in his 1932 monograph Théorie des opérations linéaires, introducing normed linear spaces (Banach spaces) that combined vector space structure with metric completeness, enabling analytic applications.

Fundamental Structures

Vectors and Operations

In linear algebra, vectors are fundamental objects in finite-dimensional spaces over the real numbers \mathbb{R} or complex numbers \mathbb{C}. A vector in \mathbb{R}^n is an ordered n-tuple of real numbers, typically represented as a column with components v = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}, where each v_i \in \mathbb{R}. Similarly, a vector in \mathbb{C}^n is an ordered n-tuple of complex numbers, v = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}, with each v_i \in \mathbb{C}. Geometrically, vectors in \mathbb{R}^2 or \mathbb{R}^3 can be interpreted as directed arrows from the origin, where the components indicate displacements along the coordinate axes, capturing both magnitude (length of the arrow) and direction.^[17]^[18]^[19] Vector addition in \mathbb{R}^n or \mathbb{C}^n is performed component-wise: for vectors \mathbf{u} = (u_1, \dots, u_n) and \mathbf{v} = (v_1, \dots, v_n), the sum is \mathbf{u} + \mathbf{v} = (u_1 + v_1, \dots, u_n + v_n). This operation is commutative (\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}) and associative ((\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w})). Geometrically, adding arrows corresponds to placing the tail of the second at the head of the first to form a resultant displacement. Scalar multiplication by a scalar c \in \mathbb{R} (or \mathbb{C}) scales the vector: c\mathbf{u} = (c u_1, \dots, c u_n). This stretches or compresses the arrow by |c| and reverses direction if c < 0. Key properties include distributivity over vector addition (c(\mathbf{u} + \mathbf{v}) = c\mathbf{u} + c\mathbf{v}) and over scalars ((c + d)\mathbf{u} = c\mathbf{u} + d\mathbf{u}), as well as associativity (c(d\mathbf{u}) = (cd)\mathbf{u}).^[17]^[17]^[18] A linear combination of vectors \mathbf{v}_1, \dots, \mathbf{v}_k in \mathbb{R}^n or \mathbb{C}^n is \sum_{i=1}^k c_i \mathbf{v}_i, where each c_i is a scalar; this generalizes addition and scaling to form new vectors within the space. The zero vector \mathbf{0} = (0, \dots, 0) satisfies \mathbf{u} + \mathbf{0} = \mathbf{u} for any \mathbf{u}, and the negative -\mathbf{u} = (-1)\mathbf{u} allows subtraction via \mathbf{u} - \mathbf{v} = \mathbf{u} + (-\mathbf{v}), ensuring closure under these operations. Vectors can be represented as column matrices in a fixed coordinate system for computational purposes.^[17]^[17]^[18] In physics, vectors model quantities with direction, such as position vectors specifying a point's location from the origin, \mathbf{r} = (x, y, z), or velocity vectors \mathbf{v} = \frac{d\mathbf{r}}{dt} = \left( \frac{dx}{dt}, \frac{dy}{dt}, \frac{dz}{dt} \right), representing speed and direction of motion. For instance, a ship's displacement combines position vectors from successive legs of a journey to yield the net position.^[20]^[21]^[20]

Matrices and Arithmetic

In linear algebra, a matrix is defined as a rectangular array of numbers, symbols, or expressions arranged in rows and columns. It is typically denoted by an uppercase letter, such as A, with elements a_{ij} where i indexes the row and j indexes the column, forming an m \times n matrix if there are m rows and n columns./Chapter_1:_Systems_of_Linear_Equations/1.03:_Matrix_Algebra) Matrix addition is performed element-wise between two matrices of the same dimensions. For matrices A and B, both m \times n, the sum C = A + B has entries c_{ij} = a_{ij} + b_{ij}.^[22] Scalar multiplication involves multiplying each entry of a matrix by a scalar c \in \mathbb{R}, yielding D = cA where d_{ij} = c a_{ij}. These operations satisfy properties analogous to those of real numbers, including commutativity and associativity for addition, and distributivity of scalar multiplication over addition: c(A + B) = cA + cB and (c + d)A = cA + dA.^[23] Matrix multiplication combines two matrices A (of size m \times p) and B (of size p \times n) to produce a product C = AB of size m \times n, where each entry is given by the dot product of a row of A and a column of B:

c_{ik} = \sum_{j=1}^p a_{ij} b_{jk}.

This operation is not commutative in general (AB \neq BA), but it is associative: (AB)C = A(BC), and distributive over addition: A(B + C) = AB + AC and (A + B)C = AC + BC.^[24]^[25] The transpose of a matrix A, denoted A^T, is obtained by interchanging its rows and columns, so that if A is m \times n, then A^T is n \times m with entries (A^T)_{ji} = a_{ij}. A key property is that the transpose operation is involutory: (A^T)^T = A. Additionally, it distributes over addition and scalar multiplication: (A + B)^T = A^T + B^T and (cA)^T = c A^T, and reverses the order in multiplication: (AB)^T = B^T A^T.^[26]^[27] Matrices serve as fundamental tools for representing linear relations, such as coefficient matrices in systems of linear equations, where the entries capture the coefficients relating variables to constants. For instance, the system $2x + 3y = 5 and $4x - y = 1 has coefficient matrix \begin{pmatrix} 2 & 3 \\ 4 & -1 \end{pmatrix}. They also represent linear transformations, like rotation matrices in the plane; a counterclockwise rotation by angle \theta is given by \begin{pmatrix} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{pmatrix}, which maps vectors while preserving lengths and angles. Matrices are essential for compactly solving linear systems through arithmetic operations.^[28]^[29]

Solving Linear Systems

Gaussian Elimination

Gaussian elimination is a systematic algorithm for solving systems of linear equations by transforming the coefficient matrix into an upper triangular form through elementary row operations. Developed by Carl Friedrich Gauss in 1809 as part of his work on celestial mechanics, the method provides an efficient way to determine solutions, identify inconsistencies, or find the general solution involving free variables.^[30] The process begins by representing the system A\mathbf{x} = \mathbf{b}, where A is an m \times n coefficient matrix, \mathbf{x} is the vector of unknowns, and \mathbf{b} is the constant vector, in augmented matrix form [A \mid \mathbf{b}]. This augmented matrix combines the coefficients and constants into a single (m \times (n+1)) matrix for unified manipulation.^[31] The core of the algorithm relies on three elementary row operations: (1) swapping two rows, (2) multiplying a row by a nonzero scalar, and (3) adding a scalar multiple of one row to another row. These operations do not alter the solution set of the system and are applied to the augmented matrix to simplify it progressively.^[32] In the forward elimination phase, the goal is to produce zeros below each pivot position—the leading nonzero entry in a row—starting from the top-left corner and proceeding column by column. For the k-th column, the entry at position (k, k) serves as the pivot; if it is zero, rows are swapped to find a nonzero entry, ensuring the process continues unless no such entry exists. This elimination transforms the matrix into row echelon form, where each subsequent row begins with zeros farther to the right, resulting in an upper triangular matrix U such that the system becomes U\mathbf{x} = \mathbf{c}, with \mathbf{c} derived from the modified \mathbf{b}.^[33] To enhance numerical stability, especially with floating-point arithmetic, partial pivoting is employed: before using the pivot in column k, the row with the largest absolute value in that column (from row k downward) is swapped to the k-th position. This strategy minimizes error growth due to division by small pivots and is crucial for ill-conditioned systems, as it bounds the growth of matrix entries during elimination.^[34] Without pivoting, the algorithm can fail or produce inaccurate results if a zero or near-zero pivot is encountered early.^[35] Once upper triangular form is achieved, back-substitution solves the system starting from the last equation and working upward. For U\mathbf{x} = \mathbf{c}, the bottom equation gives x_n = c_n / u_{nn}, assuming u_{nn} \neq 0; then, substitute this into the (n-1)-th equation to solve for x_{n-1}, and continue similarly until all variables are determined. This phase requires O(n^2) operations.^[31] The row echelon form also reveals the rank of the matrix—the number of nonzero rows—which equals the number of pivot positions. If the rank is less than the number of variables n, free variables exist, corresponding to non-pivot columns, allowing a parameterized general solution. Inconsistent systems are detected if a row in the echelon form has a nonzero entry in the augmented part but zeros elsewhere in the coefficients, indicating $0 = d where d \neq 0. If the rank of A equals the rank of the augmented matrix but is less than n, infinitely many solutions exist; otherwise, the system has no solution.^[32] The computational complexity of Gaussian elimination, including partial pivoting, is O(n^3) arithmetic operations for an n \times n system, dominated by the forward elimination phase with approximately \frac{2}{3}n^3 multiplications and additions. This cubic scaling makes it practical for moderate-sized systems but motivates iterative methods for very large ones.^[36]

Matrix Inverses and LU Decomposition

A square matrix A is invertible if there exists another square matrix A^{-1}, called the inverse of A, such that A A^{-1} = I and A^{-1} A = I, where I is the identity matrix of the same dimension.^[37] This property establishes a one-to-one correspondence between invertible matrices and those of full rank, meaning the rank equals the dimension of the matrix, as the null space then has dimension zero and the associated linear map is bijective.^[38]^[39] For a $2 \times 2 matrix A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}, the inverse is given explicitly by

A^{-1} = \frac{1}{ad - bc} \begin{pmatrix} d & -b \\ -c & a \end{pmatrix},

provided the determinant ad - bc \neq 0, which ensures full rank.^[40] This formula arises from solving the equation A A^{-1} = I directly and serves as a concrete example of invertibility for small systems.^[41] Cramer's rule provides an alternative method to find the solution to a linear system A \mathbf{x} = \mathbf{b} using determinants, where for an invertible n \times n matrix A, the i-th component of the solution is x_i = \det(A_i) / \det(A), with A_i obtained by replacing the i-th column of A by \mathbf{b}.^[42] This approach is theoretically insightful but computationally inefficient for large n, as it requires evaluating n+1 determinants.^[43] LU decomposition factors a square matrix A as A = LU, where L is a lower triangular matrix with ones on the diagonal and U is an upper triangular matrix.^[44] This decomposition, derived from Gaussian elimination without pivoting, enables efficient solving of linear systems A \mathbf{x} = \mathbf{b} by first solving L \mathbf{y} = \mathbf{b} via forward substitution to find \mathbf{y}, then solving U \mathbf{x} = \mathbf{y} via back substitution.^[45] Forward substitution exploits the structure of L by solving sequentially from the top, while back substitution for U proceeds from the bottom, each requiring O(n^2) operations for an n \times n system after the initial O(n^3) factorization.^[46] In practice, direct LU decomposition may encounter numerical instability due to small pivots during elimination, so partial pivoting is used, yielding PA = LU where P is a permutation matrix that reorders rows to select the largest pivot in each column.^[47] This variant maintains the efficiency of forward and back substitution while improving accuracy in floating-point computations, making it a cornerstone of numerical linear algebra for solving multiple systems with the same A but varying right-hand sides.^[48]

Abstract Vector Spaces

Axioms and Examples

A vector space over a field F is a set V equipped with two operations: vector addition, which combines elements of V to produce another element in V, and scalar multiplication, which combines elements of F with elements of V to produce another element in V. These operations must satisfy a set of axioms that ensure the structure behaves consistently like the familiar Euclidean space.^[49] The axioms for vector addition are: closure under addition (for all u, v \in V, u + v \in V); associativity ((u + v) + w = u + (v + w) for all u, v, w \in V); commutativity (u + v = v + u for all u, v \in V); existence of a zero vector (there exists $0 \in V such that u + 0 = u for all u \in V); and existence of additive inverses (for each u \in V, there exists -u \in V such that u + (-u) = 0). For scalar multiplication, the axioms include: closure (for all a \in F and u \in V, a u \in V); distributivity over vector addition (a(u + v) = a u + a v and (a + b)u = a u + b u for all a, b \in F and u, v \in V); compatibility with field multiplication ((a b) u = a (b u) for all a, b \in F and u \in V); and the identity scalar ( $1 \cdot u = u for all u \in V, where 1 is the multiplicative identity in F). These ten axioms collectively define the algebraic structure, with proofs of derived properties like uniqueness of zero and inverses following from them.^[49]^[50] Common examples of vector spaces include the Euclidean space \mathbb{R}^n, where vectors are n-tuples of real numbers, addition is componentwise, and scalar multiplication uses real scalars; this satisfies all axioms, as verified by checking closure (e.g., (x_1, y_1) + (x_2, y_2) = (x_1 + x_2, y_1 + y_2) \in \mathbb{R}^2), associativity and commutativity from real number properties, zero as (0, \dots, 0), inverses as negatives componentwise, and distributivity similarly inherited from \mathbb{R}. Another example is the space P_n(F) of polynomials of degree at most n over field F, with addition and scalar multiplication defined termwise (e.g., (a_0 + a_1 x + \dots + a_n x^n) + (b_0 + b_1 x + \dots + b_n x^n) = (a_0 + b_0) + (a_1 + b_1) x + \dots); closure holds since degrees do not exceed n, and other axioms follow from field operations on coefficients. The space C[0,1] of continuous real-valued functions on [0,1] forms a vector space under pointwise addition and scalar multiplication ((f + g)(x) = f(x) + g(x), (a f)(x) = a f(x)), with the zero function as identity; axioms are satisfied due to continuity preserving addition and multiplication by constants. The trivial zero space \{0\} also qualifies, as it contains only the zero vector and operations are vacuously defined.^[51]^[52] The underlying field F must itself satisfy field axioms, such as those for the reals \mathbb{R}, complexes \mathbb{C}, or rationals \mathbb{Q}, all infinite fields where every nonzero element has a multiplicative inverse. Vector spaces can also be defined over finite fields, like the Galois field \mathbb{F}_p of p elements for prime p, which are finite in contrast to infinite ones but enable structures like finite-dimensional spaces over \mathbb{F}_2 (binary vectors). Infinite fields like \mathbb{Q} allow for vector spaces such as \mathbb{Q}^n, while finite fields support applications in coding theory.^[53]^[52] A subset W \subseteq V is a subspace if it contains the zero vector and is closed under vector addition and scalar multiplication (i.e., for all u, v \in W and a \in F, u + v \in W and a u \in W); this ensures W inherits the vector space axioms from V. For instance, the set of constant polynomials is a subspace of P_n(\mathbb{R}), as sums and scalar multiples remain constant. Matrices can represent linear maps between such spaces, preserving the operations.^[54]^[55]

Bases, Dimension, and Coordinates

In a vector space V over a field F, the linear span of a set S = \{ \mathbf{v}_1, \dots, \mathbf{v}_m \} consists of all linear combinations \sum_{i=1}^m a_i \mathbf{v}_i where a_i \in F.^[56] This span, denoted \operatorname{span}(S), forms the smallest subspace of V containing S.^[56] A set S = \{ \mathbf{v}_1, \dots, \mathbf{v}_m \} is linearly independent if the equation \sum_{i=1}^m a_i \mathbf{v}_i = \mathbf{0} implies a_i = 0 for all i.^[56] Equivalently, no vector in S is a linear combination of the others.^[56] A basis for a vector space V is a linearly independent set that spans V.^[56] It can also be characterized as a minimal spanning set, where removing any vector fails to span V, or a maximal linearly independent set, where adding any vector from V creates dependence.^[56] The dimension of a finite-dimensional vector space V, denoted \dim V, is the number of vectors in any basis of V.^[56] All bases of V have the same cardinality, as established by the Steinitz exchange lemma, which allows replacing vectors in a basis while preserving linear independence and spanning properties.^[56] Given a basis \{ \mathbf{e}_1, \dots, \mathbf{e}_n \} for V, any vector \mathbf{v} \in V can be uniquely expressed as \mathbf{v} = \sum_{i=1}^n x_i \mathbf{e}_i, where the scalars x_1, \dots, x_n are the coordinates of \mathbf{v} relative to this basis.^[56] These coordinates form a column vector [ \mathbf{v} ]_{\mathcal{B}} = \begin{pmatrix} x_1 \\ \vdots \\ x_n \end{pmatrix}. To change from coordinates in basis \mathcal{B} to basis \mathcal{B}', where the columns of matrix P are the \mathcal{B}-coordinates of the \mathcal{B}'-basis vectors, the transformation is [ \mathbf{v} ]_{\mathcal{B}'} = P^{-1} [ \mathbf{v} ]_{\mathcal{B}}.^[56] In \mathbb{R}^n, the standard basis is \{ \mathbf{e}_1, \dots, \mathbf{e}_n \}, where \mathbf{e}_i has a 1 in the i-th position and 0s elsewhere, yielding dimension n.^[56] The vector space of polynomials over \mathbb{R} of degree at most n, denoted \mathcal{P}_n(\mathbb{R}), has basis \{1, x, x^2, \dots, x^n\} and dimension n+1.^[56]

Linear Transformations

Definitions and Properties

A linear transformation (or linear map) T: V \to W between vector spaces V and W over the same field is a function that preserves vector addition and scalar multiplication: for all u, v \in V and scalars c, T(u + v) = T(u) + T(v) and T(cu) = c T(u).^[57] This additivity ensures that T maps linear combinations to linear combinations, maintaining the structure of the vector spaces. In finite-dimensional spaces with chosen bases, every linear transformation corresponds to multiplication by a matrix.^[57] Common examples illustrate these properties. A projection onto a line in \mathbb{R}^2, such as T(x) = \proj_v(x) = \frac{x \cdot v}{\|v\|^2} v for a fixed nonzero v, maps any vector to its closest point on the span of v, satisfying linearity via dot product homogeneity.^[58] A rotation in \mathbb{R}^2 by angle \theta, defined by

T\begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} x \cos \theta - y \sin \theta \\ x \sin \theta + y \cos \theta \end{pmatrix},

preserves lengths and angles, confirming linearity through trigonometric identities.^[59] Another example is the differentiation operator D: \mathcal{P}_n(\mathbb{R}) \to \mathcal{P}_{n-1}(\mathbb{R}) on polynomials of degree at most n, where D(p)(x) = p'(x); for instance, D(a_0 + a_1 x + a_2 x^2 + a_3 x^3) = a_1 + 2a_2 x + 3a_3 x^2, which is linear since the derivative of a sum is the sum of derivatives and scales with constants.^[60] The kernel of T, denoted \ker T = \{ v \in V \mid T(v) = 0 \}, is the set of vectors mapped to the zero vector in W, forming a subspace of V.^[61] The image of T, denoted \operatorname{im} T = \{ T(v) \mid v \in V \}, is the set of all outputs, forming a subspace of W.^[61] These subspaces capture essential aspects of T's behavior: the kernel measures "loss" of information, while the image measures the "reach" of T. A fundamental result relating these is the rank-nullity theorem: for a linear transformation T: V \to W with \dim V < \infty, \dim(\ker T) + \dim(\operatorname{im} T) = \dim V.^[62] Here, \dim(\operatorname{im} T) is the rank of T, and \dim(\ker T) is the nullity; the theorem intuitively shows that the dimension of the domain decomposes into the part "collapsed" to zero (nullity) and the part that spans the outputs (rank), providing a balance between injectivity and surjectivity potential.^[62] Linear transformations are closed under composition: if T: V \to W and S: W \to U are linear, then S \circ T: V \to U defined by (S \circ T)(v) = S(T(v)) is linear, as it preserves addition and scalar multiplication by applying the properties sequentially.^[63] The identity map \operatorname{Id}_V: V \to V, \operatorname{Id}_V(v) = v, is the canonical linear transformation that leaves every vector unchanged.^[57]

Kernel, Image, and Isomorphisms

In linear algebra, the kernel of a linear transformation T: V \to W between vector spaces V and W over the same field is defined as the set \ker T = \{ v \in V \mid T(v) = 0_W \}, where $0_W is the zero vector in W.^[64] This set forms a subspace of V, as it is closed under addition and scalar multiplication, inheriting these properties from the linearity of T.^[64] The dimension of \ker T, denoted \dim(\ker T), is called the nullity of T, which measures the "degeneracy" or redundancy in the mapping.^[65] The image of T, denoted \operatorname{im} T = \{ T(v) \mid v \in V \}, is the subset of W consisting of all vectors that are outputs under T.^[66] Like the kernel, the image is a subspace of W, since the linearity of T ensures closure under the vector space operations in W.^[66] The dimension of \operatorname{im} T, denoted \dim(\operatorname{im} T), is the rank of T, which quantifies the "span" or effective dimensionality of the outputs.^[65] A fundamental relation between these concepts is the rank-nullity theorem, which states that for a linear transformation T: V \to W where \dim V < \infty, \dim V = \operatorname{rank} T + \operatorname{nullity} T.^[67] To sketch the proof, extend a basis \{v_1, \dots, v_k\} of \ker T (where k = \operatorname{nullity} T) to a basis \{v_1, \dots, v_k, v_{k+1}, \dots, v_n\} of V (with n = \dim V). The set \{T(v_{k+1}), \dots, T(v_n)\} is linearly independent in \operatorname{im} T because any linear dependence would imply a non-trivial relation in V involving kernel elements, contradicting the basis extension; moreover, it spans \operatorname{im} T since every T(v) is a combination of these images. Thus, \dim(\operatorname{im} T) = n - k, yielding the theorem.^[67] A linear transformation T: V \to W is an isomorphism if it is bijective, meaning both injective (one-to-one, equivalent to \ker T = \{0\}) and surjective (onto, equivalent to \operatorname{im} T = W).^[68] For finite-dimensional spaces, T is an isomorphism if and only if \dim V = \dim W and \operatorname{rank} T = \dim V, by the rank-nullity theorem.^[68] The inverse T^{-1}: W \to V is also linear, as it preserves addition and scalar multiplication via the bijectivity and linearity of T.^[69] A simple example is the scaling map T: \mathbb{R} \to \mathbb{R} given by T(x) = c x where c \neq 0; this is bijective with inverse T^{-1}(y) = y/c, which is linear.^[70] The kernel plays a central role in constructing quotient spaces: for a subspace K \subseteq V, the quotient space V/K consists of cosets v + K = \{v + k \mid k \in K\} with vector space operations defined on cosets.^[71] Specifically, the natural projection \pi: V \to V / \ker T induces an isomorphism V / \ker T \cong \operatorname{im} T, where the isomorphism sends the coset v + \ker T to T(v); this is well-defined because T(v + k) = T(v) for k \in \ker T, bijective by the first isomorphism theorem for vector spaces, and linear by construction.^[71] These structures have key applications in solving equations involving linear transformations. The equation T(v) = w has a solution v \in V if and only if w \in \operatorname{im} T, with solutions forming an affine subspace v_0 + \ker T for any particular solution v_0.^[72] This solvability condition, tied to the rank, determines consistency in systems like Ax = b when T is represented by a matrix A.^[72]

Advanced Matrix Theory

Determinants and Their Properties

The determinant of an n \times n square matrix A = (a_{ij}) is a scalar-valued function \det: M_n(\mathbb{R}) \to \mathbb{R} that arises as the unique multilinear alternating form on the columns of A normalized to equal 1 on the identity matrix.^[73] Multilinearity implies that \det is linear in each column when the remaining columns are held fixed, so for vectors \mathbf{u}, \mathbf{v}, \mathbf{w} and scalar c, \det([\mathbf{u}, \mathbf{v}, \dots, c\mathbf{w} + \mathbf{z}, \dots]) = c \det([\mathbf{u}, \mathbf{v}, \dots, \mathbf{w}, \dots]) + \det([\mathbf{u}, \mathbf{v}, \dots, \mathbf{z}, \dots]).^[73] The alternating property ensures that interchanging any two columns changes the sign of the determinant, \det(A') = -\det(A) where A' results from such a swap, and thus \det(A) = 0 if any two columns are identical.^[73] These axioms characterize the determinant uniquely among functions from matrices to scalars.^[73] An explicit expression for the determinant is given by the Leibniz formula:

\det(A) = \sum_{\sigma \in S_n} \operatorname{sgn}(\sigma) \prod_{i=1}^n a_{i,\sigma(i)},

where the sum runs over all permutations \sigma in the symmetric group S_n, and \operatorname{sgn}(\sigma) is the sign of the permutation (+1 for even, -1 for odd)./02:_II._Linear_Algebra/04:_Determinants/4.02:_Laplace_Expansion_and_Leibniz_Formula) This formula originates from the work of Gottfried Wilhelm Leibniz, who in 1693 described a similar expansion in a letter to Guillaume de l'Hôpital while solving systems of linear equations, though the modern permutation form was formalized later.^[7] For practical computation, the cofactor expansion provides a recursive method along any row or column. The minor M_{ij} of A is the (n-1) \times (n-1) submatrix obtained by deleting row i and column j, and the cofactor is C_{ij} = (-1)^{i+j} \det(M_{ij}). Expanding along row i yields

\det(A) = \sum_{j=1}^n a_{ij} C_{ij}.

This holds for any choice of row or column, with the base case \det() = a for $1 \times 1 matrices.^[74] For block-partitioned matrices of the form \begin{bmatrix} A & B \\ C & D \end{bmatrix} where A is invertible, the Schur complement formula gives \det = \det(A) \det(D - C A^{-1} B), enabling computation via smaller blocks.^[75] Several algebraic properties follow from the multilinear alternating nature of the determinant. The multiplicative property states that for compatible square matrices A and B, \det(AB) = \det(A) \det(B), which extends to arbitrary products and implies \det(I) = 1 for the identity.^[76] Transposition preserves the determinant, \det(A^T) = \det(A), since row and column operations are symmetric under this operation.^[76] Scaling the matrix by a constant c multiplies the determinant by c^n, \det(cA) = c^n \det(A), reflecting the multilinearity in all entries.^[76] Geometrically, the determinant measures the signed scaling of volumes under the linear transformation T(\mathbf{x}) = A\mathbf{x}. Specifically, if the columns of A are vectors \mathbf{v}_1, \dots, \mathbf{v}_n, then \det(A) is the signed n-dimensional volume of the parallelepiped they span, with the sign indicating orientation (positive for right-handed bases, negative otherwise).^[77] The absolute value |\det(A)| gives the unsigned volume scaling factor: the image of the unit cube under T has volume |\det(A)|, and more generally, T scales the volume of any measurable set by this factor.^[77] If \det(A) = 0, the columns are linearly dependent, collapsing the parallelepiped to a lower-dimensional object of zero volume.^[77] The adjugate of A, denoted \operatorname{adj}(A), is the transpose of the cofactor matrix whose (i,j)-entry is C_{ji}. It satisfies A \cdot \operatorname{adj}(A) = \det(A) I = \operatorname{adj}(A) \cdot A, and for invertible A, the inverse is \operatorname{adj}(A) = \det(A) A^{-1}, or equivalently A^{-1} = \frac{1}{\det(A)} \operatorname{adj}(A).^[78] A matrix A is invertible if and only if \det(A) \neq 0.^[76]

Eigenvalues, Eigenvectors, and Diagonalization

In the context of linear algebra, an eigenvector of a square matrix A \in \mathbb{R}^{n \times n} (or more generally, an endomorphism T: V \to V on a finite-dimensional vector space V) is a non-zero vector v \in V such that T(v) = \lambda v for some scalar \lambda \in \mathbb{R}, where \lambda is called the corresponding eigenvalue.^[79] This equation implies that v is mapped to a scalar multiple of itself under T, preserving direction while possibly scaling the magnitude.^[80] Eigenvalues and eigenvectors capture intrinsic scaling directions of the transformation, distinct from the global volume scaling measured by the determinant.^[80] The eigenvalues of A are the roots of its characteristic polynomial, defined as p_A(\lambda) = \det(A - \lambda I), where I is the identity matrix.^[80] By the fundamental theorem of algebra, over the complex numbers, every square matrix has exactly n eigenvalues counting multiplicity, though they may be complex even for real matrices.^[79] The algebraic multiplicity of an eigenvalue \lambda is its multiplicity as a root of p_A(\lambda), while the geometric multiplicity is the dimension of the eigenspace \ker(A - \lambda I), which equals the maximum number of linearly independent eigenvectors associated with \lambda.^[80] A matrix A is diagonalizable if and only if there exists a basis of V consisting of eigenvectors of A, which occurs precisely when the algebraic multiplicity equals the geometric multiplicity for every eigenvalue.^[79] In this case, A is similar to a diagonal matrix D via A = P D P^{-1}, where the columns of P are the eigenvectors.^[80] For real symmetric matrices, the spectral theorem guarantees stronger properties: every such matrix A = A^T has n real eigenvalues (counting multiplicity) and is orthogonally diagonalizable, meaning A = Q D Q^T where Q is an orthogonal matrix (Q^T Q = I) whose columns are orthonormal eigenvectors, and D is diagonal with the eigenvalues on the main diagonal.^[81] This decomposition simplifies computations involving symmetric matrices, such as those arising in quadratic forms or principal component analysis, and ensures all eigenvalues are real and the eigenvectors form an orthonormal basis.^[82] Not all matrices are diagonalizable; for those that are not, the Jordan canonical form provides a canonical representation under similarity.^[83] Specifically, every square matrix over an algebraically closed field like \mathbb{C} is similar to a block-diagonal Jordan matrix, where each Jordan block is an upper-triangular matrix of the form

\begin{pmatrix} \lambda & 1 & 0 & \cdots & 0 \\ 0 & \lambda & 1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \ddots & \vdots \\ 0 & 0 & \cdots & \lambda & 1 \\ 0 & 0 & \cdots & 0 & \lambda \end{pmatrix}

of size equal to the geometric multiplicity or related invariant factors for that eigenvalue \lambda, with the number and sizes of blocks determined by the dimensions of generalized eigenspaces.^[83] The off-diagonal 1's account for the deficiency in the number of independent eigenvectors when algebraic multiplicity exceeds geometric multiplicity. Diagonalization has key applications in computing matrix powers and analyzing dynamical systems. For a diagonalizable matrix A = P D P^{-1}, the powers simplify to A^k = P D^k P^{-1}, where D^k is diagonal with entries \lambda_i^k, allowing efficient exponentiation without repeated multiplication.^[80] This is particularly useful in Markov chains, where the transition matrix P (a stochastic matrix with non-negative entries summing to 1 per column) models state probabilities, and P^k gives the k-step transition probabilities; the eigenvalue 1 (with multiplicity 1 for irreducible chains) corresponds to the stationary distribution, while other eigenvalues |\lambda_i| < 1 ensure convergence to equilibrium as k \to \infty.^[84]

Inner Product and Normed Spaces

Inner Products and Norms

An inner product on a real vector space V is a function \langle \cdot, \cdot \rangle: V \times V \to \mathbb{R} that satisfies positivity (\langle u, u \rangle \geq 0 for all u \in V, with equality if and only if u = 0), symmetry (\langle u, v \rangle = \langle v, u \rangle for all u, v \in V), and bilinearity (linear in each argument).^[85]^[86] For a complex vector space V over \mathbb{C}, the inner product maps to \mathbb{C}, is conjugate symmetric (\langle u, v \rangle = \overline{\langle v, u \rangle}), linear in the second argument, and conjugate linear in the first, while retaining positivity on the real part for the norm.^[87]^[88] A vector space equipped with such an inner product is called an inner product space, which introduces a geometric structure analogous to Euclidean space.^[89] A canonical example of an inner product is the dot product on \mathbb{R}^n, defined by \langle u, v \rangle = \sum_{i=1}^n u_i v_i, which satisfies the required axioms and corresponds to the standard Euclidean geometry.^[90]^[91] For \mathbb{C}^n, the standard inner product is \langle u, v \rangle = \sum_{i=1}^n u_i \overline{v_i}, incorporating the conjugate to ensure conjugate symmetry and positivity.^[87]^[88] These examples extend to function spaces, such as L^2 spaces where \langle f, g \rangle = \int f(x) \overline{g(x)} \, dx, but the finite-dimensional cases illustrate the core bilinear form.^[92] The inner product induces a norm on the space by \|v\| = \sqrt{\langle v, v \rangle}, which satisfies the norm axioms: non-negativity and positive definiteness (from the inner product's positivity), homogeneity (\|\alpha v\| = |\alpha| \|v\| from linearity and conjugate linearity), and the triangle inequality (\|u + v\| \leq \|u\| + \|v\|), the latter derived from the Cauchy-Schwarz inequality.^[93]^[94] This norm measures vector length and turns the inner product space into a normed space, enabling notions of distance and convergence. The Cauchy-Schwarz inequality states that for any u, v in an inner product space, |\langle u, v \rangle| \leq \|u\| \|v\|, with equality if and only if u and v are linearly dependent (one is a scalar multiple of the other).^[95]^[85] This bound arises from the non-negativity of \langle u - \lambda v, u - \lambda v \rangle \geq 0 for appropriate \lambda, and it underpins the triangle inequality as well as angle definitions via \cos \theta = \frac{\langle u, v \rangle}{\|u\| \|v\|}.^[96]^[97] Two vectors u, v in an inner product space are orthogonal if \langle u, v \rangle = 0.^[98]^[99] A set of vectors is orthogonal if every pair of distinct elements is orthogonal, and orthonormal if it is orthogonal and each vector has unit norm (\|e_i\| = 1).^[85] Orthonormal sets provide convenient bases for expansions, as seen in the Gram-Schmidt process for orthogonalizing arbitrary bases.^[100] For an orthonormal basis \{e_1, \dots, e_n\} of a finite-dimensional inner product space V, Parseval's identity holds: for any v \in V, \|v\|^2 = \sum_{i=1}^n |\langle v, e_i \rangle|^2.^[101] This equates the squared norm to the sum of squared coefficients in the basis expansion v = \sum \langle v, e_i \rangle e_i, preserving energy or length in orthogonal decompositions.^[102]

Orthogonal Bases and Gram-Schmidt Process

In an inner product space, an orthogonal basis is a basis consisting of pairwise orthogonal vectors, meaning the inner product of any two distinct basis vectors is zero.^[103] If, in addition, each basis vector has unit norm, the basis is orthonormal.^[103] Every finite-dimensional inner product space admits an orthonormal basis, which simplifies computations such as expansions and projections due to the orthogonality property.^[104] The orthogonal projection of a vector \mathbf{v} onto a nonzero vector \mathbf{u} is given by

\text{proj}_{\mathbf{u}} \mathbf{v} = \frac{\langle \mathbf{v}, \mathbf{u} \rangle}{\langle \mathbf{u}, \mathbf{u} \rangle} \mathbf{u},

where \langle \cdot, \cdot \rangle denotes the inner product.^[105] This formula yields the vector in the span of \mathbf{u} closest to \mathbf{v} in the norm induced by the inner product.^[105] The Gram-Schmidt process constructs an orthogonal basis from any linearly independent set \{\mathbf{v}_1, \dots, \mathbf{v}_n\} in an inner product space.^[104] The algorithm proceeds recursively: set \mathbf{u}_1 = \mathbf{v}_1, and for k = 2, \dots, n,

\mathbf{u}_k = \mathbf{v}_k - \sum_{i=1}^{k-1} \text{proj}_{\mathbf{u}_i} \mathbf{v}_k.

Normalizing each \mathbf{u}_k by dividing by its norm produces an orthonormal basis.^[104] First described by Jørgen Pedersen Gram in 1883 and formalized by Erhard Schmidt in 1907, the process is fundamental for orthogonalization.^[104] A matrix analogue is the QR decomposition, where any real or complex matrix A with full column rank factors as A = QR, with Q having orthonormal columns and R upper triangular with positive diagonal entries.^[106] This decomposition arises directly from applying Gram-Schmidt to the columns of A.^[106] The modified Gram-Schmidt variant, which subtracts projections incrementally, enhances numerical stability in finite-precision arithmetic compared to the classical version.^[104] Orthonormal bases enable efficient solutions to least-squares problems, minimizing \|A\mathbf{x} - \mathbf{b}\|_2. Using QR decomposition, the solution simplifies to solving R\mathbf{x} = Q^T \mathbf{b} for the upper triangular system, avoiding the ill-conditioned normal equations.^[106] This approach improves computational stability, with error bounds scaling as O(\epsilon \|A\|_2) where \epsilon is machine precision.^[104] Fourier series exemplify orthogonal expansions: a periodic function f(x) on [-\pi, \pi] expands as

f(x) = \frac{a_0}{2} + \sum_{n=1}^\infty \left( a_n \cos(nx) + b_n \sin(nx) \right),

where the basis \{1, \cos(nx), \sin(nx)\}_{n=1}^\infty is orthogonal with respect to the inner product \langle f, g \rangle = \int_{-\pi}^\pi f(x) g(x) \, dx.^[107] The coefficients are projections onto these basis functions, leveraging orthogonality for Parseval's identity relating energy to coefficients.^[107]

Duality and Bilinear Forms

Dual Spaces and Functionals

In linear algebra, the dual space of a vector space V over a field F, denoted V^*, is the set of all linear functionals on V, that is, all linear maps from V to F.^[108] These functionals form a vector space under pointwise addition and scalar multiplication: for \phi, \psi \in V^* and c \in F, (\phi + \psi)(v) = \phi(v) + \psi(v) and (c\phi)(v) = c \cdot \phi(v) for all v \in V.^[109] If V is finite-dimensional with dimension n, then \dim V^* = n.^[110] Given a basis \{e_1, \dots, e_n\} for V, there exists a unique dual basis \{e^1, \dots, e^n\} for V^* such that e^i(e_j) = \delta_{ij}, where \delta_{ij} is the Kronecker delta (1 if i = j, 0 otherwise).^[111] The elements e^i are called coordinate functionals, as for any v = \sum_{i=1}^n x_i e_i \in V, the coordinate x_i = e^i(v).^[109] This duality allows vectors in V to be represented via their evaluations under functionals in V^*, providing a way to express linear algebra concepts without additional structure like inner products. For a subspace W \subseteq V, the annihilator of W, denoted W^0, is the subspace of V^* consisting of all functionals that vanish on W: W^0 = \{\phi \in V^* \mid \phi(w) = 0 \ \forall w \in W\}.^[108] If \dim V = n and \dim W = k, then \dim W^0 = n - k.^[112] This construction captures the "orthogonal" complement in the algebraic sense, independent of any metric. A concrete example is the dual space of \mathbb{R}^n, where vectors are typically column vectors; elements of (\mathbb{R}^n)^* can be identified with row vectors \xi = [\xi_1, \dots, \xi_n], acting via \xi(v) = \xi_1 v_1 + \dots + \xi_n v_n for v = [v_1, \dots, v_n]^T \in \mathbb{R}^n.^[113] Another example is the space of n \times n matrices over F, denoted M_n(F), whose dual includes the trace functional \operatorname{Tr}: M_n(F) \to F, defined by \operatorname{Tr}(A) = \sum_{i=1}^n a_{ii} for A = (a_{ij}), which is linear and satisfies \operatorname{Tr}(AB) = \operatorname{Tr}(BA).^[114] For finite-dimensional V, there is a natural isomorphism \iota: V \to V^{**}, where V^{**} is the double dual (the dual of V^*), given by \iota(v)(\phi) = \phi(v) for \phi \in V^*.^[114] This embedding is an isomorphism, meaning every continuous linear functional on V^* arises from evaluation at some vector in V, and it preserves the vector space structure.^[115] Dual maps, which transpose linear transformations between spaces to maps between their duals, arise naturally from this framework.^[108]

Dual Maps and Adjoints

In linear algebra, given a linear transformation T: V \to W between vector spaces over the same field, the dual map T^*: W^* \to V^* is defined by (T^* f)(v) = f(T v) for all f \in W^* and v \in V, where V^* and W^* denote the dual spaces of linear functionals on V and W, respectively. This construction preserves linearity, as T^*(\alpha f + \beta g) = \alpha T^* f + \beta T^* g follows directly from the definition and the linearity of f and T.^[108] The dual map induces a contravariant functoriality, reversing the direction of arrows in diagrams of linear maps. When V and W are finite-dimensional, choosing bases for V and W induces bases for their duals, and the matrix representation of T^* with respect to these dual bases is the transpose of the matrix representation of T.^[116] Specifically, if A is the matrix of T with respect to standard bases, then the matrix of T^* is A^T, reflecting how the dual map pulls back functionals through the transformation. This correspondence underscores the role of the transpose in bridging matrix algebra and duality.^[117] In the context of inner product spaces, where V and W are equipped with inner products \langle \cdot, \cdot \rangle_V and \langle \cdot, \cdot \rangle_W, the adjoint T^*: W \to V of a linear map T: V \to W is defined by the relation \langle T u, v \rangle_W = \langle u, T^* v \rangle_V for all u \in V and v \in W.^[118] The adjoint exists uniquely in finite dimensions and satisfies linearity: T^*(\alpha w_1 + \beta w_2) = \alpha T^* w_1 + \beta T^* w_2. A map T is self-adjoint if T = T^*, meaning \langle T u, v \rangle = \langle u, T v \rangle for all u, v.^[119] Key properties of the adjoint include the reverse composition rule (S T)^* = T^* S^* for composable linear maps S and T, and the double adjoint property (T^*)^* = T when V and W have compatible inner products.^[120] Additionally, a linear map T is unitary (or orthogonal in the real case) if T^* T = I, preserving the inner product via \langle T u, T v \rangle = \langle u, v \rangle. Self-adjoint maps have real eigenvalues, and the spectral theorem guarantees an orthonormal basis of eigenvectors for finite-dimensional self-adjoint operators, facilitating diagonalization.^[121] Examples illustrate these concepts vividly. Consider the differentiation operator D = \frac{d}{dx} on the space of smooth functions with compact support; its adjoint is D^* = -\frac{d}{dx}, derived via integration by parts: \int (D u) v \, dx = -\int u (D v) \, dx assuming boundary terms vanish.^[122] Another example is the Fourier transform \mathcal{F} on L^2(\mathbb{R}), which is unitary with \mathcal{F}^* = \mathcal{F}^{-1}, preserving norms and enabling the diagonalization of self-adjoint operators like the Laplacian through multiplication by frequencies in the Fourier domain.^[123]

Bilinear Forms

A bilinear form on a vector space V over a field F is a function B: V \times V \to F that is linear in each argument: for all u, v, w \in V and c \in F, B(u + c v, w) = B(u, w) + c \, B(v, w), \quad B(u, v + c w) = B(u, v) + c \, B(u, w). Bilinear forms connect to dual spaces by inducing a linear map L_B: V \to V^* given by (L_B u)(v) = B(u, v) for u, v \in V. The form B is non-degenerate if L_B is injective (equivalently, B(u, v) = 0 for all v implies u = 0); for finite-dimensional V, non-degeneracy implies L_B is an isomorphism V \cong V^*. With respect to a basis \{e_1, \dots, e_n\} of V, B has a matrix representation (b_{ij}) where b_{ij} = B(e_i, e_j), and for coordinate vectors \mathbf{u}, \mathbf{v}, B(u, v) = \mathbf{u}^T (b_{ij}) \mathbf{v}. Symmetric bilinear forms (B(u,v) = B(v,u)) over \mathbb{R} include inner products when positive definite.^[124]

Geometric Interpretations

Transformations in Euclidean Space

In Euclidean space \mathbb{R}^n with the standard inner product, linear transformations T: \mathbb{R}^n \to \mathbb{R}^n represented by matrices act geometrically by stretching, rotating, or shearing vectors while preserving vector addition and scalar multiplication. These transformations maintain the linearity inherent to the space's vector structure, allowing for interpretations such as mappings that align with the Euclidean metric's emphasis on distances and angles. Orthogonal transformations, a key subclass, preserve the Euclidean norm \|v\| = \sqrt{v^T v} for all vectors v, ensuring that lengths and angles remain unchanged under the mapping.^[125] An orthogonal transformation is given by an orthogonal matrix Q satisfying Q^T Q = I_n, where I_n is the n \times n identity matrix, which implies \det(Q) = \pm 1.^[126] These matrices form the orthogonal group O(n), comprising proper rotations in the special orthogonal group SO(n) (where \det(Q) = 1) and improper rotations including reflections (where \det(Q) = -1).^[125] In two dimensions, a counterclockwise rotation by angle \theta is represented by the matrix

\begin{pmatrix} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{pmatrix},

which rotates vectors around the origin while preserving orientation and distances.^[127] Not all linear transformations are orthogonal; for instance, shear and scaling distort angles and relative lengths. A horizontal shear in \mathbb{R}^2 by factor k has matrix \begin{pmatrix} 1 & k \\ 0 & 1 \end{pmatrix}, sliding points parallel to the x-axis based on their y-coordinate, while a uniform scaling by factor s uses \begin{pmatrix} s & 0 \\ 0 & s \end{pmatrix}, enlarging or shrinking distances from the origin.^[128] Affine transformations extend linear ones by including translations, mapping v \mapsto Av + b for some vector b, and thus do not fix the origin unless b = 0, unlike purely linear transformations that always map the origin to itself.^[129] Similarity transformations arise in changes of coordinates, where a new basis represented by invertible matrix P transforms the matrix A of a linear map to A' = P^{-1} A P, preserving eigenvalues that indicate principal axes of deformation.^[130] Any invertible matrix A admits a polar decomposition A = QP, where Q is orthogonal and P is positive semi-definite symmetric, separating the rotational component Q from the stretching P = \sqrt{A^T A}.^[131] In computer graphics, these transformations enable efficient manipulation of 3D models, such as applying rotations and scalings to vertices for rendering scenes while maintaining geometric fidelity.^[128]

Projections and Decompositions

In linear algebra, the orthogonal projection of a vector \mathbf{v} onto a subspace W of a Euclidean space is defined as the vector \mathbf{p} \in W that minimizes the Euclidean distance \| \mathbf{v} - \mathbf{p} \|, equivalently solving \mathbf{p} = \arg\min_{\mathbf{w} \in W} \| \mathbf{v} - \mathbf{w} \|.^[132] This projection operator P, represented as a linear transformation, satisfies the idempotence property P^2 = P and self-adjointness P^* = P with respect to the inner product, ensuring that the projection is orthogonal and the error \mathbf{v} - P\mathbf{v} lies in the orthogonal complement of W.^[133] When W has an orthonormal basis \{ \mathbf{u}_i \}, the projection matrix takes the explicit form

P = \sum_i |\mathbf{u}_i\rangle \langle \mathbf{u}_i|,

where the sum is over the basis vectors, allowing computation via inner products: P\mathbf{v} = \sum_i \langle \mathbf{u}_i, \mathbf{v} \rangle \mathbf{u}_i.^[134] Such orthonormal bases can be constructed from arbitrary bases using the Gram-Schmidt process. These projections decompose any vector as \mathbf{v} = P\mathbf{v} + (I - P)\mathbf{v}, partitioning the space into W and its orthogonal complement, which is fundamental for least-squares problems and error analysis.^[132] For general matrices, the singular value decomposition (SVD) extends projection concepts to rectangular arrays, factoring an m \times n matrix A as A = U \Sigma V^T, where U and V are orthogonal matrices, and \Sigma is a diagonal matrix with non-negative singular values \sigma_1 \geq \sigma_2 \geq \cdots \geq 0 on the diagonal.^[135] The singular values quantify the "stretch" factors along principal directions, with the rank of A equal to the number of positive singular values, enabling low-rank approximations by truncating smaller singular values.^[136] The Eckart-Young theorem establishes that the best rank-k approximation to A in the Frobenius or spectral norm is obtained by retaining only the top k singular values and corresponding singular vectors, yielding A_k = U_k \Sigma_k V_k^T, which minimizes \| A - B \| over all rank-k matrices B.^[137] This truncation provides optimal dimensionality reduction, preserving essential structure while discarding noise, as in principal component analysis where data variance is captured by leading singular values.^[135] SVD also facilitates computing the Moore-Penrose pseudoinverse A^+ = V \Sigma^+ U^T, where \Sigma^+ inverts the non-zero singular values and sets zeros for others, solving underdetermined or overdetermined systems in a least-squares sense with minimal norm solutions.^[138] Applications include data compression, where low-rank approximations reduce storage without significant information loss, and signal processing for noise filtering via singular value thresholding.^[135] In contrast to orthogonal projections, which minimize distance perpendicularly, oblique projections onto a subspace along a non-orthogonal direction do not satisfy self-adjointness and yield different minimizers, though they share idempotence.^[139]

Applications

In Physics and Engineering

Linear algebra provides essential tools for modeling and analyzing physical systems in physics and engineering, often through linear approximations that simplify complex dynamics into matrix equations and vector spaces. In the study of vibrations, mass-spring systems are represented as coupled oscillators whose equations of motion form a second-order linear differential equation, reducible to an eigenvalue problem for determining normal modes and natural frequencies. For a system with n masses and springs, the dynamics are governed by M \ddot{\mathbf{q}} + K \mathbf{q} = 0, where M is the mass matrix and K is the stiffness matrix; assuming solutions of the form \mathbf{q}(t) = \mathbf{v} e^{i\omega t}, this yields the generalized eigenvalue problem K \mathbf{v} = \omega^2 M \mathbf{v}, with eigenvalues \omega^2 corresponding to squared frequencies and eigenvectors \mathbf{v} describing the mode shapes.^[140] In quantum mechanics, the state of a physical system is described by a vector in a Hilbert space, an infinite-dimensional inner product space that generalizes finite-dimensional Euclidean spaces to accommodate continuous spectra. Observables, such as position or energy, are represented by self-adjoint operators on this space, whose eigenvalues yield possible measurement outcomes and eigenvectors the corresponding states. This framework, formalized in the 1930s, ensures that probabilities are given by inner products and that expectation values align with Born's rule. Electrical circuits are analyzed using Kirchhoff's laws, which translate network topologies into systems of linear equations solvable via matrix methods. Kirchhoff's current law (KCL) equates the sum of currents at a node to zero, while Kirchhoff's voltage law (KVL) equates the sum of voltages around a loop to zero; for a network with n nodes and branches, these yield A \mathbf{i} = 0 for currents \mathbf{i} under KCL, where A is the incidence matrix. In the frequency domain, impedance matrices Z relate nodal voltages \mathbf{v} to currents \mathbf{i} via \mathbf{v} = Z \mathbf{i}, enabling efficient analysis of AC circuits through inversion or factorization.^[141] Control theory employs state-space models to represent dynamic systems, with the linear time-invariant form \dot{\mathbf{x}} = A \mathbf{x} + B \mathbf{u}, \mathbf{y} = C \mathbf{x} + D \mathbf{u}, where \mathbf{x} is the state vector, \mathbf{u} the input, and A, B, C, D constant matrices. Controllability, the ability to drive the state from any initial value to any final value in finite time using inputs \mathbf{u}, is determined by the Kalman rank condition: the controllability matrix \mathcal{C} = [B \ AB \ \cdots \ A^{n-1}B] must have full rank n, equal to the state dimension. This criterion, introduced in the early 1960s, underpins stability analysis and controller design in applications like aerospace and robotics.^[142] In signal processing, discrete convolution of signals \mathbf{x} and \mathbf{h} is equivalent to multiplication by a Toeplitz matrix H formed from \mathbf{h}, yielding \mathbf{y} = H \mathbf{x}, which captures linear filtering operations. Circulant matrices, a special Toeplitz subclass approximating convolutions under periodic boundary conditions, are diagonalized by the discrete Fourier transform (DFT) matrix F, where F^{-1} C F = \Lambda with \Lambda diagonal containing the eigenvalues as DFT coefficients of the first row; this enables fast convolution via FFT, reducing complexity from O(n^2) to O(n \log n). Structural engineering utilizes the finite element method (FEM) to approximate solutions to partial differential equations governing deformation, discretizing continuous structures into finite elements connected at nodes. For linear elasticity, the global system is K \mathbf{u} = \mathbf{f}, where K is the assembled stiffness matrix relating nodal displacements \mathbf{u} to forces \mathbf{f}; each element contributes a local stiffness matrix derived from strain energy, ensuring equilibrium and compatibility. This matrix-based approach, developed in the mid-20th century, facilitates simulation of complex structures like bridges and aircraft under loads.

In Computer Science and Data Analysis

Linear algebra plays a foundational role in computer science and data analysis, enabling efficient representations, computations, and optimizations over high-dimensional data and structures. Core operations like matrix multiplication and eigenvalue decomposition underpin algorithms for processing graphs, rendering visuals, ranking information, reducing dimensions, training models, and solving optimization problems. These applications leverage the algebraic structure of vectors and matrices to handle discrete, large-scale computations, often emphasizing numerical stability and scalability in implementations. In graph theory, linear algebra provides tools for analyzing network structures through matrix representations. The adjacency matrix A of an undirected graph G = (V, E) is a symmetric |V| \times |V| matrix where A_{ij} = 1 if there is an edge between vertices i and j, and 0 otherwise, capturing the connectivity of the graph.^[143] The graph Laplacian L = D - A, where D is the diagonal degree matrix, has eigenvalues that reveal properties like connectivity and clustering; the second smallest eigenvalue, known as the algebraic connectivity or Fiedler value, measures how well-connected the graph is, with higher values indicating stronger overall connectivity.^[143] These spectral properties, rooted in the Rayleigh quotient for quadratic forms, enable algorithms for partitioning and embedding graphs into lower-dimensional spaces.^[144] Computer graphics relies on linear algebra for geometric transformations in rendering pipelines. Homogeneous coordinates extend 3D points (x, y, z) to 4D vectors (x, y, z, 1), allowing affine transformations like translation, rotation, scaling, and perspective projection to be represented uniformly as 4x4 matrix multiplications.^[145] In the graphics pipeline, a sequence of such matrices—model, view, and projection—transforms vertex coordinates from world space to screen space, facilitating efficient rasterization and clipping while preserving projective geometry.^[145] The PageRank algorithm, central to web search engines, models the web as a directed graph and uses linear algebra to compute importance scores. It finds the principal eigenvector of the Google matrix G, a stochastic matrix derived from the adjacency matrix normalized by out-degrees with added teleportation for damping, satisfying \pi = G \pi where \pi is the PageRank vector representing steady-state probabilities of random walks.^[146] This eigenvector centrality approach ranks pages by iteratively solving the linear system, powering scalable ranking in large graphs like the early web index.^[146] Principal component analysis (PCA) applies singular value decomposition (SVD) to the covariance matrix for dimensionality reduction in data analysis. For a centered data matrix X \in \mathbb{R}^{n \times p}, the covariance \Sigma = \frac{1}{n-1} X^T X is decomposed via eigendecomposition, or equivalently, the SVD X = U \Sigma V^T yields principal components as columns of V, with variances given by squared singular values, allowing projection onto top components to capture maximum variance while minimizing information loss.^[147] Originally formulated to find best-fit lines and planes to data points, PCA reduces high-dimensional datasets for visualization and noise reduction, as in gene expression analysis.^[147] In neural networks, linear layers form the building blocks of deep architectures, performing affine transformations via matrix multiplication. A fully connected layer computes y = W x + b, where W is the weight matrix, x the input vector, and b the bias, enabling feature extraction through composition of such layers.^[148] Backpropagation, the standard training algorithm, propagates errors backward using the adjoint (transpose) of the weight matrices to compute gradients efficiently via the chain rule, as in the update \frac{\partial L}{\partial W} = \frac{\partial L}{\partial y} x^T, allowing optimization of millions of parameters in models like convolutional networks.^[148] Optimization problems in computer science often reduce to linear algebra for solution. Linear programming solves \max c^T x subject to A x \leq b, x \geq 0 using the simplex method, which pivots through basic feasible solutions by Gaussian elimination on the constraint matrix to find the optimal vertex of the feasible polytope.^[149] Introduced for resource allocation, it scales to thousands of variables via revised simplex variants. Least squares minimization, \min \| A x - b \|_2^2, is solved by the normal equations A^T A x = A^T b, yielding the pseudoinverse for overdetermined systems, foundational for regression and curve fitting in data analysis.^[150]

Generalizations and Extensions

Modules and Non-Vector Spaces

A module over a ring R is an abelian group M equipped with a scalar multiplication operation R \times M \to M, (r, m) \mapsto r \cdot m, satisfying the axioms: (r_1 + r_2) \cdot m = r_1 \cdot m + r_2 \cdot m, r \cdot (m_1 + m_2) = r \cdot m_1 + r \cdot m_2, (r_1 r_2) \cdot m = r_1 \cdot (r_2 \cdot m), and $1_R \cdot m = m for all r_1, r_2 \in R, m, m_1, m_2 \in M, where $1_R is the multiplicative identity of R.^[151] Unlike vector spaces, modules do not require division by nonzero scalars, allowing for more general structures where the ring R may have zero divisors or lack inverses.^[152] Vector spaces can be viewed as modules over fields, providing a special case where every nonzero scalar has an inverse.^[151] Examples of modules include \mathbb{Z}-modules, which are precisely the abelian groups under addition, with scalar multiplication given by repeated addition.^[151] Another example is the polynomial ring k viewed as a module over itself, where k is a field and multiplication by elements of k acts by polynomial multiplication.^[153] A free module over R is a module that is isomorphic to a direct sum of copies of R itself, M \cong \bigoplus_{i \in I} R for some index set I, and thus admits a basis \{e_i\}_{i \in I} such that every element of M is a unique finite R-linear combination of the basis elements.^[154] Free modules generalize free abelian groups and vector spaces with bases, but over general rings, not every finitely generated projective module is free.^[155] In contrast, a torsion module over an integral domain R is one where every nonzero element m \in M is annihilated by some nonzero r \in R, meaning r \cdot m = 0.^[156] For R = \mathbb{Z}, the module \mathbb{Z}/n\mathbb{Z} is torsion, as

n \cdot {{grok:render&&&type=render_inline_citation&&&citation_id=1&&&citation_type=wikipedia}} = {{grok:render&&&type=render_inline_citation&&&citation_id=0&&&citation_type=wikipedia}}

for the generator

{{grok:render&&&type=render_inline_citation&&&citation_id=1&&&citation_type=wikipedia}}

, and every element is a multiple of

{{grok:render&&&type=render_inline_citation&&&citation_id=1&&&citation_type=wikipedia}}

.^[156] Homology in the context of modules arises from chain complexes, sequences of modules and homomorphisms \cdots \to M_{n+1} \xrightarrow{d_{n+1}} M_n \xrightarrow{d_n} M_{n-1} \to \cdots with d_n \circ d_{n+1} = 0 for all n, where the homology groups are H_n(M_\bullet) = \ker d_n / \operatorname{im} d_{n+1}.^[157] A chain complex is exact if all homology groups vanish, meaning \operatorname{im} d_{n+1} = \ker d_n at each degree, analogous to the kernel-image exactness in linear maps.^[157] Concepts from linear algebra, such as the rank-nullity theorem, generalize to modules through tools like projective resolutions; for a module M, a short exact sequence $0 \to P_1 \to P_0 \to M \to 0with free (or projective) modulesP_iallows the minimal number of generators ofMto be computed as the rank ofP_0$ in a minimal free resolution.^[157] Applications of these concepts appear in homological algebra, particularly in topology, where chain complexes of modules compute topological invariants like singular homology groups of spaces, enabling classifications of manifolds and computations of fundamental groups via exact sequences.^[158]

Tensors and Multilinear Algebra

In multilinear algebra, a multilinear map is a function that is linear in each of its arguments separately.^[159] For vector spaces V_1, \dots, V_k and W_1, \dots, W_m over a field F, a tensor of type (k, m) is an element of the space V_1^* \otimes \cdots \otimes V_k^* \otimes W_1 \otimes \cdots \otimes W_m, where V_i^* denotes the dual space of V_i, corresponding to a multilinear map from V_1 \times \cdots \times V_k \times W_1^* \times \cdots \times W_m^* to F.^[159] This perspective unifies scalars (type (0,0)), vectors (type (1,0) or (0,1)), and matrices (type (1,1)) as special cases of tensors.^[160] The tensor product of two vector spaces V and W over F, denoted V \otimes W, is constructed as the quotient of the free vector space on V \times W by the relations enforcing bilinearity, yielding a universal space for bilinear maps from V \times W to any vector space X.^[161] If \{e_i\} and \{f_j\} are bases for V and W, respectively, then \{e_i \otimes f_j\} forms a basis for V \otimes W, and the tensor product map extends bilinearly to all elements.^[162] Higher-order tensor products, such as V_1 \otimes \cdots \otimes V_k, generalize this construction multilinearly.^[161] Tensor contraction is a multilinear operation that pairs a covariant index of one tensor with a contravariant index of another (or within the same tensor), summing over that index to reduce the total rank by 2, analogous to the trace for matrices.^[163] For simple tensors u \otimes v \in V^* \otimes W and x \otimes y \in V \otimes W^*, the contraction yields \langle u \otimes v, x \otimes y \rangle = \langle u, x \rangle \langle v, y \rangle, where \langle \cdot, \cdot \rangle denotes duality pairing.^[163] The outer product of vectors u \in V and v \in W^* produces the rank-1 tensor u \otimes v \in V \otimes W^*, which acts as a linear map from W to V via w \mapsto \langle v, w \rangle u.^[164] In matrix terms, if u is a column vector and v a row vector, their outer product u v is a rank-1 matrix.^[164] In differential geometry, the Riemann curvature tensor serves as a canonical example of a (0,4)-tensor on a Riemannian manifold, defined as a multilinear map R: T_pM \times T_pM \times T_pM \to T_pM (or its fully covariant version via the metric), quantifying intrinsic curvature at point p.^[165] Tensors find applications in statistics, where the covariance of multivariate data generalizes to a (0,2)-tensor capturing bilinear dependencies, and higher-order tensors represent multi-way arrays for analyzing interactions in multidimensional datasets, such as in psychometrics or signal processing.^[160] For instance, tensor decompositions like the CP model factor multi-way arrays into sums of rank-1 tensors to reveal latent structures in data.^[160]

Infinite-Dimensional and Topological Spaces

Linear algebra extends naturally to infinite-dimensional settings, where vector spaces can have uncountably infinite bases, leading to structures essential for analysis and applications in physics. Key examples include the sequence spaces \ell^p for $1 \leq p \leq \infty, consisting of all sequences (x_n) of complex numbers such that \sum |x_n|^p < \infty (or \sup |x_n| < \infty for p = \infty), equipped with the norm \|x\|_p = \left( \sum |x_n|^p \right)^{1/p}.^[166] These spaces are infinite-dimensional and serve as models for studying convergence and completeness in functional analysis. Another fundamental example is the space C[0,1] of continuous real-valued functions on the interval [0,1], normed by the supremum \|f\|_\infty = \sup_{t \in [0,1]} |f(t)|, which captures the behavior of smooth functions in infinite dimensions.^[166] To handle limits and continuity in these spaces, a topology is introduced that is compatible with the vector space operations, resulting in a topological vector space. This is a vector space over \mathbb{R} or \mathbb{C} endowed with a topology such that the maps X \times X \to X given by addition (x,y) \mapsto x + y and \mathbb{K} \times X \to X given by scalar multiplication (\lambda, x) \mapsto \lambda x (where \mathbb{K} is the scalar field with its standard topology) are continuous.^[167] Such topologies enable the study of convergence, allowing infinite-dimensional analogs of finite-dimensional concepts like boundedness and compactness. A Banach space is a complete normed topological vector space, meaning every Cauchy sequence converges to an element within the space.^[168] Both \ell^p and C[0,1] are Banach spaces, providing rigorous frameworks for infinite series and function approximations. A cornerstone result is the Hahn-Banach theorem, which states that if M is a subspace of a normed space X and f: M \to \mathbb{K} is a bounded linear functional, then there exists an extension \tilde{f}: X \to \mathbb{K} that is bounded with the same norm.^[169] This theorem facilitates the extension of linear functionals, crucial for duality theory in infinite dimensions. A Hilbert space is a complete inner product space, where the inner product \langle \cdot, \cdot \rangle induces a norm \|x\| = \sqrt{\langle x, x \rangle} making it a Banach space, with the added structure of orthogonality.^[170] The Riesz representation theorem asserts that every continuous linear functional f on a Hilbert space H can be expressed uniquely as f(x) = \langle x, y \rangle for some y \in H.^[170] This identifies the dual space H^* with H itself, simplifying operator theory compared to general Banach spaces. In infinite dimensions, bounded linear operators T: X \to Y between normed spaces satisfy \|Tx\| \leq M \|x\| for some M > 0 and all x, generalizing matrix norms.^[171] Their spectrum \sigma(T) is the set of \lambda \in \mathbb{C} such that T - \lambda I is not invertible, which is nonempty and compact in Banach spaces, unlike the finite-dimensional case where it coincides exactly with eigenvalues.^[172] This spectral theory extends the finite-dimensional spectral theorem, aiding the analysis of differential operators. Applications abound in solving partial differential equations (PDEs), where Hilbert spaces like L^2 domains allow separation of variables: for instance, the heat equation u_t = \Delta u on a bounded domain separates into ordinary differential equations via eigenfunction expansions in the Hilbert space, yielding solutions as series converging in the L^2 norm.^[173] In quantum field theory, Hilbert spaces describe the state space of quantum fields, with operators representing observables and the vacuum state as the origin, enabling the quantization of fields over spacetime.^[174]

References

[1]
[PDF] What is linear algebra? - UC Berkeley math
Linear algebra studies systems of equations, geometric configurations, and transformations of spaces that carry lines to lines.
[2]
[PDF] Linear Algebra Review and Reference - CS229
Sep 30, 2015 · Linear algebra provides a way of compactly representing and operating on sets of linear equations. For example, Ax = b.
[3]
[PDF] A Brief History of Linear Algebra - University of Utah Math Dept.
Some of the things Linear Algebra is used for are to solve systems of linear format, to find least-square best fit lines to Page 5 predict future outcomes or ...
[4]
[PDF] A Classic from China: The Nine Chapters
a process identical to what Europeans would later call Gaussian. Elimination. ... Lun, The. Nine Chapters on the Mathematical Art: Companion and. Commentary ...
[5]
[PDF] A Chinese Classic: The Nine Chapters - CSUSM
Nov 14, 2005 · The Jiuzhang suanshu, or Nine Chapters on the Mathematical Art was written for the training of ... Gaussian elimination. Negative numbers ...
[6]
Matrices and determinants - MacTutor History of Mathematics
The first appearance of a determinant in Europe was ten years later. In 1693 Leibniz wrote to de l'Hôpital. He explained that the system of equations. 10 + ...
[7]
[PDF] A Brief History of Algebra Thomas Q. Sibley
Around 1750 Leonhard Euler's textbooks set the standard for algebra notation. ... separately Colin Maclaurin (1729) invented determinants to solve systems, and ...
[8]
The Theory of Equations in the 18th Century - jstor
In 1754 an eighteen-year-old mathematician, Joseph Louis Lagrange, addressed a letter to Leonard Euler, one of the giants of European mathematics. In ithe ...
[9]
How ordinary elimination became Gaussian elimination
The familiar method for solving simultaneous linear equations, Gaussian elimination, originated independently in ancient China and early modern Europe.
[10]
[PDF] hermann grassmann and the - creation of linear algebra
Geometry. In the Ausdehnungslehre of 1844, Grassmann describes the geometric consider- ations which led him to the theory that we now call linear algebra ...
[11]
II. A memoir on the theory of matrices - Journals
The theorem shows that every rational and integral function (or indeed every rational function) of a matrix may be considered as a rational and integral ...
[12]
The axiomatization of linear algebra: 1875-1940 - ScienceDirect.com
Modern linear algebra is based on vector spaces, or more generally, on modules. The abstract notion of vector space was first isolated by Peano (1888) in ...
[13]
Abstract linear spaces - MacTutor History of Mathematics
The first to give an axiomatic definition of a real linear space was Peano in a book published in Torino in 1888. He credits Leibniz, Möbius's 1827 work, ...Missing: axiomatization | Show results with:axiomatization
[14]
Moore-Smith Convergence in General Topology - jstor
THEOREM 7: If X is any system of points, in which it is decided which directed sets converge to which points, then X is a topological space under the criterion ...
[15]
Riesz-Fischer Theorem -- from Wolfram MathWorld
In analysis, the phrase "Riesz-Fischer theorem" is used to describe a number of results concerning the convergence of Cauchy sequences in L-p spaces.
[16]
[PDF] 1. Vectors and Geometry - Purdue Math
The notation a ∈ S means that the object a is in the set S, so v ∈ Rn means that the vector v is in the set. Rn of n-dimensional space. has n entries, we say ...
[17]
[PDF] Vectors and Vector Spaces - Home - Virtual Math Learning Center
Definition 1. A real vector is a column of n real numbers. We call such a vector an n dimensional real vector. The set of all n dimensional vectors is Rn.
[18]
[PDF] A Geometric Review of Linear Algebra - Center for Neural Science
Jan 20, 1999 · Vectors of dimension 2 or 3 can be graph- ically depicted as arrows, with the tail at the origin and the head at the coordinate lo- cation ...
[19]
[PDF] Chapter 4: Vectors, Matrices, and Linear Algebra
The definition of a set of basis vectors is twofold: (1) linear combinations (meaning addition, subtraction and multiplication by scalars) of the basis vectors ...<|control11|><|separator|>
[20]
The Feynman Lectures on Physics Vol. I Ch. 11: Vectors - Caltech
Thus velocity is a vector because it is the difference of two vectors. It is also the right definition of velocity because its components are dx/dt, dy/dt, and ...
[21]
[PDF] Chapter 2. Matrix Algebra §2-1. Matrix Addition, Scalar Multiplication ...
Jan 31, 2021 · Definition. Let m and n be positive integers. ▷ An m × n matrix is a rectangular array of numbers having m rows and n columns.Missing: authoritative source
[22]
M.2 Matrix Arithmetic | STAT ONLINE - Penn State
Matrix arithmetic includes transpose, addition (same dimensions), scalar multiplication (multiply by scalar), and multiplication (specific dimension rules).
[23]
[PDF] 1.4 Matrix Multiplication AB and CR - MIT Mathematics
If that vector x has a single 1 in component j, then the associative law is (AB)x = A(Bx). This tells us how to multiply matrices ! The left side is column j of ...
[24]
[PDF] Linear Algebra Review and Reference - CS229
Sep 20, 2020 · Matrix multiplication is associative: (AB)C = A(BC). • Matrix multiplication is distributive: A(B + C) = AB + AC. • Matrix multiplication is, in ...
[25]
[PDF] MATH 304 Linear Algebra Lecture 6: Transpose of a matrix ...
Definition. Given a matrix A, the transpose of A, denoted AT , is the matrix whose rows are columns of A (and whose columns are rows of A). That is, if A = ( ...
[26]
https://people.tamu.edu/~yvorobets/MATH304-2010A/Lect1-06web.pdf
[27]
[PDF] Linear Algebra and Differential Equations Chapter Summaries
A system of m linear equations in n unknowns may be expressed in matrix notation as Ax = b where A is an m x n matrix of coefficients of the equations, x is ...
[28]
[PDF] Linear Algebra - Arizona Math
May 4, 2005 · with a2 + b2 = 1 may be written as a rotation matrix. The rotation matrix acts as a linear transformation of vectors. It rotates them. 7 ...
[29]
[PDF] Mathematicians of Gaussian Elimination - CIS UPenn
Gaussian elimination is universally known as “the” method for solving simultaneous linear equations. As. Leonhard Euler remarked, it is the.
[30]
[PDF] Gaussian Elimination - Purdue Math
May 2, 2010 · We now illustrate how elementary row-operations applied to the augmented matrix of a system of linear equations can be used first to determine ...
[31]
Gaussian Elimination and Rank - Ximera - The Ohio State University
The process of using the elementary row operations on a matrix to transform it into row-echelon form is called Gaussian Elimination.
[32]
Gaussian Elimination — Linear Algebra, Geometry, and Computation
Eight years later, in 1809, Gauss revealed his methods of orbit computation in his book Theoria Motus Corporum Coelestium. Although Gauss invented this method ...
[33]
[PDF] A summary Partial pivoting - CS@Cornell
Gaussian elimination with partial pivoting is almost always backward stable in practice. There are some artificial examples where “pivot growth” breaks ...
[34]
[PDF] Lecture 7 - Gaussian Elimination with Pivoting
Gaussian elimination with pivoting involves partial pivoting, which exchanges rows to avoid zero pivots, and uses the largest pivot for stability.
[35]
[PDF] Linear Systems - cs.Princeton
Questions on Gaussian Elimination? Page 27. Complexity of Gaussian Elimination. • Forward elimination: 2/3 * n3 + O(n2). (triple for-loops yield n3). • Back ...<|control11|><|separator|>
[36]
[PDF] Linear Algebra Review Notation An,m is an n×m matrix, ie
A square matrix is invertible if and only if it is of full rank. We denote the inverse of A as A. -1. , so if A. -1.Missing: correspondence | Show results with:correspondence
[37]
[PDF] Matrices - Lecture Notes 2
Square matrices have a unique inverse if they are full rank, since in that case the null space has dimension 0 and the associated linear map is a bijection. The ...
[38]
[PDF] 1 Introduction 2 Matrices: Definition - The University of Texas at Dallas
Matrices whose rank is equal to their dimensions are full rank and they are invertible. When the rank of a matrix is smaller than its dimensions, the matrix is ...
[39]
Inverse of a Square Matrix
Inverse of a square matrix. Written by Paul Bourke ... The inverse of a 2x2 matrix can be written explicitly, namely. a. b. c. d. = 1. ad - bc. d. -b. -c. a ...Missing: formula | Show results with:formula
[40]
[PDF] 12.1 Matrices
The following example shows how to use this trick to find the matrix for a linear transformation. EXAMPLE 2. Find the matrix for a 45◦ counterclockwise rotation ...
[41]
Cramer's Rule
Cramer's Rule. Theorem: Let "A" be an invertible n x n matrix. For any in the unique solution, , of has these entries: where is the "A" matrix with its ith ...
[42]
[PDF] 5.3 Determinants and Cramer's Rule
This result, called Cramer's Rule for 2 × 2 systems, is usually learned in college algebra as part of determinant theory. Determinants of Order 2. College ...
[43]
LU Decomposition for Solving Linear Equations - CS 357
LU decomposition is the factorization of a matrix A into L and U, where A=LU. L is lower-triangular, U is upper-triangular, and it provides an efficient way to ...
[44]
[PDF] 11. LU Decomposition - UC Davis Math
For lower triangular matrices, back substitution gives a quick solution; for upper triangular matrices, forward substitution gives the solution. 1. Page 2 ...
[45]
[PDF] LU Decomposition 1.0 Introduction We have seen how to construct ...
U is an upper-triangular matrix, meaning that all elements below the diagonal are 0. •. L is a lower-triangular matrix, meaning that all elements above the ...
[46]
[PDF] LU Factorization with Partial Pivoting (Numerical Linear Algebra ...
the LU decomposition PA = LU where P is the associated permutation matrix. Solution: We can keep the information about permuted rows of A in the permutaion.
[47]
A LU Pivoting - Penn Math
LU Decomposition With Pivoting · Permute the rows of A using P. · Apply Gassian elimination without pivoting to PA.
[48]
Axioms of vector spaces
A real vector space is a set X with a special element 0, and three operations. These operations must satisfy the following axioms.
[49]
[PDF] The vector space axioms
A vector space over a field F is a set V , equipped with. • an element 0 ∈ V called zero, • an addition law α : V × V → V (usually written α(v, w) = v + w), ...
[50]
[PDF] Examples of Vector Spaces - Cornell University
The phrase “treated formally” means that two polynomials in F[x] are equal if and only if their corresponding coefficients are equal. They are NOT functions.
[51]
[PDF] Mathematics Course 111: Algebra I Part IV: Vector Spaces
Example. The set of all polynomials with real coefficients is a real vector space, with the usual oper- ations of addition of polynomials and multiplication of ...
[52]
[PDF] Lecture 4 - Math 5111 (Algebra 1)
Fields and Vector Spaces. Fields + Basic Examples. Vector Spaces. Subfields and Field Extensions. This material represents §2.1.1-2.2.1 from the course notes.
[53]
Axioms and Properties of Vector Spaces | Abstract Linear Algebra I ...
Vector spaces are the foundation of linear algebra, defining sets of objects that behave like vectors. They're governed by axioms that dictate how vectors ...
[54]
[PDF] 8 Vector space
The eight properties in the definition of a vector space are called the vector space axioms. These axioms can be used to prove other properties about vector ...
[55]
[PDF] LADR4e.pdf - Linear Algebra Done Right - Sheldon Axler
In contrast, this course will emphasize abstract vector spaces and linear maps. The title of this book deserves an explanation. Most linear algebra textbooks.
[56]
https://linear.axler.net/LADR4e.pdf
[57]
[PDF] MATH 235: Linear Algebra 2 for Honours Mathematics
... linear maps. For example, inte- gration and differentiation of polynomials are both linear maps. Example 2.1.8. The differentiation map. D: 𝒫3(R) → 𝒫2(R).<|control11|><|separator|>
[58]
https://math.libretexts.org/Bookshelves/Linear_Algebra/A_First_Course_in_Linear_Algebra_(Kuttler)/05%3A_Linear_Transformations/5.02%3A_The_Matrix_of_a_Linear_Transformation_I
[59]
[PDF] 23. Kernel, Rank, Range - UC Davis Mathematics
Definition Let L : V → W be a linear transformation. The set of all vectors v such that Lv = 0W is called the kernel of L: kerL = {v ∈ V |Lv = 0}. 1 The ...
[60]
[PDF] 2.2 Kernel and Range of a Linear Transformation
Definition 2.6: Let T : V → W be a linear transformation. The nullity of T is the dimension of the kernel of T, and the rank of T is the dimension of the ...
[61]
[PDF] Lecture 13: Image and Kernel
Mar 2, 2011 · The kernel of a matrix. If T : Rm → Rn is a linear transformation, then the set {x | T(x)=0 } is called the kernel of T. These are all ...
[62]
[PDF] The Rank-Nullity Theorem - Purdue Math
Feb 16, 2007 · Theorem 4.9.1. (Rank-Nullity Theorem). For any m × n matrix A, rank(A) + nullity(A) = n. (4.9.1). Proof If rank(A) = n, then by the Invertible ...
[63]
[PDF] 7.3 Isomorphisms and Composition
A linear transformation T :V → W is called an isomorphism if it is both onto and one-to-one. The vector spaces V and W are said to be isomorphic if there exists ...
[64]
[PDF] Isomorphisms Math 130 Linear Algebra
Definition 1 (Isomorphism of vector spaces). Two vector spaces V and W over the same field F are isomorphic if there is a bijection T : V → W which.
[65]
12 Linear Maps: Isomorphisms and Homomorphisms
A homomorphism that is bijective is said to be an isomorphism. An isomorphism from a vector space to itself is called an automorphism.
[66]
2.3 Linear Maps out of quotients
5. First Isomorphism theorem. Let T : V → W be a linear map. Then ⁡ ( T ) ≅ Im ( T ) via the map T ∗ ( v + ker ⁡
[67]
LTR-0050: Image and Kernel of a Linear Transformation - Ximera
We define the image and kernel of a linear transformation and prove the Rank-Nullity Theorem for linear transformations.
[68]
[PDF] Multilinearity of the Determinant. Professor Karen Smith A. Theorem
The determinant is multilinear in the rows. This means that if we fix all but one column of an n × n matrix, the determinant function is linear in the remaining ...
[69]
Cofactor Expansions
Cofactor expansions are recursive formulas for computing a matrix's determinant, using minors and cofactors, and can be done along rows or columns.
[70]
Determinants and Volumes
In this section we give a geometric interpretation of determinants, in terms of volumes. This will shed light on the reason behind three of the four defining ...
[71]
https://math.dartmouth.edu/~trs/PreTeXtProjects/linear-algebra-refresher-CLI/output/web/section-factoring-through-a-quotient.html
[72]
[PDF] Math 113: Linear Algebra Eigenvectors and Eigenvalues
Oct 30, 2008 · We need some extra structure, which will be the focus of Chapter 6. 2 Eigenvectors and Eigenvalues. Definition 1 (Eigenvector, eigenvalue).
[73]
[PDF] Chapter 6 Eigenvalues and Eigenvectors
This chapter enters a new part of linear algebra. The first part was about ... If two eigenvectors share the same λ, so do all their linear combinations.
[74]
[PDF] Proof of the spectral theorem
Nov 5, 2013 · The theorem says first of all that a selfadjoint operator is diagonalizable, and that all the eigenvalues are real. The orthogonality of the ...
[75]
[PDF] Math 63CM Section 4 Handout - Mathematics
Jan 30, 2020 · Theorem 1.14. If A is real symmetric, then A is diagonalizable, i.e. for an orthogonal matrix O and a diagonal matrix D, we have A = OT DO.
[76]
[PDF] a note on the jordan canonical form - arXiv
Dec 12, 2010 · A proof of the Jordan canonical form, suitable for a first course in linear algebra, is given. The proof includes the uniqueness of the number ...
[77]
[PDF] 10.3 Markov Matrices, Population, and Economics - MIT Mathematics
Questions 1–12 are about Markov matrices and their eigenvalues and powers. 1. Find the eigenvalues of this Markov matrix (their sum is the trace):. A = .90 .15.
[78]
[PDF] inner product spaces - UC Davis Mathematics
Definition. An inner product on a real vector space V is a function that associates a real number 〈u, v〉 with each pair of vectors in V.
[79]
Inner Product Spaces - Ximera - The Ohio State University
An inner product on a real vector space is a function that assigns a real number to every pair , of vectors in in such a way that the following properties are ...<|control11|><|separator|>
[80]
[PDF] Complex Inner Product Spaces 1 The definition
A complex inner product ⟨x|y⟩ is linear in y and conjugate linear in x. Definition 1 A complex inner product space is a vector space V over the field C of.
[81]
[PDF] MATH 304 Linear Algebra Lecture 20: Inner product spaces ...
The notion of inner product generalizes the notion of dot product of vectors in Rn. Definition. Let V be a vector space. A function β : V × V → R, usually ...
[82]
5.2 Definition and Properties of an Inner Product - BOOKS
The inner product is always a rule that takes two vectors as inputs and the output is a scalar (often a complex number).
[83]
[PDF] 9 Inner product
The generalization of the dot product to an arbitrary vector space is called an. “inner product.” Just like the dot product, this is a certain way of ...
[84]
[PDF] Inner products - Purdue Math
Sep 7, 2024 · An inner product, also known as dot product, is an operation on vectors that produces a number, denoted by (x,y). It has three properties.<|separator|>
[85]
[PDF] Appendix: Norms and Inner Products
Inner Product Spaces. Definition: Inner Product. Let V be a vector space. An inner product on V is a function h−,−i: V ×V → R satisfying the following ...
[86]
[PDF] Math 20F Linear Algebra Lecture 25 1 Slide 1
An inner product space induces a norm, that is, a notion of length of a vector. Definition 2 (Norm) Let V , ( , ) be a inner product space. The norm function, ...
[87]
[PDF] 5. Inner Products and Norms - Numerical Analysis Lecture Notes
May 18, 2008 · The triangle inequality turns out to be an elementary consequence of the. Cauchy–Schwarz inequality, and hence is valid in any inner product ...
[88]
[PDF] 6.7 Cauchy-Schwarz Inequality - UC Berkeley math
Thus the Cauchy-Schwarz inequality is an equality if and only if u is a scalar multiple of v or v is a scalar multiple of u (or both; the phrasing has been ...
[89]
[PDF] Cauchy-Schwarz - CS@Cornell
For any inner product,. 0 ≤ ksu + vk2 = hsu + v, su + vi = s2kuk2 + 2shu, vi ... With a little algebra, we have the Cauchy-Schwarz inequality,. |hu, vi ...
[90]
[PDF] Math 113: Linear Algebra Norms and Inner Products
Nov 5, 2008 · Proposition 1 (Cauchy-Schwarz Inequality). |(v, w)|2 ≤ (v, v)(w, w ... Left as an exercise; use linearity properties of the inner product.
[91]
INNER PRODUCT & ORTHOGONALITY
Two vectors are orthogonal to each other if their inner product is zero. That means that the projection of one vector onto the other "collapses" to a point.
[92]
[PDF] MATH 323 Linear Algebra Lecture 35: Orthogonality in inner product ...
Let V be an inner product space with an inner product h·,·i. Definition 1. Vectors x,y ∈ V are said to be orthogonal (denoted x ⊥ y) if hx,yi = 0.
[93]
[PDF] Orthogonality in inner product spaces.
Theorem Any finite-dimensional vector space with an inner product has an orthonormal basis. Remark. An infinite-dimensional vector space with an inner product ...
[94]
[PDF] Lecture 5: October 16, 2018 1 Orthogonality and orthonormality. - TTIC
Oct 16, 2018 · Proposition 1.5 (Parseval's identity) Let V be a finite dimensional inner product space and let. {w1,..., wn} be an orthonormal basis for V.
[95]
[PDF] 18.102 S2021 Lecture 15. Orthonormal Bases and Fourier Series
Apr 13, 2021 · Theorem 164 (Parseval's identity). Let H be a Hilbert space, and let {en} be a countable orthonormal basis of H. Then for all u ∈ H,. X n. |hu ...
[96]
[PDF] Orthogonal Bases and the QR Algorithm
Jun 5, 2010 · Two vectors v, w ∈ V are called orthogonal if their inner product vanishes: v·w = 0. In the case of vectors in Euclidean space, orthogonality ...
[97]
[PDF] Gram--Schmidt Orthogonalization: 100 Years and More - CIS UPenn
Jun 8, 2010 · Schmidt used what is now known as the classical. Gram–Schmidt process. Erhard Schmidt (1876–1959), studied in. Göttingen under Hilbert. In 1917 ...
[98]
[PDF] Inner Product Spaces and Orthogonality - HKUST Math Department
We define the vector. Projv(y) = (v, y). (v, v) v, called the orthogonal projection of y along v. The linear transformation Proju : V → V is called the.
[99]
[PDF] QR decomposition: History and its Applications - Duke People
Dec 17, 2010 · QR decomposition is the matrix version of the Gram-Schmidt orthonor- malization process. where Q ∈ Cm×n has orthonormal columns and R ∈ Cn×n is ...
[100]
Fourier Series -- from Wolfram MathWorld
A Fourier series is an expansion of a periodic function using an infinite sum of sines and cosines, using orthogonality relationships.
[101]
[PDF] Notes on Dual Spaces
The dual space of V , denoted by V ∗, is the space of all linear functionals on V ; i.e. V ∗ := L(V,F).
[102]
[PDF] Dual Spaces - Cornell University
Sep 26, 2019 · We define the dual space of V to be V ∗ = Hom. F. (V,F) . The linear transformations in V ∗ are usually called linear functionals.
[103]
[PDF] Math 4377/6308 Advanced Linear Algebra - 2.5 Change of Bases ...
Definition. For a vector space V over F, the dual space of V is the vector space V∗ = L(V,F). Note that for finite-dimensional V, dim(V. ∗. ) = dim(L( ...
[104]
[PDF] MATH 423 Linear Algebra II Lecture 31: Dual space. Adjoint operator.
Let V be a vector space over a field F. Definition. The vector space L(V,F) of all linear functionals. ℓ : V → F is called the dual space of V (denoted V′ or V∗) ...
[105]
[PDF] Annihilators : Linear Algebra Notes - Satya Mandal
Sep 21, 2005 · It is possible to mix up two annihilator of S. The first one is a subspace of the double dual V ∗∗ and the second one is the subspace of V.
[106]
[PDF] The Dual of a Vector Space - OSU Math
Sep 12, 2016 · In physics the elements of the vector space V ∗ are called covectors. 2. That V ∗ does indeed form a vector space is verified by observing that.
[107]
Linear Algebra, Part 3: Dual Spaces (Mathematica)
A duality can be viewed as a property to see an object similar to the original picture when it is applied twice, analogous to a mirror image.
[108]
Math 55a: Duality basics
The dual of a quotient space V/U is naturally a subspace of V*, namely the annihilator of U in V*. If V has finite dimension n then so does V*. The dimension of ...Missing: algebra | Show results with:algebra
[109]
[PDF] Duality, part 2: Annihilators and the Matrix of a Dual Map
Duality, part 2: Annihilators and the Matrix of a Dual Map ... The Matrix of the Dual of a Linear Map. The matrix of T/ is the transpose of the matrix of T.
[110]
[PDF] Lecture 2.5: The transpose of a linear map
The transpose of a matrix is what results from swapping rows with columns. In our setting, we like to think about vectors in X as column vectors, and dual ...
[111]
[PDF] Math 113: Linear Algebra Adjoints of Linear Transformations
Nov 12, 2008 · Roughly, an inner product gives a way to equate V and V ∗. Definition 1 (Adjoint). If V and W are finite dimensional inner product spaces and T ...
[112]
[PDF] Adjoints of linear operators • Self
Thus the adjoint is like the complex conjugate, but for linear transformations rather than for scalars.
[113]
[PDF] MATH 423 Linear Algebra II Lecture 32: Adjoint operator (continued ...
Theorem (i) If the adjoint operator L∗ exists, it is unique and linear. (ii) If V is finite-dimensional, then L∗ always exists. Properties of adjoint operators:.
[114]
[PDF] The Spectral Theorem for Self-Adjoint and Unitary Operators
If dimH = n < ∞, each self-adjoint A ∈ L(H) has the property that H has an orthonormal basis of eigenvectors of A. The same holds for each unitary U ∈ L ...
[115]
[PDF] Chapter 10: Linear Differential Operators and Green's Functions
As we will see below, the adjoint of a differential operator is another differential operator, which we obtain by using integration by parts. The domain V(A) ...
[116]
[PDF] Fourier Transform - UCLA Department of Mathematics
which shows, among other things, that the Fourier transform is in fact a unitary operation, and so one can view the frequency space representation of a function ...
[117]
[PDF] CLASSICAL GROUPS 1. Orthogonal groups These notes are about ...
which is the definition of the special orthogonal group SO(n). It is the identity component of O(n), and therefore has the same dimension and the same Lie ...
[118]
Orthogonal Transformation -- from Wolfram MathWorld
An orthogonal transformation is a linear transformation T:V->V which preserves a symmetric inner product. In particular, an orthogonal transformation ...
[119]
[PDF] Linear algebra and geometric transformations in 2D - UCSD CSE
• 2D translation using a 3x3 matrix. • Inverse of 2D translation is ... 2D rotation about a point using homogeneous coordinates. CSE 167, Winter 2018.
[120]
[PDF] Linear Transformations in Graphics
A linear transformation is matrix-vector multiplication, where a matrix maps a vector from a domain to a co-domain. It meets the linearity condition.
[121]
[PDF] Linear and affine transformations - FD intranet
Nov 10, 2020 · An affine transformation is any transformation that preserves co-linearity (i.e., all points lying on a line initially still lie on a line after.
[122]
[PDF] Unit 5: Change of Coordinates
Change of coordinates involves writing a vector in a new basis using a coordinate change matrix, and is used to figure out the matrix of a transformation.
[123]
[PDF] Lecture 7.4: Polar decomposition
Theorem. Every linear map A: X → U can be written as A = UP where P ≥ 0 and U is unitary. This is called the (left) polar decomposition of A.
[124]
Orthogonal Projection
Learn the basic properties of orthogonal projections as linear transformations and as matrix transformations.
[125]
[PDF] Lecture 15: Projections onto subspaces - MIT OpenCourseWare
We can see from Figure 1 that this closest point p is at the intersection formed by a line through b that is orthogonal to a. If we think of p as an.
[126]
6.3 Orthogonal bases and projections - Understanding Linear Algebra
Definition 6.3. Given a vector in and a subspace of , the orthogonal projection of onto is the vector in that is closest to . b . It is characterized by the ...
[127]
[PDF] Chapter 7 The Singular Value Decomposition (SVD)
The Singular Value Decomposition is a highlight of linear algebra. A is any m by n matrix, square or rectangular. Its rank is r. We will diagonalize this A, ...Missing: seminal | Show results with:seminal
[128]
[PDF] Early History of the Singular Value Decomposition - UC Davis Math
Eugenio. Beltrami (1835-1899), Camille Jordan (1838-1921), ...Missing: seminal | Show results with:seminal<|separator|>
[129]
[PDF] The Approximation of One Matrix by Another of Lower Rank
The problem is to find a matrix of lower rank that most closely approximates a given matrix of higher rank, related to factor theory.
[130]
[PDF] 1 Singular Value Decomposition 2 Inverses - pillow lab @ princeton
We can obtain the pseudoinverse from the SVD by inverting all singular values that are non-zero, and leaving all zero singular values at zero.
[131]
Projection matrix - StatLect
Learn how projection matrices are defined in linear algebra. Discover their properties. With detailed explanations, proofs, examples and solved exercises.Projections · Projection operator · How to derive the projection...
[132]
[PDF] NUMERICAL METHODS FOR LARGE EIGENVALUE PROBLEMS ...
Figure 10.2: A spring system with two masses. At equilibrium the displacements as well as their derivatives, and the external forces are zero. As a result ...
[133]
Linear equations from electrical circuits
Developing linear equations from electric circuits is based on two Kirchhoff's laws: Kirchhoff's current law (KCL): at any node (junction) in an electrical ...Missing: impedance | Show results with:impedance
[134]
[PDF] Kalman's controllability rank condition - Sontag Lab
Kalman's controllability rank condition, for linear systems, states that the rank of the Kalman block matrix must equal the state space dimension.
[135]
[PDF] Eigenvalues and the Laplacian of a graph - Fan Chung Graham
Spectral graph theory has a long history. In the early days, matrix theory and linear algebra were used to analyze adjacency matrices of graphs.
[136]
https://www.math.ucdavis.edu/~saito/courses/229A/stewart-svd.pdf
[137]
Homogeneous Coordinates and Projective Planes in Computer ...
Homogeneous Coordinates and Projective Planes in Computer Graphics · Riesenfeld · Published in IEEE Computer Graphics and… 1981 · Computer Science.Missing: original | Show results with:original
[138]
[PDF] The PageRank Citation Ranking: Bringing Order to the Web
Jan 29, 1998 · This paper describes PageRank, a method for rating Web pages objectively and mechanically, effectively measuring the human interest and.
[139]
[PDF] Pearson, K. 1901. On lines and planes of closest fit to systems of ...
Pearson, K. 1901. On lines and planes of closest fit to systems of points in space. Philosophical Magazine ... http://pbil.univ-lyon1.fr/R/pearson1901.pdf.
[140]
[PDF] Learning representations by back-propagating errors
We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in ...
[141]
[PDF] Origins of the Simplex Method - DTIC
For forty years the simplex method has reigned supreme as the preferred method for solving linear programs. It is historically the reason for the practical ...
[142]
Gauss and the Invention of Least Squares - jstor
The most famous priority dispute in the history of statistics is that between Gauss and Legendre, over the discovery of the method of least squares.<|control11|><|separator|>
[143]
module in nLab
Jul 10, 2025 · The ordinary case of modules over rings is phrased in terms of stabilized overcategories by the following observation, which goes back at least ...
[144]
Module -- from Wolfram MathWorld
A module is a mathematical object in which things can be added together commutatively by multiplying coefficients and in which most of the rules of ...
[145]
[PDF] atiyahmacdonald.pdf
Commutative algebra is essentially the study of commutative rings. Roughly speaking, it has developed from two sources: (1) algebraic geometry and (2).
[146]
free module in nLab
Apr 26, 2023 · A free module over some ring R R is freely generated on a set of basis elements. Under the interpretation of modules as generalized vector ...
[147]
Free Module -- from Wolfram MathWorld
The free module of rank n over a nonzero unit ring R, usually denoted R^n, is the set of all sequences {a_1,a_2,...,a_n} that can be formed by picking n (Missing: textbook | Show results with:textbook
[148]
[PDF] rotman.pdf
SPECIAL MODULES. CH. 3. Definition. If R is a domain and M is an R-module, then its torsion4 sub- module is. tM = m ∈ M : rm = 0 for some nonzero r ∈ R . We ...
[149]
[PDF] AN INTRODUCTION TO HOMOLOGICAL ALGEBRA
Homological algebra is a tool used to prove nonconstructive existence theo- rems in algebra (and in algebraic topology). It also provides obstructions to.
[150]
[PDF] Algebraic Topology - Cornell Mathematics
This book covers geometric notions, the fundamental group, homology, cohomology, and homotopy theory, with a classical approach.
[151]
https://ncatlab.org/nlab/show/module
[152]
[PDF] Tensor Decompositions and Applications
A tensor is a multidimensional or N-way array. Decompositions of higher-order tensors (i.e., N-way arrays with N ≥ 3) have applications in psycho- metrics, ...
[153]
[PDF] Lecture 16: Tensor Products
Definition. Let U and V be vector spaces. Then the tensor product is vector space U ⊗ V such that (1) There is a bilinear map i: U × V → U ⊗ V and (2) Given ...
[154]
https://ncatlab.org/nlab/show/free%2Bmodule
[155]
Tensor Contraction -- from Wolfram MathWorld
Tensor Contraction. The contraction of a tensor is obtained by setting unlike indices equal and summing according to the Einstein summation convention.Missing: linear | Show results with:linear
[156]
Linear Algebra, Part 3: Vector Products (Mathematica)
In particular, u ⊗ v is a matrix of rank 1, which means that most matrices cannot be written as tensor products of two vectors. The special case constitute ...
[157]
Riemann Tensor -- from Wolfram MathWorld
The Riemann tensor is in some sense the only tensor that can be constructed from the metric tensor and its first and second derivatives.
[158]
[PDF] 1 Lp Spaces and Banach Spaces
The basic analytic fact is that Lp is complete in the sense that every Cauchy sequence in the norm I·ILp converges to an element in Lp.
[159]
[PDF] Topological Vector Spaces I: Basic Theory - KSU Math
The product space KI (defined as the space of all functions I → K) is obviously a vector space (with pointwise addition and scalar multiplication). The product ...
[160]
[PDF] An Introduction To Banach Space Theory
The genesis of Banach space theory can be traced back to Stefan Banach's seminal work in the early 20th century, particularly his 1932 book Théorie des ...
[161]
[PDF] The Hahn–Banach theorem
Here is the first step: 1The Hahn–Banach theorem was first proven in 1912 by the Austrian mathematician Eduard Helly. (1884–1943). It was rediscovered ...
[162]
[PDF] hilbert spaces and the riesz representation theorem - UChicago Math
Abstract. The Riesz representation theorem is a powerful result in the theory of Hilbert spaces which classifies continuous linear functionals in terms of ...
[163]
[PDF] Chapter 9: The Spectrum of Bounded Linear Operators
A bounded linear operator on an infinite-dimensional & ilbert space need not have any eigenvalues at all, even if it is self-adjoint (seeCX xample ./} below) .
[164]
[PDF] 7 Spectrum of linear operators
Theorem 7.2. The spectrum σ(A) of any bounded linear operator A is a closed subset of contained in |λ|≤kAk. Proof.
[165]
[PDF] Separation of Variables in Linear PDE: One-Dimensional Problems
Separation of Variables in Linear PDE: One-Dimensional Problems. Now we apply the theory of Hilbert spaces to linear differential equations with partial ...
[166]
Hilbert space of Quantum Field Theory in de Sitter spacetime - arXiv
Jan 10, 2023 · We study the decomposition of the Hilbert space of quantum field theory in (d+1) dimensional de Sitter spacetime into Unitary Irreducible Representations (UIRs)