Linear Algebra: What It Is and How Vectors and Matrices Work

What linear algebra is

Wolfram MathWorld defines linear algebra as the study of linear systems of equations and their transformation properties — matrices and determinants sit at its computational core, and in a broader sense so do vector spaces and the linear maps between them. The same entry flags a small terminological quirk: in the strict technical sense, an “algebra” is a structure with its own internal multiplication, and an ordinary vector space doesn’t carry one by default, so the name “linear algebra”, applied to the whole field, is a bit misleading relative to that narrower meaning.

For anyone arriving from algebra and arithmetic, linear algebra doesn’t introduce a new kind of calculation so much as a new set of objects — vectors, matrices, vector spaces — to which familiar calculation rules still largely apply.

Vectors, matrices and vector spaces

A vector space is a set of elements, called vectors, on which two operations are defined: addition between vectors, and multiplication of a vector by a scalar — that is, a plain number. Giuseppe Peano gave the first axiomatic definition of a real vector space in 1888, building on the work of Hermann Grassmann, who in 1844 had published Die lineale Ausdehnungslehre (Theory of Linear Extension), the text usually credited as the origin of the field.

A matrix is a rectangular table of numbers. The term was coined in 1850 by the English mathematician Sylvester, who described it as an oblong arrangement of terms; the first abstract definition of a matrix, together with the first proof of the Cayley-Hamilton theorem for 2×2 matrices, appears in Arthur Cayley’s Memoir on the Theory of Matrices, from 1858. Matrix calculations — addition, multiplication, computing a determinant — form, in MathWorld’s phrasing, one of the extremely useful toolkits of linear algebra.

	Scalar	Vector	Matrix
What it is	A single number	An element of a vector space	A rectangular table of numbers
Example	5	(2, -1, 3)	a 2×2 grid of numbers
Used for	Scaling a vector up or down	Representing a quantity or a point in space	Representing a system of equations or a transformation

Linear systems and Gaussian elimination

A system of linear equations in several unknowns can be written compactly as Ax = b, where A is the matrix of coefficients, x the vector of unknowns and b the vector of known values. MathWorld notes that although the equation can, in principle, be solved by inverting the matrix (x = A⁻¹b), Gaussian elimination remains the numerically most robust method for finding x in practice.

Procedures of this kind are older than the name attached to them. In the Chinese text Nine Chapters on the Mathematical Art, dated by historians to around the second century BCE, MacTutor identifies what it calls the earliest known example of matrix methods: the text instructs the reader to multiply one column of coefficients by a factor and subtract it from another, a column-by-column way of eliminating unknowns one at a time that anticipates, in substance, Gaussian elimination. In Europe, Gottfried Leibniz developed a double-index notation for systems of equations in 1693, a forerunner of determinant notation; Gabriel Cramer described in 1750 the method now known as Cramer’s rule, which expresses each unknown as a ratio of determinants. Carl Friedrich Gauss introduced the term “determinant” in the Disquisitiones Arithmeticae of 1801; years later, while computing the orbit of the asteroid Pallas from observations taken between 1803 and 1809, he solved a system of six linear equations in six unknowns using precisely the elimination procedure that today bears his name.

Practical example: a system Ax = b shows up whenever several quantities are tied together by several equations at once — for instance an electrical circuit with multiple loops, where the unknown currents are found by solving Kirchhoff’s equations together, written in matrix form.

Linear transformations, eigenvalues and eigenvectors

A linear transformation is a function between vector spaces that preserves the two defining operations of a vector space: transforming the sum of two vectors gives the same result as summing their transformations, and the same holds for scalar multiplication. Linear transformations, and their representation as matrices, sit alongside linear dependence among the central ideas of the field.

A particular but central case is that of eigenvalues and eigenvectors. Given a square matrix A and a nonzero vector X, if there is a scalar λ such that AX = λX, then λ is called an eigenvalue of A and X the corresponding eigenvector. Eigenvalues are found by solving the characteristic equation det(A - λI) = 0, where I is the identity matrix; MathWorld notes that for symmetric matrices, eigenvectors associated with distinct eigenvalues are always orthogonal to one another.

MathWorld links the search for eigenvalues and eigenvectors to matrix diagonalization, a calculation that comes up in concrete applications such as stability analysis, the physics of rotating bodies, and the study of small oscillations in vibrating systems.

Linear algebra and geometry

Linear algebra is used widely beyond abstract algebra and functional analysis — Treccani’s Enciclopedia della Matematica singles out geometry in particular. The connection has clear historical roots: in 1827 Möbius published Der barycentrische Calcul, a geometry book studying transformations of lines and conics; in 1832 Bellavitis defined two segments as “equipollent” when equal and parallel, an idea MacTutor describes as essentially a vector space. Hamilton gave, in 1843, an important example of a four-dimensional vector space with his quaternions.

Vectors, matrices and linear transformations are the tools used to describe rotations, projections and changes of coordinate system in space. Many university courses teach the two subjects together, often under a combined name such as geometry and linear algebra.

A history spanning more than two thousand years

The remote origins of linear algebra stretch from ancient China through seventeenth- and eighteenth-century Europe, while its modern formalization is compressed into a few decades of the nineteenth century. Cayley introduced matrix algebras in 1857, opening the way toward more general abstract systems; the following year, in 1858, he published the first abstract definition of a matrix. Grassmann had already published the concepts of a space and a linear combination in 1844, though MacTutor describes his text as very difficult to read and notes that it did not find favor with mathematicians at the time. It was Peano, in 1888, who gave the idea a rigorous axiomatic form.

Even the concept of a vector predates the nineteenth century in its earliest form: MacTutor traces the first steps of the idea to the work of Bernard Bolzano, at the start of the 1800s.

Where it gets used

The applications of linear algebra go well beyond pure mathematics. Encyclopaedia Britannica lists, among the uses of matrix multiplication, plotting graphs, rendering images, analyzing electrical circuits, guiding robots, developing medical scans, encrypting messages and carrying out statistical analysis — fields as different as engineering, computer science and probability and statistics.

Data science and machine learning add a more recent layer to that list: data sets and model parameters are routinely stored as vectors and matrices, and training a model largely comes down to matrix operations on them.

MIT’s course 18.06, taught by Gilbert Strang, sums up the backbone of the subject in six core topics: systems of equations, vector spaces, determinants, eigenvalues, similarity between matrices, and positive definite matrices — the same core of ideas covered in this Recap, from linear systems to eigenvalues. Anyone continuing their studies from here runs into calculus, which studies limits, derivatives and integrals using functions built with the tools of linear algebra.