Dot Product & Projection

Concept Overview

The dot product (or scalar product) is an algebraic operation that takes two equal-length sequences of numbers (usually coordinate vectors) and returns a single number. It tells us how much two vectors point in the same direction. Vector projection uses the dot product to find the component of one vector that lies along the direction of another.

Mathematical Definition

Algebraic Form

For two vectors u and v in n-dimensional space, the dot product is the sum of the products of their corresponding entries:

u · v = Σ_i=1ⁿ u_iv_i = u₁v₁ + u₂v₂ + ... + u_nv_n

Geometric Form

Geometrically, it is the product of the Euclidean magnitudes of the two vectors and the cosine of the angle θ between them:

u · v = ||u|| ||v|| cos(θ)

Vector Projection

The projection of vector u onto vector v is the orthogonal projection of u onto a straight line parallel to v. It results in a vector that is a scaled version of v:

proj_v(u) = [(u · v) / ||v||²] v

Key Concepts

Orthogonality

Two non-zero vectors are orthogonal (perpendicular) if and only if their dot product is zero. In this case, cos(90°) = 0, meaning the vectors share no directional components.

Sign and Angle

The sign of the dot product indicates the angle between vectors:

Positive: The angle is acute (< 90°). They generally point in the same direction.
Zero: The angle is exactly 90°. The vectors are orthogonal.
Negative: The angle is obtuse (> 90°). They generally point in opposite directions.

Scalar vs. Vector Projection

The scalar projection is the magnitude of the projected vector, representing the length of the shadow u casts onto v. It is given by (u · v) / ||v||. The vector projection multiplies this scalar by the unit vector in the direction of v to produce a vector.

Historical Context

The concept of the dot product originated from the development of quaternions by William Rowan Hamilton in 1843, and the later vector analysis developed independently by Josiah Willard Gibbs and Oliver Heaviside in the late 19th century. Gibbs originally called it the "direct product," emphasizing its distinction from the "skew product" (which we now know as the cross product).

Projections have deep roots in ancient geometry, particularly in the study of shadows and perspective, but formalizing them algebraically via vector operations enabled sweeping generalizations into infinite-dimensional spaces (Hilbert spaces), paving the way for modern quantum mechanics and signal processing.

Real-world Applications

Computer Graphics: Used extensively to calculate illumination (how light hits a surface), back-face culling, and reflections. The angle between a light ray and a surface normal determines the brightness of a pixel.
Physics (Work): In classical mechanics, work is the dot product of the force vector and the displacement vector (W = F · d), meaning only the force applied in the direction of movement contributes to work.
Machine Learning: Fundamental to neural networks (computing weighted sums of inputs) and finding similarities between documents or items (cosine similarity).
Signal Processing: Vector projection is used to decompose signals into orthogonal components, forming the basis of tools like the Fourier transform.

Related Concepts

Cross Product: While the dot product yields a scalar measuring parallelism, the cross product yields a vector measuring perpendicularity (area).
Gram-Schmidt Process: An algorithm that uses vector projections repeatedly to orthogonalize a set of vectors.
Linear Transformations: Dot products can be expressed as matrix multiplications (u^Tv).

Dot Product & Projection