Newton’s Method

Newton’s Method

Mathematical Definition

At x_n, the tangent line has slope f′(x_n) and passes through (x_n, f(x_n)). Setting its height to zero gives the update, provided the derivative is nonzero.

The amber segment shows the latest completed tangent update. A dashed tangent at the initial point is only a preview. Green vertical guides return an x-intercept to the curve to evaluate the next function value. The ledger includes the starting point as row zero; completed updates count the transitions after it.

Key Concepts

A square-root example

For f(x) = x² − 2, the rule simplifies to x_n+1 = (x_n + 2/x_n)/2. Starting at 1 gives 3/2, then 17/12, then 577/408. The third estimate is approximately 1.414215686275; its exact residual is 1/166464, approximately 0.000006007305. Starting at −1 produces the corresponding negative sequence and approaches −√2. The starting guess can determine which root is found.

Local convergence and repeated roots

For a twice continuously differentiable function near a simple root r, with f′(r) ≠ 0 and a sufficiently close initial estimate, Newton’s method converges locally and quadratically. Writing e_n = x_n − r, its leading error relation is:

These are local conditions. They do not promise success from every starting point. For the repeated-root preset f(x) = (x − 1)², the unmodified update is (x + 1)/2 whenever x ≠ 1. Its distance to the root halves at every step. Starting at 3 gives 2, 1.5 and 1.25, whose residual is 0.0625. This is linear convergence. Starting exactly at 1 already solves the equation, so no division is needed despite the zero derivative.

Failure is part of the lesson

• Horizontal tangent: For x² − 2 at x = 0, f = −2 and f′ = 0. The tangent never reaches the x-axis, so the update is undefined.
• A two-cycle: For x³ − 2x + 2 at x = 0, the next estimate is 1; at 1, the next estimate is 0. Neither point is a root. The exact repetition demonstrates failure even though both slopes are nonzero.
• Small slope: A tiny nonzero derivative can produce a very large step. A large or repeated step is not evidence of a root.

Numerical stopping rules

This bounded demonstration uses double-precision arithmetic and at most twelve updates. It stops first when |f(x)| ≤ 1e-10. Otherwise, a derivative magnitude ≤ 1e-12, a repeated estimate within 1e-12, or a proposed nonfinite estimate or magnitude above 1e6 stops the run. These thresholds are computational safeguards. A nearly repeated estimate is reported as a repeated iterate, not a proof of an exact mathematical cycle. A small residual measures the equation’s remaining mismatch; it does not certify the distance to a root or guarantee uniqueness. Near a flat root those quantities can differ substantially.

The iteration budget is an upper bound. In the positive square-root example, the fourth iterate is 665857/470832. Its exact residual is 1/221682772224, about 4.51e-12, so the residual test stops there even with a budget of twelve. Floating-point evaluation can differ slightly from that exact rational residual. The plot limits its displayed iterates to |x| ≤ 12; the ledger preserves any larger valid numerical values.

Historical Context

Newton’s method belongs to the long tradition of iterative root calculation. The square-root recurrence above is also known as the Babylonian method. Modern numerical analysis studies the derivative-based rule alongside fixed-point iteration, secant methods and bracketing methods, emphasizing convergence conditions and finite-precision computation rather than a universal success guarantee.

Real-world Applications

• Nonlinear equations: Solve model equations when the function and its derivative can be evaluated.
• Square roots: Repeatedly refine a numerical estimate using the simplified recurrence.
• Scientific software: Derivative-based updates are often combined with damping or a bracket to improve reliability. Those extensions are outside this demonstration.

Related Concepts

• Fixed Point Iteration — Newton’s update can be written as x_n+1 = g(x_n).
• Taylor Series — the tangent is the first-order local approximation.
• Gradient Descent — reduces an objective; Newton root-finding instead solves f(x) = 0.

References

• OpenStax, Calculus Volume 1, §4.9 — tangent construction, examples and failures.
• NIST Digital Library of Mathematical Functions, §3.8(ii) — local convergence and multiplicity.
• MIT OpenCourseWare, §13.1 — linear approximation and starting-point limitations.
• Cornell CS 4220, nonlinear equation lecture — local analysis and stopping criteria.