The Secant Method

Newton’s method is an extremely powerful technique, but it has a major weakness: the need to know the value of the derivative of $f$ at each approximation. Frequently, $f'(x)$ is far more difficult and needs more arithmetic operations to calculate than $f(x)$.  

To circumvent the problem of the derivative evaluation in Newton’s method, we introduce a slight variation. By definition,  

\[f'(p_{n-1}) = \lim_{x \to p_{n-1}} \frac{f(x) - f(p_{n-1})}{x - p_{n-1}}.\]

If $p_{n-2}$ is close to $p_{n-1}$, then  

\[f'(p_{n-1}) \approx \frac{f(p_{n-2}) - f(p_{n-1})}{p_{n-2} - p_{n-1}}= \frac{f(p_{n-1}) - f(p_{n-2})}{p_{n-1} - p_{n-2}}.\]

Using this approximation for $f'(p_{n-1})$ in Newton’s formula gives  

\[p_n = p_{n-1} - \frac{f(p_{n-1})(p_{n-1} - p_{n-2})}{f(p_{n-1}) - f(p_{n-2})}. \tag{2.12}\]

This technique is called the Secant method and is presented in Algorithm 2.4. (See Figure 2.10.) Starting with the two initial approximations $p_0$ and $p_1$, the approximation $p_2$ is the $x$-intercept of the line joining $(p_0, f(p_0))$ and $(p_1, f(p_1))$. The approximation $p_3$ is the $x$-intercept of the line joining $(p_1, f(p_1))$ and $(p_2, f(p_2))$, and so on. Note that only one function evaluation is needed per step for the Secant method after $p_2$ has been determined. In contrast, each step of Newton’s method requires an evaluation of both the function and its derivative. 

Figure 2.10

To find a solution to $f(x) = 0$ given initial approximations $p_0$ and $p_1$:  

INPUT initial approximations $p_0, p_1$; tolerance $TOL$; maximum number of iterations $N_0$.  

OUTPUT approximate solution $p$ or message of failure.  

• Step 1 Set $i = 2$;  \[q_0 = f(p_0); \quad q_1 = f(p_1).\]
• Step 2 While $i \leq N_0$ do Steps 3–6.  
• Step 3 Set\[p = p_1 - q_1 \frac{(p_1 - p_0)}{(q_1 - q_0)}. \quad \text{(Compute $p$.)}\]
• Step 4 If $|p - p_1| < TOL$ then 
  OUTPUT $(p;$ \quad The procedure was successful.) 
  STOP.  
• Step 5 Set $i = i + 1$.  
• Step 6 Set $p_0 = p_1; \quad (\text{Update } p_0, q_0, p_1, q_1.)$ 
  \[p_0 = p_1; \quad q_0 = q_1; \quad p_1 = p; \quad q_1 = f(p).\]
• Step 7 OUTPUT (‘The method failed after $N_0$ iterations, $N_0 =$’, $N_0$; The procedure was unsuccessful.) 
  STOP.

The next example involves a problem considered in Example 1, where we used Newton’s method with $p_0 = \pi/4$.

Example 1  

Use the Secant method to find a solution to $x = \cos x$, and compare the approximations with those given in Example 1 which applied Newton’s method.  

Solution In Example 1 we compared fixed-point iteration and Newton’s method starting with the initial approximation $p_0 = \pi/4$. For the Secant method we need two initial approximations. Suppose we use $p_0 = 0.5$ and $p_1 = \pi/4$. Succeeding approximations are generated by the formula  

\[p_n = p_{n-1} - \frac{(p_{n-1} - p_{n-2})(\cos p_{n-1} - p_{n-1})}{(\cos p_{n-1} - p_{n-1}) - (\cos p_{n-2} - p_{n-2})}, \qquad n \geq 2.\]

These give the results in Table 2.5.  

Comparing the results in Table 2.5 from the Secant method and Newton’s method, we see that the Secant method approximation $p_5$ is accurate to the tenth decimal place, whereas Newton’s method obtained this accuracy by $p_3$. For this example, the convergence of the Secant method is much slower than functional iteration but slightly slower than Newton’s method. (This is generally the case. See Exercise 14 of Section 2.4.)  

Newton’s method or the Secant method is often used to refine an answer obtained by another technique, such as the Bisection method, since these methods require good first approximations but generally give rapid convergence.