An Introduction To Numerical Analysis - Part 1 (Preliminaries)

 Numerical analysis is a powerful tool for finding approximate solutions to complex mathematical problems, particularly differential equations. In the fields of space plasma physics and solar physics, whole courses are devoted to numerical analysis approaches that can then be applied to numerical simulations of specific problems - say the development of auroral substorms, or the origins of coronal mass ejections. In this first part of my introduction we will not dive in at that deep end but confine attention to preliminaries.  

In fact though, we've already seen some cases of how numerical analysis can be applied, for instance in power series solutions to differential equations:

• Looking At Power Series Solutions To Differential Equations...   
•    In the above post/link we see a key element in numerical analysis, the use of Taylor series as an approximation to a function.      

      Let f  then be a function defined on the closed interval [a, b] and f has n continuous derivatives on [a, b] then f ⁿ⁺¹ (x)  exists for all x in [a, b]. And:

f(b) = f(a) + f ’(a) × (b – a) + f ” (a)/ 2! (b – a)² +

….+   f ⁿ (a) / n! (b – a)ⁿ  + f ⁿ⁺¹ (c)/ (n +1)! [(b- a) ⁿ⁺¹]

Where c lies between a and b. 

So in the case of f(x) in the power series solns. post:

f(x) = f(x _o ) + f ’(x _o) × (x – x _o) + f ” (x _o) (x – x _o)² / 2!  +

 f ”’ (x _o) (x – x _o)³ / 2!  +   f ⁿ (x _o) (x– x _o)ⁿ / n!  +  ….

This is provided all derivatives exist at x = x _o

An example of use of Taylor series in a specific problem:

Integrate:  ò ^b _a  f(x)  dx

 Using a Taylor series method and justify it. 

Solution:

Take f(x) = å ^¥ _{n = 0}    f ⁿ (a) / n! (x – a) ⁿ

Then:    ò ^b _a    f(x)  dx = ò ^b _a    f(a)  dx +  ò ^b _a  f ’(a) × (x – a)   dx

+ ò ^b _a     f ” (a)/ 2! (x – a)²

ò ^b _a  f(x)  dx = [f (a) × (b – a)  + f ’(a)/ 2! (x – a)²  ÷ ^b _a 

+ ò ^b _a f ” (c)/ 2! (x – a)²

 The addition of the two terms in the bracket, with b, a limits, will yield a least upper bound to the value of the integral.  Given a function f, assume it is infinitely differentiable at some point a. Then we can write:

f(x) =  å ^¥ _{n = 0}    f ⁿ (a) / n! (x – a) ⁿ

Series and partial sums can also factor into numerical analysis, and convergence is often at the forefront. 

If {S _n } converges we say the series converges.

Where:  S _n  = å ^k _k = 1   u _n  =   u ₁  +   u ₂  +   u ₃  +…. u _n

From which we can get a sequence of partial sums:

E.g.  S ₁   =  u _{1 ,}    S ₂   =  u ₁ +   u ₂ ,     S ₃   =  u ₁ +   u ₂ +   u ₃

 Example:

å ^¥ _n = 1    1/ (n+1) =   å ^¥ _n = 1   (1/n  -  1/ (n +!)  =   1

S _k  = ( 1 – ½)  +(1/2 – 1/3) +  (1/3 – ¼)  + …..(1/k – 1/ k -1) =  1

So: S _k  =  1 - 1/ k + 1   converges to 1

If:  å u _n   converges then:  lim  u _n ® 0o  =  0  

For absolute convergence:  Given: å ^¥ _n = 1  u _n

If  å ^¥ _n = 1  ÷ u _n ÷  converges then å u _n   converges absolutely

Ratio Test for absolute convergence.  

Compute:    lim _n® 0o÷ u _n+1  / u _n ÷ =  L

If  L   <  1  the series converges

If  L   >  1  the series diverges

If  L   =  1  the test fails, so no result

Example: å ^¥ _n = 1   n  (2/3) ⁿ

lim _n® 0÷ n + 1  (2/3) ⁿ⁺¹ /  n (2/3) ⁿ ÷ =  lim _n® 0o ÷ (2/3)(1 + 1/n)÷  =

=  2/3  < 1   (So converges absolutely)

Now back to more basic generalities to see how numerical analysis works in the most primitive settings.

We take x and y as real numbers ( R ) and let F be a generic computing system. Then x* and y*  are the representations of x and y in F.   Thus we can write for translation sake:  

x =  x*  +   e

y  =  y*  +  h

Let w denote a binary operation and let w* be the pseudo operation correction for w. Then if I want:  x w y  I actually get: x* w* y*.  So that:

x w y =  x* w* y*   = (x w y -  x* w y*) + (x* w y*  -   x* w* y*)

Hence:   

÷ x w y -  x* w* y*÷  <  ÷ x w y -  x* w  y*÷  + 

÷ x* w y* -  x* w*  y*÷ 

Then the relative error is:

 ÷ x w y -  x* w* y*÷ / x* w*  y*

While the absolute error is:  ÷ x w y -  x* w* y*÷  

This primitive and admittedly reductive example gets to the heart of numerical analysis.

Suggested Problems For Math Nerds:

1) Evaluate:   5 x⁴  + 4 x³   +   6 x²  + 2x + 2 

At x = 1.2  using Horner's approximation method

You may use the "Horner calculator" e.g.

Online calculator for the Horner scheme (elsenaju.eu)

But then write all steps in long form ultimately comparing: 

a) 4 multiplications, 4 additions, and 

b) 10 multiplications, 4 additions.

2) Check to see if the series:

å ^¥ _n = 1   2ⁿ  (x + 5)ⁿ   /n ²

Converges.

Challenge Problem:

The Taylor series can be used to find an approximation to a function’s value, i.e. of a square root, in terms of the order n desired viz.

P _n  =   f(n) + f ’(n) × (x – n) + f ” (n)/ 2! (x – n)² +

….+   f ^”’ (n) / 3! (x – n)³  …..+ f ⁿ⁺¹ (n)/ (n +1)! [(x- n) ⁿ⁺¹]

Use the Taylor series to 4^th order (n = 4 or  P ₄) to obtain the value of Ö 5