Difference Methods

Overview:

In this notebook, we will start to consider a few difference methods - ways to numerically estimate derivatives. These methods provide us with recipes to construct numerical algorithms that can be used to solve differential equations.

In the examples in the previous two notebooks, we saw several different ways that we could approach the time stepping. However, in each of the examples, we had a clear way to estimate the derivative at any timestep - it was given by the differential equation. As we move toward more complex equations with higher order derivatives, we’ll need to approximate derivates in order to formulate a time stepping scheme.

Let’s review some common approximations here.

Difference Methods for First Derivatives

Forward Difference

A forward difference uses estimates of the solution at a time now and one time step in the future to estimate the derivative:

\[ y'(t) \approx \frac{y(t+\Delta t) - y(t)}{\Delta t} \]

Backward Difference

A backward difference uses estimates of the solution at a time now and one time step in the past to estimate the derivative:

\[ y'(t) \approx \frac{y(t) - y(t-\Delta t)}{\Delta t} \]

Centered Difference

A centered difference uses estimates of the solution one time step in the future and one time step in the past to estimate the derivative:

\[ y'(t) \approx \frac{y(t+\Delta t) - y(t-\Delta t)}{2\Delta t} \]

🤔 Sanity Check

In the solution above, we averaged the forward and backwards velocities to obtain a better solution - this is equivalent to the centered difference. Show that the centered difference is equivalent to averaging the forward and backward differences.

Difference Methods for Second Derivatives

In the examples above, we explicitly solved for velocity through time by considering the gravitational acceleration. However, we could also approximate the second derivative in the same way. Consider the forward difference as applied to the second derivative:

\[ y''(t) \approx \frac{y'(t+\Delta t) - y'(t)}{\Delta t} \]

What happens if we appoximate \(y'(t+\Delta t)\) and \(y'(t)\) each by backwards differences? We arrive at the formula below for the second derivative.

\[ y''(t) \approx \frac{y(t+\Delta t) - 2y(t) + y(t-\Delta t)}{\Delta t^2} \]

Hopefully you can see how you might apply formulas above to apprimxate higher order derivatives - try it yourself on a third derivative \(y'''(t)\)!