freakingpenguin blog

Fourier Part 2: Integrating a discontinuous function

2021-04-10T00:00:00Z

Checking if `gsl_integration_qng()` will work

At the end of part 1, I suggested that gsl_integration_qng() would not be able to successfully integrate the function we want to find the Fourier series of. Here is the function I will find the Fourier series of.

Because this function is discontinuous, we have to approximate it with a Fourier series instead of a Taylor series. If we recall the formulas shown in the last part, we know that we will have to integrate $f(x)$ from $0$ to $2l$ , among other things. Before calculating the Fourier series, let's try using gsl_integration_qng() to perform basic integration.

#include <stdio.h>
#include <math.h>
#include <gsl/gsl_integration.h>

/* f(x) = x for 0 <= x < 4, repeating */
double f(double x, void *params) {
  const double period = 4;
  return fmod(x, period);
}

int main() {
  double result, error;
  double low = 0, high = 8;
  gsl_function F = {.function = &f};
  double err_abs = 0, err_rel = 1;
  size_t num_evals;

  gsl_integration_qng(&F, low, high,
      		err_abs, err_rel,
      		&result, &error, &num_evals);
  printf("result error num_evals\n");
  printf("%f %f %zu\n", result, error, num_evals);
}

Results:
|    result |    error | num_evals |
| 14.804436 | 8.015135 |        21 |

Well, something happened here, but it's not quite what we wanted. We told GSL to take gsl_integral-i. Since we're dealing with a discontinuous function, we can rewrite this as gsl-integral-rewrite-i. Now I'm no mathematician, but this answer should be 16, not 14.8. Additionally, our error is very large, and if I attempt to lower err_rel to a smaller value, GSL yells at me saying that it failed to reach the requested error.

This is not a problem on GSL's end. We aren't using the right integration function for the job! According to the GSL manual,

The QNG algorithm is a non-adaptive procedure which uses fixed Gauss-Kronrod-Patterson abscissae to sample the integrand at a maximum of 87 points. It is provided for fast integration of smooth functions.

While that's a lot of words, the important part is the last sentence. gsl_integration_qng() is for smooth functions. Our function is not smooth, it has a discontinuity! We will need to find an alternative function in GSL that can handle discontinuous graphs.

`gsl_integration_qng()` and discontinuities

Fortunately we do not have to look far. The very next function mentioned in the manual is what we need. gsl_integration_qag() has the following signature.

int gsl_integration_qag(const gsl_function * f,
            double a, double b,
            double epsabs, double epsrel,
            size_t limit, int key,
            gsl_integration_workspace * workspace,
            double * result, double * abserr)

f, a, b, epsabs, epsrel, result, and abserr are all the same as gsl_integration_qng(). We can see that several new terms are introduced however.

workspace is a pointer to an area of memory used by gsl_integration_qag. We allocate space by using the gsl_integration_workspace_alloc() function, This function is passed an integer to adjust how much memory we want to allocate. Whatever integer we pass to gsl_integration_workspace_alloc()we need to ensure that limit is the same. This way, gsl_integration_qag knows how much memory it has available.

key is a integer between 0 through 6. GSL recommends using higher values when integrating smooth functions, and lower values when functions are discontinuous.

Let's see how well gsl_integration_qag() performs!

#include <stdio.h>
#include <math.h>
#include <gsl/gsl_integration.h>

double f(double x, void *params) {
  const double period = 4;
  return fmod(x, period);
}

int main() {
  size_t limit = 1024;
  gsl_integration_workspace *w
    = gsl_integration_workspace_alloc(limit);
  double result, error;
  double low = 0, high = 8;
  gsl_function F = {.function = &f};
  double err_abs = 0, err_rel = 1e-7;

  gsl_integration_qag(&F, low, high,
      		err_abs, err_rel,
      		limit, 1, w,
      		&result, &error);
  /* get into the habit of freeing memory when done! */
  gsl_integration_workspace_free(w);
  printf("result error\n");
  printf("%f %f\n", result, error);
}

Results:
| result | error |
|   16.0 |   0.0 |

Look at that! We are getting the exact value we expected, even though we're integrating with a discontinuity. We are now at the point where we can calculate the Fourier series for our function.

Generating our Fourier series

Let's recall that we can find our $a_{n}$ and $b_{n}$ coefficients with the following formulas.

a_{n}=\frac{1}{l}\int_{0}^{2l}f(x)\cos(\frac{n\pi x}{l})dx

b_{n}=\frac{1}{l}\int_{0}^{2l}f(x)\sin(\frac{n\pi x}{l})dx

GSL expects us to only pass one function to it as an argument. That means that, unlike before, we can't just pass a pointer to our function f(x), as GSL won't be multiplying it by the cosine and sine terms.

There are a couple of solutions to this that I can think of. The first is to make use of the void * argument that GSL includes in the gsl_function structure. Using this, we could pass extra information to f(x), adapting it to our specific n value.

Alternatively, we could write a parent function, like aorb_subn(). This function would then be stored in the gsl_function structure. The advantage of this approach is that the function we want the Fourier series of, f(x), is distinct in our code. This should make it a bit easier to change f(x) to any function that we want. This is the approach that I will take.

To make this code hopefully a bit more modular, I moved the integration out of main, and instead into a function called get_aorb_subn(). This function calculates the nth Fourier coefficient for a function f(x), using the variable get_a to determine if it should calculate $a_{n}$ or $b_{n}$ . It may be better to wrap this in a fourier structure that contains the two terms, but I elected not to do that to hopefully maintain some vague semblance of readability.

This is what the code to calculate the Fourier series looks like.

#include <stdbool.h>
#include <stdio.h>
#include <math.h>
#include <gsl/gsl_integration.h>

#define PI 3.14159
const double period = 4;
const double l = period / 2;

struct aorb_params {
  double (*f)(double x, void *params);
  int n;
  bool calc_a;
};

double f(double x, void *params) {
  return fmod(x, period);
}

double aorb_subn(double x, struct aorb_params *params) {
  int n = params->n;
  double trig = params->calc_a ? cos(PI*n*x/l) : sin(PI*n*x/l);
  return params->f(x, NULL) * trig;
}

double get_aorb_subn(double (*f)(double x, void *params), int n,
      	       double low, double high, bool get_a) {
  double normalization = 1/l;
  if (get_a && n == 0) normalization /= 2;

  size_t limit = 1024;
  gsl_integration_workspace *w
    = gsl_integration_workspace_alloc(limit);
  double result, error;
  gsl_function F = { .function = &aorb_subn,
    .params = &(struct aorb_params){ .f = f, .n = n,
      		 .calc_a = get_a } };
  double err_abs = 0, err_rel = 1e-7;

  gsl_integration_qag(&F, low, high,
      		err_abs, err_rel,
      		limit, 1, w,
      		&result, &error);
  gsl_integration_workspace_free(w);

  return normalization * result;
}

int main() {
  const int upto = 10;
  double a_subn[upto], b_subn[upto];
  double low = 0, high = period;

  for (int i = 0; i < upto; i++) {
    a_subn[i] = get_aorb_subn(f, i, low, high, true);
    b_subn[i] = get_aorb_subn(f, i, low, high, false);
    printf("%d %f %f\n", i, a_subn[i], b_subn[i]);
  }
}

Results:
| n | a_subn |    b_subn |
|---+--------+-----------|
| 0 |    2.0 |       0.0 |
| 1 | -7e-06 | -1.273242 |
| 2 | -7e-06 | -0.636621 |
| 3 | -7e-06 | -0.424414 |
| 4 | -7e-06 |  -0.31831 |
| 5 | -7e-06 | -0.254648 |
| 6 | -7e-06 | -0.212207 |
| 7 | -7e-06 | -0.181892 |
| 8 | -7e-06 | -0.159155 |
| 9 | -7e-06 | -0.141471 |

Like I mentioned before, aorb_subn is a "parent function" that combines f(x) and the sine/cosine term. This is the function we integrate, and we provide it enough information to know what term we are calculating.

Interestingly, if we use the $\pi$ constant M_PI, the integration fails due to roundoff error. Fortunately we can get "close enough" values by just using a few less digits.

Once we graph the Fourier series generated by these coefficients, this is what we get.

And there we are! Here is a Fourier series generated for a discontinuous periodic function. The more terms we add, the more accurate the approximation.

Fourier Part 1: What is a Fourier series?

2021-04-06T00:00:00Z

An overview of Taylor series

A fourier expansion is a way for us to approximate a function. If you've taken calculus before, you may have heard of a similar concept, Taylor series. With Taylor series, we can approximate any function as a sum of polynomials. For example, we can write $\sin x$ as

\sin x=x-\frac{x^{3}}{3!}+\frac{x^{5}}{5!}-\frac{x^{7}}{7!}

If we were to graph this, we see that as more terms are added, our approximation becomes more and more accurate. Here's a small demonstration.

In the graph below, I tried demonstrating how the Taylor series can "home in" on a function. Since we can't add part of a term in a Taylor series, I tried to demonstrate this effect by multiplying the term by a value. For example, when you see 1.1 terms, that means the first term + 0.1 * the second term. For $\sin x$ , that's $\sin x=x-0.1*\frac{x^{3}}{3!}$ .

Now, Taylor series can be useful, but they have significant limitations. First, we see that the Taylor series does a poor job modeling periodic functions. Even though $\sin x$ repeats, our Taylor series does not. At large $x$ values, this Taylor series is completely wrong!

Second, a Taylor series relies on the function being continuous (no holes or jumps). In addition to the function being continous, its derivatives must be as well. Let's consider the following graph.

Even though $f(x)$ is continuous, its derivative $f^{\prime}(x)$ is not. As such, a Taylor series cannot be used to approximate this function. This issue would come up even if $f^{\prime\prime\prime\prime\prime\prime\prime\prime\prime\prime}(x)$ was discontinuous. (Keep in mind that discontinuous is not the same thing as 0! We can model $f(x)=x$ as a Taylor series, as its higher order derivatives are continuous. They just also happen to be 0. Also, $f(x)=x$ is its own Taylor series.)

So, to summarize, Taylor series have the following issues:

They do not model periodic functions well
They require the function and all of its derivatives to be continuous

Fortunately, the Fourier series provides answers to both of these problems! (At least for most functions. Some functions, like a function that is discontinuous everywhere, exist solely for the purpose of making us sad.)

What we're all here for, Fourier series

I'll be sticking to the basics of Fourier series for now. Let's assume we have a periodic function $f(x)$ that has a period of $T$ . I am going to fintroduce the symbol l-i where $l=\frac{T}{2}$ . While this isn't the only option, we can write the Fourier series as

f⁢(x)=∑n=0∞an⁢cos⁡(n⁢π⁢xl)+bn⁢sin⁡(n⁢π⁢xl)

In this case, we can find $a_{n}$ and $b_{n}$ with the formulas

a_{n}=\frac{1}{l}\int_{0}^{2l}f(x)\cos(\frac{n\pi x}{l})dx

b_{n}=\frac{1}{l}\int_{0}^{2l}f(x)\sin(\frac{n\pi x}{l})dx

I won't go into detail as to where these formulas come from. (That is left as an exercise for the reader. Hah!) However, I will point out that we can adjust the limits of integration to any values we want, just as long as the difference between the upper and lower limits equals our period.

There is one special case that we need to discuss. When $n=0$ , we need a new formula for $a_{0}$ . This formula will look like

a_{0}=\frac{1}{2l}\int_{0}^{2l}f(x)dx

Fortunately there is no special case for $b_{0}$ . This occurs due to the fact that $a_{0}$ is a constant term ( $\cos 0=1$ ) as opposed to periodic.

That is all the theory that we need to calculate the Fourier series! As long as we can find a library that can perform integration for us, we should be able to calculate the Fourier series for any periodic function.

Using GSL to calculate an integral

Explanation of `gsl_integration_qng()`

Because of inscrutible magic mumbo jumbo, I decided to use C for calculating the Fourier expansion. In order to do that, I needed to pick out a library that could perform the integration for me. I settled on GSL or the GNU Scientific Library. There are many, many, MANY functions available in this library, but luckily I only need to worry about integration.

Before going too far into Fourier stuff, I'm going to do a simple sanity check so I can make sure I'm calculating integrals correctly. I want to calculate the following integral.

\int_{0}^{8}xdx

To do this using GSL, I can use the gsl_integration_qng() function. This function has the following signature.

int gsl_integration_qng(const gsl_function * f,
            double a, double b,
            double epsabs, double epsrel,
            double * result, double * abserr,
            size_t * neval)

f is a pointer to a structure that contains a function pointer. For those who don't speak nerd, this is how gsl-integration-qng() knows what function to integrate. It's our f(x). (Mostly. The reason for making it a structure is because the structure also contains a void * or void pointer. This void pointer can be used to pass parameters to the function. This could be used to let us change the slope of the function without having to modify the function itself.

a and b are the lower and upper limits of integration. That's fairly straightforward.

epsabs and epsrel help gsl_integration_qng() decide when to stop integrating. It's not possible to integrate the function to an exact value with GSL. Instead, it tries to zero in on a specific value or best guess as to what the answer is. epsabs is the absolute error that we want. If epsabs = 0.1, we don't know what the answer is, but we know we are no more than 0.1 away from it. epsrel is similar, but percentage based instead of absolute. (e.g. epsrel = 0.01 means our answer is within 1% of the actual value.)

result and abserr are used by the function to store the result and estimated absolute error, respectively. The number of iterations it took to calculate the result is stored in neval.

It is possible for the integration to fail. This might happen if we set the error tolerances too tight. Since the function we're integrating is so simple, I don't think that's a likely concern.

`gsl_integration_qng()` in use

To compile this program, you will need to tell the compiler what external libraries to use. You can do with with the -lgsl compile flag, e.g. gcc main.c -lgsl. Because GSL depends on another library, CBLAS, you will also need the -lcblas flag. If CBLAS isn't available, you can use a version of CBLAS provided by GSL with -lgslcblas. (Don't forget to install GSL!) Since I'm using some math function, I'm also going to include the math library with -lm.

In the end, our command will look like gcc main.c -lgsl -lcblas -lm, which will compile main.c and create an output file a.out. (Actually I'm using org-babel so I don't have to deal with this, but I'm assuming most readers here are not.)

Alright! Time to integrate! I need to preface the gsl_integration.h header with gsl/ as the headers are installed in a gsl subdirectory. (If you installed this through your systems package manager, you can probably find this file in /usr/include/gsl/.)

#include <stdio.h>
#include <gsl/gsl_integration.h>
#include <math.h>

/* This is where we define the function */
double f(double x, void *params) {
  return x;
}

int main() {
  double result, error;
  double low = 0, high = 8;
  gsl_function F = {.function = &f};
  double err_abs = 0, err_rel = 1;
  size_t num_evals;

  gsl_integration_qng(&F, low, high, err_abs, err_rel,
      		&result, &error, &num_evals);
  printf("result error num_evals\n");
  printf("%f %f %zu\n", result, error, num_evals);
}

Results:
| result | error | num_evals |
|   32.0 |   0.0 |        21 |

There we go! GSL was able to successfully integrate $\int_{0}^{8}xdx$ .

In the next part, we'll start using GSL to calculate the Fourier series of a discontinuous, periodic function. We'll see if we can naively get away with using the simple gsl_integration_qng() function, or if we need to find a more complicated, but more powerful, alternative.