Function Sequences and Series

Cauchy Sequences §

We say the sequence

$$\{a_n{\}}_{n=1}^{\infty }\in \mathbb{R}$$

is Cauchy if $\forall ϵ>0$, $\exists N\in \mathbb{N}$ such that $\forall n,m\ge N$,

$$|a_n-a_m|<ϵ$$

Theorem: Boundedness of Cauchy Sequences

Let $\{a_n\}$ be a Cauchy sequence. Then, $\{a_n\}$ is bounded.

Proof

This proof is very similar to our proof on boundedness of convergent sequences!

Let $\{a_n\}$ be Cauchy. Then,

$$\begin{array}{rl}|a_n|& =|a_n-a_N+a_N|\\ & \le |a_n-a_N|+|a_N|\\ & <ϵ+|a_N|\end{array}$$

This guarantees a bound for our larger values! So, we take $M=\max \{ϵ+|a_N|,|a_n|,\dots |a_{N-1}|\}$ to find a bound.

Theorem: Convergence of Cauchy Sequences

Let $\{a_n\}$ be a sequence of real numbers. Then, $\{a_n\}$ is Cauchy if and only if it converges.

Note that this only holds because our sequence is on real numbers! In other spaces, this may not hold.

Proof

Proof ($\leftarrow$) §

Suppose $\{a_n\}$ converges. Let $a$ be what it converges to. Then,

$$\begin{array}{rl}|a_n-a_m|& =|a_n+a-a-a_m|\\ & \le |a_n-a|+|-(a_m-a)|\\ & <\frac{ϵ}{2}+\frac{ϵ}{2}\\ & <ϵ\end{array}$$

Proof ($\to$) §

Suppose $\{a_n\}$ is Cauchy. By definition, $\forall ϵ>0$, $\exists N\in \mathbb{N}$ such that $\forall n,m\ge N$,

$$|a_n-a|<ϵ$$

We know that Cauchy sequences are bounded. By the Bolzano-Weierstrass Theorem, we can extract a convergent subsequence $\{a_{n_k}\}\to L$ forall $k\ge K$.

Finally, fix $N$ and $K$. Then,

$$\begin{array}{rl}|a_n-L|& \le |a_n-a_{n_k}|+|a_{n_k}-L|\\ & <\frac{ϵ}{2}+\frac{ϵ}{2}\\ & <ϵ\end{array}$$

For $n\ge N$, $n_k\ge N$, and $k\ge K$. Note that we can always obtain this, as $n_k$ is a strictly increasing seqeuence, so we can keep increasing $K$ until $n_k\ge N$.

Cauchy sequences can be very helpful in determining convergence! Recall how before, we could show that a sequence converges, but we had to know what it converged to in order to prove this! Cauchy sequences avoids this limitation.

Infinite Series §

Let $S_n$ be a partial sum, given as

$$S_n=\sum _{k=0}^n{a}_k$$

Where $\{a_k\}$ are terms of a sequence. We say it is a series when $n\to \infty$, and this series converges if

$$S=\underset{n\to \infty }{\lim }{S}_{n}S\in \mathbb{R}$$

Otherwise, if $S$ does not exist, then the series diverges.

Proofs for some of the series tests are given below.

Telescoping Sums (Proof)

By assumption, suppose that $\{c_n\}\to 0$.

$$\begin{array}{rl}{s}_n& =\sum _{k=m}^{\infty }(c_{k-1}-c_k)\\ & =(c_{m_1}-c_m)+(c_m-c_{m+1})+\cdots +(c_{n-1}-c_n)\\ & =c_{m-1}-c_n\to c_{m-1}\end{array}$$

Divergence Test (Proof)

Suppose ${\sum }_{k=0}^{\infty }{a}_k$ converges. Then,

$$a_n=S_n-S_{n-1}\to 0$$

So by contrapositive, if $a_n\nrightarrow 0$, then the series diverges.

Geometric Series (Proof)

$$\begin{array}{r}{s}_n=\sum _{k=0}^n{r}^k=1+r+r^2+\cdots +r^n\\ r{s}_n=r+r^2+r^3+\cdots +r^{n+1}\\ (1-r)s_n=1-r^{n+1}\\ s_n=\frac{1-r^{n+1}}{1-r}\to \frac{1}{1-r}|r|<1\end{array}$$

Series with Non-Negative Terms (Proof)

$$s_n=\sum _{k=0}^n{a}_k{a}_k\ge 0$$

By the Monotone Convergence Theorem, if $\{s_n\}$ is bounded, then it converges, and it converges to its supremum.

Direct Comparison Test (Proof)

$$\begin{array}{r}0\le a_k\le b_k\\ 0\le \sum _{k=1}^n{a}_k\le \sum _{k=1}^n{b}_k\end{array}$$

Suppose $\sum b_k$ exists. Then, $\sum a_k$ exists by the Monotone Convergence Theorem.

Otherwise, if $\sum a_k=\infty$, then $\sum b_k=\infty$ as no number can be greater than infinity.

Limit Comparison Test (Proof)

We have

$$a_k\ge 0,b_k\ge 0\underset{k\to \infty }{\lim }\frac{a_k}{b_k}=c>0$$

Thus, $\forall ϵ>0$, $\exists N\in \mathbb{N}$ such that $\forall k\ge N$,

$$\left|\frac{a_k}{b_k}-c\right|<ϵ$$

This gives us

$$0\le b_k(-ϵ+c)<a_k<b_k(ϵ+c)$$

Taking the summation, we can apply the direct comparison test to determine convergence or divergence.

Basically, we find that our partial sums are approximately equal (with a multiplier), so anything one sum does, the other must also do.

Integral Test & P-Test (Proof)

We have

$$0\le \sum _{n=1}^{\infty }f(n+1)\le {\int }_a^{\infty }f(x)dx\le \sum _{n=1}^{\infty }f(n)$$

Formed with the left-hand and right-hand Riemann sums, whose comparisons are known as $f$ is monotonically decreasing.

Apply direct comparison test.

Apply integral test on $p-Test$ to show convergence on $p>1$.

Cauchy Convergence Criterion (Proof)

Suppose $\{s_n\}$ is Cauchy. Then, we can combine $s_m,s_n$ into a single sum, which by definition is bounded by $ϵ$.

Absolute Convergence Test (Proof)

Let $s_n={\sum }_{k=1}^n{a}_k$. We show $s_n$ is Cauchy

$$0\le |s_{n+k}-s_n|=|\sum _{i=n+1}^{n+k}{a}_i|\le \sum _{i=n+1}^{n+k}|a_i|<ϵ$$

We have $<ϵ$ provided our $n$ is large enough.

We now apply the Cauchy Sequence Criterion to obtain convergence.

Convergence of Function Sequences §

Pointwise Convergence §

We say a sequence of functions $\{f_n\}\to f$ pointwise on $D\subseteq \mathbb{R}$, if $\forall x\in D,\forall ϵ>0,\exists N\in \mathbb{N}$ such that $\forall n\ge N$,

$$|f_n(x)-f(x)|<ϵ$$

Or in other words, $\forall x\in D$,

$$\underset{n\to \infty }{\lim }{f}_n(x)=f(x)$$

Notice how we fix our $x$ first, before choosing $N$! This means that $N$ can rely on both $x$ and $ϵ$.

Example: Pointwise Convergence Proof

Let $f_n(x)=\frac{nx}{(1+nx{)}^2}$. Show that $\{f_n(x)\}\to f(x)=0$ pointwise on $x\in [0,\infty )$.

Pick any arbitrary $ϵ>0$, $x\in [0,\infty )$. To show pointwise convergence, we want to find an $N\in \mathbb{N}$ such that

$$|f_n(x)-f(x)|<ϵ$$

We find this $N$. Assume $x\ne 0$.

Note that if $x=0$, then $|f_n(x)-f(x)|=0<ϵ$.

$$\begin{array}{rl}|f_n(x)-f(x)|& =\left|\frac{nx}{(1+nx{)}^2}-0\right|=\left|\frac{nx}{(1+nx)(1+nx)}\right|\\ & \le \left|\frac{nx}{(nx)(1+nx)}\right|=\left|\frac{1}{1+nx}\right|\\ & \le \left|\frac{1}{nx}\right|<ϵ\end{array}$$

By the Archimedian property, we can guarantee the existence of a $N\in \mathbb{N}$ such that

$$\frac{1}{N}<ϵx$$

Choose this $N$. Then, $\forall n\ge N$, we have $|f_n(x)-f(x)|<ϵ$, giving us pointwise convergence!

Pointwise convergence is a very weak notion of convergence! Let’s see why.

Example: Weakness of Pointwise Covergence

$$f_n(x)=x+\frac{1}{n}$$

Fix $x\in \mathbb{R}$. Then, as $n\to \infty$, $\{f_n(x)\}\to x$, giving us pointwise convergence.

Some notable things we can see from this example:

1. All functions $f_n$ are continuous on $x\in \mathbb{R}$, and so is $f$.
2. The limit of our derivatives is equal to our function derivative
   $$\underset{n\to \infty }{\lim }{f}_n^{\prime }(x)=1=f^{\prime }(x)$$
3. The limit of our integrals is equal to our function integral
   $$\underset{n\to \infty }{\lim }{\int }_a^b{f}_n(x)dx=\frac{1}{2}{x}^2{|}_a^b={\int }_a^{b}f(x)dx$$

However none of these properties always hold for pointwise convergence! This makes pointwise convergence is extremely weak, as we can’t really use it for anything.

Below, we provide counterexamples to each of the “observations” above.

Example: Weakness of Pointwise Convergence (1)

Consider $f_n(x)=x^n$ on $[0,1]$. Then,

$$\{f_n\}\to f(x)\text{pointwise}=\{\begin{array}{ll}0& x\in [0,1)\\ 1& x=1\end{array}$$

So, we have pointwise convergence, but the limit ($f$) is not continuous!

Example: Weakness of Pointwise Convergence (2)

Consider $f_n(x)=\sin (nx)/n$. Observe that as $n\to \infty$,

$$\underset{n\to \infty }{\lim }{f}_n(x)=0$$

However,

$$f_n^{\prime }(x)=\cos (nx)$$

Does not converge to $f$’s derivative which is 0!

We can show this by taking $\cos (2nx)$, applying a trig identity, and obtaining a contradiction by seeing that the identity gives us convergence to -1!

Example: Weakness of Pointwise Convergence (3)

$$f_n(x)=\{\begin{array}{ll}0& x\in \{q_1,q_2,\dots q_n\},q_i\in \mathbb{Q}\\ 1& \text{else}\end{array}$$

Then,

$$f_n(x)\to f(x)\text{pointwise}=\{\begin{array}{ll}0& x\in \mathbb{Q}\cap [0,1]\\ 1& \text{else}\end{array}$$

So the integral of ${\int }_0^1{f}_n(x)=1$ (we have a finite number of points), but the integral of the limit doesn’t exist!

Uniform Convergence §

We say a sequence of functions $\{f_n\}\to f$ uniformly on $D\subseteq \mathbb{R}$, if $\forall ϵ>0$, $\exists N\in \mathbb{N}$ such that $\forall n\ge N$, $\forall x\in D$

$$|f_n(x)-f(x)|<ϵ$$

Notice how we fix our $x$ after choosing $N$! This is called uniform convergence, as our $N$ is no longer dependent on $x$, so it should work “uniformly” for all $x$.

Uniform convergence is a much stronger notion of convergence than pointwise. In fact, it implies pointwise convergence.

Theorem: Uniform and Pointwise Convergence

If $\{f_n\}\to f$ uniformly on $D$, then $\{f_n\}\to f$ pointwise.

Example: Uniform Convergence Proof

$$f_n(x)=x+\frac{1}{n}$$

Note that ${\lim }_{n\to \infty }{f}_n(x)=f(x)=x$. Let’s show $\{f_n\}\to f$ uniformly. Fix $ϵ>0$. We want

$$|f_n(x)-f(x)|=\left|x+\frac{1}{n}-x\right|=\left|\frac{1}{n}\right|\le \left|\frac{1}{N}\right|<ϵ$$

Choose $N$ such that $\frac{1}{N}<ϵ$, whose existence is guaranteed by the Archimedian Property. Then, we have uniform convergence.

Example: Uniform Convergence Disproof

Let $f_n(x)=\frac{n}{n+x}$.

Show that $f_n(x)\nrightarrow f(x)$ uniformly on $x\in [0,\infty )$.

First, observe that ${\lim }_{n\to \infty }{f}_n(x)=f(x)=1$. Now, suppose by way of contradiction that $f_n(x)\to f(x)$ uniformly.

By definition, $\forall ϵ>0$, $\exists N\in \mathbb{N}$ such that $\forall n\ge N,\forall x\in [0,\infty )$,

$$|f_n(x)-f(x)|<ϵ$$

Let $ϵ=\frac{1}{6}$, and choose the $N$ satisfying this definition. Then, it should be true that $\forall n\ge N,\forall x\in [0,\infty )$,

$$|f_n(x)-f(x)|=\left|\frac{n}{n+x}-1\right|=\left|\frac{x}{n+x}\right|<ϵ$$

Let $x=\frac{1}{2}$, which is in our domain. Then, we have

$$\left|\frac{x}{n+x}\right|=\left|\frac{n/2}{3n/2}\right|=\frac{1}{3}\nless \frac{1}{6}$$

Which is a contradiction! Thus, we do not have uniform convergence.

Example: Uniform Convergence Disproof (2)

$$f_n(x)=x^{n}x\in [0,1]$$

As shown before, this converges pointwise to

$$f(x)=\{\begin{array}{ll}1& x=1\\ 0& x\in [0,1)\end{array}$$

Assume $f_n\to f$ uniformly. Then, $\forall ϵ>0$, $\exists N\in \mathbb{N}$ such that $\forall n\ge N,\forall x\in [0,1]$,

$$|x^n-f(x)|<ϵ$$

Let’s choose $ϵ=\frac{1}{10}$. So,

$$|x^n-f(x)|<\frac{1}{10}\forall x\in [0,1]$$

Choose $x={\left(\frac{1}{2}\right)}^{1/n}$, which is in our interval. Then,

$$\left|{\left({\left(\frac{1}{2}\right)}^{1/n}\right)}^n-f(x)\right|\nless \frac{1}{10}$$

We say sequence $\{f_n\}$ is uniformly Cauchy on $D\subseteq \mathbb{R}$ if $\forall ϵ>0$, $\exists N\in \mathbb{N}$ such that $\forall n\ge N,\forall k\in n,\forall x\in D$,

$$|f_{n+k}(x)-f_n(x)|<ϵ$$

While a complex definition, this lets us show convergence even if we don’t know the limit!

Theorem: Uniform Cauchy and Uniform Convergence

Uniform Cauchy convergence implies uniform convergence.

Example: Uniform Cauchy Convergence

$$f_n(x)=\sum _{j=1}^n\frac{x^j}{j{2}^j}{f}_n:[-1,1]\to \mathbb{R}$$

We show this is uniformly convergent using our Cauchy definition. Let $ϵ>0$. We want

$$\begin{array}{rl}\left|f_{n+k}(x)-f_n(x)\right|& =\left|\sum _{j=1}^{n+k}\frac{x^j}{j{2}^j}-\sum _{j=1}^n\frac{x^j}{j{2}^j}\right|\\ & =\left|\sum _{j=n+1}^{n+k}\frac{x^j}{j{2}^j}\right|\le \sum _{j=n+1}^{n+k}\frac{|x^j|}{j{2}^j}\\ & \le \sum _{j=n+1}^{n+k}\frac{1}{j{2}^j}\le \sum _{j=n+1}^{\infty }\frac{1}{j{2}^j}\\ & \le \sum _{j=n+1}^{\infty }\frac{1}{2^j}<ϵ\end{array}$$

Provided $N$ is large enough. We find this $N$ using the Geometric Series Test to finish.

Uniform convergence is much stronger than pointwise convergence! In fact, properties of continuity, differentiation, and integration that did not hold for pointwise convergence, now hold for uniform convergence!

Theorem: Continuity and Uniform Convergence

If $f_n:D\to \mathbb{R}$ is continuous for each $n$, and $f_n\to f$ uniformly, then $f$ is continuous.

The same holds for uniform continuity on $f_n$ and $f$.

Proof

By definition of $f_n$’s continuity, $\forall ϵ>0$, $\exists \delta >0$ such that $\forall x\in D$,

$$|x-x_0|<\delta \to |f_n(x)-f_n(x_0)|<\frac{ϵ}{3}$$

And by definition of uniform convergence, $\forall ϵ>0$, $\exists N\in \mathbb{N}$ such that $\forall n\ge N,\forall x\in D$,

$$|f_n(x)-f(x)|<\frac{ϵ}{3}$$

We want to show that $\forall ϵ>0$, $\exists \delta >0$ such that $\forall x\in D$,

$$|x-x_0|<\delta \to |f(x)-f(x_0)|<ϵ$$

Fix $ϵ>0$. If $|x-x_0|<\delta$, then

$$\begin{array}{rl}|f(x)-f(x_0)|& =|f(x)-f(x_0)+f_n(x)-f_n(x)+f_n(x_0)-f_n(x_0)\\ & \le |f_n(x)-f_n(x_0)|+|f_n(x)-f(x)|+|f_n(x_0)-f(x_0)|\\ & <\frac{ϵ}{3}+\frac{ϵ}{3}+\frac{ϵ}{3}=ϵ\end{array}$$

Theorem: Integration and Uniform Convergence

Let $f_n\to f$ uniformly on $[a,b]$, and assume ${\int }_a^b{f}_n$ exists for al $n$. Then,

$$\underset{n\to \infty }{\lim }{\int }_a^b{f}_n={\int }_a^b\underset{n\to \infty }{\lim }{f}_n={\int }_a^{b}f$$

Proof

By definition, $\forall ϵ>0$, $\exists N\in \mathbb{N}$ such that $\forall n\ge N,\forall x\in [a,b]$,

$$|f_n(x)-f(x)|<ϵ⟹f_n(x)-ϵ<f(x)<f_n(x)+ϵ$$

Taking the upper integral on the right-side inequality, and the lower integral on the left-side inequality,

$$\begin{array}{r}\overline{{\int }_a^b}f<\overline{{\int }_a^b}{f}_n(x)dx+ϵ(b-a)\\ \underline{{\int }_a^b}{f}_n(x)dx-ϵ(b-a)<\underline{{\int }_a^b}f\end{array}$$

Combining these, we get

$$\begin{array}{r}0\le \overline{{\int }_a^b}f-\underline{{\int }_a^b}f<\left(\overline{{\int }_a^b}{f}_{n}dx+ϵ(b-a)\right)+\left(ϵ(b-a)-\underline{{\int }_a^b}{f}_{n}dx\right)\\ 0\le \overline{{\int }_a^b}f-\underline{{\int }_a^b}f<2ϵ(b-a)\end{array}$$

Thus, by squeeze theorem, the upper and lower integral of $f$ is the same, so $f$ is integrable. Knowing that $f$ is integrable, we can now integrate our first inequality to get

$$\begin{array}{r}{\int }_a^b{f}_n(x)dx-ϵ(b-a)<{\int }_a^{b}f(x)<{\int }_a^b{f}_n(x)dx+ϵ(b-a)\\ \left|{\int }_a^b{f}_n(x)-{\int }_a^{b}f\right|<ϵ(b-a)\end{array}$$

Letting $ϵ\to 0$, we get equality! So, as $ϵ\to 0$, ${\int }_a^b{f}_n\to {\int }_a^{b}f$.

Theorem: Differentiation and Uniform Convergence

Let $f_n\in C^1(D)$ for all $n$. Suppose $f_n\to f$ pointwise on $D$, and suppose $f_n^{\prime }\to g$ uniformly on $D$, where $g$ is some function.

Then, $g(x)=f^{\prime }(x)$, and $f^{\prime }\in C^1(D)$.

If we don’t know the limit $g$, we can also find that $f_n^{\prime }$ is uniformly Cauchy.

Proof

$\forall x,x_0\in D$, we have that

$$f_n(x)-f_n(x_0)={\int }_{x_0}^x{f}_n^{\prime }(t)dt$$

Taking the limit as $n\to \infty$, we get

$$\begin{array}{r}\underset{n\to \infty }{\lim }{f}_n(x)-f_n(x_0)=\underset{n\to \infty }{\lim }{\int }_{x_0}^x{f}_n^{\prime }(t)dt\\ f(x)-f(x_0)={\int }_{x_0}^{x}g(t)dt\end{array}$$

Since the map $x\to {\int }_{x_0}^{x}g$ is continuous, we get that $f$ is continuous.

Also, $f^{\prime }(x)=g(x)$, as we take the derivative of both sides and apply the second fundamental theorem. Note that this works because $g$ is continuous, as $f_n^{\prime }$ are continuous, and they converge uniformly to $g$.

Power Series §

A power series is defined as

$$f(x)=\sum _{n=0}^{\infty }{a}_n(x-x_0{)}^n$$

We’ll typically just use $x_0=0$.

We can often find the domain of $f$ using a Ratio Test (or Root Test) where we check the endpoints as well. Can we differentiate and integrate this series?

Theorem (9.40 Fitzpatrick)

Define the power series

$$f(x)=\sum _{k=0}^{\infty }{a}_k{x}^k$$

And let $x_0$ be in its domain of convergence, $D$. Furthermore, let $0<r<|x_0|$.

Then, $[-r,r]\subseteq D$ and

$$\sum _{k=0}^{\infty }{a}_k{x}^k\sum _{k=1}^{\infty }{a}_{k}k{x}^{k-1}$$

Both converge uniformly on $[-r,r]$.

This means that the partial sums converge uniformly!

This does not mean that we have uniform convergence on $D$! For example,

Example: Uniform Convergence Counterexample

$$f(x)=\sum _{n=0}^{\infty }{x}^n=\frac{1}{1-x}$$

We have that the domain of convergence is $(-1,1)$. Since our partialsums are bounded on $(-1,1)$, but the limit is unbounded, we cannot have uniform convergence!

Theorem: Integrability of Power Series

Let $[a,b]\subseteq (-r,r)\subset D$, where

$$f(x)=\sum _{k=0}^{\infty }{a}_k{x}^k$$

Then, we’re allowed to integrate our term!

$${\int }_a^{b}f(x)dx=\sum _{k=0}^{\infty }\left({\int }_a^b{a}_k{x}^{k}dx\right)=\sum _{k=0}^{\infty }{a}_k\frac{x^{k+1}}{k+1}{|}_a^b$$

Proof

We know by Theorem 9.40 that our partial sums converge uniformly.

So,

$$\underset{n\to \infty }{\lim }{\int }_a^b{s}_n(x)dx={\int }_a^b\underset{n\to \infty }{\lim }{s}_n(x)dx$$

This is enabled by our uniform convergence!

This simplifies to

$$\begin{array}{rl}& \sum _{k=0}^{\infty }\left({\int }_a^b{a}_k{x}^k\right)=\underset{n\to \infty }{\lim }{\int }_a^b{s}_n(x)dx={\int }_a^b\underset{n\to \infty }{\lim }{s}_n(x)dx={\int }_a^{b}f(x)dx\\ & {\int }_a^{b}f(x)dx=\sum _{k=0}^{\infty }\frac{a_k{x}^{k+1}}{k+1}{|}_a^b\end{array}$$

Theorem: Differentiability of Power Series

Let $(-r,r)\subset D$ where

$$f(x)=\sum _{k=0}^{\infty }{a}_k{x}^k$$

Then, $f\in C^{\infty }(-r,r)$ and

$$f^n(x)=\sum _{k=n}^{\infty }{a}_k\frac{d^n}{d{x}^n}{x}^k=\frac{k!}{(k-n)!}{x}^{k-n}$$

Note that $f^n(0)=a_{n}n!$, as we have a $0^0$ term which we define as equal to 1.

Proof

We know the partial sums $\{s_n(x)\}$ and $\{s_n^{\prime }(x)\}$ converge uniformly on $[-R,R]\subseteq (-r,r)$ by Theorem 9.40.

So $s_n\to f$ uniformly on $[-R,R]$, $s_n^{\prime }\to g$ uniformly to some $g$. But as $s_n\to f$, then $s_n^{\prime }=f^{\prime }$ by property of uniform convergence, so $s_n\to f^{\prime }$ uniformly on $[-R,R]$. Taking our limit, we get

$$\begin{array}{rl}\underset{n\to \infty }{\lim }{s}_n^{\prime }(x)& =\underset{n\to \infty }{\lim }\frac{d}{dx}\sum _{k=0}^n{a}_k{x}^k\\ & =\underset{n\to \infty }{\lim }\sum _{k=1}^n{a}_{k}k{x}^{k-1}\\ & =f^{\prime }(x)=\frac{d}{dx}\sum _{k=0}^{\infty }{a}_k{x}^k\end{array}$$