Characteristic function

A real-valued random variable $X$ with a pdf $f_X(x)$ can be completely defined by its characteristic function ${\varphi }_X(t)$, which is defined as:

$${\varphi }_X(t)=E[e^{itX}]={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{itx}{f}_X(x)dx$$

where $i$ is the imaginary unit and $t$ is a real number.

Characteristic function provides a way to analyze the distribution of a random variable in the frequency domain (Fourier transform with sign reversal), and can be good alternative route to analytical results compared to using pdf or cdf.

TIP

In addition to univariate distributions, characteristic functions can also be defined for multivariate distributions.

Properties of characteristic function

1. Uniqueness: There is a one-to-one correspondence between cumulative distribution functions and characteristic functions. The pdf $f_X(x)$ can be recovered from the characteristic function ${\varphi }_X(t)$ by the inverse transform:

$$f_X(x)=\frac{1}{2\pi }{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{-itx}{\varphi }_X(t)dt$$

2. Continuity: ${\varphi }_X(t)$ is continuous for all $t$.

3. Convex combination: The convex combination of characteristic functions is also a characteristic function. $\sum _i{\alpha }_i\cdot {\varphi }_i(t)$ with $\sum _i{\alpha }_i=1$ and ${\alpha }_i\ge 0$ is also a characteristic function.

4. Multiplication: The product of finite characteristic functions is also a characteristic function.

Sum of independent random variables

If $X$ and $Y$ are independent random variables, then:

$${\varphi }_{X+Y}(t)={\varphi }_X(t)\cdot {\varphi }_Y(t)$$

5. For a real number $a$, ${\varphi }_{aX}(t)={\varphi }_X(at)$.

Generating moments from characteristic function

As ${\varphi }_X(t)$ can be written as a Taylor series expansion for $e^{itX}$:

$${\varphi }_X(t)=E[e^{itX}]=\sum _{n=0}^{\mathrm{\infty }}\frac{(it{)}^n}{n!}E[X^n]=1+itE[X]-\frac{t^{2}E[X^2]}{2}+\dots$$

To get the $n$'th moment of $X$, we can differentiate the characteristic function $n$ times and evaluate it at $t=0$:

$$E[X^n]={\frac{d^n}{d{t}^n}{\varphi }_X(t)|}_{t=0}$$

Moment generating function