Some distributions

Introduction of some distribution

random variable

probability density function

For continous variable x:

$E(x) = \int xf(x)dx = \int x dF(x)$

For discrete variable x:

$E(x)=\sum x_i p_i$

Variance:

$Var(x) = E(x^2)-(E(x))^2$

Binomial distribution

For example, flip a coin for 5 times, define random variable X as number of heads after 5 flips. The probability is defined as:

$p(X=n)=\frac{5!}{n!(5-n)!}=C_5^n$

The general probability is

$p(X=k)=\frac{n!}{k!(n-k)!}=C_n^k$

And the

$E(X) = np=\sum_{k=0}^{n}kC_n^kp^k(1-p)^{n-k}$

The term for $k=0$ is 0, hence it can be rewrite as

$E(X) =\sum_{k=1}^{n}kC_n^kp^k(1-p)^{n-k}$

Then

$\begin{aligned} E(x) &= \sum_{k=1}^{n}k\frac{n!}{k!(n-k)!}p^k(1-p)^{(n-k)} \\ &= \sum_{k=1}^{n}k\frac{n(n-1)!}{k(k-1)!(n-k)!}p p^{k-1}(1-p)^{(n-k)} \\ &= np\sum_{k=1}^{n}\frac{(n-1)!}{(k-1)!(n-k)!}p^{k-1}(1-p)^{(n-k)} \end{aligned}$

Let $a = k-1$, $b=n-1$ $\Rightarrow k=a+1, n=b+1$

$\begin{aligned} E(x)&= np\sum_{k=1}^{n}\frac{(n-1)!}{(k-1)!(n-k)!}p^{k-1}(1-p)^{(n-k)} \\ &= np\sum_{a=0}^{b}\frac{b!}{a!(b-n)!}p^{a}(1-p)^{(b-a)} \end{aligned}$

Where $\sum_{a=0}^{b}\frac{b!}{a!(b-n)!}p^{a}(1-p)^{(b-a)}$ is the sum of all the probabilities which equals to 1.

Hence the $E(x)$ is $np$

Poisson distribution

For example, we want to know X = how many car pass in an hour. Assume the car is randomly pass on the road.

Say $E(X) = \lambda = np$, where $\lambda$ has unit cars/hour $\Rightarrow 60min/hour \frac{\lambda}{60} cars/min$

Hence the probability can be written as whether there is a car pass in that minute which follows binomial distribution

$p(X=k)=C_{60}^k(\frac{\lambda}{60})^k(1-\frac{\lambda}{60})^{60-k}$

But the question is if there are more than 1 car pass in a minute, how do we deal with it? The most straight way is to set minutes into second

$p(X=k) =C_{3600}^k(\frac{\lambda}{3600})^k(1-\frac{\lambda}{3600})^{3600-k}$

As we know

$\lim_{x\rightarrow \inf}(1+\frac{a}{x})^x=e^a$

The proof is let $1/n = a/x, x= na$ . Then rewrite as:

$\begin{aligned} & \lim_{n\rightarrow \infty}(1+\frac{1}{n})^{na} \\ &=\lim_{n\rightarrow \infty}((1+\frac{1}{n})^{n})^a \\ &= e^a \end{aligned}$

Then as we increased $n$ . The probability can be written as

$\begin{aligned} p(X=k) &=\lim_{n\rightarrow \infty} C_{n}^k(\frac{\lambda}{n})^k(1-\frac{\lambda}{n})^{n-k} \\ &= \lim_{n\to \infty} \frac{n!}{(n-k)!k!} \frac{\lambda ^k}{n^k} (1-\frac{\lambda}{n})^{n} (1-\frac{\lambda}{n})^{-k} \\ &= \lim_{n \to \infty} \frac{n(n-1)...1}{(n-k)(n-k-1)...1}\frac{\lambda ^k}{k!} \lim_{n \to \infty} (1-\frac{\lambda}{n})^n(1-\frac{\lambda}{n})^{-k} \\ &= \lim_{n \to \infty} \frac{n^k...}{(n^k...)}\frac{\lambda ^k}{k!} \lim_{n \to \infty} (1-\frac{\lambda}{n})^n(1-\frac{\lambda}{n})^{-k} \\ &= 1\cdot\frac{\lambda ^k}{k!} e^{-\lambda}\cdot1 \\ \end{aligned}$

Gamma Function

gamma function is to find the common function for factorial. 当x不为整数时，阶乘就只能靠gamma function.

$\Gamma(x)=\int_0^{+\infty}e^{-t}t^{x-1}dt (x>0)$

For example

$\Gamma(1) = \int_0^{+\infty}e^{-t}dt=-e^{-t}|_0^{+\infty}=1$

Also the $\Gamma(1/2) = \sqrt\pi$

Beta Distribution

beta distribution 是定义在区间（0，1）的连续概率分布，with two parameter $\alpha$ and $\beta$, $\alpha, \beta >0$

$\begin{aligned} f(x;\alpha,\beta)&=\frac{x^{\alpha-1}(1-x)^{\beta -1}}{\int_0^1u^{\alpha-1}(1-u)^{\beta-1}du} \\ &= \frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}x^{\alpha-1}(1-x)^{\beta-1} \\ &= \frac{1}{B(\alpha, \beta)}x^{\alpha-1}(1-x)^{\beta-1} \end{aligned}$

Proof that

paper

$B(a,b) = \frac{\Gamma(a)\Gamma(b)}{\Gamma(a+b)} = \int_0^1\mu^{a-1}(1-\mu)^{b-1}d\mu$

approach 1:

$Beta(\mu|a,b)=\frac{\Gamma(a+b)}{\Gamma(a)\Gamma(b)} \mu^{a-1}(1-\mu)^{b-1}$

$\Gamma(a)\Gamma(b)=\int_0^\infty e^{-x}x^{a-1}dx\int_0^{\infty}y^{b-1}e^{-y}dy$

Let $t=x+y$, then $dy=d(t-x)=dt$

…

Approach 2:

…