The Poisson Distribution

The poisson and the binomial distribution are actually the same distribution.

fishy fishy

Let us try to derive the poisson distribution from nothing but the binomial distribution.

We assume that in any moment in time an event may happen and that no two events can occur at the same time and that during any given period of time of the same duration the chance of an event happening is the same. Lets also say that in some period of time the expected number of events is λ\lambda.

Let's just assume for now that no two events can happen within a second of one another. Well what this would mean that in a period of 11 second there is a probability pp of the even happening. If there are nn seconds in a period then the probability of exactly rr events happening over this period would be modelled by the binomial distribution with

P(# of events=r)=(nr)prqn−r\mathbb{P}(\text{\# of events} = r) = {n \choose r} p^r q^{n-r}

We also know that during this period of nn seconds we would expect there to be npnp events. So if we know that during these nn seconds we should expect there to be λ\lambda events, we have that p=λnp = \frac{\lambda}{n}.

But of course we don't actually know that two events cannot happen within a second of one another, what if two atoms just happen to decay within a second of one another. What if two customers just happen to click order within a second of one another. We just don't know.

Now the trick is that we can just consider a period of time even smaller than a second, even smaller than a millisecond, what if we just take the limit as our period of time in which no events can happen becomes infinitely small. Or in other words, we assume that the number of "seconds" within our time period nn gets really really large. This is perfect because at some point the probability of two events happening within 0.000000…000010.000000\dots00001 seconds of one another is so small that it could not possibly impact the final probability.

If we know generating functions we may recognize that the probability generating function for the probability of XX events is now

GX(t)=lim⁡x→∞(1−λn+λn⋅t)n=lim⁡x→∞(1+λn⋅(t−1))nG_X(t) = \lim_{x \to \infty} \left( 1 - \frac{\lambda}{n} + \frac \lambda n \cdot t \right) ^n = \lim_{x \to \infty} \left( 1 + \frac \lambda n \cdot (t-1) \right) ^n

and then using the definition of erxe^{rx} we have that

Gx(t)=eλ(t−1)=e−λ⋅(1+λ1!+λ22!+λ33!+… )G_x(t) = e^{\lambda(t-1)} = e^{-\lambda} \cdot \left( 1 + \frac{\lambda}{1!} + \frac {\lambda^2}{2!} + \frac{\lambda^3}{3!} + \dots \right)

This gives us exactly what we are looking for, that is

P(# of events=r)=eλλrr!\mathbb{P}(\text{\# of events} = r) = \frac{e^\lambda \lambda^r}{r!}

But it is okay if we do not know generating functions. We would have that

P(# of events=r)=lim⁡n→∞(nr)⋅(λn)r⋅(1−λn)n−r\mathbb{P}(\text{\# of events} = r) = \lim_{n \to \infty} {n \choose r} \cdot \left(\frac \lambda n\right)^r\cdot \left(1-\frac\lambda n\right)^{n-r}

Now lets just evaluate this limit.

=lim⁡n→∞(nr)⋅(λn1−λn)r⋅lim⁡n→∞(1−λn)n=e−λ⋅lim⁡n→∞(nr)⋅(λn1−λn)r=e−λ⋅λr⋅lim⁡n→∞(nr)⋅(1n1−λn)r=e−λ⋅λr⋅lim⁡n→∞(nr)⋅(1n1)r=e−λ⋅λr⋅lim⁡n→∞(nr)⋅(1n)r=e−λ⋅λr⋅lim⁡n→∞n!r!⋅(n−r)!⋅nr=e−λ⋅λrr!lim⁡n→∞n!(n−r)!⋅nr=e−λ⋅λrr!lim⁡n→∞(nn⋅n−1n⋅⋯⋅n−r+1n) \begin{align*} &= \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{\frac \lambda n}{1-\frac{\lambda}n}\right)^r \cdot \lim_{n \to \infty} \left(1-\frac\lambda n\right)^n \\ &= e^{-\lambda} \cdot \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{\frac \lambda n}{1-\frac{\lambda}n}\right)^r \\ &= e^{-\lambda} \cdot \lambda^r \cdot \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{\frac 1 n}{1-\frac{\lambda}n}\right)^r \\ &= e^{-\lambda} \cdot \lambda^r \cdot \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{\frac 1 n}{1}\right)^r \\ &= e^{-\lambda} \cdot \lambda^r \cdot \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{1}{n }\right)^r \\ &= e^{-\lambda} \cdot \lambda^r \cdot \lim_{n \to \infty} \frac{n!}{r! \cdot (n-r)! \cdot n^r} \\ &= \frac{e^{-\lambda} \cdot \lambda^r}{r!} \lim_{n \to \infty} \frac{n!}{(n-r)! \cdot n^r} \\ &= \frac{e^{-\lambda} \cdot \lambda^r}{r!} \lim_{n \to \infty} \left( \frac{n}{n} \cdot \frac{n-1}{n} \cdot \dots \cdot \frac{n-r+1}{n} \right) \end{align*} \\

Now it is not too much work to show that the limit evaluates to 11 which gives us the poisson distribution formula again.

Now lets prove a cool fact about the poisson distribution:

if X∼Po(λ)X \sim \text{Po}(\lambda) then Var(X)=E(X)=λ\text{Var}(X) = \mathbb{E}(X) = \lambda

We know that in a binomial distribution the variance is equal to npqnpq. Where we know that np=λnp = \lambda. Lets recall how we derived the poisson distribution in the first place. We considered as nn approached infinity. When this happens q=1−λnq = 1 - \frac \lambda n approaches 11. We we have that npq=λ(1−λn)npq = \lambda \left( 1 - \frac \lambda n \right) approaches λ\lambda as nn approaches infinity.

And in the limiting case when we get the poisson distribution. The variance is equal to λ\lambda

Lets prove one more cool fact about the poisson distribution

if X∼Po(a)X \sim \text{Po}(a) and Y∼Po(b)Y \sim \text{Po}(b)

then X+Y∼Po(a+b)X+Y \sim \text{Po}(a+b)

To do this we use the generating function of the poisson distribution

GX(t)=ea(x−1)andGY(t)=eb(x−1)G_X(t) = e^{a(x-1)} \quad \text{and} \quad G_Y(t) = e^{b(x-1)}

So we then have that

GX+Y=e(a+b)(x−1)G_{X+Y} = e^{(a+b)(x-1)}