The Poisson Distribution

The poisson and the binomial distribution are actually the same distribution.

fishy fishy

Let us try to derive the poisson distribution from nothing but the binomial distribution.

We assume that in any moment in time an event may happen and that no two events can occur at the same time and that during any given period of time of the same duration the chance of an event happening is the same. Lets also say that in some period of time the expected number of events is λ\lambda.

Let's just assume for now that no two events can happen within a second of one another. Well what this would mean that in a period of 11 second there is a probability pp of the event happening. If there are nn seconds in a period of time then the probability of exactly rr events happening over this period would be modelled by the binomial distribution with

P(# of events=r)=(nr)prqn−r\mathbb{P}(\text{\# of events} = r) = {n \choose r} p^r q^{n-r}

We also know that during this period of nn seconds we would expect there to be npnp events. So if we know that during these nn seconds we should expect there to be λ\lambda events, we have that p=λnp = \frac{\lambda}{n}.

But of course we don't actually know that two events cannot happen within a second of one another, what if two atoms just happen to decay within a second of one another. What if two customers just happen to click order within a second of one another. We just don't know.

Now the trick is that we can just consider a period of time even smaller than a second, even smaller than a millisecond, what if we just take the limit as our period of time in which no events can happen becomes infinitely small. Or in other words, we assume that the number of "seconds" within our time period nn gets really really large. This is perfect because at some point the probability of two events happening within 0.000000…000010.000000\dots00001 seconds of one another is so small that it could not possibly impact the final probability.

If we know generating functions we may recognize that the probability generating function for the probability of XX events is now

GX(t)=lim⁡x→∞(1−λn+λn⋅t)n=lim⁡x→∞(1+λn⋅(t−1))nG_X(t) = \lim_{x \to \infty} \left( 1 - \frac{\lambda}{n} + \frac \lambda n \cdot t \right) ^n = \lim_{x \to \infty} \left( 1 + \frac \lambda n \cdot (t-1) \right) ^n

and then using the definition of erxe^{rx} we have that

Gx(t)=eλ(t−1)=e−λ⋅(1+λ1!+λ22!+λ33!+… )G_x(t) = e^{\lambda(t-1)} = e^{-\lambda} \cdot \left( 1 + \frac{\lambda}{1!} + \frac {\lambda^2}{2!} + \frac{\lambda^3}{3!} + \dots \right)

This gives us exactly what we are looking for, that is

P(# of events=r)=eλλrr!\mathbb{P}(\text{\# of events} = r) = \frac{e^\lambda \lambda^r}{r!}

But it is okay if we do not know generating functions. We would have that

P(# of events=r)=lim⁡n→∞(nr)⋅(λn)r⋅(1−λn)n−r\mathbb{P}(\text{\# of events} = r) = \lim_{n \to \infty} {n \choose r} \cdot \left(\frac \lambda n\right)^r\cdot \left(1-\frac\lambda n\right)^{n-r}

Now lets just evaluate this limit.

=lim⁡n→∞(nr)⋅(λn1−λn)r⋅lim⁡n→∞(1−λn)n=e−λ⋅lim⁡n→∞(nr)⋅(λn1−λn)r=e−λ⋅λr⋅lim⁡n→∞(nr)⋅(1n1−λn)r=e−λ⋅λr⋅lim⁡n→∞(nr)⋅(1n1)r=e−λ⋅λr⋅lim⁡n→∞(nr)⋅(1n)r=e−λ⋅λr⋅lim⁡n→∞n!r!⋅(n−r)!⋅nr=e−λ⋅λrr!lim⁡n→∞n!(n−r)!⋅nr=e−λ⋅λrr!lim⁡n→∞(nn⋅n−1n⋅⋯⋅n−r+1n) \begin{align*} &= \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{\frac \lambda n}{1-\frac{\lambda}n}\right)^r \cdot \lim_{n \to \infty} \left(1-\frac\lambda n\right)^n \\ &= e^{-\lambda} \cdot \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{\frac \lambda n}{1-\frac{\lambda}n}\right)^r \\ &= e^{-\lambda} \cdot \lambda^r \cdot \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{\frac 1 n}{1-\frac{\lambda}n}\right)^r \\ &= e^{-\lambda} \cdot \lambda^r \cdot \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{\frac 1 n}{1}\right)^r \\ &= e^{-\lambda} \cdot \lambda^r \cdot \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{1}{n }\right)^r \\ &= e^{-\lambda} \cdot \lambda^r \cdot \lim_{n \to \infty} \frac{n!}{r! \cdot (n-r)! \cdot n^r} \\ &= \frac{e^{-\lambda} \cdot \lambda^r}{r!} \lim_{n \to \infty} \frac{n!}{(n-r)! \cdot n^r} \\ &= \frac{e^{-\lambda} \cdot \lambda^r}{r!} \lim_{n \to \infty} \left( \frac{n}{n} \cdot \frac{n-1}{n} \cdot \dots \cdot \frac{n-r+1}{n} \right) \end{align*} \\

Now it is not too much work to show that the limit evaluates to 11 which gives us the poisson distribution formula again.

By realising that the poisson distribution is the limit of the binomial distribution we can instantly get the following fact about the poisson distribution:

If X∼Po(λ)X \sim \text{Po}(\lambda) then Var(X)=E(X)=λ\text{Var}(X) = \mathbb{E}(X) = \lambda

We know that in a binomial distribution the variance is equal to npqnpq. We also know that np=λnp = \lambda. As pp approaches 00, q=1−pq = 1 - p approaches 11, So np=npqnp = npq, Mean = Variance.