The Poisson Distribution

The poisson and the binomial distribution are actually the same distribution.

fishy fishy

Let us try to derive the poisson distribution from nothing but the binomial distribution.

We assume that in any moment in time an event may happen and that no two events can occur at the same time and that during any given period of time of the same duration the chance of an event happening is the same. Lets also say that in some period of time the expected number of events is λ \lambda .

Let's just assume for now that no two events can happen within a second of one another. Well what this would mean that in a period of 1 1 second there is a probability p p of the event happening. If there are n n seconds in a period of time then the probability of exactly r r events happening over this period would be modelled by the binomial distribution with

P ( # of events = r ) = ( n r ) p r q n − r \mathbb{P}(\text{\# of events} = r) = {n \choose r} p^r q^{n-r}

We also know that during this period of n n seconds we would expect there to be n p np events. So if we know that during these n n seconds we should expect there to be λ \lambda events, we have that p = λ n p = \frac{\lambda}{n} .

But of course we don't actually know that two events cannot happen within a second of one another, what if two atoms just happen to decay within a second of one another. What if two customers just happen to click order within a second of one another. We just don't know.

Now the trick is that we can just consider a period of time even smaller than a second, even smaller than a millisecond, what if we just take the limit as our period of time in which no events can happen becomes infinitely small. Or in other words, we assume that the number of "seconds" within our time period n n gets really really large. This is perfect because at some point the probability of two events happening within 0.000000 … 00001 0.000000\dots00001 seconds of one another is so small that it could not possibly impact the final probability.

If we know generating functions we may recognize that the probability generating function for the probability of X X events is now

G X ( t ) = lim ⁡ x → ∞ ( 1 − λ n + λ n ⋅ t ) n = lim ⁡ x → ∞ ( 1 + λ n ⋅ ( t − 1 ) ) n G_X(t) = \lim_{x \to \infty} \left( 1 - \frac{\lambda}{n} + \frac \lambda n \cdot t \right) ^n = \lim_{x \to \infty} \left( 1 + \frac \lambda n \cdot (t-1) \right) ^n

and then using the definition of e r x e^{rx} we have that

G x ( t ) = e λ ( t − 1 ) = e − λ ⋅ ( 1 + λ 1 ! + λ 2 2 ! + λ 3 3 ! + …   ) G_x(t) = e^{\lambda(t-1)} = e^{-\lambda} \cdot \left( 1 + \frac{\lambda}{1!} + \frac {\lambda^2}{2!} + \frac{\lambda^3}{3!} + \dots \right)

This gives us exactly what we are looking for, that is

P ( # of events = r ) = e λ λ r r ! \mathbb{P}(\text{\# of events} = r) = \frac{e^\lambda \lambda^r}{r!}

But it is okay if we do not know generating functions. We would have that

P ( # of events = r ) = lim ⁡ n → ∞ ( n r ) ⋅ ( λ n ) r ⋅ ( 1 − λ n ) n − r \mathbb{P}(\text{\# of events} = r) = \lim_{n \to \infty} {n \choose r} \cdot \left(\frac \lambda n\right)^r\cdot \left(1-\frac\lambda n\right)^{n-r}

Now lets just evaluate this limit.

= lim ⁡ n → ∞ ( n r ) ⋅ ( λ n 1 − λ n ) r ⋅ lim ⁡ n → ∞ ( 1 − λ n ) n = e − λ ⋅ lim ⁡ n → ∞ ( n r ) ⋅ ( λ n 1 − λ n ) r = e − λ ⋅ λ r ⋅ lim ⁡ n → ∞ ( n r ) ⋅ ( 1 n 1 − λ n ) r = e − λ ⋅ λ r ⋅ lim ⁡ n → ∞ ( n r ) ⋅ ( 1 n 1 ) r = e − λ ⋅ λ r ⋅ lim ⁡ n → ∞ ( n r ) ⋅ ( 1 n ) r = e − λ ⋅ λ r ⋅ lim ⁡ n → ∞ n ! r ! ⋅ ( n − r ) ! ⋅ n r = e − λ ⋅ λ r r ! lim ⁡ n → ∞ n ! ( n − r ) ! ⋅ n r = e − λ ⋅ λ r r ! lim ⁡ n → ∞ ( n n ⋅ n − 1 n ⋅ ⋯ ⋅ n − r + 1 n ) \begin{align*} &= \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{\frac \lambda n}{1-\frac{\lambda}n}\right)^r \cdot \lim_{n \to \infty} \left(1-\frac\lambda n\right)^n \\ &= e^{-\lambda} \cdot \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{\frac \lambda n}{1-\frac{\lambda}n}\right)^r \\ &= e^{-\lambda} \cdot \lambda^r \cdot \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{\frac 1 n}{1-\frac{\lambda}n}\right)^r \\ &= e^{-\lambda} \cdot \lambda^r \cdot \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{\frac 1 n}{1}\right)^r \\ &= e^{-\lambda} \cdot \lambda^r \cdot \lim_{n \to \infty} {n \choose r} \cdot \left(\frac{1}{n }\right)^r \\ &= e^{-\lambda} \cdot \lambda^r \cdot \lim_{n \to \infty} \frac{n!}{r! \cdot (n-r)! \cdot n^r} \\ &= \frac{e^{-\lambda} \cdot \lambda^r}{r!} \lim_{n \to \infty} \frac{n!}{(n-r)! \cdot n^r} \\ &= \frac{e^{-\lambda} \cdot \lambda^r}{r!} \lim_{n \to \infty} \left( \frac{n}{n} \cdot \frac{n-1}{n} \cdot \dots \cdot \frac{n-r+1}{n} \right) \end{align*} \\

Now it is not too much work to show that the limit evaluates to 1 1 which gives us the poisson distribution formula again.

By realising that the poisson distribution is the limit of the binomial distribution we can instantly get the following fact about the poisson distribution:

If X ∼ Po ( λ ) X \sim \text{Po}(\lambda) then Var ( X ) = E ( X ) = λ \text{Var}(X) = \mathbb{E}(X) = \lambda

We know that in a binomial distribution the variance is equal to n p q npq . We also know that n p = λ np = \lambda . As p p approaches 0 0 , q = 1 − p q = 1 - p approaches 1 1 , So n p = n p q np = npq , Mean = Variance.