Probability
Probability and counting basics
A sample space is the set of all possible outcomes of an experiment. An event is a subset of the sample space.
Naive definition of probability
$$ \text{the probability of an event} \quad P(A) = \frac{\text{number of favorable outcomes}}{\text{number of possible outcomes}} $$
The naive definition assumes all outcomes are equally likely and there is a finite sample space.
We say a coin is fair when heads and tails are equally likely to occur. If we flip a coin twice, there are four outcomes possible: HH, HT, TH, TT. To calculate the probability of getting heads twice, we get 1/4.
Multiplication rule of counting: if we have an experiment with \(n_1\) possible outcomes, and for each outcome of the first experiment there are \(n_2\) possible outcomes for the second experiment, ..., for each there are \(n_r\) outcomes for the \(r^{th}\) experiment, then there are \(n_1 \cdot n_2 \cdots n_r\) overall possible outcomes.
Binomial coefficient \(n\) choose \(k\) is the number of subsets \(k\) of a group of \(n\) objects where order does not matter:
$$ {n \choose k} = \frac{n!}{(n-k)! k!} $$
If we want to choose \(k\) object out from \(n\) objects, this could be done in \(n \cdot (n-1) \cdot (n-2) \cdots (n-k+1)\) ways. This will our an answer if we pick objects in specific order. However, if we want to choose objects in any order, we have to devide by \(k!\) because we overcounted them:
$$ \frac{n \cdot (n-1) \cdot (n-2) \cdots (n-k+1)}{k!} = \frac{n!}{(n-k)! k!} $$
The probability of getting a full house in pocker (for example, 3 7's and 2 10's):
$$ P(\textrm{full house}) = \frac{13 {4 \choose 3} 12 {4 \choose 2}}{{52 \choose 5}} = \frac{3744}{2598960} \approx 0.00144. $$
Sampling table: choose \(k\) objects out of \(n\). We can sample with or without replacement, order may or may not matter:
| --- | Order matters | Order doesn't matter |
|---|---|---|
| Replace | \(n^k\) | \(n+k-1 \choose k\) |
| Don't replace | \(n \cdot (n-1) \cdot (n-2) \cdots (n-k+1)\) | \(n \choose k\) |