Sample space

The concept of probability has two interpretations: - Frequency of occurence (number of heads when we toss a coin 100 times); - Subjective beliefs (uncertainty about the existence of life on Mars).

The uncertainty has two reasons: - Epistemic uncertainty: ignorance about the mechanism generating data; - Aleatoric uncertainty: intrinsic variability in the mechanism generating data.

A probabilistic model is a description of an uncertain situation in math symbols with the following elements: - There is only one experiment, which could be a single toss of a coin, an infinite sequence of tosses, or it could be a thought experiment about the existence of life on Mars. - The result of an experiement is called an outcome. Description of outcomes should have enough details to distinguish one from another, but avoid irrelevant details. - The sample space \(\Omega\) is the set of all possible mutually exclusive outcomes of an experiment. The number of possible outcomes could be finite or infinite. In general, the sample space must be collectively exhaustive, so that we always get an outcome that is included in the sample space. - An event is a set \(A\) of possible outcomes. - The probability \(P(A)\) assigns our degree of belief about the overall chance that outcomes in the set \(A\) will happen.

Probability laws: 1. (Non-negativity) \(P(A) \ge 0\) for every event \(A\). 2. (Additivity) If \(A\) and \(B\) are two disjoint events, then

$$ P(A \cup B) = P(A) + P(B)$$

1. (Normalization) The probability of entire sample space is equal to 1, that is \(P(\Omega) = 1\).

Discrete probability law: If the sample space sonsists of a finite number of possible outcomes, then each single outcome has a specified probability, and the probability of any event \({s_1,s_2,\ldots,s_n}\) is the sum of the probabilities of its elements:

$$P({s_1,s_2,\ldots,s_n}) = P({s_1}) + P({s_2}) + \cdots + P({s_n})$$

Discrete uniform probability law: If the sample space consists of \(n\) possible outcomes which are equally likely, then

$$P(A) = \frac{\text{number of elements of \(A\)}}{n}$$

Example. Consider an experiment of tossing a single coin. There are two outcomes, heads (\(H\)) and tails (\(T\)). The sample space is \(\Omega={H,T}\). The events are \({H,T},{H},{T}, \emptyset\). If the coin if fair, \(P({H}) = P({T}) = 0.5\) and \(P({H,T}) = P({H}) + P({T}) = 1\).

Example. Consider an experiment of tossing a coin 3 times. An event

$$A={\text{exactly 2 tails occur}} = {HTT,THT,TTH}$$

then

$$\begin{align} P(A) &= P({HTT,THT,TTH})\ &= P({HTT}) + P({THT}) + P({TTH})\ &= \frac{1}{8} + \frac{1}{8} + \frac{1}{8}\ &= \frac{3}{8} \end{align}$$

To simplify notation, the convention is to drop the curly braces set notation when denoting an event which is still considered as a set of outcomes.