Probability
Probability refers to the likelihood of an event E happening, expressed as the ratio between the number of favorable outcomes (f) and the total number of possible outcomes (n). $$ p(E) = \frac{f}{n} $$
This ratio gives an estimate of how likely the event is to occur.
The probability of a random event is a number between 0 and 1.
$$ 0 \le p(E) \le 1 $$
A certain event has a probability of 1, while an impossible event has a probability of 0.
Sometimes, probabilities are expressed as percentages, where 0% corresponds to 0 (impossible event) and 100% corresponds to 1 (certain event).
$$ 0 \% \le p(E) \le 100 \% $$
Note: There are two main interpretations of probability. The frequentist interpretation treats probability as an objective property of an outcome, where the probability converges to a fixed value after many repetitions of the experiment. The subjectivist interpretation, on the other hand, views probability as a subjective estimate by the observer, meaning different analysts can assign different probabilities to the same outcome.
The set of all possible outcomes of an event is called the sample space (or universal set) and is often denoted by S, U, or Ω.
The set of favorable outcomes (F) is a subset of the sample space S, containing only the outcomes that favor event E.
Since F is a subset of U, the number of favorable outcomes is always less than or equal to the total number of possible outcomes.
$$ |F| \le |U| $$
Note: If the set of favorable outcomes is empty, F=Ø, the event is impossible. Conversely, if F matches U, meaning F=U, the event is certain.
A Practical Example
When rolling a die, there are several possible outcomes.
Since the die has six faces, there are six possible outcomes.
The sample space consists of n=6 elements (six possible outcomes).
$$ S = \{1,2,3,4,5,6 \} $$
To calculate the probability of rolling a 3, you need to find the ratio between the number of favorable outcomes (f) and the total number of possible outcomes (n).
In this case, the set of favorable outcomes (F) contains only one favorable outcome (f=1), which is rolling a 3.
$$ F=\{ 3 \} $$
Thus, the probability of rolling a 3 (A="rolling a 3") is 0.16
$$ P(A) = \frac{|F|}{|S|} = \frac{f}{n} = \frac{1}{6} = 0.16 $$
In percentage terms, the probability of this event is 16%.
Example 2
What is the probability of rolling an even number?
In this case, event E is "rolling an even number."
The sample space is the same, with n=6 possible outcomes.
$$ S = \{1,2,3,4,5,6 \} $$
The set of favorable outcomes contains f=3 outcomes.
$$ F = \{ 2 , 4, 6 \} $$
So, the probability of rolling an even number (A="rolling an even number") is 0.5
$$ P(A) = \frac{|F|}{|S|} = \frac{f}{n} = \frac{3}{6} = 0.5 $$
In percentage terms, the probability of this event is 50%.
Understanding A Priori and A Posteriori Probability
Probability can be calculated in two ways, depending on the nature of the random phenomenon or event.
- A Priori (Theoretical) Probability
A priori probability is calculated without performing any real-world experiments (empirical data). It is based on a theoretical model built using classical probability theory. In this case, probability is defined as the ratio between the number of favorable outcomes (F) and the number of possible outcomes (N). $$ p = \frac{F}{N} $$ Where F is a theoretical value derived from analyzing the problem.
Example: A common example is a fair six-sided die. Each face has an equal chance of landing, so no estimation is needed to calculate the probability. I just need to know that each face has a probability of 1/6.
- A Posteriori (Statistical) Probability
A posteriori probability is determined through the collection of real-world empirical data, based on repeated trials, following the empirical law of probability. This method uses direct observation to provide a more accurate and representative estimate of an event’s probability. It is mainly used in inferential statistics. In practice, N trials are performed, and the number of favorable outcomes is recorded. Statistical probability is the ratio of favorable outcomes (F) to the total number of trials (N). $$ p = \frac{F}{N} $$ Where F represents the number of favorable outcomes observed in the trials.
Example: When a die is biased, the theoretical probability no longer accurately reflects the real-world outcomes. In such cases, a posteriori probability is used, where the probability of the event is measured through the relative frequency of favorable outcomes in repeated trials or experiments. However, many trials are needed for a reliable estimate. According to the empirical law of probability, the more observations are made, the closer the relative frequency (F/N) comes to the actual probability (P) of the event.
Which Should You Use?
A priori (theoretical) probability is easier and faster to calculate, but it can't always be applied to every random event.
A posteriori (statistical) probability is useful when the theoretical probability can't be determined, but it requires more data and time to be reliable.
For example, if a die is fair, a priori probability is reliable. However, if the die is biased, a posteriori probability must be used.
In general, the choice depends on the context and the problem at hand.
Theoretical probability may be less precise in some situations, but statistical probability is more time-consuming, as it requires extensive data collection to be accurate.
Often, the decision is also influenced by the level of risk you're willing to accept, making it a subjective choice.
For instance, in situations where I have limited data, a tight deadline, and the events are already somewhat predictable, a priori (theoretical) probability might be an acceptable estimate. It's easier and faster to calculate, provided the margin of error is low and the consequences of a mistake aren't too significant. In these cases, a careful cost-benefit analysis is crucial.
And so on...
