Sample Space

The sample space (or sample set) refers to the collection of all possible samples that can be drawn from a population.

Each sample is made up of a subset of elements from the population, and the sample space represents all of these subsets.

For example, if I have a population of $N$ elements and I want to draw samples of size $n$, the sample space includes all possible combinations of $n$ elements from the population of $N$ elements.

A practical example
How many samples are in the sample space?

A practical example

Let's consider a population consisting of three elements:

$$ \{A, B, C\} $$

I want to draw samples of size $n = 2$.

There are several sampling methods to choose from.

A] Sampling without replacement and ignoring order (combinations)

In this case, the order of elements in the sample doesn’t matter, and elements cannot be repeated.

So, the sample space will be:

$$ \{\{A, B\}, \{A, C\}, \{B, C\}\} $$

The sample space consists of the possible combinations of 2 elements chosen from 3, which is $\binom{N}{n} = \binom{3}{2} = 3$.

$$ \binom{N}{n} = \frac{3!}{2!(3-2)!} = \frac{6}{2} = 3 $$

In total, there are 3 possible samples.

B] Sampling without replacement but considering order (permutations without replacement)

When the order matters but repetition of elements is not allowed, we consider the permutations of 2 elements chosen from 3, i.e., $A, B, C$.

The sample space is as follows:

$$ \{(A, B), (A, C), (B, A), (B, C), (C, A), (C, B)\} $$

The number of possible samples is given by the permutations without replacement.

$$ \frac{N!}{(N-n)!} = \frac{3!}{(3-2)!} = \frac{6}{1} = 6 $$

In total, there are 6 possible samples.

C] Sampling with replacement and ignoring order (combinations with replacement)

Now let’s look at the case where elements can be repeated, but the order doesn’t matter.

In this scenario, the sample space includes all pairs, allowing the repetition of elements.

$$ \{ \{A, A\}, \{A, B\}, \{A, C\}, \{B, B\}, \{B, C\}, \{C, C\} \} $$

The number of possible samples is given by the combinations with replacement.

$$ \binom{N + n - 1}{n} = \frac{(N + n - 1)!}{n!(N - 1)!} = \binom{3 + 2 - 1}{2} = \binom{4}{2} = \frac{4!}{2!2!} = \frac{24}{4} = 6 $$

So, there are 6 possible samples in the sample space.

D] Sampling with replacement and considering order (permutations with replacement)

Finally, let’s consider the case where the order matters and elements can be repeated.

In this case, the sample space includes all ordered samples with repetition.

$$ \{(A, A), (A, B), (A, C), (B, A), (B, B), (B, C), (C, A), (C, B), (C, C)\} $$

The total number of possible samples is given by the permutations with replacement, which is 9.

$$ N^n = 3^2 = 9 $$

In conclusion, these examples show how the sample space can change depending on whether or not the order of elements matters and whether repetition of elements is allowed in the sample.

How many samples are in the sample space?

The total number of possible samples depends on the size of the population $N$ and the sample $n$, as well as the type of sampling, whether repetition is allowed, and whether or not the order of elements matters.

Order matters, and sampling with replacement
$N^n$
Order matters, and sampling without replacement
$\frac{N!}{(N-n)!}$ (simple permutations)
Order doesn’t matter, and sampling with replacement
$\binom{N + n - 1}{n}$ (combinations with replacement)
Order doesn’t matter, and sampling without replacement
$\binom{N}{n}$

Thus, the formula $N^n$ is used to calculate the size of the sample space only in cases where sampling allows replacement and the order is important.

If the sampling is without replacement or if the order doesn’t matter, we use the binomial coefficient $\binom{N}{n}$.

Note: In general, the sample space becomes significantly large if the population size $N$ is large or if the sample size $n$ is relatively large.