1.1. What is the Sampling Distribution of the mean ?

In Tutorial 3, we considered the Normal Distribution as well as another variant, the Standard Normal Distribution.

The Sampling Distribution of the mean is an additional variant of the Normal Distribution. The Sampling Distribution of the mean looks very much like any other Normal Distribution, with its own mean (\(\mu\)) and standard deviation (\(\sigma\)), except that each value on the x-axis represents a sample mean rather than an individual score.

Reminder: Inferential statistics allow us to characterise a population.

Let’s look at an example, which illustrates the difference between the Sampling Distribution of the mean and the Normal Distribution

HIV-AIDS & CD4 count
The CD4 count, a type of white blood cell that plays a key role in immune system functioning, was recorded in a sample of HIV-AIDS patients. Table 1 below contains the CD4 counts of 5 individuals. The mean (\(\mu\)) CD4 count in the South African HIV-AIDS population is 110

...=c(...)
...(...)#Calculate the mean
...(...)#Calculate the standard deviation

CD4=c(100,120,200,150,80)
mean(CD4)#Calculate the mean
sd(CD4)#Calculate the standard deviation

...=c(...)
...(...)#Calculate the mean
...(...)#Calculate the standard deviation

CD4.SDM=c(120,110,140,100,135)
mean(CD4.SDM)#Calculate the mean
sd(CD4.SDM)#Calculate the standard deviation

2.1 Sample size and Normal Distribution

The extent to which an empirical distribution (i.e. based on a sample) approximates a population distribution (i.e. theoretical) is influenced by the sample size.

Grade 5 reading ability in Western Cape
Reading ability of Grade 5 learners follows a normal distribution, with a population mean (\(\mu\)) of 139 words per minute, and a standard deviation (\(\sigma\)) of 35 words per minute.

set.seed(...)
reading.ability = rnorm(..., ...,...)

set.seed(12)
reading.ability = rnorm(10000, 139, 35)

small.sample = sample(x=..., size=..., replace=...)
medium.sample = sample(x=..., size=..., replace=...)
large.sample = sample(x=..., size=..., replace=...)

small.sample = sample(x = reading.ability,size = 10,replace = TRUE)
medium.sample = sample(x = reading.ability,size = 100,replace = TRUE)
large.sample = sample(x = reading.ability,size = 1000,replace = TRUE)

par(mfrow = c(1, 3))
hist(..., xlab="...", main="...")
hist(..., xlab="...", main="...")
hist(..., xlab="...", main="...")

par(mfrow = c(1, 3))
hist(small.sample, xlab= "Reading speed per minute", main="Reading ability in sample of 10")
hist(medium.sample, xlab= "Reading speed per minute", main="Reading ability in sample of 100")
hist(large.sample, xlab= "Reading speed per minute", main="Reading ability in sample of 1000")

2.2. Sample size and Sampling Distribution of the mean

We have discussed that when we sample from the population, the sample mean varies as a function of (i.e. is based on) the sample size. More specifically, the sample mean better approximates the population mean when the sample size increases.

We can extend this principle to the Sampling Distribution of the mean, where we now draw multiple samples of a particular size (instead of one sample), calculate the mean for each of these samples, and plot the means on distribution of their own, i.e. the Sampling Distribution of the mean.

set.seed(2)
yourmeans <- replicate(100, mean(sample(x=reading.ability, size=10, replace = TRUE)))
hist(..., xlab="...", main = "...")

set.seed(2)
yourmeans <- replicate(100, mean(sample(x=reading.ability, size=10, replace = TRUE)))
hist(yourmeans, xlab= "Sampling Distribution means", main="Sampling Distribution of the mean for reading ability, based on sample size of 10")

set.seed(2)
... <- replicate(..., mean(sample(x=reading.ability, size=.., replace = TRUE)))
hist(..., xlab="...", main= "...")

set.seed(2)
yourmeans <- replicate(100, mean(sample(x=reading.ability, size=100, replace = TRUE)))
hist(yourmeans, xlab= "Sampling Distribution means", main="Sampling Distribution of the mean for reading ability, based on sample size of 100")

mean(yourmeans)
mean(yourmeans.2)

sd(yourmeans)
sd(yourmeans.2)

3.1 Application of the Central Limit Theorem

While we were able to generate sample/empirical Sampling Distributions in section 2, where means were generated from samples of a particular size, we are more interested in a theoretical Sampling Distribution of the mean, where means are generated from infinitely large samples. However, it is impossible to simulate such samples, but we can apply what is referred to as the Central Limit Theorem

The Central Limit Theorem states the following:

1. Shape: The distribution of sample means (i.e. Sampling Distribution of the mean) will always approximate a normal distribution as the sample size increases, irrespective of the shape of population distribution.
2. Mean parameter(\(\mu\)): The mean (i.e. mean of means) of the Sampling Distribution will be equal to the mean of the population.

\[ \begin{aligned} \mu_{\overline{X}} = \mu \end{aligned} \]

\(\mu_{\overline{X}}\) = Mean of means in Sampling Distribution
\(\mu\) = Population mean

3. Spread parameter (\(\sigma\)): The standard deviation of the Sampling Distribution of the mean will be equal to the population standard deviation divided by the square root of sample size (n)

\[ \begin{aligned} \sigma_{\overline{X}} = {\frac{\sigma}{\sqrt{n}}} \end{aligned} \]

\(\sigma_{\overline{X}}\) = Standard deviation of the Sampling Distribution of the mean
\(\sigma\) = Standard deviation of the population
N = Total sample size

The alternate spread formula can be used, which defines spread in terms of variance or \(\sigma^{2}\) instead of standard deviation \(\sigma\)

\[ \begin{aligned} \sigma^2_{\overline{X}} = {\frac{\sigma^2}{n}} \end{aligned} \]

Central Limit Theorem applied to reading ability example:

The Population Distribution of Grade 5 reading ability in the Western Cape follows a normal distribution with mean of 139 words per minute and standard deviation of 35 words per minute. If the Sampling Distribution of the mean is based on samples of size 10, then

1. The mean of Sampling Distribution will equal the population mean (139)
2. The standard deviation of the Sampling Distribution will equal the population standard deviation divided by the square root of the sample size (35/sqrt(10)=11.06)

Depression and night shift work
Let’s say that the mean (\(\mu\)) depression of night shift workers is 30 on the Beck Depression Inventory (BDI), and the standard deviation is 10 (\(\sigma\)). Samples of 100 were drawn from the population

10/sqrt(100)

100/2000

3.2 Central Limit Theorem application to non-normal distributions

The major strength of the Central Limit Theorem is its applicability to both normal and non-normal population distributions.

The Central Limit Theorem states the following:

1. Regardless of the shape of the population distribution, the sampling distribution of the mean will approximate the Normal Distribution

This is a particularly useful extension of the theorem, given that…

1. Not all variables we wish to study are normally distributed (e.g. reaction time tasks)
2. We have an immense range of tools developed for use with normal distributions that can now be applied to other types of distributions

Continuous Performance Task (CPT) and reaction time Let’s say that the Continuous Performance Task (CPT), which is a measure of sustained attention, has a population mean reaction time of 0.5 seconds (\(\mu\)). Scores are stored in the object vector CPT.rtime

hist(CPT.rtime, main= "Distribution of CPT reaction times", xlab= "Reaction time")

set.seed(2)
CPT.rtime.means <- replicate(.., mean(sample(x=..., size=..., replace = TRUE)))
hist(CPT.rtime.means, xlab="...", main = "...")

set.seed(2)
CPT.rtime.means <- replicate(200, mean(sample(x=CPT.rtime, size=1000, replace = TRUE)))
hist(CPT.rtime.means, xlab= "Reaction times", main="Sampling Distribution of the mean for CPT reaction times")

mean(CPT.rtime) 
...(...)

mean(CPT.rtime) 
mean(CPT.rtime.means)

4.1. Mean score to probability

Mean scores from the Sampling Distribution can be converted to probabilities much like individual scores from the Normal Distribution can be converted to probabilities, using the pnorm() command. The only difference is the application of Central Limit Theorem.

Grade 5 reading ability in the Western Cape
Reading ability of Grade 5 learners follows a normal distribution, with a population mean (\(\mu\)) of 139 words per minute, and a standard deviation (\(\sigma\)) of 35 words per minute. We draw a random sample of 100 learners.

se = 35/sqrt(100)  # se = standard error
pnorm(q=90, mean = 139, sd = se)

pnorm(q=.., mean = ..., sd= .../sqrt(...))

pnorm(q=150, mean = 139, sd= 35/sqrt(100))

4.2 Probability to mean score

Probabilities from the Sampling Distribution can also be converted back to a mean score, much like with the Normal Distribution, using the qnorm() command.

Job satisfaction of social scientists
The mean (\(\mu\)) job satisfaction of social scientists is 130 as measured on the Job Satisfaction Survey (JSS), and the variance is 100 (\(\sigma^2\)). A random sample of 2000 social scientists was drawn from the population

standard.dev= sqrt(100)
qnorm(p=..., mean =..., sd=.../sqrt(...))

standard.dev= sqrt(100)
qnorm(p=0.10, mean =130, sd=standard.dev/sqrt(2000))

standard.dev= sqrt(...)
qnorm(p=..., mean =..., sd=standard.dev/sqrt(...))

standard.dev= sqrt(100)
qnorm(p=0.90, mean =130, sd=standard.dev/sqrt(2000))

4.3. Transform Sampling Distribution to Standard Normal Distribution (converting to Z-scores)

The Sampling Distribution can be converted to a Standard Normal Distribution by converting mean scores to z-scores. The Z-score formula is much the same as that for converting a Normal to a Standard Normal Distribution, except that

1. the random score, X, is replaced with a random mean score \({\overline{X}}\)
2. the population mean of the raw scores, \(\mu\), is replaced with the population mean of the Sampling Distribution, \(\mu_{\overline{X}}\).
3. the population standard deviation \(\sigma\), is replaced with the population standard deviation of the Sampling Distribution, \(\sigma_{\overline{X}}\), which is equivalent to \({\frac{\sigma}{\sqrt{n}}}\)

See two equivalent versions of Z-score formula for Sampling Distribution below:

\[ \begin{aligned} Zscore =\frac{\overline{X} - \mu_{\overline{X}}}{\sigma_{\overline{X}}} \ or \ Zscore =\frac{\overline{X} - \mu_{\overline{X}}}{\frac{\sigma}{\sqrt{n}}} \end{aligned} \] \({\overline{X}}\) = any mean score in the range of possible values of some Sampling Distribution
\({\mu_{\overline{X}}}\) = mean of the Sampling Distribution
\({\sigma_{\overline{X}}}\) = standard deviation of the Sampling Distribution

(...-139)/(35/sqrt(...))

(145-139)/(35/sqrt(100))

zscore =(...-...)/(.../sqrt(...))
pnorm (zscore, mean =..., sd=...)

zscore =(145-139)/(35/sqrt(100))
pnorm (zscore, mean = 0, sd = 1)

5.1 Standard error

The Central Limit Theorem suggests that the variance of Sampling Distributions can be controlled by the researcher. This is because when looking at the standard deviation of the Sampling Distribution, \({\frac{\sigma}{\sqrt{n}}}\), as the size of n increases, the standard deviation decreases.

This is why researchers typically want large samples, with the aim of

1. decreasing the standard deviation of the sampling distribution,
2. which thus increases the accuracy of the estimation of the population mean.

In turn, the standard deviation of the Sampling Distribution is commonly referred to as the standard error of the estimate or standard error for short. When we sample means, it is the standard error of the mean, but we could also obtain the standard error of some other estimator.

par(mfrow = c(..., ...))
hist(..., xlab=..., main=...xlim=c(110, 160))
hist(..., xlab=..., main=..., xlim=c(110, 160))

par(mfrow = c(2,1))
hist(yourmeans, xlab= "Reading speed", main = "Sampling Distribution of 5th Grader reading ability \nbased on sample size of 10", xlim=c(110, 160))
hist(yourmeans.2, xlab= "Reading speed", main = "Sampling Distribution of 5th Grader reading ability \nbased on sample size of 100", xlim=c(110, 160))

5.2 Confidence limits/Intervals

When we use a sample to estimate characteristics of a population, there is always a degree of uncertainty in the estimation of parameters (e.g. the mean), which is represented by the standard error. The degree of uncertainty is small when the sample is representative of the population, and is demonstrated as the sample size increases in size.

One can account for this degree of uncertainty with the calculation of confidence limits/intervals. The confidence interval represents the range within which the true parameter lies. (Actually, this is not quite technically correct, but it is close enough in meaning for our purposes)

If one wished to calculate the 95% confidence interval for an estimate, we could say that amongst 95% of all samples, the true population parameter value falls within this interval.

For example, the 95% confidence interval for reading ability of 5th graders is between 132 and 146 words per minute (Note that the interval includes the population mean 139), which means that the true parameter value lies somewhere within this interval. See a graphical representation of the confidence interval below.

In order to set up confidence intervals, we use the Standard Normal Distribution. The figure below displays the 95% confidence interval within which the true average reading ability of 5th graders lies. In other words, the entire black area of the curve represents the 95% confidence interval. Notice that on the x-axis, z-scores are presented instead of scores.

The confidence interval is derived from the Z-score formula for the Sampling Distribution

\[ \begin{aligned} Z =\frac{\overline{X} - \mu_{\overline{X}}}{\frac{\sigma}{\sqrt{n}}} \ ,thus\ Z \frac{\sigma}{\sqrt{n}} = {\overline{X}} - {\mu}\ ,thus\ \mu = \overline{X} + Z \frac{\sigma}{\sqrt{n}} \end{aligned} \]

Confidence Interval formula

\[ \begin{aligned} \mu = \overline{X} \pm Z \frac{\sigma}{\sqrt{n}} \end{aligned} \]

Let’s look at how to calculate the confidence interval in r using the 5th grader reading ability example:

Grade 5 reading ability in Western Cape
Reading ability of Grade 5 learners follows a normal distribution, with a population mean (\(\mu\)) of 139 words per minute, and a standard deviation (\(\sigma\)) of 35 words per minute. We draw a random sample of 100 learners.

error <- qnorm(0.975)*35/sqrt(100)

upper =139 + error
lower = 139 - error

c(lower, upper)

error <- qnorm(0.95)*35/sqrt(100)

upper =139 + error
lower = 139 - error

c(lower, upper)

error <- qnorm(0.975)*.../sqrt(...)

upper = ... + error
lower = ... - error

c(lower, upper)

error <- qnorm(0.975)*46.9/sqrt(5)

upper = 130 + error
lower = 130 - error

c(lower, upper)

error <- qnorm(...)*.../sqrt(...)

upper = ... + error
lower = ... - error

c(lower, upper)

error <- qnorm(0.95)*46.9/sqrt(5)

upper = 130 + error
lower = 130 - error

c(lower, upper)

Exercise 6.1.1.1

standdev=sqrt(...)

error <- qnorm(...)*standdev/sqrt(...)

upper = ... + error
lower = ... - error

c(lower, upper)

standdev=sqrt(100)

error <- qnorm(0.975)*standdev/sqrt(2000)

upper = 130 + error
lower = 130 - error

c(lower, upper)

Exercise 6.1.1.2

standdev2=sqrt(1000)

error2 <- qnorm(0.975)*standdev2/sqrt(2000)

upper2 = 130 + error2
lower2 = 130 - error2

c(lower2, upper2)

Exercise 6.1.1.3

Exercise 6.1.1.4

Move on to and complete Statistics Tutorial Assignment 4 on Amathuba (Activities | Assignments)