Exercise 1.1.1.1

FearIntimacyRelationship = ...("...")

head(...)

FearIntimacyRelationship = read.csv("FearIntimacyRelationship.csv")

head(FearIntimacyRelationship)

Exercise 1.1.1.2

plot(FearIntimacyRelationship$fear.intimacy, FearIntimacyRelationship$length.relationship, xlab ="Fear of Intimacy", ylab = "Relationship length", main = "Relationship between fear of intimacy and relationship length")

Exercise 2.0.1.2

AgeBMI = ...("...")

plot(AgeBMI$..., AgeBMI$...,xlab ="...", ylab = "...", main ="...")

AgeBMI = read.csv("AgeBMI.csv")

plot(AgeBMI$Age, AgeBMI$BMI,xlab ="Age", ylab = "BMI", main= "Relationship between Age and BMI")

Exercise 2.0.1.3

plot(....$..., ....$...,xlab ="...", ylab = "...", main ="...")

abline(lm(AgeBMI$Age~AgeBMI$BMI))

plot(AgeBMI$Age, AgeBMI$BMI,xlab ="Age", ylab = "BMI", main= "Relationship between Age and BMI")

abline(lm(AgeBMI$Age~AgeBMI$BMI))

Exercise 2.0.1.4

2.1 Direction of relationship

As was previously illustrated, relationships between variables can differ in direction, which include:

1. Positive relationship - as one variable increases so does the other
2. Negative relationship - as one variable increases the other decreases
3. No relationship - there is no pattern in the way two variables vary

2.2 Form of relationship (Linearity)

You should notice that all the regression lines previously presented are all represented by a straight line. This is because the use of a straight regression line suggests a linear relationship between variables.

1. Linear relationship is one where any change in the x-variable leads to a constant change in the y-variable
   • For example: Every additional year one grows older, BMI is predicted to increase by 0.15.
   • Implication: Despite whether the increase in age is from 20 to 21, or 50 to 51 years older, the incremental increase in BMI is equal to 0.15

There are other relationship forms…

2. A nonlinear relationship is one in which change in x-variable leads to a varying change in the y-variable (see plot directly below)
   • For example: Heroin and euphoria. Heroin use may initially generate strong feelings of euphoria, but with continued use can sharply decrease feelings of euphoria as tolerance to the drug develops. In other words, the drop in euphoria with continued use does not occur at a constant rate
   • Implication: From the 1st to 2nd month of heroin use, the drop in feelings of euphoria is greater than from the 5th to 6th month of heroin use

2.3. Strength of relationship

Whilst the regression line or “line of best fit” can tell us

1. direction of the relationship (positive vs negative) - by whether the line is upwards (positive) or downwards (negative) facing, and
2. form of the relationship (linear vs nonlinear) - whether the line is straight (linear) or curved (nonlinear)

the regression line can also tell us

3. strength of the relationship (weak vs strong) - by the closeness of the points to the regression line
   • closer to the line = stronger linear relationship
   • Less close to the line = weaker linear relationship

3.1. Information from regression equation

Linear regression equation example: Age and Body mass index (BMI)
See regression equation below, which estimates the relationship between age and BMI

\[ \begin{aligned} BMI = 0.15*Age+24.2 \ ~~~~OR~~~~\ y = 0.15*x + 24.2 \end{aligned} \]

\(x\) = \(Age\)

\(y\) = \(BMI\)

How do we interpret this regression equation?

1. For every year one gets older (i.e. for every 1 year), there is a predicted 0.15 increase in BMI

The regression equation is a formula

0.15*...+ 24.2

0.15*40 + 24.2

0.15*24 + 24.2
0.15*25 + 24.2
0.15*30 + 24.2
0.15*32 + 24.2
0.15*36 + 24.2
0.15*41 + 24.2
0.15*45 + 24.2

plot(...$xcoord.age, ...$ycoord.BMI, xlab="...", ylab="...", main ="...")

plot(xycoord$xcoord.age, xycoord$ycoord.BMI, xlab="Age", ylab="BMI", main ="Regression line")

0.15*28 + 24.2

3.2. Altering aspects of regression equations

When the estimates \(b\) and \(c\) are altered in the regression equation, this alters the appearance of the regression line

\[ \begin{aligned} y =bx+c \end{aligned} \]

3.3 Producing the regression equation

In order to calculate components of the regression equation based on the raw data, the lm() command, in conjunction with the summary() command, can be used

Fear of intimacy and relationship length
Researchers are interested in investigating the relationship between fear of intimacy (named “fear.intimacy”) and relationship length (named “length.relationship”). Data on both variables is collected from a sample of 200.

mod2=lm(length.relationship~fear.intimacy, data = FearIntimacyRelationship)

summary(mod2)

Exercise 4.1.1.1

Exercise 4.1.1.2

5.1 Calculating Standard Error of Estimate (SE)

The Standard Error of Estimate captures the amount of error in the regression model, or can alternatively be viewed as a measure of the accuracy in the regression model, where the lower the SE, the more accurate/less error in the regression model. The SE is based on the OLS method mentioned in the previous section (section 4), with some additional features from point 3 onwards:

1. Square the distances \((y - y')^2\) between observed (\(y\)) and predicted (\(y'\)) values - remember why we need to square the differences?
2. Take the sum of the squared distances (\(\sum(y - y')^2\))
3. Calculate the average sum of squared distances (\(\frac{\sum(y - y')^2}{N}\)) - An average is taken because the sum of squared distances is influenced by sample size (as sample size increases, so does \(\sum(y - y')^2\))
4. Square root the value obtained in 3 - we are undoing the earlier squaring here, converting a measure of variance to a standard deviation - remember how variance is converted to standard deviation?

Thus, the Standard Error of Estimate can be described as attempting, in principle, to compute an average of the vertical distances between each observed value and predicted values along the regression line (but uses squaring and square-rooting to deal with the difficulties we mentioned earlier).

See population SE formula below:

\[ \begin{aligned} \sigma_{y.x} = \sqrt{\frac{\sum(y - y')^2}{N}} \end{aligned} \]

\(y\) = observed values (each observed value is entered into the formula)
\(y'\) = predicted values (each corresponding predicted value is entered into the formula)
\(N\) = population size

See sample SE formula below

\[ \begin{aligned} s_{y.x} = \sqrt{\frac{\sum(y - y')^2}{n-2}} \end{aligned} \]

Unlike the population SE, when you are dealing with an entire population, the formula for the sample SE differs somewhat.

\(N\) is replaced with \(n-2\), where \(n\) refers to the size of the sample. This is because \(n-2\) represents the degrees of freedom (see Guided tutorial 6)

5.2 Evaluating Standard Error of Estimate (SE)

In order to evaluate whether a SE is large or small, we need to compare it to something.

The SE (\(s_{y.x}\)) is compared against the standard deviation of y (\(s_{y}\)), Why?

1. SE (\(s_{y.x}\)) uses the regression line to predict y for every x-value
2. We can think of computing the standard deviation of y (\(s_{y}\)) as a regression type of calculation, in that there is a regression line through \(\bar{Y}\), and we find the “average” of the distances of the y values from this line. Another way of thinking about this is to ask ‘what is the best prediction we could make of y, if we did not have information about x?’ The answer to that is that we could guess \(\bar{Y}\), the mean of y, and that would not be a good prediction, but the best we could do in the absence of knowing anything about the predictor, x.

Run the chunk below, and you will see what we mean. Note that sd(y) is very similar to the standard error of estimate in the output; the differences arise because for the SE of estimate we divide by (n-2), and for the sd by (n-1). Remember that in the output of lm in R, what is called the ‘residual standard error’ is what is more generally called the standard error of estimate, \(s_{y.x}\) In the plot, the red line is the line through \(\bar{Y}\), and represents the equation \(Y = \bar{Y}\). The distance between each point and the red line can be measured by \(S_Y\). The blue line that is shown in the plot is the line of best fit, and you can see that it does a better job of fitting the data than the red line (which is the best we can do in the absence of x)

x <- rnorm(50)
y <- x + rnorm(50,1,1) # x represents the sequence number of the y values
plot(x,y)
abline(a = mean(y), b= 0, col = "red")
abline(lm(y~x),col = "blue")
summary(lm(y ~ x))
sd(y)

We look at the ratio of \(s_{y.x}\) and \(s_{y}\) to make sense of the SE

\[ \begin{aligned} \frac{s_{y.x}}{s_{y}} \end{aligned} \]

If the SE (\(s_{y.x}\)) is smaller than the standard deviation of y (\(s_{y}\)), we can say that the regression model improves the ability to predict y-values over what we would achieve in the absence of the model

6.1. Covariance

Covariance is an important aspect of both regression and correlation (which we will discuss later). Covariance measures the extent to which two variables co-vary

Covariance formula:

\[ \begin{aligned} cov_{xy} = {\frac{\sum(x - \overline{x})(y - \overline{y})}{n-1}} \end{aligned} \]

\(x\) = x-variable values
\(\overline{x}\) = x-variable average
\(y\) = y-variable values
\(\overline{y}\) = y-variable average
\(n\) = sample size

cov.out=cov(FearIntimacyRelationship$..., FearIntimacyRelationship$...)
cov.out

cov.out=cov(FearIntimacyRelationship$length.relationship, FearIntimacyRelationship$fear.intimacy)
cov.out

6.2 Covariance and regression coefficients

Covariance is important because it is used to calculate the regression coefficient/s (\(m\)) in the regression equation

Recall the general form of the regression equation below:

\[ \begin{aligned} y = bx+a \end{aligned} \] \(b\) is the regression coefficient, and captures the relationship between x- (independent) and y- (dependent) variables.

It is calculated using the following formula:

\[ \begin{aligned} b = \frac{cov_{xy}}{s_{x}^2} \end{aligned} \]

\(s_{x}^2\) = variance of x

.../var(FearIntimacyRelationship$...)

cov.out/var(FearIntimacyRelationship$fear.intimacy)

6.3 Correlation

Correlation is another way of quantifying the degree of accuracy, or error, of the regression equation (although a correlation/s can be estimated independently of a regression model).

Correlation is best described as quantifying the strength of the relationship between two variables

The covariance is used to calculate correlation - as covariance increases, so does correlation.

See correlation formula below, otherwise referred to as the product moment correlation coefficient or pearson product moment:

\[ \begin{aligned} r = \frac{cov_{xy}}{s_{x}s_{y}} \end{aligned} \] \(r\) = Pearson product moment/correlation coefficient
\(s_{x}s_{y}\) = product (i.e. multiplicative) of the standard deviations of both variables

.../(sd(...$...)*sd(...$...))

cov.out/(sd(FearIntimacyRelationship$fear.intimacy)*sd(FearIntimacyRelationship$length.relationship))

cor(...$..., ...$...)

cor(FearIntimacyRelationship$fear.intimacy, FearIntimacyRelationship$length.relationship)

6.4 Correlation vs regression coefficient

The correlation is very similar to the regression coefficient found in the regression equation in that it can be interpreted as having

1. direction of relationship -> (e.g. an \(r\) of -0.7 indicates a negative relationship, whilst an \(r\) of 0.7 indicates a positive relationship)
2. strength of relationship -> (e.g. an \(r\) of 0.7 indicates a stronger relationship than an \(r\) of 0.3)

In the case of the relationship between fear of intimacy and relationship length, both \(r\) (-0.69) and \(b\) (-0.33) are shown in the figure below.

Despite the similarities between correlation and regression coefficients, there are some key differences:

1. the correlation coefficient is a standardised measure of the strength of the relationship between 2 variables, meaning that despite the units in which variables are measured, correlations can only range from -1 to 1. On the other hand, regression coefficients are expressed in the same unit of measurement and on the same scale as the variable
2. the correlation coefficient cannot be used to predict values, whilst the regression coefficient can

6.5 Correlation strength classification

A classification system can be used to describe the strength of a relationship based on the size of the correlation coefficient (see below)

Exercise 7.1.1.1

cor(...$..., ...$...)

cor(AgeBMI$Age, AgeBMI$BMI)

Exercise 7.1.1.2

Exercise 7.1.1.3

AgeBMI.mod=lm(...~..., data =...)
summary(AgeBMI.mod)

AgeBMI.mod=lm(BMI~Age, data =AgeBMI)
summary(AgeBMI.mod)

Exercise 8.1.1.1

Exercise 8.1.1.2

cor.test(...$..., ...$..., alternative = "...")

cor.test(AgeBMI$Age, AgeBMI$BMI, alternative = "two.sided")

Exercise 8.1.1.3

Move on to and complete Statistics Tutorial Assignment 7 on Amathuba (Activities | Assignments)