variance

Understanding variance and standard deviation is fundamental in statistics, as they are key measures that describe the spread or dispersion of a dataset. These concepts help us understand how much individual data points deviate from the mean (average) value.

Variance measures how far each number in the set is from the mean and thus from every other number in the set. In simpler terms, it’s a numerical value that describes the variability of observations from the central tendency (mean). A higher variance indicates that the data points are more spread out from the mean, while a lower variance indicates that the data points are closer to the mean.

The formula for the variance (\(\sigma^2\)) of a population (all members of a set) is:

\[ \sigma^2 = \frac{\sum (x_i - \mu)^2}{N} \]

Where:

\(\sigma^2\) is the population variance.
\(x_i\) represents each value in the population.
\(\mu\) is the population mean.
\(N\) is the number of values in the population.

For a sample (a subset of a population), the formula adjusts to account for the bias in estimating a population parameter from a sample:

\[ s^2 = \frac{\sum (x_i - \bar{x})^2}{n-1} \]

Where:

\(s^2\) is the sample variance.
\(\bar{x}\) is the sample mean.
\(n\) is the number of values in the sample.

Standard Deviation

The standard deviation is the square root of the variance and provides a measure of the spread of the data points in the same units as the data itself. For this reason, it’s more commonly used than variance in data analysis.

For a population:

\[ \sigma = \sqrt{\sigma^2} \]

For a sample:

\[ \sigma = \sqrt{s^2} \]

Example in R

# Sample data
data <- data.frame(x = 1:5, y = c(1, 2, 3, 4, 5))

# Calculate the mean of y
mean_y <- mean(data$y)

# Variance and Standard Deviation
variance <- var(data$y) # Sample variance by default
std_deviation <- sd(data$y) # Sample standard deviation by default
print(paste("Variance:", variance))

## [1] "Variance: 2.5"

print(paste("Standard Deviation:", std_deviation))

## [1] "Standard Deviation: 1.58113883008419"

Sum of ‘Squares’

The following figure illustrates the concept of variance through the visualization of the sum of squares for the simple dataset above (1,2,3,4,5). Each blue point represents an individual data value plotted against its index. The red dashed line indicates the mean (\(μ\)) of the dataset, serving as a central point of reference. From each data point, a dark gray dotted line extends vertically to meet the mean, representing the distance of each point from the mean. The shaded gray areas under these lines symbolize the squared distances of each point from the mean, highlighting the ‘squared’ aspect of variance calculation. Variance is computed as the sum of these squared distances, normalized by the number of observations minus one (\(N−1\)) for a sample variance, providing a measure of the dataset’s spread.

# Load necessary library
library(ggplot2)

# Create the base plot
plot <- ggplot(data, aes(x, y)) +
  geom_point(color = 'blue', size = 3) + # Plot data points
  geom_hline(yintercept = mean_y, color = 'red', linetype = "dashed") + 
  geom_segment(aes(xend = x, yend = mean_y), linetype = "dotted", color = 'darkgray') + 
  theme_minimal() +
  labs(title = "Sum of 'Squares' Visualization",
       x = "Data Point",
       y = "Value") +
  annotate("text", x = Inf, y = mean_y+0.05, label = paste("Mean =", mean_y), hjust = 1.1, vjust = 0, color = "red")

# Adding shaded squares for each data point
data$square_bottom <- mean_y # Bottom of the square is at the mean
data$square_top <- data$y # Top of the square is at the data point
for(i in 1:nrow(data)) {
  plot <- plot + geom_rect(data = data[i, ], aes(xmin = x-0.4, xmax = x+0.4, ymin = square_bottom, ymax = square_top), alpha = 0.2, fill = "gray")
}

# Display the plot
print(plot)

Visualization

Visual aids can provide intuition behind the concepts of variance and standard deviation. Let’s consider a dataset with two different distributions:

Distribution A: Less spread out (lower variance and standard deviation).
Distribution B: More spread out (higher variance and standard deviation).

We’ll visualize these distributions to understand the concepts better.

# Visualizing in R
set.seed(123) # For reproducibility

# Generating data
data_A <- rnorm(100, mean = 50, sd = 5) # Less spread
data_B <- rnorm(100, mean = 50, sd = 15) # More spread

# Plotting
par(mfrow=c(1,2)) # Set the plotting area into a 1x2 array
hist(data_A, main="Distribution A", col="skyblue", xlim=c(20,80), breaks=20)
hist(data_B, main="Distribution B", col="lightpink", xlim=c(20,80), breaks=20)

Distribution A, with a smaller standard deviation, shows data points clustered closely around the mean. In contrast, Distribution B, with a larger standard deviation, displays data points spread out over a wider range of values, indicating higher variability.