Comprehensive Guide to Principal Component Analysis (PCA) in R

Introduction

The session walked participants through the theory and practical implementation of Principal Component Analysis (PCA) using R. PCA is a multivariate technique that transforms a set of correlated variables into a smaller set of uncorrelated components, preserving most of the original variability.

Why Use PCA?

• Correlated predictors: Traditional regression assumes independent predictors; when variables are highly correlated, multicollinearity inflates variance and invalidates results.
• Dimensionality reduction: PCA condenses many measurements (e.g., flower morphometrics) into a few components that capture the bulk of information.
• Data screening: Outliers, clusters, and data quality issues become visible in PCA plots.

Theoretical Foundations

1. Data matrix: An (n \times p) matrix (X) where rows are experimental units and columns are variables (X_1 … X_p).
2. Standardization: When variables have different units (cm, mm, inches), they are centered (mean = 0) and scaled (SD = 1) to make them comparable.
3. Covariance vs. Correlation matrix:
4. Covariance retains original scales; suitable only when variables share similar units.
5. Correlation matrix is based on standardized data and is preferred for most PCA applications.
6. Eigenvalues & eigenvectors:
7. Eigenvalues ("latent roots") indicate the amount of variance each component explains.
8. Eigenvectors ("latent vectors") provide the loadings – the weights that combine original variables into a component.
9. Component ordering: PC1 explains the greatest variance, PC2 the next greatest, and so on. All PCs are orthogonal (uncorrelated).

Preparing Data in R

• Load required packages: FactoMineR, factoextra, ggplot2, gridExtra, ggraph, GGally, etc.
• Set working directory and import the dataset (e.g., the classic iris CSV).
• Explore the data: histograms for normality, scatter‑matrix for pairwise relationships, boxplots for group comparisons.
• Standardize using scale() before feeding the data to PCA.

Running PCA in R

library(FactoMineR)
library(factoextra)
# Remove the categorical column (Species) and standardize
pca_res <- PCA(iris[,-5], scale.unit = TRUE, graph = FALSE)
summary(pca_res)          # eigenvalues, % variance
pca_res$var$coord          # component scores (PC1, PC2, …)

Key functions: - PCA() – performs the analysis. - summary() – shows eigenvalues and cumulative variance. - get_eig() (from factoextra) – extracts eigenvalues for scree plots. - fviz_eig() – visual scree ("elbow") plot. - fviz_pca_ind() – individuals (observations) plot, colored by species. - fviz_pca_var() – variable contributions (loadings) plot. - fviz_pca_biplot() – combined biplot of individuals and variables.

Interpreting Results

• Eigenvalues: Components with eigenvalue > 1 are typically retained. In the example, PC1 = 2.9 (≈73 % variance) and PC2 = 0.91 (≈23 % variance); together they explain ~96 % of the data.
• Loadings: Large absolute loadings indicate strong contribution. PC1 is driven by Sepal.Length, Petal.Length, and Petal.Width. PC2 is dominated by Sepal.Width.
• Scores: Plotting PC1 vs. PC2 reveals three clusters corresponding to the iris species. Overlap between versicolor and virginica reflects their similar petal measurements.
• Biplot interpretation: Vectors close together (< 90°) are positively correlated; vectors > 90° are negatively correlated; orthogonal vectors indicate little or no correlation.

Choosing the Number of Components

• Scree plot (elbow method) – visualizes eigenvalues; the point where the curve flattens suggests the optimal cut‑off.
• Cumulative variance – aim for > 80 % explained variance for most applications.
• In the demo, the elbow appears after PC2, so PC1 and PC2 were selected for downstream analysis.

Visualizations & Diagnostics

• Scatter matrix (GGally::ggpairs) – simultaneous histograms, scatter plots, and correlation coefficients.
• Boxplots by species – assess group differences before PCA.
• Biplot – combines scores and loadings; useful for spotting outliers and interpreting component directions.
• Contribution plots – bar charts of variable contributions to each PC.

Practical Tips & Common Errors

• Package installation: Install all listed packages before sourcing the script; missing packages cause "function not found" errors.
• Version mismatch: Warnings about packages built under a different R version can be ignored temporarily, but updating R is advisable.
• File handling: Ensure the CSV is unzipped and placed in the working directory; use read.csv() or read_excel() accordingly.
• Standardization: Forgetting scale = TRUE leads to misleading PCs when variables have different units.
• Interpretation: Remember PCA is exploratory; it does not provide p‑values. For confirmatory analysis, follow up with clustering or discriminant analysis.

Next Steps After PCA

• Cluster analysis: Use the PC scores as input for k‑means or hierarchical clustering to formalize the observed groups.
• Discriminant analysis: Test whether the identified groups are statistically separable and obtain classification probabilities.
• Regression on PCs: If a predictive model is needed, regress the response on the retained PCs (they are orthogonal, satisfying the independence assumption).

Closing Remarks

The session emphasized the importance of aligning statistical techniques with research objectives, avoiding blind application of methods, and continuously expanding one’s toolbox through practice and community resources.

Principal Component Analysis converts many correlated measurements into a few orthogonal components that retain most of the original information, making it indispensable for dimensionality reduction, data exploration, and preparing data for further multivariate modeling in R.