Factor Analysis
1. Introduction
Factor Analysis is a multivariate statistical technique used to identify latent (unobserved)
variables, called factors, that explain the patterns of correlations among a set of observed
variables. It is particularly useful when dealing with complex datasets where many variables are
interrelated and may be influenced by common underlying constructs.
The main purpose of factor analysis is data reduction and structure detection. Instead of analyzing
many observed variables individually, factor analysis groups them into a smaller number of factors
that capture the essential information in the data. This allows researchers to simplify analysis
while maintaining interpretability.
Factor analysis is widely applied in fields such as psychology, education, sociology, economics,
marketing, and finance. For example, multiple survey questions measuring attitudes or behaviors
can often be explained by a few psychological traits.
Unlike Principal Component Analysis (PCA), factor analysis is based on a statistical model that
explicitly accounts for measurement error. It assumes that observed variables are influenced by
common factors as well as unique, variable-specific components.
2. The Factor Analysis Model
The factor analysis model expresses each observed variable as a linear combination of common
factors plus a unique error term.
Mathematically, the model is written as:
𝑋 = Λ𝐹 + 𝜀
where:
• 𝑋is the vector of observed variables
• Λis the matrix of factor loadings
, • 𝐹is the vector of latent common factors
• 𝜀represents unique factors (specific variance and measurement error)
Key Assumptions of the Model:
• Common factors explain the shared variance among variables
• Unique factors are uncorrelated with each other
• Unique factors are uncorrelated with common factors
• The mean of factors and errors is zero
Factor loadings indicate the strength and direction of the relationship between observed variables
and factors. A high loading means that the variable is strongly influenced by the corresponding
factor.
3. Estimation of the Parameters
The goal of parameter estimation in factor analysis is to determine:
• Factor loadings
• Unique variances
• Factor correlations (in oblique models)
The estimated model should reproduce the observed covariance or correlation matrix as closely
as possible.
Common Estimation Methods:
1. Principal Axis Factoring (PAF)
• Focuses on shared variance only
• Does not assume multivariate normality
• Commonly used in exploratory factor analysis
2. Maximum Likelihood Estimation (MLE)
• Assumes multivariate normality
1. Introduction
Factor Analysis is a multivariate statistical technique used to identify latent (unobserved)
variables, called factors, that explain the patterns of correlations among a set of observed
variables. It is particularly useful when dealing with complex datasets where many variables are
interrelated and may be influenced by common underlying constructs.
The main purpose of factor analysis is data reduction and structure detection. Instead of analyzing
many observed variables individually, factor analysis groups them into a smaller number of factors
that capture the essential information in the data. This allows researchers to simplify analysis
while maintaining interpretability.
Factor analysis is widely applied in fields such as psychology, education, sociology, economics,
marketing, and finance. For example, multiple survey questions measuring attitudes or behaviors
can often be explained by a few psychological traits.
Unlike Principal Component Analysis (PCA), factor analysis is based on a statistical model that
explicitly accounts for measurement error. It assumes that observed variables are influenced by
common factors as well as unique, variable-specific components.
2. The Factor Analysis Model
The factor analysis model expresses each observed variable as a linear combination of common
factors plus a unique error term.
Mathematically, the model is written as:
𝑋 = Λ𝐹 + 𝜀
where:
• 𝑋is the vector of observed variables
• Λis the matrix of factor loadings
, • 𝐹is the vector of latent common factors
• 𝜀represents unique factors (specific variance and measurement error)
Key Assumptions of the Model:
• Common factors explain the shared variance among variables
• Unique factors are uncorrelated with each other
• Unique factors are uncorrelated with common factors
• The mean of factors and errors is zero
Factor loadings indicate the strength and direction of the relationship between observed variables
and factors. A high loading means that the variable is strongly influenced by the corresponding
factor.
3. Estimation of the Parameters
The goal of parameter estimation in factor analysis is to determine:
• Factor loadings
• Unique variances
• Factor correlations (in oblique models)
The estimated model should reproduce the observed covariance or correlation matrix as closely
as possible.
Common Estimation Methods:
1. Principal Axis Factoring (PAF)
• Focuses on shared variance only
• Does not assume multivariate normality
• Commonly used in exploratory factor analysis
2. Maximum Likelihood Estimation (MLE)
• Assumes multivariate normality
