Course material:
- Book: recommended, not required
→ Only chapter 2 self study, other chapters are background
- Written exam (80%)
- Assignments (20%)
Lecture 1 - Introduction IRM
Multivariate analysis
- All statistical methods that simultaneously analyze multiple measurements on each individual
or object under investigation.
- Any simultaneous analysis of two or more variables.
- Examining relationships between/among more than two variables.
Measurement scales
How different variables can be collected and/or measured.
1. Non-metric scales:
→ Qualitative
I. Nominal
Assigns numbers to label or identify subjects or objects. These numbers have no
quantitative meaning beyond indicating the presence or absence of the attribute.
- Characteristics: Unique definition/identification
- Phenomena: E.g. brand name, gender, student number
- Methods of analysis: %, mode, Chi square test
II. Ordinal
- Characteristics: Indicate ‘’order’’, sequence
- Phenomena: Preference ranking, level of education
- Methods of analysis: Percentiles, median, rank correlation, + previous methods
2. Metric scales:
→ Quantitative
→ Variables can be ordered in relation to the amount of the attribute possessed
I. Interval
, Interval scales use an arbitrary zero point
- Characteristics: Arbitrary origin
- Phenomena: Attribute scores, price index, celcius
- Methods of analysis: Arithmetic average, rane, SD, + previous methods
II. Ratio
Ratio scales have an absolute zero point
- Characteristics: Unique origin
- Phenomena: Age, cost, number of customers
- Methods of analysis: Geometric average, coefficient of variation, + previous
methods
Errors
- Reliability
The degree to which the observed variables measure the ‘’true’’ value or is error free; if
the measure is asked repeatedly, more reliable measures will show greater consistency
than less reliable measures.
→ Opposite of measurement error
→ Test and retest to make sure data is reliable
- Validity
The degree to which a measure accurately represents what it is supposed to; does the
measure capture the concept it is supposed to measure?
Measurement error
The degree to which the observed values are not representative of the ‘’true’’ values.
→ When variables with measurement error are used to compute correlations or means,
the ‘’true’’ effect is partially masked by the measurement error, causing correlations to
weaken and the means to be less precise.
Statistical significance and power
Interpreting statistical inferences requires the researcher to specify the acceptable levels of
statistical error that result from using a sample (sampling error).
1. Type 1 error (α)
,The probability of a test showing statistical
significance when it is not present.
- Measurement error: false positive
- The probability of incorrectly rejecting the
null hypothesis.
- Saying a correlation exists when it
actually does not.
2. Type 2 error (β)
The probability of incorrectly failing to
reject the null hypothesis.
- Not reject the null hypothesis when it is actually false.
- The chance of not finding a correlation of means
difference when it does exist.
3. Power (1-β)
The probability of a test showing statistical significance
when it is present.
- The probability of correctly rejecting the null hypothesis when it should be rejected.
- Limiting type 1 error: Use a stricter decision rule
- Cutoff: Set it in such way that alpha is smaller than 0.05
Power depends on:
1. Alpha (+)
As alpha becomes more restrictive, power decreases.
- As the researcher reduces the chance of incorrectly saying an effect is significant when
it is not, the probability of correctly finding an effect decreases.
- If you are willing to accept a higher type 1 error, power increases
2. Effect size (+)
The probability of achieving statistical significance is also based on the actual size of the
effect. It helps researchers determine whether the observed relationship is meaningful or
not.
- Increase in effect size increases power
, 3. Sample size n (+)
Increased sample sizes always produce greater power for the statistical test.
- As sample size increases, researchers must decide if the power is too high; by
increasing sample size, smaller and smaller effects (correlations) will be found
statistically significant until at very large samples almost any effect is significant.
→ Implications: anticipate consequences of alpha, effect, and n; assess/incorporate power
when interpreting results.
Rules of thumb: Statistical power analysis
- Researcher should design studies to achieve a power level of 0.80 at the desired significance
level.
- More stringent significance levels (e.g. 0.01 instead of 0.05) require larger samples to achieve
the desired power level.
- Power can be increased by choosing a less stringent alpha level (e.g. 0.10 instead of 0.05).
- Smaller effect sizes require larger sample sizes to achieve the desired power.
- An increase in power is most likely achieved by increasing the sample size.
Types of multivariate methods:
1. Dependence techniques
- One or more variables can be identified as dependent variables and the remaining as
independent variables.
- Choice of dependence technique depends on the number of dependent variables
involved in analysis.
- When looking for a causal relationship; causes and outcome variables.
- Either metric or non-metric.
2. Interdependence techniques
- Whole set of interdependent relationships is examined to find the underlying structure.
- Further classified as having focus on variables or objects.
- No single variable or group of variables is defined as being dependent or independent.
- No distinction between causes and outputs, just looking for links without grouping them
into dependent and independent variables.
Conducting preliminary analysis:
- Getting a feel for the data
- Suggesting possible problems and remedies in the next steps
> Graphical inspection
> Univariate profiling (one variable at the time)
> Bivariate analysis (two variables at the time)
Outliers
Observations with a unique combination of characteristics identifiable as distinctly different from
- Book: recommended, not required
→ Only chapter 2 self study, other chapters are background
- Written exam (80%)
- Assignments (20%)
Lecture 1 - Introduction IRM
Multivariate analysis
- All statistical methods that simultaneously analyze multiple measurements on each individual
or object under investigation.
- Any simultaneous analysis of two or more variables.
- Examining relationships between/among more than two variables.
Measurement scales
How different variables can be collected and/or measured.
1. Non-metric scales:
→ Qualitative
I. Nominal
Assigns numbers to label or identify subjects or objects. These numbers have no
quantitative meaning beyond indicating the presence or absence of the attribute.
- Characteristics: Unique definition/identification
- Phenomena: E.g. brand name, gender, student number
- Methods of analysis: %, mode, Chi square test
II. Ordinal
- Characteristics: Indicate ‘’order’’, sequence
- Phenomena: Preference ranking, level of education
- Methods of analysis: Percentiles, median, rank correlation, + previous methods
2. Metric scales:
→ Quantitative
→ Variables can be ordered in relation to the amount of the attribute possessed
I. Interval
, Interval scales use an arbitrary zero point
- Characteristics: Arbitrary origin
- Phenomena: Attribute scores, price index, celcius
- Methods of analysis: Arithmetic average, rane, SD, + previous methods
II. Ratio
Ratio scales have an absolute zero point
- Characteristics: Unique origin
- Phenomena: Age, cost, number of customers
- Methods of analysis: Geometric average, coefficient of variation, + previous
methods
Errors
- Reliability
The degree to which the observed variables measure the ‘’true’’ value or is error free; if
the measure is asked repeatedly, more reliable measures will show greater consistency
than less reliable measures.
→ Opposite of measurement error
→ Test and retest to make sure data is reliable
- Validity
The degree to which a measure accurately represents what it is supposed to; does the
measure capture the concept it is supposed to measure?
Measurement error
The degree to which the observed values are not representative of the ‘’true’’ values.
→ When variables with measurement error are used to compute correlations or means,
the ‘’true’’ effect is partially masked by the measurement error, causing correlations to
weaken and the means to be less precise.
Statistical significance and power
Interpreting statistical inferences requires the researcher to specify the acceptable levels of
statistical error that result from using a sample (sampling error).
1. Type 1 error (α)
,The probability of a test showing statistical
significance when it is not present.
- Measurement error: false positive
- The probability of incorrectly rejecting the
null hypothesis.
- Saying a correlation exists when it
actually does not.
2. Type 2 error (β)
The probability of incorrectly failing to
reject the null hypothesis.
- Not reject the null hypothesis when it is actually false.
- The chance of not finding a correlation of means
difference when it does exist.
3. Power (1-β)
The probability of a test showing statistical significance
when it is present.
- The probability of correctly rejecting the null hypothesis when it should be rejected.
- Limiting type 1 error: Use a stricter decision rule
- Cutoff: Set it in such way that alpha is smaller than 0.05
Power depends on:
1. Alpha (+)
As alpha becomes more restrictive, power decreases.
- As the researcher reduces the chance of incorrectly saying an effect is significant when
it is not, the probability of correctly finding an effect decreases.
- If you are willing to accept a higher type 1 error, power increases
2. Effect size (+)
The probability of achieving statistical significance is also based on the actual size of the
effect. It helps researchers determine whether the observed relationship is meaningful or
not.
- Increase in effect size increases power
, 3. Sample size n (+)
Increased sample sizes always produce greater power for the statistical test.
- As sample size increases, researchers must decide if the power is too high; by
increasing sample size, smaller and smaller effects (correlations) will be found
statistically significant until at very large samples almost any effect is significant.
→ Implications: anticipate consequences of alpha, effect, and n; assess/incorporate power
when interpreting results.
Rules of thumb: Statistical power analysis
- Researcher should design studies to achieve a power level of 0.80 at the desired significance
level.
- More stringent significance levels (e.g. 0.01 instead of 0.05) require larger samples to achieve
the desired power level.
- Power can be increased by choosing a less stringent alpha level (e.g. 0.10 instead of 0.05).
- Smaller effect sizes require larger sample sizes to achieve the desired power.
- An increase in power is most likely achieved by increasing the sample size.
Types of multivariate methods:
1. Dependence techniques
- One or more variables can be identified as dependent variables and the remaining as
independent variables.
- Choice of dependence technique depends on the number of dependent variables
involved in analysis.
- When looking for a causal relationship; causes and outcome variables.
- Either metric or non-metric.
2. Interdependence techniques
- Whole set of interdependent relationships is examined to find the underlying structure.
- Further classified as having focus on variables or objects.
- No single variable or group of variables is defined as being dependent or independent.
- No distinction between causes and outputs, just looking for links without grouping them
into dependent and independent variables.
Conducting preliminary analysis:
- Getting a feel for the data
- Suggesting possible problems and remedies in the next steps
> Graphical inspection
> Univariate profiling (one variable at the time)
> Bivariate analysis (two variables at the time)
Outliers
Observations with a unique combination of characteristics identifiable as distinctly different from
