Short summary QRMfSD MCK
Lecture 1: The Logic of Experiments: Turning Ideas into Testable Designs
Experimental research: helps evaluate the impact of products, services, interfaces, campaigns, and policies.
Experiment = a research method in which a researcher deliberately manipulates one variable and observes its effect on
another variable while controlling all other factors.
> Variables: - Independent Variable (IV) = the variable manipulated by the researcher.
- Dependent Variable (DV) = the outcome that is measured.
- Covariate = an additional variable that may influence the dependent variable and is therefore
measured and controlled.
- Demographic Variables = Demographic variables describe participants and help researchers
understand the composition of their sample.
Hypothesis = a specific prediction about the relationship between an independent variable and a dependent variable.
- Null Hypothesis (H₀) = The null hypothesis states that there is no effect or difference.
o Mathematically: μ₁ = μ₂
- Alternative Hypothesis (H₁) = The alternative hypothesis states that an effect or difference exists.
Correlation = Correlation means that two variables change together.
Causation = Causation means that one variable directly produces changes in another variable.
Randomization = the process of assigning participants to experimental conditions by chance.
- A quasi-experiment resembles an experiment but lacks random assignment -> occurs when the independent
variable cannot be manipulated.
Significance testing: p < .05, the result is considered statistically significant, and researchers reject the null hypothesis. ->
Statistical significance answers: "Does it work?"
Type I Error (False Positive): occurs when the researcher concludes that an effect exists when it actually does not ->
Probability of making a Type I error is called alpha (α) (often: α = .05).
Type II Error (False Negative): occurs when the researcher concludes that no effect exists when an effect actually does
exist -> Probability of making a Type II error is beta (β) (often: β = .20).
Statistical power refers to the probability of correctly detecting a true effect.
> Power = 1 – β -> Often a minimum power of .80
> Power can be increased in several ways: Larger Sample Sizes, Larger Effect Sizes, Better Measurements, Better
Experimental Control, and Within-Subjects Designs.
Effect size measures the strength of the relationship between variables and indicates the practical importance of a result.
-> Answers "How much does it work?"
> Effect size measures: Used with Minimum Medium Large
Cohen’s d t-tests 0.20 0.50 0.80
Eta Squared (η²) ANOVA 0.01 0.06 0.14
R² regression <0.10 0.10-0.25 >0.25
Researchers must choose how participants are exposed to conditions:
- Between-Subjects Design: Each participant experiences only one condition.
- Within-Subjects Design: Each participant experiences all conditions.
- Mixed Design: A mixed design combines both between-subjects and within-subjects factors.
T-test is used to compare two means and determine whether they differ significantly.
- Independent Samples T-Test = Used when comparing two separate groups.
- Paired Samples T-Test = Used when the same participants are measured twice.
T-Test is used when: One independent variable, Two groups only
, Short summary QRMfSD MCK
Lecture 2: Measuring Impact: Choosing Dependent Variables and Measures
Dependent variables can be measured in two fundamentally different ways:
- Explicit Measures directly ask participants about their thoughts, feelings, attitudes, intentions, or behaviours.
- Implicit Measures not directly ask participants what they think or feel. Instead, researchers infer
psychological processes from behaviour or physiological responses.
Measuring sustainable behaviour:
- Measuring Behavioural Intentions: Researchers often measure intentions because actual behaviour is difficult to
observe.
- Measuring Actual Behaviour: Actual behaviour is often preferred because it avoids the intention-behaviour gap.
Measuring Behaviour in Different Contexts: Online Experiments, Laboratory Experiments, and Field Experiments
Some concepts are difficult to measure directly. Researchers therefore use alternative indicators: Eye Tracking, Reaction
Time Measures, and Self-Report Alternatives.
Scientific rigor requires transparency and replicability. Researchers must clearly describe: What was measured, How it
was measured, Which scales were used, How responses were coded, and Why specific measures were chosen.
Variability refers to the differences observed between individuals, groups, or observations.
> Variance is a statistical measure of variability. It describes how spread out scores are around the mean.
o In ANOVA, variance is often represented as the Sum of Squares (SS).
o Three important sources of variance:
Treatment Variance (Systematic Variance): caused by the independent variable itself. This is the
variance researchers want to observe.
Confound Variance: caused by variables other than the independent variable.
Error Variance (Unsystematic Variance): consists of random influences that cannot easily be
controlled.
o Total Variance = Treatment Variance + Confound Variance + Error Variance
Internal validity refers to the degree to which observed effects can truly be attributed to the independent variable.
- Researchers increase internal validity by: Using strong manipulations, Using validated measures, Controlling
confounding variables, Randomly assigning participants, Using homogeneous samples, Screening inattentive
participants, Removing outliers when justified, and Including covariates.
ANOVA (Analysis of Variance): used when researchers want to compare the means of three or more groups or examine the
effects of multiple independent variables.
- ANOVA is used when: One or more independent variables, Three or more conditions, and Multiple factors are
studied simultaneously
- Types: - One-Way ANOVA (One IV) -> One independent variable, Three or more levels
- Two-Way ANOVA (Factorial ANOVA) -> Two independent variables (2×2 design)
- Three-Way ANOVA -> Three independent variables
ANOVA uses the F-statistic: F-test compares: Variance between groups VS Variance within groups
- Between-Group Variance: Differences caused by the independent variable(s).
- Within-Group Variance: Differences caused by random variation and individual differences.
> F = Mean Square Between / Mean Square Within
Large F-values indicate that group differences are much larger than random noise, suggesting a significant effect.
In factorial ANOVA, researchers examine both main effects and interaction effects.
- Main Effect: A main effect occurs when one independent variable influences the dependent variable regardless of
other variables.
- Interaction Effect: An interaction occurs when the effect of one independent variable depends on another
independent variable.
Scientific reporting requires researchers to report: F-value, Degrees of freedom, p-value, Effect size (η²), Means and
standard deviations, and Follow-up analyses
Lecture 1: The Logic of Experiments: Turning Ideas into Testable Designs
Experimental research: helps evaluate the impact of products, services, interfaces, campaigns, and policies.
Experiment = a research method in which a researcher deliberately manipulates one variable and observes its effect on
another variable while controlling all other factors.
> Variables: - Independent Variable (IV) = the variable manipulated by the researcher.
- Dependent Variable (DV) = the outcome that is measured.
- Covariate = an additional variable that may influence the dependent variable and is therefore
measured and controlled.
- Demographic Variables = Demographic variables describe participants and help researchers
understand the composition of their sample.
Hypothesis = a specific prediction about the relationship between an independent variable and a dependent variable.
- Null Hypothesis (H₀) = The null hypothesis states that there is no effect or difference.
o Mathematically: μ₁ = μ₂
- Alternative Hypothesis (H₁) = The alternative hypothesis states that an effect or difference exists.
Correlation = Correlation means that two variables change together.
Causation = Causation means that one variable directly produces changes in another variable.
Randomization = the process of assigning participants to experimental conditions by chance.
- A quasi-experiment resembles an experiment but lacks random assignment -> occurs when the independent
variable cannot be manipulated.
Significance testing: p < .05, the result is considered statistically significant, and researchers reject the null hypothesis. ->
Statistical significance answers: "Does it work?"
Type I Error (False Positive): occurs when the researcher concludes that an effect exists when it actually does not ->
Probability of making a Type I error is called alpha (α) (often: α = .05).
Type II Error (False Negative): occurs when the researcher concludes that no effect exists when an effect actually does
exist -> Probability of making a Type II error is beta (β) (often: β = .20).
Statistical power refers to the probability of correctly detecting a true effect.
> Power = 1 – β -> Often a minimum power of .80
> Power can be increased in several ways: Larger Sample Sizes, Larger Effect Sizes, Better Measurements, Better
Experimental Control, and Within-Subjects Designs.
Effect size measures the strength of the relationship between variables and indicates the practical importance of a result.
-> Answers "How much does it work?"
> Effect size measures: Used with Minimum Medium Large
Cohen’s d t-tests 0.20 0.50 0.80
Eta Squared (η²) ANOVA 0.01 0.06 0.14
R² regression <0.10 0.10-0.25 >0.25
Researchers must choose how participants are exposed to conditions:
- Between-Subjects Design: Each participant experiences only one condition.
- Within-Subjects Design: Each participant experiences all conditions.
- Mixed Design: A mixed design combines both between-subjects and within-subjects factors.
T-test is used to compare two means and determine whether they differ significantly.
- Independent Samples T-Test = Used when comparing two separate groups.
- Paired Samples T-Test = Used when the same participants are measured twice.
T-Test is used when: One independent variable, Two groups only
, Short summary QRMfSD MCK
Lecture 2: Measuring Impact: Choosing Dependent Variables and Measures
Dependent variables can be measured in two fundamentally different ways:
- Explicit Measures directly ask participants about their thoughts, feelings, attitudes, intentions, or behaviours.
- Implicit Measures not directly ask participants what they think or feel. Instead, researchers infer
psychological processes from behaviour or physiological responses.
Measuring sustainable behaviour:
- Measuring Behavioural Intentions: Researchers often measure intentions because actual behaviour is difficult to
observe.
- Measuring Actual Behaviour: Actual behaviour is often preferred because it avoids the intention-behaviour gap.
Measuring Behaviour in Different Contexts: Online Experiments, Laboratory Experiments, and Field Experiments
Some concepts are difficult to measure directly. Researchers therefore use alternative indicators: Eye Tracking, Reaction
Time Measures, and Self-Report Alternatives.
Scientific rigor requires transparency and replicability. Researchers must clearly describe: What was measured, How it
was measured, Which scales were used, How responses were coded, and Why specific measures were chosen.
Variability refers to the differences observed between individuals, groups, or observations.
> Variance is a statistical measure of variability. It describes how spread out scores are around the mean.
o In ANOVA, variance is often represented as the Sum of Squares (SS).
o Three important sources of variance:
Treatment Variance (Systematic Variance): caused by the independent variable itself. This is the
variance researchers want to observe.
Confound Variance: caused by variables other than the independent variable.
Error Variance (Unsystematic Variance): consists of random influences that cannot easily be
controlled.
o Total Variance = Treatment Variance + Confound Variance + Error Variance
Internal validity refers to the degree to which observed effects can truly be attributed to the independent variable.
- Researchers increase internal validity by: Using strong manipulations, Using validated measures, Controlling
confounding variables, Randomly assigning participants, Using homogeneous samples, Screening inattentive
participants, Removing outliers when justified, and Including covariates.
ANOVA (Analysis of Variance): used when researchers want to compare the means of three or more groups or examine the
effects of multiple independent variables.
- ANOVA is used when: One or more independent variables, Three or more conditions, and Multiple factors are
studied simultaneously
- Types: - One-Way ANOVA (One IV) -> One independent variable, Three or more levels
- Two-Way ANOVA (Factorial ANOVA) -> Two independent variables (2×2 design)
- Three-Way ANOVA -> Three independent variables
ANOVA uses the F-statistic: F-test compares: Variance between groups VS Variance within groups
- Between-Group Variance: Differences caused by the independent variable(s).
- Within-Group Variance: Differences caused by random variation and individual differences.
> F = Mean Square Between / Mean Square Within
Large F-values indicate that group differences are much larger than random noise, suggesting a significant effect.
In factorial ANOVA, researchers examine both main effects and interaction effects.
- Main Effect: A main effect occurs when one independent variable influences the dependent variable regardless of
other variables.
- Interaction Effect: An interaction occurs when the effect of one independent variable depends on another
independent variable.
Scientific reporting requires researchers to report: F-value, Degrees of freedom, p-value, Effect size (η²), Means and
standard deviations, and Follow-up analyses
