RESEARCH METHODS AND TECHNIQUES II
HOC 1: INTRODUCTION TO SCIENTIFIC REASONING
1. PSYCHOLOGY AS A WAY OF THINKING
Producing versus consuming research
• Research producer – research consumer
o Some of us will produce research, but most of us will consume it
o Producers create new knowledge; consumers apply existing knowledge
o Psychologists often take on both roles – doing research and learning from others
o Both value empiricism: answering psychological questions with direct, formal
observations and communicating with others about what they have learned
• Example to illustrate the importance of the research consumer role
o Ik ben in de VS alle mogelijke opleidingen gaan volgen, bij de absolute toppers’: Griet op
de Beeck reageert op kritiek
- She doesn’t have a degree in psychology but does provide therapy
- She has attended trainings and workshops that give her the skills to help people
- She gives internal family systems therapy
o Internal family systems (IFS) therapy for posttraumatic stress disorder (PTSD) among
survivors of multiple childhood trauma: a pilot effectiveness study
- They invited people suffering from PTSD and measured the degree of PTSD, then
gave them therapy and then measured their degree of PTSD again
- There was a weakening in the degree of PTSD
o But the study was not good → the evidence is really weak
- Sample is too small
- False positives
- No control group
• Feeling the future: experimental evidence for anomalous retroactive influences on cognition and
affect
o This study shows that people can predict the future
o In some studies people could 'predict the future' but in most not, but he only published
the studies in which it succeeded
• Critical mindset is essential
• Not all published research is correct or robust
• Replication crisis in psychology
How scientists work
• The fundamental ways psychologists approach their work: they
o Acts as empiricists in their investigations
o Test theories through research and in turn revise their theories based on the resulting data
o Follow norms in the scientific community that prioritize objectivity and fairness
o Take an empirical approach to both applied research and basic research
o Make their work public
• Science is based on empiricism
, • Scientists test theories
• Scientists work on fundamental and applied problems
• Science is continuously evolving
• Scientists publish their findings in scientific journals
• Scientists communicate with the general public via journalists
Empiricism
• Empiricism means collecting data to test certain theories/hypotheses
• Empirical method is based on data obtained through:
o Our senses (sight, hearing, touch)
o Instruments that assist our senses (thermometer, questionnaires, timer)
• Empiricists aim to do research in a systematic, rigorous, and replicable manner
o Systematic: you use the same method for everyone
o Rigorous: is carried out accurately, thoroughly and according to strict scientific standards
o Replicable: make their work independently verified by other observers
• Empiricism is not based on own experiences, intuition, or authority figures
Scientists test theories
• In the theory-data cycle, scientists collect data to test, change, or update their theories
o First, you asked a particular set of questions that reflected your theorie
- Because of your theory, you chose not to ask other kinds of questions
- Your theory sets you up to ask certain questions and not others
o Next, you make specific predictions, which you tested by collecting data
- You tested your first idea by making a specific prediction
- You tested your prediction
- The data told you your initial prediction was wrong
- You used that outcome to change your idea about the problem
• There is a difference between exploratory (inductive) and confirmatory (deductive) research → this
cycle is only for confirmatory research
• Harlow (1958): cupboard theory versus the contact comfort theory
o Two competing theories
- Cupboard theory: infants attach to their mother because she provides food, creating
positive associations through hunger reduction
- Contact comfort theory: infants attach due to the physical comfort of warmth and
softness (Harlow)
o Harlow’s experiment
, - Two artificial ‘mothers’ were created:
- A wire mother with a milk bottle (food, no comfort)
- A cloth mother with warmth and softness (comfort, no food)
- Predictions
- Cupboard theory: monkeys spend more time with wire mother
- Contact comfort theory: monkeys spend more time with cloth mother
- Neither: monkeys divide time equally
- Results and conclusions
- Monkeys preferred the cloth mother, even though she had no food
- This supported the contact comfort theory, not the cupboard theory
- Harlow's study is a clear example of the theory-data cycle: testing competing
predictions and letting the data decide
• Theory-data cycle
o A theory is a simple set of statements that describes general principles about how
variables relate to one another and leads to specific hypotheses
o A hypothesis is stated in terms of the study design, it is the specific outcome the
researcher will observe in a study if the theory is accurate
- One theory can generate many hypotheses, each tested in separate empirical studies
o Data is a set of observations collected in the study
- If the data matches the hypothesis, they support the theory
- If not, the theory may need to be revised
o Ideally, hypotheses are preregistered, meaning researchers publicly state their expected
outcomes before collecting data
o The theory-data cycle is like a gamble: researchers place a public bet in advance that the
study will come out in favor of the theory, and they are willing to risk being wrong every
time they collect data
• Characteristics of good theories
o Supported by data
o Falsifiable
- A theory should lead to hypotheses that, when tested, could fail to support the theory
- Examples of non-falsifiable theories
- Facilitated communication threatment believers: some therapists use
facilitated communication, believing it helps people with developmental
disorders express themselves. However, controlled studies show that the
messages come from the therapist, not the client - falsifying the theory
behind FC. Supporters reject this evidence, claiming that scientific testing
undermines trust and that FC only works when not scrutinized, making the
belief unfalsifiable.
- See book ‘De ongelovige Thomas heeft een punt’
o Parsimonious (“Occam’s razor”)
• Theories are evaluated based on all available evidence
o It is important to use multiple studies before forming conclusions about a theory
o A theory can never be ‘proven’, but it can be falsified
- Scientists avoid using the word ‘prove’ because, as empiricists, they base
conclusions only on direct observations
o Replication is crucial
, - Scientists therefore evaluate their theories based on the weight of the evidence – the
collection of studies, including replications, of the same theory
Scientists form a community
• Scientists are members of a community, and as such, they follow a set of norms -shared
expectations about how they should act
• Scientists Merton’s scientific norms:
o Universalism = anyone can conduct research, regardless of their background
- Scientific claims are evaluated according to their merit, independent of the
researcher’s credentials or reputation
- The same standards apply to all scientists and all research
- Even if students can do science, you don’t need an advanced degree or research
position
o Communality = researchers form a community and share study results
- Scientific knowledge is created by a community, and its findings belong to the
community
- Scientists should transparently and freely share the results of their work with other
scientists and the public
o Disinterestedness = research should be motivated by the pursuit of knowledge, not
personal gain, for example becoming famous or rich
- Scientists strive to discover the truth, whatever it is; they are not swayed by
conviction, idealism, politics or profit
- Personal beliefs, income, or prestige should not influence a scientist’s interpretation
or reporting data
o Organized skepticism = researchers maintain a critical perspective on others’ work
- Scientists question everything, including their own ideas and widely accepted theories
- Scientists accept almost nothing at face value, they always ask to see the evidence
Fundamental versus applied research
• The empirical method can be used for both applied and basic research questions
• Applied research addresses practical problems in real-world settings
o Is done with a practical problem in mind
o Researchers conduct their work in a local, real-world context
• Basic research aims to expand general knowledge, not solve specific issues
o The knowledge generated by basis research may later be applied to real-world problems
• Translational research uses insights from basic research to develop and test applications in areas
like healthcare, psychotherapy or other interventions
o It serves as a bridge between basic and applied research
HOC 1: INTRODUCTION TO SCIENTIFIC REASONING
1. PSYCHOLOGY AS A WAY OF THINKING
Producing versus consuming research
• Research producer – research consumer
o Some of us will produce research, but most of us will consume it
o Producers create new knowledge; consumers apply existing knowledge
o Psychologists often take on both roles – doing research and learning from others
o Both value empiricism: answering psychological questions with direct, formal
observations and communicating with others about what they have learned
• Example to illustrate the importance of the research consumer role
o Ik ben in de VS alle mogelijke opleidingen gaan volgen, bij de absolute toppers’: Griet op
de Beeck reageert op kritiek
- She doesn’t have a degree in psychology but does provide therapy
- She has attended trainings and workshops that give her the skills to help people
- She gives internal family systems therapy
o Internal family systems (IFS) therapy for posttraumatic stress disorder (PTSD) among
survivors of multiple childhood trauma: a pilot effectiveness study
- They invited people suffering from PTSD and measured the degree of PTSD, then
gave them therapy and then measured their degree of PTSD again
- There was a weakening in the degree of PTSD
o But the study was not good → the evidence is really weak
- Sample is too small
- False positives
- No control group
• Feeling the future: experimental evidence for anomalous retroactive influences on cognition and
affect
o This study shows that people can predict the future
o In some studies people could 'predict the future' but in most not, but he only published
the studies in which it succeeded
• Critical mindset is essential
• Not all published research is correct or robust
• Replication crisis in psychology
How scientists work
• The fundamental ways psychologists approach their work: they
o Acts as empiricists in their investigations
o Test theories through research and in turn revise their theories based on the resulting data
o Follow norms in the scientific community that prioritize objectivity and fairness
o Take an empirical approach to both applied research and basic research
o Make their work public
• Science is based on empiricism
, • Scientists test theories
• Scientists work on fundamental and applied problems
• Science is continuously evolving
• Scientists publish their findings in scientific journals
• Scientists communicate with the general public via journalists
Empiricism
• Empiricism means collecting data to test certain theories/hypotheses
• Empirical method is based on data obtained through:
o Our senses (sight, hearing, touch)
o Instruments that assist our senses (thermometer, questionnaires, timer)
• Empiricists aim to do research in a systematic, rigorous, and replicable manner
o Systematic: you use the same method for everyone
o Rigorous: is carried out accurately, thoroughly and according to strict scientific standards
o Replicable: make their work independently verified by other observers
• Empiricism is not based on own experiences, intuition, or authority figures
Scientists test theories
• In the theory-data cycle, scientists collect data to test, change, or update their theories
o First, you asked a particular set of questions that reflected your theorie
- Because of your theory, you chose not to ask other kinds of questions
- Your theory sets you up to ask certain questions and not others
o Next, you make specific predictions, which you tested by collecting data
- You tested your first idea by making a specific prediction
- You tested your prediction
- The data told you your initial prediction was wrong
- You used that outcome to change your idea about the problem
• There is a difference between exploratory (inductive) and confirmatory (deductive) research → this
cycle is only for confirmatory research
• Harlow (1958): cupboard theory versus the contact comfort theory
o Two competing theories
- Cupboard theory: infants attach to their mother because she provides food, creating
positive associations through hunger reduction
- Contact comfort theory: infants attach due to the physical comfort of warmth and
softness (Harlow)
o Harlow’s experiment
, - Two artificial ‘mothers’ were created:
- A wire mother with a milk bottle (food, no comfort)
- A cloth mother with warmth and softness (comfort, no food)
- Predictions
- Cupboard theory: monkeys spend more time with wire mother
- Contact comfort theory: monkeys spend more time with cloth mother
- Neither: monkeys divide time equally
- Results and conclusions
- Monkeys preferred the cloth mother, even though she had no food
- This supported the contact comfort theory, not the cupboard theory
- Harlow's study is a clear example of the theory-data cycle: testing competing
predictions and letting the data decide
• Theory-data cycle
o A theory is a simple set of statements that describes general principles about how
variables relate to one another and leads to specific hypotheses
o A hypothesis is stated in terms of the study design, it is the specific outcome the
researcher will observe in a study if the theory is accurate
- One theory can generate many hypotheses, each tested in separate empirical studies
o Data is a set of observations collected in the study
- If the data matches the hypothesis, they support the theory
- If not, the theory may need to be revised
o Ideally, hypotheses are preregistered, meaning researchers publicly state their expected
outcomes before collecting data
o The theory-data cycle is like a gamble: researchers place a public bet in advance that the
study will come out in favor of the theory, and they are willing to risk being wrong every
time they collect data
• Characteristics of good theories
o Supported by data
o Falsifiable
- A theory should lead to hypotheses that, when tested, could fail to support the theory
- Examples of non-falsifiable theories
- Facilitated communication threatment believers: some therapists use
facilitated communication, believing it helps people with developmental
disorders express themselves. However, controlled studies show that the
messages come from the therapist, not the client - falsifying the theory
behind FC. Supporters reject this evidence, claiming that scientific testing
undermines trust and that FC only works when not scrutinized, making the
belief unfalsifiable.
- See book ‘De ongelovige Thomas heeft een punt’
o Parsimonious (“Occam’s razor”)
• Theories are evaluated based on all available evidence
o It is important to use multiple studies before forming conclusions about a theory
o A theory can never be ‘proven’, but it can be falsified
- Scientists avoid using the word ‘prove’ because, as empiricists, they base
conclusions only on direct observations
o Replication is crucial
, - Scientists therefore evaluate their theories based on the weight of the evidence – the
collection of studies, including replications, of the same theory
Scientists form a community
• Scientists are members of a community, and as such, they follow a set of norms -shared
expectations about how they should act
• Scientists Merton’s scientific norms:
o Universalism = anyone can conduct research, regardless of their background
- Scientific claims are evaluated according to their merit, independent of the
researcher’s credentials or reputation
- The same standards apply to all scientists and all research
- Even if students can do science, you don’t need an advanced degree or research
position
o Communality = researchers form a community and share study results
- Scientific knowledge is created by a community, and its findings belong to the
community
- Scientists should transparently and freely share the results of their work with other
scientists and the public
o Disinterestedness = research should be motivated by the pursuit of knowledge, not
personal gain, for example becoming famous or rich
- Scientists strive to discover the truth, whatever it is; they are not swayed by
conviction, idealism, politics or profit
- Personal beliefs, income, or prestige should not influence a scientist’s interpretation
or reporting data
o Organized skepticism = researchers maintain a critical perspective on others’ work
- Scientists question everything, including their own ideas and widely accepted theories
- Scientists accept almost nothing at face value, they always ask to see the evidence
Fundamental versus applied research
• The empirical method can be used for both applied and basic research questions
• Applied research addresses practical problems in real-world settings
o Is done with a practical problem in mind
o Researchers conduct their work in a local, real-world context
• Basic research aims to expand general knowledge, not solve specific issues
o The knowledge generated by basis research may later be applied to real-world problems
• Translational research uses insights from basic research to develop and test applications in areas
like healthcare, psychotherapy or other interventions
o It serves as a bridge between basic and applied research
