Summary covering all lectures on methods in the clinical neuropsychological research
course
Lecture 1 - Introduction & science and value for society.
Neuropsychological assessment is only meaningful when it is valid and interpretable:
- Access and familiarity with technology
- Environment —> controlled situation
- Privacy
- Interpretable? —> for example, in telehealth, from the clinical setting to an online platform. Is
this still interpretable in this new environment?
Scienti c practioner:
An individual who has a strong background in science but is also able to provide care.
From the 19th century —> the mind was seen as something immaterial, but around 1900, the rst
recognition of neurons was.
In 1895, the rst x-ray was performed. From that, pneumoencephalography was performed
(dangerous) —> CSF is replaced with oxygen which made the brain better visible. A CT scan was
also followed —> makes multiple photos and puts them together.
Broadman areas were discovered
—> Localized areas and gave these di erent functions.
—> Used as an atlas of the brain. —> Big brain 3D atlas used more often as 3D map of the brain.
Having all these techniques, which method do you choose:
Take ethics into account —> gain knowledge should outweigh the burden of the participant
Code of ethics —> applies when you want to do research
1) Avoidance of exploitation
2) Outweigh bene ts and burden
3) Respect
4) Scienti c validity
5) Scienti c relevance
6) Respect
Milgram experiment (participants need to decide whether they want to shock somebody and are
forced to continue) some ethics are exceeded:
- Deception
- Protection of participants
- Right to withdraw
The WHO law —> Medical law with the subject
Validity
1. Internal validity
A causal relationship between the variables. —> Measuring what you are supposed to measure.
2. External validity
Whether the outcomes are generalizable across the population.
3. Ecological validity
Research ndings related to real-world settings.
For example. —> you want to test whether music a ects driving. So e.g. real-life setting in a car
on the road —> this also increases external validity —> however it decreases internal validity due
to e ects and other stimuli on the road.
ff fifi fi fifi ff ff fi
, 4. Construct validity
Measure that can indicate a latent variable, through for example valid tests or measurements.
Latent refers to the fact that even though these variables were not measured directly in the
research design they are the ultimate goal of the project.
5. Statistical conclusion validity
Whether the statistical methods that are used are appropriate, for example statistical power, use
of control group, etc.
Replication crisis
Only 36% is replicated which has the following causes:
1. Publication bias
Only signi cant results are published even though they may be false positives.
2. Publish or perish
Pressure from the scienti c eld —> when you don’t publish enough papers, you might lose your
job, so people want to publish really hard.
3. Lack of institutional oversight
4. Commercial con ict of interest. —> Bias towards results work for a certain company, so
obscured results?
5. Indaquete training
6. Desire to nd a signi cant result
Leads to biases.
However, these causes do not come from evil intentions necessarily, however, there is ambiguity
about how to make these decisions.
What could improve these problems?
- More dedicated replications
- Improve education
- Pre-registration —> state the hypothesis and methods before publishing the paper, this way
the paper will get published no matter the results.
- Sharing —> create transparency
- Adversarial collaborations —> combine e ects.
Research integrity code
- Honestly
- Scrupulousness —> moral integrity
- Transparency —> able to replicate
- Independence —> no con ict of interest
- Responsibility
Threats to research integrity
1. Plagiarism
2. Data fabrication
3. Unethical use of data
4. Position —> cannot misuse your power
—> Open science good movement
Accessible to everyone, increases reproducibility, increases research integrity
fi fi fl fififl fi ff
, Translational sciences
From labwork to e ective interventions to improve individuals and the public.
Valorization —> The value of research
Who will bene t from what?
Stakeholders:
1. Patients and physicians —> clinical
2. Funding and taxpayers —> societal
3. Researchers —> scienti c
Recommendations for translation:
• Strict reporting guidelines
• Optimizing the science
- Computer-based randomization
- Blinding at outcome measurement, and where possible
- Report exclusions (with rationale), ALL outcome measures - Sample size calculations
• Publication/communication
- How will you reach a stakeholder audience?
• Less focus on impact factor (in the journal, how often is a journal cited), and more on the actual
impact
Levels of translation
From ndings to clinical and vice versa.
Multidisciplinarity is important —> Working parallel together (multiple departments working
together). —> Synergy in treatment.
Lecture 2 - Large datasets in clinical research
> Data sharing —> make data openly accessible.
Advantages of data sharing:
1. Reduce needed resource
2. Pooling datasets
3. Allow other questions to be addressed
4. Research more inclusive
fi fi ff fi
, 5. Transparency
6. Collaboration
7. Increase trust
Disadvantages of data sharing:
1. Personal investment
2. Fears of being scooped —> Somebody else nds a greater nding
3. Fear of errors
4. Fear of misuse —> misrepresents data
5. Lack of time —> takes time to make a suitable dataset for sharing
6. Privacy concerns —> permission to share
However —> Research integrity should always come over personal gain.
Datasets
A) Created for sharing
1. Biobanks
2. Crossectional/longitudinal
3. Multi-lab collaborations
4. Respositories
B) Not created for sharing
1. Medical/Health
2. Dating sites/social media
3. Search engine use
You can only share these data when ethical permission is granted.
Practical implications
1. Ethical treatment highest priority
—> Anominyzed and permission to share.
2. Requested by journals and funders
3. Shared data sets are a lot of work
a) To make it understandable
b) Hard to store les, as they are often very large
4. Using a shared data set also a lot of work
a) Clearing/processing
b) Easier to collaborate with people that have collected it, but not always possible
Statistical implications (when working with large datasets)
1. Usually not a direct re ection of the actual population
a) Depends on the sample size
2. More power
a) Lower standard error
3. Slightly di erent assumptions
a) normal distribution not that important
4. Reduced possibility of manual checks and often missing data points.
> Cross-sectional design
1. Between-group design
- Subgroups of nominal variables
- Subgroups based on continuous variables based on aspects of the data
ff fi fl fi fi
course
Lecture 1 - Introduction & science and value for society.
Neuropsychological assessment is only meaningful when it is valid and interpretable:
- Access and familiarity with technology
- Environment —> controlled situation
- Privacy
- Interpretable? —> for example, in telehealth, from the clinical setting to an online platform. Is
this still interpretable in this new environment?
Scienti c practioner:
An individual who has a strong background in science but is also able to provide care.
From the 19th century —> the mind was seen as something immaterial, but around 1900, the rst
recognition of neurons was.
In 1895, the rst x-ray was performed. From that, pneumoencephalography was performed
(dangerous) —> CSF is replaced with oxygen which made the brain better visible. A CT scan was
also followed —> makes multiple photos and puts them together.
Broadman areas were discovered
—> Localized areas and gave these di erent functions.
—> Used as an atlas of the brain. —> Big brain 3D atlas used more often as 3D map of the brain.
Having all these techniques, which method do you choose:
Take ethics into account —> gain knowledge should outweigh the burden of the participant
Code of ethics —> applies when you want to do research
1) Avoidance of exploitation
2) Outweigh bene ts and burden
3) Respect
4) Scienti c validity
5) Scienti c relevance
6) Respect
Milgram experiment (participants need to decide whether they want to shock somebody and are
forced to continue) some ethics are exceeded:
- Deception
- Protection of participants
- Right to withdraw
The WHO law —> Medical law with the subject
Validity
1. Internal validity
A causal relationship between the variables. —> Measuring what you are supposed to measure.
2. External validity
Whether the outcomes are generalizable across the population.
3. Ecological validity
Research ndings related to real-world settings.
For example. —> you want to test whether music a ects driving. So e.g. real-life setting in a car
on the road —> this also increases external validity —> however it decreases internal validity due
to e ects and other stimuli on the road.
ff fifi fi fifi ff ff fi
, 4. Construct validity
Measure that can indicate a latent variable, through for example valid tests or measurements.
Latent refers to the fact that even though these variables were not measured directly in the
research design they are the ultimate goal of the project.
5. Statistical conclusion validity
Whether the statistical methods that are used are appropriate, for example statistical power, use
of control group, etc.
Replication crisis
Only 36% is replicated which has the following causes:
1. Publication bias
Only signi cant results are published even though they may be false positives.
2. Publish or perish
Pressure from the scienti c eld —> when you don’t publish enough papers, you might lose your
job, so people want to publish really hard.
3. Lack of institutional oversight
4. Commercial con ict of interest. —> Bias towards results work for a certain company, so
obscured results?
5. Indaquete training
6. Desire to nd a signi cant result
Leads to biases.
However, these causes do not come from evil intentions necessarily, however, there is ambiguity
about how to make these decisions.
What could improve these problems?
- More dedicated replications
- Improve education
- Pre-registration —> state the hypothesis and methods before publishing the paper, this way
the paper will get published no matter the results.
- Sharing —> create transparency
- Adversarial collaborations —> combine e ects.
Research integrity code
- Honestly
- Scrupulousness —> moral integrity
- Transparency —> able to replicate
- Independence —> no con ict of interest
- Responsibility
Threats to research integrity
1. Plagiarism
2. Data fabrication
3. Unethical use of data
4. Position —> cannot misuse your power
—> Open science good movement
Accessible to everyone, increases reproducibility, increases research integrity
fi fi fl fififl fi ff
, Translational sciences
From labwork to e ective interventions to improve individuals and the public.
Valorization —> The value of research
Who will bene t from what?
Stakeholders:
1. Patients and physicians —> clinical
2. Funding and taxpayers —> societal
3. Researchers —> scienti c
Recommendations for translation:
• Strict reporting guidelines
• Optimizing the science
- Computer-based randomization
- Blinding at outcome measurement, and where possible
- Report exclusions (with rationale), ALL outcome measures - Sample size calculations
• Publication/communication
- How will you reach a stakeholder audience?
• Less focus on impact factor (in the journal, how often is a journal cited), and more on the actual
impact
Levels of translation
From ndings to clinical and vice versa.
Multidisciplinarity is important —> Working parallel together (multiple departments working
together). —> Synergy in treatment.
Lecture 2 - Large datasets in clinical research
> Data sharing —> make data openly accessible.
Advantages of data sharing:
1. Reduce needed resource
2. Pooling datasets
3. Allow other questions to be addressed
4. Research more inclusive
fi fi ff fi
, 5. Transparency
6. Collaboration
7. Increase trust
Disadvantages of data sharing:
1. Personal investment
2. Fears of being scooped —> Somebody else nds a greater nding
3. Fear of errors
4. Fear of misuse —> misrepresents data
5. Lack of time —> takes time to make a suitable dataset for sharing
6. Privacy concerns —> permission to share
However —> Research integrity should always come over personal gain.
Datasets
A) Created for sharing
1. Biobanks
2. Crossectional/longitudinal
3. Multi-lab collaborations
4. Respositories
B) Not created for sharing
1. Medical/Health
2. Dating sites/social media
3. Search engine use
You can only share these data when ethical permission is granted.
Practical implications
1. Ethical treatment highest priority
—> Anominyzed and permission to share.
2. Requested by journals and funders
3. Shared data sets are a lot of work
a) To make it understandable
b) Hard to store les, as they are often very large
4. Using a shared data set also a lot of work
a) Clearing/processing
b) Easier to collaborate with people that have collected it, but not always possible
Statistical implications (when working with large datasets)
1. Usually not a direct re ection of the actual population
a) Depends on the sample size
2. More power
a) Lower standard error
3. Slightly di erent assumptions
a) normal distribution not that important
4. Reduced possibility of manual checks and often missing data points.
> Cross-sectional design
1. Between-group design
- Subgroups of nominal variables
- Subgroups based on continuous variables based on aspects of the data
ff fi fl fi fi
