Table of Contents
1. Introduction and Overview.................................................................................................................. 8
1.1 Brain Enthusiasm: The Relevance of Distinguishing Fact from Fiction....................... 8
1.2 The Basis of Neural Signals ............................................................................................. 11
1.2.1 Information Transfer in Neurons ............................................................................. 11
1.2.2 Signal Processing ..................................................................................................... 12
1.2.3 Other Signals in the Brain: Molecular and Hemodynamic Signals .......................... 15
1.2.4 Maps in the Brain: From the Activity of Single Neurons to Signals without Single-
Neuron Resolution............................................................................................................ 16
1.3 A Short Overview of Methods in Human Neuroscience ................................................ 18
1.3.1 Techniques to Measure Brain Structure .................................................................. 18
1.3.2 Techniques to Measure Hemodynamic Correlates of Neural Activity .................... 19
1.3.3 Techniques to Measure Electrophysiological Activity ............................................. 20
2. The Physics behind Magnetic Resonance Imaging (MRI) .................................................................. 23
2.1 The Effect of Magnetic Fields on the Human Body ........................................................ 23
2.2 From Resonance to Imaging ........................................................................................... 25
2.3 How Do These Physical Principles Give Rise to an Image with Anatomical Structure? . 30
2.4 The Hardware of a Scanner ............................................................................................ 32
2.5 Parameters That Are Chosen by the User ...................................................................... 33
3. Structural Imaging Methods .............................................................................................................. 34
3.1 Structural T1-Weighted MRI ........................................................................................... 34
3.1.1 Quality Check........................................................................................................... 34
3.1.2 Finding Structure in Anatomical Images and Normalization ................................... 35
3.1.3 Approaches to Investigate Brain Morphometry ...................................................... 38
Intermezzo: quantitative imaging..................................................................................... 39
3.1.4 Statistical Analysis and Interpretation ..................................................................... 40
3.1.5 Voxel-Based Lesion Behavior Mapping (VLBM)....................................................... 40
3.1.6 The Relevance of Brain Structure for Behavior and Mind ....................................... 41
3.2 Diffusion-Weighted Imaging (DWI) ................................................................................ 42
3.2.1 Data Acquisition and Preprocessing ........................................................................ 42
3.2.2 Diffusion Tensor Imaging (DTI) ................................................................................ 43
3.2.3 Advanced Approaches to Capture Heterogeneity of Diffusion in Voxels ................ 45
1
, 3.2.4 The Relevance of Anatomical Connectivity for Behavior and Mind ........................ 46
3.3 Magnetic Resonance Spectroscopy (MRS) ..................................................................... 47
3.3.1 Data Acquisition ...................................................................................................... 47
3.3.2 Data Analysis ........................................................................................................... 49
3.3.3 The Relevance of Molecular Indices for Behavior and Mind .................................. 50
4. Hemodynamic Imaging Methods ...................................................................................................... 51
4.1 Hemodynamics and Its Relationship to Neural Activity ................................................. 51
4.1.1 The Hemodynamic Response Function ................................................................... 51
4.1.2 The Relationship between the HRF and Different Aspects of Neural Activity ........ 53
4.2 Functional Magnetic Resonance Imaging (fMRI) ........................................................... 55
4.2.1 Blood-Oxygenation-Level Dependent fMRI............................................................. 55
4.2.2 Arterial Spin Labeling fMRI ...................................................................................... 57
4.2.3 The Relevance of fMRI for Behavior ........................................................................ 57
4.3 Positron Emission Tomography (PET) ............................................................................. 58
4.3.1 The Physics of PET ................................................................................................... 58
4.3.2 Using PET for Measuring Neural Activity................................................................. 58
4.3.3 Unique Contributions of PET ................................................................................... 59
4.4 Functional Near-Infrared Spectroscopy (fNIRS) ............................................................. 60
4.5 A Comparison of Research with fMRI, PET, and fNIRS ................................................... 61
5. Designing a Hemodynamic Imaging Experiment ............................................................................... 63
5.1 Think Before You Start an Experiment ........................................................................... 63
5.2 Which Conditions to Include: The Subtraction Method ................................................. 63
5.2.1 The Subtraction Method ......................................................................................... 63
5.2.2 Considerations About the Subtraction Method ...................................................... 64
5.3 How to Present the Conditions: The Block Design ......................................................... 65
5.3.1 The Block Design and the Hemodynamic Response Function ................................ 65
5.3.2 The Block Design in Practice in fMRI and fNIRS ...................................................... 67
5.3.3 A Few Examples of Classical Studies Using a Block Design ..................................... 68
5.4 The Event-Related Design ............................................................................................... 69
5.4.1 More Advanced Designs and Analyses .................................................................... 70
5.5 The Baseline or Rest Condition ...................................................................................... 70
5.5.1 The Role of a Baseline in Task-Based fMRI .............................................................. 70
2
, 5.5.2 Regions Activated During a Resting Baseline .......................................................... 71
5.6 Task and Stimuli in the Scanner ...................................................................................... 71
6. Image Processing ............................................................................................................................... 73
6.1 Software Packages .................................................................................................... 73
6.2 Properties of the Images ................................................................................................ 73
Preprocessing Step 0: quality control ............................................................................... 74
6.3 Preprocessing Step 1: Slice Timing ................................................................................. 75
6.4 Preprocessing Step 2: Motion Correction ...................................................................... 76
6.5 Preprocessing Step 3: Co-registration ...................................................................... 78
6.6 Preprocessing Step 4: Normalization ............................................................................. 79
6.7 Preprocessing Step 5: Spatial Smoothing ....................................................................... 79
External quality control through transparency and reproducibility................................. 80
7. Basis Statistical Analyses.................................................................................................................... 82
7.1 Statistical Analyses: The General Linear Model ............................................................. 82
7.1.1 Simple Linear Regression......................................................................................... 82
7.1.2 Multiple Linear Regression ...................................................................................... 83
7.1.3 The General Linear Model Applied to fMRI Data .................................................... 83
7.1.4 Data Cleaning Prior to Applying the GLM ............................................................... 85
7.1.5 The Efficiency of a Design and Correlation between Predictors ............................. 85
7.2 Determining Significance and Interpreting It ................................................................. 87
7.2.1 Calculating a Simple Test Statistic: A t-Contrast ...................................................... 87
7.2.2 Correction for Multiple Comparisons, or How to Avoid Brain Activity in Dead
Salmon .............................................................................................................................. 89
7.2.3 Combining Data across Participants: Second-Level Whole-Brain Analyses ............ 90
7.2.4 Region-of-Interest Analyses..................................................................................... 91
7.2.5 Another Statistical Caveat: Double Dipping and Circular Analyses ......................... 92
7.2.6 Statistical Inference ................................................................................................. 93
8. Advanced Statistical Analyses ............................................................................................................ 94
8.1 Functional Connectivity: Designs and Analyses ............................................................. 94
8.1.1 Correlations in Brain Activity ................................................................................... 94
8.1.2 The Interpretation of Correlations in Brain Activity ................................................ 95
8.1.3 Modeling Directional Functional Connectivity ........................................................ 96
3
, 8.1.4 Task-Related Modulations of Connectivity .............................................................. 98
8.1.5 Resting-State fMRI (RS fMRI) ................................................................................... 99
8.2 Multi-voxel Pattern Analyses (MVPA) ........................................................................... 101
8.2.1 A Schematic Tutorial of MVPA ............................................................................... 102
8.2.2 A Specific Example of MVPA .................................................................................. 103
8.2.3 The Potential of MVPA to Move beyond Neophrenology ..................................... 104
8.2.4 What Do We Measure with MVPA? ...................................................................... 105
8.3 Functional MRI Adaptation........................................................................................... 106
9. Electromagnetic Field of the Brain .................................................................................................. 107
9.1 Electrophysiological Activity of the Brain ..................................................................... 107
9.1.1 From Neurons to Electric Field .............................................................................. 107
9.1.2 Magnetic Field of the Neural Activity .................................................................... 108
9.1.3 From the Field to Sensors ...................................................................................... 108
9.2 Electromagnetic Field Signals ....................................................................................... 108
9.2.1 Properties of the Field Signal ................................................................................ 108
9.2.2 Dimensions and Resolution of the Field Signal ..................................................... 109
9.3 Brain Dynamics vs. Mind Dynamics .............................................................................. 111
10. Electroencephalography and Magnetoencephalography ............................................................. 112
10.1 Electroencephalography (EEG) ................................................................................... 112
10.1.1 EEG Electrodes..................................................................................................... 112
10.1.2 EEG Amplifier....................................................................................................... 115
10.1.3 Procedure for EEG Data Acquisition .................................................................... 116
10.2 Magnetoencephalography (MEG) .............................................................................. 116
10.2.1 MEG Sensors........................................................................................................ 116
10.2.2 Magnetically Shielded Room ............................................................................... 118
10.2.3 Procedure for MEG Data Acquisition .................................................................. 118
10.3 Comparison between EEG and MEG .......................................................................... 119
11. Basic Analysis of Electrophysiological Signals................................................................................ 120
11.1 Noises ......................................................................................................................... 120
11.1.1 Biological Noises .................................................................................................. 120
11.1.2 Artifactual and Environmental Noises ................................................................. 121
11.1.3 Visual Inspection ................................................................................................. 121
4
, 11.2 Data Format and Analysis Software ........................................................................... 121
11.2.1 Data Format ......................................................................................................... 121
11.2.2 Analysis Software ................................................................................................ 122
11.3 Preprocessing ............................................................................................................. 122
11.3.1 Referencing .......................................................................................................... 122
11.3.2 Segmentation and Channel Rejection ................................................................. 123
11.3.3 Independent Component Analysis for Preprocessing ......................................... 124
11.3.4 Filtering for Preprocessing................................................................................... 125
11.3.5 Resampling .......................................................................................................... 127
11.3.6 More About Preprocessing.................................................................................. 127
11.4 Spectral Analysis ......................................................................................................... 127
11.4.1 Workflow of Spectral Analysis ............................................................................. 128
11.4.2 More about FFT ................................................................................................... 128
11.4.3 Spectral Plots ....................................................................................................... 128
11.4.4 Frequency Bands of M/EEG Signal ...................................................................... 128
11.4.5 Statistical Analysis ............................................................................................... 129
11.4.6 Pros and Cons ...................................................................................................... 130
11.5 Event-Related Potential and Event-Related Field Analysis ......................................... 130
11.5.1 More About Trial Averaging ................................................................................ 130
11.5.2 Workflow of ERR Analysis .................................................................................... 131
11.5.3 Time- and Surface Plot of ERR ............................................................................. 131
11.5.4 Statistical Analysis ............................................................................................... 131
11.5.5 The Naming of ERR .............................................................................................. 132
11.5.6 Pros and Cons ...................................................................................................... 133
11.6 Steady-State Evoked Response ................................................................................... 134
11.6.1 More About ssER ................................................................................................. 134
11.6.2 Statistical Analysis ............................................................................................... 136
11.6.3 Pros and Cons ...................................................................................................... 136
12. Advanced Data Analysis................................................................................................................. 137
12.1 Time-Frequency Analysis ............................................................................................ 137
12.1.1 Short Time Fourier Transform ............................................................................. 137
12.1.2 Wavelet Transform .............................................................................................. 138
5
, 12.1.3 STFT vs. Wavelet .................................................................................................. 140
12.1.4 Analysis of Time-Frequency Data ........................................................................ 140
12.2 Phase Analysis ............................................................................................................ 142
12.2.1 Computation of the Phase................................................................................... 142
12.2.2 Circular Statistics ................................................................................................. 142
12.2.3 Phase Synchrony.................................................................................................. 143
12.2.4 Inter-trial Phase Coherence................................................................................. 146
12.2.5 Trial Averaging Revisited...................................................................................... 146
12.3 Autoregression and Granger Causality ....................................................................... 147
12.3.1 Autoregression .................................................................................................... 147
12.3.2 Granger Causality ................................................................................................ 148
13. Multi-modal Imaging ..................................................................................................................... 150
13.1 The Spatial and Temporal Unfolding of Neural Representations ............................... 151
Note: From biological to artificial brains ........................................................................ 153
13.2 Simultaneous Application of EEG and fMRI ............................................................... 154
13.3 M/EEG Source Localization (involving MRI) ............................................................... 156
13.4 Differentiating between Representational and Access Theories of Disorders ........... 157
13.5 Clinical Diagnostics with Multi-modal Imaging .......................................................... 159
14. Causal Methods to Modulate Brain Activity .................................................................................. 160
14.1 Microstimulation and Deep Brain Stimulation ........................................................... 160
14.2 Focused Ultrasound Stimulation (FUS)....................................................................... 161
14.3 Transcranial Magnetic Stimulation (TMS) .................................................................. 162
14.4 Transcranial Current Stimulation (TCS)....................................................................... 165
15. Computational Neuroimaging ....................................................................................................... 168
15.1 Computational Models ............................................................................................... 168
15.2 Computational Models of Behavior and Cognition .................................................... 169
15.2.1 Effects of Social Networks on Brain Function ...................................................... 169
15.2.2 Prediction Error and Reinforcement Learning .................................................... 169
15.2.3 Drift Diffusion and Evidence Accumulation ......................................................... 171
15.3 Computational Models of Neural Processing ............................................................. 172
15.3.1 Population Receptive Fields (pRFs) ..................................................................... 172
15.3.2 Encoding Models ................................................................................................. 175
6
,16. Conclusion ..................................................................................................................................... 179
7
,1. Introduction and Overview
• In real life, we cannot see the signals emitted from someone’s brain. Nevertheless, the signals
are there in every person
• Sometimes the underlying physical principles sound very complex, but at other times these
principles come surprisingly close to an optic signal
In the past decades, scientists’ ability to measure brain signals has improved radically
• 1930: the advent of electroencephalography (EEG)
• 1970: radiographic methods (ex. computerized tomography)
• 1980-1990: brain imaging techniques (ex. positron emission tomography (PET) and functional
magnetic resonance imaging (fMRI))
• 1990-2000: Decade of the Brain → an evolution not without criticism
o The application of these brain imaging methods has only increased in frequency since
then, across all scientific disciplines
1.1 Brain Enthusiasm: The Relevance of Distinguishing Fact from Fiction
Some basic knowledge about brain imaging will allow us to distinguish the actual scientific potential
of these methods from science fiction (we give four examples)
Example 1: 2009, Verbeke
• Claim: in 5 years, people applying for important jobs would undergo brain scans as an
important supplement to traditional job interviews and behavioral testing
o Brain scan (fMRI) to determine if person is a good fit for the job, and so avoid hiring
the wrong person
o Brainscan as part of the selection process
• Verbeke’s own company can do this for you, for only € 5.000
• Situation today (2024)
o Job interviews and psychometric tests are still standard practice when hiring people,
brain scans are not
o Main problem: it is not easy to make big claims about 1 single person and his brain
scans
Example 2: brain scans as evidence in court of law
• Using brain imaging to make very important decisions about individual people’s lives
o Lie detection
o Personality assessment (fMRI)
▪ Want to see if someone is really to be trusted
▪ Understand why a certain criminal murdered someone
▪ Some lawyers have used brain scans and said “there is a difference in the
activity in the frontal cortex and has this personality problem and because of
that you should not put hem/her in jail”
o Control of actions
▪ “He did not know what he was doing, in this brain scan you can see that the
person had no control over his actions”
8
, • Brain scans often lack the validity and reliability to justify strong claims at the level of
individual subjects
o The brain scans might be overinterpreted by laypersons and as such provide
misleading evidence that negatively impacts legal decisions making
Example 3: state of consciousness tests
• Research on patients in a persistent vegetative state or suffering from locked-in syndrome
o If they can communicate it is often through eye-movement
o But some patients have no way to communicate anymore with people around them
▪ This research suggests that brain imaging can be used to test the state of
consciousness of these patients even though they lack the ability to
communicate with their environment
• Ask to answer yes/no questions by imagining two very different events which are so different
that they can easily be distinguished based on elicited brain activity
o Just thinking yes or no is not enough, brain activation is almost the same
o You have to give instructions
▪ “If it is ‘yes’ think about watching a tennis game, if it is ‘no’ think about
walking into a house”
o This gives different brain activation patterns and this way yes or no questions can be
answered
o These people have a very high level of consciousness, they need to understand the
instructions and can imagen the events
• Applied to former prime minister of Israel, Ariel Sharon
• The results from such scans are not sufficiently conclusive to be the basis for important
decisions about life and death
➔ Brain reading
• Lie detection and the locked-in syndrome → brain imaging is proposed as a tool to read
people’s mind without needing their consent
o The level of sophistication is simple (binary question)
• More sophisticated demonstrations of brain reading include the use of imaging data to
reconstruct more nuanced aspects of cognition
o Enhanced with artificial intelligence (AI) tools
9
, Example 4: objective diagnosis of diseases
• For several neurological syndromes we are able to diagnose them objectively like brain
tumor, dementia, mild cognitive impairments
• Neuropsychological cases can be diagnosed with brainscans: detection and prognosis of
tumors, CVA, dementia…
o A lot of progress in neuropsychological science in these domains
• But less progress for psychiatric and mental syndromes
o Depression, autism spectrum disorder, schizophrenia, ...
o There are differences between normal and “diseased” brains at the group level!
▪ Findings are very exciting and help in understanding the disorders
▪ But the differences are not large and consistent enough to allow diagnosis at
the individual level
So: The media does not report these nuances when presenting and discussing the results and
unrealistic expectations might be caused
• Media coverage is based on scientific investigation that appear in peer-reviewed journals
• Science is primarily valid and important in its own right and the peer-reviewed studies
advance our knowledge of brain functioning often in a very meaningful way
• The info and claims that the media reports stretches far beyond the original scope of these
reports
➔ Here we recognize an important role of having an in-depth knowledge of the implicated
methodology, which is needed to judge the true potential of these techniques
• Necessary to avoid being a victim of overenthusiasm or overskepticism
Neuroskepticism and the media
• The skepticism targets the scientific use of the methods as well as the claims found in the
popular press
• They argue that brain imaging only informs us about where mental functions are in the brain
o The terms “neolocalizationism” and “neophrenology” are often used in this context
▪ Brain imaging as neo-phrenology?
• Is a reaction to other people being to positive
• Problem with the use and understanding of brain science in popular media
o Leaning towards science fiction
o Data/methods do not support strong conclusion
o Neuromania of neurophiles and neurohawks leads to neurononsense, neurotrash,
and neurobabble
o Brain-hype in media might be over, but number of scientific studies keeps increasing
10
1. Introduction and Overview.................................................................................................................. 8
1.1 Brain Enthusiasm: The Relevance of Distinguishing Fact from Fiction....................... 8
1.2 The Basis of Neural Signals ............................................................................................. 11
1.2.1 Information Transfer in Neurons ............................................................................. 11
1.2.2 Signal Processing ..................................................................................................... 12
1.2.3 Other Signals in the Brain: Molecular and Hemodynamic Signals .......................... 15
1.2.4 Maps in the Brain: From the Activity of Single Neurons to Signals without Single-
Neuron Resolution............................................................................................................ 16
1.3 A Short Overview of Methods in Human Neuroscience ................................................ 18
1.3.1 Techniques to Measure Brain Structure .................................................................. 18
1.3.2 Techniques to Measure Hemodynamic Correlates of Neural Activity .................... 19
1.3.3 Techniques to Measure Electrophysiological Activity ............................................. 20
2. The Physics behind Magnetic Resonance Imaging (MRI) .................................................................. 23
2.1 The Effect of Magnetic Fields on the Human Body ........................................................ 23
2.2 From Resonance to Imaging ........................................................................................... 25
2.3 How Do These Physical Principles Give Rise to an Image with Anatomical Structure? . 30
2.4 The Hardware of a Scanner ............................................................................................ 32
2.5 Parameters That Are Chosen by the User ...................................................................... 33
3. Structural Imaging Methods .............................................................................................................. 34
3.1 Structural T1-Weighted MRI ........................................................................................... 34
3.1.1 Quality Check........................................................................................................... 34
3.1.2 Finding Structure in Anatomical Images and Normalization ................................... 35
3.1.3 Approaches to Investigate Brain Morphometry ...................................................... 38
Intermezzo: quantitative imaging..................................................................................... 39
3.1.4 Statistical Analysis and Interpretation ..................................................................... 40
3.1.5 Voxel-Based Lesion Behavior Mapping (VLBM)....................................................... 40
3.1.6 The Relevance of Brain Structure for Behavior and Mind ....................................... 41
3.2 Diffusion-Weighted Imaging (DWI) ................................................................................ 42
3.2.1 Data Acquisition and Preprocessing ........................................................................ 42
3.2.2 Diffusion Tensor Imaging (DTI) ................................................................................ 43
3.2.3 Advanced Approaches to Capture Heterogeneity of Diffusion in Voxels ................ 45
1
, 3.2.4 The Relevance of Anatomical Connectivity for Behavior and Mind ........................ 46
3.3 Magnetic Resonance Spectroscopy (MRS) ..................................................................... 47
3.3.1 Data Acquisition ...................................................................................................... 47
3.3.2 Data Analysis ........................................................................................................... 49
3.3.3 The Relevance of Molecular Indices for Behavior and Mind .................................. 50
4. Hemodynamic Imaging Methods ...................................................................................................... 51
4.1 Hemodynamics and Its Relationship to Neural Activity ................................................. 51
4.1.1 The Hemodynamic Response Function ................................................................... 51
4.1.2 The Relationship between the HRF and Different Aspects of Neural Activity ........ 53
4.2 Functional Magnetic Resonance Imaging (fMRI) ........................................................... 55
4.2.1 Blood-Oxygenation-Level Dependent fMRI............................................................. 55
4.2.2 Arterial Spin Labeling fMRI ...................................................................................... 57
4.2.3 The Relevance of fMRI for Behavior ........................................................................ 57
4.3 Positron Emission Tomography (PET) ............................................................................. 58
4.3.1 The Physics of PET ................................................................................................... 58
4.3.2 Using PET for Measuring Neural Activity................................................................. 58
4.3.3 Unique Contributions of PET ................................................................................... 59
4.4 Functional Near-Infrared Spectroscopy (fNIRS) ............................................................. 60
4.5 A Comparison of Research with fMRI, PET, and fNIRS ................................................... 61
5. Designing a Hemodynamic Imaging Experiment ............................................................................... 63
5.1 Think Before You Start an Experiment ........................................................................... 63
5.2 Which Conditions to Include: The Subtraction Method ................................................. 63
5.2.1 The Subtraction Method ......................................................................................... 63
5.2.2 Considerations About the Subtraction Method ...................................................... 64
5.3 How to Present the Conditions: The Block Design ......................................................... 65
5.3.1 The Block Design and the Hemodynamic Response Function ................................ 65
5.3.2 The Block Design in Practice in fMRI and fNIRS ...................................................... 67
5.3.3 A Few Examples of Classical Studies Using a Block Design ..................................... 68
5.4 The Event-Related Design ............................................................................................... 69
5.4.1 More Advanced Designs and Analyses .................................................................... 70
5.5 The Baseline or Rest Condition ...................................................................................... 70
5.5.1 The Role of a Baseline in Task-Based fMRI .............................................................. 70
2
, 5.5.2 Regions Activated During a Resting Baseline .......................................................... 71
5.6 Task and Stimuli in the Scanner ...................................................................................... 71
6. Image Processing ............................................................................................................................... 73
6.1 Software Packages .................................................................................................... 73
6.2 Properties of the Images ................................................................................................ 73
Preprocessing Step 0: quality control ............................................................................... 74
6.3 Preprocessing Step 1: Slice Timing ................................................................................. 75
6.4 Preprocessing Step 2: Motion Correction ...................................................................... 76
6.5 Preprocessing Step 3: Co-registration ...................................................................... 78
6.6 Preprocessing Step 4: Normalization ............................................................................. 79
6.7 Preprocessing Step 5: Spatial Smoothing ....................................................................... 79
External quality control through transparency and reproducibility................................. 80
7. Basis Statistical Analyses.................................................................................................................... 82
7.1 Statistical Analyses: The General Linear Model ............................................................. 82
7.1.1 Simple Linear Regression......................................................................................... 82
7.1.2 Multiple Linear Regression ...................................................................................... 83
7.1.3 The General Linear Model Applied to fMRI Data .................................................... 83
7.1.4 Data Cleaning Prior to Applying the GLM ............................................................... 85
7.1.5 The Efficiency of a Design and Correlation between Predictors ............................. 85
7.2 Determining Significance and Interpreting It ................................................................. 87
7.2.1 Calculating a Simple Test Statistic: A t-Contrast ...................................................... 87
7.2.2 Correction for Multiple Comparisons, or How to Avoid Brain Activity in Dead
Salmon .............................................................................................................................. 89
7.2.3 Combining Data across Participants: Second-Level Whole-Brain Analyses ............ 90
7.2.4 Region-of-Interest Analyses..................................................................................... 91
7.2.5 Another Statistical Caveat: Double Dipping and Circular Analyses ......................... 92
7.2.6 Statistical Inference ................................................................................................. 93
8. Advanced Statistical Analyses ............................................................................................................ 94
8.1 Functional Connectivity: Designs and Analyses ............................................................. 94
8.1.1 Correlations in Brain Activity ................................................................................... 94
8.1.2 The Interpretation of Correlations in Brain Activity ................................................ 95
8.1.3 Modeling Directional Functional Connectivity ........................................................ 96
3
, 8.1.4 Task-Related Modulations of Connectivity .............................................................. 98
8.1.5 Resting-State fMRI (RS fMRI) ................................................................................... 99
8.2 Multi-voxel Pattern Analyses (MVPA) ........................................................................... 101
8.2.1 A Schematic Tutorial of MVPA ............................................................................... 102
8.2.2 A Specific Example of MVPA .................................................................................. 103
8.2.3 The Potential of MVPA to Move beyond Neophrenology ..................................... 104
8.2.4 What Do We Measure with MVPA? ...................................................................... 105
8.3 Functional MRI Adaptation........................................................................................... 106
9. Electromagnetic Field of the Brain .................................................................................................. 107
9.1 Electrophysiological Activity of the Brain ..................................................................... 107
9.1.1 From Neurons to Electric Field .............................................................................. 107
9.1.2 Magnetic Field of the Neural Activity .................................................................... 108
9.1.3 From the Field to Sensors ...................................................................................... 108
9.2 Electromagnetic Field Signals ....................................................................................... 108
9.2.1 Properties of the Field Signal ................................................................................ 108
9.2.2 Dimensions and Resolution of the Field Signal ..................................................... 109
9.3 Brain Dynamics vs. Mind Dynamics .............................................................................. 111
10. Electroencephalography and Magnetoencephalography ............................................................. 112
10.1 Electroencephalography (EEG) ................................................................................... 112
10.1.1 EEG Electrodes..................................................................................................... 112
10.1.2 EEG Amplifier....................................................................................................... 115
10.1.3 Procedure for EEG Data Acquisition .................................................................... 116
10.2 Magnetoencephalography (MEG) .............................................................................. 116
10.2.1 MEG Sensors........................................................................................................ 116
10.2.2 Magnetically Shielded Room ............................................................................... 118
10.2.3 Procedure for MEG Data Acquisition .................................................................. 118
10.3 Comparison between EEG and MEG .......................................................................... 119
11. Basic Analysis of Electrophysiological Signals................................................................................ 120
11.1 Noises ......................................................................................................................... 120
11.1.1 Biological Noises .................................................................................................. 120
11.1.2 Artifactual and Environmental Noises ................................................................. 121
11.1.3 Visual Inspection ................................................................................................. 121
4
, 11.2 Data Format and Analysis Software ........................................................................... 121
11.2.1 Data Format ......................................................................................................... 121
11.2.2 Analysis Software ................................................................................................ 122
11.3 Preprocessing ............................................................................................................. 122
11.3.1 Referencing .......................................................................................................... 122
11.3.2 Segmentation and Channel Rejection ................................................................. 123
11.3.3 Independent Component Analysis for Preprocessing ......................................... 124
11.3.4 Filtering for Preprocessing................................................................................... 125
11.3.5 Resampling .......................................................................................................... 127
11.3.6 More About Preprocessing.................................................................................. 127
11.4 Spectral Analysis ......................................................................................................... 127
11.4.1 Workflow of Spectral Analysis ............................................................................. 128
11.4.2 More about FFT ................................................................................................... 128
11.4.3 Spectral Plots ....................................................................................................... 128
11.4.4 Frequency Bands of M/EEG Signal ...................................................................... 128
11.4.5 Statistical Analysis ............................................................................................... 129
11.4.6 Pros and Cons ...................................................................................................... 130
11.5 Event-Related Potential and Event-Related Field Analysis ......................................... 130
11.5.1 More About Trial Averaging ................................................................................ 130
11.5.2 Workflow of ERR Analysis .................................................................................... 131
11.5.3 Time- and Surface Plot of ERR ............................................................................. 131
11.5.4 Statistical Analysis ............................................................................................... 131
11.5.5 The Naming of ERR .............................................................................................. 132
11.5.6 Pros and Cons ...................................................................................................... 133
11.6 Steady-State Evoked Response ................................................................................... 134
11.6.1 More About ssER ................................................................................................. 134
11.6.2 Statistical Analysis ............................................................................................... 136
11.6.3 Pros and Cons ...................................................................................................... 136
12. Advanced Data Analysis................................................................................................................. 137
12.1 Time-Frequency Analysis ............................................................................................ 137
12.1.1 Short Time Fourier Transform ............................................................................. 137
12.1.2 Wavelet Transform .............................................................................................. 138
5
, 12.1.3 STFT vs. Wavelet .................................................................................................. 140
12.1.4 Analysis of Time-Frequency Data ........................................................................ 140
12.2 Phase Analysis ............................................................................................................ 142
12.2.1 Computation of the Phase................................................................................... 142
12.2.2 Circular Statistics ................................................................................................. 142
12.2.3 Phase Synchrony.................................................................................................. 143
12.2.4 Inter-trial Phase Coherence................................................................................. 146
12.2.5 Trial Averaging Revisited...................................................................................... 146
12.3 Autoregression and Granger Causality ....................................................................... 147
12.3.1 Autoregression .................................................................................................... 147
12.3.2 Granger Causality ................................................................................................ 148
13. Multi-modal Imaging ..................................................................................................................... 150
13.1 The Spatial and Temporal Unfolding of Neural Representations ............................... 151
Note: From biological to artificial brains ........................................................................ 153
13.2 Simultaneous Application of EEG and fMRI ............................................................... 154
13.3 M/EEG Source Localization (involving MRI) ............................................................... 156
13.4 Differentiating between Representational and Access Theories of Disorders ........... 157
13.5 Clinical Diagnostics with Multi-modal Imaging .......................................................... 159
14. Causal Methods to Modulate Brain Activity .................................................................................. 160
14.1 Microstimulation and Deep Brain Stimulation ........................................................... 160
14.2 Focused Ultrasound Stimulation (FUS)....................................................................... 161
14.3 Transcranial Magnetic Stimulation (TMS) .................................................................. 162
14.4 Transcranial Current Stimulation (TCS)....................................................................... 165
15. Computational Neuroimaging ....................................................................................................... 168
15.1 Computational Models ............................................................................................... 168
15.2 Computational Models of Behavior and Cognition .................................................... 169
15.2.1 Effects of Social Networks on Brain Function ...................................................... 169
15.2.2 Prediction Error and Reinforcement Learning .................................................... 169
15.2.3 Drift Diffusion and Evidence Accumulation ......................................................... 171
15.3 Computational Models of Neural Processing ............................................................. 172
15.3.1 Population Receptive Fields (pRFs) ..................................................................... 172
15.3.2 Encoding Models ................................................................................................. 175
6
,16. Conclusion ..................................................................................................................................... 179
7
,1. Introduction and Overview
• In real life, we cannot see the signals emitted from someone’s brain. Nevertheless, the signals
are there in every person
• Sometimes the underlying physical principles sound very complex, but at other times these
principles come surprisingly close to an optic signal
In the past decades, scientists’ ability to measure brain signals has improved radically
• 1930: the advent of electroencephalography (EEG)
• 1970: radiographic methods (ex. computerized tomography)
• 1980-1990: brain imaging techniques (ex. positron emission tomography (PET) and functional
magnetic resonance imaging (fMRI))
• 1990-2000: Decade of the Brain → an evolution not without criticism
o The application of these brain imaging methods has only increased in frequency since
then, across all scientific disciplines
1.1 Brain Enthusiasm: The Relevance of Distinguishing Fact from Fiction
Some basic knowledge about brain imaging will allow us to distinguish the actual scientific potential
of these methods from science fiction (we give four examples)
Example 1: 2009, Verbeke
• Claim: in 5 years, people applying for important jobs would undergo brain scans as an
important supplement to traditional job interviews and behavioral testing
o Brain scan (fMRI) to determine if person is a good fit for the job, and so avoid hiring
the wrong person
o Brainscan as part of the selection process
• Verbeke’s own company can do this for you, for only € 5.000
• Situation today (2024)
o Job interviews and psychometric tests are still standard practice when hiring people,
brain scans are not
o Main problem: it is not easy to make big claims about 1 single person and his brain
scans
Example 2: brain scans as evidence in court of law
• Using brain imaging to make very important decisions about individual people’s lives
o Lie detection
o Personality assessment (fMRI)
▪ Want to see if someone is really to be trusted
▪ Understand why a certain criminal murdered someone
▪ Some lawyers have used brain scans and said “there is a difference in the
activity in the frontal cortex and has this personality problem and because of
that you should not put hem/her in jail”
o Control of actions
▪ “He did not know what he was doing, in this brain scan you can see that the
person had no control over his actions”
8
, • Brain scans often lack the validity and reliability to justify strong claims at the level of
individual subjects
o The brain scans might be overinterpreted by laypersons and as such provide
misleading evidence that negatively impacts legal decisions making
Example 3: state of consciousness tests
• Research on patients in a persistent vegetative state or suffering from locked-in syndrome
o If they can communicate it is often through eye-movement
o But some patients have no way to communicate anymore with people around them
▪ This research suggests that brain imaging can be used to test the state of
consciousness of these patients even though they lack the ability to
communicate with their environment
• Ask to answer yes/no questions by imagining two very different events which are so different
that they can easily be distinguished based on elicited brain activity
o Just thinking yes or no is not enough, brain activation is almost the same
o You have to give instructions
▪ “If it is ‘yes’ think about watching a tennis game, if it is ‘no’ think about
walking into a house”
o This gives different brain activation patterns and this way yes or no questions can be
answered
o These people have a very high level of consciousness, they need to understand the
instructions and can imagen the events
• Applied to former prime minister of Israel, Ariel Sharon
• The results from such scans are not sufficiently conclusive to be the basis for important
decisions about life and death
➔ Brain reading
• Lie detection and the locked-in syndrome → brain imaging is proposed as a tool to read
people’s mind without needing their consent
o The level of sophistication is simple (binary question)
• More sophisticated demonstrations of brain reading include the use of imaging data to
reconstruct more nuanced aspects of cognition
o Enhanced with artificial intelligence (AI) tools
9
, Example 4: objective diagnosis of diseases
• For several neurological syndromes we are able to diagnose them objectively like brain
tumor, dementia, mild cognitive impairments
• Neuropsychological cases can be diagnosed with brainscans: detection and prognosis of
tumors, CVA, dementia…
o A lot of progress in neuropsychological science in these domains
• But less progress for psychiatric and mental syndromes
o Depression, autism spectrum disorder, schizophrenia, ...
o There are differences between normal and “diseased” brains at the group level!
▪ Findings are very exciting and help in understanding the disorders
▪ But the differences are not large and consistent enough to allow diagnosis at
the individual level
So: The media does not report these nuances when presenting and discussing the results and
unrealistic expectations might be caused
• Media coverage is based on scientific investigation that appear in peer-reviewed journals
• Science is primarily valid and important in its own right and the peer-reviewed studies
advance our knowledge of brain functioning often in a very meaningful way
• The info and claims that the media reports stretches far beyond the original scope of these
reports
➔ Here we recognize an important role of having an in-depth knowledge of the implicated
methodology, which is needed to judge the true potential of these techniques
• Necessary to avoid being a victim of overenthusiasm or overskepticism
Neuroskepticism and the media
• The skepticism targets the scientific use of the methods as well as the claims found in the
popular press
• They argue that brain imaging only informs us about where mental functions are in the brain
o The terms “neolocalizationism” and “neophrenology” are often used in this context
▪ Brain imaging as neo-phrenology?
• Is a reaction to other people being to positive
• Problem with the use and understanding of brain science in popular media
o Leaning towards science fiction
o Data/methods do not support strong conclusion
o Neuromania of neurophiles and neurohawks leads to neurononsense, neurotrash,
and neurobabble
o Brain-hype in media might be over, but number of scientific studies keeps increasing
10
