14. Fair prediction with disparate impact: a study of bias in recidivism
prediction instruments. - Chouldechova (2017).
In dit artikel worden verschillende noties van fairness gepresenteerd en wordt uitgelegd waarom
deze soms op verbazingwekkende manieren niet tegelijkertijd voldaan kunnen worden. Ook
wordt uitgelegd hoe we afwegingen kunnen maken tussen verschillende noties van fairness.
Dit artikel gebruikt gevorderde statistische concepten:
• Positive predictive value (PPV)
• False positive rate (FPR)
• False negative rate (FNR)
- recidivism prediction instruments (RPIs): provide decision-makers with an assessment of the
likelihood that a criminal defendant will reoffend in the future
→ important that they are free from discriminatory biases
- false positive rate (FPR): how often someone innocent is mistakenly thought to be guilty
- false negative rate (FNR): how often someone guilty is mistakenly thought to be innocent
→ in COMPAS: higher FPRs and lower FNRs for black defendants than for white ones
- disparate impact: refers to settings wherein a penalty policy has unintended disproportionate
adverse impact on a particular group
- high-risk score threshold (sHR): defendants whose score S exceeds sHR are referred to as high
risk, the rest is referred to as low risk
- calibration: an algorithm is fairly calibrated if the probability of a positive outcome for each
score of the algorithm is the same for all people, regardless of which group they are in
→ a well calibrated instrument: free from predictive bias or differential prediction
- predictive parity: an algorithm meets predictive parity when the probability of a positive
outcome is the same among all people with a positive prediction value (PPV), regardless of
which group those people are in
- error rate balance: FPRs and PNRs are equal for both groups
- statistical parity: an algorithm satisfies statistical parity if the proportion of individuals
classified as high risk is the same for each group (equal acceptance rates / group fairness)
- predictive parity is incompatible with error rate balance when prevalence differs across groups
- when the recidivism prevalence differs across groups, any instrument that satisfies predictive
parity at a given threshold (sHR) must have imbalanced false positive or false negative error
rates at that threshold
prediction instruments. - Chouldechova (2017).
In dit artikel worden verschillende noties van fairness gepresenteerd en wordt uitgelegd waarom
deze soms op verbazingwekkende manieren niet tegelijkertijd voldaan kunnen worden. Ook
wordt uitgelegd hoe we afwegingen kunnen maken tussen verschillende noties van fairness.
Dit artikel gebruikt gevorderde statistische concepten:
• Positive predictive value (PPV)
• False positive rate (FPR)
• False negative rate (FNR)
- recidivism prediction instruments (RPIs): provide decision-makers with an assessment of the
likelihood that a criminal defendant will reoffend in the future
→ important that they are free from discriminatory biases
- false positive rate (FPR): how often someone innocent is mistakenly thought to be guilty
- false negative rate (FNR): how often someone guilty is mistakenly thought to be innocent
→ in COMPAS: higher FPRs and lower FNRs for black defendants than for white ones
- disparate impact: refers to settings wherein a penalty policy has unintended disproportionate
adverse impact on a particular group
- high-risk score threshold (sHR): defendants whose score S exceeds sHR are referred to as high
risk, the rest is referred to as low risk
- calibration: an algorithm is fairly calibrated if the probability of a positive outcome for each
score of the algorithm is the same for all people, regardless of which group they are in
→ a well calibrated instrument: free from predictive bias or differential prediction
- predictive parity: an algorithm meets predictive parity when the probability of a positive
outcome is the same among all people with a positive prediction value (PPV), regardless of
which group those people are in
- error rate balance: FPRs and PNRs are equal for both groups
- statistical parity: an algorithm satisfies statistical parity if the proportion of individuals
classified as high risk is the same for each group (equal acceptance rates / group fairness)
- predictive parity is incompatible with error rate balance when prevalence differs across groups
- when the recidivism prevalence differs across groups, any instrument that satisfies predictive
parity at a given threshold (sHR) must have imbalanced false positive or false negative error
rates at that threshold
