Fairness Tradeoffs

Fairness tradeoffs arise when different requirements for a prediction system cannot all be satisfied under the same conditions. The requirements answer different questions: whether a score means the same thing across groups, whether flagged cases have the same outcome frequency, or whether people with the same recorded outcome face the same error rates. Choosing which requirement matters also depends on what the prediction is used to do.^[1]

Measures and denominators

For a binary outcome, a true positive is a correctly flagged positive case; a false positive is a flagged negative case; a false negative is a missed positive case. Chouldechova’s definitions distinguish:

Requirement	Quantity compared across groups	Denominator or conditioning set
Calibration across groups	Outcome frequency at each score	Cases receiving that score
Predictive parity	Positive predictive value: TP / (TP + FP)	Flagged cases at a chosen threshold
False-positive-rate balance	FP / (FP + TN)	Actual negative cases
False-negative-rate balance	FN / (FN + TP)	Actual positive cases
Statistical parity	Fraction flagged	All cases in each group

Error-rate balance requires both error rates to match. Calibration at each score need not imply predictive parity after thresholding: groups can have different mixtures of scores above the threshold. A requirement about the score and a requirement about the resulting binary decision are therefore distinct.^[1]

Why base rates matter

This is teaching arithmetic for a generic binary outcome, not criminal-justice data. Each group contains 1,000 cases. Suppose a classifier catches 80% of positives and flags 10% of negatives:

Quantity	Group A	Group B
Actual positive cases	200	500
Actual negative cases	800	500
Correctly flagged positives	160	400
Incorrectly flagged negatives	80	50
Missed positives	40	100
False-positive rate	80/800 = 10%	50/500 = 10%
False-negative rate	40/200 = 20%	100/500 = 20%
Positive predictive value	160/240 ≈ 66.7%	400/450 ≈ 88.9%

Equal error rates leave different proportions of true positives among flags because the positive outcome is more common in Group B. These counts describe a thresholded classifier; they do not establish calibration of any underlying score.

Chouldechova’s Equation 2.6 gives the exact relationship at an operating threshold:

FPR = [p / (1 − p)] × [(1 − PPV) / PPV] × (1 − FNR)

Here p is the group’s positive-outcome prevalence, and PPV is its positive predictive value. With both outcome classes present (0 < p < 1), a common 0 < PPV < 1, and a common true-positive rate greater than zero (FNR < 1), differing base rates require differing false-positive rates. Thus predictive parity and both equal error rates cannot all hold under these conditions.^[1]

Boundary cases matter. If we flag 160 and 400 true positives with zero false positives, both predictive values are 100%, both false-positive rates are zero and both false-negative rates remain 20%, despite different base rates. This is not perfect prediction: positives are still missed. A rule that flags nobody has undefined PPV. Neither case supports an unconditional impossibility claim.

A different theorem about scores

Kleinberg and colleagues study probability scores, group calibration to those probabilities, and equal average scores within the positive and negative classes. These class-conditional averages are not threshold false-positive and false-negative rates. In their model, all three score requirements can be satisfied only if the instance permits perfect prediction from its features or the groups have equal base rates. Equal base rates allow, for example, assigning everyone the common prevalence as a score, though that score is uninformative.^[2]

From prediction to consequences

Chouldechova analyzes Broward County COMPAS scores and recorded two-year recidivism outcomes, then separately studies hypothetical penalty policies. Her empirical comparison finds approximate predictive parity at some thresholds alongside unequal error rates; the policy models show how errors could translate into unequal burdens when high-risk classifications trigger stricter penalties. They do not measure actual sentencing effects. A high-risk flag used to offer assistance has different consequences from one used to impose a penalty.^[1]

The COMPAS investigation supplies the documented-case reading path. A statistical criterion alone does not settle whether the outcome label, population, decision threshold or institutional use is justified. Social Alignment and Preference Aggregation address whose interests count and how disagreement is resolved; fairness measures make parts of that decision inspectable rather than replacing it.