One Number from Four Cells

Question

A signal classifier produces the confusion table: TP = 30, FP = 20, FN = 10, TN = 940. Compute precision, recall, and the F1 score. Why is F1 the harmonic mean rather than the arithmetic mean of precision and recall?

Accepted Answer

The three numbers. - Precision = fracTPTP + FP = frac3050 = 0.6: when the model fires, it is right 60% of the time. - Recall = fracTPTP + FN = frac3040 = 0.75: it catches 75% of the true events. - F1 = frac2 times 0.6 times 0.750.6 + 0.75 = frac0.91.35 = mathbffrac23 approx 0.667. Why harmonic. The harmonic mean is dominated by the smaller input. A degenerate classifier that predicts positive on everything gets recall 1 and tiny precision; the arithmetic mean would still award it approx 0.5, while the harmonic mean collapses toward the tiny precision. F1 therefore cannot be gamed by maxing one side, you must be simultaneously selective and complete. Formally, the harmonic mean of rates is the right average when the quantities are ratios with different denominators, and it always sits below the arithmetic mean unless P = R. The trap to flag: accuracy here is frac9701000 = 97%, yet a model that never fires scores 96%. With 4% prevalence, accuracy is nearly information-free, which is exactly why the question hands you a confusion table instead. In production signal research, where tradeable events are rare, precision/recall style metrics (or better, PnL-weighted versions of them) are

One Number from Four Cells

Problem

Your answer

Hints