From 66.7% to 90.1%: fine-tuning ResNet50 for pneumonia detection

66.7% is a trap

A classifier that lands at 66.7% accuracy on an imbalanced chest X-ray dataset can look like it's learning when it's mostly just predicting the majority class. That was roughly where an out-of-the-box ResNet50 landed before any real tuning, and it's a useful reminder that the headline accuracy number can hide a model that hasn't actually learned the signal you care about.

What actually moved the needle

Three changes, in order of impact: starting from ImageNet-pretrained weights instead of random init, using differential learning rates so the early general-purpose layers fine-tune slower than the task-specific later layers, and data augmentation (rotation, zoom, flips) to fight overfitting on a dataset that's small by deep-learning standards. Together those took accuracy to 90.1% — recall specifically mattered more than raw accuracy here, since a missed pneumonia case (a false negative) is the expensive failure mode, not a false alarm.

Why Grad-CAM was non-negotiable

A 90.1%-accurate model that can't show its work isn't something you'd trust, and it isn't something a clinician should trust either. Grad-CAM visualizations let you check whether the model's attention actually lands on lung opacity regions rather than, say, a scanner artifact or text stamped on the image — the kind of shortcut a model will happily learn if you let it. Adding that interpretability layer mattered as much to me as the accuracy gain itself.

Final numbers: 96% recall, 89% precision, 0.9647 ROC-AUC, and just 15 false negatives out of 390 pneumonia cases across a 624-image test set.