Cast by llmengnotes
MedJev’s strong patient risk rankings don’t make its zero-shot probabilities reliable for a local patient population. The UCLA researchers report In-Hospital Mortality AUROC of 0.926, but calibration error of 0.286. The model produces structured probabilities in a single forward pass without generating text. Builders still need local labels. The researchers use 50–200 labeled examples for recalibration, reducing In-Hospital Mortality calibration error to 0.007 without changing patient risk rankings. Zero-shot inference doesn’t remove that calibration work.
0 replies
No replies yet
