Native Probability Outputs for Clinical Triage: An Evaluation of Calibration and Decision Accuracy
Overview & Synopsis
Patient-facing triage requires an appropriate care recommendation and recognition of when information is insufficient. We evaluate whether Jev, a decision model that returns probabilities over predefined actions, supports both judgments. Using CARE-Bench, we compare Jev with four general-purpose language models and three input-independent baselines across 925 decision points from 439 public cases. On the public test subset, Jev ranks fourth of five models by macro-F1 (0.461) but has the lowest observed multiclass Brier score (0.610), an 18.7\% improvement over uniform prediction (case-bootstrap 95\% interval, 4.9--31.1\%). Noul has lower binary Brier than Choice (0.215 versus 0.276). A evaluation of 453 HealthBench conversations again gives Jev the lowest observed Brier score (0.299), but measures a distinct emergency-referral judgment. Jev combines lower probability error with missed clarification needs in the four-action task. Asking about information sufficiency separately increases recall but also produces frequent unnecessary clarification assignments.
arxiv.org)
← / → for pages • + / - for zoom • 0 to fit width
Powered by Mozilla PDF.js v3.11
Document Information
| Document Title | Native Probability Outputs for Clinical Triage: An Evaluation of Calibration and Decision Accuracy |
|---|---|
| Author / Organization | Liang You, Zhiyin Zhou, Dawei Tang, Chuchu Wu, Jingzhuo Hu |
| Subject & Discipline | cs.LG |
| Publication Year | 2026 |
| License / Rights | arXiv Open Access / Creative Commons |
| Resource Verification | Listed PDF (Verification Pending) |
| Original Host Website |
arXiv
(arxiv.org)
|
| File Format | application/pdf (Portable Document Format) |
| Language | EN |
| Catalog ID | DOC-276794 |
| Original Source Link | Visit Source Page on arxiv.org » |