BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/New_York
X-LIC-LOCATION:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20210402T160552Z
LOCATION:Track 3
DTSTART;TZID=America/New_York:20201113T171000
DTEND;TZID=America/New_York:20201113T174000
UID:submissions.supercomputing.org_SC20_sess222_ws_cafcw132@linklings.com
SUMMARY:Why I’m Not Answering: Understanding Determinants of Classificatio
 n of an Abstaining Classifier for Cancer Pathology Reports
DESCRIPTION:Workshop\n\nWhy I’m Not Answering: Understanding Determinants 
 of Classification of an Abstaining Classifier for Cancer Pathology Reports
 \n\nDhaubhadel, McMahon\n\nSafe deployment of deep learning systems to cri
 tical real world applications requires models to make few mistakes, and on
 ly under predictable circumstances. Development of such a model is not yet
  possible, in general. In this work, we address this problem with an absta
 ining classifier tuned to have >95% accuracy, and identify the determinant
 s of abstention with LIME (Local Interpretable Model-agnostic Explanations
 ).  \n\nEssentially, we are training our model to learn the attributes of 
 pathology reports that are likely to lead to incorrect classifications, al
 beit at the cost of reduced sensitivity. We demonstrate our method in a mu
 ltitask setting to classify cancer pathology reports from the NCI SEER can
 cer registries on six tasks of greatest importance. For these tasks, we re
 duce the classification error rate by factors of 2–5 by abstaining on 25–4
 5% of the reports. For the specific case of cancer site, we are able to id
 entify metastasis and reports involving lymph nodes as responsible for man
 y of the classification mistakes, and that the extent and types of mistake
 s vary systematically with cancer site (eg. breast, lung, and prostate). W
 hen combining across three of the tasks, our model classifies 50% of the r
 eports with an accuracy greater than 95% for three of the six tasks and gr
 eater than 85% for all six tasks on the retained samples. By using this in
 formation, we expect to define work flows that incorporate machine learnin
 g only in the areas where it is sufficiently robust and accurate, saving h
 uman attention to areas where it is required.\n\nRegistration Category: Wo
 rkshop Reg Pass
END:VEVENT
END:VCALENDAR