BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/New_York
X-LIC-LOCATION:America/New_York
BEGIN:DAYLIGHT
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20210402T160102Z
LOCATION:Track 2
DTSTART;TZID=America/New_York:20201118T160000
DTEND;TZID=America/New_York:20201118T163000
UID:submissions.supercomputing.org_SC20_sess146_pap331@linklings.com
SUMMARY:BATCH:  Machine Learning Inference Serving on Serverless Platforms
  with Adaptive Batching
DESCRIPTION:Paper\n\nBATCH:  Machine Learning Inference Serving on Serverl
 ess Platforms with Adaptive Batching\n\nAli, Pinciroli, Yan, Smirni\n\nSer
 verless computing is a new pay-per-use cloud service paradigm that automat
 es resource scaling for stateless functions and can potentially facilitate
  bursty machine learning serving. Batching is critical for latency perform
 ance and cost-effectiveness of machine learning inference, but unfortunate
 ly it is not supported by existing serverless platforms due to their state
 less design. Our experiments show that without batching, machine learning 
 serving cannot reap the benefits of serverless computing. In this paper, w
 e present BATCH, a framework for supporting efficient machine learning ser
 ving on serverless platforms. BATCH uses an optimizer to provide inference
  tail latency guarantees and cost optimization and to enable adaptive batc
 hing support.  We prototype BATCH atop of AWS Lambda and popular machine l
 earning inference systems. The evaluation verifies the accuracy of the ana
 lytic optimizer and demonstrates performance and cost advantages over the 
 state-of-the-art method MArk and the state-of-the-practice tool SageMaker.
 \n\nTag: Cloud and Distributed Computing, Containers, Machine Learning, De
 ep Learning and Artificial Intelligence, Resource Management and Schedulin
 g\n\nRegistration Category: Tech Program Reg Pass
END:VEVENT
END:VCALENDAR