SC20 Is Everywhere We Are

SC20 Virtual Platform
Can We Build a Virtuous Cycle Between Machine Learning and HPC?
Presenter
Event Type
Invited Talk
Tags
Applications
Big Data
Extreme Scale Computing
Machine Learning, Deep Learning and Artificial Intelligence
Registration Categories
TP
XO
TimeWednesday, 18 November 202010:45am - 11:30am EST
LocationTrack 1
DescriptionMachine learning (ML) draws on many HPC components; AlexNet was enabled by GPUs, and GPU-based MLPerf submissions connect data replicas using Infiniband. In other ways, it feels like HPC and ML are diverging. Just a handful of the current ML startups are considering HPC applications. Google's TPUs use a proprietary interconnect and a limited set of collective operations. Many HPC codes are written for double precision; ML machines reach for peta-ops of low-precision floating-point. Programming stacks are very different, with rare commonality (Horovod integrates MPI into TensorFlow). Can we construct a better, virtuous cycle between ML and HPC? In this talk I'll examine the opportunities for parallelization, mixed precision and new algorithms (particularly sparsity) in uniting the two fields and solving common problems.
Back To Top Button