Can We Build a Virtuous Cycle Between Machine Learning and HPC?
Machine Learning, Deep Learning and Artificial Intelligence
TimeWednesday, 18 November 202010:45am - 11:30am EDT
DescriptionMachine learning (ML) draws on many HPC components; AlexNet was enabled by GPUs, and GPU-based MLPerf submissions connect data replicas using Infiniband. In other ways, it feels like HPC and ML are diverging. Just a handful of the current ML startups are considering HPC applications. Google's TPUs use a proprietary interconnect and a limited set of collective operations. Many HPC codes are written for double precision; ML machines reach for peta-ops of low-precision floating-point. Programming stacks are very different, with rare commonality (Horovod integrates MPI into TensorFlow). Can we construct a better, virtuous cycle between ML and HPC? In this talk I'll examine the opportunities for parallelization, mixed precision and new algorithms (particularly sparsity) in uniting the two fields and solving common problems.