Workshop:The 5th Deep Learning on Supercomputers Workshop
Authors: Hamsa Shwetha Venkataram and Chris A. Mattmann (Jet Propulsion Laboratory, California Institute of Technology) and Scott Penberthy (Google LLC)
Abstract: We all have questions, about today's temperature, scores of our favorite baseball team, the Universe, and about life during COVID-19. Life, physical and natural scientists have been trying to find answers to various topics using scientific methods and experiments, while computer scientists have built language models as a tiny step towards automatically answering all of these questions across domains, given a little bit of context. In this paper, we propose an architecture using state-of-the-art Natural Language Processing language models, namely Topic Models and Bidirectional Encoder Representations from Transformers (BERT), that can transparently and automatically retrieve articles of relevance to questions across domains, and fetch answers to topical questions related to COVID-19 current and historical medical research literature. We demonstrate the benefits of using domain-specific supercomputers like tensor processing units (TPUs), residing on cloud-based infrastructure, with which we could achieve significant gains in training and inference times, with very minimal cost.