WekaFS for HPC and AI Converging Pipelines
Presenter
Event Type
Exhibitor Forum
Machine Learning, Deep Learning and Artificial Intelligence
Storage
XO
TimeThursday, 19 November 20202pm - 2:30pm EDT
LocationTrack 6
DescriptionThis introduction is to the WekaFS scale out parallel file system data and metadata distribution. WekaFS uses commodity servers without specialized hardware to create a parallel file system that provides the fastest throughput, IOPS, metadata and low latency required for today's converging HPC and AI worlds. Winner of the SC19 IO500 Award, the system scales data and metadata across any increasing number of nodes without the need of a centralized metadata server (which creates a hotspot), and is designed to scale to exabytes. The system protects the data across storage nodes as well as providing predictable scalable performance, in flight and at rest encryption, in flight data correctness and self healing. It also has the ability to snapshot and expand the name space between flash and S3, and provide backup and DR to HPC and AI environments that contain billions of files. WekaFS can be used with on-prem systems or in the cloud.
Presenter
