The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Authors: Stephen Simms (Indiana University, OpenSFS Inc), Frank Baetke (European Open File System Association (EOFS))
Abstract: Lustre is the leading open-source and open-development file system for HPC. Around two thirds of the top 100 supercomputers use Lustre. It is a community-developed technology with contributors from around the world. Lustre currently supports many HPC infrastructures beyond scientific research, such as financial services, energy, manufacturing and life sciences. Lustre clients are available for broadly deployed instruction set architectures such as x86, POWER, and Arm.
At this BOF, Lustre developers, administrators and solution providers will gather to discuss recent Lustre developments and challenges, including the role of Lustre in AI and its use in cloud environments.
Long Description: Lustre is the leading open-source and open-development file system for HPC. Around two thirds of the top 100 supercomputers use Lustre file systems. Lustre is a community developed file system with contributors from around the world. Lustre supports many HPC infrastructures beyond its traditional stronghold of scientific research including financial services, energy, manufacturing, life sciences and animation and Lustre clients are available for ISAs such as x86, POWER, and Arm.
At this BOF, Lustre developers, administrators, and solution providers will gather to discuss recent developments, such as Persistent Client Caching (PCC) and Data on Metadata (DoM) and new challenges and corresponding opportunities, including the role of Lustre in AI, its use in Cloud environments and its extension towards upcoming exascale systems. Today Lustre is one of the most widely adopted technologies in HPC, from the University of Cambridge’s Data Accelerator system to the Tianhe-2 system in China, and the Frontera system at TACC, to the Fugaku computer at RIKEN in Japan, and many more. Lustre is also widely used across mid- and small-scale HPC-systems with continued adoption attributable to the stability of the Lustre file system as it has matured.
Vital to the technology is the community that continues to drive Lustre forward. As Lustre has evolved into a true open-source and open-development model, the end users, developers, and solution providers have come together through the worldwide OpenSFS and EOFS communities.
This community development model has resulted in significant new features, improved stability and broader adoption.
The 2020 Lustre BOF will focus on feature developments and discuss how they will shape future Lustre deployments. Across all key-application segments, Lustre is at the heart of many HPC infrastructures and must continue to evolve in order to support emerging use cases including the
scalability challenges associated with exascale systems. We will explore these cases and discuss the Lustre roadmap for meeting the requirements that they present.
Given current circumstances the session is now planned as a virtual town hall meeting. OpenSFS and EOFS will provide initial discussion topics focusing on a brief set of challenges. OpenSFS and EOFS will compile the list of challenges by polling the community before the BOF and address those at the session.
The Lustre BOF has been held at previous SC events and has been well attended and enthusiastically received. SC presents a rare opportunity for the worldwide Lustre community to meet and discuss how to make Lustre more successful. Attendance is usually very good with well over 150 attendees. The Lustre BOF at SC17 had about 190 attendees, the BOF at SC18 had approximately 180 and the BOF at SC19 had about 150 attendees.
The targeted audience includes all who are involved with Lustre deployments such as administrators, system architects, developers, solution providers and end users. It includes all who have been contributing or reviewing new features or providing additional tools for management and control.
A written report summarizing the Lustre Community BOF will be developed and will be made available to the Lustre community.
URL: http://www.opensfs.org
Back to Birds of a Feather Archive Listing