Globus Services for Data-Intensive Experimental Research Automation
Presenter
Event Type
Workshop
Applications
Scientific Computing
Simulation
State of the Practice
Technology Challenge
W
TimeFriday, 13 November 20206:05pm - 6:10pm EDT
LocationTrack 10
DescriptionLightning talk: The Globus Architecture for Data-Intensive Experimental Research (Gladier) enables the rapid development of customized automation flows linking many data services to perform data capture, storage and analysis solutions for experimental facilities. Globus provides a collection of data services built for science including: Globus Auth, Transfer, Search, Groups and Flows, and funcX to enable distributed function-as-a-service execution. Globus services are highly reliable, professionally operated cloud-hosted services that support the work of over 150,000 researchers worldwide as foundational capabilities for scientific applications and workflows: using them greatly reduces the burden on local systems, administrators and programmers.
In this lightning talk, we describe the application of these services to establish the Kanzus pipeline, integrating Argonne's Advanced Photon Source (APS) with HPC resources at the Leadership Computing Facility (ALCF) to perform online serial synchrotron crystallography (SSX) analysis to guide experimentation. The Kanzus pipeline automates capture, analysis, transfer, indexing, storage, and cataloging of tens of thousands of images for a given sample. These data are rapidly analyzed
to inform the scientist about sample quality in near real-time, enabling dynamic steering of data acquisition. Once suitable data are captured the flow automatically produces quality statistics and solves the structure, reducing the time to structure from weeks to hours.
In this lightning talk, we describe the application of these services to establish the Kanzus pipeline, integrating Argonne's Advanced Photon Source (APS) with HPC resources at the Leadership Computing Facility (ALCF) to perform online serial synchrotron crystallography (SSX) analysis to guide experimentation. The Kanzus pipeline automates capture, analysis, transfer, indexing, storage, and cataloging of tens of thousands of images for a given sample. These data are rapidly analyzed
to inform the scientist about sample quality in near real-time, enabling dynamic steering of data acquisition. Once suitable data are captured the flow automatically produces quality statistics and solves the structure, reducing the time to structure from weeks to hours.