How XSEDE will facilitate collaborative science - 1 views
started by Bonnie Sutton on 23 Jul 11
started by Bonnie Sutton on 23 Jul 11
Start a New Topic » « Back to the Educational Technology and Change Journal group
by Warren Froelich, San Diego Supercomputer Center
How will XSEDE provide private, secure environments that have all the resources, services and collaboration support users need to be productive? Along with other project experts, J.P. Navarro, senior HPC administrator with Argonne National Laboratory and deputy manager for Software Development & Integration with XSEDE, outlines how XSEDE plans to achieve that vision.
Q. Scientists need to form communities with their peers. What innovations/tools and space is XSEDE creating to facilitate knowledge sharing, collaboration, and community building?
A.Expanding on the concept of project- or allocation-based groups, XSEDE will provide new services enabling user groups to self-organize to create and share information.
For community or group formation, a new GO-Team group management service will be launched to help registered XSEDE users create collaborations, determine membership, establish membership roles, and arrange and manage compute, data and other resources. This service, to be available in Year 2, will allow these self-managed groups to control and share their content within or outside the project team, providing links to other system components, including a specified email list and a specified wiki, in addition to relevant XSEDE resource allocations. Group administrators will specify criteria that allow membership requests and other group-management tasks to be handled autonomously.
Q. Networking is fundamental to collaboration. How will XSEDE provide a reliable, high-performance, interoperable, effective, network infrastructure to conduct collaborative science?
A.XSEDE's networking vision includes an infrastructure that can meet near-term needs with sufficient flexibility and growth options to extend and evolve to an expanding set of users, campus, and resources.
During the first year, XSEDEnet will provide dedicated 10 Gbps connectivity to all current core XD Service Providers (Indiana, NCSA, NICS, NCAR, PSC, Purdue, SDSC and TACC) using National LambdaRail's (NLR) FrameNet services. By building on NLR's FrameNet service, XSEDE will significantly lower the entry barrier for institutions and collaborations to provision high-performance connections from their resources to XSEDEnet.
In Year 2, XSEDE will build on the framework established in Year 1 with a service that will allow research collaborators to gain access to a high-performance network without having to commit to permanent infrastructure. Specifically, XSEDE is investigating NLR's FrameNet Dynamic VLAN Services for reservation-based (on-demand) high-performance networks between XSEDE service providers and many other potential sites around the country, to enable collaborative teams and their research.
Q. A key component of collaborative science is the ability to manage, store and efficiently move data. How will XSEDE make this functionality easy to use and transparent to a team of users?
A.XSEDE will provide a collection of data replication, remote data access, high-performance data movement, and data collection publishing services that allow collaborative science teams to create, use, manage, and share their data products.
To support transparent management and shared access to data, XSEDE will provide both a parallel wide-area file system with high-performance access from all XSEDE resources and a global, federated file system (GFFS) that will provide transparent file-system level access to a collection of XSEDE data resources to machines outside of XSEDE.
To support data movement, XSEDE is providing a new GO-Data service, a Globus Online-hosted solution providing integrated data management and storage capabilities. Initially, GO-Data will provide site-to-site and desktop-to-site managed data movement and replication. In future months, XD-Datawill incorporate data mirroring (keeping multiple copies in sync), publication (registration of metadata in XD-Registry), allocation and management of "scratch" storage, and other related functions.
Q. How will XSEDE assure users that collaborative data, shared across the network, will remain private and secure to that community?
A. All XSEDE services will be designed and operated to ensure that only authorized users and teams can access the data.
Services that create or move data will require user authentication and enforce file-access permission based on user-controlled permissions. Users will be able to determine when they want to share their data with the public by advertising it in the XD-Registry service and by placing it in publicly accessible storage space.
Q. All XSEDE users need to be able to move their computational work and their data between and among members of their community, using a diversity of technologies that may not be limited to the NSF XD program. How will XSEDE provide these services to user communities in a seamless manner?
A. XSEDE will offer a variety of tools and service to create a consistent user environment for research teams that employ a wide range of technologies both inside and outside the XSEDE program.
To access remote data, for example, users will have a couple of options:
They can choose between moving data using GO-Data managed data movement services and the Globus Connect client package.
They can directly access remote data using the GFFS software and service, which enables communities to plug their file-systems into an XSEDE-wide global file-system that can be transparently accessed from XSEDE and non-XSEDE resources.
Using both of these approaches XSEDE will ensure that users can benefit from the usability and performance characteristics that best match their data access requirements.
To easily move data from campus servers, laptops, and desktops, XSEDE will provide the Globus Connect software that can easily be installed to Mac, Windows, and Linux machines, allowing high-performance data movement to and from XSEDE resources. Globus Connect requires only outbound connections to transfer data and thus works behind most firewalls and NATs.
The GFFS will allow external resources to be included into the federated file system, allowing non-XSEDE data resources to be directly accessed from XSEDE workflows. Using the GFFS, most users will never have to explicitly move data from one site to another, but can access data as if they were local regardless of their location.
Q. Many user communities have advanced, specialized needs. What are XSEDE's plans to help these communities with specialized modes of computation which they need to produce scientific advances?
A. Many users have specialized compute and data-related needs-that may not be shared by many others-that nevertheless are critical to their research. XSEDE services collectively will address specialized needs in multiple ways.
First, services like GO-Job and GO-Data will provide multiple programmable interfaces, allowing users with advanced requirements to access many different resources and services in an integrated manner directly from their applications and software.
Second, XSEDE will provide the XD-Registry service and the Digital Product Advisor that enable registration and exploration of a broad spectrum of computing, storage, data, software, tutorial, and other digital resources. These services leverage and extend the successful TeraGrid Integrated Information Service.
Finally, communities requiring advanced, specialized, and customized capabilities can leverage XSEDE's Science Gateway support framework to design and build their own science gateway infrastructure so they can access XSEDE's many resource-level services and high-level user services. For example, highly customized and targeted science gateways available to XSEDE users on day one can directly access low-level data, file-system, computation, and information resources and services, or to leverage XSEDE's high-level software services-GO-Data, GO-Job, GO-User, GO-Team-which can integrate service complex activities that XSEDE operates and supports. These software services are built on the same principles as the highly successful Software-as-a-Service "SaaS" Web 2.0 services provided by Google, Amazon, and Facebook.
Q&A contributors: Mary Bass, Maytal Dahan, Ian Foster, Andrew Grimshaw, Wendy Huntoon, Christopher Jordan, Stuart Martin, J.P. Navarro, Warren Smith, Steve Tuecke, Nancy Wilkins-Diehr, and Linda Winkler.