imported>Widefox: /* top */ update tags

2025-05-27T19:30:20Z

top: update tags

New page

{{more citations needed|date=August 2014}}
{{Infobox software
| name = High Performance Storage System
| developer = HPSS Collaboration ([[IBM]], [[LANL]], [[LBNL]], [[LLNL]], [[ORNL]], [[Sandia National Laboratories|SNL]])
| latest_release_version = 11.2
| latest_release_date = February 2025
| operating_system = Linux
| genre = [[Hierarchical Storage Management]]
| license = [[Proprietary software|Proprietary]]
| website = [http://www.hpss-collaboration.org hpss-collaboration]
}}

'''High Performance Storage System''' ('''HPSS''') is a flexible, [[scalable]], policy-based, software-defined [[hierarchical storage management]] (HSM) product developed by [[#The HPSS Collaboration|the HPSS Collaboration]]. It provides scalable HSM, archive, and file system services using cluster, [[LAN]] and [[storage area network]] (SAN) technologies to aggregate the capacity and performance of many computers, disks, disk systems, tape drives, and tape libraries.<ref name ="hpss_collab">{{cite web |url=http://www.hpss-collaboration.org |title=Official HPSS Collaboration Website |publisher=IBM}}</ref>

== Architecture ==
HPSS supports a variety of methods for accessing and creating data. Among them are support for [[FTP]], parallel FTP, [[Filesystem in Userspace|FUSE]] (Linux), as well as a robust client [[API]] with support for parallel I/O.

As of version 7.5, HPSS has full support on [[Linux]]. The HPSS client [[API]] is supported on [[AIX]], [[Linux]], and [[Solaris (operating system)|Solaris]].<ref name="hpss_collab"/>

The implementation is built around IBM's [[IBM Db2|Db2]], a scalable [[relational database management system]].

== The HPSS Collaboration ==
In early 1992, several [[United States Department of Energy]] (DOE) [[United States Department of Energy national laboratories|National Laboratories]] — [[Lawrence Livermore National Laboratory|Lawrence Livermore]] (LLNL), [[Los Alamos National Laboratory|Los Alamos]] (LANL), [[Oak Ridge National Laboratory|Oak Ridge]] (ORNL), and [[Sandia National Laboratory|Sandia]] (SNL) — joined with IBM to form the National Storage Laboratory (NSL).<ref>{{Cite web |title=High Performance Storage System Taking the Long View |url=https://str.llnl.gov/2022-09/heer |access-date=2023-03-29 |website=str.llnl.gov}}</ref> The NSL's purpose was to commercialize software and hardware technologies that would overcome computing and data storage bottlenecks.<ref>{{Cite book |last1=Watson |first1=R.W. |last2=Coyne |first2=R.A. |title=Proceedings Thirteenth IEEE Symposium on Mass Storage Systems. Toward Distributed Storage and Data Management Systems |chapter=The National Storage Laboratory (NSL): Overview and status |date=June 1994 |chapter-url=https://ieeexplore.ieee.org/document/373025 |pages=39–43 |doi=10.1109/MASS.1994.373025|isbn=0-8186-5580-1 |s2cid=206444692 }}</ref> The NSL's research on data storage gave birth to the collaboration which produces HPSS. This collaboration began in the fall of 1992<ref name ="hpss_llnl">{{cite web |url=https://computing.llnl.gov/projects/hpss |title=HPSS at LLNL |publisher=LLNL}}</ref> and involved [[IBM]]'s [[Houston]] Global Services and five DOE national labs ([[Lawrence Berkeley National Laboratory|Lawrence Berkeley]] [LBL], LLNL, LANL, ORNL, and SNL).<ref name="hpss_collab" /> At that time, the HPSS design team at the DOE national laboratories and IBM recognized there would be a data storage explosion driven by computing power rising to [[teraflops]]/[[PetaFLOPS|petaflops]] requiring data stored in HSMs to rise to petabytes and beyond, data transfer rates with the HSM to rise to gigabytes/s and higher, and daily throughput with a HSM in tens of terabytes per day. Therefore, the collaboration set out to design and deploy a system that would scale by a factor of 1,000 or more and evolve from the base above toward these expected targets and beyond.<ref name="petabyte">[http://www.hpss-collaboration.org/learn_who_petabyte_data.shtml Largest HPSS Sites 1+ petabytes]</ref>

The HPSS collaboration is based on the premise that no single organization has the experience and resources to meet all the challenges represented by the growing imbalance between computing power and data collection capabilities, and storage system I/O, capacity, and functionality. Over twenty organizations worldwide including industry, US Department of Energy (DOE), other federal laboratories, universities, [[National Science Foundation]] (NSF) supercomputer centers, French [[Commissariat à l'énergie atomique et aux énergies alternatives|Commissariat a l'Energie Atomique]] (CEA), and Gleicher Enterprises have contributed to various aspects of this effort.

As of 2022, the primary HPSS development team consists of:
* [[IBM]] Global Business Services ([[Houston]], [[Texas|TX]])
* [[Los Alamos National Laboratory]] ([[Los Alamos, New Mexico|Los Alamos]], [[New Mexico|NM]])
* [[Lawrence Livermore National Laboratory]] ([[Livermore, California|Livermore]], [[California|CA]])
* [[Lawrence Berkeley National Laboratory|Lawrence Berkeley]] [[National Energy Research Scientific Computing Center]] ([[Berkeley, California|Berkeley]], [[California|CA]])
* [[Oak Ridge National Laboratory]] ([[Oak Ridge, Tennessee|Oak Ridge]], [[Tennessee|TN]])
* [[Sandia National Laboratory]] ([[Albuquerque]], [[New Mexico|NM]])

== Notable achievements ==
* In March 2024, [[European Centre for Medium-Range Weather Forecasts|ECMWF]] became the first HPSS site to reach an [[exabyte]] of data stored in a single HPSS namespace.<ref name="exabyte">{{cite web |title=To an Exabyte and beyond! |url=https://hpss-collaboration.org/news/ |publisher=HPSS |access-date=26 February 2025}}</ref>
* On November 14, 2007, the [[San Diego Supercomputer Center]] along with [[IBM]], DataDirect, and Brocade demonstrated a "Billion File" test which successfully backed up a billion files from [[GPFS]] into HPSS.<ref name="sdsc">[http://www.hpcwire.com/hpc/1896617.html HPCWire Nov 15, 2007] {{webarchive |url=https://web.archive.org/web/20071117014105/http://www.hpcwire.com/hpc/1896617.html |date=November 17, 2007 }}</ref>
* In May 2013 a 380 Petabyte HPSS installation entered service at the [[National Center for Supercomputing Applications]] (NCSA) at the [[University of Illinois at Urbana-Champaign]].<ref>{{cite web |url=http://www.ncsa.illinois.edu/news/story/ncsa_puts_worlds_largest_high_performance_storage_system_into_production |title=NCSA puts world's largest High Performance Storage System into production |date=2013-05-30 |accessdate=2014-08-30}}</ref>

== References ==
<references/>

[[Category:IBM storage devices]]
[[Category:Storage software]]

High Performance Storage System - Revision history

imported>Widefox: /* top */ update tags