Lustre (file system)
Lustre is a type of parallel distributed file system, generally used for large-scale cluster computing. The name Lustre is a portmanteau word derived from Linux and cluster. Lustre file system software is available under the GNU General Public License (version 2 only) and provides high performance file systems for computer clusters ranging in size from small workgroup clusters to large-scale, multi-site systems. Since June 2005, Lustre has consistently been used by at least half of the top ten, and more than 60 of the top 100 fastest supercomputers in the world, including the world's No. 1 ranked TOP500 supercomputer in November 2022, Frontier, as well as previous top supercomputers such as Fugaku, Titan and Sequoia.
| Initial release | December 16, 2003 |
|---|---|
| Stable release | 2.15.0 (latest major release),
2.15.4 (latest maintenance release), / December 22, 2023 |
| Preview release | 2.15.59
/ 2023-11-08 |
| Repository | |
| Written in | C |
| Operating system | Linux kernel |
| Type | Distributed file system |
| License | GPL v2, LGPL |
| Website | www |
| Company type | Private |
|---|---|
| Founded | 2001 |
| Founder | Peter J. Braam |
| Headquarters | |
Key people | Andreas Dilger, Eric Barton (HPC), Phil Schwan |
| Products | Lustre file system |
| Introduced | December, 2003 with Linux |
|---|---|
| Structures | |
| Directory contents | Hash, Interleaved Hash with DNE in 2.7+ |
| File type | file, directory, hardlink, symlink, block special, character special, socket, FIFO |
| Bootable | No |
| Limits | |
| Min volume size | 32 MB |
| Max volume size | 700 PB (production), over 16 EB (theoretical) |
| Max file size | 32 PB (ext4), 16 EB (ZFS) |
| File size granularity | 4 KB |
| Max no. of files | Per Metadata Target (MDT): 4 billion files (ldiskfs backend), 256 trillion files (ZFS backend), up to 128 MDTs per filesystem |
| Max filename length | 255 bytes |
| Max dirname length | 255 bytes |
| Max directory depth | 4096 bytes |
| Allowed filename characters | All bytes except NUL ('\0') and '/' and the special file names "." and ".." |
| Features | |
| Dates recorded | modification (mtime), attribute modification (ctime), access (atime), delete (dtime), create (crtime) |
| Date range | 2^34 bits (ext4), 2^64 bits (ZFS) |
| Date resolution | 1 s |
| Forks | No |
| Attributes | 32bitapi, acl, checksum, flock, lazystatfs, localflock, lruresize, noacl, nochecksum, noflock, nolazystatfs, nolruresize, nouser_fid2path, nouser_xattr, user_fid2path, user_xattr |
| File system permissions | POSIX, POSIX.1e ACL, SELinux |
| Transparent compression | Yes (ZFS only) |
| Transparent encryption | Yes (network, storage with ZFS 0.8+, fscrypt with Lustre 2.14.0+) |
| Data deduplication | Yes (ZFS only) |
| Copy-on-write | Yes (ZFS only) |
| Other | |
| Supported operating systems | Linux kernel |
Lustre file systems are scalable and can be part of multiple computer clusters with tens of thousands of client nodes, hundreds of petabytes (PB) of storage on hundreds of servers, and tens of terabytes per second (TB/s) of aggregate I/O throughput. This makes Lustre file systems a popular choice for businesses with large data centers, including those in industries such as meteorology, simulation, artificial intelligence and machine learning, oil and gas, life science, rich media, and finance. The I/O performance of Lustre has widespread impact on these applications and has attracted broad attention.