In this article, we look at several digital storage-related product introductions at Supercomputing Conference 2023. Digital storage and memory are essential components of high-performance computing, including AI. In particular, introductions by DDN, Arcitecta and WDC for software-defined scalable storage and high-performance storage applications.
DDN announced its Infinia software-defined storage (SDS) platform for AI and cloud applications. The figure below shows the attributes of this SDS system.
The primary data plane is based on flash for all storage, not just primary storage, including the use of QLC flash for higher latency storage, scaling up to 100 PB of storage. Writes are batched in cache to reduce wear on QLC flash. It is intended to integrate with existing client orchestration tools, is hardware agnostic, and supports multi-tenancy. The Infinia is intended to support applications that don’t need the performance and scalability of the company’s ESAScaler storage systems.
The Infinia system is currently object-based, but in the longer term it is intended to support containers and files as well as objects, with each service having its own container. DDN will feature 1U AMD-based servers as reference architectures with 12 hot-swappable 2.5″ NVMe U.2 drive bays and up to four 200G network ports, with a minimum of 6 or these used to create a cluster acting as an S3 object.store.
By 2024, DDN says it will have a parallel file system designed for AI and the cloud and support advanced databases and data movement.
Australian data management software company Arcitecta has announced its MediaFlux universal data system, which combines data management, data orchestration, multi-protocol access and storage in a single platform. It can manage data on-premises and in the cloud and includes globally distributed access for hundreds of billions of files. The company says it can do this at a much lower cost than existing cluster file systems.
Arcitecta was built on top of third-party file and object storage systems from other vendors, as shown below. One example is its use with NetApp E-Series block storage, although their largest market is Isilon storage systems.
Clustered file system storage costs between $100 and $1,000 per TB, but Arcitecta licenses are per user rather than per capacity. They quoted me a price as low as $100 per user, regardless of storage capacity used. There may still be maintenance fees.
Western Digital was showcasing the company’s OpenFlex Data24 3200 NVMe-oF (NVMe over Fabric) storage platform. This system can create a storage pool that can be shared by up to six hosts without a switch and supports Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE). It is available in a 2U, 24-bay platform with storage capacities up to 368TB using the company’s Ultrastar DC SN655 dual-port PCIe Gen 4.0 SSDs.
WDC also introduced its Ultrastar Data102 and Data60 JBOD hybrid storage platforms to support disaggregated storage and software-defined storage (SDS). These are dual-port SAS or single-port SATA configurations. The Data102 has storage capacities of up to 2.65 PB and the Data60 up to 1.56 TB in a 4U enclosure that includes IsoVibe and ArticFlow technologies for improved performance and reliability. Data102 and Data60 capacity figures assume the use of 26TB SMR hard drives.
WDC also demonstrated a proof of concept of GPUDirect storage combining the company’s RaidFlex technology with Ingrasys ES2100 with integrated NVIDIA Spectrum Ethernet switches as well as NVIDIA GPUs, Magnum IO GPUDirect storage, BlueField DPUs and ConnectX SmartNICs. The proof-of-concept demonstration can provide 25 GB/s bandwidth for a single NVIDIA A100 Tensor Core GPU and over 100 GB/s for four NVIDIA A100 GPUs.
At SC23, Arcitecta and DDN demonstrated software-defined storage solutions for AI and cloud applications. WDC also showed off SDS, its OpenFlex NVMe storage, and its GPUDirect storage.