What Is AI Storage? How TuringData Platform and TuringData Flash Support AI Workloads

← Back to blog

AI performance depends on how efficiently data reaches the processors doing the work. Powerful GPUs can accelerate model development, but when data arrives too slowly, those GPUs spend valuable time waiting.

As datasets grow and more teams share AI infrastructure, storage becomes a critical part of the performance equation. NVIDIA identifies slow storage I/O as a reason fast GPUs can become starved of data.

TuringData addresses this challenge through two connected offerings: TuringData Platform, its storage software, and TuringData Flash, an integrated all-flash appliance powered by that software. Together, they provide a data foundation for workloads across the AI lifecycle.

What is AI storage?

AI storage is the data infrastructure that supplies model training, retrieval and inference. It can include scale-out file and object platforms, parallel file systems, caches and data orchestration software, delivered through software, appliances or managed services.

Its role extends beyond holding datasets. Storage must deliver data at the speed, scale and consistency required by the applications using it. As GigaOm’s analysis explains, evaluating AI storage increasingly involves inference integration, checkpointing, data preparation and operational controls alongside throughput.

Why do AI workloads need specialized storage?

An AI pipeline generates several kinds of storage activity. Data preparation reads source material and writes transformed datasets. Training repeatedly retrieves samples. Checkpointing creates bursts of writes as training progress is saved. Inference introduces requirements around model loading and application data access.

Supporting these activities requires a balance of:

  • Throughput: Moving large volumes of data quickly.
  • Low latency: Completing individual requests with minimal delay.
  • Concurrency: Serving many workers and jobs simultaneously.
  • Metadata performance: Finding and opening large numbers of files efficiently.
  • Scalability: Accommodating growing datasets and compute resources.

Different workloads emphasize different capabilities. MLCommons describes 3D U-Net as a bandwidth-intensive workload involving concurrent large-file reads, while RetinaNet places greater pressure on small-file access and metadata operations. A storage system therefore needs to be evaluated against the application’s access pattern.

TuringData Platform: a flexible software foundation

TuringData Platform is a software-defined storage platform supporting deployment across bare-metal, on-premises, cloud and hybrid environments.

It combines high-throughput I/O, distributed metadata, file and object storage integration, and high-speed RDMA networking. Its Elastic Data Network supports multiple network planes within one storage cluster, enabling training and inference environments to share a common storage foundation.

For organizations choosing their own infrastructure, the platform provides deployment flexibility alongside shared data services. Directory-based quality-of-service controls and quotas help administrators allocate performance and capacity across teams.

TuringData Flash: an integrated appliance with MLPerf Storage v3.0 results

TuringData Flash packages TuringData Platform software with NVMe flash storage and high-speed InfiniBand, RoCE or Ethernet networking. Organizations can start with three storage nodes and expand as requirements grow.

Flash also supports NVIDIA GPUDirect Storage, providing a path for compatible applications to transfer data between storage and GPU memory. This makes it an integrated deployment option for organizations building infrastructure for training, checkpointing and inference.

Demonstrated performance in MLPerf Storage v3.0

MLPerf® Storage v3.0 evaluates storage against representative AI workload demands. As Blocks & Files reports, the benchmark includes training, checkpointing, vector database and KV-cache workloads, with version 3.0 adding S3 object storage support and broader inference testing.

StorageReview’s coverage of the published results reports the following performance from a cluster of three TuringData Flash F9200 storage nodes:

Benchmark workloadReported result
3D U-Net training541.5 GiB/s read bandwidth, supporting 96 simulated B200 accelerators
Checkpoint-70B read539.9 GiB/s
Checkpoint-70B write307.1 GiB/s

Both workloads used the same three-node storage cluster.

What the results mean for AI workloads

The 3D U-Net result demonstrates sustained data delivery under concurrent, bandwidth-intensive training reads. Supporting 96 simulated accelerators shows how much demand the submitted configuration could serve while meeting the benchmark’s utilization requirement.

MLPerf Storage exercises real storage I/O while emulating accelerator computation. For 3D U-Net, valid results must meet a 90% simulated accelerator-utilization threshold. This provides a defined measure of whether storage can keep pace with the benchmark’s training demand.

The Checkpoint-70B results address saving and restoring training state. Write bandwidth matters when persisting checkpoints; read bandwidth matters when retrieving that state for recovery. Faster checkpoint I/O can shorten the storage portion of these operations and reduce interruptions to productive compute time.

Together, the results demonstrate high throughput for training-data reads and checkpoint traffic from a three-node storage deployment. Actual application performance depends on the dataset, network, compute environment and software integration.

How TuringData supports the AI lifecycle

Data preparation: accessing large collections of files

Before training begins, teams collect, clean and transform data. An image pipeline, for example, may need to locate and open millions of individual files.

File-access overhead can therefore matter as much as raw transfer speed. TuringData Flash uses distributed metadata and parallel I/O processing to address workloads involving large file collections, helping preparation processes access the material they need.

Training and fine-tuning: supplying data consistently

Training requires a steady flow of data as workers progress through a dataset. Adding GPUs increases the pressure on the storage and network infrastructure feeding them.

For compatible applications and configurations, GPUDirect Storage can streamline transfers by avoiding an extra copy through a buffer in CPU memory. This can reduce CPU overhead and improve data-movement efficiency.

Checkpointing: preserving progress efficiently

Checkpoints allow training to resume after an interruption. Large distributed jobs can generate substantial write traffic when saving this state.

The Flash Checkpoint-70B results provide a concrete reference for this requirement. End-to-end checkpoint time also depends on the training framework and checkpointing approach. Asynchronous checkpointing, for example, can move writes off the critical training path.

Inference: supporting model and application data access

Once a model is deployed, storage remains relevant to loading model artifacts and supplying application data. Some serving architectures also use storage to offload and retrieve KV-cache state.

The benefit depends on the serving software, model size, context length and concurrency. GigaOm recommends testing inference performance against the organization’s own serving stack and a clearly defined baseline.

Managing growth and storage costs

AI environments accumulate raw datasets, prepared data, checkpoints and model versions. These assets do not all require the same performance tier.

TuringData Platform’s Smart Tiering keeps hot data on NVMe and automatically offloads cold data to object storage. This helps align data placement with access patterns as storage requirements grow.

Choosing storage for your AI environment

Start with representative datasets, file sizes, concurrent jobs and checkpoint activity. Measure data-loading time, checkpoint duration and GPU utilization alongside storage throughput. Test read and write performance at more than one configuration size, and verify that a checkpoint can be saved and restored at your model’s scale.

Explore TuringData Platform and TuringData Flash to build a storage foundation suited to your AI workloads—from flexible software deployment to an integrated appliance with demonstrated MLPerf Storage v3.0 performance.

Find out how much GPU utilization you're leaving on the table. Schedule a free 30-minute architecture review with our experts.