Overview of Breaking your AI storage bottlenecks
In this Stack Overflow Podcast episode, host Ben talks with MinIO co-founders Garima Kapoor and Anand Babu Periasamy about why AI systems are increasingly limited by storage and data movement rather than raw GPU compute. The conversation centers on NVIDIA’s new storage reference architecture, STX, and how MinIO is adapting its object store to feed GPUs faster using DPUs, ARM-based systems, PCIe Gen 6, 800 Gb networking, and RDMA-based data paths.
The big idea: as AI workloads grow, the infrastructure problem shifts from “How do we compute more?” to “How do we keep GPUs continuously supplied with data, memory, checkpoints, and context?”
Why AI storage is becoming a bottleneck
GPUs are starving for data
- Modern GPUs are fast enough that storage and networking can’t always keep up.
- This creates a bottleneck where expensive compute sits idle waiting for data.
- The issue affects training, inference, checkpoints, context memory, and other AI pipeline stages.
Traditional storage hardware hits physical limits
- Commodity x86 storage servers were designed for general-purpose workloads, not 800 Gb AI pipelines.
- Typical limits include:
- insufficient PCIe lanes
- limited CPU-to-memory bandwidth
- network saturation before disks are fully utilized
- Even with dual 400 Gb NICs, older architectures can still bottleneck on memory and PCIe sharing.
NVIDIA STX and the new storage architecture
What STX is
- NVIDIA’s STX is a reference architecture for AI storage, analogous to:
- DGX for compute
- STX for storage/data
- It is built around a specialized DPU-based system designed to feed GPUs at very high speed.
What makes it different
- Uses an ARM-based Vera CPU
- Includes:
- 88 cores
- HBM-style high-bandwidth memory
- PCIe Gen 6
- 800 Gb networking
- The goal is to remove the usual storage bottlenecks and provide a purpose-built path for AI data movement.
Why MinIO was an early partner
- MinIO was already designed around:
- software-defined storage
- ARM optimization
- simplicity at scale
- Because of that, it could adapt more quickly to NVIDIA’s new architecture than legacy appliance-based vendors.
How MinIO fits into the AI data stack
Object storage is the foundation
- The discussion emphasizes that modern cloud and AI infrastructure are built on object storage, not legacy file/NAS/block systems.
- MinIO is positioning object storage as the standard foundation for:
- unstructured data
- structured data
- tables
- AI memory and context data
S3, tables, and formats
- S3 is presented as the modern object store model.
- Parquet is described as a file format used to store structured/tabular data inside object storage.
- Iceberg and similar open table formats sit on top of object storage.
- MinIO supports both objects and tables in the same system.
Performance gains and technical advantages
Major throughput improvements
- With S3 over RDMA on STX, MinIO reports up to 5x read performance gains versus non-RDMA deployments.
- The architecture is meant to saturate very fast networks rather than becoming the limiting factor.
Low-latency memory offload
- MinIO also supports KV cache offload to provide sub-millisecond latency to GPUs.
- This is especially important as AI systems become more memory-intensive.
Hardware acceleration matters
- The team highlighted how MinIO has long taken advantage of:
- Intel AVX/AVX2/AVX-512
- ARM NEON/SVE/SVE2
- Power VSX
- The point: AI storage benefits from SIMD/vector acceleration just like compute does.
Power and density are now first-class concerns
Power is the new currency
- The guests emphasized that AI infrastructure must be both:
- fast
- power-efficient
- Simply slowing things down to save power is not acceptable in AI.
Higher density reduces power waste
- Better throughput means fewer nodes are needed for the same workload.
- They gave an example of shrinking a 1,000-node cluster down to 128 nodes through architecture efficiency.
- Fewer nodes means less idle power and better overall efficiency.
Commodity hardware still matters
- Their recommendation is to build on open standards and commodity hardware whenever possible.
- Appliance-style, closed systems make it hard to adopt new accelerators and optimize for future workloads.
The future: memory becomes the next storage frontier
“G3.5 memory” and AI context
- A major forward-looking theme was NVIDIA’s concept of G3.5 memory:
- not quite GPU memory
- not quite persistent storage
- something in between: memory that behaves like storage
- This is aimed at inference context memory, where AI systems need huge, fast-access state without requiring full enterprise durability.
Why this matters
- AI systems repeatedly recompute context, which increases cost and latency.
- A memory-like storage layer can:
- reduce recomputation
- lower token usage and inference cost
- maintain long-lived context for AI agents and robots
MinIO’s response
- MinIO introduced AI Store Memory Edition to support this emerging use case.
- The system is designed to store huge amounts of “memory” on NVMe-backed, commodity infrastructure with high performance.
Key takeaways
- AI bottlenecks are shifting from compute to data movement and memory access.
- NVIDIA’s STX is a new storage reference architecture designed specifically for AI-era throughput.
- MinIO is positioned as the object storage layer that can feed GPUs quickly over modern interconnects like RDMA.
- ARM, DPUs, PCIe Gen 6, and 800 Gb networking are central to the new architecture.
- The next frontier is AI memory: persistent, large-scale, low-latency context storage for inference and agents.
Where to learn more
- Visit min.io
- Look for the section covering NVIDIA STX reference architecture
- Contact the team at hello@min.io