NVIDIA details Python path to GPU memory reads with GPUDirect Storage

NVIDIA published a technical guide on October 22, 2026 explaining how Python applications can read data directly into GPU memory with Magnum IO GPUDirect Storage, and how to verify whether a read used the accelerated path or a CPU-staged fallback. The post says the distinction matters because KvikIO and cuFile make separate routing decisions before a request reaches storage.

How the path is selected

According to NVIDIA, GPUDirect Storage creates a direct data path between supported storage and GPU memory, avoiding unnecessary staging through CPU memory and reducing overall CPU utilization. In the Python stack described in the post, KvikIO sits above cuFile: KvikIO decides whether to call cuFile at all, while cuFile decides whether the request uses the accelerated nvidia-fs path or a compatibility path that stages data through host memory.

The company says reaching the cuFile API alone does not prove the accelerated path was used. That distinction is central to the tutorial, which shows how to inspect KvikIO defaults, confirm GDS components with gdscheck.py, and then use gds_stats to determine whether operations were counted under nvfs or posix.

What the tutorial used

NVIDIA says the example was tested on an x86_64 system with an H100 GPU, CUDA 13.3 and cuFile 1.18. The setup also used a local NVMe device formatted with ext4, a PIX connection between the GPU and the NVMe device, IOMMU and ACS disabled, and the nvidia-fs kernel module loaded to provide the accelerated path.

In the Python example, the guide creates a 1 GiB file, allocates a CuPy array in GPU memory, and reads the file into that buffer with KvikIO. The post says KvikIO reported compat_mode as AUTO and a gds_threshold of 16 KiB on the tutorial system. AUTO tells KvikIO to try cuFile first and fall back to POSIX if cuFile is unavailable, while reads below the threshold use the POSIX path directly.

Verification with cuFile statistics

The article says one way to confirm the actual transfer path is to enable CUFILE_STATS_LEVEL=3 and inspect the running process with gds_stats. In the tutorial example, KvikIO's default 4 MiB task size split the 1 GiB request into 256 reads, and the statistics showed n=256, nvfs=256 and posix=0, indicating that all operations used the nvidia-fs path.

The post also compares the layers by changing settings. With KVIKIO_COMPAT_MODE=off, KvikIO still uses cuFile, and cuFile can be forced into its compatibility mode with CUFILE_FORCE_COMPAT_MODE=true, which moves operations under posix. With KVIKIO_COMPAT_MODE=on, KvikIO bypasses cuFile entirely and performs the read through its own CPU-staged POSIX backend.

cuFile scheduling and request sizing

NVIDIA says cuFile has its own execution settings in cufile.json that affect how requests are broken up and scheduled, including execution.parallel_io, execution.max_io_threads, execution.min_io_threshold_size_kb, execution.max_request_parallelism and execution.max_io_queue_depth. In the example configuration, four cuFile worker threads, max_request_parallelism set to four, and min_io_threshold_size_kb set to 8192 meant a 1 GiB read could be subdivided further once cuFile received it.

The guide notes that if KVIKIO_TASK_SIZE is set to 1 GiB, KvikIO sends one large request to cuFile, which then uses its unregistered buffer scheduling path because the destination buffer is not explicitly registered. The post says that in this case cuFile limits the request to four pieces, each 256 MiB, and that those pieces are scheduled work units rather than individual storage transfers. It also says the 8 MiB scheduling target would matter for a 16 MiB request, which cuFile would split into two 8 MiB pieces.

Why NVIDIA is emphasizing the stack

The article places the tutorial in the context of larger AI data movement demands, citing the 2025 Stanford AI Index as saying compute for notable AI models had doubled about every five months and training datasets for large language models had doubled about every eight months. NVIDIA also says Python was the main application interface for many AI projects, while cuFile remains a C API, making KvikIO a bridge between Python code and the storage-acceleration layer.

According to the post, KvikIO is part of NVIDIA CUDA-X for data science and has roots in earlier cuDF work on accelerated Parquet and ORC ingestion. The company says the same storage path has since been used in cuCIM digital pathology workflows, cuPhoton xDataReader and GPU-accelerated Presto work, but the tutorial itself focuses on how to confirm whether a Python read actually took the accelerated route.

The post does not assign performance gains to every configuration. Instead, it stresses that direct-path operation depends on compatible hardware, drivers, filesystems, topology and settings, and that unsupported configurations can fall back to compatibility mode. NVIDIA says users should verify their own workloads with gds_stats rather than assume a speedup.