Teams running GPU-based LLM or agentic inference at scale, especially on NVIDIA hardware, who are seeing KV cache evicted and recomputed on long-context or multi-turn sessions and want to test a shared, RDMA-based cache tier before adding more GPUs.
.svg)
Long and agentic workloads create a "prefill tax." When GPU memory can't hold the KV cache long enough, previously computed context gets evicted, so the next turn has to recompute it from scratch instead of continuing where it left off.
MemKV adds a new, shared memory tier between local GPU memory and network storage, using RDMA so any inference server on the cluster can restore KV cache into GPU memory, not just the node that originally created it.
MemKV is built KV-cache-native and share-nothing, running on standard NVMe and Ethernet infrastructure, deployable as bare metal or via Kubernetes, and designed to integrate with existing inference server plugins without requiring a new stack.
.svg)
