Serving models across GPU, RAM and NVMe
A September 12, 2026 post reports running a model on an RTX 5090 with 125.7 GiB of RAM and part of the model file on NVMe.
Source: x.com (x.com). Text prepared with AI from this source.
What happened and what to do
On September 12, 2026, the post described running a model on an RTX 5090 with 125.7 GiB of RAM, while part of a 502 GB GGUF file was stored on NVMe. The author reported 5.12 tokens per second for new content and up to 21.27 when the data was resident.
For a company assessing model serving on constrained hardware, the case points to measuring latency and throughput under different data-residency conditions and distributions across GPU, RAM and NVMe. A monitoring pipeline and dashboard can track performance, resource use and operating costs, providing a basis for tuning the configuration.
How the consultancy can help
Wendelmaques can assess the workload, infrastructure limits and performance requirements; define a serving architecture and metrics; and deliver a monitored implementation with operations scoped to the case.
Next step
Send a short description of the model, infrastructure and expected workload to receive a scoped proposal.
Let's assess your project
I reply with scope and a proposal.