Local inference: performance and power on an RTX 3090
Tests of eight local LLMs on one RTX 3090 compared throughput and energy efficiency at power limits ranging from 100 W to 450 W.
Source: Testes de LLM na RTX 3090 comparam throughput e eficiência energética (x.com). Text prepared with AI from this source.
What happened and what to do
Published on May 8, 2026, the post reports tests of eight local LLMs on a single RTX 3090, using power limits from 100 W to 450 W. Average throughput was 90.4 tokens per second at 225 W and 107.1 at 450 W; efficiency fell from 0.4167 to 0.2731 tokens per second per watt. The results show the trade-off between speed and energy efficiency in local inference.
A company running models on its own GPUs can measure this trade-off using workloads representative of its usage, tracking throughput, power consumption, and response quality at different power limits. A monitoring pipeline and comparison dashboard can help select configurations based on latency, capacity, and energy targets rather than defaulting to maximum power.
How the consultancy can help
Wendelmaques can assess the inference operation and its objectives, define a measurement scope, and implement the monitoring pipeline and dashboard. It can also support operation and periodic configuration reviews.
Next step
Send a short description of your inference environment and what you want to optimize to receive a scoped proposal.
Let's assess your project
I reply with scope and a proposal.