Serving MoE models with AWS EFA
On November 6, 2025, a Perplexity article described kernels for serving MoE models across AWS GPUs using EFA, without GPUDirect Async.
Source: Perplexity cria kernels de inferência MoE para AWS EFA (x.com). Text prepared with AI from this source.
What happened and what to do
On November 6, 2025, a Perplexity article presented expert-parallelism kernels for serving large MoE models across multiple AWS GPUs using EFA. The approach groups GPU dispatch and combine operations into RDMA writes and uses a host-side proxy thread to coordinate transfers alongside grouped GEMM, in a setting where EFA does not offer GPUDirect Async.
For a company running MoE models across multiple nodes, a practical response is to assess the network, GPU communication patterns, and current serving stack. The company could instrument dispatch, combine, and GEMM, monitor latency and utilization, and, if needed, implement and test kernels and a serving layer suited to its infrastructure.
How the consultancy can help
Wendelmaques can diagnose the serving architecture and communication bottlenecks, define an implementation scope, and support operation of instrumentation, monitoring, and kernels on the client's infrastructure.
Next step
Send a short description of your infrastructure and serving challenge to receive a scoped proposal.
Let's assess your project
I reply with scope and a proposal.