KohakuFA proposes an alternative for backward-pass precision
On October 3, 2026, a post presents the KohakuFA kernel as an alternative amid claims of precision issues in backward passes involving cuDNN, FA4, and FlexAttention.
Source: KohakuFA mira problemas de precisão no backward (github.com). Text prepared with AI from this source.
What happened and what to do
The post dated October 3, 2026, claims that cuDNN, FA4, and FlexAttention have precision issues in backward passes, and presents KohakuFA as an alternative kernel. The project is available in a GitHub repository.
To assess the impact on a company, its team can compare the kernel with its current implementation on representative training workloads, check gradient precision, and track stability, performance, and divergence across runs. Adoption can depend on validation and monitoring criteria tailored to the operation.
How the consultancy can help
Wendelmaques can diagnose risks in your training operation, define precision and performance tests, and propose a scoped implementation with appropriate instrumentation and monitoring. It can also support operations according to the agreed scope.
Next step
Send a short description of your training workload and the risks you want to assess to receive a scoped proposal.
Consulting for your project
AssessmentDeploymentOngoing operations
Quoted per project
Email