Loading
Life sciences · Preprint
arXiv · September 9, 2026
Early or partial results. Treat as a signal, not a conclusion.
EFQ-Softmax is a proposed low-bit quantization method for attention softmax that bypasses high-precision exponential computation and directly generates E2M1 probability codes, validated on three large language and vision models. The method shows modest improvements in aggregate benchmark scores and substantial latency reduction on specialized hardware, but the work is unrefereed and lacks statistical testing or broader architectural validation.
Uncontrolled empirical benchmark evaluation with baseline comparison. Three large-scale transformer models in production use; no human subjects or clinical population.. Intervention: EFQ-Softmax: direct low-bit probability generation using exponent-only scaling and E2M1 quantization, replacing conventional exp-then-quantize softmax path.. Compared with: MXFP4 low-bit quantization (baseline) and FP16 (reference); FlashAttention architecture held constant..
Qwen3-8B seven-task mean improved from 0.6749 with MXFP4 to 0.6773 with EFQ-Softmax Qwen3-VL-8B-Instruct nine-task mean improved from 0.7826 to 0.8000 WAN2.2-TI2V-5B maintained temporal consistency and visual quality comparable to FP16 and MXFP4 baselines
Safety was not reported in the material analysed. Check the source before drawing any conclusion about harm.
The source did not state who this applies to in practice.
Early-stage technical contribution demonstrating a novel quantization method with end-to-end validation on specific models and kernel-level performance gains, but lacking peer review, comparative statistical rigor, and broad generalization evidence.
As stated by the source record.
Quoted from the source exactly as published.
Graded across the dimensions that decide whether you should act, each from what the source actually supports. There is no single score, and where a dimension was not assessed it says so.
Low-bit attention accelerates Transformer inference by moving the $QK^\top$ and $PV$ matrix multiplications to FP8 or FP4 matrix engines. However, the softmax path often evaluates shifted-score exponentials in higher precision, forms a temporary probability block, and quantizes it before low-bit $PV$ multiplication. This exp-then-quantize path creates a mismatch between a high-precision probability producer and a low-bit matrix consumer. We propose EFQ-Softmax (Exp-Free Quantization for Softmax), a low-bit probability-generation method that directly maps shifted attention scores to block-scaled E2M1 operands. For each microscaling block, EFQ-Softmax selects an exponent-only scale from the local maximum, maps the shifted scores to a normalized residual domain, and generates nonnegative E2M1 probability codes using a single affine rule. The resulting operand is used consistently in both the $\widetilde{P}V$ numerator update and the $\widetilde{P}\mathbf{1}$ denominator update. The FlashAttention-style row-maximum update, historical rescaling, high-precision accumulation, and final normalization remain unchanged. We evaluate end-to-end quality on Qwen3-8B, Qwen3-VL-8B-Instruct, and WAN2.2-TI2V-5B, and separately measure kernel-level performance on the A5 vector unit. EFQ-Softmax improves the Qwen3-8B seven-task mean from 0.6749 with MXFP4 to 0.6773 and the Qwen3-VL nine-task mean from 0.7826 to 0.8000. On WAN2.2, it maintains temporal consistency and visual quality comparable to the FP16 and MXFP4 baselines under VBench. On the A5 vector unit, EFQ-Softmax reduces the vector-stage latency of the fused probability-generation kernel by 40.33% on average across sequence lengths from 16K to 128K. These results show that direct low-bit probability generation can replace the conventional exp-then-quantize path while preserving end-to-end model quality.
Taken from the source record, never inferred. Follow any of these and new work involving them reaches your briefing.