Inco AI’s DFlash 2 asks whether a model can produce answers faster without replacing the model that decides which answer is valid. Released on August 18, the method improves the small drafting system used in speculative decoding. Inco reports 16–25% gains across its benchmarks over the preceding approach and 2.7–3.4 times autoregressive throughput for its Qwen3.8-27B configuration at batch size one. Those conditions define the claim. The wider opportunity is to make existing model capability less expensive to deliver. Inco’s technical announcement describes the method and tests.
Draft quickly, verify carefully
Speculative decoding separates proposing tokens from accepting them. A smaller system drafts several candidates, and the target model checks them. The original DFlash paper applies a lightweight diffusion model to draft a block in parallel. DFlash 2 adds a mechanism to select more coherent candidate sequences and improve predictions toward the end of a block. The target still verifies the draft. This is different from simply asking a smaller model to supply the final answer.
The conceptual advantage is that useful speed can come from spending cheap computation to avoid repeated expensive work. But the cheap work must actually help. If proposed tokens are rejected frequently, the system has added drafting overhead without recovering enough target-model time. An effective drafter is therefore defined by its relationship to a target and workload, not by its standalone score on a general benchmark.
The business gain is conditional on the traffic
A low-concurrency result can be highly relevant to an individual using a local model or to a latency-sensitive request. It is not automatically a measure of how many customers a busy hosted service can support. At higher load, the target model may already use hardware differently, and allocating resources to a drafter changes the balance. Providers need to publish both the fast individual experience and the capacity available when many requests arrive together.
An illustrative buying decision makes the distinction clear. A developer might accept fewer total concurrent users to shorten an interactive coding response. A batch-processing team might prefer maximum accepted tokens over a longer window. Both can use the same model and reach different conclusions about the same speculative configuration. The relevant metric is the service objective, not the most attractive multiplier in the launch post.
Preserving the target is not the same as proving correctness
Inco describes a verification and rejection-sampling path intended to preserve the target distribution. That is a claim about the decoding method. It does not mean the target model always gives a correct answer, or that every implementation combination will reproduce the same numerical behavior. A deployment still needs to validate the target, runtime, precision, sampling settings, and supported features together.
This distinction matters when communicating a speed improvement to customers. Faster delivery of the same model behavior is valuable because it avoids making an explicit capability trade for every request. It does not erase the need to check the output. A trustworthy comparison should retain the same task evaluation while changing only the intended acceleration mechanism.
The next competition is over useful verification work
Our reading is that speculative decoding creates another dimension of competition between inference providers. They can compete on how much useful output they obtain from each expensive target pass. Better candidate selection, efficient verification, and matching the drafter to the traffic may allow one provider to offer a more responsive service from otherwise similar model weights.
The improvement also changes where optimization effort belongs. Simply training a larger drafter may consume the saving it was supposed to create. A small mechanism that improves acceptance at low overhead can be more commercially useful than a generally stronger assistant. That makes the drafter–target pair the unit of engineering. The model name alone no longer describes the performance customers receive.
The evidence to watch
Look for repeated measurements at different concurrency levels, including memory overhead, acceptance rates, and exact runtime versions. The strongest result will preserve evaluated answer quality while reducing full-request latency or cost under representative demand.
Questions, news tips or corrections? Contact [email protected].
