modal-labs/Qwen3.6-35B-A3B-DFlash
Qwen3.6-35B-A3B-DFlash Paper(https://arxiv.org/abs/2602.06036) | Github(https://github.com/z-lab/dflash) | Blog(https://z-lab.ai/projects/dflash) This DFlash draft model is a joint retrain from Z-Lab(https://z-lab.ai) and Modal(https://modal.com), t...
Thành tựu nổi bật
Những danh hiệu mới nhất của dự án
#1 Repository Của Tuần
#2 Repository Của Tuần
#3 Repository Của Tuần
🚀 Qwen3.6-35B-A3B-DFlash — Tăng tốc sinh text với DFlash Speculative Decoding
DFlash dùng mô hình khối diffusion nhẹ để đề xuất đa token, target model xác minh — tăng throughput lên tới 3.61x.
Đây là mô hình draft do Z-Lab và Modal cùng huấn luyện lại, dựa trên Qwen3.6-35B-A3B với sequence length 40k và sliding-window attention. Không phải mô hình độc lập, cần dùng kèm target model trong server hỗ trợ DFlash.
✨ Điểm nổi bật
- Huấn luyện lại với 40k sequence length: tối ưu ngữ cảnh dài.
- Sliding-window attention: cải thiện hiệu năng ngữ cảnh dài.
- Block diffusion draft model: đề xuất đa token đồng thời.
- Speculative decoding: target model xác minh đề xuất, tăng throughput.
- Hỗ trợ SGLang: triển khai dễ dàng qua cờ
--speculative-algorithm DFLASH. - Tốc độ tăng lên tới 3.61x: benchmark trên B200, bf16.
- Accept length cao: trung bình 5–7 token mỗi bước sinh.
💡 Vì sao đáng chú ý
DFlash khác biệt từ MTP bằng cách dùng diffusion thay vì autoregressive để đề xuất token. Kết quả: tốc độ cao hơn MTP ở hầu hết workload (GSM8K, MATH500, HumanEval, MBPP, MT-Bench), đặc biệt ở concurrency 1. Phù hợp triển khai trên B200 với SGLang để tối ưu chi phí phục vụ mô hình lớn.
#DFlash #SpeculativeDecoding #Qwen3.6 #SGLang #TiniX
Similar Projects
Discover other similar projects and AI models.
Qwen3.6-35B-A3B-DFlash Paper(https://arxiv.org/abs/2602.06036) | Github(https://github.com/z-lab/dflash) | Blog(https://z-lab.ai/projects/dflash) This DFlash draft model is a joint retrain from Z-Lab(https://z-lab.ai) and Modal(https://modal.com), t...
Qwen3-0.6B Qwen3 Highlights Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancement...
DeepSeek-R1 Paper Link👁️ 1. Introduction We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1. DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a p...
Model Information The Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models in 8B, 70B and 405B sizes (text in/text out). The Llama 3.1 instruction tuned text onl...
GPT-2 Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in this paper(https://d4mucfpksywv.cloudfro...
Model Details Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instruction tuned models are optimized for dia...
Try gpt-oss · Guides · Model card · OpenAI blog Welcome to the gpt-oss series, OpenAI’s open-weight models(https://openai.com/open-models) designed for powerful reasoning, agentic tasks, and versatile developer use cases. We’re releasing two flavors...
Try gpt-oss · Guides · Model card · OpenAI blog Welcome to the gpt-oss series, OpenAI’s open-weight models(https://openai.com/open-models) designed for powerful reasoning, agentic tasks, and versatile developer use cases. We’re releasing two flavors...
Model Details Meta developed and released the Meta Llama 3 family of large language models (LLMs), a collection of pretrained and instruction tuned generative text models in 8 and 70B sizes. The Llama 3 instruction tuned models are optimized for dia...