mtfyun/ds4
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
853 Commits 6 Branches 0 Tags
4f806387f71d3f0e8517650d5cd13b559003cdfd
Commit Graph
4 Commits
Author SHA1 Message Date
Salvatore Sanfilippo ab38bc6b7a Serve Qwen batched MTP with safe session state and output limits 
Keep exact sampled requests on ordinary batches. Clear stale verification snapshots, honor logical context limits, and reset speculative timing estimates when batch width changes. Exercise forced rejection, mixed decode, snapshot restore, disk failures and arena replacement.
 2026-09-16 15:17:25 +02:00
antirez 147109a3cc Add GLM 5.3 Flash inference 
Implement GLM 5.3 Q2 and Q4 inference across Metal and CUDA, including tensor parallelism, RDMA, batching, MTP, snapshots, SSD streaming, and GLM 5.2 shared-path improvements.

Add quantization and validation tools, official quality fixtures, model downloads, regression coverage, release QA, and user documentation.
 2026-08-28 23:26:29 +02:00
antirez 8579ec82c0 Make batched server prefill quantum configurable 2026-08-03 10:07:30 +02:00
antirez 36ee8c1407 Add CUDA tensor parallelism and session batching 2026-07-18 18:50:03 +02:00