Collect references from Alibaba through OpenRouter with an explicit empty system message and no provider fallback. Validate token byte alignment and reject lossy Unicode metadata for API agreement while retaining continuation likelihood scoring.
Track 100 short and 12 long reference cases, Q2/Q4 default and quality controls, continued-prefill results, and independent long-context answers. Include collector and scorer regressions and document the hosted-checkpoint and precision limitations.
Share the Metal session graph with CUDA for text and vision, MTP and exact sampling, steering, checkpoints, and disk-only BF16 n-grams. Add bounded concurrent Linux n-gram reads, vectorized dense and expert kernels, register-held recurrent prefill state, and efficient small speculative batches.
Keep Qwen out of DeepSeek batch dispatch and exclude disk n-grams from resident memory accounting. Use normal file-backed weight loading in diagnostics and link n-gram support into the CUDA quality scorer.
Include independent kernel and memory-safety tests, mixed-prefill and recovery checks, long-context server and agent coverage, and user documentation with measured DGX Spark performance.
Keep the table outside model mappings and Metal residency. Use bounded explicit reads, reject failed lookups before GPU consumption, and remove the runtime sidecar option and paging controls. Preserve Qwen hash and speculative state handling; cover malformed files and exact recovery after prefill, decode, and MTP read failures.
Add --ple to the official continuation scorer and use its existing --rendered-prompt option instead of rewriting hardcoded token IDs through an environment variable. Remove the unreferenced multi-shard mapping stub introduced by the PR.
Implement qwen4exp loading, tokenizer and chat formatting, PLE sidecars,
Metal text and vision graphs, built-in MTP, and recurrent KV persistence.
Integrate the CLI, agent, server, download targets, and conversion tools.
Preserve the native Q4_1 encoder, Q4_K accumulation guard, and shared
quantizer build target. Keep exact-MTP rewind and vision-cache row-width
fixes. Persist valid logits and recurrent state at completed prefill chunks,
and reuse Qwen text-answer history when tools clients omit reasoning.
Retain exact reasoning replay and test continued checkpoints, cancellation,
cache eviction, and restart reuse with and without reasoning_content. Include
reproducible charts and measurements comparing throughput before and after
the checkpoint fixes on M3 Ultra.
Add Qwen trunk steering and prompt activation capture, including GDN layers.
Keep the MTP predictor outside the 48-row bank and preserve prompt dumps
through one-token prefill tails and multi-row speculative verification.
Recover Chrome target creation and retain its last page target. Preserve
full agent transcripts in stripped-session format when KV no longer fits,
while retaining the newer automatic compaction and continuation logic.
Add steering/MTP/capture, Chrome CDP recovery, and full-session save/reload
regressions. Adapt the intended changes from fork PRs #13 and #7.
Reconcile with antirez main bd66c40, preserving DeepSeek v4.1 model IDs,
tool syntax, session state, and Metal kernel dispatch. Exclude superseded
engine code and prototypes.
Co-authored-by: Andreas Parusel <aparuael@7170c824-6152-4cf1-985e-9503832ce673.fritz.box>
Co-authored-by: david <contact@davidberard.fr>
Co-authored-by: Michael Kuebbeler <cropduster@users.noreply.github.com>
Co-authored-by: Ogilthorp3 <bertrand@nepveu.name>
Co-authored-by: anemll <realanemll@gmail.com>
Add two offline standard-library utilities with 30 host tests. Require complete API-covered official-score manifests, finite consistent values and matched denominators; optionally enforce per-case identity. Validate every expected ds4-bench frontier and complete float32 vector, preserving signed zero and reporting drift separately from invalid artifacts.
The existing comparator silently accepts missing rows, duplicate rows and NaN in reproduced controls; the validator rejects them. Saved Flash/Vision 100-case pairs pass unchanged. The frontier tool detects full-vector drift despite matching argmax IDs at all four measured frontiers. Document use and scope without changing inference or introducing model/GPU dependencies.
Implement GLM 5.3 Q2 and Q4 inference across Metal and CUDA, including tensor parallelism, RDMA, batching, MTP, snapshots, SSD streaming, and GLM 5.2 shared-path improvements.
Add quantization and validation tools, official quality fixtures, model downloads, regression coverage, release QA, and user documentation.
Keep the preview oracle intact, add the 100-case official 0813 fixture, and record the source, conversion, safety, quality, hash, and publication results.
Load compression ratios from the source checkpoint instead of retaining stale template metadata. Stop Q2_K/Q4_K refinement when an accepted fit collapses the search range.
Bound safetensors and GGUF length fields by the bytes remaining in the file, cross-check FP8/FP4 tensor storage against declared shapes, and reject impossible metadata and tensor counts before allocation.
Closes#542Closes#543Closes#545
Add DeepSeek V4 Pro support on top of main.
- add Flash/Pro shape selection and Pro Metal inference support
- fix shared SwiGLU clamp and indexer QAT behavior for Flash and Pro
- update eval/context tooling and imatrix quantization for Pro
- tag KV caches/sessions with model ids so Flash and Pro state cannot cross-load
- remove misc/PRO.md from tracked files while leaving it as a local scratch log
Remove SuperGPQA rows with broken upstream keys from ds4-eval instead of locally re-keying them. Replace them with cleaner self-contained SuperGPQA items, document the audited SuperGPQA slice, and regenerate the imatrix calibration prompts so benchmark-derived calibration no longer includes the broken items.
Add language/prose calibration tasks, include ds4-eval benchmark prompts without answer keys, and expand long-context prompts with code synthesis, agent transcript replay, trace diagnosis, prose fact recovery, document comparison, delayed constraints, and a small needle task. Regenerate the rendered prompt files and manifest so the corpus stays reproducible.
Render tool result bodies as raw text so file contents containing angle brackets reach the model unchanged. Escape only the tool_result closing sentinel, and keep the imatrix prompt renderer mirror in sync.
Fixes#152
Tarjei Mandt
Salvatore Sanfilippo
Donato Capitella
Luigi Colluto
user