Keep the table outside model mappings and Metal residency. Use bounded explicit reads, reject failed lookups before GPU consumption, and remove the runtime sidecar option and paging controls. Preserve Qwen hash and speculative state handling; cover malformed files and exact recovery after prefill, decode, and MTP read failures.
Implement qwen4exp loading, tokenizer and chat formatting, PLE sidecars,
Metal text and vision graphs, built-in MTP, and recurrent KV persistence.
Integrate the CLI, agent, server, download targets, and conversion tools.
Preserve the native Q4_1 encoder, Q4_K accumulation guard, and shared
quantizer build target. Keep exact-MTP rewind and vision-cache row-width
fixes. Persist valid logits and recurrent state at completed prefill chunks,
and reuse Qwen text-answer history when tools clients omit reasoning.
Retain exact reasoning replay and test continued checkpoints, cancellation,
cache eviction, and restart reuse with and without reasoning_content. Include
reproducible charts and measurements comparing throughput before and after
the checkpoint fixes on M3 Ultra.
Add Qwen trunk steering and prompt activation capture, including GDN layers.
Keep the MTP predictor outside the 48-row bank and preserve prompt dumps
through one-token prefill tails and multi-row speculative verification.
Recover Chrome target creation and retain its last page target. Preserve
full agent transcripts in stripped-session format when KV no longer fits,
while retaining the newer automatic compaction and continuation logic.
Add steering/MTP/capture, Chrome CDP recovery, and full-session save/reload
regressions. Adapt the intended changes from fork PRs #13 and #7.
Reconcile with antirez main bd66c40, preserving DeepSeek v4.1 model IDs,
tool syntax, session state, and Metal kernel dispatch. Exclude superseded
engine code and prototypes.
Co-authored-by: Andreas Parusel <aparuael@7170c824-6152-4cf1-985e-9503832ce673.fritz.box>
Co-authored-by: david <contact@davidberard.fr>
Co-authored-by: Michael Kuebbeler <cropduster@users.noreply.github.com>
Co-authored-by: Ogilthorp3 <bertrand@nepveu.name>
Co-authored-by: anemll <realanemll@gmail.com>
Implement GLM 5.3 Q2 and Q4 inference across Metal and CUDA, including tensor parallelism, RDMA, batching, MTP, snapshots, SSD streaming, and GLM 5.2 shared-path improvements.
Add quantization and validation tools, official quality fixtures, model downloads, regression coverage, release QA, and user documentation.
Resolves the residual multiple-choice false negative tracked in #321
(follow-up to #319). find_answer_letter returned the first
boundary-isolated in-range capital on the answer line, so when the model
rejects a distractor before stating its pick the rejected letter won:
"Answer: It is not B, the answer is D" -> B (should be D)
"Answer: rules out C, leaving D" -> C (should be D)
The forward scan now skips an in-range capital that is the object of an
explicit, adjacent rejection -- "not B", "isn't B", "rule(s/d) out C",
"eliminate/reject/except E". The cue set is deliberately small and
high-precision: it rewrites only clear elimination phrasing and makes no
attempt at general selection-vs-rejection sentence parsing, which is the
principled limit noted in #321. It never looks before the answer marker
or across a newline, so a leading pick is accepted before any later
rejected distractor is inspected -- "Answer: D, not B" still grades D,
the regression that blocks the naive "take the last letter" fix.
Locked by new --self-test-extractors cases (model-free, no model/GPU),
including the "D, not B" guard. All prior extractor, integer, and
regrade self-tests continue to pass.
Tested: make ds4-eval (Metal) and make cpu build clean (-Wall -Wextra);
./ds4-eval --self-test-extractors passes on both the Metal and CPU
binaries.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
find_answer_letter returned the first boundary-isolated in-range capital
after "Answer:", so on 10-choice (A-J) cases a leading English pronoun or
article was graded as the choice:
"Answer: I think it is C" -> I (should be C)
"Answer: I'll go with C." -> I (should be C)
"Answer: A careful reading ... C" -> A (should be C)
24 embedded cases are 10-choice, so this is reachable. The forward scan
now skips a standalone capital that begins a same-line word or a
contraction; the reverse scan still recovers it when it is the only
candidate ("Answer: A is correct"), and a genuine standalone answer
("Answer: I.") is unchanged.
Known limitation: a distractor explicitly rejected *before* the chosen
letter on the same line ("... rules out C, leaving D") is still misread;
that needs sentence-level parsing and is left for discussion.
Locked by new --self-test-extractors cases (model-free). All prior
self-tests and the integer/regrade cases continue to pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two model-free correctness fixes in the answer grader, locked by new
--self-test-extractors cases (no model/GPU needed):
- find_integer_answer: the answer-marker path took the first integer in
the window, so an answer line that shows its arithmetic
("Answer: m+n = 256+37 = 293") was graded as the first summand (256)
instead of the stated total (293) -> false negative. Reachable on the
many embedded AIME2025 "Find m+n / a+b+c" cases. The scan is now bound
to the answer line and, when it shows arithmetic, reads the value after
the last '='. "Final answer: 082" -> "82" and the loose fallback are
preserved.
- regrade_trace_file: every case with a MODEL_OUTPUT block was graded,
including interrupted ones (STOPPED/SKIPPED/SWITCHED/ERROR). Their
partial output inflated passed/failed and raised spurious "changed"
drift. Grading is now factored into regrade_case_outcome(), which only
grades PASSED/FAILED (and legacy status-less) traces; others are
reported via a new not_graded counter.
Tested: make (Metal) and make cpu build clean (-Wall -Wextra);
./ds4-eval --self-test-extractors passes; regrade of a 3-case trace
(1 PASSED, 2 STOPPED) reports passed=1 not_graded=2 changed=0
(was passed=2 failed=1 changed=1).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add shared help text across the CLI, server, agent, bench, and eval tools. Expand distributed-mode guidance, clean up endpoint naming, and use a TTY-only 256-color layout with clearer section titles, option arguments, separators, examples, and explanatory text.
Add DeepSeek V4 Pro support on top of main.
- add Flash/Pro shape selection and Pro Metal inference support
- fix shared SwiGLU clamp and indexer QAT behavior for Flash and Pro
- update eval/context tooling and imatrix quantization for Pro
- tag KV caches/sessions with model ids so Flash and Pro state cannot cross-load
- remove misc/PRO.md from tracked files while leaving it as a local scratch log
Use a shared DS4_DEFAULT_MIN_P of 0.05 for sampled generation. The default keeps top_p at 1.0 and uses min_p as the active filter, preventing very low relative-probability tokens from entering the sampling set while preserving plausible alternatives.
Apply the same default to the CLI, server, and ds4-eval, expose --min-p in CLI/eval, and make the server thinking-mode fixed sampler use min_p=0.05 instead of disabling min-p filtering. Explicit non-thinking API min_p values continue to be honored.
Builds checked with make -B ds4 ds4-server ds4-eval. Server unit tests pass; full make test could not run because another DS4 process was already holding /tmp/ds4.lock.
Add a q key path that exits the TUI cleanly and prints the normal plain-text report, show the pause/quit hints in the TUI title, and keep the final report compact by omitting question titles and ANSI escapes. Also clamp per-case generation to the remaining context budget so small-ctx runs stop or skip individual cases instead of aborting the whole benchmark on KV capacity errors.
Remove SuperGPQA rows with broken upstream keys from ds4-eval instead of locally re-keying them. Replace them with cleaner self-contained SuperGPQA items, document the audited SuperGPQA slice, and regenerate the imatrix calibration prompts so benchmark-derived calibration no longer includes the broken items.
antirez
rinaldofestaandClaude Opus 4.8
Salvatore Sanfilippo
user