Compare commits

..
Author SHA1 Message Date
alvinfeiandClaude Opus 5.5 d0a4fd60f2 fix(memhub): pass every UI language to the SDK, zh-Hans included
The owner's decision for #243: a zh-Hans UI now sets
MemoryConfig.language="zh-Hans" like every other UI language, so App users
never fall back to the SDK's zh/en auto-detect. Only an unset language
(CLI/API-only installs, or 'settings language system') keeps auto-detect.
status --json pipeline.sdk_language shows zh-Hans for zh-Hans users.

Desktop now syncs the language the GUI shows (snapshot.locale), not the raw
preference: an App that follows the system sends the resolved locale
(e.g. zh-Hans on a Chinese macOS) instead of null.

Refs #243 (MemHub passes the language explicitly; the SDK's System 2
auto-detect is left unchanged for direct SDK users)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-02 12:35:25 +08:00
alvinfeiandClaude Opus 5.5 aae877b880 fix(memhub): accept only a full 40-character commit at the packaged-runtime boundary
resolvePackagedCommit accepted 7-40 hex characters in any case, while every
downstream check (packaged-runtime install, local install, release download,
release scripts, after-pack) requires 40 lowercase hex characters. It now
accepts exactly 40 hex characters, case-insensitive, returns them lowercased,
and skips a short or malformed value so a bad ONE_MEMORY_COMMIT no longer
shadows the embedded commit. The tsup build fails on a malformed supplied
ONE_MEMORY_COMMIT instead of baking it.

Fixes #236, Fixes #235

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-02 12:35:14 +08:00
feitianxiang 670c7c6cb1 Merge pull request #267 from Tencent-Hunyuan/docs/release-1.0.6
docs: prepare MemHub v1.0.6 (news, changelog)
2026-11-02 11:50:58 +08:00
alvinfeiandClaude Opus 5.5 aa633623da docs: MemHub 1.0.6 news and changelog
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-02 11:46:06 +08:00
feitianxiang 37d8662b93 Merge pull request #266 from Tencent-Hunyuan/chore/build-37
chore(release): MemHub 1.0.6, build 37
2026-11-02 11:44:12 +08:00
alvinfeiandClaude Opus 5.5 9459a1107b chore(release): MemHub 1.0.6, build 37
Sets every MemHub package, the Core project and its lock entry to 1.0.6
with set-release-version.mjs, and the build number to 37. The release
facts in memhub/README.md and docs/INSTALLATION.md now name v1.0.6
(build 37).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-02 11:36:55 +08:00
feitianxiang 2c6f269170 Merge pull request #264 from Tencent-Hunyuan/chore/pin-hy-memory-2.0.11
chore(memhub): pin hy-memory 2.0.11 and set the SDK language directly
2026-11-02 11:34:45 +08:00
alvinfeiandClaude Opus 5.5 7a2ff2f4c4 build(memhub): pin hy-memory 2.0.11 and set the SDK language directly
Core pins and locks the published hy-memory 2.0.11 from PyPI, which adds
MemoryConfig.language and the prompt registry. apply_sdk_language now sets
the language directly; the guard that skipped it on an SDK without the
field (and its debug log) is gone, and the tests use the real field. The
version constant, README, architecture notes, SBOM, notices and their tests
follow. CHANGELOG records 2.0.11 as published.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 22:22:32 +08:00
feitianxiang 35128f2c84 Merge pull request #262 from Tencent-Hunyuan/feat/memhub-eleven-languages
feat(memhub): 11 display languages; non-zh/en choices set the SDK prompt language
2026-11-01 22:17:21 +08:00
feitianxiang f12a95ce7a Merge pull request #263 from Tencent-Hunyuan/release/hy-memory-2.0.11
release: hy-memory 2.0.11
2026-11-01 22:13:44 +08:00
alvinfeiandClaude Opus 5.5 2b75adc718 release: hy-memory 2.0.11
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 21:55:18 +08:00
feitianxiang 75ac1b01f2 Merge pull request #261 from Tencent-Hunyuan/feat/sdk-multilingual-prompts
feat(sdk): prompts in 11 languages and an explicit language option
2026-11-01 21:53:38 +08:00
alvinfeiandClaude Opus 5.5 e786657a0b feat(memhub): 11 display languages; non-zh/en choices set the SDK prompt language
UI (packages/i18n, web, desktop):
- Locales: en, zh-Hans, zh-Hant, ja, es, de, pt, fr, ko, ru, id. zh-CN is
  now zh-Hans; a stored zh-CN preference (Desktop gui-language.json,
  browser localStorage) is read as zh-Hans, so existing users keep it.
- One complete catalog per new locale (packages/i18n/src/locales), every
  key with the same placeholders; product names stay untranslated.
- The language selector lists all 11 by their own names.
- The system language resolves to the new locales (zh-TW/HK/MO -> zh-Hant).
- The migration failure dialog and its GitHub issue prefill follow the
  display language (new "migration.*" catalog).
- Completeness test: every locale has every key, matching placeholders,
  no empty string; a missing key falls back to en.
- The language button keeps long names on one line (CSS ellipsis).

SDK prompt language (Core):
- config.json `ui_language` (written only once set), GET/PUT
  /api/v1/settings/ui-language. The Desktop main process sends the
  choice at launch and on every change; the browser UI on a change.
- zh-Hans or no choice ("system") passes no language: the SDK keeps its
  zh/en detection. Every other code, en included, is set as
  MemoryConfig.language in build_sdk_config, only when the pinned SDK has
  the field (else logged once at debug level).
- The language joins the read and write handle keys, so a change rebuilds
  clients with the #218 use-count retirement; collection names unchanged.
- Status: pipeline.ui_language and pipeline.sdk_language (auto or code).
- CLI: memhub settings language [<code>|system].

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 21:43:20 +08:00
feitianxiang b2c91d9751 Merge pull request #260 from Tencent-Hunyuan/feat/memhub-embedding-switch
feat(memhub): switch the embedding model safely — space registry, paused re-embed, startup merge of stray spaces
2026-11-01 21:34:53 +08:00
alvinfeiandClaude Opus 5.5 7df371bb10 fix(memhub): make the embedding switch work on real graphs and SDK services
Reviewing the switch against the real SDK, Kuzu and EmbedService found what
the in-memory backend could not:

- Verifying the graph ran beside live clients, and the SDK refuses a graph
  one holds open, so an ultra switch failed whenever a read landed after the
  copy. Clients are closed for the verify too.
- The graph was only re-embedded in ultra, so a pro home's graph from its
  ultra days went with the old space. It moves in every mode now.
- The re-embed built a full client config, which needs the generation key;
  an Ollama embedding has none resolved. It uses the retrieval view.
- The scan opened Chroma outside the first-open lock, the race that once lost
  a clean install's first write. It takes the client-construction lock.
- A live space whose profile was deleted raised KeyError on every read. It
  now answers `embedding_space_profile_missing`.

Also tests the legacy-collection blocker and the real startup migration on a
home mid-switch.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 21:19:01 +08:00
alvinfeiandClaude Opus 5.5 f38c370fbd feat(memhub-cli): embedding switch commands and the App contract
`memhub models activate` asks Core for the switch plan, shows it, and needs
`--yes` (a prompt on a terminal, exit 2 otherwise); with `--yes` it starts
the switch and follows it (`--wait`, JSON lines with `--json`). Ctrl-C only
stops watching. `models add --activate` now probes before activating and
takes the same path. New `models switch status|cancel` and `models spaces
[--cleanup <id> --yes]`; `memhub status` shows the switch, merge or scan
line; `embedding_unavailable` reads as "the embedding service is
unavailable".

docs/contracts/embedding-switch-v1.md is the App's contract: endpoints,
states, progress fields, error codes, and zh/en/es copy for the
loading-screen merge, the confirmation dialog, the progress banner, the
failure states and the `embedding_unavailable` toast.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 20:49:23 +08:00
alvinfeiandClaude Opus 5.5 e87d6305be feat(memhub): switch the embedding model safely in Core
The embedding space is explicit now. An `embedding_spaces` registry in the
main Core SQLite pins the live space; existing homes register the
collection they already use under its four-attribute name, byte for byte
(an "auto" width keeps its unsuffixed name and only records the measured
width). The identity of a space is the embedding model and width: a new
endpoint for the same model keeps it. `build_sdk_config(profile, space,
mode, features=)` takes the space and refuses a profile whose identity is
not the space's (`embedding_space_mismatch`); mode and features never
change a name. The read handle and the write coordinator resolve the live
space through the registry and key their handles by it.

B2: a probe on the profile that fills the live space no longer records a
width that would move writes to new collections; the same stored width is
a relabel only.

An identity change becomes a switch job (`embedding_switch_jobs`):
preparing, reembedding, verifying, committing, done / failed (rollback) /
rolled_back, with a heartbeat and owner pid. Model writes pause (claim and
admit gates, direct edit/delete/graph-width answer 409
`embedding_switch_in_progress`); intake keeps queueing. Every vector of
every user is re-embedded into a new space through the SDK re-embed
module (`hy_sdk_migration.SdkReembedBackend`), the graph too in ultra,
verified per role and per user, then one SQLite transaction makes the new
space live, retires the old one and activates the profile. Reads serve the
old space until that commit.

Every Core start scans the vector store before model writes resume. A
non-empty stray space (an earlier build wrote one collection and read
another) is merged into the live space with the live embedder; live
records win, merged strays retire. Unfinished jobs resume at startup,
including after an App update: `sdk-migration-status` reports an open job
without requiring or blocking a migration, and `sdk-migrate` leaves the
switch tables alone. Retired spaces are dropped after 14 days or on
request.

Activation answers 409 `embedding_switch_required` with the plan unless
the identity is unchanged; an empty home activates directly. A search,
recall or context call whose remote embedding does not answer fails with
503 `embedding_unavailable` (no lexical fallback).

API: GET/DELETE /api/v1/embedding-spaces, POST
/api/v1/embedding-switches/plan, POST /api/v1/embedding-switches, GET
/api/v1/embedding-switches/current and /{id}, POST
/api/v1/embedding-switches/{id}/cancel; `embedding_switch` on health and
memory-processing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 20:49:12 +08:00
alvinfeiandClaude Opus 5.5 7a78fca807 build(memhub): pin hy-memory 2.0.10
Core pins and locks the published hy-memory 2.0.10 from PyPI, which adds
the re-embed module the embedding switch needs. The version constant,
README, architecture notes, SBOM, notices and their tests follow.
CHANGELOG records 2.0.10 as published.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 20:48:55 +08:00
154 changed files with 27763 additions and 486 deletions

No files matched your search

+45 -3
View File
@@ -4,7 +4,47 @@ Hy Memory and MemHub are versioned and released independently.
## Unreleased
- Hy Memory: LLM prompts now ship in 11 languages: `en`, `zh-Hans`,
## MemHub 1.0.6 — 2026-11-02
Build 37, on `hy-memory==2.0.11`. Linux x64/arm64 Runtime and macOS arm64
App and Runtime.
- Interfaces in 11 languages: English, Simplified and Traditional Chinese,
Japanese, Spanish, German, Portuguese, French, Korean, Russian and
Indonesian. Choosing any language other than Simplified Chinese also sets
the SDK prompt language; Simplified Chinese keeps the automatic zh/en
detection.
- Memory modes: lite (no LLM, keeps raw conversations), pro (LLM extraction)
and ultra (pro plus the knowledge graph and a nightly evolve run). Set with
`GET`/`PUT /api/v1/settings/memory-mode` or `memhub settings mode`. Raw
conversations are now kept permanently. `POST /api/v2/evolve` and
`memhub evolve` run System 2 on demand.
- Safe embedding-model switching: an embedding-space registry, a switch that
pauses writes, re-embeds every user's vectors and the graph, verifies and
commits atomically, and resumes after a crash or an App update. Every start
scans the vector store and merges data left in other spaces. When the
embedding service is unavailable, search returns `embedding_unavailable`.
- Multi-user Runtime: memory APIs and the CLI take `user_id` (default
`local`); records are isolated per user; Hermes profiles can be connected as
separate users. The App keeps using `local`.
- Configurable SDK features through the CLI and API (`memhub settings
features`): graph rerank, episode recall, session episodes, episode
summaries, intention and few-shot. Defaults are unchanged.
- Search and recall pass through the SDK's search parameters with server-side
score floors; the memory library filters by layer; `/api/v2/raw` returns
data again; v2 source and import routes; profile version history.
- Upgrade fixes: migration no longer stops without a model profile (#234) or
on a pre-existing readback mismatch (#229, #240); source grants work after
the cutover (#233); reads use the active embedding (#242); the embedding
endpoint no longer follows the generation `base_url`.
- Hermes import drops gateway notices and probes and unwraps relayed
messages; `memhub logs` no longer reports valid older records as damaged.
## Hy Memory 2.0.11 — 2026-11-01
- Published
[`hy-memory==2.0.11`](https://pypi.org/project/hy-memory/2.0.11/).
- LLM prompts now ship in 11 languages: `en`, `zh-Hans`,
`zh-Hant`, `ja`, `es`, `de`, `pt`, `fr`, `ko`, `ru` and `id`. They live in
one registry, `hy_memory/prompts/`, with one module per language. Call
sites use `get_prompt(name, language)`, so a new language only needs a new
@@ -17,7 +57,7 @@ Hy Memory and MemHub are versioned and released independently.
(`prompts_zh.*_ZH`, `extractor.EXTRACT_SYSTEM_PROMPT`, ...) still import.
The nine new languages are machine translations, and native-speaker review
is recommended.
- Hy Memory: new optional `language` to choose the prompt language
- New optional `language` to choose the prompt language
explicitly. Set it on `MemoryConfig.language` (env `MEMORY_LANGUAGE`) for
a client default, or per call on `add` / `async_add`, `add_extracted`,
`finalize_session`, `evolve` and `sweep`. A per-call value overrides the
@@ -30,11 +70,13 @@ Hy Memory and MemHub are versioned and released independently.
Without a language, the existing zh/en auto-detection runs unchanged and
memories keep the source language. `search` has no `language`: its judge
and rewrite prompts are English-only and keep the query's language.
- Hy Memory: the HTTP server accepts `language` on `/api/v1/add` and
- The HTTP server accepts `language` on `/api/v1/add` and
`/api/v1/add_extracted`.
## Hy Memory 2.0.10 — 2026-11-01
- Published
[`hy-memory==2.0.10`](https://pypi.org/project/hy-memory/2.0.10/).
- New `hy_memory.reembed` module for switching the embedding
model. Writers must be paused for the whole run. Nothing is derived from
the provider or endpoint: source and target are named explicitly, and the
+1
View File
@@ -33,6 +33,7 @@ English | [中文](README.zh.md)
## Latest News
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Open-sourced** quantized **FP8** (**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**) and **GGUF** (**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**) versions of **Hy-Memory-30B-A3B** on Hugging Face!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-11]**: 📦 **Released [MemHub v1.0.6](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.6)** with interfaces in 11 languages, memory modes (lite / pro / ultra with nightly evolve), safe embedding-model switching, multi-user support for the CLI and API, and more reliable upgrades, on **`hy-memory==2.0.11`**. See the release notes for details.
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **Released [MemHub v1.0.5](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.5)** with Chinese, English and Spanish interfaces, one-time pairing codes for more custom Harnesses, and smoother browsing, search, startup and recovery, on **`hy-memory==2.0.8`**. See the release notes for details.
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **Released [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** and **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**, with a signed macOS desktop app and verified macOS/Linux Runtimes!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Released** **Hy Memory 2.0 (`hy-memory==2.0.4`)** on [PyPI](https://pypi.org/project/hy-memory/2.0.4/) and open-sourced the **Hy-Memory** monorepo together with **[MemHub](memhub/README.md)**!
+1
View File
@@ -33,6 +33,7 @@
## 最新动态
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **模型开源** 在 Hugging Face 发布 **Hy-Memory-30B-A3B** 的 **FP8**(**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**)与 **GGUF**(**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**)量化版本模型!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-11]**: 📦 **正式发布 [MemHub v1.0.6](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.6)**:界面支持 11 种语言,新增记忆模式(lite / pro / ultra,ultra 每晚自动整理),可安全切换嵌入模型,CLI 和 API 支持多用户,升级更可靠;底层使用 **`hy-memory==2.0.11`**。详见发布说明。
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **正式发布 [MemHub v1.0.5](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.5)**:支持中文、英文和西班牙语界面,可通过一次性配对码连接更多自定义 Harness,并优化了记忆浏览、搜索、启动和异常恢复;底层使用 **`hy-memory==2.0.8`**。详见发布说明。
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **正式发布 [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** 与 **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**,提供已签名的 macOS 桌面应用及经过验证的 macOS/Linux Runtime!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **正式发布** **Hy Memory 2.0 (`hy-memory==2.0.4`)**(已上线 [PyPI](https://pypi.org/project/hy-memory/2.0.4/)),并开源 **Hy-Memory** monorepo 与 **[MemHub](memhub/README.md)**!
+4 -4
View File
@@ -18,13 +18,13 @@ If you run into a problem, please open an Issue with the version, platform,repro
## 发行与平台合同
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.9`。MemHub 源码版本为 `1.0.5`
(build 36),GitHub 仓库为
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.11`。MemHub 源码版本为 `1.0.6`
(build 37),GitHub 仓库为
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
`memhub-cli`。当前代码架构见
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)。
`v1.0.5` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64 App 和 Runtime。
`v1.0.6` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64 App 和 Runtime。
Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命周期验收;macOS
制品由发行流水线完成 Developer ID 签名、公证和验收后手动上传。Windows 不支持。
只有同时出现在 GitHub Release 与 `release-manifest.json` 中、且通过公开校验的制品才属于
@@ -118,7 +118,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
## Hy Memory
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.9`。SDK 配置、权限边界及 search
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.11`。SDK 配置、权限边界及 search
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
+1 -1
View File
@@ -2352,7 +2352,7 @@
{
"SPDXID": "SPDXRef-Package-hy-memory",
"name": "hy-memory",
"versionInfo": "2.0.9",
"versionInfo": "2.0.11",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"licenseConcluded": "Apache-2.0",
+1 -1
View File
@@ -9,7 +9,7 @@ Per-package license texts are not reproduced here yet; the license
column is filled in as part of release review, and the SBOM records
`NOASSERTION` until then rather than guessing.
## hy-memory 2.0.9
## hy-memory 2.0.11
License: Apache-2.0
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/cli",
"version": "1.0.5",
"version": "1.0.6",
"description": "MemHub bootstrap CLI for installing and managing the local-first shared memory runtime",
"license": "Apache-2.0",
"repository": {
+243 -25
View File
@@ -122,6 +122,14 @@ import {
doctorExitCode,
renderDoctorText,
} from "./doctor-report.js";
import {
activateWithEmbeddingSwitch,
cleanupSpace,
listSpaces,
type SwitchIo,
switchCancel,
switchStatus,
} from "./embedding-switch-flow.js";
import {
EXIT_FAILED,
EXIT_OK,
@@ -195,6 +203,13 @@ import {
renderFeaturesText,
runFeaturesCommand,
} from "./memory-features-flow.js";
import {
type LanguageDocument,
languageExitCode,
parseLanguageCommand,
renderLanguageText,
runLanguageCommand,
} from "./memory-language-flow.js";
import {
type EvolveDocument,
evolveExitCode,
@@ -248,7 +263,6 @@ import {
renderModelReportText,
} from "./model-report.js";
import {
activateModelProfile,
editModelProfile,
type ModelWriteDocument,
modelWriteExitCode,
@@ -1703,6 +1717,18 @@ async function runCommand(
);
if (apiKey === undefined) return usage();
}
if (
parsed.kind === "spaces" &&
parsed.cleanup !== undefined &&
!parsed.confirmed
) {
return emitConfirmPreview(asJson, {
command: "models spaces",
target: parsed.cleanup,
effect: `删除已停用的向量空间 ${parsed.cleanup}(不可恢复;正在使用的空间不能删除)`,
retry: `${PRODUCT} models spaces --cleanup ${parsed.cleanup} --yes`,
});
}
if (parsed.kind === "remove" && !parsed.confirmed) {
return emitConfirmPreview(asJson, {
command: "models remove",
@@ -1725,6 +1751,50 @@ async function runCommand(
return connected.exitCode;
}
const { reader, operations } = connected;
// Ctrl-C while following a switch stops watching, never the switch.
const interrupt = new AbortController();
const switchIo: SwitchIo = {
reader,
operations,
emit: (text) => {
emit(text);
},
asJson,
interactive:
!asJson && (controls.stdinIsTTY ?? process.stdin.isTTY) === true,
confirm: async (question) => {
const prompter =
controls.wizardPrompter ?? (await createTerminalPrompter());
try {
return /^(y|yes)$/i.test((await prompter.ask(question)).trim());
} finally {
if ("close" in prompter && typeof prompter.close === "function") {
prompter.close();
}
}
},
...(controls.watchWait ? { wait: controls.watchWait } : {}),
signal: controls.watchSignal ?? interrupt.signal,
};
const withInterrupt = async (run: () => Promise<number>) => {
const onInterrupt = () => interrupt.abort();
process.once("SIGINT", onInterrupt);
try {
return await run();
} finally {
process.removeListener("SIGINT", onInterrupt);
}
};
if (parsed.kind === "switch") {
return parsed.action === "status"
? await switchStatus(switchIo)
: await switchCancel(switchIo);
}
if (parsed.kind === "spaces") {
return parsed.cleanup === undefined
? await listSpaces(switchIo)
: await cleanupSpace(parsed.cleanup, switchIo);
}
if (parsed.kind === "add") {
const created = await createModelProfile(
config,
@@ -1739,33 +1809,66 @@ async function runCommand(
emitFlowDocument(created, asJson);
return flowExitCode(created);
}
const activated = await activateModelProfile(
String(created.receipt.profile_id),
// Core activates only a probed profile: probe first, then the same
// path as `models activate`, so an embedding change shows its plan.
const profileId = String(created.receipt.profile_id);
const probe = await probeUntilSettled(
reader,
operations,
profileId,
controls.watchWait,
);
const document = {
...created,
activation:
activated.state === "committed"
? { state: "committed" }
: { state: activated.state, code: activated.code },
};
const probed =
probe.state === "completed" &&
Boolean(
(
(probe.receipt.result as Record<string, unknown> | undefined)
?.capabilities as Record<string, unknown> | undefined
)?.embedding_available,
);
emit(
asJson
? `${JSON.stringify(document)}\n`
? `${JSON.stringify({
...created,
probe: {
state: probe.state,
...("operation_id" in probe && probe.operation_id
? { operation_id: probe.operation_id }
: {}),
...("code" in probe ? { code: probe.code } : {}),
},
})}\n`
: `${renderFlowText(created)}${
activated.state === "committed"
? "已设为当前使用的模型配置。\n"
: `设为当前使用未完成:${activated.state}(${activated.code})\n`
probed
? "探测通过。\n"
: `探测未通过(${"code" in probe ? probe.code : probe.state}),没有设为当前使用;用 ${PRODUCT} models test ${profileId} 查看原因。\n`
}`,
);
return activated.state === "committed" ? EXIT_OK : EXIT_FAILED;
if (!probed) return EXIT_FAILED;
return await withInterrupt(() =>
activateWithEmbeddingSwitch(
profileId,
{ confirmed: parsed.confirmed, follow: parsed.follow },
switchIo,
),
);
}
if (parsed.kind === "edit" || parsed.kind === "activate") {
const document =
parsed.kind === "edit"
? await editModelProfile(parsed.profileId, config, apiKey, operations)
: await activateModelProfile(parsed.profileId, operations);
if (parsed.kind === "activate") {
return await withInterrupt(() =>
activateWithEmbeddingSwitch(
parsed.profileId,
{ confirmed: parsed.confirmed, follow: parsed.follow },
switchIo,
),
);
}
if (parsed.kind === "edit") {
const document = await editModelProfile(
parsed.profileId,
config,
apiKey,
operations,
);
emitModelWriteDocument(document, asJson);
return modelWriteExitCode(document);
}
@@ -2283,6 +2386,40 @@ async function runCommand(
);
return featuresExitCode(document);
}
if (
named === "settings" &&
arguments_.slice(1).find((value) => value !== "--json") === "language"
) {
const parsed = parseLanguageCommand(
arguments_
.slice(1)
.filter((value, index, all) => index !== all.indexOf("language")),
);
if (!parsed) return usage();
const connected = await connectRuntime(environment);
if (!connected.ok) {
const document: LanguageDocument = {
schema_version: 1,
product: PRODUCT,
command: "settings language",
state: "unavailable",
code: connected.code,
};
emit(
asJson ? `${JSON.stringify(document)}\n` : renderLanguageText(document),
);
return connected.exitCode;
}
const document = await runLanguageCommand(
parsed,
connected.reader,
connected.operations,
);
emit(
asJson ? `${JSON.stringify(document)}\n` : renderLanguageText(document),
);
return languageExitCode(document);
}
if (named === "settings") {
const parsed = parseSettingsCommand(arguments_.slice(1));
if (!parsed) return usage();
@@ -5289,6 +5426,8 @@ type ModelsCommand =
configFile: string;
apiKeyFile?: string;
activate: boolean;
confirmed: boolean;
follow?: boolean;
}
| {
kind: "edit";
@@ -5296,7 +5435,14 @@ type ModelsCommand =
configFile: string;
apiKeyFile?: string;
}
| { kind: "activate"; profileId: string }
| {
kind: "activate";
profileId: string;
confirmed: boolean;
follow?: boolean;
}
| { kind: "switch"; action: "status" | "cancel" }
| { kind: "spaces"; cleanup?: string; confirmed: boolean }
| { kind: "test"; profileId: string }
| { kind: "receipt"; operationId: string }
| { kind: "remove"; profileId: string; confirmed: boolean };
@@ -5337,16 +5483,46 @@ function parseModelsCommand(rest: readonly string[]): ModelsCommand | null {
const flags = readFlags(
tokens.slice(1),
["--config-file", "--api-key-file"],
["--activate"],
["--activate", "--yes", "--wait", "--no-wait"],
);
const configFile = flags?.values.get("--config-file");
if (!flags || configFile === undefined) return null;
const follow = followFlag(flags.switches);
if (follow === null) return null;
const activate = flags.switches.has("--activate");
// The switch flags only mean something when the add also activates.
if (
!activate &&
["--yes", "--wait", "--no-wait"].some((flag) => flags.switches.has(flag))
) {
return null;
}
const apiKeyFile = flags.values.get("--api-key-file");
return {
kind: "add",
configFile,
...(apiKeyFile === undefined ? {} : { apiKeyFile }),
activate: flags.switches.has("--activate"),
activate,
confirmed: flags.switches.has("--yes"),
...(follow === undefined ? {} : { follow }),
};
}
if (action === "switch") {
if (tokens.length !== 2 || (target !== "status" && target !== "cancel")) {
return null;
}
return { kind: "switch", action: target };
}
if (action === "spaces") {
const flags = readFlags(tokens.slice(1), ["--cleanup"], ["--yes"]);
if (!flags) return null;
const cleanup = flags.values.get("--cleanup");
if (cleanup === undefined && flags.switches.has("--yes")) return null;
if (cleanup !== undefined && !OPERATION_ID.test(cleanup)) return null;
return {
kind: "spaces",
...(cleanup === undefined ? {} : { cleanup }),
confirmed: flags.switches.has("--yes"),
};
}
if (target === undefined || !OPERATION_ID.test(target)) return null;
@@ -5367,8 +5543,21 @@ function parseModelsCommand(rest: readonly string[]): ModelsCommand | null {
};
}
if (action === "activate") {
const flags = readFlags(tokens.slice(2), [], ["--confirm", "--yes"]);
return flags ? { kind: "activate", profileId: target } : null;
const flags = readFlags(
tokens.slice(2),
[],
["--confirm", "--yes", "--wait", "--no-wait"],
);
if (!flags) return null;
const follow = followFlag(flags.switches);
if (follow === null) return null;
return {
kind: "activate",
profileId: target,
// `--confirm` stays the hidden alias of `--yes`.
confirmed: flags.switches.has("--yes") || flags.switches.has("--confirm"),
...(follow === undefined ? {} : { follow }),
};
}
if (action === "test") {
return tokens.length === 2 ? { kind: "test", profileId: target } : null;
@@ -5391,6 +5580,35 @@ function parseModelsCommand(rest: readonly string[]): ModelsCommand | null {
return null;
}
/** `--wait` / `--no-wait`: undefined when neither, null when both. */
function followFlag(switches: ReadonlySet<string>): boolean | undefined | null {
const wait = switches.has("--wait");
const noWait = switches.has("--no-wait");
if (wait && noWait) return null;
return wait ? true : noWait ? false : undefined;
}
/** Start a probe and read its receipt until it settles (about two minutes). */
async function probeUntilSettled(
reader: CoreReader,
operations: CoreOperations,
profileId: string,
wait?: (milliseconds: number, signal: AbortSignal) => Promise<void>,
): Promise<ModelProbeFlowDocument> {
let document = await startModelProbe(reader, operations, profileId);
const pause =
wait ??
((milliseconds: number) =>
new Promise<void>((resolve) => setTimeout(resolve, milliseconds)));
const signal = new AbortController().signal;
for (let attempt = 0; attempt < 120; attempt += 1) {
if (document.state !== "pending" || !document.operation_id) break;
await pause(1_000, signal);
document = await readModelProbeReceipt(reader, document.operation_id);
}
return document;
}
type SourcesCommand =
| {
kind: "ids";
+21 -5
View File
@@ -361,9 +361,9 @@ export const COMMANDS: readonly CommandDescriptor[] = [
{
name: "models",
summary:
"管理生成模型配置。add/test 直接提交并返回操作 ID(异步,用 receipt 查结果);edit/activate 直接执行;remove 不可逆,需 --yes。<profile-id>/<operation-id> 都是 UUID;脚本模式的凭据只从受控文件读取。",
"管理生成模型配置。add/test 直接提交并返回操作 ID(异步,用 receipt 查结果);edit 直接执行;activate 换了向量模型(模型或维度)时先显示切换计划,需 --yes 确认后重新生成全部向量,switch 查看或取消、spaces 查看或清理旧向量数据;remove 不可逆,需 --yes。<profile-id>/<operation-id> 都是 UUID;脚本模式的凭据只从受控文件读取。",
usage:
"memhub models list [--json]\n memhub models show <profile-id> [--json]\n memhub models add --config-file <path> [--api-key-file <path>] [--activate] [--json]\n memhub models test <profile-id> [--json]\n memhub models receipt <operation-id> [--json]\n memhub models edit <profile-id> --config-file <path> [--api-key-file <path>] [--json]\n memhub models activate <profile-id> [--json]\n memhub models remove <profile-id> [--yes] [--json]",
"memhub models list [--json]\n memhub models show <profile-id> [--json]\n memhub models add --config-file <path> [--api-key-file <path>] [--activate [--yes] [--wait|--no-wait]] [--json]\n memhub models test <profile-id> [--json]\n memhub models receipt <operation-id> [--json]\n memhub models edit <profile-id> --config-file <path> [--api-key-file <path>] [--json]\n memhub models activate <profile-id> [--yes] [--wait|--no-wait] [--json]\n memhub models switch <status|cancel> [--json]\n memhub models spaces [--cleanup <space-id> --yes] [--json]\n memhub models remove <profile-id> [--yes] [--json]",
alternative_usage: "memhub models setup",
options: [
{
@@ -377,7 +377,17 @@ export const COMMANDS: readonly CommandDescriptor[] = [
},
{
flag: "--activate",
summary: "add 成功后把新配置设为当前使用。",
summary:
"add 成功后先探测,再按 activate 的规则设为当前使用(向量模型变化时同样要确认切换)。",
},
{
flag: "--wait | --no-wait",
summary:
"activate 开始向量模型切换后是否跟随进度;终端里默认跟随,Ctrl-C 只停止显示。",
},
{
flag: "--cleanup <space-id>",
summary: "spaces:立即删除一个已停用的向量空间(不可恢复,需 --yes)。",
},
DESTRUCTIVE_CONFIRM_OPTION,
JSON_OPTION,
@@ -451,12 +461,13 @@ export const COMMANDS: readonly CommandDescriptor[] = [
{
name: "settings",
summary:
"查看或切换记忆模式(lite / pro / ultra)。lite 不调用模型,只在原始对话里检索;pro 由模型提取记忆(System 1);ultra 在 pro 之上每晚自动进化(System 2,使用知识图谱)。任何模式都保留全部原始对话。切换不迁移数据:已处理的不会重做,排队中的和新的按新模式处理;需 --yes,不加时只显示计划。环境变量 ONE_MEMORY_GRAPH_STORE 会强制 ultra。settings features 查看或开关 SDK 的可选检索与提取功能(全局设置,默认全部保持 SDK 默认值):graph_rerank 图谱重排(仅 ultra)、same_series_timeline 同系列时间线(需 graph_rerank)、episode_recall 片段召回(pro / ultra,需先开 session_episodes 写入片段)、session_episodes 片段写入、episode_summary 片段摘要(每个片段一次模型调用)、intention 意图提取(默认开)、few_shot 提取示例;set 需 --yes,不加时只显示计划;reset 恢复默认值。",
"查看或切换记忆模式(lite / pro / ultra)。lite 不调用模型,只在原始对话里检索;pro 由模型提取记忆(System 1);ultra 在 pro 之上每晚自动进化(System 2,使用知识图谱)。任何模式都保留全部原始对话。切换不迁移数据:已处理的不会重做,排队中的和新的按新模式处理;需 --yes,不加时只显示计划。环境变量 ONE_MEMORY_GRAPH_STORE 会强制 ultra。settings features 查看或开关 SDK 的可选检索与提取功能(全局设置,默认全部保持 SDK 默认值):graph_rerank 图谱重排(仅 ultra)、same_series_timeline 同系列时间线(需 graph_rerank)、episode_recall 片段召回(pro / ultra,需先开 session_episodes 写入片段)、session_episodes 片段写入、episode_summary 片段摘要(每个片段一次模型调用)、intention 意图提取(默认开)、few_shot 提取示例;set 需 --yes,不加时只显示计划;reset 恢复默认值。settings language 查看或设置界面语言(App 切换语言时会自动设置):简体中文(zh-Hans)和 system 不指定 SDK 语言,SDK 照旧自动识别中文或英文;其余语言(en、zh-Hant、ja、es、de、pt、fr、ko、ru、id)会让提取等 SDK 提示使用该语言,立即生效。",
usage:
"memhub settings mode [lite|pro|ultra] [--yes] [--json]\n" +
" memhub settings features [--json]\n" +
" memhub settings features set <name>=<true|false> [...] [--yes] [--json]\n" +
" memhub settings features reset [<name> ...] [--json]",
" memhub settings features reset [<name> ...] [--json]\n" +
" memhub settings language [<code>|system] [--json]",
options: [
{
flag: "mode [lite|pro|ultra]",
@@ -471,6 +482,11 @@ export const COMMANDS: readonly CommandDescriptor[] = [
flag: "features reset [<name> ...]",
summary: "把指定功能(不写则全部)恢复为 SDK 默认值。",
},
{
flag: "language [<code>|system]",
summary:
"不带参数显示界面语言和 SDK 提示语言;带语言代码(en、zh-Hans、zh-Hant、ja、es、de、pt、fr、ko、ru、id)或 system 时设置。",
},
CONFIRM_OPTION,
JSON_OPTION,
],
@@ -635,6 +635,8 @@ describe("documented commands", () => {
"receipt",
"edit",
"activate",
"switch",
"spaces",
"remove",
],
// Source processing is one Freeze/Thaw switch under `sync`; the obsolete
+6 -1
View File
@@ -97,7 +97,12 @@ export function contextExitCode(document: ContextSearchDocument): number {
}
export function renderContextText(document: ContextSearchDocument): string {
if (document.state === "unavailable") return "暂时无法查记忆,请稍后重试。\n";
if (document.state === "unavailable") {
// No lexical fallback: without the embedding service nothing is searched.
return document.code === "embedding_unavailable"
? "向量服务不可用,无法检索;请检查向量模型配置。\n"
: "暂时无法查记忆,请稍后重试。\n";
}
if (document.state === "denied") return "当前应用没有查记忆权限。\n";
if (document.state === "invalid") return "查记忆返回了无法识别的结果。\n";
if (document.state === "rejected") {
+22 -2
View File
@@ -8,6 +8,9 @@ import axios from "axios";
*/
const READ_ONLY_PATHS = [
"/api/v1/call-events",
// Read-only: the registry plus a scan of the vector store's collection names.
"/api/v1/embedding-spaces",
"/api/v1/embedding-switches/current",
"/api/v1/import-jobs",
"/api/v1/live-sync",
"/api/v1/memory-processing",
@@ -20,6 +23,7 @@ const READ_ONLY_PATHS = [
"/api/v2/user-profile/versions",
"/api/v1/settings/memory-mode",
"/api/v1/settings/memory-features",
"/api/v1/settings/ui-language",
"/api/v2/evolve",
] as const;
/** One evolve run's receipt (`memhub evolve --wait`). */
@@ -163,10 +167,23 @@ export function createCoreReader(attachment: {
code: detailCode(response.data, response.status),
};
}
return { state: "unavailable" as const, code: `http_${response.status}` };
return {
state: "unavailable" as const,
code: unavailableCode(response.data, response.status),
};
};
}
/** A 503 that names `embedding_unavailable` keeps it: the search could not
* embed its query, which needs another fix than "try again later". */
function unavailableCode(value: unknown, status: number): string {
return status === 503 && detailCode(value, status) === EMBEDDING_UNAVAILABLE
? EMBEDDING_UNAVAILABLE
: `http_${status}`;
}
export const EMBEDDING_UNAVAILABLE = "embedding_unavailable";
/** Core's stable `detail` code when it sent one, else the bare status. */
function detailCode(value: unknown, status: number): string {
const detail =
@@ -218,7 +235,10 @@ export function createCoreContextQuery(attachment: {
if (response.status === 404) {
return { state: "not_found" as const, code: "http_404" };
}
return { state: "unavailable" as const, code: `http_${response.status}` };
return {
state: "unavailable" as const,
code: unavailableCode(response.data, response.status),
};
};
}
+34 -3
View File
@@ -31,12 +31,29 @@ const EVOLVE_PATH = "/api/v2/evolve";
const MEMORY_MODE_PATH = "/api/v1/settings/memory-mode";
/** `memhub settings features set|reset` sets the SDK features. */
const MEMORY_FEATURES_PATH = "/api/v1/settings/memory-features";
/** `memhub settings language <code>` sets the display language Core uses. */
const UI_LANGUAGE_PATH = "/api/v1/settings/ui-language";
const ONBOARDING_COMPLETE_PATH = "/api/v1/onboarding/complete";
const EMBEDDING_SWITCH_PATHS = [
"/api/v1/embedding-switches",
"/api/v1/embedding-switches/plan",
] as const;
const EMBEDDING_SWITCH_CANCEL =
/^\/api\/v1\/embedding-switches\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\/cancel$/;
const EMBEDDING_SPACE_PATH =
/^\/api\/v1\/embedding-spaces\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const MAX_BYTES = 4 * 1024 * 1024;
export type OperationOutcome<T> =
| { state: "committed"; data: T }
| { state: "rejected"; code: string; http_status: number }
| {
state: "rejected";
code: string;
http_status: number;
/** Core's refusal body, when it carries more than the code (a switch
* plan rides along with `embedding_switch_required`). */
body?: unknown;
}
| { state: "unknown"; code: string };
export interface CoreOperations {
@@ -104,6 +121,8 @@ export function createCoreOperations(attachment: {
path !== GRAPH_WIDTH_PATH &&
path !== EVOLVE_PATH &&
path !== ONBOARDING_COMPLETE_PATH &&
!(EMBEDDING_SWITCH_PATHS as readonly string[]).includes(path) &&
!EMBEDDING_SWITCH_CANCEL.test(path) &&
!MODEL_PROFILE_ACTIVATE.test(path)
) {
throw new Error("core_operation_path_not_allowed");
@@ -121,11 +140,18 @@ export function createCoreOperations(attachment: {
if (response.status >= 200 && response.status < 300) {
return { state: "committed", data: response.data as T };
}
if (response.status >= 400 && response.status < 500) {
if (
(response.status >= 400 && response.status < 500) ||
// Core answers a refusal it can name with 503 (`sdk_reembed_unavailable`)
// or 507 (`insufficient_disk_space`) on the switch routes.
([503, 507].includes(response.status) &&
(EMBEDDING_SWITCH_PATHS as readonly string[]).includes(path))
) {
return {
state: "rejected",
code: safeErrorCode(response.data, response.status),
http_status: response.status,
body: response.data,
};
}
return {
@@ -141,6 +167,7 @@ export function createCoreOperations(attachment: {
if (
path !== MEMORY_MODE_PATH &&
path !== MEMORY_FEATURES_PATH &&
path !== UI_LANGUAGE_PATH &&
(!SDK_MEMORY_PATH.test(path) || path.includes("?"))
) {
throw new Error("core_operation_path_not_allowed");
@@ -204,7 +231,11 @@ export function createCoreOperations(attachment: {
};
},
async delete<T>(path: string, signal?: AbortSignal) {
if (!SDK_MEMORY_PATH.test(path) && !MODEL_PROFILE_PATH.test(path)) {
if (
!SDK_MEMORY_PATH.test(path) &&
!MODEL_PROFILE_PATH.test(path) &&
!EMBEDDING_SPACE_PATH.test(path)
) {
throw new Error("core_operation_path_not_allowed");
}
let response: { status: number; data: unknown };
@@ -0,0 +1,488 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { run } from "./cli.js";
import { renderContextText } from "./context-search.js";
import { describeEmbeddingSwitchSummary } from "./embedding-switch-flow.js";
import { composeStatusReport, renderStatusText } from "./status-report.js";
const transport = vi.hoisted(() => ({
get: vi.fn(),
post: vi.fn(),
put: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
}));
vi.mock("axios", () => ({
default: { create: vi.fn(() => transport), get: vi.fn() },
}));
const runtime = vi.hoisted(() => ({
inspectRuntimeStatus: vi.fn(),
discoverRuntimeAttachment: vi.fn(),
}));
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
...runtime,
}));
const PROFILE = "44444444-4444-4444-8444-444444444444";
const JOB = "55555555-5555-4555-8555-555555555555";
const SPACE = "66666666-6666-4666-8666-666666666666";
function plan(overrides: Record<string, unknown> = {}) {
return {
needed: true,
reason: "embedding_identity_changed",
plan_token: "token-1",
source: {
space_id: "77777777-7777-4777-8777-777777777777",
model: "bge-small-zh-v1.5",
dims: 512,
collection_base: "memhub_aaaaaaaaaaaaaaaa",
vectors: 12480,
},
target: {
space_id: null,
model: "text-embedding-v3",
dims: 1024,
collection_base: "",
},
target_profile_id: PROFILE,
target_revision: 5,
target_provider_kind: "openai_compatible",
target_local: false,
sources: [],
total_vectors: 12480,
users: 2,
graph_vectors: null,
estimate_s: 120,
requests: 195,
disk_bytes_needed: 70 * 1024 * 1024,
disk_bytes_free: 10 * 1024 * 1024 * 1024,
retention_days: 14,
blockers: [],
...overrides,
};
}
function job(
state: string,
percent: number,
extra: Record<string, unknown> = {},
) {
return {
job_id: JOB,
kind: "switch",
state,
source: plan().source,
target: { ...plan().target, space_id: SPACE },
stray_space_ids: [],
target_profile_id: PROFILE,
target_revision: 5,
requested_by: "cli",
progress: {
phase: state === "reembedding" ? "vectors" : "prepare",
done: Math.round((12480 * percent) / 100),
total: 12480,
percent,
eta_s: 60,
},
cancel_requested: false,
error_code: null,
attempt: 1,
created_at: "T1",
updated_at: "T1",
finished_at: state === "done" ? "T2" : null,
...extra,
};
}
beforeEach(() => {
vi.clearAllMocks();
for (const method of Object.values(transport)) method.mockReset();
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
version: "1.2.3",
});
runtime.discoverRuntimeAttachment.mockResolvedValue({
instanceId: "instance-1",
apiToken: "synthetic-runtime-token",
host: "127.0.0.1",
port: 8765,
servicePid: 4242,
health: {
status: "ok",
service: "one-memory-core",
version: "1.2.3",
instance_id: "instance-1",
},
});
});
describe("models activate with an embedding change", () => {
it("prints the plan and exits 2 without --yes on a non-terminal", async () => {
transport.post.mockImplementation(async (url: string) =>
url === "/api/v1/embedding-switches/plan"
? { status: 200, data: plan() }
: { status: 404, data: {} },
);
const result = await capturedRun(["models", "activate", PROFILE]);
expect(result.code).toBe(2);
expect(result.stdout).toContain("将更换向量模型");
expect(result.stdout).toContain("bge-small-zh-v1.5(512 维)");
expect(result.stdout).toContain("text-embedding-v3(1024 维)");
expect(result.stdout).toContain("12,480");
expect(result.stdout).toContain("重新运行并加上 --yes 以确认");
expect(transport.post.mock.calls.map(([url]) => url)).toEqual([
"/api/v1/embedding-switches/plan",
]);
});
it("starts the switch with --yes and emits JSON progress lines", async () => {
transport.post.mockImplementation(async (url: string) => {
if (url === "/api/v1/embedding-switches/plan") {
return { status: 200, data: plan() };
}
if (url === "/api/v1/embedding-switches") {
return { status: 202, data: job("preparing", 0) };
}
return { status: 404, data: {} };
});
const answers = [
{
phase: "running",
writes_paused: true,
job: job("reembedding", 50),
last_finished: null,
},
{
phase: "idle",
writes_paused: false,
job: null,
last_finished: job("done", 100),
},
];
transport.get.mockImplementation(async (url: string) =>
url === "/api/v1/embedding-switches/current"
? { status: 200, data: { schema_version: 1, ...answers.shift() } }
: { status: 404, data: {} },
);
const result = await capturedRun(
["models", "activate", PROFILE, "--yes", "--wait", "--json"],
{ watchWait: async () => undefined },
);
expect(result.code).toBe(0);
const lines = result.stdout
.trim()
.split("\n")
.map((line) => JSON.parse(line));
expect(lines.map((line) => line.state)).toEqual([
"preparing",
"reembedding",
"done",
"done",
]);
expect(lines[1]).toMatchObject({ percent: 50, job_id: JOB });
expect(lines.at(-1)).toMatchObject({ result: "switched" });
expect(transport.post).toHaveBeenCalledWith(
"/api/v1/embedding-switches",
{
profile_id: PROFILE,
expected_revision: 5,
plan_token: "token-1",
requested_by: "cli",
},
undefined,
);
});
it("refuses a plan with blockers and starts nothing", async () => {
transport.post.mockResolvedValue({
status: 200,
data: plan({ blockers: ["sdk_reembed_unavailable"] }),
});
const result = await capturedRun([
"models",
"activate",
PROFILE,
"--yes",
"--json",
]);
expect(result.code).toBe(1);
expect(JSON.parse(result.stdout)).toMatchObject({
state: "blocked",
code: "sdk_reembed_unavailable",
});
expect(transport.post).toHaveBeenCalledTimes(1);
});
it("says nothing is re-embedded when only generation changed", async () => {
transport.post.mockImplementation(async (url: string) =>
url === "/api/v1/embedding-switches/plan"
? {
status: 200,
data: plan({ needed: false, reason: "same_embedding" }),
}
: { status: 200, data: profileResponse() },
);
const result = await capturedRun(["models", "activate", PROFILE]);
expect(result.code).toBe(0);
expect(result.stdout).toContain("向量模型未变化,无需重新生成向量。");
});
it("reads the plan carried by a 409 embedding_switch_required", async () => {
transport.post.mockImplementation(async (url: string) =>
url.endsWith("/activate")
? {
status: 409,
data: { detail: "embedding_switch_required", plan: plan() },
}
: { status: 404, data: { detail: "Not Found" } },
);
const result = await capturedRun(["models", "activate", PROFILE, "--json"]);
expect(result.code).toBe(2);
expect(JSON.parse(result.stdout)).toMatchObject({
state: "confirmation_required",
code: "embedding_switch_required",
});
});
});
describe("models switch and spaces", () => {
it("reports no switch, then cancels the running one", async () => {
transport.get.mockResolvedValueOnce({
status: 200,
data: {
schema_version: 1,
phase: "idle",
writes_paused: false,
job: null,
last_finished: null,
},
});
const idle = await capturedRun(["models", "switch", "status"]);
expect(idle.code).toBe(0);
expect(idle.stdout).toContain("没有进行中的向量模型切换");
transport.get.mockResolvedValueOnce({
status: 200,
data: {
schema_version: 1,
phase: "running",
writes_paused: true,
job: job("reembedding", 20),
last_finished: null,
},
});
transport.post.mockResolvedValue({
status: 200,
data: job("reembedding", 20, { cancel_requested: true }),
});
const cancelled = await capturedRun([
"models",
"switch",
"cancel",
"--json",
]);
expect(cancelled.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
`/api/v1/embedding-switches/${JOB}/cancel`,
{},
undefined,
);
expect(JSON.parse(cancelled.stdout)).toMatchObject({
cancel_requested: true,
});
});
it("lists spaces and needs --yes before cleaning one up", async () => {
transport.get.mockResolvedValue({
status: 200,
data: {
schema_version: 1,
live_space_id: "x",
spaces: [
{
space_id: SPACE,
state: "retired",
model: "bge-small-zh-v1.5",
dims: 512,
vectors: 10,
keep_until: "2026-11-15T00:00:00+00:00",
},
{
space_id: "88888888-8888-4888-8888-888888888888",
state: "dropped",
model: "m",
dims: 8,
},
],
scanned: [],
scan_error: null,
},
});
const listed = await capturedRun(["models", "spaces"]);
expect(listed.code).toBe(0);
expect(listed.stdout).toContain("已停用");
expect(listed.stdout).toContain("保留到 2026-11-15");
expect(listed.stdout).not.toContain("88888888");
const preview = await capturedRun(["models", "spaces", "--cleanup", SPACE]);
expect(preview.code).toBe(0);
expect(preview.stdout).toContain("--yes");
expect(transport.delete).not.toHaveBeenCalled();
transport.delete.mockResolvedValue({
status: 200,
data: { schema_version: 1 },
});
const dropped = await capturedRun([
"models",
"spaces",
"--cleanup",
SPACE,
"--yes",
]);
expect(dropped.code).toBe(0);
expect(transport.delete).toHaveBeenCalledWith(
`/api/v1/embedding-spaces/${SPACE}`,
undefined,
);
});
});
describe("status and search wording", () => {
it("shows the switch line in memhub status", async () => {
const reader = async (path: string) =>
path === "/api/v1/memory-processing"
? {
state: "ok" as const,
data: {
pending: 3,
fresh_pending: 1,
backlog_pending: 2,
outside_window_pending: 0,
running: 0,
waiting_for_model: 0,
completed: 9,
failed: 0,
embedding_switch: {
job_id: JOB,
kind: "switch",
state: "reembedding",
percent: 50,
eta_s: 60,
},
},
}
: { state: "unavailable" as const, code: "http_503" };
const report = await composeStatusReport({
dataDir: "/synthetic/data",
runtime: {
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
} as never,
background: null,
reader: reader as never,
});
const text = renderStatusText(report, "MemHub");
expect(text).toContain("向量模型:切换中 50%,记忆整理已暂停");
expect(
describeEmbeddingSwitchSummary({
kind: "merge",
state: "reembedding",
percent: 30,
}),
).toBe("正在合并旧向量数据 30%,记忆整理已暂停");
expect(
describeEmbeddingSwitchSummary({
kind: "merge",
state: "scanning",
percent: 0,
}),
).toBe("正在检查向量数据…");
});
it("names an unavailable embedding service instead of 'try later'", () => {
expect(
renderContextText({
schema_version: 2,
product: "memhub",
state: "unavailable",
code: "embedding_unavailable",
items: [],
}),
).toContain("向量服务不可用");
});
});
function profileResponse() {
return {
profile_id: PROFILE,
name: "Synthetic",
provider_kind: "openai_compatible",
base_url: "https://example.invalid/v1",
model: "synthetic-model-2",
embedding_model: "text-embedding-v3",
embedding_provider_kind: "openai_compatible",
embedding_base_url: "https://example.invalid/v1",
embedding_dimensions: 1024,
has_api_key: true,
credential_state: "available",
timeout_seconds: 30,
max_concurrency: 1,
context_window: null,
disable_thinking: true,
capabilities: {
endpoint_reachable: true,
model_available: true,
structured_output: true,
embedding_available: true,
},
revision: 5,
is_active: true,
created_at: "2026-09-19T00:00:00Z",
updated_at: "2026-09-19T00:00:00Z",
};
}
async function capturedRun(
arguments_: string[],
controls: Parameters<typeof run>[2] = {},
) {
let stdout = "";
let stderr = "";
const stdoutSpy = vi
.spyOn(process.stdout, "write")
.mockImplementation((chunk) => {
stdout += String(chunk);
return true;
});
const stderrSpy = vi
.spyOn(process.stderr, "write")
.mockImplementation((chunk) => {
stderr += String(chunk);
return true;
});
try {
return {
code: await run(
arguments_,
{ ONE_MEMORY_DATA_DIR: "/synthetic/data" },
{ stdinIsTTY: false, ...controls },
),
stdout,
stderr,
};
} finally {
stdoutSpy.mockRestore();
stderrSpy.mockRestore();
}
}
@@ -0,0 +1,847 @@
/**
* Switching the embedding model from the CLI: `models activate` when the
* embedding changes, `models switch status|cancel`, `models spaces`.
*
* Core owns every decision (`/api/v1/embedding-switches*`, contract
* `docs/contracts/embedding-switch-v1.md`). A change of embedding model or
* width is never applied by activation alone: Core plans a switch, the user
* confirms it, Core re-embeds every memory into a new space and only then
* makes it live. The CLI shows the plan, asks, starts the job and follows it.
*/
import type { CoreReader } from "./core-client.js";
import type {
CoreDeleteOperations,
CoreOperations,
OperationOutcome,
} from "./core-operations.js";
import { EXIT_FAILED, EXIT_OK, EXIT_USAGE } from "./exit-codes.js";
import {
activateModelProfile,
type ModelWriteDocument,
modelProfileResult,
renderModelWriteText,
} from "./model-write-flows.js";
import { PRODUCT } from "./version.js";
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const CODE = /^[a-z0-9_.:-]{1,128}$/;
const TERMINAL = new Set(["done", "rolled_back"]);
const POLL_MS = 1_000;
export interface SpaceSummary {
space_id: string | null;
model: string | null;
dims: number | null;
collection_base: string;
vectors?: number | null;
}
export interface EmbeddingSwitchPlan {
needed: boolean;
reason: string;
plan_token: string;
source: SpaceSummary;
target: SpaceSummary;
target_profile_id: string;
target_revision: number;
target_provider_kind: string;
target_local: boolean;
total_vectors: number;
users: number | null;
graph_vectors: number | null;
estimate_s: number | null;
requests: number | null;
disk_bytes_needed: number;
disk_bytes_free: number | null;
retention_days: number;
blockers: string[];
}
export interface SwitchProgress {
phase: string;
done: number;
total: number;
percent: number;
eta_s: number | null;
}
export interface EmbeddingSwitchJob {
job_id: string;
kind: "switch" | "merge";
state: string;
source: SpaceSummary;
target: SpaceSummary;
progress: SwitchProgress;
cancel_requested: boolean;
error_code: string | null;
finished_at: string | null;
}
export interface SwitchCurrent {
phase: "idle" | "scanning" | "running";
writes_paused: boolean;
job: EmbeddingSwitchJob | null;
last_finished: EmbeddingSwitchJob | null;
}
export interface SwitchIo {
reader: CoreReader;
operations: CoreOperations & CoreDeleteOperations;
emit: (text: string) => void;
asJson: boolean;
/** A terminal is attached: may prompt, and follows progress by default. */
interactive: boolean;
confirm?: (question: string) => Promise<boolean>;
wait?: (milliseconds: number, signal: AbortSignal) => Promise<void>;
signal?: AbortSignal;
}
export interface ActivateOptions {
confirmed: boolean;
/** `--wait` / `--no-wait`; undefined follows on a terminal only. */
follow?: boolean;
}
// ---------------------------------------------------------------------------
// Parsing: bounded projections of Core's answers, never passthrough.
// ---------------------------------------------------------------------------
function record(value: unknown): Record<string, unknown> | null {
return value && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: null;
}
function int(value: unknown): number | null {
return typeof value === "number" && Number.isSafeInteger(value) && value >= 0
? value
: null;
}
function str(value: unknown, max = 256): string | null {
return typeof value === "string" && value.length <= max ? value : null;
}
function space(value: unknown): SpaceSummary | null {
const item = record(value);
if (!item) return null;
return {
space_id: str(item.space_id, 64),
model: str(item.model),
dims: int(item.dims),
collection_base: str(item.collection_base) ?? "",
vectors: int(item.vectors),
};
}
export function parsePlan(value: unknown): EmbeddingSwitchPlan | null {
const plan = record(value);
if (!plan || typeof plan.needed !== "boolean") return null;
const source = space(plan.source);
const target = space(plan.target);
const token = str(plan.plan_token, 128);
const profile = str(plan.target_profile_id, 128);
const revision = int(plan.target_revision);
if (!source || !target || !token || !profile || revision === null) {
return null;
}
const blockers = Array.isArray(plan.blockers)
? plan.blockers.filter(
(item): item is string => typeof item === "string" && CODE.test(item),
)
: [];
return {
needed: plan.needed,
reason: str(plan.reason, 64) ?? "",
plan_token: token,
source,
target,
target_profile_id: profile,
target_revision: revision,
target_provider_kind: str(plan.target_provider_kind, 64) ?? "",
target_local: plan.target_local === true,
total_vectors: int(plan.total_vectors) ?? 0,
users: int(plan.users),
graph_vectors: int(plan.graph_vectors),
estimate_s: int(plan.estimate_s),
requests: int(plan.requests),
disk_bytes_needed: int(plan.disk_bytes_needed) ?? 0,
disk_bytes_free: int(plan.disk_bytes_free),
retention_days: int(plan.retention_days) ?? 14,
blockers,
};
}
export function parseJob(value: unknown): EmbeddingSwitchJob | null {
const job = record(value);
if (!job) return null;
const jobId = str(job.job_id, 64);
const state = str(job.state, 32);
const source = space(job.source);
const target = space(job.target);
const progress = record(job.progress) ?? {};
if (!jobId || !state || !source || !target) return null;
if (job.kind !== "switch" && job.kind !== "merge") return null;
const percent =
typeof progress.percent === "number" && Number.isFinite(progress.percent)
? Math.min(100, Math.max(0, progress.percent))
: 0;
return {
job_id: jobId,
kind: job.kind,
state,
source,
target,
progress: {
phase: str(progress.phase, 32) ?? "prepare",
done: int(progress.done) ?? 0,
total: int(progress.total) ?? 0,
percent,
eta_s: int(progress.eta_s),
},
cancel_requested: job.cancel_requested === true,
error_code:
typeof job.error_code === "string" && CODE.test(job.error_code)
? job.error_code
: null,
finished_at: str(job.finished_at, 64),
};
}
export function parseCurrent(value: unknown): SwitchCurrent | null {
const current = record(value);
if (!current) return null;
const phase = current.phase;
if (phase !== "idle" && phase !== "scanning" && phase !== "running") {
return null;
}
return {
phase,
writes_paused: current.writes_paused === true,
job: current.job == null ? null : parseJob(current.job),
last_finished:
current.last_finished == null ? null : parseJob(current.last_finished),
};
}
// ---------------------------------------------------------------------------
// Wording
// ---------------------------------------------------------------------------
const BLOCKER_TEXT: Record<string, string> = {
embedding_switch_running: "上一次向量模型切换还在进行,请等待完成或先取消。",
embedding_probe_required: "新配置还没有通过探测;先运行 memhub models test。",
sdk_reembed_unavailable:
"当前运行时的 SDK 不支持重新生成向量,请先升级 MemHub。",
embedding_switch_requires_migration:
"还有旧版本的向量数据没有迁移完,请先完成升级迁移。",
embedding_dimensions_unknown:
"新配置的向量维度未知;先探测或在配置里写明 embedding_dimensions。",
insufficient_disk_space: "磁盘空间不足,无法容纳新旧两份向量数据。",
};
const FAILURE_TEXT: Record<string, string> = {
writes_did_not_drain: "等待正在进行的记忆整理结束超时",
embedding_service_failed: "向量服务连续请求失败",
embedding_dimensions_mismatch: "向量服务返回的维度与探测到的不一致",
embedding_verification_failed: "新向量没有通过校验",
target_profile_changed: "切换期间目标配置被修改",
embedding_switch_cancelled: "已取消",
model_secret_unavailable: "读不到向量服务的 API Key",
secure_storage_unavailable: "系统钥匙串不可用",
embedding_switch_clients_busy: "记忆图谱一直被占用,无法重新生成",
};
const STATE_TEXT: Record<string, string> = {
preparing: "准备中(等待记忆整理暂停)",
reembedding: "重新生成向量",
verifying: "校验新向量",
committing: "正式切换",
failed: "正在回滚",
done: "已完成",
rolled_back: "已回滚",
};
export function blockerText(code: string): string {
return BLOCKER_TEXT[code] ?? code;
}
function spaceLabel(item: SpaceSummary, local?: boolean): string {
const dims = item.dims === null ? "维度未知" : `${item.dims} 维`;
return `${item.model ?? "未知模型"}(${dims}${local ? ",本地" : ""})`;
}
function duration(seconds: number | null): string {
if (seconds === null) return "未知";
if (seconds < 60) return `${Math.max(1, seconds)} 秒`;
return `${Math.round(seconds / 60)} 分钟`;
}
function megabytes(bytes: number | null): string {
return bytes === null ? "未知" : `${Math.ceil(bytes / (1024 * 1024))} MB`;
}
export function renderPlanText(plan: EmbeddingSwitchPlan): string {
const lines = [
`将更换向量模型:${spaceLabel(plan.source)} → ${spaceLabel(plan.target, plan.target_local)}`,
`需要用新模型重新生成 ${plan.total_vectors.toLocaleString("en-US")} 条向量` +
`${plan.users === null ? "" : `(${plan.users} 个用户)`}` +
`,预计约 ${duration(plan.estimate_s)}` +
`${plan.requests === null ? ",在本机计算,不产生费用" : `,约 ${plan.requests} 次请求`}` +
`,额外磁盘约 ${megabytes(plan.disk_bytes_needed)}。`,
];
if (plan.graph_vectors) {
lines.push(`记忆图谱另有 ${plan.graph_vectors} 条向量一并重新生成。`);
}
if (
plan.source.dims !== null &&
plan.source.dims === plan.target.dims &&
plan.source.model !== plan.target.model
) {
lines.push("新旧模型维度相同,但向量不能通用,仍需重新生成。");
}
lines.push(
"切换期间记忆整理暂停,检索继续使用旧模型;完成并校验通过后才会生效。",
`旧的向量数据保留 ${plan.retention_days} 天,可用 memhub models spaces 清理。`,
);
for (const blocker of plan.blockers) {
lines.push(`无法开始:${blockerText(blocker)}`);
}
return `${lines.join("\n")}\n`;
}
export function renderProgressLine(job: EmbeddingSwitchJob): string {
const what = job.kind === "merge" ? "正在合并旧向量数据" : "正在切换向量模型";
const state = STATE_TEXT[job.state] ?? job.state;
if (job.state === "reembedding" && job.progress.total > 0) {
const eta =
job.progress.eta_s === null
? ""
: ` 预计还需 ${duration(job.progress.eta_s)}`;
return `${what}:${state} ${job.progress.done.toLocaleString("en-US")}/${job.progress.total.toLocaleString("en-US")} ${Math.floor(job.progress.percent)}%${eta}`;
}
return `${what}:${state}`;
}
export function renderOutcomeText(job: EmbeddingSwitchJob): string {
if (job.state === "done") {
return job.kind === "merge"
? "旧向量数据已合并,记忆整理已恢复。\n"
: `已切换到 ${job.target.model ?? "新模型"}。\n`;
}
const code = job.error_code ?? "embedding_switch_failed";
if (code === "embedding_switch_cancelled") {
return `已取消切换,继续使用 ${job.source.model ?? "旧模型"}。\n`;
}
return `切换未完成,已恢复使用 ${job.source.model ?? "旧模型"},数据未受影响。原因:${FAILURE_TEXT[code] ?? code}(${code})\n`;
}
/** One line for `memhub status`, from `memory-processing.embedding_switch`. */
export function describeEmbeddingSwitchSummary(value: unknown): string | null {
const summary = record(value);
if (!summary) return null;
const percent =
typeof summary.percent === "number" && Number.isFinite(summary.percent)
? Math.floor(summary.percent)
: 0;
if (summary.state === "scanning") return "正在检查向量数据…";
if (summary.kind === "merge") {
return `正在合并旧向量数据 ${percent}%,记忆整理已暂停`;
}
if (summary.state === "failed") return "切换未完成,正在恢复旧向量模型";
return `切换中 ${percent}%,记忆整理已暂停`;
}
// ---------------------------------------------------------------------------
// Flows
// ---------------------------------------------------------------------------
function document(
command: string,
fields: Record<string, unknown>,
): Record<string, unknown> {
return { schema_version: 1, product: PRODUCT, command, ...fields };
}
function emitDocument(io: SwitchIo, value: Record<string, unknown>): void {
io.emit(`${JSON.stringify(value)}\n`);
}
async function requestPlan(
profileId: string,
io: SwitchIo,
): Promise<
| { state: "plan"; plan: EmbeddingSwitchPlan }
| { state: "unsupported" }
| { state: "failed"; code: string }
> {
const outcome = await io.operations.post<unknown>(
"/api/v1/embedding-switches/plan",
{ profile_id: profileId },
);
if (outcome.state === "committed") {
const plan = parsePlan(outcome.data);
// Not a plan: activation itself still refuses an embedding change.
return plan ? { state: "plan", plan } : { state: "unsupported" };
}
if (outcome.state === "rejected") {
// A Core before the registry has no plan route: activation as before.
if (outcome.http_status === 404 && outcome.code === "http_404") {
return { state: "unsupported" };
}
const plan = parsePlan(record(outcome.body)?.plan);
if (plan) return { state: "plan", plan };
return { state: "failed", code: outcome.code };
}
return { state: "failed", code: outcome.code };
}
function emitActivation(io: SwitchIo, result: ModelWriteDocument): number {
io.emit(
io.asJson ? `${JSON.stringify(result)}\n` : renderModelWriteText(result),
);
return result.state === "committed" ? EXIT_OK : EXIT_FAILED;
}
/**
* `memhub models activate <id> [--yes] [--wait|--no-wait] [--json]`.
*
* Asks Core for the plan first. No embedding change: activate as before.
* An embedding change: show the plan, confirm (`--yes`, or a prompt on a
* terminal; exit 2 otherwise), start the switch and follow it.
*/
export async function activateWithEmbeddingSwitch(
profileId: string,
options: ActivateOptions,
io: SwitchIo,
): Promise<number> {
if (!UUID.test(profileId)) {
return emitActivation(
io,
await activateModelProfile(profileId, io.operations),
);
}
const planned = await requestPlan(profileId, io);
if (planned.state === "unsupported") {
const activated = await activateOrPlan(profileId, io);
return activated.state === "plan"
? confirmAndSwitch(activated.plan, options, io)
: emitActivation(io, activated.result);
}
if (planned.state === "failed") {
return reject(io, "models activate", planned.code);
}
const plan = planned.plan;
if (!plan.needed) {
if (!io.asJson && plan.reason === "same_embedding") {
io.emit("向量模型未变化,无需重新生成向量。\n");
}
const activated = await activateOrPlan(profileId, io);
if (activated.state === "plan") {
return confirmAndSwitch(activated.plan, options, io);
}
return emitActivation(io, activated.result);
}
return confirmAndSwitch(plan, options, io);
}
async function activateOrPlan(
profileId: string,
io: SwitchIo,
): Promise<
| { state: "plan"; plan: EmbeddingSwitchPlan }
| { state: "activated"; result: ModelWriteDocument }
> {
const result: OperationOutcome<unknown> = await io.operations.post<unknown>(
`/api/v1/model-profiles/${encodeURIComponent(profileId)}/activate`,
{},
);
if (
result.state === "rejected" &&
result.code === "embedding_switch_required"
) {
const plan = parsePlan(record(result.body)?.plan);
if (plan) return { state: "plan", plan };
}
return { state: "activated", result: modelProfileResult(profileId, result) };
}
function reject(io: SwitchIo, command: string, code: string): number {
if (io.asJson) {
emitDocument(io, document(command, { state: "rejected", code }));
} else {
io.emit(`${command} 未完成:${blockerText(code)}(${code})\n`);
}
return EXIT_FAILED;
}
async function confirmAndSwitch(
plan: EmbeddingSwitchPlan,
options: ActivateOptions,
io: SwitchIo,
): Promise<number> {
if (plan.blockers.length > 0) {
if (io.asJson) {
emitDocument(
io,
document("models activate", {
state: "blocked",
code: plan.blockers[0],
plan,
}),
);
} else {
io.emit(renderPlanText(plan));
}
return EXIT_FAILED;
}
if (!options.confirmed) {
const ask = io.interactive && !io.asJson ? io.confirm : undefined;
if (ask === undefined) {
if (io.asJson) {
emitDocument(
io,
document("models activate", {
state: "confirmation_required",
code: "embedding_switch_required",
plan,
next_step: `${PRODUCT} models activate ${plan.target_profile_id} --yes`,
}),
);
} else {
io.emit(`${renderPlanText(plan)}重新运行并加上 --yes 以确认\n`);
}
return EXIT_USAGE;
}
io.emit(renderPlanText(plan));
if (!(await ask("继续?[y/N] "))) {
io.emit("未开始切换,仍在使用原来的向量模型。\n");
return EXIT_OK;
}
}
const started = await io.operations.post<unknown>(
"/api/v1/embedding-switches",
{
profile_id: plan.target_profile_id,
expected_revision: plan.target_revision,
plan_token: plan.plan_token,
requested_by: "cli",
},
);
if (started.state !== "committed") {
return reject(io, "models activate", started.code);
}
const job = parseJob(started.data);
if (!job) return reject(io, "models activate", "embedding_switch_invalid");
const follow = options.follow ?? io.interactive;
if (!follow) {
if (io.asJson) {
emitDocument(io, document("models activate", progressFields(job)));
} else {
io.emit(
`已开始切换向量模型(${job.job_id})。用 memhub models switch status 查看进度。\n`,
);
}
return EXIT_OK;
}
return followJob(job, io, "models activate");
}
function progressFields(job: EmbeddingSwitchJob): Record<string, unknown> {
return {
state: job.state,
job_id: job.job_id,
kind: job.kind,
phase: job.progress.phase,
percent: job.progress.percent,
done: job.progress.done,
total: job.progress.total,
eta_s: job.progress.eta_s,
...(job.error_code ? { code: job.error_code } : {}),
};
}
async function readCurrent(io: SwitchIo): Promise<SwitchCurrent | null> {
const outcome = await io.reader<unknown>(
"/api/v1/embedding-switches/current",
);
return outcome.state === "ok" ? parseCurrent(outcome.data) : null;
}
/** Poll `current` until the job ends or the user stops watching. */
async function followJob(
started: EmbeddingSwitchJob,
io: SwitchIo,
command: string,
): Promise<number> {
const wait = io.wait ?? defaultWait;
const signal = io.signal ?? new AbortController().signal;
let last = "";
let job = started;
const show = (value: EmbeddingSwitchJob) => {
const line = io.asJson
? JSON.stringify(document(command, progressFields(value)))
: renderProgressLine(value);
if (line !== last) {
io.emit(`${line}\n`);
last = line;
}
};
show(job);
if (!io.asJson) {
io.emit(
"(Ctrl-C 只停止显示;用 `memhub models switch cancel` 取消切换)\n",
);
}
while (!TERMINAL.has(job.state)) {
try {
await wait(POLL_MS, signal);
} catch {
// Aborted below.
}
if (signal.aborted) {
if (!io.asJson) {
io.emit(
"已停止显示进度;切换仍在后台进行。用 `memhub models switch cancel` 取消。\n",
);
}
return EXIT_OK;
}
const current = await readCurrent(io);
if (current === null) continue;
const next =
current.job?.job_id === job.job_id
? current.job
: current.last_finished?.job_id === job.job_id
? current.last_finished
: null;
if (next === null) continue;
job = next;
show(job);
}
if (io.asJson) {
emitDocument(
io,
document(command, {
...progressFields(job),
result: job.state === "done" ? "switched" : "rolled_back",
}),
);
} else {
io.emit(renderOutcomeText(job));
}
return job.state === "done" ? EXIT_OK : EXIT_FAILED;
}
function defaultWait(milliseconds: number, signal: AbortSignal): Promise<void> {
return new Promise((resolve, reject) => {
if (signal.aborted) {
reject(new Error("aborted"));
return;
}
const timer = setTimeout(() => {
signal.removeEventListener("abort", onAbort);
resolve();
}, milliseconds);
const onAbort = () => {
clearTimeout(timer);
reject(new Error("aborted"));
};
signal.addEventListener("abort", onAbort, { once: true });
});
}
/** `memhub models switch status [--json]`. */
export async function switchStatus(io: SwitchIo): Promise<number> {
const outcome = await io.reader<unknown>(
"/api/v1/embedding-switches/current",
);
if (outcome.state !== "ok")
return reject(io, "models switch status", outcome.code);
const current = parseCurrent(outcome.data);
if (!current) {
return reject(io, "models switch status", "embedding_switch_invalid");
}
if (io.asJson) {
emitDocument(
io,
document("models switch status", {
state: current.phase,
writes_paused: current.writes_paused,
job: current.job ? progressFields(current.job) : null,
last_finished: current.last_finished
? progressFields(current.last_finished)
: null,
}),
);
return EXIT_OK;
}
if (current.job) {
io.emit(
`${renderProgressLine(current.job)}\n记忆整理已暂停,检索不受影响。\n`,
);
} else if (current.phase === "scanning") {
io.emit("正在检查向量数据…\n");
} else if (current.last_finished) {
io.emit(
`没有进行中的向量模型切换。上一次:${renderOutcomeText(current.last_finished)}`,
);
} else {
io.emit("没有进行中的向量模型切换。\n");
}
return EXIT_OK;
}
/** `memhub models switch cancel [--json]`: cancel the running job, if any. */
export async function switchCancel(io: SwitchIo): Promise<number> {
const current = await readCurrent(io);
if (current === null) {
return reject(io, "models switch cancel", "embedding_switch_unavailable");
}
if (!current.job) {
return reject(io, "models switch cancel", "embedding_switch_not_found");
}
const outcome = await io.operations.post<unknown>(
`/api/v1/embedding-switches/${current.job.job_id}/cancel`,
{},
);
if (outcome.state !== "committed") {
return reject(io, "models switch cancel", outcome.code);
}
const job = parseJob(outcome.data) ?? current.job;
if (io.asJson) {
emitDocument(
io,
document("models switch cancel", {
...progressFields(job),
cancel_requested: true,
}),
);
} else {
io.emit("已请求取消;当前批次完成后回滚,继续使用原来的向量模型。\n");
}
return EXIT_OK;
}
interface SpaceRow {
space_id: string;
state: string;
model: string | null;
dims: number | null;
vectors: number | null;
keep_until: string | null;
}
function spaceRows(value: unknown): SpaceRow[] | null {
const body = record(value);
if (!body || !Array.isArray(body.spaces)) return null;
const rows: SpaceRow[] = [];
for (const item of body.spaces) {
const row = record(item);
const spaceId = str(row?.space_id, 64);
const state = str(row?.state, 16);
if (!row || !spaceId || !state) return null;
if (state === "dropped") continue;
rows.push({
space_id: spaceId,
state,
model: str(row.model),
dims: int(row.dims) ?? int(row.name_dims),
vectors: int(row.vectors),
keep_until: str(row.keep_until, 64),
});
}
return rows;
}
const SPACE_STATE: Record<string, string> = {
live: "使用中",
building: "正在生成",
stray: "待合并",
retired: "已停用",
};
/** `memhub models spaces [--json]`. */
export async function listSpaces(io: SwitchIo): Promise<number> {
const outcome = await io.reader<unknown>("/api/v1/embedding-spaces");
if (outcome.state !== "ok") return reject(io, "models spaces", outcome.code);
const rows = spaceRows(outcome.data);
if (!rows) return reject(io, "models spaces", "embedding_spaces_invalid");
const body = record(outcome.data) ?? {};
const unregistered = Array.isArray(body.scanned)
? body.scanned
.map(record)
.filter(
(item): item is Record<string, unknown> =>
item !== null && item.state === "unregistered",
)
.map((item) => ({
collection_base: str(item.collection_base) ?? "",
vectors: int(item.vectors) ?? 0,
}))
: [];
if (io.asJson) {
emitDocument(
io,
document("models spaces", {
state: "ok",
spaces: rows,
unregistered,
scan_error:
typeof body.scan_error === "string" && CODE.test(body.scan_error)
? body.scan_error
: null,
}),
);
return EXIT_OK;
}
const lines = ["向量空间:"];
for (const row of rows) {
const dims = row.dims === null ? "维度未知" : `${row.dims} 维`;
const keep =
row.state === "retired" && row.keep_until
? `,保留到 ${row.keep_until.slice(0, 10)}`
: "";
lines.push(
` ${row.space_id} ${SPACE_STATE[row.state] ?? row.state} ${row.model ?? "未知模型"}(${dims}) ${row.vectors ?? "?"} 条${keep}`,
);
}
for (const item of unregistered) {
lines.push(
` (未登记)${item.collection_base} ${item.vectors} 条,下次启动时合并`,
);
}
if (rows.some((row) => row.state === "retired")) {
lines.push(
"清理已停用的空间:memhub models spaces --cleanup <space-id> --yes",
);
}
io.emit(`${lines.join("\n")}\n`);
return EXIT_OK;
}
/** `memhub models spaces --cleanup <space-id> --yes`: drop a retired space now. */
export async function cleanupSpace(
spaceId: string,
io: SwitchIo,
): Promise<number> {
if (!UUID.test(spaceId)) {
return reject(io, "models spaces", "embedding_space_not_found");
}
const outcome = await io.operations.delete<unknown>(
`/api/v1/embedding-spaces/${spaceId}`,
);
if (outcome.state !== "committed") {
return reject(io, "models spaces", outcome.code);
}
if (io.asJson) {
emitDocument(
io,
document("models spaces", { state: "dropped", space_id: spaceId }),
);
} else {
io.emit(`已清理旧向量数据:${spaceId}\n`);
}
return EXIT_OK;
}
+149 -17
View File
@@ -510,15 +510,98 @@ describe("models add", () => {
expect(result.stdout + result.stderr).not.toContain("SYNTHETIC_LOOSE_KEY");
});
it("activates the new profile with --activate", async () => {
it("probes the new profile before activating it with --activate", async () => {
const configFile = await modelConfigFile();
transport.post.mockImplementation(async (url: string) =>
url === "/api/v1/model-profiles"
? { status: 200, data: PROFILE }
: url === "/api/v1/model-profiles/profile-1/activate"
? { status: 200, data: { ...PROFILE, is_active: true } }
: { status: 404, data: {} },
let probeId: string | undefined;
transport.get.mockImplementation(async (url: string) => {
if (url === "/api/v1/model-profiles") {
return { status: 200, data: [PROBE_PROFILE] };
}
const match = /^\/api\/v1\/model-probe-operations\/(.+)$/.exec(url);
if (match) {
return {
status: 200,
data:
probeId === undefined
? probeNotFound(match[1])
: probeReceiptPassing(match[1]),
};
}
return { status: 404, data: {} };
});
transport.post.mockImplementation(async (url: string, body: unknown) => {
if (url === "/api/v1/model-profiles") {
return { status: 200, data: PROBE_PROFILE };
}
if (url === `/api/v1/model-profiles/${PROBE_PROFILE.profile_id}/probe`) {
probeId = (body as { operation_id: string }).operation_id;
return { status: 200, data: probePending(probeId) };
}
if (url === "/api/v1/embedding-switches/plan") {
return { status: 200, data: samePlan() };
}
if (
url === `/api/v1/model-profiles/${PROBE_PROFILE.profile_id}/activate`
) {
return {
status: 200,
data: {
...PROBE_PROFILE,
is_active: true,
embedding_model: "synthetic-embedding",
},
};
}
return { status: 404, data: {} };
});
const result = await capturedRun(
["models", "add", "--config-file", configFile, "--activate", "--json"],
{ watchWait: async () => undefined },
);
expect(result.code).toBe(0);
const lines = result.stdout
.trim()
.split("\n")
.map((line) => JSON.parse(line));
expect(lines[0].probe).toMatchObject({ state: "completed" });
expect(lines.at(-1)).toMatchObject({ state: "committed" });
expect(transport.post.mock.calls.map(([url]) => url)).toEqual([
"/api/v1/model-profiles",
`/api/v1/model-profiles/${PROBE_PROFILE.profile_id}/probe`,
"/api/v1/embedding-switches/plan",
`/api/v1/model-profiles/${PROBE_PROFILE.profile_id}/activate`,
]);
});
it("does not activate a profile whose probe fails", async () => {
const configFile = await modelConfigFile();
let probeId: string | undefined;
transport.get.mockImplementation(async (url: string) => {
if (url === "/api/v1/model-profiles") {
return { status: 200, data: [PROBE_PROFILE] };
}
const match = /^\/api\/v1\/model-probe-operations\/(.+)$/.exec(url);
if (match) {
return {
status: 200,
data:
probeId === undefined
? probeNotFound(match[1])
: probeReceipt(match[1]),
};
}
return { status: 404, data: {} };
});
transport.post.mockImplementation(async (url: string, body: unknown) => {
if (url === "/api/v1/model-profiles") {
return { status: 200, data: PROBE_PROFILE };
}
if (url.endsWith("/probe")) {
probeId = (body as { operation_id: string }).operation_id;
return { status: 200, data: probeReceipt(probeId) };
}
return { status: 404, data: {} };
});
const result = await capturedRun([
"models",
"add",
@@ -527,14 +610,12 @@ describe("models add", () => {
"--activate",
"--json",
]);
expect(result.code).toBe(0);
expect(JSON.parse(result.stdout).activation).toEqual({
state: "committed",
});
expect(transport.post.mock.calls.map(([url]) => url)).toEqual([
"/api/v1/model-profiles",
"/api/v1/model-profiles/profile-1/activate",
]);
expect(result.code).toBe(1);
expect(
transport.post.mock.calls.some(([url]) =>
String(url).endsWith("/activate"),
),
).toBe(false);
});
it("recovers an unknown create through its operation receipt", async () => {
@@ -752,6 +833,54 @@ function probePending(operationId: string) {
};
}
function samePlan() {
const space = {
space_id: null,
model: "synthetic-embedding",
dims: 8,
collection_base: "",
};
return {
needed: false,
reason: "same_embedding",
plan_token: "token-1",
source: space,
target: space,
target_profile_id: PROBE_PROFILE.profile_id,
target_revision: 3,
target_provider_kind: "openai_compatible",
target_local: false,
blockers: [],
};
}
function probeReceiptPassing(operationId: string) {
const receipt = probeReceipt(operationId);
return {
...receipt,
result: {
...receipt.result,
stages: [
{
stage: "embedding",
ok: true,
code: "ok",
latency_ms: 1,
embedding_dimensions: 8,
},
],
capabilities: {
endpoint_reachable: true,
model_available: true,
structured_output: true,
embedding_available: true,
},
embedding_width: 8,
embedding_switch_required: false,
},
};
}
function probeNotFound(operationId: string) {
return {
schema_version: 1,
@@ -819,7 +948,10 @@ function importJob() {
};
}
async function capturedRun(argv: string[]) {
async function capturedRun(
argv: string[],
controls: Parameters<typeof run>[2] = {},
) {
let stdout = "";
let stderr = "";
const originalOut = process.stdout.write;
@@ -833,7 +965,7 @@ async function capturedRun(argv: string[]) {
return true;
}) as typeof process.stderr.write;
try {
const code = await run(argv, {});
const code = await run(argv, {}, controls);
return { code, stdout, stderr };
} finally {
process.stdout.write = originalOut;
@@ -0,0 +1,273 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { run } from "./cli.js";
import { parseLanguageCommand } from "./memory-language-flow.js";
import { composeStatusReport, renderStatusText } from "./status-report.js";
const transport = vi.hoisted(() => ({
get: vi.fn(),
post: vi.fn(),
put: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
}));
vi.mock("axios", () => ({
default: {
create: vi.fn(() => transport),
get: vi.fn(),
post: vi.fn(),
isAxiosError: () => false,
},
}));
const runtime = vi.hoisted(() => ({
inspectRuntimeStatus: vi.fn(),
discoverRuntimeAttachment: vi.fn(),
}));
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
...runtime,
}));
function languageState(uiLanguage: string | null, extra = {}) {
const sdk = uiLanguage === null ? "auto" : uiLanguage;
return {
ui_language: uiLanguage,
sdk_language: sdk,
languages: [
"en",
"zh-Hans",
"zh-Hant",
"ja",
"es",
"de",
"pt",
"fr",
"ko",
"ru",
"id",
],
...extra,
};
}
beforeEach(() => {
vi.clearAllMocks();
for (const method of Object.values(transport)) method.mockReset();
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
version: "1.2.3",
});
runtime.discoverRuntimeAttachment.mockResolvedValue({
instanceId: "instance-1",
apiToken: "synthetic-runtime-token",
host: "127.0.0.1",
port: 8765,
servicePid: 4242,
health: {
status: "ok",
service: "one-memory-core",
version: "1.2.3",
instance_id: "instance-1",
},
});
transport.get.mockImplementation(async (url: string) =>
url === "/api/v1/settings/ui-language"
? { status: 200, data: languageState(null) }
: { status: 404, data: {} },
);
});
describe("memhub settings language", () => {
it("shows the display language and the SDK prompt language", async () => {
const result = await capturedRun(["settings", "language", "--json"]);
expect(result.code).toBe(0);
const json = JSON.parse(result.stdout);
expect(json).toMatchObject({
command: "settings language",
state: "ok",
language: { ui_language: null, sdk_language: "auto" },
});
expect(transport.put).not.toHaveBeenCalled();
const text = (await capturedRun(["settings", "language"])).stdout;
expect(text).toContain("界面语言:未设置(SDK 自动识别)");
expect(text).toContain("SDK 提示语言:auto");
});
it("sets a language at once and reports the SDK language", async () => {
transport.put.mockResolvedValue({
status: 200,
data: languageState("ja", { rebuilt: true }),
});
const result = await capturedRun(["settings", "language", "ja", "--json"]);
expect(result.code).toBe(0);
expect(transport.put).toHaveBeenCalledWith(
"/api/v1/settings/ui-language",
{ ui_language: "ja" },
undefined,
);
expect(JSON.parse(result.stdout)).toMatchObject({
state: "committed",
rebuilt: true,
language: { ui_language: "ja", sdk_language: "ja" },
});
const text = (await capturedRun(["settings", "language", "ja"])).stdout;
expect(text).toContain("界面语言:ja(日本語)");
});
it("sends null for system and maps the legacy zh-CN code", async () => {
transport.put.mockResolvedValue({
status: 200,
data: languageState("zh-Hans", { rebuilt: false }),
});
await capturedRun(["settings", "language", "zh-CN"]);
expect(transport.put).toHaveBeenLastCalledWith(
"/api/v1/settings/ui-language",
{ ui_language: "zh-Hans" },
undefined,
);
transport.get.mockResolvedValue({
status: 200,
data: languageState("en"),
});
transport.put.mockResolvedValue({
status: 200,
data: languageState(null, { rebuilt: true }),
});
await capturedRun(["settings", "language", "system"]);
expect(transport.put).toHaveBeenLastCalledWith(
"/api/v1/settings/ui-language",
{ ui_language: null },
undefined,
);
});
it("does not write a language Core already has", async () => {
const result = await capturedRun(["settings", "language", "system"]);
expect(result.code).toBe(0);
expect(
JSON.parse(
(await capturedRun(["settings", "language", "system", "--json"]))
.stdout,
).state,
).toBe("unchanged");
expect(transport.put).not.toHaveBeenCalled();
});
it("refuses unknown codes before the Runtime", async () => {
for (const bad of [
["settings", "language", "it"],
["settings", "language", "EN"],
["settings", "language", "ja", "ko"],
]) {
expect((await capturedRun(bad)).code, bad.join(" ")).toBe(2);
}
expect(transport.put).not.toHaveBeenCalled();
});
it("parses the forms the help shows", () => {
expect(parseLanguageCommand([])).toEqual({ kind: "show" });
expect(parseLanguageCommand(["system"])).toEqual({
kind: "set",
language: null,
});
expect(parseLanguageCommand(["zh-Hant", "--json"])).toEqual({
kind: "set",
language: "zh-Hant",
});
});
});
describe("status language", () => {
it.each([
[null, "auto", "跟随系统", "auto(自动识别中/英)"],
["ja", "ja", "ja", "SDK 提示语言 ja"],
])(
"shows the UI language %s and the SDK language %s",
async (uiLanguage, sdkLanguage, uiText, sdkText) => {
const processing = {
pending: 0,
completed: 0,
failed: 0,
blocked_stages: [],
pipeline: {
mode: "pro",
configured_mode: "pro",
forced_by_env: false,
last_evolve_at: null,
graph_build: { state: "idle", users: {} },
evolve: { mode: "pro", schedule: null, running: [], users: {} },
features: {},
ui_language: uiLanguage,
sdk_language: sdkLanguage,
},
};
const report = await composeStatusReport({
dataDir: "/tmp/isolated",
runtime: { state: "running", code: "ok", host: "127.0.0.1", port: 1 },
background: null,
reader: (async (path: string) =>
path === "/api/v1/memory-processing"
? { state: "ok", data: processing }
: { state: "unavailable", code: "x" }) as never,
});
expect(report.pipeline?.ui_language).toBe(uiLanguage);
expect(report.pipeline?.sdk_language).toBe(sdkLanguage);
const text = renderStatusText(report, "MemHub");
expect(text).toContain(uiText);
expect(text).toContain(sdkText);
},
);
it("reads an older Runtime without the setting as auto", async () => {
const report = await composeStatusReport({
dataDir: "/tmp/isolated",
runtime: { state: "running", code: "ok", host: "127.0.0.1", port: 1 },
background: null,
reader: (async () => ({
state: "ok",
data: {
pipeline: {
mode: "pro",
configured_mode: "pro",
evolve: {},
graph_build: {},
},
},
})) as never,
});
expect(report.pipeline?.sdk_language).toBe("auto");
expect(report.pipeline?.ui_language).toBeNull();
});
});
async function capturedRun(arguments_: string[]) {
let stdout = "";
let stderr = "";
const stdoutSpy = vi
.spyOn(process.stdout, "write")
.mockImplementation((chunk) => {
stdout += String(chunk);
return true;
});
const stderrSpy = vi
.spyOn(process.stderr, "write")
.mockImplementation((chunk) => {
stderr += String(chunk);
return true;
});
try {
return {
code: await run(arguments_, { ONE_MEMORY_DATA_DIR: "/synthetic/data" }),
stdout,
stderr,
};
} finally {
stdoutSpy.mockRestore();
stderrSpy.mockRestore();
}
}
+185
View File
@@ -0,0 +1,185 @@
/**
* `memhub settings language [<code>|system]`.
*
* The display language the App reports to the Runtime (`ui_language`), and
* the SDK prompt language it implies. Every code, zh-Hans and en included,
* makes extraction and the other SDK prompts use that language; "system"
* (no language set) passes none, so the SDK keeps detecting Chinese or
* English as before. The App sends the language it shows, so it is only
* unset on an install without the App. A change applies at once: a write in flight
* finishes in the old language, the next ones use the new one. The App sends
* its own choice again whenever it changes there.
*/
import type { CoreReader } from "./core-client.js";
import type { CorePutOperations } from "./core-operations.js";
import { PRODUCT } from "./version.js";
export const UI_LANGUAGES = [
"en",
"zh-Hans",
"zh-Hant",
"ja",
"es",
"de",
"pt",
"fr",
"ko",
"ru",
"id",
] as const;
export type UiLanguage = (typeof UI_LANGUAGES)[number];
/** Codes earlier builds used, and the code each one is now. */
const LEGACY: Record<string, UiLanguage> = { "zh-CN": "zh-Hans" };
const NAMES: Record<UiLanguage, string> = {
en: "English",
"zh-Hans": "简体中文",
"zh-Hant": "繁體中文",
ja: "日本語",
es: "Español",
de: "Deutsch",
pt: "Português",
fr: "Français",
ko: "한국어",
ru: "Русский",
id: "Bahasa Indonesia",
};
export type LanguageCommand =
| { kind: "show" }
| { kind: "set"; language: UiLanguage | null };
function isUiLanguage(value: string): value is UiLanguage {
return (UI_LANGUAGES as readonly string[]).includes(value);
}
/** Arguments after `settings language`; null = usage. */
export function parseLanguageCommand(
values: readonly string[],
): LanguageCommand | null {
const rest = values.filter((value) => value !== "--json");
if (rest.length === 0) return { kind: "show" };
if (rest.length !== 1) return null;
const raw = rest[0] ?? "";
const value: string = Object.hasOwn(LEGACY, raw) ? LEGACY[raw] : raw;
if (value === "system") return { kind: "set", language: null };
return isUiLanguage(value) ? { kind: "set", language: value } : null;
}
export interface LanguageState {
ui_language: UiLanguage | null;
sdk_language: string;
rebuilt?: boolean;
}
function languageState(value: unknown): LanguageState | null {
if (!value || typeof value !== "object") return null;
const item = value as Record<string, unknown>;
if (
!(item.ui_language === null || isUiLanguage(String(item.ui_language))) ||
typeof item.sdk_language !== "string"
) {
return null;
}
return {
ui_language: item.ui_language as UiLanguage | null,
sdk_language: item.sdk_language,
...(typeof item.rebuilt === "boolean" ? { rebuilt: item.rebuilt } : {}),
};
}
export type LanguageDocument = {
schema_version: 1;
product: string;
command: "settings language";
} & (
| { state: "ok" | "unchanged"; language: LanguageState }
| { state: "committed"; language: LanguageState; rebuilt: boolean }
| { state: "rejected" | "unknown" | "unavailable"; code: string }
);
const BASE = {
schema_version: 1 as const,
product: PRODUCT,
command: "settings language" as const,
};
const PATH = "/api/v1/settings/ui-language";
export async function runLanguageCommand(
command: LanguageCommand,
reader: CoreReader,
operations: CorePutOperations,
): Promise<LanguageDocument> {
const read = await reader<unknown>(PATH);
if (read.state !== "ok") {
return { ...BASE, state: "unavailable", code: read.code };
}
const current = languageState(read.data);
if (current === null) {
return { ...BASE, state: "unavailable", code: "response_invalid" };
}
if (command.kind === "show") {
return { ...BASE, state: "ok", language: current };
}
if (current.ui_language === command.language) {
return { ...BASE, state: "unchanged", language: current };
}
const result = await operations.put<unknown>(PATH, {
ui_language: command.language,
});
if (result.state !== "committed") {
return { ...BASE, state: result.state, code: result.code };
}
const changed = languageState(result.data);
if (changed === null) {
return { ...BASE, state: "unknown", code: "response_invalid" };
}
return {
...BASE,
state: "committed",
language: changed,
rebuilt: changed.rebuilt === true,
};
}
export function languageExitCode(document: LanguageDocument): number {
return ["ok", "unchanged", "committed"].includes(document.state) ? 0 : 1;
}
function sdkText(sdkLanguage: string): string {
return sdkLanguage === "auto"
? "auto(SDK 自动识别中文或英文)"
: `${sdkLanguage}(提取等 SDK 提示使用该语言)`;
}
export function renderLanguageText(document: LanguageDocument): string {
if ("code" in document) {
return `界面语言未能${document.state === "unavailable" ? "读取" : "修改"}:${document.code}\n`;
}
const state = document.language;
const lines = [
`界面语言:${
state.ui_language === null
? "未设置(SDK 自动识别)"
: `${state.ui_language}(${NAMES[state.ui_language]})`
}`,
`SDK 提示语言:${sdkText(state.sdk_language)}`,
];
if (document.state === "ok") {
lines.push(
`可选:${UI_LANGUAGES.join("、")}、system。任一语言(包括 zh-Hans、en)都会让 SDK 使用该语言;system 不指定语言,SDK 自动识别中文或英文。`,
);
}
if (document.state === "unchanged") lines.push("没有需要改动的设置。");
if (document.state === "committed") {
lines.push(
document.rebuilt
? "已更新。正在进行的写入按原语言完成,之后的写入和检索使用新语言。"
: "已保存,SDK 提示语言没有变化。",
);
}
return `${lines.join("\n")}\n`;
}
+1 -1
View File
@@ -142,7 +142,7 @@ describe("memhub settings mode", () => {
["settings", "mode", "max"],
["settings", "mode", "--yes"],
["settings", "mode", "lite", "pro"],
["settings", "language", "en"],
["settings", "theme", "dark"],
]) {
expect((await capturedRun(bad)).code, bad.join(" ")).toBe(2);
}
+3
View File
@@ -372,6 +372,9 @@ export function renderMemoryAddText(document: MemoryAddDocument): string {
export function renderMemoryPageText(document: MemoryPageDocument): string {
const page = document.page;
if (page.state !== "ok") {
if (page.state === "unavailable" && page.code === "embedding_unavailable") {
return "向量服务不可用,无法检索;请检查向量模型配置。\n";
}
if (page.state === "unavailable") {
return "后台正忙,记忆列表这次没查完,不代表没有记忆,请稍后再试\n";
}
+30 -1
View File
@@ -22,6 +22,10 @@ const RESULT_KEYS = new Set([
"stages",
"capabilities",
]);
const OPTIONAL_RESULT_KEYS = [
"embedding_width",
"embedding_switch_required",
] as const;
const STAGE_KEYS = new Set([
"stage",
"ok",
@@ -272,9 +276,34 @@ function validateResult(
inputRevision: number,
): Record<string, unknown> | null {
const result = record(value);
// A Core with the embedding registry adds what the embedding stage measured
// and whether using it needs an embedding switch; older Cores send neither.
const optional = Object.fromEntries(
OPTIONAL_RESULT_KEYS.filter((key) => key in (result ?? {})).map((key) => [
key,
result?.[key],
]),
);
const required = result
? Object.fromEntries(
Object.entries(result).filter(
([key]) => !(OPTIONAL_RESULT_KEYS as readonly string[]).includes(key),
),
)
: null;
if (
!result ||
!exactKeys(result, RESULT_KEYS) ||
!required ||
!exactKeys(required, RESULT_KEYS) ||
!(
optional.embedding_width === undefined ||
optional.embedding_width === null ||
positiveInteger(optional.embedding_width)
) ||
!(
optional.embedding_switch_required === undefined ||
typeof optional.embedding_switch_required === "boolean"
) ||
result.profile_id !== profileId ||
result.input_profile_revision !== inputRevision ||
!positiveInteger(result.profile_revision) ||
+1 -1
View File
@@ -117,7 +117,7 @@ export async function editModelProfile(
return modelProfileResult(profileId, result);
}
function modelProfileResult(
export function modelProfileResult(
operationId: string,
result:
| Awaited<ReturnType<CoreOperations["post"]>>
+45 -1
View File
@@ -18,6 +18,7 @@ import {
validateProcessing,
validateSources,
} from "./core-responses.js";
import { describeEmbeddingSwitchSummary } from "./embedding-switch-flow.js";
import { PRODUCT } from "./version.js";
const MIGRATION_ISSUES_URL =
@@ -85,6 +86,10 @@ export interface PipelineSummary {
graph_build: string | null;
/** SDK features (`memory_features`) whose value differs from the default. */
features: Record<string, boolean>;
/** The App's display language; null = never chosen or "follow the system". */
ui_language: string | null;
/** The SDK prompt language: `auto` (zh/en detection) or the code passed. */
sdk_language: string;
}
const MODES = new Set(["lite", "pro", "ultra"]);
@@ -157,6 +162,10 @@ export function describePipeline(value: unknown): PipelineSummary | null {
evolve_users: users,
graph_build: typeof graph.state === "string" ? graph.state : null,
features,
ui_language: typeof item.ui_language === "string" ? item.ui_language : null,
// An older Runtime has no setting: it always left the SDK on auto.
sdk_language:
typeof item.sdk_language === "string" ? item.sdk_language : "auto",
};
}
@@ -306,6 +315,10 @@ export function renderStatusText(
);
}
if (report.pipeline) lines.push(`记忆模式:${pipelineLine(report.pipeline)}`);
const embeddingSwitch = describeEmbeddingSwitchSummary(
report.sections.processing?.embedding_switch,
);
if (embeddingSwitch !== null) lines.push(`向量模型:${embeddingSwitch}`);
if (report.summary.state === "partial") {
const notes = [
report.summary.degraded.length > 0
@@ -348,6 +361,13 @@ export function pipelineLine(pipeline: PipelineSummary): string {
.join("、")}`,
);
}
parts.push(
`;界面语言 ${pipeline.ui_language ?? "跟随系统"},SDK 提示语言 ${
pipeline.sdk_language === "auto"
? "auto(自动识别中/英)"
: pipeline.sdk_language
}`,
);
return parts.join("");
}
@@ -365,7 +385,10 @@ function sectionLine(section: Section | undefined): string {
if (section.state === "not_checked") return "未检查";
if (section.state !== "ok") return "暂时不可用,请稍后重试";
return Object.entries(section)
.filter(([key]) => key !== "state" && key !== "code")
.filter(
([key]) =>
key !== "state" && key !== "code" && key !== "embedding_switch",
)
.map(([key, value]) => `${fieldLabel(key)} ${formatValue(value)}`)
.join(" ");
}
@@ -481,9 +504,30 @@ function describeProcessing(
reason: block.reason,
count: block.count,
})),
...embeddingSwitchField(summary),
};
}
/** The running switch or merge, projected; absent when none runs. */
function embeddingSwitchField(
summary: MemoryProcessingSummary,
): Record<string, unknown> {
const value = (summary as unknown as Record<string, unknown>)
.embedding_switch;
if (!value || typeof value !== "object" || Array.isArray(value)) return {};
const item = value as Record<string, unknown>;
const kind = item.kind === "merge" ? "merge" : "switch";
const state =
typeof item.state === "string" && /^[a-z_]{1,32}$/.test(item.state)
? item.state
: "unknown";
const percent =
typeof item.percent === "number" && Number.isFinite(item.percent)
? Math.min(100, Math.max(0, item.percent))
: 0;
return { embedding_switch: { kind, state, percent } };
}
/** Projection, not passthrough: a Core that leaks a key must not make us leak. */
function describeModel(profiles: ModelProfile[]): Record<string, unknown> {
const active = profiles.find((profile) => profile.is_active);
+2 -2
View File
@@ -4,7 +4,7 @@ build-backend = "uv_build"
[project]
name = "one-memory"
version = "1.0.5"
version = "1.0.6"
description = "MemHub local-first memory core"
readme = "README.md"
requires-python = ">=3.12"
@@ -12,7 +12,7 @@ dependencies = [
"fastapi>=0.141.1",
"filelock>=3.32.4",
# Public SDK version validated against MemHub's permission and recall contracts.
"hy-memory==2.0.9",
"hy-memory==2.0.11",
# chromadb arrives through the HY SDK; locked here so an unrelated upgrade
# cannot move the SDK's store. Built-in embedding consumes numpy directly.
"chromadb==1.5.9",
+336 -7
View File
@@ -71,6 +71,9 @@ from one_memory.diagnostic_log import (
reset_trace_id,
resource_snapshot,
)
from one_memory.embedding_health import EMBEDDING_UNAVAILABLE, EmbeddingUnavailableError
from one_memory.embedding_spaces import EmbeddingSpaceStore
from one_memory.embedding_switch import EmbeddingSwitchJob, EmbeddingSwitchService
from one_memory.exception_reporting import (
install_asyncio_exception_handler,
install_threading_excepthook,
@@ -237,6 +240,7 @@ from one_memory.model_profiles import (
ModelProfilePatch,
ModelProfileStore,
ModelProfileUnavailableError,
ProviderKind,
)
from one_memory.model_request_audit import (
AuditedModelTransport,
@@ -296,6 +300,12 @@ from one_memory.source_read_access import SourceReadAccess
from one_memory.source_totals import SourceTotals
from one_memory.strict_query import forbid_unknown_query_params
from one_memory.target_schema import bootstrap_fresh_management_schema
from one_memory.ui_language import (
SDK_LANGUAGE_AUTO,
UI_LANGUAGES,
UiLanguage,
sdk_language,
)
from one_memory.user_scope import (
DEFAULT_USER_ID,
MULTI_USER_SOURCES,
@@ -330,6 +340,10 @@ class HealthResponse(BaseModel):
# failed steps are named so a stuck recovery is visible without a log.
recovery: Literal["pending", "running", "done", "failed"] = "done"
recovery_failed_steps: tuple[str, ...] = ()
# An embedding switch or a startup merge of stray spaces, while one runs
# (`{kind, state, percent, eta_s}`): the loading screen and the status
# line read it. Reads are served throughout; model writes are paused.
embedding_switch: dict[str, object] | None = None
class BackupRequest(BaseModel):
@@ -666,6 +680,9 @@ class MemoryProcessingSummary(BaseModel):
#: R10: the SDK mode and System 2 state. Optional so older readers and
#: test doubles stay valid; Core always sends it.
pipeline: "PipelineStatus | None" = None
#: The running embedding switch or merge, `{job_id, kind, state, percent,
#: eta_s}`; None when none runs. Model writes wait while it is set.
embedding_switch: dict[str, object] | None = None
class MemoryModeState(BaseModel):
@@ -748,6 +765,38 @@ class PipelineStatus(BaseModel):
#: The SDK features (`memory_features`) whose value differs from the SDK
#: default; empty while every feature is at its default.
features: dict[MemoryFeatureName, bool] = Field(default_factory=dict)
#: The display language the App last reported (`ui_language`); null =
#: none set (the App sends the language it shows, even following the system).
ui_language: UiLanguage | None = None
#: The SDK prompt language: `auto` (the SDK detects zh/en) or a code.
sdk_language: str = SDK_LANGUAGE_AUTO
class UiLanguageState(BaseModel):
"""`GET /api/v1/settings/ui-language`."""
model_config = ConfigDict(extra="forbid")
#: What the App reported; null = none set (CLI/API-only, or `system`).
ui_language: UiLanguage | None
#: What SDK clients are built with: `auto` (no language set: the SDK
#: detects zh/en, as before) or the code passed as `MemoryConfig.language`.
sdk_language: str
languages: tuple[UiLanguage, ...] = UI_LANGUAGES
class UiLanguageUpdate(BaseModel):
model_config = ConfigDict(extra="forbid")
#: A supported code, or null for "follow the system" (SDK auto-detect).
ui_language: UiLanguage | None
class UiLanguageChange(UiLanguageState):
"""`PUT /api/v1/settings/ui-language`."""
#: Whether the SDK language changed, so the SDK clients were rebuilt.
rebuilt: bool
class MemoryFeatureState(BaseModel):
@@ -1044,6 +1093,16 @@ def create_app(
# state, so initialize it before authority selection.
app.state.secret_store = secret_store or KeyringSecretStore()
app.state.model_profile_store = ModelProfileStore(database_path)
# The live embedding space, pinned before any SDK client opens.
# A home this build has not seen registers the space it already
# uses, under the name it already has (`EmbeddingSpaceStore.bootstrap`).
app.state.embedding_spaces = EmbeddingSpaceStore(database_path)
active_identity = app.state.model_profile_store.active_identity()
app.state.embedding_spaces.bootstrap(
app.state.model_profile_store.snapshot(*active_identity)
if active_identity is not None
else None
)
app.state.sdk_data_plane_migrations = SdkDataPlaneMigrationStore(database_path)
app.state.memory_authority = app.state.sdk_data_plane_migrations.authority()
_record_startup_authority(
@@ -1280,6 +1339,9 @@ def create_app(
sedimentation_enabled=lambda: _config_store(app).sedimentation_enabled(),
memory_mode=lambda: _memory_mode(app),
memory_features=lambda: _memory_features(app),
spaces=app.state.embedding_spaces,
writes_open=lambda: _embedding_writes_open(app),
sdk_language=lambda: _sdk_language(app),
)
app.state.sdk_memory_runtime = SdkMemoryRuntime(
coordinator=app.state.sdk_write_coordinator,
@@ -1288,6 +1350,7 @@ def create_app(
extraction_settings=app.state.extraction_settings_store,
data_dir=runtime_settings.data_dir,
memory_mode=lambda: _memory_mode(app),
writes_open=lambda: _embedding_writes_open(app),
)
app.state.evolve_state_store = EvolveStateStore(runtime_settings.data_dir)
app.state.evolve_service = EvolveService(
@@ -1342,6 +1405,10 @@ def create_app(
async with app.state.sedimentation_lock, source_lock:
if not _config_store(app).sedimentation_enabled():
raise RuntimeError("model_sedimentation_disabled")
if not _embedding_writes_open(app):
# Re-checked at the last boundary: a switch that began
# after the claim keeps this job queued, unsent.
raise RuntimeError("embedding_switch_in_progress")
if not _source_writes_on(app, source_id):
raise RuntimeError("model_content_source_disabled")
return await mark_invoked()
@@ -1393,6 +1460,64 @@ def create_app(
source_claim_enabled=lambda source_id: _sedimentation_claim_enabled(app, source_id),
)
app.state.memory_write_drain = memory_write_drain
async def embedding_writes_drained() -> bool:
"""No model write runs: none past its claim, none holding a client."""
await asyncio.to_thread(memory_write_queue.reclaim_expired)
running, _ = await asyncio.to_thread(app.state.sdk_memory_runtime.active_depths)
in_use = cast(SdkWriteCoordinator, app.state.sdk_write_coordinator).in_use()
return running == 0 and in_use == 0
async def quiesce_sdk_clients() -> Callable[[], None]:
"""Close every SDK client and hold the stores, for the graph re-embed."""
deadline = time.monotonic() + 120
while True:
await _read_handle(app).close_idle()
await app.state.sdk_write_coordinator.close_idle()
if hold_stores_offline(runtime_settings.data_dir):
return lambda: release_stores(runtime_settings.data_dir)
if time.monotonic() >= deadline:
raise RuntimeError("embedding_switch_clients_busy")
await asyncio.sleep(0.2)
async def after_embedding_commit(job: EmbeddingSwitchJob) -> None:
from one_memory.hy_graph_admin import forget_graph_widths
forget_graph_widths()
_read_handle(app).health.forget()
if job.kind == "switch" and job.target_profile_id is not None:
profile = await asyncio.to_thread(
_model_profile_store(app).require,
job.target_profile_id,
_secret_store(app),
)
await asyncio.to_thread(
app.state.memory_task_store.rebind_unstarted_profile, profile
)
await asyncio.to_thread(memory_write_queue.readmit_model_profile_waiters)
_resume_sedimentation(app)
_start_backfill(app)
from one_memory.embedding_switch import default_reembed_backend
app.state.embedding_switch = EmbeddingSwitchService(
database_path=database_path,
data_dir=runtime_settings.data_dir,
profiles=app.state.model_profile_store,
spaces=app.state.embedding_spaces,
secrets=app.state.secret_store,
backend=default_reembed_backend(runtime_settings.data_dir),
drain_idle=embedding_writes_drained,
on_commit=after_embedding_commit,
quiesce_clients=quiesce_sdk_clients,
# Only ultra keeps a graph; its width follows the space.
graph_enabled=lambda: _memory_mode(app) == "ultra",
worker_enabled=runtime_settings.embedding_switch_worker,
drain_timeout_s=runtime_settings.embedding_switch_drain_timeout_seconds,
)
if app.state.memory_authority is not MemoryAuthority.SDK:
# Only an SDK-authority home has SDK spaces to scan and merge.
app.state.embedding_switch.release_scan_hold()
# Started by `recover_after_ready`: a running drain claims jobs and
# holds the single writer, which the rest of startup still needs.
app.state.memory_write_drain_task = None
@@ -1631,6 +1756,12 @@ def create_app(
repair.coding_rows,
)
async def resume_embedding_spaces() -> None:
try:
await app.state.embedding_switch.recover()
finally:
app.state.embedding_switch.start_retention()
async def start_write_drain() -> None:
app.state.memory_write_drain_task = watched_task(
asyncio.get_running_loop(),
@@ -1719,6 +1850,13 @@ def create_app(
# Before the drain: the repair restores metadata the
# queued writes are read back against.
await step("sdk_metadata_repair", repair_sdk_metadata)
# Before the drain too: resume an unfinished embedding
# switch (an App update may have stopped it), or scan for
# stray spaces and start merging them. Until this decides,
# the write gate holds every model write back; reads are
# served from the live space throughout.
if sdk:
await step("embedding_spaces", resume_embedding_spaces)
await step("write_drain", start_write_drain)
if sdk:
await step("evolve_schedule", start_evolve_schedule)
@@ -1821,6 +1959,8 @@ def create_app(
await app.state.evolve_scheduler.shutdown()
await app.state.evolve_service.shutdown()
await app.state.memory_graph_backfill.shutdown()
# The job stays where it is; the next start resumes it.
await app.state.embedding_switch.shutdown()
await app.state.sdk_memory_runtime.shutdown()
await app.state.sdk_write_coordinator.close()
if app.state.sdk_client_handle is not None:
@@ -1906,6 +2046,9 @@ def create_app(
from one_memory.call_event_routes import call_event_router
app.include_router(call_event_router(protected))
from one_memory.embedding_switch_routes import embedding_switch_router
app.include_router(embedding_switch_router(protected))
app.add_middleware(
CORSMiddleware,
@@ -1999,6 +2142,12 @@ def create_app(
},
)
@app.exception_handler(EmbeddingUnavailableError)
async def embedding_unavailable(_: Request, _error: EmbeddingUnavailableError) -> JSONResponse:
# No lexical fallback: a search that cannot embed its query fails,
# and says why, so the App can show it (top-right toast).
return JSONResponse(status_code=503, content={"detail": EMBEDDING_UNAVAILABLE})
@app.exception_handler(sqlite3.OperationalError)
async def sqlite_error(request: Request, error: sqlite3.OperationalError) -> JSONResponse:
record_database_error("core.api", error, route_path=request.url.path)
@@ -2044,6 +2193,7 @@ def create_app(
),
recovery=getattr(app.state, "recovery", "pending"),
recovery_failed_steps=getattr(app.state, "recovery_failed_steps", ()),
embedding_switch=_embedding_switch_summary(app),
)
@app.get("/health", response_model=HealthResponse, include_in_schema=False)
@@ -3322,6 +3472,7 @@ def create_app(
profile_id: str,
request: ModelProfilePatch,
) -> ModelProfile:
_refuse_embedding_edit_during_switch(app, profile_id, request)
try:
return _model_profile_store(app).update(
profile_id,
@@ -3342,25 +3493,62 @@ def create_app(
@app.post(
"/api/v1/model-profiles/{profile_id}/activate",
response_model=ModelProfile,
response_model=None,
dependencies=protected,
)
async def activate_model_profile(profile_id: str) -> ModelProfile:
async def activate_model_profile(profile_id: str) -> Any:
# Asynchronous for the sake of what follows the activation: scheduling
# work needs the event loop this coroutine runs on, and a synchronous
# path operation is handed to a worker thread that has none. Activating
# with any evidence already imported — the ordinary order, since
# importing comes first — used to fail here outright.
#
# Activation never changes the embedding space by itself. A profile
# with the live space's model and width (a generation-only change, or
# another endpoint for the same model) activates as before. Another
# embedding on a home with no vector at all moves the space in the
# same transaction. Anything else answers 409 `embedding_switch_required`
# with the plan: `POST /api/v1/embedding-switches` carries it out.
from one_memory.embedding_switch import EmbeddingSwitchError
from one_memory.embedding_switch_routes import refusal
switch = _embedding_switch(app)
try:
profile = await asyncio.to_thread(
_model_profile_store(app).activate,
profile_id,
_secret_store(app),
)
# The refusals activation always made come first, with their codes.
await asyncio.to_thread(_store_activation_preflight, app, profile_id)
plan = await switch.plan(profile_id)
except KeyError:
raise HTTPException(status_code=404, detail="model_profile_not_found") from None
except ModelProfileUnavailableError as error:
raise HTTPException(status_code=409, detail=str(error)) from None
except EmbeddingSwitchError as error:
if error.code == "model_profile_not_found":
raise HTTPException(status_code=404, detail="model_profile_not_found") from None
return refusal(error.code, plan=error.plan)
if "embedding_switch_running" in plan.blockers:
return refusal("embedding_switch_running", plan=plan)
try:
if plan.needed:
return refusal("embedding_switch_required", plan=plan)
if plan.reason == "empty_home":
await asyncio.to_thread(switch.activate_empty_home, profile_id)
profile = await asyncio.to_thread(
_model_profile_store(app).require, profile_id, _secret_store(app)
)
else:
profile = await asyncio.to_thread(
_model_profile_store(app).activate,
profile_id,
_secret_store(app),
)
except KeyError:
raise HTTPException(status_code=404, detail="model_profile_not_found") from None
except ModelProfileUnavailableError as error:
raise HTTPException(status_code=409, detail=str(error)) from None
except ModelProfileConflictError:
raise HTTPException(status_code=409, detail="model_profile_conflict") from None
except EmbeddingSwitchError as error:
return refusal(error.code)
# D75: work queued for the previous model continues with this one.
# Only tasks that never reached the model are re-bound.
await asyncio.to_thread(app.state.memory_task_store.rebind_unstarted_profile, profile)
@@ -3933,6 +4121,7 @@ def create_app(
if request.task is not None and request.kind in {"chat", "fact"}:
raise HTTPException(status_code=422, detail="memory_task_requires_experience")
namespace = _namespace_kwargs(request.kind, request.variant)
_refuse_write_during_embedding_switch(app)
try:
async with app.state.sdk_write_coordinator.use_client() as client:
# `update` takes no user: read the memory first and refuse
@@ -3988,6 +4177,7 @@ def create_app(
user_id: str = Query(default=DEFAULT_USER_ID, pattern=USER_ID_PATTERN),
) -> Response:
_check_memory_namespace(kind, variant)
_refuse_write_during_embedding_switch(app)
try:
async with app.state.sdk_write_coordinator.use_client() as client:
# `delete` takes no user. Another user's memory is answered
@@ -4296,6 +4486,7 @@ def create_app(
in_flight_count=in_flight_count,
metrics=ProcessingMetrics.model_validate(store.processing_metrics()),
pipeline=_pipeline_status(app),
embedding_switch=_embedding_switch_summary(app),
)
@app.get(
@@ -4408,6 +4599,53 @@ def create_app(
changed=changed,
)
@app.get(
"/api/v1/settings/ui-language",
response_model=UiLanguageState,
dependencies=strict,
)
def get_ui_language() -> UiLanguageState:
"""The App's display language and the SDK prompt language it implies."""
return _ui_language_state(app)
@app.put(
"/api/v1/settings/ui-language",
response_model=UiLanguageChange,
dependencies=strict,
)
async def put_ui_language(request: UiLanguageUpdate) -> UiLanguageChange:
"""Record the App's display language; one global setting, not per user.
Every code (zh-Hans and en included) is passed as
`MemoryConfig.language`; only null passes no language, so the SDK
keeps detecting zh/en as before. A change of that SDK language rebuilds the
SDK clients like a feature change: a write in flight finishes in the
old language. Nothing stored is migrated and no collection is renamed.
"""
store = _config_store(app)
async with app.state.memory_mode_lock:
previous = _sdk_language(app)
if store.ui_language() != request.ui_language:
await asyncio.to_thread(store.set_ui_language, request.ui_language)
current = _sdk_language(app)
rebuilt = current != previous
if rebuilt:
coordinator = getattr(app.state, "sdk_write_coordinator", None)
if coordinator is not None:
await coordinator.retire_stale_handles()
# Reads rebuild on their next request (`ActiveProfileReadHandle`).
diagnostic_event(
level="info",
component="memory.language",
event="sdk_language.changed",
outcome="succeeded",
details={
"sdk_language": current or SDK_LANGUAGE_AUTO,
"previous": previous or SDK_LANGUAGE_AUTO,
},
)
return UiLanguageChange(**_ui_language_state(app).model_dump(), rebuilt=rebuilt)
@app.get(
"/api/v1/settings/evolve-schedule",
response_model=EvolveSchedule,
@@ -4545,6 +4783,7 @@ def create_app(
assumption that cost a live graph once was made in code, not by a user.
"""
supervisor = _sdk_memory_runtime(app)
_refuse_write_during_embedding_switch(app)
try:
outcome = await supervisor.decide_graph_width(request.decision)
except HySdkError as error:
@@ -5768,6 +6007,8 @@ def _sdk_client_handle(app: FastAPI) -> SdkClientHandle:
data_dir=cast(Settings, app.state.settings).data_dir,
memory_mode=lambda: _memory_mode(app),
features=lambda: _memory_features(app),
spaces=getattr(app.state, "embedding_spaces", None),
language=lambda: _sdk_language(app),
)
app.state.sdk_client_handle = handle
return handle
@@ -5810,6 +6051,20 @@ def _memory_features(app: FastAPI) -> FeatureOverrides:
return store.memory_features().overrides() if store is not None else ()
def _sdk_language(app: FastAPI) -> str | None:
"""The SDK prompt language every client is built with; None = auto-detect."""
store = getattr(app.state, "config_store", None)
return sdk_language(store.ui_language()) if store is not None else None
def _ui_language_state(app: FastAPI) -> UiLanguageState:
store = getattr(app.state, "config_store", None)
return UiLanguageState(
ui_language=store.ui_language() if store is not None else None,
sdk_language=_sdk_language(app) or SDK_LANGUAGE_AUTO,
)
def _memory_features_state(
app: FastAPI, defaults: dict[str, bool] | None = None
) -> MemoryFeaturesState:
@@ -5865,9 +6120,14 @@ def _pipeline_status(app: FastAPI) -> PipelineStatus | None:
dict[MemoryFeatureName, bool],
non_default_features(_memory_features(app), sdk_feature_defaults()),
),
**_ui_language_state(app).model_dump(exclude={"languages"}),
)
def _read_handle(app: FastAPI) -> ActiveProfileReadHandle:
return cast(ActiveProfileReadHandle, _sdk_client_handle(app))
def _log_skipped_sdk_memories(count: int) -> None:
if count:
logger.warning("sdk_memory_invalid_rows_skipped count=%d", count)
@@ -6123,6 +6383,7 @@ def _sedimentation_claim_enabled(app: FastAPI, source_id: str) -> bool:
return (
_config_store(app).sedimentation_enabled()
and _embedding_writes_open(app)
and (_memory_mode(app) == "lite" or _model_profile_store(app).active_identity() is not None)
and _history_import_gate_released(app)
and _source_writes_on(app, source_id)
@@ -6137,6 +6398,74 @@ def _user_evolving(app: FastAPI, user_id: str) -> bool:
return bool(callable(evolving) and evolving(user_id))
def _embedding_switch(app: FastAPI) -> EmbeddingSwitchService:
return cast(EmbeddingSwitchService, app.state.embedding_switch)
def _embedding_writes_open(app: FastAPI) -> bool:
"""False while an embedding switch or merge runs, or the startup scan has not decided.
Intake is not affected: conversations keep being queued, and sediment
into the live space once the gate opens again.
"""
switch = getattr(app.state, "embedding_switch", None)
return switch is None or switch.writes_open()
def _embedding_switch_summary(app: FastAPI) -> dict[str, object] | None:
switch = getattr(app.state, "embedding_switch", None)
if switch is None:
return None
try:
return cast(EmbeddingSwitchService, switch).summary()
except sqlite3.Error:
return None
def _refuse_write_during_embedding_switch(app: FastAPI) -> None:
"""Direct edits and deletes wait for the switch: 409, never a write into the old space."""
if not _embedding_writes_open(app):
raise HTTPException(status_code=409, detail="embedding_switch_in_progress")
_EMBEDDING_PATCH_FIELDS = frozenset(
{"embedding_model", "embedding_provider_kind", "embedding_base_url", "embedding_dimensions"}
)
def _refuse_embedding_edit_during_switch(
app: FastAPI, profile_id: str, patch: ModelProfilePatch
) -> None:
"""The profiles a running switch reads from or commits to keep their embedding."""
if not (_EMBEDDING_PATCH_FIELDS & patch.model_fields_set):
return
switch = getattr(app.state, "embedding_switch", None)
if switch is None:
return
row = cast(EmbeddingSwitchService, switch).jobs.open_job()
if row is None:
return
active = _model_profile_store(app).active_identity()
if profile_id == row["target_profile_id"] or (active is not None and active[0] == profile_id):
raise HTTPException(status_code=409, detail="embedding_switch_running")
def _store_activation_preflight(app: FastAPI, profile_id: str) -> None:
"""The checks `ModelProfileStore.activate` makes, for the empty-home path."""
profile = _model_profile_store(app).require(profile_id, _secret_store(app))
if profile.provider_kind in {ProviderKind.HY_DEFAULT, ProviderKind.ANTHROPIC}:
raise ModelProfileUnavailableError("model_provider_not_supported")
if not profile.embedding_model:
raise ModelProfileUnavailableError("embedding_model_required")
capabilities = profile.capabilities
if not (
capabilities.endpoint_reachable
and capabilities.model_available
and capabilities.embedding_available
):
raise ModelProfileUnavailableError("model_profile_probe_required")
def _history_import_gate_released(app: FastAPI) -> bool:
"""Hold first sedimentation until every accepted history import is terminal."""
+26 -3
View File
@@ -35,7 +35,7 @@ import re # noqa: E402
import time # noqa: E402
from collections.abc import Iterator, Sequence # noqa: E402
from pathlib import Path # noqa: E402
from typing import TextIO # noqa: E402
from typing import Any, TextIO # noqa: E402
from one_memory.config import load_settings # noqa: E402
from one_memory.runtime_client import RuntimeStatus, inspect_runtime # noqa: E402
@@ -609,6 +609,14 @@ def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
else:
state = "required"
document = {"schema_version": 1, "authority": authority.value, "state": state}
from one_memory.embedding_switch import read_open_job
# Informational. An unfinished embedding switch never makes a
# migration required or blocks it: the upgrade installs, `sdk-migrate`
# leaves the switch's tables alone, and the new Core resumes the job.
switch = read_open_job(database_path)
if switch is not None:
document["embedding_switch"] = switch
from one_memory.sdk_migration_log import write_migration_log
write_migration_log(
@@ -616,7 +624,15 @@ def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
writer="core-sdk-migration-status",
event="status.probed",
outcome="succeeded",
details={"authority": document["authority"], "state": document["state"]},
details={
"authority": document["authority"],
"state": document["state"],
**(
{"embedding_switch": document["embedding_switch"]}
if "embedding_switch" in document
else {}
),
},
)
if as_json:
stream.write(f"{json.dumps(document, ensure_ascii=False)}\n")
@@ -696,6 +712,12 @@ def _refuse_unmigrated_home(data_dir: Path) -> None:
raise SystemExit(SDK_MIGRATION_REQUIRED_EXIT)
def _without_embedding_switch(settings: Any) -> Any:
"""The maintenance Core's settings: it never resumes or runs a switch."""
copy = getattr(settings, "model_copy", None)
return copy(update={"embedding_switch_worker": False}) if callable(copy) else settings
def serve(*, maintenance: bool = False) -> None:
import uvicorn
@@ -740,7 +762,8 @@ def serve(*, maintenance: bool = False) -> None:
if launcher is not None:
ParentWatchdog(launcher).start()
application = create_app(
settings,
# Only the serving Core runs or resumes an embedding switch.
_without_embedding_switch(settings) if maintenance else settings,
manage_runtime=not maintenance,
)
if maintenance:
@@ -45,6 +45,11 @@ class Settings(BaseSettings):
model_request_limit: int | None = Field(default=None, ge=1, le=100_000)
model_request_body_byte_limit: int = Field(default=24_576, ge=1)
model_request_output_token_limit: int = Field(default=16_384, ge=1)
# The embedding switch worker (and the startup merge). The maintenance
# Core turns it off: only the serving Core may resume or run a switch.
embedding_switch_worker: bool = True
# How long a switch waits for model writes already past their claim.
embedding_switch_drain_timeout_seconds: float = Field(default=600, gt=0)
@field_validator("development_origin")
@classmethod
@@ -19,6 +19,7 @@ from one_memory.context_models import (
SdkMemoryContextItem,
SdkRawContextItem,
)
from one_memory.embedding_health import EMBEDDING_UNAVAILABLE, EmbeddingUnavailableError
from one_memory.external_memory_access import external_access
from one_memory.hy_runtime_config import HY_MEMORY_READER
from one_memory.hy_sdk_runtime import sdk_relevance_floor
@@ -263,6 +264,10 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
)
await record(final, timed_out=True)
return final
for lane in (raw_items, memory_items):
if isinstance(lane, EmbeddingUnavailableError):
# No lexical fallback: the query could not be embedded.
raise HTTPException(503, EMBEDDING_UNAVAILABLE)
partial = isinstance(raw_items, Exception) or isinstance(memory_items, Exception)
raw_candidates = (
() if isinstance(raw_items, Exception) else cast(tuple[SdkContextItem, ...], raw_items)
@@ -0,0 +1,180 @@
"""What the embedding switch needs from the SDK's re-embed module, in Core terms.
The switch worker, the startup merge and the retention sweep only talk to a
`ReembedBackend`. The production one (`hy_sdk_migration.SdkReembedBackend`)
is the single place that calls `hy_memory`'s re-embed functions; tests use
an in-memory one. Every space is named explicitly -- collection names and
graph paths -- and nothing here derives a name from a provider or endpoint.
"""
from __future__ import annotations
from collections.abc import Callable, Mapping
from dataclasses import dataclass, field
from pathlib import Path
from typing import Protocol
from one_memory.embedding_spaces import EmbeddingSpace, space_collection_names
from one_memory.model_profiles import ModelProfileSnapshot
class ReembedUnavailableError(RuntimeError):
"""The installed SDK has no re-embed module (before hy-memory 2.0.10)."""
def __init__(self, code: str = "sdk_reembed_unavailable") -> None:
super().__init__(code)
self.code = code
class ReembedCancelledError(RuntimeError):
"""A cancel was seen at a batch boundary; every finished batch is written."""
class EmbeddingWidthError(RuntimeError):
"""The embedding service returned vectors of another width than the target's."""
@dataclass(frozen=True)
class PhysicalSpace:
"""One space's collections, named explicitly."""
collection_base: str
name_dims: int | None
#: The width of the vectors in it, or to be written to it.
dims: int | None
model: str | None = None
#: Explicit role -> collection name; None derives them from the base.
collections: Mapping[str, str] | None = None
def names(self) -> dict[str, str]:
if self.collections is not None:
return dict(self.collections)
return space_collection_names(self.collection_base, self.name_dims)
@classmethod
def of(cls, space: EmbeddingSpace) -> PhysicalSpace:
return cls(
collection_base=space.collection_base,
name_dims=space.name_dims,
dims=space.dims if space.dims is not None else space.name_dims,
model=space.model,
)
@dataclass(frozen=True)
class ScannedSpace:
"""A space found in the vector store, grouped from its collection names."""
collection_base: str
name_dims: int | None
#: The stored width when every non-empty role agrees.
width: int | None
counts: Mapping[str, int]
collections: Mapping[str, str]
per_user: Mapping[str, int] = field(default_factory=dict)
@property
def vectors(self) -> int:
return sum(self.counts.values())
@property
def key(self) -> tuple[str, int]:
return (self.collection_base, self.name_dims or 0)
def physical(self) -> PhysicalSpace:
return PhysicalSpace(
collection_base=self.collection_base,
name_dims=self.name_dims,
dims=self.width if self.width is not None else self.name_dims,
collections=dict(self.collections),
)
@dataclass(frozen=True)
class ReembedStep:
"""Progress of one role (vectors) or table (graph)."""
phase: str
role: str
done: int
total: int
embedded: int = 0
reused: int = 0
kept: int = 0
@dataclass(frozen=True)
class VectorCheck:
per_role: Mapping[str, Mapping[str, int]]
#: Target records per metadata `user_id`.
per_user: Mapping[str, int]
problems: tuple[str, ...]
warnings: tuple[str, ...] = ()
self_recall_hits: int = 0
self_recall_samples: int = 0
@dataclass(frozen=True)
class GraphCheck:
ok: bool
problems: tuple[str, ...] = ()
@dataclass(frozen=True)
class EmbeddingTarget:
"""The embedding that fills a space: the profile to reach it, and its key."""
profile: ModelProfileSnapshot
space: EmbeddingSpace
api_key: str | None
Cancel = Callable[[], bool]
OnStep = Callable[[ReembedStep], None]
class ReembedBackend(Protocol):
def unavailable_reason(self) -> str | None:
"""None when re-embedding works; otherwise the code that says why not."""
async def scan(self, *, per_user: bool = False) -> tuple[ScannedSpace, ...]:
"""Every space with collections in the home's vector store. Read-only."""
async def diff(self, source: PhysicalSpace, target: PhysicalSpace) -> dict[str, int]:
"""Per role, how many ids the source holds that the target does not."""
async def reembed(
self,
source: PhysicalSpace,
target: PhysicalSpace,
*,
embedding: EmbeddingTarget,
cancel: Cancel,
on_step: OnStep,
batch_size: int = 64,
) -> None:
"""Embed every source record the target lacks into the target. Target wins."""
async def verify(
self, source: PhysicalSpace, target: PhysicalSpace, *, embedding: EmbeddingTarget
) -> VectorCheck: ...
async def graph_vectors(self, path: Path) -> int | None:
"""Embeddings a graph holds, or None when there is no graph there."""
async def reembed_graph(
self,
source: Path,
target: Path,
*,
embedding: EmbeddingTarget,
target_dims: int,
cancel: Cancel,
on_step: OnStep,
) -> None: ...
async def verify_graph(self, source: Path, target: Path, *, target_dims: int) -> GraphCheck: ...
async def drop(self, space: PhysicalSpace) -> None: ...
async def drop_graph(self, path: Path) -> None: ...
@@ -0,0 +1,147 @@
"""`embedding_unavailable`: a search that could not embed its query says so.
With a remote embedding model, every search embeds the query through that
service. When the service is down, the SDK swallows the failure and answers
with no hits, which reads as "you have no such memory". There is no lexical
fallback (owner decision): the search fails with the explicit code
`embedding_unavailable`, the API answers 503 with that code, and the App
shows a toast.
The check is only made when a search came back empty or raised, so a working
service costs nothing on searches that find something. It sends one tiny
embedding request to the same endpoint, model and width the read client
uses; the answer is remembered briefly so a burst of searches sends one.
"""
from __future__ import annotations
import threading
import time
from collections.abc import Callable, Mapping
from typing import Any
import httpx
from one_memory.http_client import create_client
from one_memory.hy_sdk_types import HySdkError
from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind
EMBEDDING_UNAVAILABLE = "embedding_unavailable"
_OK_TTL_S = 30.0
_FAILED_TTL_S = 10.0
_TIMEOUT_S = 8.0
def _no_transport() -> httpx.BaseTransport | None:
return None
#: Tests replace this to answer the check without a network.
transport_factory: Callable[[], httpx.BaseTransport | None] = _no_transport
class EmbeddingUnavailableError(HySdkError):
"""The live space's embedding service did not answer; search cannot run."""
def __init__(self) -> None:
super().__init__(EMBEDDING_UNAVAILABLE)
class EmbeddingHealth:
"""Whether a remote embedding answers, remembered for a few seconds."""
def __init__(self) -> None:
self._lock = threading.Lock()
self._known: dict[tuple[str, str, int | None], tuple[bool, float]] = {}
def available(
self,
profile: ModelProfileSnapshot,
*,
endpoint: str | None,
name_dims: int | None,
api_key: str | None,
) -> bool:
provider = profile.embedding_provider_kind or profile.provider_kind
if provider is ProviderKind.LOCAL_ONNX or endpoint is None:
return True
model = profile.embedding_model or ""
key = (endpoint, model, name_dims)
now = time.monotonic()
with self._lock:
known = self._known.get(key)
if known is not None and known[1] > now:
return known[0]
ok = _answers(
endpoint,
model=model,
name_dims=name_dims,
api_key="ollama" if provider is ProviderKind.OLLAMA else api_key,
)
with self._lock:
self._known[key] = (ok, now + (_OK_TTL_S if ok else _FAILED_TTL_S))
return ok
def forget(self) -> None:
with self._lock:
self._known.clear()
def _answers(endpoint: str, *, model: str, name_dims: int | None, api_key: str | None) -> bool:
body: dict[str, Any] = {"model": model, "input": ["ping"]}
if name_dims:
body["dimensions"] = name_dims
headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}
transport = transport_factory()
try:
with create_client(
timeout=_TIMEOUT_S,
**({"transport": transport} if transport is not None else {}),
) as client:
response = client.post(f"{endpoint.rstrip('/')}/embeddings", json=body, headers=headers)
if response.status_code != 200:
return False
data = response.json().get("data")
return bool(isinstance(data, list) and data and data[0].get("embedding"))
except (httpx.HTTPError, ValueError, AttributeError):
return False
def has_hits(result: object) -> bool:
"""Whether an SDK search result holds any hit, whatever its scene's shape."""
if not isinstance(result, Mapping):
return bool(result)
for key in ("memories", "facts", "experiences"):
value = result.get(key)
if isinstance(value, list) and value:
return True
if isinstance(value, Mapping) and any(
isinstance(item, list) and item for item in value.values()
):
return True
return False
class EmbeddingGuardedClient:
"""An SDK client whose `search` reports a dead embedding instead of nothing.
Everything else is the client's own attribute, unchanged.
"""
def __init__(self, client: Any, available: Callable[[], bool]) -> None:
self._client = client
self._available = available
def search(self, *args: Any, **kwargs: Any) -> Any:
try:
result = self._client.search(*args, **kwargs)
except Exception:
if not self._available():
raise EmbeddingUnavailableError() from None
raise
if not has_hits(result) and not self._available():
raise EmbeddingUnavailableError()
return result
def __getattr__(self, name: str) -> Any:
return getattr(self._client, name)
@@ -0,0 +1,554 @@
"""Which embedding space the home reads and writes: an explicit registry.
The SDK names its vector collections after `vector_store.collection_name`
and the configured width, so the space used to follow whatever the active
profile hashed to (`_collection_base`). Any change to that identity -- a
probe recording the measured width, a new endpoint -- silently opened new,
empty collections beside the old ones, and nothing carried the memories over.
The live space is pinned here instead, in the main Core database beside the
model profiles, so a switch commits the new space and the profile that fills
it in one transaction. The identity of a space is its embedding model and
width only: a new endpoint or provider serving the same model at the same
width keeps the space.
Existing homes keep their collections byte-for-byte. The first start registers
the space the active profile (or the built-in model) already opened under its
existing four-attribute name; only spaces created by a switch use the
model-and-width name. A space whose width was never declared ("auto") keeps
its unsuffixed name and only records the width it was measured at.
"""
from __future__ import annotations
import hashlib
import sqlite3
from collections.abc import Iterator, Mapping
from contextlib import contextmanager
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Literal
from uuid import uuid4
from pydantic import BaseModel, ConfigDict
from one_memory.database import take_the_write_lock_first
from one_memory.database_access import main_database
from one_memory.embedding_assets import BUILTIN_EMBEDDING_DIMENSIONS, BUILTIN_EMBEDDING_MODEL
from one_memory.memory_server.migrations import embedding_spaces_ddl
from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind
#: How long a space replaced by a switch or folded in by a merge is kept.
RETIRED_SPACE_RETENTION = timedelta(days=14)
SpaceState = Literal["live", "building", "stray", "retired", "dropped"]
SpaceNaming = Literal["legacy", "v2", "discovered"]
#: Every vector collection one space consists of, as the SDK names them.
VECTOR_ROLES = (
"memory",
"entities",
"coding_keys",
"coding_keys_cv",
"working_facts",
"working_facts_cv",
)
#: The graph directory of the first space; later spaces get their own.
DEFAULT_GRAPH_DIR = "graph"
EMBEDDING_SPACES_SCHEMA = embedding_spaces_ddl()
#: Bumped by every registry write in this process, so cached live rows are
#: reread after a change made through any connection (a probe's, a commit's).
_CHANGES = [0]
def _changed() -> None:
_CHANGES[0] += 1
class EmbeddingSpaceError(RuntimeError):
"""A registry refusal with a stable, product-facing code."""
def __init__(self, code: str) -> None:
super().__init__(code)
self.code = code
class EmbeddingIdentity(BaseModel):
"""What makes two vectors comparable: the model and the width.
`dims=None` is a width nobody declared or measured yet; it matches any
width, because the collection it names was opened without one.
"""
model_config = ConfigDict(frozen=True, extra="forbid")
model: str
dims: int | None = None
def matches(self, other: EmbeddingIdentity) -> bool:
if self.model != other.model:
return False
return self.dims is None or other.dims is None or self.dims == other.dims
class EmbeddingSpace(BaseModel):
"""One embedding space: the physical collections and graph one model fills."""
model_config = ConfigDict(frozen=True, extra="forbid")
space_id: str
state: SpaceState
#: None only for a space found on disk that no registry row named.
model: str | None = None
#: The vector width stored in this space, once known.
dims: int | None = None
#: The width the collection names and embedding requests carry. None is
#: an "auto" space: unsuffixed names and no `dimensions` request field.
name_dims: int | None = None
collection_base: str
naming: SpaceNaming = "legacy"
graph_dir: str | None = DEFAULT_GRAPH_DIR
provider_kind: str | None = None
#: The profile version whose embedding fills this space; None = built-in.
profile_id: str | None = None
profile_revision: int | None = None
#: Vectors counted the last time the space was scanned.
vectors: int | None = None
created_at: str
live_since: str | None = None
retired_at: str | None = None
keep_until: str | None = None
dropped_at: str | None = None
@property
def identity(self) -> EmbeddingIdentity | None:
if self.model is None:
return None
return EmbeddingIdentity(model=self.model, dims=self.dims)
@property
def builtin(self) -> bool:
return self.provider_kind == ProviderKind.LOCAL_ONNX.value and self.profile_id is None
def collection_names(self) -> dict[str, str]:
return space_collection_names(self.collection_base, self.name_dims)
def space_collection_names(base: str, name_dims: int | None) -> dict[str, str]:
"""The SDK's physical collection name per role for one space.
Mirrors `VectorStoreBase` (the `_<dims>` suffix), the entity store and the
coding stores. The SDK's own `vector_space_names` is checked against this
in the migration adapter when the installed SDK provides it.
"""
suffix = f"_{name_dims}" if name_dims else ""
return {
"memory": f"{base}{suffix}",
"entities": f"{base}{suffix}_entities",
"coding_keys": f"{base}_coding_keys{suffix}",
"coding_keys_cv": f"{base}_coding_keys_cv{suffix}",
"working_facts": f"{base}_working_facts{suffix}",
"working_facts_cv": f"{base}_working_facts_cv{suffix}",
}
def embedding_identity(profile: ModelProfileSnapshot) -> EmbeddingIdentity:
"""The embedding identity a profile asks for: model name and width only."""
provider = profile.embedding_provider_kind or profile.provider_kind
local = provider is ProviderKind.LOCAL_ONNX
model = profile.embedding_model or (BUILTIN_EMBEDDING_MODEL if local else None)
if not model:
raise EmbeddingSpaceError("hy_sdk_embedding_model_required")
dims = profile.embedding_dimensions
if local and dims is None:
dims = BUILTIN_EMBEDDING_DIMENSIONS
return EmbeddingIdentity(model=model, dims=dims)
def v2_collection_base(identity: EmbeddingIdentity, generation: int = 0) -> str:
"""The base name a switch gives a new space: model and width, nothing else."""
digest = hashlib.sha256(f"{identity.model}\0{identity.dims or 'auto'}".encode()).hexdigest()
base = f"memhub_e_{digest[:16]}"
return f"{base}_g{generation}" if generation else base
def graph_dir_for(space_id: str) -> str:
return f"graph-{space_id[:8]}"
def derived_space(profile: ModelProfileSnapshot) -> EmbeddingSpace:
"""The space `profile` opened before the registry existed, under its old name.
Never stored as is: bootstrap stores it, and offline tools that run beside
a home with no registry yet read through it.
"""
from one_memory.hy_sdk_runtime import legacy_collection_base
provider = profile.embedding_provider_kind or profile.provider_kind
identity = embedding_identity(profile)
builtin = profile.profile_id == _BUILTIN_PROFILE_ID
return EmbeddingSpace(
space_id="derived",
state="live",
model=identity.model,
dims=identity.dims,
name_dims=identity.dims,
collection_base=legacy_collection_base(profile),
naming="legacy",
graph_dir=DEFAULT_GRAPH_DIR,
provider_kind=provider.value,
profile_id=None if builtin else profile.profile_id,
profile_revision=None if builtin else profile.revision,
created_at=_now(),
)
_BUILTIN_PROFILE_ID = "builtin-retrieval"
def _now() -> str:
return datetime.now(UTC).isoformat()
def _row(row: sqlite3.Row | Mapping[str, object]) -> EmbeddingSpace:
return EmbeddingSpace.model_validate(dict(row))
def ensure_schema(connection: sqlite3.Connection) -> None:
connection.executescript(EMBEDDING_SPACES_SCHEMA)
def registry_present(connection: sqlite3.Connection) -> bool:
return (
connection.execute(
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='embedding_spaces'"
).fetchone()
is not None
)
def live_space_in(connection: sqlite3.Connection) -> EmbeddingSpace | None:
if not registry_present(connection):
return None
previous = connection.row_factory
connection.row_factory = sqlite3.Row
try:
row = connection.execute("SELECT * FROM embedding_spaces WHERE state='live'").fetchone()
finally:
connection.row_factory = previous
return _row(row) if row is not None else None
def read_live_space(database_path: Path) -> EmbeddingSpace | None:
"""The live space, read only, or None for a home with no registry yet."""
if not database_path.is_file():
return None
with main_database(database_path).read(row_factory=sqlite3.Row) as connection:
return live_space_in(connection)
class EmbeddingSpaceStore:
"""The `embedding_spaces` table of the main Core database."""
def __init__(self, database_path: Path) -> None:
self.database_path = database_path
#: The live row, read once per change: every read and write asks for
#: it. Only this process writes the registry while Core serves.
self._live_cache: tuple[int, EmbeddingSpace] | None = None
with self._connect(writing=True) as connection:
ensure_schema(connection)
def invalidate(self) -> None:
_changed()
@contextmanager
def _connect(
self, *, writing: bool = False, lock_first: bool = True
) -> Iterator[sqlite3.Connection]:
database = main_database(self.database_path)
checkout = (
database.write(row_factory=sqlite3.Row)
if writing
else database.read(row_factory=sqlite3.Row)
)
with checkout as connection, connection:
if writing and lock_first:
take_the_write_lock_first(connection)
yield connection
@contextmanager
def transaction(self) -> Iterator[sqlite3.Connection]:
"""One `BEGIN IMMEDIATE` on the main database, committed on success."""
_changed()
try:
with self._connect(writing=True, lock_first=False) as connection:
connection.execute("BEGIN IMMEDIATE")
yield connection
finally:
_changed()
def live(self) -> EmbeddingSpace | None:
generation = _CHANGES[0]
cached = self._live_cache
if cached is not None and cached[0] == generation:
return cached[1]
with self._connect() as connection:
space = live_space_in(connection)
self._live_cache = (generation, space) if space is not None else None
return space
def require_live(self) -> EmbeddingSpace:
space = self.live()
if space is None:
raise EmbeddingSpaceError("embedding_space_not_bootstrapped")
return space
def get(self, space_id: str) -> EmbeddingSpace | None:
with self._connect() as connection:
row = connection.execute(
"SELECT * FROM embedding_spaces WHERE space_id=?", (space_id,)
).fetchone()
return _row(row) if row is not None else None
def list_spaces(self, *, include_dropped: bool = False) -> list[EmbeddingSpace]:
with self._connect() as connection:
rows = connection.execute(
"SELECT * FROM embedding_spaces "
+ ("" if include_dropped else "WHERE state != 'dropped' ")
+ "ORDER BY created_at, space_id"
).fetchall()
return [_row(row) for row in rows]
def bootstrap(self, active: ModelProfileSnapshot | None) -> EmbeddingSpace:
"""Register the space this home already uses, once, under its existing name.
`active` is the active profile, or None for the built-in model. The
collection name is derived exactly as before the registry, so nothing
on disk moves.
"""
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot
with self.transaction() as connection:
existing = live_space_in(connection)
if existing is not None:
return existing
derived = derived_space(active or builtin_retrieval_snapshot())
space = derived.model_copy(
update={"space_id": str(uuid4()), "live_since": derived.created_at}
)
_insert(connection, space)
return space
def record_width(self, space_id: str, dims: int) -> EmbeddingSpace:
"""A relabel: the space's stored width is now known. Names never change."""
with self.transaction() as connection:
record_width_in(connection, space_id, dims)
space = self.get(space_id)
if space is None: # pragma: no cover - the row was just updated
raise EmbeddingSpaceError("embedding_space_not_found")
return space
def record_vectors(self, counts: Mapping[str, int]) -> None:
"""Cache the vector count per space id from the last scan (display only)."""
if not counts:
return
_changed()
with self._connect(writing=True) as connection:
connection.executemany(
"UPDATE embedding_spaces SET vectors=? WHERE space_id=?",
[(count, space_id) for space_id, count in counts.items()],
)
def register_stray(
self,
*,
collection_base: str,
name_dims: int | None,
dims: int | None,
vectors: int,
) -> EmbeddingSpace:
"""Name a non-empty space the scan found that no row described."""
with self.transaction() as connection:
row = connection.execute(
"SELECT * FROM embedding_spaces WHERE collection_base=? "
"AND COALESCE(name_dims,0)=? AND state != 'dropped'",
(collection_base, name_dims or 0),
).fetchone()
if row is not None:
return _row(row)
space = EmbeddingSpace(
space_id=str(uuid4()),
state="stray",
dims=dims,
name_dims=name_dims,
collection_base=collection_base,
naming="discovered",
graph_dir=None,
vectors=vectors,
created_at=_now(),
)
_insert(connection, space)
return space
def mark_dropped(self, space_id: str) -> None:
_changed()
with self._connect(writing=True) as connection:
connection.execute(
"UPDATE embedding_spaces SET state='dropped', dropped_at=? "
"WHERE space_id=? AND state IN ('retired','building')",
(_now(), space_id),
)
def expired(self, *, now: datetime | None = None) -> list[EmbeddingSpace]:
moment = (now or datetime.now(UTC)).isoformat()
with self._connect() as connection:
rows = connection.execute(
"SELECT * FROM embedding_spaces WHERE state='retired' "
"AND keep_until IS NOT NULL AND keep_until <= ?",
(moment,),
).fetchall()
return [_row(row) for row in rows]
def registry_for(profiles: object) -> EmbeddingSpaceStore | None:
"""The registry beside a profile store, or None for a store with no database."""
path = getattr(profiles, "database_path", None)
return EmbeddingSpaceStore(path) if isinstance(path, Path) else None
def _insert(connection: sqlite3.Connection, space: EmbeddingSpace) -> None:
values = space.model_dump()
columns = ", ".join(values)
marks = ", ".join("?" for _ in values)
connection.execute(
f"INSERT INTO embedding_spaces ({columns}) VALUES ({marks})",
tuple(values.values()),
)
def insert_space_in(connection: sqlite3.Connection, space: EmbeddingSpace) -> None:
_changed()
_insert(connection, space)
def record_width_in(connection: sqlite3.Connection, space_id: str, dims: int) -> None:
_changed()
changed = connection.execute(
"UPDATE embedding_spaces SET dims=? WHERE space_id=? AND (dims IS NULL OR dims=?)",
(dims, space_id, dims),
)
if changed.rowcount != 1:
raise EmbeddingSpaceError("embedding_space_width_conflict")
def space_in(connection: sqlite3.Connection, space_id: str) -> EmbeddingSpace | None:
previous = connection.row_factory
connection.row_factory = sqlite3.Row
try:
row = connection.execute(
"SELECT * FROM embedding_spaces WHERE space_id=?", (space_id,)
).fetchone()
finally:
connection.row_factory = previous
return _row(row) if row is not None else None
def spaces_in(connection: sqlite3.Connection) -> list[EmbeddingSpace]:
previous = connection.row_factory
connection.row_factory = sqlite3.Row
try:
rows = connection.execute("SELECT * FROM embedding_spaces").fetchall()
finally:
connection.row_factory = previous
return [_row(row) for row in rows]
def set_state_in(
connection: sqlite3.Connection,
space_id: str,
*,
state: SpaceState,
expected: tuple[SpaceState, ...],
now: datetime | None = None,
) -> None:
"""Move one space between states, refusing any it was not expected in."""
_changed()
moment = now or datetime.now(UTC)
updates: dict[str, object] = {"state": state}
if state == "live":
updates["live_since"] = moment.isoformat()
elif state == "retired":
updates["retired_at"] = moment.isoformat()
updates["keep_until"] = (moment + RETIRED_SPACE_RETENTION).isoformat()
elif state == "dropped":
updates["dropped_at"] = moment.isoformat()
assignments = ", ".join(f"{column}=?" for column in updates)
marks = ",".join("?" for _ in expected)
changed = connection.execute(
f"UPDATE embedding_spaces SET {assignments} WHERE space_id=? AND state IN ({marks})",
(*updates.values(), space_id, *expected),
)
if changed.rowcount != 1:
raise EmbeddingSpaceError("embedding_space_state_changed")
def next_v2_base(
connection: sqlite3.Connection,
identity: EmbeddingIdentity,
*,
stored_names: frozenset[str],
) -> str:
"""A base for a new space no registry row and no stored collection uses.
Switching back to a model whose retired space is still kept gets a fresh
`_g<n>` base, so stale retired vectors are never mixed with fresh ones.
"""
used = {space.collection_base for space in spaces_in(connection)}
generation = 0
while True:
base = v2_collection_base(identity, generation)
names = space_collection_names(base, identity.dims).values()
if base not in used and not any(name in stored_names for name in names):
return base
generation += 1
def probe_width_allowed(
connection: sqlite3.Connection,
profile_id: str,
width: int,
*,
data_dir: Path,
) -> bool:
"""Whether a probe may record `width` on a profile without moving the space.
A profile that fills the live space (the active one, or the one the space
was built with) may record its measured width only when it is the
space's: equal to the width already known, or -- for an "auto" space
whose width was never recorded -- equal to the width Chroma stored, or
any width when nothing is stored yet. That is a relabel; the names never
change. Any other width needs a switch (B2: it used to silently open a
new, empty collection). Every other profile records what it measured.
"""
live = live_space_in(connection)
if live is None:
return True
active = connection.execute(
"SELECT profile_id FROM model_profiles WHERE is_active=1"
).fetchone()
fills_live = live.profile_id == profile_id or (active is not None and active[0] == profile_id)
if not fills_live:
return True
if live.dims is not None:
return live.dims == width
from one_memory.sdk_vector_reattach import stored_collection_width
stored = stored_collection_width(data_dir, live.collection_names()["memory"])
if stored is not None and stored != width:
return False
record_width_in(connection, live.space_id, width)
return True
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,161 @@
"""HTTP surface of the embedding switch: spaces, plans, jobs, cleanup.
The contract the App and the CLI build on is
`memhub/docs/contracts/embedding-switch-v1.md`. Errors keep Core's shape,
`{"detail": "<code>"}`; a refusal that computed a plan also carries it as
`"plan"`, so a client can show the confirmation without asking again.
"""
from __future__ import annotations
import asyncio
from collections.abc import Sequence
from typing import Any, cast
from fastapi import APIRouter, Request, Response
from fastapi.params import Depends
from fastapi.responses import JSONResponse
from pydantic import BaseModel, ConfigDict, Field
from one_memory.embedding_backend import ReembedUnavailableError
from one_memory.embedding_spaces import EmbeddingSpaceError
from one_memory.embedding_switch import (
EmbeddingSwitchError,
EmbeddingSwitchPlan,
EmbeddingSwitchService,
)
#: HTTP status per refusal code; anything not listed is a 409.
_STATUS = {
"model_profile_not_found": 404,
"embedding_switch_not_found": 404,
"embedding_space_not_found": 404,
"sdk_reembed_unavailable": 503,
"secure_storage_unavailable": 503,
"insufficient_disk_space": 507,
}
class EmbeddingSwitchPlanRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
profile_id: str = Field(min_length=1, max_length=128)
expected_revision: int | None = Field(default=None, ge=1)
class EmbeddingSwitchStartRequest(BaseModel):
model_config = ConfigDict(extra="forbid")
profile_id: str = Field(min_length=1, max_length=128)
expected_revision: int = Field(ge=1)
plan_token: str = Field(min_length=1, max_length=128)
requested_by: str = Field(default="app", pattern=r"^(app|cli)$")
def refusal(code: str, *, plan: EmbeddingSwitchPlan | None = None) -> JSONResponse:
content: dict[str, Any] = {"detail": code}
if plan is not None:
content["plan"] = plan.model_dump(mode="json")
return JSONResponse(status_code=_STATUS.get(code, 409), content=content)
def _service(request: Request) -> EmbeddingSwitchService:
return cast(EmbeddingSwitchService, request.app.state.embedding_switch)
def embedding_switch_router(dependencies: Sequence[Depends]) -> APIRouter:
router = APIRouter(dependencies=dependencies)
@router.get("/api/v1/embedding-spaces")
async def list_embedding_spaces(request: Request) -> dict[str, object]:
"""The registry and what the vector store holds, space by space."""
service = _service(request)
spaces = await asyncio.to_thread(service.spaces.list_spaces)
live = next((space for space in spaces if space.state == "live"), None)
scanned: list[dict[str, object]] | None = None
scan_error = service.backend.unavailable_reason()
if scan_error is None:
try:
found = await service.backend.scan()
except Exception as error: # noqa: BLE001 - the registry still answers
scan_error = getattr(error, "code", None) or "embedding_space_scan_failed"
else:
known = {(space.collection_base, space.name_dims or 0): space for space in spaces}
scanned = [
{
"collection_base": item.collection_base,
"name_dims": item.name_dims,
"width": item.width,
"vectors": item.vectors,
"counts": dict(item.counts),
"space_id": known[item.key].space_id if item.key in known else None,
"state": known[item.key].state if item.key in known else "unregistered",
}
for item in found
]
return {
"schema_version": 1,
"live_space_id": live.space_id if live is not None else None,
"spaces": [space.model_dump(mode="json") for space in spaces],
"scanned": scanned,
"scan_error": scan_error,
}
@router.delete("/api/v1/embedding-spaces/{space_id}", response_model=None)
async def delete_embedding_space(space_id: str, request: Request) -> Any:
"""Clean up a retired space now instead of after its 14 days."""
try:
space = await _service(request).drop_space(space_id)
except (EmbeddingSwitchError, EmbeddingSpaceError, ReembedUnavailableError) as error:
return refusal(error.code)
except RuntimeError as error:
if "held open" in str(error):
return refusal("embedding_space_in_use")
raise
return {"schema_version": 1, "space": space.model_dump(mode="json")}
@router.post("/api/v1/embedding-switches/plan", response_model=None)
async def plan_embedding_switch(value: EmbeddingSwitchPlanRequest, request: Request) -> Any:
try:
plan = await _service(request).plan(
value.profile_id, expected_revision=value.expected_revision
)
except EmbeddingSwitchError as error:
return refusal(error.code, plan=error.plan)
return plan.model_dump(mode="json")
@router.post("/api/v1/embedding-switches", status_code=202, response_model=None)
async def start_embedding_switch(value: EmbeddingSwitchStartRequest, request: Request) -> Any:
try:
job = await _service(request).start_switch(
value.profile_id,
expected_revision=value.expected_revision,
plan_token=value.plan_token,
requested_by=value.requested_by,
)
except EmbeddingSwitchError as error:
return refusal(error.code, plan=error.plan)
return job.model_dump(mode="json")
@router.get("/api/v1/embedding-switches/current")
async def current_embedding_switch(request: Request, response: Response) -> dict[str, object]:
"""Polled by the loading screen, the banner and `memhub models switch status`."""
response.headers["Cache-Control"] = "no-store"
return await asyncio.to_thread(_service(request).current)
@router.get("/api/v1/embedding-switches/{job_id}", response_model=None)
async def get_embedding_switch(job_id: str, request: Request) -> Any:
job = await asyncio.to_thread(_service(request).get_job, job_id)
if job is None:
return refusal("embedding_switch_not_found")
return job.model_dump(mode="json")
@router.post("/api/v1/embedding-switches/{job_id}/cancel", response_model=None)
async def cancel_embedding_switch(job_id: str, request: Request) -> Any:
try:
job = await asyncio.to_thread(_service(request).cancel, job_id)
except EmbeddingSwitchError as error:
return refusal(error.code)
return job.model_dump(mode="json")
return router
@@ -94,11 +94,15 @@ def inspect_profile_graph_width(
api_key: str | None,
data_dir: Path,
mode: str | None = None,
graph_dir: str = "graph",
) -> GraphWidthMismatch | None:
"""Inspect the graph path and width selected by the active profile."""
"""Inspect the graph path and width selected by the active profile.
`graph_dir` is the live embedding space's graph (`embedding_spaces`).
"""
del api_key
return _inspect_graph_width(
_profile_graph_path(data_dir, mode),
_profile_graph_path(data_dir, mode, graph_dir),
_profile_embedding_width(profile),
)
@@ -110,11 +114,12 @@ def decide_profile_graph_width(
data_dir: Path,
decision: GraphWidthDecision,
mode: str | None = None,
graph_dir: str = "graph",
) -> GraphWidthOutcome:
"""Apply the user's decision to the active profile's graph."""
del api_key
config = _GraphAdminConfig(
graph_store=_GraphStoreConfig(str(_profile_graph_path(data_dir, mode))),
graph_store=_GraphStoreConfig(str(_profile_graph_path(data_dir, mode, graph_dir))),
embedder=_EmbeddingWidthConfig(_profile_embedding_width(profile)),
)
return apply_graph_width_decision(
@@ -204,6 +209,12 @@ _GRAPH_WIDTH_CACHE: dict[str, int | None] = {}
_GRAPH_WIDTH_LOCK = threading.Lock()
def forget_graph_widths() -> None:
"""Drop every remembered width: an embedding switch moved the graph."""
with _GRAPH_WIDTH_LOCK:
_GRAPH_WIDTH_CACHE.clear()
def _graph_embedding_width(db_path: Path) -> int | None:
"""Read the Memory embedding width directly from Kuzu's schema."""
try:
@@ -252,9 +263,9 @@ def _inspect_graph_width(
)
def _profile_graph_path(data_dir: Path, mode: str | None = None) -> Path:
def _profile_graph_path(data_dir: Path, mode: str | None = None, graph_dir: str = "graph") -> Path:
ultra = mode == "ultra" if mode is not None else graph_store_enabled()
name = "graph" if ultra else "graph-disabled"
name = graph_dir if ultra else "graph-disabled"
return data_dir / "hy-memory" / HY_MEMORY_NAMESPACE / name
@@ -14,7 +14,7 @@ from pathlib import Path
from typing import Literal, cast
HY_MEMORY_DISTRIBUTION = "hy-memory"
HY_MEMORY_VERSION = "2.0.9"
HY_MEMORY_VERSION = "2.0.11"
HY_MEMORY_NAMESPACE = "governed-v2"
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
@@ -3,20 +3,26 @@
Runtime product code must use public ``HyMemoryClient`` methods. The startup
migration boundary is the isolated exception: all writers are stopped, so it
may copy, restore and repair SDK-owned local stores.
The embedding switch is the second such boundary: model writes are paused
while it re-embeds one space into another (`SdkReembedBackend`), so it may
read and fill SDK collections through the SDK's re-embed module.
"""
from __future__ import annotations
import asyncio
import json
import sqlite3
from collections.abc import Iterator
from contextlib import closing
from contextlib import closing, contextmanager
from dataclasses import dataclass, field
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any
from hy_memory import ( # type: ignore[import-untyped]
import hy_memory # type: ignore[import-untyped]
from hy_memory import (
DataPlaneRestore,
DataPlaneSnapshot,
restore_local_data_plane,
@@ -497,7 +503,7 @@ def _ensure_coding_schema(path: Path) -> None:
CodingMemoryStore,
)
# Relies on the pinned SDK's internals (hy-memory==2.0.9 is pinned exactly).
# Relies on the pinned SDK's internals (hy-memory==2.0.11 is pinned exactly).
store = CodingMemoryStore.__new__(CodingMemoryStore)
store._db_path = str(path)
store._conn = None
@@ -511,3 +517,256 @@ def _ensure_coding_schema(path: Path) -> None:
def _batches[T](values: list[T] | tuple[T, ...]) -> Iterator[tuple[T, ...]]:
for start in range(0, len(values), _BATCH):
yield tuple(values[start : start + _BATCH])
# The SDK functions the embedding switch needs. They arrived in hy-memory
# 2.0.10; an older SDK reports `sdk_reembed_unavailable` instead of failing.
_REEMBED_API = (
"VectorSpace",
"scan_vector_spaces",
"diff_spaces",
"reembed_vector_space",
"verify_reembed",
"drop_vector_space",
"reembed_graph",
"verify_graph_reembed",
"drop_graph",
"ReembedCancelled",
)
class SdkReembedBackend:
"""`ReembedBackend` over `hy_memory`'s re-embed module, for one home.
Runs beside live SDK clients: the vector functions open Chroma with the
SDK's own settings, so they share its client system. The graph functions
refuse a graph a live client holds open; the switch closes them first.
"""
def __init__(self, data_dir: Path) -> None:
self.data_dir = data_dir
self.root = data_dir.resolve() / "hy-memory" / HY_MEMORY_NAMESPACE
self.persist_directory = self.root / "vector"
self._chroma_opened = False
def unavailable_reason(self) -> str | None:
missing = [name for name in _REEMBED_API if not hasattr(hy_memory, name)]
return "sdk_reembed_unavailable" if missing else None
def _sdk(self) -> Any:
from one_memory.embedding_backend import ReembedUnavailableError
reason = self.unavailable_reason()
if reason is not None:
raise ReembedUnavailableError(reason)
return hy_memory
def _open_chroma_once(self) -> None:
"""Open Chroma once under the client-construction lock.
Every later open beside the Runtime finds that system. Two first
opens race over Chroma's unlocked cache, and a client under
construction lost its stores that way (see `hy_sdk_runtime`).
"""
if self._chroma_opened or not self.persist_directory.is_dir():
return
import chromadb
from one_memory.hy_sdk_runtime import first_store_open
with first_store_open(self.data_dir):
chromadb.PersistentClient(path=str(self.persist_directory))
self._chroma_opened = True
def _vector_space(self, space: Any) -> Any:
return self._sdk().VectorSpace(
base_name=space.collection_base,
dims=space.dims,
persist_directory=str(self.persist_directory),
model=space.model,
collections=space.names(),
)
async def scan(self, *, per_user: bool = False) -> tuple[Any, ...]:
from one_memory.embedding_backend import ScannedSpace
if not self.persist_directory.is_dir():
return ()
await asyncio.to_thread(self._open_chroma_once)
found = await self._sdk().scan_vector_spaces(str(self.persist_directory), per_user=per_user)
return tuple(
ScannedSpace(
collection_base=str(item.base_name),
name_dims=item.dims,
width=item.width,
counts={role: int(entry.count) for role, entry in item.roles.items()},
collections={role: str(entry.name) for role, entry in item.roles.items()},
per_user=dict(item.per_user) if per_user else {},
)
for item in found
)
async def diff(self, source: Any, target: Any) -> dict[str, int]:
await asyncio.to_thread(self._open_chroma_once)
result = await self._sdk().diff_spaces(
self._vector_space(source), self._vector_space(target)
)
return {role: len(ids) for role, ids in result.missing.items()}
async def reembed(
self,
source: Any,
target: Any,
*,
embedding: Any,
cancel: Any,
on_step: Any,
batch_size: int = 64,
) -> None:
from one_memory.embedding_backend import ReembedStep
await asyncio.to_thread(self._open_chroma_once)
sdk = self._sdk()
def progress(item: Any) -> None:
on_step(
ReembedStep(
phase="vectors",
role=str(item.role),
done=int(item.done),
total=int(item.total),
embedded=int(item.embedded),
reused=int(item.reused),
kept=int(getattr(item, "kept", 0)),
)
)
with self._translated():
await sdk.reembed_vector_space(
self._vector_space(source),
self._vector_space(target),
embedding=self._embedding_service(embedding),
batch_size=batch_size,
cancel=cancel,
on_progress=progress,
# The target wins: an id it already holds is never touched, so
# a resumed run continues and a merge never overwrites live data.
skip_existing=True,
)
async def verify(self, source: Any, target: Any, *, embedding: Any) -> Any:
from one_memory.embedding_backend import VectorCheck
await asyncio.to_thread(self._open_chroma_once)
with self._translated():
result = await self._sdk().verify_reembed(
self._vector_space(source),
self._vector_space(target),
embedding=self._embedding_service(embedding),
)
return VectorCheck(
per_role={role: dict(values) for role, values in result.per_role.items()},
per_user=dict(result.per_user),
problems=tuple(result.problems),
warnings=tuple(result.warnings),
self_recall_hits=int(result.self_recall_hits),
self_recall_samples=int(result.self_recall_samples),
)
async def graph_vectors(self, path: Path) -> int | None:
if not path.exists():
return None
# Not re-exported at the package root; the module is the SDK's own.
plan = await self._sdk().reembed.plan_graph_reembed(path)
return int(plan.vectors)
async def reembed_graph(
self,
source: Path,
target: Path,
*,
embedding: Any,
target_dims: int,
cancel: Any,
on_step: Any,
) -> None:
from one_memory.embedding_backend import ReembedStep
def progress(item: Any) -> None:
on_step(
ReembedStep(
phase="graph",
role=str(item.table),
done=int(item.done),
total=int(item.total),
embedded=int(item.embedded),
)
)
with self._translated():
await self._sdk().reembed_graph(
source,
target,
embedding=self._embedding_service(embedding),
target_dims=target_dims,
cancel=cancel,
on_progress=progress,
# A resumed switch rebuilds the copy from the untouched source.
overwrite=True,
)
async def verify_graph(self, source: Path, target: Path, *, target_dims: int) -> Any:
from one_memory.embedding_backend import GraphCheck
result = await self._sdk().verify_graph_reembed(source, target, target_dims=target_dims)
return GraphCheck(ok=bool(result.ok), problems=tuple(result.problems))
async def drop(self, space: Any) -> None:
await asyncio.to_thread(self._open_chroma_once)
await self._sdk().drop_vector_space(self._vector_space(space))
async def drop_graph(self, path: Path) -> None:
await self._sdk().drop_graph(path)
def _embedding_service(self, embedding: Any) -> Any:
"""The service that computes a target space's vectors.
The built-in model is MemHub's ONNX service, exactly what the Runtime
client embeds with. A remote model gets the SDK's own service, built
from the same configuration the Runtime client would use.
"""
from one_memory.hy_sdk_runtime import (
LocalOnnxEmbeddingService,
build_sdk_config,
retrieval_view,
)
from one_memory.model_profiles import ProviderKind
profile = embedding.profile
provider = profile.embedding_provider_kind or profile.provider_kind
if provider is ProviderKind.LOCAL_ONNX:
return LocalOnnxEmbeddingService()
from hy_memory.core.embed_service import EmbedService # type: ignore[import-untyped]
return EmbedService(
# The retrieval view: re-embedding never needs the generation model.
build_sdk_config(
retrieval_view(profile),
embedding.space,
api_key=embedding.api_key,
data_dir=self.data_dir,
)
)
@contextmanager
def _translated(self) -> Iterator[None]:
"""SDK refusals as the Core errors the switch classifies."""
from one_memory.embedding_backend import EmbeddingWidthError, ReembedCancelledError
sdk = self._sdk()
try:
yield
except sdk.ReembedCancelled:
raise ReembedCancelledError("embedding_switch_cancelled") from None
except hy_memory.EmbeddingDimensionMismatchError as error:
raise EmbeddingWidthError("embedding_dimensions_mismatch") from error
+207 -33
View File
@@ -11,10 +11,11 @@ import asyncio
import hashlib
import os
import threading
from collections.abc import Callable, Coroutine, Iterable, Mapping
from collections.abc import Callable, Coroutine, Iterable, Iterator, Mapping
from contextlib import contextmanager
from dataclasses import dataclass
from pathlib import Path
from typing import Any, ClassVar
from typing import Any, ClassVar, Literal, TypedDict, cast
from hy_memory import ( # type: ignore[import-untyped]
AmbiguousMemoryIdError as AmbiguousMemoryIdError, # re-export for api.py
@@ -29,6 +30,13 @@ from one_memory.embedding_assets import (
BUILTIN_EMBEDDING_DIMENSIONS,
BUILTIN_EMBEDDING_MODEL,
)
from one_memory.embedding_spaces import (
DEFAULT_GRAPH_DIR,
EmbeddingSpace,
EmbeddingSpaceError,
derived_space,
embedding_identity,
)
from one_memory.hy_graph_admin import (
GraphWidthMismatchError,
effective_memory_mode,
@@ -54,6 +62,10 @@ BUILTIN_RETRIEVAL_PROFILE_ID = "builtin-retrieval"
_LOCAL_ONNX_PLACEHOLDER_URL = "http://127.0.0.1/local-onnx"
DEFAULT_RELEVANCE_FLOOR = 0.35
#: The SDK's memory modes. A mode never changes a collection name: the space
#: names the collections, the mode only which stores the client opens.
MemoryMode = Literal["lite", "pro", "ultra"]
@dataclass
class _RuntimeEntry:
@@ -176,37 +188,77 @@ class SdkClientHandle:
api_key: str | None,
data_dir: Path,
http_transport_factory: Callable[[], Any] | None = None,
space: EmbeddingSpace | None = None,
mode: str | None = None,
features: FeatureOverrides = (),
language: str | None = None,
) -> None:
self._profile = profile
self._api_key = api_key
self._data_dir = data_dir
self._http_transport_factory = http_transport_factory
self._space = space
#: The SDK mode this handle's client is built with, fixed for its life.
self.mode = mode or effective_memory_mode()
#: The SDK features set on its config (`memory_features`), fixed likewise.
self.features = features
#: The SDK prompt language (`ui_language`); None = SDK auto-detect.
self.language = language
self._lease: ClientLease | None = None
self._constructing: asyncio.Future[ClientLease] | None = None
self._lock = asyncio.Lock()
@property
def profile(self) -> ModelProfileSnapshot:
return self._profile
@property
def space(self) -> EmbeddingSpace | None:
return self._space
@property
def api_key(self) -> str | None:
return self._api_key
async def client(self) -> HyMemoryClient:
if self._lease is not None:
return self._lease.client
async with self._lock:
if self._lease is None:
self._lease = await asyncio.to_thread(
acquire_client,
self._profile,
api_key=self._api_key,
data_dir=self._data_dir,
http_transport_factory=self._http_transport_factory,
mode=self.mode,
features=self.features,
constructing = asyncio.ensure_future(
asyncio.to_thread(
_acquire_for_handle,
self._profile,
self._space,
api_key=self._api_key,
data_dir=self._data_dir,
http_transport_factory=self._http_transport_factory,
mode=self.mode,
features=self.features,
**language_kwargs(self.language),
)
)
self._constructing = constructing
try:
self._lease = await asyncio.shield(constructing)
except asyncio.CancelledError:
# The construction thread runs on. Keep what it builds, so
# `close` releases it instead of leaving its runtime open.
constructing.add_done_callback(self._keep)
raise
return self._lease.client
def _keep(self, constructing: asyncio.Future[ClientLease]) -> None:
if constructing.cancelled() or constructing.exception() is not None:
return
if self._lease is None:
self._lease = constructing.result()
async def close(self) -> None:
constructing = self._constructing
if constructing is not None and not constructing.done():
# A construction a cancelled caller left behind: wait for its lease.
await asyncio.gather(constructing, return_exceptions=True)
async with self._lock:
lease, self._lease = self._lease, None
if lease is not None:
@@ -299,33 +351,67 @@ def apply_sdk_features(
return config
class LanguageKwargs(TypedDict, total=False):
language: str
def language_kwargs(language: str | None) -> LanguageKwargs:
"""`language=` for a client constructor, only when there is one.
Without a language every call is the one it was before the setting
existed, so stand-ins written for that call keep working.
"""
return {"language": language} if language is not None else {}
def apply_sdk_language(config: MemoryConfig, language: str | None) -> MemoryConfig:
"""Set the SDK prompt language (`MemoryConfig.language`), when there is one.
None (no language set) sets nothing: the SDK detects zh/en as it always
did. Every display language, zh-Hans included, is set explicitly. Like a feature, the language never touches `vector_store`, so
the collection identity never depends on it.
"""
if language is not None:
config.language = language
return config
def build_sdk_config(
profile: ModelProfileSnapshot,
space: EmbeddingSpace | None = None,
mode: MemoryMode | None = None,
*,
api_key: str | None,
data_dir: Path,
features: Mapping[str, bool] | Iterable[tuple[str, bool]] = (),
language: str | None = None,
) -> MemoryConfig:
"""Map a model snapshot to SDK models, endpoints, backends, and local paths.
"""Map a model snapshot and its embedding space to an SDK configuration.
`features` (`memory_features`) are set on top; with none the config is
exactly what it was before the setting existed.
`space` names the collections and the graph; the profile only says how to
reach the models. A profile whose embedding identity (model and width) is
not the space's raises `embedding_space_mismatch` instead of opening new,
empty collections. Without `space`, the space the profile opened before
the registry existed is used: offline tools on a home with no registry.
`mode` never changes a name, and neither do `features`
(`memory_features`), which are set on top; with none the config is
exactly what it was before the setting existed. So is `language`
(`apply_sdk_language`); None leaves the SDK's own zh/en detection.
"""
if mode is not None and mode not in {"lite", "pro", "ultra"}:
raise HySdkRuntimeError("hy_sdk_memory_mode_invalid")
generation_endpoint, generation_key = _generation_access(profile, api_key)
embedding_provider = profile.embedding_provider_kind or profile.provider_kind
embedding_service_is_local = embedding_provider is ProviderKind.LOCAL_ONNX
embedding_model = profile.embedding_model or (
BUILTIN_EMBEDDING_MODEL if embedding_service_is_local else None
)
if not embedding_model:
raise HySdkRuntimeError("hy_sdk_embedding_model_required")
dimensions: int | None
try:
identity = embedding_identity(profile)
except EmbeddingSpaceError as error:
raise HySdkRuntimeError(error.code) from None
if embedding_service_is_local:
embedding_endpoint = None
embedding_key = None
dimensions = profile.embedding_dimensions or BUILTIN_EMBEDDING_DIMENSIONS
else:
# Never the generation endpoint: the collection name hashes this one.
# Never the generation endpoint: a remote embedding has its own.
embedding_endpoint = _openai_endpoint(embedding_provider, profile.embedding_base_url)
embedding_key = "ollama" if embedding_provider is ProviderKind.OLLAMA else api_key
if embedding_provider is ProviderKind.OPENAI_COMPATIBLE and not embedding_key:
@@ -335,13 +421,18 @@ def build_sdk_config(
ProviderKind.OLLAMA,
}:
raise HySdkRuntimeError("hy_sdk_embedding_provider_unsupported")
dimensions = profile.embedding_dimensions
if space is None:
space = derived_space(profile)
elif space.identity is None or not identity.matches(space.identity):
raise HySdkRuntimeError("embedding_space_mismatch")
# The width the space was opened with: its names carry it, and a remote
# request sends it as `dimensions`. An "auto" space sends none, as before.
dimensions = space.name_dims
embedding_model = identity.model
root = data_dir.resolve() / "hy-memory" / "governed-v2"
root.mkdir(parents=True, exist_ok=True)
collection = _collection_base(
embedding_provider, embedding_endpoint, embedding_model, dimensions
)
collection = space.collection_base
llm: dict[str, Any] = {
"provider": "openai",
@@ -379,7 +470,7 @@ def build_sdk_config(
},
"graph_store": {
"provider": "kuzu",
"db_path": str(root / "graph"),
"db_path": str(root / (space.graph_dir or DEFAULT_GRAPH_DIR)),
"url": None,
"username": None,
"password": None,
@@ -393,25 +484,54 @@ def build_sdk_config(
"coding": {"db_path": str(root / "coding.sqlite3")},
}
)
return apply_sdk_features(config, features)
return apply_sdk_language(apply_sdk_features(config, features), language)
def _acquire_for_handle(
profile: ModelProfileSnapshot,
space: EmbeddingSpace | None,
*,
api_key: str | None,
data_dir: Path,
http_transport_factory: Callable[[], Any] | None,
mode: str | None,
features: FeatureOverrides = (),
language: str | None = None,
) -> ClientLease:
# `acquire_client` is looked up at call time, so a stand-in replaces every
# construction. A handle without a space keeps the call it always made.
named: dict[str, Any] = {"space": space} if space is not None else {}
return acquire_client(
profile,
api_key=api_key,
data_dir=data_dir,
http_transport_factory=http_transport_factory,
mode=mode,
features=features,
**named,
**language_kwargs(language),
)
def acquire_client(
profile: ModelProfileSnapshot,
space: EmbeddingSpace | None = None,
*,
api_key: str | None,
data_dir: Path,
http_transport_factory: Callable[[], Any] | None = None,
mode: str | None = None,
features: FeatureOverrides = (),
language: str | None = None,
) -> ClientLease:
"""Construct a public SDK client and return its explicit runtime lease.
`mode` is the SDK pipeline mode (lite / pro / ultra); None keeps the
pre-setting default (ultra when `ONE_MEMORY_GRAPH_STORE` is set, else pro).
`features` are the `memory_features` set on its config; none keeps every
SDK default.
Waits while `hold_stores_offline` holds this home's stores.
`space` names the collections (`build_sdk_config`). `mode` is the SDK
pipeline mode (lite / pro / ultra) and never changes a name; None keeps
the pre-setting default (ultra when `ONE_MEMORY_GRAPH_STORE` is set, else
pro). `features` are the `memory_features` set on its config; none keeps
every SDK default. `language` is the SDK prompt language; None
auto-detects. Waits while `hold_stores_offline` holds this home's stores.
"""
key = data_dir.resolve()
with _STORE_GATE:
@@ -419,6 +539,7 @@ def acquire_client(
_STORE_GATE.wait()
_OPENING[key] = _OPENING.get(key, 0) + 1
try:
named: dict[str, Any] = {"space": space} if space is not None else {}
return _acquire_client(
profile,
api_key=api_key,
@@ -426,6 +547,8 @@ def acquire_client(
http_transport_factory=http_transport_factory,
mode=mode or effective_memory_mode(),
features=features,
**named,
**language_kwargs(language),
)
finally:
with _STORE_GATE:
@@ -435,6 +558,20 @@ def acquire_client(
_STORE_GATE.notify_all()
@contextmanager
def first_store_open(data_dir: Path) -> Iterator[None]:
"""One opening of this home's stores at a time: Chroma's first open is unlocked.
The same lock client construction takes, for anything else that opens
Chroma beside the Runtime (the re-embed functions).
"""
key = data_dir.resolve()
with _STORE_GATE:
constructing = _CONSTRUCTING.setdefault(key, threading.Lock())
with constructing:
yield
def hold_stores_offline(data_dir: Path) -> bool:
"""Hold this home's SDK stores for offline maintenance; False if a client is open.
@@ -460,14 +597,24 @@ def release_stores(data_dir: Path) -> None:
def _acquire_client(
profile: ModelProfileSnapshot,
space: EmbeddingSpace | None = None,
*,
api_key: str | None,
data_dir: Path,
http_transport_factory: Callable[[], Any] | None,
mode: str = "pro",
features: FeatureOverrides = (),
language: str | None = None,
) -> ClientLease:
config = build_sdk_config(profile, api_key=api_key, data_dir=data_dir, features=features)
config = build_sdk_config(
profile,
space,
cast(MemoryMode, mode),
api_key=api_key,
data_dir=data_dir,
features=features,
language=language,
)
config.http_transport_factory = http_transport_factory
mismatch = inspect_graph_width_mismatch(config)
if mismatch is not None:
@@ -523,6 +670,33 @@ def _generation_access(
return endpoint, api_key
def embedding_endpoint(profile: ModelProfileSnapshot) -> str | None:
"""The OpenAI-compatible base URL the SDK embeds through; None when local."""
provider = profile.embedding_provider_kind or profile.provider_kind
if provider is ProviderKind.LOCAL_ONNX:
return None
return _openai_endpoint(provider, profile.embedding_base_url)
def legacy_collection_base(profile: ModelProfileSnapshot) -> str:
"""The four-attribute name a profile's space had before the registry.
Existing homes are registered under exactly this name, so it must never
change: provider, embedding endpoint, model and declared width.
"""
provider = profile.embedding_provider_kind or profile.provider_kind
local = provider is ProviderKind.LOCAL_ONNX
model = profile.embedding_model or (BUILTIN_EMBEDDING_MODEL if local else None)
if not model:
raise HySdkRuntimeError("hy_sdk_embedding_model_required")
if local:
return _collection_base(
provider, None, model, profile.embedding_dimensions or BUILTIN_EMBEDDING_DIMENSIONS
)
endpoint = _openai_endpoint(provider, profile.embedding_base_url)
return _collection_base(provider, endpoint, model, profile.embedding_dimensions)
def _collection_base(
provider: ProviderKind,
endpoint: str | None,
@@ -19,6 +19,7 @@ from pydantic import BaseModel, ConfigDict, Field, StrictBool, field_validator
from one_memory.memory_features import MemoryFeatures
from one_memory.memory_queue import ExtractionWindowPreset
from one_memory.ui_language import UI_LANGUAGES, UiLanguage
_SOURCE_ID = re.compile(r"^[a-z0-9][a-z0-9-]{0,127}$")
@@ -73,6 +74,10 @@ class ConfigDocument(BaseModel):
#: SDK retrieval and extraction features a person set; an unset one keeps
#: the SDK default. See `one_memory.memory_features`.
memory_features: MemoryFeatures = Field(default_factory=MemoryFeatures)
#: The display language the App shows (its choice, or the system language
#: it follows); None = none set. See `one_memory.ui_language` for what it
#: does to the SDK.
ui_language: UiLanguage | None = None
@field_validator("sources")
@classmethod
@@ -95,7 +100,7 @@ class ConfigFileError(RuntimeError):
pass
_WRITTEN_WHEN_SET = frozenset({"memory_mode", "evolve_schedule", "memory_features"})
_WRITTEN_WHEN_SET = frozenset({"memory_mode", "evolve_schedule", "memory_features", "ui_language"})
class ConfigStore:
@@ -183,6 +188,19 @@ class ConfigStore:
self._document = updated
return updated
def ui_language(self) -> UiLanguage | None:
with self._lock:
return self._document.ui_language
def set_ui_language(self, language: UiLanguage | None) -> ConfigDocument:
if language is not None and language not in UI_LANGUAGES:
raise ValueError("invalid_ui_language")
with self._lock:
updated = self._document.model_copy(update={"ui_language": language})
self._write(updated)
self._document = updated
return updated
def migrate_extraction_window_preset(self, preset: ExtractionWindowPreset) -> ConfigDocument:
"""Adopt the legacy SQL value only when config.json has no value."""
if not isinstance(preset, ExtractionWindowPreset):
@@ -253,6 +253,9 @@ MODEL_NOT_READY_CODES = frozenset(
"model_secret_unavailable",
"secure_storage_unavailable",
"hy_memory_client_unavailable",
# The active profile's embedding is not the live space's: activation
# must go through an embedding switch, so the write waits for it.
"embedding_space_mismatch",
}
)
@@ -2565,6 +2568,9 @@ async def execute_job(
"model_sedimentation_disabled",
# Evolve holds this user; the job runs once it is done.
"memory_evolve_in_progress",
# An embedding switch paused writes: the job stays queued and
# sediments into the new space once the switch commits.
"embedding_switch_in_progress",
}
and not hy_invoked
):
@@ -1383,6 +1383,83 @@ async def _050_model_probe_operations_reserved_at(cursor: Any) -> None:
)
def embedding_spaces_ddl() -> str:
"""The embedding space registry (`embedding_spaces.py`); the main Core SQLite only.
Unversioned on purpose, like the visibility tables: a migration raises
the build's schema version, and a library this build opened would then
refuse an older package. The table is additive and created idempotently;
an older build ignores it and opens the space its active profile names,
which is the registered space on every home that never switched. SQLite
only (partial unique indexes): it lives in the main Core database.
"""
return """
CREATE TABLE IF NOT EXISTS embedding_spaces (
space_id TEXT PRIMARY KEY NOT NULL,
state TEXT NOT NULL CHECK(state IN ('live','building','stray','retired','dropped')),
model TEXT,
dims INTEGER,
name_dims INTEGER,
collection_base TEXT NOT NULL,
naming TEXT NOT NULL CHECK(naming IN ('legacy','v2','discovered')),
graph_dir TEXT,
provider_kind TEXT,
profile_id TEXT,
profile_revision INTEGER,
vectors INTEGER,
created_at TEXT NOT NULL,
live_since TEXT,
retired_at TEXT,
keep_until TEXT,
dropped_at TEXT
);
CREATE UNIQUE INDEX IF NOT EXISTS embedding_spaces_one_live
ON embedding_spaces(state) WHERE state = 'live';
CREATE UNIQUE INDEX IF NOT EXISTS embedding_spaces_one_physical
ON embedding_spaces(collection_base, COALESCE(name_dims, 0))
WHERE state != 'dropped';
"""
def embedding_switch_jobs_ddl() -> str:
"""Embedding switch and merge jobs (`embedding_switch.py`); unversioned, as above.
At most one job is open at a time (the partial unique index), and it
lives beside the profiles and the registry so the commit is one
transaction over all three.
"""
return """
CREATE TABLE IF NOT EXISTS embedding_switch_jobs (
job_id TEXT PRIMARY KEY NOT NULL,
kind TEXT NOT NULL CHECK(kind IN ('switch','merge')),
state TEXT NOT NULL CHECK(state IN (
'preparing','reembedding','verifying','committing','done','failed','rolled_back'
)),
source_space_id TEXT NOT NULL,
target_space_id TEXT NOT NULL,
stray_space_ids TEXT NOT NULL DEFAULT '[]',
target_profile_id TEXT,
target_revision INTEGER,
prior_profile_id TEXT,
requested_by TEXT NOT NULL,
plan_json TEXT NOT NULL,
progress_json TEXT NOT NULL DEFAULT '{}',
verification_json TEXT,
error_code TEXT,
error_detail TEXT,
cancel_requested INTEGER NOT NULL DEFAULT 0,
owner_pid INTEGER,
heartbeat_at TEXT,
attempt INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
finished_at TEXT
);
CREATE UNIQUE INDEX IF NOT EXISTS embedding_switch_one_open
ON embedding_switch_jobs((1)) WHERE state NOT IN ('done','rolled_back');
"""
def model_profile_seeds_ddl(dialect: str = "mysql") -> str:
"""Which presets have been seeded, so seeding happens once."""
if dialect == "sqlite":
+41 -18
View File
@@ -423,7 +423,7 @@ class ModelProber:
ModelProbeOperations(self.store).assert_no_pending(profile.profile_id)
# Measuring embedding width may create a new profile version of its own.
input_revision = profile.revision
profile = self._record_embedding_width(profile, stages)
profile, _switch_required = self._record_embedding_width(profile, stages)
self.store.record_capabilities(
profile.profile_id,
expected_revision=profile.revision,
@@ -442,32 +442,55 @@ class ModelProber:
self,
profile: ModelProfile,
stages: tuple[ProbeStage, ...],
) -> ModelProfile:
"""Store the measured vector width on the profile.
) -> tuple[ModelProfile, bool]:
"""Store the measured vector width on the profile, unless it moves the space.
The graph store fixes its embedding column width when it is first
created, so guessing this leaves every relation write failing later.
On the profile that fills the live embedding space a different width
is not recorded: it would name other collections (B2). True when the
width was refused that way: using it takes an embedding switch, which
the switch plan reports (the probe result keeps its shape).
"""
width = next(
(
stage.embedding_dimensions
for stage in stages
if stage.stage == "embedding" and stage.embedding_dimensions
),
None,
)
width = measured_width(stages)
if width is None or width == profile.embedding_dimensions:
return profile
return self.store.update(
profile.profile_id,
ModelProfilePatch(
expected_revision=profile.revision,
embedding_dimensions=width,
return profile, False
from one_memory.embedding_spaces import EmbeddingSpaceStore, probe_width_allowed
spaces = EmbeddingSpaceStore(self.store.database_path)
with spaces.transaction() as connection:
allowed = probe_width_allowed(
connection,
profile.profile_id,
width,
data_dir=self.store.database_path.parent,
)
if not allowed:
return profile, True
return (
self.store.update(
profile.profile_id,
ModelProfilePatch(
expected_revision=profile.revision,
embedding_dimensions=width,
),
self.secrets,
),
self.secrets,
False,
)
def measured_width(stages: tuple[ProbeStage, ...]) -> int | None:
return next(
(
stage.embedding_dimensions
for stage in stages
if stage.stage == "embedding" and stage.embedding_dimensions
),
None,
)
def _models_request(
profile: ModelProfile,
secret: str | None,
@@ -13,6 +13,7 @@ from pydantic import BaseModel, ConfigDict, Field, field_validator
from one_memory.database import take_the_write_lock_first
from one_memory.database_access import main_database
from one_memory.embedding_spaces import probe_width_allowed
from one_memory.memory_server.migrations import model_probe_operations_ddl
from one_memory.model_probe import ModelProbeResult, ProbeStage
from one_memory.model_profiles import ModelCapabilities, ModelProfileStore
@@ -349,7 +350,19 @@ class ModelProbeOperations:
)
revision = profile["revision"]
now = datetime.now(UTC).isoformat()
if width is not None and width != profile["embedding_dimensions"]:
# The profile that fills the live embedding space records only the
# space's own width: another one would move writes to new, empty
# collections (B2). Using it takes an embedding switch.
if (
width is not None
and width != profile["embedding_dimensions"]
and probe_width_allowed(
db,
profile["profile_id"],
width,
data_dir=self.store.database_path.parent,
)
):
revision += 1
db.execute(
"UPDATE model_profiles SET embedding_dimensions=?,revision=? "
@@ -703,10 +703,16 @@ class ModelProfileStore:
}
)
# User changes to service/model/credentials need a new explicit activation.
# Probe-discovered embedding width alone retains its existing behavior.
# Probe-discovered embedding width alone retains its existing behavior;
# replacing one declared width with another is a new embedding
# identity, and activating it goes through an embedding switch.
deactivate = (
patch.api_key is not None
or patch.clear_api_key
or (
row["embedding_dimensions"] is not None
and next_values["embedding_dimensions"] != row["embedding_dimensions"]
)
or any(
row[key] != next_values[key]
for key in (
@@ -47,10 +47,13 @@ class SdkMemoryRuntime:
extraction_settings: ExtractionSettingsStore,
data_dir: Path,
memory_mode: Callable[[], str] = lambda: "pro",
writes_open: Callable[[], bool] = lambda: True,
) -> None:
self.coordinator = coordinator
#: The SDK mode, for the graph store's path (`hy_graph_admin`).
self.memory_mode = memory_mode
#: False while an embedding switch pauses model writes.
self.writes_open = writes_open
self.read_handle = read_handle
self.write_queue = write_queue
self.extraction_settings = extraction_settings
@@ -111,6 +114,8 @@ class SdkMemoryRuntime:
)
async def digest(self) -> dict[str, object]:
if not self.writes_open():
raise HySdkError("embedding_switch_in_progress")
async with self.coordinator.use_client() as client:
try:
raw = await asyncio.to_thread(
@@ -151,12 +156,14 @@ class SdkMemoryRuntime:
if str(error) == "model_profile_not_active":
return None
raise
space = await self.coordinator.live_space()
return await asyncio.to_thread(
inspect_profile_graph_width,
profile,
api_key=secret,
data_dir=self.data_dir,
mode=self.memory_mode(),
graph_dir=space.graph_dir if space is not None and space.graph_dir else "graph",
)
async def decide_graph_width(
@@ -167,6 +174,7 @@ class SdkMemoryRuntime:
profile, secret = await self.coordinator.active_profile_credentials()
except RuntimeError as error:
raise HySdkError(str(error)) from error
space = await self.coordinator.live_space()
return await asyncio.to_thread(
decide_profile_graph_width,
profile,
@@ -174,6 +182,7 @@ class SdkMemoryRuntime:
data_dir=self.data_dir,
decision=decision,
mode=self.memory_mode(),
graph_dir=space.graph_dir if space is not None and space.graph_dir else "graph",
)
@asynccontextmanager
@@ -183,6 +192,8 @@ class SdkMemoryRuntime:
Callers record these against the edit or deletion instead of failing
the request that already committed it.
"""
if not self.writes_open():
raise HySdkError("embedding_switch_in_progress")
async with AsyncExitStack() as stack:
try:
client = await stack.enter_async_context(self.coordinator.use_client())
@@ -222,13 +222,34 @@ def runtime_sdk_client(
raise DataPlaneMigrationError("sdk_migration_secure_storage_unavailable") from None
if secret is None:
raise DataPlaneMigrationError("sdk_migration_model_secret_unavailable")
lease = acquire_client(profile, api_key=secret, data_dir=data_dir)
# The registry's live space when this home has one and the profile fills
# it; otherwise the space the profile named before the registry existed.
# A switch in progress never changes it: the profile and the live space
# change together, at the switch's commit.
space = live_space_for(database_path, profile)
lease = (
acquire_client(profile, space, api_key=secret, data_dir=data_dir)
if space is not None
else acquire_client(profile, api_key=secret, data_dir=data_dir)
)
try:
yield lease.client
finally:
lease.close()
def live_space_for(database_path: Path, profile: ModelProfileSnapshot) -> Any:
"""The live embedding space `profile` reads, or None to derive it from the profile."""
from one_memory.embedding_spaces import embedding_identity, read_live_space
space = read_live_space(database_path)
if space is None or space.identity is None:
return None
if not embedding_identity(profile).matches(space.identity):
return None
return space
def verify_readback(
client: Any, legacy: LegacyMemoryIds, *, chat_missing_before_migration: int = 0
) -> ReadbackReport:
@@ -1,20 +1,25 @@
"""The read client, kept in the embedding space the active profile writes to.
"""The read client, kept in the live embedding space.
Every read used to open the built-in 512-dim collection while writes followed
the active profile, so a custom embedding read an empty space (#242). This
handle opens the active profile's `retrieval_view` instead, and the built-in
space only when no profile is active.
the active profile, so a custom embedding read an empty space (#242). Reads
now open the space the registry pins as live (`embedding_spaces`), which is
the space writes fill. The embedding that reaches it is the active profile's
when its model and width are the space's -- a new endpoint for the same model
keeps the space -- and otherwise the profile version that filled the space
(the built-in model for the built-in space). A switch that is still running
never changes this: reads keep serving the old space until its commit.
A profile change builds a new client and retires the old one; a retired client
is closed once no task still uses it, as `SdkWriteCoordinator` does for writes
(#218). A use is the task that asked for the client: callers keep their
`client = await handle.client()` shape, and the use ends with that task (one
HTTP request, one retention pass).
A change of space or profile builds a new client and retires the old one; a
retired client is closed once no task still uses it, as `SdkWriteCoordinator`
does for writes (#218). A use is the task that asked for the client: callers
keep their `client = await handle.client()` shape, and the use ends with that
task (one HTTP request, one retention pass).
The memory mode (lite / pro / ultra) is fixed per SDK client, so a mode change
rebuilds and retires the same way, and so do the SDK features
(`memory_features`): handles are keyed by (profile, mode, features). Neither
mode nor features names a collection; both are separate client parameters.
(`memory_features`) and the SDK prompt language (`ui_language`): handles are
keyed by (space, profile, mode, features, language). None of mode, features
or language names a collection; each is a separate client parameter.
"""
from __future__ import annotations
@@ -24,23 +29,34 @@ from collections.abc import Callable
from pathlib import Path
from typing import Any
from one_memory.embedding_health import EmbeddingGuardedClient, EmbeddingHealth
from one_memory.embedding_spaces import (
EmbeddingSpace,
EmbeddingSpaceStore,
embedding_identity,
registry_for,
)
from one_memory.hy_graph_admin import effective_memory_mode
from one_memory.hy_sdk_runtime import (
BUILTIN_RETRIEVAL_PROFILE_ID,
SdkClientHandle,
builtin_retrieval_snapshot,
embedding_endpoint,
language_kwargs,
retrieval_view,
)
from one_memory.hy_sdk_types import HySdkRuntimeError
from one_memory.memory_features import FeatureOverrides
from one_memory.model_profiles import ModelProfileStore, ProviderKind
from one_memory.model_profiles import ModelProfileSnapshot, ModelProfileStore, ProviderKind
from one_memory.secrets import SecretStore, SecretStoreUnavailableError
_BUILTIN = (BUILTIN_RETRIEVAL_PROFILE_ID, 0)
#: (space id, profile id, revision, memory mode, features, SDK language)
_HandleKey = tuple[str, str, int, str, FeatureOverrides, str | None]
class ActiveProfileReadHandle(SdkClientHandle):
"""An `SdkClientHandle` whose profile follows the active one."""
"""An `SdkClientHandle` that follows the live space and its embedding."""
def __init__(
self,
@@ -50,6 +66,9 @@ class ActiveProfileReadHandle(SdkClientHandle):
data_dir: Path,
memory_mode: Callable[[], str] = effective_memory_mode,
features: Callable[[], FeatureOverrides] = lambda: (),
spaces: EmbeddingSpaceStore | None = None,
health: EmbeddingHealth | None = None,
language: Callable[[], str | None] = lambda: None,
) -> None:
super().__init__(builtin_retrieval_snapshot(), api_key=None, data_dir=data_dir)
self.profiles = profiles
@@ -57,7 +76,13 @@ class ActiveProfileReadHandle(SdkClientHandle):
self.data_dir = data_dir
self.memory_mode = memory_mode
self.current_features = features
self._handles: dict[tuple[str, int, str, FeatureOverrides], SdkClientHandle] = {}
self.current_language = language
#: None only for a profile store without a database (test doubles).
self.spaces = spaces if spaces is not None else registry_for(profiles)
self.health = health or EmbeddingHealth()
self._handles: dict[_HandleKey, SdkClientHandle] = {}
#: Whether an active profile version reads a space, per (profile, space).
self._matches: dict[tuple[str, int, str], bool] = {}
#: Tasks in flight per handle; a retired handle closes at zero.
self._uses: dict[SdkClientHandle, int] = {}
self._retired: set[SdkClientHandle] = set()
@@ -65,22 +90,54 @@ class ActiveProfileReadHandle(SdkClientHandle):
self._handle_lock = asyncio.Lock()
async def client(self) -> Any:
profile = await asyncio.to_thread(self.profiles.active_identity) or _BUILTIN
identity = (*profile, self.memory_mode(), self.current_features())
handle = await self.current_handle()
return self._guarded(handle, await handle.client())
def _guarded(self, handle: SdkClientHandle, client: Any) -> Any:
"""A remote embedding's searches report `embedding_unavailable` (no fallback)."""
profile = getattr(handle, "profile", None)
if not isinstance(profile, ModelProfileSnapshot):
return client
endpoint = embedding_endpoint(profile)
if endpoint is None:
return client
space = handle.space
name_dims = space.name_dims if space is not None else profile.embedding_dimensions
return EmbeddingGuardedClient(
client,
lambda: self.health.available(
profile, endpoint=endpoint, name_dims=name_dims, api_key=handle.api_key
),
)
async def current_handle(self) -> SdkClientHandle:
"""The handle of the live space, held for the asking task."""
space, identity = await asyncio.to_thread(self._resolve)
mode, features = self.memory_mode(), self.current_features()
language = self.current_language()
key = (
space.space_id if space is not None else "",
*identity,
mode,
features,
language,
)
async with self._handle_lock:
handle = self._handles.get(identity)
handle = self._handles.get(key)
if handle is None:
# Only on a change of profile: resolving a key can reach the
# system keychain, so it is not done on every read.
handle = await asyncio.to_thread(self._build, identity)
self._handles[identity] = handle
self._retired.update(value for key, value in self._handles.items() if key != identity)
self._handles = {identity: handle}
# Only on a change of space or profile: resolving a key can
# reach the system keychain, so it is not done on every read.
handle = await asyncio.to_thread(
self._build, space, identity, mode, features, language
)
self._handles[key] = handle
self._retired.update(value for name, value in self._handles.items() if name != key)
self._handles = {key: handle}
unused = {value for value in self._retired if not self._uses.get(value)}
self._retired -= unused
self._hold(handle)
await asyncio.gather(*(value.close() for value in unused))
return await handle.client()
return handle
async def close(self) -> None:
async with self._handle_lock:
@@ -88,6 +145,51 @@ class ActiveProfileReadHandle(SdkClientHandle):
self._handles, self._retired, self._uses = {}, set(), {}
await asyncio.gather(*(handle.close() for handle in handles), *self._closing)
async def close_idle(self) -> None:
"""Retire every client; the unused close now, the rest when their task ends."""
async with self._handle_lock:
self._retired.update(self._handles.values())
self._handles = {}
unused = {value for value in self._retired if not self._uses.get(value)}
self._retired -= unused
await asyncio.gather(*(value.close() for value in unused))
def _resolve(self) -> tuple[EmbeddingSpace | None, tuple[str, int]]:
active = self.profiles.active_identity()
if self.spaces is None:
return None, active or _BUILTIN
space = self.spaces.live()
if space is None:
# A home this process has not bootstrapped yet: register what it
# already uses, under the name it already has.
space = self.spaces.bootstrap(
self.profiles.snapshot(*active) if active is not None else None
)
if active is not None and self._reads(active, space):
return space, active
if space.profile_id is not None and space.profile_revision is not None:
identity = (space.profile_id, space.profile_revision)
try:
self.profiles.snapshot(*identity)
except KeyError:
# Its endpoint and key went with the profile; the built-in
# model cannot read these vectors. Say so, never read wrong.
raise HySdkRuntimeError("embedding_space_profile_missing") from None
return space, identity
return space, _BUILTIN
def _reads(self, identity: tuple[str, int], space: EmbeddingSpace) -> bool:
key = (*identity, space.space_id)
known = self._matches.get(key)
if known is None:
if space.identity is None:
known = False
else:
profile = self.profiles.snapshot(*identity)
known = embedding_identity(profile).matches(space.identity)
self._matches[key] = known
return known
def _hold(self, handle: SdkClientHandle) -> None:
"""Count a use of `handle` until the asking task finishes."""
task = asyncio.current_task()
@@ -108,9 +210,18 @@ class ActiveProfileReadHandle(SdkClientHandle):
self._closing.add(closing)
closing.add_done_callback(self._closing.discard)
def _build(self, key: tuple[str, int, str, FeatureOverrides]) -> SdkClientHandle:
profile_id, revision, mode, features = key
identity = (profile_id, revision)
def _build(
self,
space: EmbeddingSpace | None,
identity: tuple[str, int],
mode: str,
features: FeatureOverrides = (),
language: str | None = None,
) -> SdkClientHandle:
# Without a registry (a profile store with no database) the handle
# opens the space its profile names, exactly as before.
named: dict[str, Any] = {"space": space} if space is not None else {}
named.update(language_kwargs(language))
if identity == _BUILTIN:
return SdkClientHandle(
builtin_retrieval_snapshot(),
@@ -118,8 +229,9 @@ class ActiveProfileReadHandle(SdkClientHandle):
data_dir=self.data_dir,
mode=mode,
features=features,
**named,
)
profile = self.profiles.snapshot(*identity)
profile: ModelProfileSnapshot = self.profiles.snapshot(*identity)
view = retrieval_view(profile)
secret: str | None = None
# Only a remote OpenAI-compatible embedding sends the shared key.
@@ -131,5 +243,5 @@ class ActiveProfileReadHandle(SdkClientHandle):
if secret is None:
raise HySdkRuntimeError("model_secret_unavailable")
return SdkClientHandle(
view, api_key=secret, data_dir=self.data_dir, mode=mode, features=features
view, api_key=secret, data_dir=self.data_dir, mode=mode, features=features, **named
)
@@ -73,6 +73,30 @@ def _stored_names(data_dir: Path, *, filled: bool = False) -> tuple[str, ...]:
return tuple(sorted(str(row[0]) for row in rows))
def filled_collection_names(data_dir: Path) -> tuple[str, ...]:
"""Every vector collection holding at least one record. Read-only.
Safe beside a serving Core: it reads Chroma's own SQLite file, never the
client system the SDK shares.
"""
return _stored_names(data_dir, filled=True)
def stored_collection_width(data_dir: Path, name: str) -> int | None:
"""The vector width Chroma fixed for `name` on its first add; None if unset."""
database = _vector_directory(data_dir) / "chroma.sqlite3"
if not database.is_file():
return None
with closing(sqlite3.connect(f"file:{database}?mode=ro", uri=True)) as connection:
try:
row = connection.execute(
"SELECT dimension FROM collections WHERE name=?", (name,)
).fetchone()
except sqlite3.Error:
return None
return int(row[0]) if row is not None and row[0] is not None else None
def legacy_vector_collections(
data_dir: Path, profile: ModelProfileSnapshot
) -> tuple[tuple[str, str], ...]:
@@ -9,8 +9,18 @@ from dataclasses import dataclass
from datetime import datetime
from typing import Any
from one_memory.embedding_spaces import (
EmbeddingSpace,
EmbeddingSpaceStore,
embedding_identity,
registry_for,
)
from one_memory.hy_graph_admin import effective_memory_mode
from one_memory.hy_sdk_runtime import SdkClientHandle, builtin_retrieval_snapshot
from one_memory.hy_sdk_runtime import (
SdkClientHandle,
builtin_retrieval_snapshot,
language_kwargs,
)
from one_memory.imports.models import EvidenceContentKind, EvidenceOrigin, EvidenceRecord
from one_memory.input_sanitization import sanitize_model_input
from one_memory.memory_features import FeatureOverrides
@@ -147,15 +157,22 @@ class SdkWriteCoordinator:
the SharedRuntime closed mid-`add`, the job failed with AttributeError
after HY was invoked, and it was left `unknown`.
Handles are keyed by profile, memory mode and the SDK features
(`memory_features`), so a mode or feature change retires handles the same
way. In lite no generation model is needed: without an
active profile the built-in local embedder writes raw turns.
Handles are keyed by profile, memory mode, the SDK features
(`memory_features`) and the SDK language (`ui_language`), so a mode,
feature or language change retires handles the same way. In lite no
generation model is needed: without an active profile the built-in
local embedder writes raw turns.
`exclusive_for_evolve(user_id)` keeps evolve and that user's writes
apart: it waits for the user's write in flight, and while it runs that
user's jobs are refused before the SDK is called
(`memory_evolve_in_progress`), so the drain puts them back.
Writes go to the live embedding space only. An active profile whose
embedding (model and width) is not the live space's is refused with
`embedding_space_mismatch`: it used to open a new, empty collection.
While an embedding switch runs, `writes_open` is false and no write
starts (`embedding_switch_in_progress`); the job waits in the queue.
"""
def __init__(
@@ -168,6 +185,9 @@ class SdkWriteCoordinator:
sedimentation_enabled: Callable[[], bool],
memory_mode: Callable[[], str] = effective_memory_mode,
memory_features: Callable[[], FeatureOverrides] = lambda: (),
spaces: EmbeddingSpaceStore | None = None,
writes_open: Callable[[], bool] = lambda: True,
sdk_language: Callable[[], str | None] = lambda: None,
) -> None:
self.profiles = profiles
self.secrets = secrets
@@ -176,7 +196,12 @@ class SdkWriteCoordinator:
self.sedimentation_enabled = sedimentation_enabled
self.memory_mode = memory_mode
self.memory_features = memory_features
self._handles: dict[tuple[str, int, str, FeatureOverrides], SdkClientHandle] = {}
self.sdk_language = sdk_language
#: Keyed by (profile id, revision, space id, memory mode, features,
#: SDK language).
self._handles: dict[
tuple[str, int, str, str, FeatureOverrides, str | None], SdkClientHandle
] = {}
#: Users evolve (or the graph backfill) is running for, the writes in
#: flight per user, and one exclusive run per user at a time. Writes
#: of one user still run side by side; only evolve waits for them.
@@ -184,6 +209,10 @@ class SdkWriteCoordinator:
self._writing: dict[str, int] = {}
self._writes_idle = asyncio.Condition()
self._exclusive: dict[str, asyncio.Lock] = {}
#: None only for a profile store without a database (test doubles):
#: then every client opens the space its profile names, as before.
self.spaces = spaces if spaces is not None else registry_for(profiles)
self.writes_open = writes_open
#: Uses in flight per handle; a retired handle closes at zero.
self._uses: dict[SdkClientHandle, int] = {}
self._retired: set[SdkClientHandle] = set()
@@ -198,6 +227,9 @@ class SdkWriteCoordinator:
) -> SdkWriteReceipt:
if not self.sedimentation_enabled():
raise RuntimeError("model_sedimentation_disabled")
if not self.writes_open():
# An embedding switch paused writes: the drain puts the job back.
raise RuntimeError("embedding_switch_in_progress")
user_id = records[0].user_id if records else "local"
if user_id in self._evolving:
# Before any SDK call: the drain returns the job to pending.
@@ -222,15 +254,15 @@ class SdkWriteCoordinator:
self._writes_idle.notify_all()
async def retire_stale_handles(self) -> None:
"""After a mode or feature change: retire handles built otherwise.
"""After a mode, feature or language change: retire handles built otherwise.
The idle ones close now. A handle still writing is closed when its
last use ends (#218), so a write that started in the old mode, or
with the old features, finishes in it.
"""
build = (self.memory_mode(), self.memory_features())
build = (self.memory_mode(), self.memory_features(), self.sdk_language())
async with self._lock:
stale = {key: handle for key, handle in self._handles.items() if key[2:] != build}
stale = {key: handle for key, handle in self._handles.items() if key[3:] != build}
for key in stale:
del self._handles[key]
self._retired.update(stale.values())
@@ -250,8 +282,11 @@ class SdkWriteCoordinator:
"""Hold one user's writes off while evolve runs for them.
New writes of the user are refused from here on; the ones already in
flight are waited for before the body runs.
flight are waited for before the body runs. Evolve and the graph
backfill write too, so an embedding switch refuses them as well.
"""
if not self.writes_open():
raise RuntimeError("embedding_switch_in_progress")
async with self._exclusive.setdefault(user_id, asyncio.Lock()):
self._evolving.add(user_id)
try:
@@ -271,6 +306,19 @@ class SdkWriteCoordinator:
self._handles, self._retired, self._uses = {}, set(), {}
await asyncio.gather(*(handle.close() for handle in handles))
def in_use(self) -> int:
"""Writes and reads holding a client right now."""
return sum(self._uses.values())
async def close_idle(self) -> None:
"""Retire every handle; the unused close now, the rest when released."""
async with self._lock:
self._retired.update(self._handles.values())
self._handles = {}
unused = {value for value in self._retired if not self._uses.get(value)}
self._retired -= unused
await asyncio.gather(*(value.close() for value in unused))
@asynccontextmanager
async def use_client(self) -> AsyncIterator[Any]:
"""The public SDK client of the active write profile, kept open while used."""
@@ -281,20 +329,32 @@ class SdkWriteCoordinator:
self,
) -> tuple[ModelProfileSnapshot, str | None]:
"""Resolve one stable profile snapshot for graph administration."""
_identity, profile, secret = await asyncio.to_thread(self._read_active_credentials)
_identity, profile, secret, _space = await asyncio.to_thread(self._read_active_credentials)
return profile, secret
async def live_space(self) -> EmbeddingSpace | None:
if self.spaces is None:
return None
return await asyncio.to_thread(self._live_space)
@asynccontextmanager
async def _active_handle(self) -> AsyncIterator[SdkClientHandle]:
"""Use the active profile's handle; retire the others, closing the unused."""
mode = self.memory_mode()
features = self.memory_features()
profile_identity, profile, secret = await asyncio.to_thread(
language = self.sdk_language()
profile_identity, profile, secret, space = await asyncio.to_thread(
self._read_active_credentials, mode
)
identity = (*profile_identity, mode, features)
key = (
*profile_identity,
space.space_id if space is not None else "",
mode,
features,
language,
)
async with self._lock:
handle = self._handles.get(identity)
handle = self._handles.get(key)
if handle is None:
handle = SdkClientHandle(
profile,
@@ -303,10 +363,12 @@ class SdkWriteCoordinator:
http_transport_factory=self.transport_factory(profile),
mode=mode,
features=features,
space=space,
**language_kwargs(language),
)
self._handles[identity] = handle
self._retired.update(value for key, value in self._handles.items() if key != identity)
self._handles = {identity: handle}
self._handles[key] = handle
self._retired.update(value for name, value in self._handles.items() if name != key)
self._handles = {key: handle}
self._uses[handle] = self._uses.get(handle, 0) + 1
unused = {value for value in self._retired if not self._uses.get(value)}
self._retired -= unused
@@ -325,18 +387,51 @@ class SdkWriteCoordinator:
self._retired.discard(handle)
await handle.close()
def _live_space(self) -> EmbeddingSpace:
assert self.spaces is not None
space = self.spaces.live()
if space is None:
active = self.profiles.active_identity()
space = self.spaces.bootstrap(
self.profiles.snapshot(*active) if active is not None else None
)
return space
def _lite_credentials(
self,
) -> tuple[tuple[str, int], ModelProfileSnapshot, str | None, EmbeddingSpace | None]:
space = self._live_space() if self.spaces is not None else None
if space is None or space.profile_id is None or space.profile_revision is None:
builtin = builtin_retrieval_snapshot()
return (builtin.profile_id, builtin.revision), builtin, None, space
identity = (space.profile_id, space.profile_revision)
profile = self.profiles.snapshot(*identity)
try:
secret = self.profiles.resolve_snapshot_secret(*identity, self.secrets)
except SecretStoreUnavailableError:
raise RuntimeError("secure_storage_unavailable") from None
if profile.has_api_key and secret is None:
raise RuntimeError("model_secret_unavailable")
return identity, profile, secret, space
def _read_active_credentials(
self,
mode: str = "pro",
) -> tuple[tuple[str, int], ModelProfileSnapshot, str | None]:
) -> tuple[tuple[str, int], ModelProfileSnapshot, str | None, EmbeddingSpace | None]:
identity = self.profiles.active_identity()
if identity is None:
if mode == "lite":
# lite stores raw turns with one embedding and calls no model.
builtin = builtin_retrieval_snapshot()
return (builtin.profile_id, builtin.revision), builtin, None
# lite stores raw turns with one embedding and calls no model:
# the live space's own, so they land where reads look.
return self._lite_credentials()
raise RuntimeError("model_profile_not_active")
profile = self.profiles.snapshot(*identity)
space = self._live_space() if self.spaces is not None else None
if space is not None and (
space.identity is None or not embedding_identity(profile).matches(space.identity)
):
# Never a new, empty collection: the switch moves the space.
raise RuntimeError("embedding_space_mismatch")
try:
secret = self.profiles.resolve_snapshot_secret(
*identity,
@@ -348,7 +443,7 @@ class SdkWriteCoordinator:
raise RuntimeError("model_profile_changed")
if profile.has_api_key and secret is None:
raise RuntimeError("model_secret_unavailable")
return identity, profile, secret
return identity, profile, secret, space
@dataclass(frozen=True)
@@ -145,6 +145,10 @@ async def search_within(
return (), "failed"
error = pending.exception()
if error is not None:
if getattr(error, "code", None) == "embedding_unavailable":
# Not "HY found nothing": the query could not be embedded, and
# the caller must say so instead of answering empty.
raise error
return (), "failed"
found = pending.result()
normalized = normalize_semantic_hits(found)
@@ -0,0 +1,77 @@
"""The display language a person chose, and the SDK prompt language it implies.
The App owns the choice (its language selector) and tells Core whenever it
changes, so Core can build every SDK client with it. It is one global value in
`config.json` (`ui_language`), beside `memory_mode` and `memory_features`;
absent means no language was set (a CLI/API-only install, or
`settings language system`). An App that follows the system sends the
language it resolved and shows, not null.
The owner's rule for the SDK:
- Any supported code, zh-Hans and en included: pass it as
`MemoryConfig.language`, so extraction and the other prompts use that
language. Every display language is passed explicitly (#243).
- No language at all (never set, as on a CLI/API-only install): pass no
language. The SDK keeps detecting Chinese or English from the text, exactly
as before this setting existed. The App does not leave it unset: it sends
the language it shows, including when it follows the system.
The language never names a collection; a change rebuilds the SDK handles the
way a mode or feature change does (#218 use count).
No SDK import here: only `hy_sdk_runtime` and `hy_sdk_migration` import it.
`config_state` imports this module, so it imports nothing from there.
"""
from __future__ import annotations
from typing import Literal
UiLanguage = Literal[
"en",
"zh-Hans",
"zh-Hant",
"ja",
"es",
"de",
"pt",
"fr",
"ko",
"ru",
"id",
]
UI_LANGUAGES: tuple[UiLanguage, ...] = (
"en",
"zh-Hans",
"zh-Hant",
"ja",
"es",
"de",
"pt",
"fr",
"ko",
"ru",
"id",
)
#: Codes earlier App builds stored, and the code each one is now.
LEGACY_UI_LANGUAGES: dict[str, UiLanguage] = {"zh-CN": "zh-Hans"}
#: What status shows when no language is passed to the SDK.
SDK_LANGUAGE_AUTO = "auto"
def normalize_ui_language(value: object) -> UiLanguage | None:
"""A supported code (legacy ones mapped), or None for anything else."""
if not isinstance(value, str):
return None
value = LEGACY_UI_LANGUAGES.get(value, value)
return value if value in UI_LANGUAGES else None
def sdk_language(ui_language: str | None) -> str | None:
"""The `MemoryConfig.language` for a display language; None = SDK auto-detect.
Every supported code is passed as is; only an unset (or unknown) value
leaves the SDK to detect zh/en.
"""
return normalize_ui_language(ui_language)
+5 -1
View File
@@ -88,6 +88,9 @@
| `test_diagnostic_trace_api.py` | `integration` | 覆盖 trace lookup is disabled until detailed logging is enabled、trace lookup returns only safe structured steps、log level route applies current process level 等 4 个场景。 |
| `test_disable_thinking.py` | `contract` | 运行期生成请求须按档案可选关闭思考;默认不向厂商下发未知字段。 |
| `test_dual_engine_helper.py` | `contract` | Tests for the dual-engine harness itself. |
| `test_embedding_switch.py` | `contract` | Switching the embedding model, and merging stray spaces, without losing a memory. |
| `test_embedding_switch_api.py` | `integration` | The embedding switch through Core's HTTP surface and lifespan. |
| `test_embedding_switch_sdk_adapter.py` | `contract` | The re-embed adapter against the SDK's real re-embed module and Chroma. |
| `test_event_loop_blocking.py` | `contract` | Proof, as a test, that blocking work on the loop freezes it — and the fix works. |
| `test_evidence_store.py` | `contract` | 覆盖 same source message merges origins without duplication、identical text from different sources is not deduplicated、rescanning an unchanged file does not write the message again 等 26 个场景。 |
| `test_export_probe.py` | `contract` | What a probe may say about a stranger's chat history, and what it may not. |
@@ -301,6 +304,7 @@
| `test_sync_facade.py` | `contract` | A2: the sync facade must keep the sync shape call sites already use. |
| `test_tail_metadata_lane.py` | `contract` | Post-response metadata retries must be bounded and never block the route. |
| `test_two_level_discovery.py` | `contract` | 3925a kernel item 1: two-level discovery. |
| `test_ui_language.py` | `contract` | The App's display language, and the SDK prompt language it implies. |
| `test_ui_session.py` | `contract` | 覆盖 exchange is single use and creates a session、expired exchange cannot create a session、exchange is bound to the core instance 等 5 个场景。 |
| `test_ui_session_api.py` | `integration` | 覆盖 ui session exchange sets httponly strict cookie、ui session requires bearer for exchange、ui session reports missing web assets 场景。 |
| `test_unanswered_turn.py` | `contract` | BUG-06: which conversations end on a question nobody answered. |
@@ -314,4 +318,4 @@
| `test_workspace_recall.py` | `contract` | Workspace memories were unreachable from every host; these pin the way in. |
| `test_write_queue_lease_renewal.py` | `contract` | Rule 8: a lease must outlive the work it covers, and be renewed while it runs. |
共 309 个测试文件。
共 313 个测试文件。
@@ -14,8 +14,8 @@ from pathlib import Path
import pytest
PRIMARY_MARKERS = ("unit", "contract", "integration", "performance", "live")
EXPECTED_TEST_FILE_COUNT = 309
EXPECTED_TEST_FILES_SHA256 = "a7d6ed33175f618d2d40f401d447a5553e1347ffd51dd878370c0fc84d13396d"
EXPECTED_TEST_FILE_COUNT = 313
EXPECTED_TEST_FILES_SHA256 = "dcada5d5421767196a79e415dd47c2b6c6d67681972dbe81481b31867d318928"
# Pure transformations, validation models, parity checks and state machines.
# This list is intentionally small: uncertain files default to contract.
@@ -54,6 +54,7 @@ INTEGRATION_FILES = frozenset(
"test_backup_endpoint.py",
"test_recall_raw_fallback.py",
"test_core_ready_before_recovery.py",
"test_embedding_switch_api.py",
"test_backup_restore_api.py",
"test_c1_import_via_api_path.py",
"test_capture_deadline.py",
+301
View File
@@ -0,0 +1,301 @@
"""An in-memory `ReembedBackend` for embedding switch tests.
It keeps collections as plain dicts named exactly as the SDK names them, so a
test can lay out a home -- a live space, a stray 512 space left by an older
build, a retired space -- and check what a switch or merge did to each. The
vectors themselves are not computed: a record remembers the model and width
it was "embedded" with, which is what the assertions need.
"""
from __future__ import annotations
import asyncio
import re
import shutil
from collections.abc import Callable
from dataclasses import dataclass, field
from pathlib import Path
from one_memory.embedding_backend import (
EmbeddingTarget,
EmbeddingWidthError,
GraphCheck,
PhysicalSpace,
ReembedCancelledError,
ReembedStep,
ScannedSpace,
VectorCheck,
)
from one_memory.embedding_spaces import VECTOR_ROLES, space_collection_names
_ROLE_SUFFIX = re.compile(
r"^(?P<base>.+?)_(?P<role>coding_keys_cv|coding_keys|working_facts_cv|working_facts)"
r"(?:_(?P<dims>\d+))?$"
)
_DIMS_SUFFIX = re.compile(r"^(?P<base>.+)_(?P<dims>\d+)$")
class Crash(BaseException):
"""Stands in for the process dying: nothing in the worker catches it."""
@dataclass
class Record:
document: str
metadata: dict[str, object]
width: int
model: str
@dataclass
class FakeVectorStore:
collections: dict[str, dict[str, Record]] = field(default_factory=dict)
def put(
self,
base: str,
name_dims: int | None,
records: dict[str, tuple[str, str]],
*,
role: str = "memory",
width: int | None = None,
model: str = "old-model",
) -> None:
"""`records` maps id -> (document, user_id)."""
name = space_collection_names(base, name_dims)[role]
target = self.collections.setdefault(name, {})
for record_id, (document, user) in records.items():
target[record_id] = Record(
document=document,
metadata={"user_id": user, "content": document},
width=width or name_dims or 0,
model=model,
)
def space(self, base: str, name_dims: int | None) -> dict[str, dict[str, Record]]:
return {
role: dict(self.collections.get(name, {}))
for role, name in space_collection_names(base, name_dims).items()
if name in self.collections
}
def ids(self, base: str, name_dims: int | None, role: str = "memory") -> set[str]:
return set(self.collections.get(space_collection_names(base, name_dims)[role], {}))
def parse(name: str) -> tuple[str, int | None, str] | None:
if name.endswith("_tag_index"):
return None
match = _ROLE_SUFFIX.match(name)
if match:
dims = match.group("dims")
return match.group("base"), int(dims) if dims else None, match.group("role")
role = "memory"
if name.endswith("_entities"):
name, role = name[: -len("_entities")], "entities"
match = _DIMS_SUFFIX.match(name)
if match:
return match.group("base"), int(match.group("dims")), role
return name, None, role
class FakeReembedBackend:
"""The backend contract, over `FakeVectorStore`."""
def __init__(self, store: FakeVectorStore | None = None) -> None:
self.store = store or FakeVectorStore()
self.unavailable: str | None = None
#: Called before every batch; a test raises `Crash` or flips a cancel here.
self.before_batch: Callable[[str, int], None] = lambda _role, _done: None
#: Width the "embedding service" returns; None returns the target's.
self.returned_width: int | None = None
#: Ids the re-embed silently loses, to prove verification catches it.
self.lose_ids: set[str] = set()
self.batch_size = 2
self.reembed_calls: list[tuple[str, str]] = []
self.dropped: list[str] = []
self.graphs_built: list[Path] = []
#: When set, every batch waits for it: a switch a test holds mid-way.
self.pause: asyncio.Event | None = None
def unavailable_reason(self) -> str | None:
return self.unavailable
async def scan(self, *, per_user: bool = False) -> tuple[ScannedSpace, ...]:
grouped: dict[tuple[str, int | None], dict[str, str]] = {}
for name in self.store.collections:
parsed = parse(name)
if parsed is None:
continue
base, dims, role = parsed
grouped.setdefault((base, dims), {})[role] = name
out = []
for (base, dims), roles in sorted(
grouped.items(), key=lambda kv: (kv[0][0], kv[0][1] or 0)
):
counts = {role: len(self.store.collections[name]) for role, name in roles.items()}
widths = {
record.width
for name in roles.values()
for record in self.store.collections[name].values()
}
users: dict[str, int] = {}
if per_user:
for name in roles.values():
for record in self.store.collections[name].values():
user = str(record.metadata.get("user_id") or "")
users[user] = users.get(user, 0) + 1
out.append(
ScannedSpace(
collection_base=base,
name_dims=dims,
width=widths.pop() if len(widths) == 1 else None,
counts=counts,
collections=roles,
per_user=users,
)
)
return tuple(out)
async def diff(self, source: PhysicalSpace, target: PhysicalSpace) -> dict[str, int]:
target_names = target.names()
out = {}
for role, name in source.names().items():
have = set(self.store.collections.get(target_names.get(role, ""), {}))
out[role] = len(set(self.store.collections.get(name, {})) - have)
return out
async def reembed(
self,
source: PhysicalSpace,
target: PhysicalSpace,
*,
embedding: EmbeddingTarget,
cancel: Callable[[], bool],
on_step: Callable[[ReembedStep], None],
batch_size: int = 64,
) -> None:
del batch_size
if target.dims is None:
raise ValueError("the target space needs an explicit width")
self.reembed_calls.append((source.collection_base, target.collection_base))
target_names = target.names()
model = embedding.space.model or "?"
for role in VECTOR_ROLES:
name = source.names().get(role)
if name is None or name not in self.store.collections:
continue
records = sorted(self.store.collections[name].items())
destination = self.store.collections.setdefault(target_names[role], {})
done = embedded = kept = 0
for start in range(0, len(records), self.batch_size):
if cancel():
raise ReembedCancelledError("embedding_switch_cancelled")
self.before_batch(role, done)
if self.pause is not None:
await self.pause.wait()
batch = records[start : start + self.batch_size]
width = self.returned_width or target.dims
if width != target.dims:
raise EmbeddingWidthError("embedding_dimensions_mismatch")
for record_id, record in batch:
if record_id in destination:
kept += 1
continue
if record_id in self.lose_ids:
continue
destination[record_id] = Record(
document=record.document,
metadata=dict(record.metadata),
width=width,
model=model,
)
embedded += 1
done += len(batch)
on_step(
ReembedStep(
phase="vectors",
role=role,
done=done,
total=len(records),
embedded=embedded,
kept=kept,
)
)
async def verify(
self, source: PhysicalSpace, target: PhysicalSpace, *, embedding: EmbeddingTarget
) -> VectorCheck:
del embedding
problems: list[str] = []
per_role: dict[str, dict[str, int]] = {}
users_source: dict[str, int] = {}
users_target: dict[str, int] = {}
target_names = target.names()
for role, name in source.names().items():
src = self.store.collections.get(name, {})
tgt = self.store.collections.get(target_names.get(role, ""), {})
missing = set(src) - set(tgt)
extra = set(tgt) - set(src)
mismatched = {
key for key in set(src) & set(tgt) if src[key].document != tgt[key].document
}
bad = sum(1 for record in tgt.values() if record.width != target.dims)
per_role[role] = {
"source": len(src),
"target": len(tgt),
"missing": len(missing),
"extra": len(extra),
"mismatched": len(mismatched),
"bad_dims": bad,
}
for label, value in (
("missing", missing),
("extra", extra),
("mismatched", mismatched),
):
if value:
problems.append(f"{role}: {len(value)} {label}")
if bad:
problems.append(f"{role}: {bad} bad dims")
for record in src.values():
user = str(record.metadata.get("user_id") or "")
users_source[user] = users_source.get(user, 0) + 1
for record in tgt.values():
user = str(record.metadata.get("user_id") or "")
users_target[user] = users_target.get(user, 0) + 1
for user in sorted(set(users_source) | set(users_target)):
if users_source.get(user, 0) != users_target.get(user, 0):
problems.append(f"user {user!r}: counts differ")
return VectorCheck(per_role=per_role, per_user=users_target, problems=tuple(problems))
async def graph_vectors(self, path: Path) -> int | None:
return 3 if path.exists() else None
async def reembed_graph(
self,
source: Path,
target: Path,
*,
embedding: EmbeddingTarget,
target_dims: int,
cancel: Callable[[], bool],
on_step: Callable[[ReembedStep], None],
) -> None:
del embedding, cancel
target.mkdir(parents=True, exist_ok=True)
(target / "width").write_text(str(target_dims))
self.graphs_built.append(target)
on_step(ReembedStep(phase="graph", role="Memory", done=3, total=3, embedded=3))
async def verify_graph(self, source: Path, target: Path, *, target_dims: int) -> GraphCheck:
ok = (target / "width").is_file() and (target / "width").read_text() == str(target_dims)
return GraphCheck(ok=ok, problems=() if ok else ("graph width",))
async def drop(self, space: PhysicalSpace) -> None:
for name in space.names().values():
if self.store.collections.pop(name, None) is not None:
self.dropped.append(name)
async def drop_graph(self, path: Path) -> None:
shutil.rmtree(path, ignore_errors=True)
@@ -164,7 +164,16 @@ def test_metadata_repair_never_overlaps_an_sdk_client(tmp_path, monkeypatch):
events.append("repair_finished")
return MetadataRepairResult(None, 0, 0, 0)
def open_client(profile, *, api_key, data_dir, http_transport_factory, mode="pro", features=()):
def open_client(
profile,
space=None,
*,
api_key,
data_dir,
http_transport_factory,
mode="pro",
features=(),
):
events.append("client_opened")
return ClientLease(object(), data_dir.resolve())
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,383 @@
"""The embedding switch through Core's HTTP surface and lifespan.
Activation refusals and the empty-home path, a whole switch driven over the
API with writes paused and intake still queueing, the startup merge of a
stray space before the drain starts, and `embedding_unavailable` on recall
and context search when the remote embedding does not answer.
"""
from __future__ import annotations
import asyncio
import hashlib
import time
from types import SimpleNamespace
from uuid import uuid4
import httpx
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory import embedding_health
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.model_profiles import ModelCapabilities, ModelProfileInput, ProviderKind
from one_memory.secrets import InMemorySecretStore
from tests.embedding_fakes import FakeReembedBackend
pytestmark = pytest.mark.fresh_sdk
ENDPOINT = "https://models.example/v1"
def builtin_base() -> str:
identity = "\0".join(("local_onnx", "local", "bge-small-zh-v1.5", "512"))
return "memhub_" + hashlib.sha256(identity.encode()).hexdigest()[:16]
@pytest.fixture
def backend(monkeypatch) -> FakeReembedBackend:
fake = FakeReembedBackend()
monkeypatch.setattr(
"one_memory.embedding_switch.default_reembed_backend", lambda _data_dir: fake
)
return fake
def application(tmp_path):
return create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("switch-test"),
imports_enabled=False,
automatic_memory_processing=False,
live_sync_enabled=False,
),
secret_store=InMemorySecretStore(),
)
def client_for(app):
return AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer switch-test"},
)
async def recovered(client) -> dict:
deadline = time.monotonic() + 10
while time.monotonic() < deadline:
body = (await client.get("/api/v1/health")).json()
if body["recovery"] in {"done", "failed"}:
return body
await asyncio.sleep(0.01)
raise AssertionError("recovery did not finish")
async def until(client, predicate, *, seconds: float = 10.0) -> dict:
deadline = time.monotonic() + seconds
while time.monotonic() < deadline:
body = (await client.get("/api/v1/embedding-switches/current")).json()
if predicate(body):
return body
await asyncio.sleep(0.01)
raise AssertionError(f"switch never reached the expected state: {body}")
def remote_profile(app, *, model="bge-m3", dims=1024, name="custom", endpoint=ENDPOINT):
store, secrets = app.state.model_profile_store, app.state.secret_store
created = store.create(
ModelProfileInput(
name=name,
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
base_url=endpoint,
model="chat",
embedding_provider_kind=ProviderKind.OPENAI_COMPATIBLE,
embedding_base_url=endpoint,
embedding_model=model,
embedding_dimensions=dims,
api_key=SecretStr("sk-test"),
),
secrets,
)
store.record_capabilities(
created.profile_id,
expected_revision=created.revision,
capabilities=ModelCapabilities(
endpoint_reachable=True,
model_available=True,
structured_output=True,
embedding_available=True,
),
secrets=secrets,
)
return created
def test_activating_another_embedding_answers_with_the_plan(tmp_path, backend):
backend.store.put(builtin_base(), 512, {"a": ("one", "local"), "b": ("two", "bob")})
async def scenario():
app = application(tmp_path)
async with app.router.lifespan_context(app), client_for(app) as client:
await recovered(client)
profile = remote_profile(app)
response = await client.post(f"/api/v1/model-profiles/{profile.profile_id}/activate")
assert response.status_code == 409
body = response.json()
assert body["detail"] == "embedding_switch_required"
plan = body["plan"]
assert plan["needed"] is True and plan["total_vectors"] == 2 and plan["users"] == 2
assert plan["source"]["model"] == "bge-small-zh-v1.5"
assert (plan["target"]["model"], plan["target"]["dims"]) == ("bge-m3", 1024)
assert app.state.model_profile_store.active_identity() is None
asyncio.run(scenario())
def test_an_empty_home_activates_another_embedding_directly(tmp_path, backend):
async def scenario():
app = application(tmp_path)
async with app.router.lifespan_context(app), client_for(app) as client:
await recovered(client)
builtin = app.state.embedding_spaces.require_live()
profile = remote_profile(app)
response = await client.post(f"/api/v1/model-profiles/{profile.profile_id}/activate")
assert response.status_code == 200, response.text
assert response.json()["is_active"] is True
live = app.state.embedding_spaces.require_live()
assert (live.model, live.dims) == ("bge-m3", 1024)
assert app.state.embedding_spaces.get(builtin.space_id).state == "dropped"
# The same embedding again: a plain activation, no plan.
again = remote_profile(app, name="same model, other chat model")
assert (
await client.post(f"/api/v1/model-profiles/{again.profile_id}/activate")
).status_code == 200
assert app.state.embedding_spaces.require_live().space_id == live.space_id
asyncio.run(scenario())
def test_a_switch_over_the_api_pauses_writes_and_keeps_intake(tmp_path, backend):
backend.store.put(
builtin_base(),
512,
{f"m{i}": (f"memory {i}", "local") for i in range(6)},
model="bge-small-zh-v1.5",
)
async def scenario():
app = application(tmp_path)
async with app.router.lifespan_context(app), client_for(app) as client:
await recovered(client)
old = app.state.embedding_spaces.require_live()
profile = remote_profile(app)
planned = await client.post(
"/api/v1/embedding-switches/plan", json={"profile_id": profile.profile_id}
)
assert planned.status_code == 200
plan = planned.json()
stale = await client.post(
"/api/v1/embedding-switches",
json={
"profile_id": profile.profile_id,
"expected_revision": profile.revision,
"plan_token": "stale",
},
)
assert stale.status_code == 409
assert stale.json()["detail"] == "embedding_switch_plan_stale"
backend.pause = asyncio.Event()
started = await client.post(
"/api/v1/embedding-switches",
json={
"profile_id": profile.profile_id,
"expected_revision": profile.revision,
"plan_token": plan["plan_token"],
"requested_by": "cli",
},
)
assert started.status_code == 202, started.text
job_id = started.json()["job_id"]
running = await until(
client, lambda body: (body["job"] or {}).get("state") == "reembedding"
)
assert running["writes_paused"] is True and running["phase"] == "running"
# Direct writers wait; intake keeps queueing; status shows the switch.
edit = await client.put(
f"/api/v2/memories/{uuid4()}", json={"content": "changed", "user_id": "local"}
)
assert edit.status_code == 409
assert edit.json()["detail"] == "embedding_switch_in_progress"
delete = await client.delete(f"/api/v2/memories/{uuid4()}")
assert delete.status_code == 409
added = await client.post(
"/api/v2/memories",
json={"request_id": str(uuid4()), "text": "请记住这件事", "scope": "all-apps"},
)
assert added.status_code == 201
processing = (await client.get("/api/v1/memory-processing")).json()
assert processing["embedding_switch"]["job_id"] == job_id
assert (await client.get("/api/v1/health")).json()["embedding_switch"]["kind"] == (
"switch"
)
other = remote_profile(app, model="text-embedding-v3", dims=768, name="other")
refused = await client.post(f"/api/v1/model-profiles/{other.profile_id}/activate")
assert refused.json()["detail"] == "embedding_switch_running"
backend.pause.set()
done = await until(
client, lambda body: (body["last_finished"] or {}).get("job_id") == job_id
)
assert done["last_finished"]["state"] == "done"
assert done["writes_paused"] is False
assert app.state.model_profile_store.active_identity()[0] == profile.profile_id
spaces = (await client.get("/api/v1/embedding-spaces")).json()
states = {space["space_id"]: space["state"] for space in spaces["spaces"]}
assert states[old.space_id] == "retired"
assert spaces["live_space_id"] == done["last_finished"]["target"]["space_id"]
cleaned = await client.delete(f"/api/v1/embedding-spaces/{old.space_id}")
assert cleaned.status_code == 200
assert cleaned.json()["space"]["state"] == "dropped"
live_cleanup = await client.delete(
f"/api/v1/embedding-spaces/{spaces['live_space_id']}"
)
assert live_cleanup.json()["detail"] == "embedding_space_not_retired"
# Same model and width behind another endpoint: no re-embed.
proxied = remote_profile(app, name="proxy", endpoint="https://proxy.example/v1")
moved = await client.post(f"/api/v1/model-profiles/{proxied.profile_id}/activate")
assert moved.status_code == 200, moved.text
assert app.state.embedding_spaces.require_live().space_id == spaces["live_space_id"]
asyncio.run(scenario())
def test_cancel_over_the_api(tmp_path, backend):
backend.store.put(builtin_base(), 512, {f"m{i}": (f"m {i}", "local") for i in range(4)})
async def scenario():
app = application(tmp_path)
async with app.router.lifespan_context(app), client_for(app) as client:
await recovered(client)
profile = remote_profile(app)
plan = (
await client.post(
"/api/v1/embedding-switches/plan", json={"profile_id": profile.profile_id}
)
).json()
backend.pause = asyncio.Event()
job_id = (
await client.post(
"/api/v1/embedding-switches",
json={
"profile_id": profile.profile_id,
"expected_revision": profile.revision,
"plan_token": plan["plan_token"],
},
)
).json()["job_id"]
await until(client, lambda body: (body["job"] or {}).get("state") == "reembedding")
cancelled = await client.post(f"/api/v1/embedding-switches/{job_id}/cancel")
assert cancelled.status_code == 200 and cancelled.json()["cancel_requested"]
backend.pause.set()
done = await until(client, lambda body: body["last_finished"] is not None)
assert done["last_finished"]["state"] == "rolled_back"
assert done["last_finished"]["error_code"] == "embedding_switch_cancelled"
missing = await client.post(f"/api/v1/embedding-switches/{uuid4()}/cancel")
assert missing.status_code == 404
asyncio.run(scenario())
def test_startup_merges_a_stray_space_before_writes_resume(tmp_path, backend):
backend.store.put(builtin_base(), 512, {"live-1": ("live", "local")})
stray = "memhub_" + "a" * 16
backend.store.put(stray, 1024, {"stray-1": ("left by an older build", "hermes-1")})
async def scenario():
app = application(tmp_path)
async with app.router.lifespan_context(app), client_for(app) as client:
# Core answers at once; the merge runs behind the loading screen.
assert (await client.get("/api/v1/health")).status_code == 200
await recovered(client)
done = await until(client, lambda body: body["last_finished"] is not None)
assert done["last_finished"]["kind"] == "merge"
assert done["last_finished"]["state"] == "done"
assert backend.store.ids(builtin_base(), 512) == {"live-1", "stray-1"}
spaces = (await client.get("/api/v1/embedding-spaces")).json()
stray_row = next(row for row in spaces["spaces"] if row["collection_base"] == stray)
assert stray_row["state"] == "retired" and stray_row["keep_until"]
assert app.state.embedding_switch.writes_open()
asyncio.run(scenario())
class _EmptySearchClient:
def search(self, *_args, **_kwargs):
return {"request_id": "r", "memories": {"schema": [], "proactive": [], "normal": []}}
def get(self, *_args, **_kwargs):
return None
def test_a_dead_remote_embedding_answers_embedding_unavailable(tmp_path, backend, monkeypatch):
status = {"code": 503}
def transport():
return httpx.MockTransport(
lambda request: httpx.Response(
status["code"], json={"data": [{"embedding": [0.1] * 4}]}
)
)
monkeypatch.setattr(embedding_health, "transport_factory", transport)
monkeypatch.setattr(
"one_memory.hy_sdk_runtime.acquire_client",
lambda *args, **kwargs: SimpleNamespace(client=_EmptySearchClient(), close=lambda: None),
)
async def ready(_app):
return True
monkeypatch.setattr("one_memory.api._recall_model_ready", ready)
async def scenario():
app = application(tmp_path)
async with app.router.lifespan_context(app), client_for(app) as client:
await recovered(client)
profile = remote_profile(app)
assert (
await client.post(f"/api/v1/model-profiles/{profile.profile_id}/activate")
).status_code == 200
recall = await client.post("/api/v1/recall", json={"query": "数据库"})
assert recall.status_code == 503
assert recall.json()["detail"] == "embedding_unavailable"
context = await client.post(
"/api/v2/context/search",
json={"query": "数据库", "user_id": "local", "kinds": ["memory"]},
)
assert context.status_code == 503
assert context.json()["detail"] == "embedding_unavailable"
# The service answers again: an empty result is just empty.
status["code"] = 200
app.state.sdk_client_handle.health.forget()
recall = await client.post("/api/v1/recall", json={"query": "数据库"})
assert recall.status_code == 200
asyncio.run(scenario())
@@ -0,0 +1,604 @@
"""The re-embed adapter against the SDK's real re-embed module and Chroma.
Skipped on an SDK without `hy_memory.reembed` (before 2.0.10): there the
switch is unavailable and the plan says `sdk_reembed_unavailable`, which
`test_embedding_switch.py` covers.
"""
from __future__ import annotations
import asyncio
from pathlib import Path
import hy_memory
import pytest
from one_memory.embedding_backend import EmbeddingTarget, PhysicalSpace
from one_memory.embedding_spaces import EmbeddingSpace, space_collection_names
from one_memory.hy_sdk_migration import SdkReembedBackend
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot
pytestmark = pytest.mark.skipif(
SdkReembedBackend(Path("/nonexistent")).unavailable_reason() is not None,
reason="the installed SDK has no re-embed module (hy-memory < 2.0.10)",
)
SOURCE = "memhub_" + "1" * 16
TARGET = "memhub_e_" + "2" * 16
class _Embedder:
"""A deterministic 8-wide stand-in for the target model."""
def __init__(self, width: int = 8) -> None:
self.width = width
self.texts: list[str] = []
async def embed_batch(self, texts: list[str], use_cache: bool = True) -> list[list[float]]:
self.texts.extend(texts)
return [[float(len(text) % 7 + 1)] + [0.5] * (self.width - 1) for text in texts]
async def embed(self, text: str, use_cache: bool = True) -> list[float]:
return (await self.embed_batch([text]))[0]
def get_embedding_dims(self) -> int:
return self.width
def _seed(data_dir: Path) -> None:
import chromadb
vector = data_dir / "hy-memory" / "governed-v2" / "vector"
vector.mkdir(parents=True)
client = chromadb.PersistentClient(path=str(vector))
names = space_collection_names(SOURCE, 4)
memory = client.get_or_create_collection(names["memory"], metadata={"hnsw:space": "cosine"})
memory.add(
ids=["m1", "m2", "m3"],
embeddings=[[0.1, 0.2, 0.3, 0.4]] * 3,
documents=["one", "two", "three"],
metadatas=[{"user_id": "local"}, {"user_id": "hermes-a"}, {"user_id": "hermes-b"}],
)
keys = client.get_or_create_collection(names["coding_keys"], metadata={"hnsw:space": "cosine"})
keys.add(
ids=["e1:0"],
embeddings=[[0.4, 0.3, 0.2, 0.1]],
documents=["task"],
metadatas=[{"user_id": "local"}],
)
def _target() -> EmbeddingTarget:
space = EmbeddingSpace(
space_id="target",
state="building",
model="stand-in",
dims=8,
name_dims=8,
collection_base=TARGET,
naming="v2",
created_at="2026-11-01T00:00:00+00:00",
)
return EmbeddingTarget(profile=builtin_retrieval_snapshot(), space=space, api_key=None)
def test_core_names_a_space_exactly_as_the_sdk_does() -> None:
import hy_memory
for dims in (None, 512, 1024):
sdk = hy_memory.vector_space_names(hy_memory.VectorSpace(base_name=SOURCE, dims=dims))
assert sdk == space_collection_names(SOURCE, dims)
def test_scan_reembed_verify_and_drop_through_the_sdk(tmp_path: Path, monkeypatch) -> None:
_seed(tmp_path)
backend = SdkReembedBackend(tmp_path)
embedder = _Embedder()
monkeypatch.setattr(backend, "_embedding_service", lambda _target: embedder)
source = PhysicalSpace(collection_base=SOURCE, name_dims=4, dims=4)
target = PhysicalSpace(collection_base=TARGET, name_dims=8, dims=8, model="stand-in")
steps = []
async def scenario() -> None:
scanned = await backend.scan(per_user=True)
found = {item.collection_base: item for item in scanned}
assert found[SOURCE].vectors == 4
assert found[SOURCE].per_user == {"local": 2, "hermes-a": 1, "hermes-b": 1}
await backend.reembed(
source, target, embedding=_target(), cancel=lambda: False, on_step=steps.append
)
assert not any((await backend.diff(source, target)).values())
check = await backend.verify(source, target, embedding=_target())
assert check.problems == ()
assert check.per_user == {"local": 2, "hermes-a": 1, "hermes-b": 1}
# Resumable: a second run embeds nothing more.
before = len(embedder.texts)
await backend.reembed(
source, target, embedding=_target(), cancel=lambda: False, on_step=steps.append
)
assert len(embedder.texts) == before
await backend.drop(target)
remaining = {item.collection_base for item in await backend.scan()}
assert TARGET not in remaining and SOURCE in remaining
asyncio.run(scenario())
assert {step.role for step in steps} >= {"memory", "coding_keys"}
def test_a_wrong_width_fails_before_anything_is_written(tmp_path: Path, monkeypatch) -> None:
from one_memory.embedding_backend import EmbeddingWidthError
_seed(tmp_path)
backend = SdkReembedBackend(tmp_path)
monkeypatch.setattr(backend, "_embedding_service", lambda _target: _Embedder(width=6))
source = PhysicalSpace(collection_base=SOURCE, name_dims=4, dims=4)
target = PhysicalSpace(collection_base=TARGET, name_dims=8, dims=8)
async def scenario() -> None:
with pytest.raises(EmbeddingWidthError):
await backend.reembed(
source, target, embedding=_target(), cancel=lambda: False, on_step=lambda _: None
)
found = {item.collection_base: item for item in await backend.scan()}
assert TARGET not in found or found[TARGET].vectors == 0
asyncio.run(scenario())
def test_cancel_stops_at_a_batch_boundary(tmp_path: Path, monkeypatch) -> None:
from one_memory.embedding_backend import ReembedCancelledError
_seed(tmp_path)
backend = SdkReembedBackend(tmp_path)
monkeypatch.setattr(backend, "_embedding_service", lambda _target: _Embedder())
source = PhysicalSpace(collection_base=SOURCE, name_dims=4, dims=4)
target = PhysicalSpace(collection_base=TARGET, name_dims=8, dims=8)
async def scenario() -> None:
with pytest.raises(ReembedCancelledError):
await backend.reembed(
source, target, embedding=_target(), cancel=lambda: True, on_step=lambda _: None
)
asyncio.run(scenario())
def test_a_whole_switch_through_the_real_sdk(tmp_path: Path, monkeypatch) -> None:
"""Registry, worker and commit over real Chroma collections and the SDK."""
import chromadb
from pydantic import SecretStr
from one_memory.embedding_spaces import EmbeddingSpaceStore
from one_memory.embedding_switch import EmbeddingSwitchService
from one_memory.model_profiles import (
ModelCapabilities,
ModelProfileInput,
ModelProfileStore,
ProviderKind,
)
from one_memory.secrets import InMemorySecretStore
database = tmp_path / "one-memory.sqlite3"
secrets = InMemorySecretStore()
profiles = ModelProfileStore(database)
spaces = EmbeddingSpaceStore(database)
live = spaces.bootstrap(None)
vector = tmp_path / "hy-memory" / "governed-v2" / "vector"
vector.mkdir(parents=True)
client = chromadb.PersistentClient(path=str(vector))
memory = client.get_or_create_collection(
live.collection_names()["memory"], metadata={"hnsw:space": "cosine"}
)
memory.add(
ids=["a", "b", "c"],
embeddings=[[0.1] * 512, [0.2] * 512, [0.3] * 512],
documents=["one", "two", "three"],
metadatas=[{"user_id": "local"}, {"user_id": "hermes-a"}, {"user_id": "local"}],
)
stray_base = "memhub_" + "9" * 16
stray = client.get_or_create_collection(
space_collection_names(stray_base, 768)["memory"], metadata={"hnsw:space": "cosine"}
)
stray.add(
ids=["a", "z"],
embeddings=[[0.5] * 768, [0.6] * 768],
documents=["stale", "only in the stray"],
metadatas=[{"user_id": "local"}, {"user_id": "hermes-b"}],
)
created = profiles.create(
ModelProfileInput(
name="remote",
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
base_url="https://models.example/v1",
model="chat",
embedding_provider_kind=ProviderKind.OPENAI_COMPATIBLE,
embedding_base_url="https://models.example/v1",
embedding_model="stand-in",
embedding_dimensions=8,
api_key=SecretStr("sk-test"),
),
secrets,
)
profiles.record_capabilities(
created.profile_id,
expected_revision=created.revision,
capabilities=ModelCapabilities(
endpoint_reachable=True,
model_available=True,
structured_output=True,
embedding_available=True,
),
secrets=secrets,
)
backend = SdkReembedBackend(tmp_path)
monkeypatch.setattr(backend, "_embedding_service", lambda _target: _Embedder())
service = EmbeddingSwitchService(
database_path=database,
data_dir=tmp_path,
profiles=profiles,
spaces=spaces,
secrets=secrets,
backend=backend,
worker_enabled=False,
)
service.release_scan_hold()
async def scenario() -> None:
plan = await service.plan(created.profile_id)
assert plan.needed and plan.total_vectors == 5 and plan.users == 3
job = await service.start_switch(
created.profile_id,
expected_revision=created.revision,
plan_token=plan.plan_token,
requested_by="cli",
)
await service.run(job.job_id)
finished = service.get_job(job.job_id)
assert finished.state == "done", service.jobs.get(job.job_id)["verification_json"]
asyncio.run(scenario())
new = spaces.require_live()
target = client.get_collection(new.collection_names()["memory"])
got = target.get(include=["documents", "embeddings"])
documents = dict(zip(got["ids"], got["documents"], strict=True))
assert documents == {"a": "one", "b": "two", "c": "three", "z": "only in the stray"}
assert {len(vector) for vector in got["embeddings"]} == {8}
assert profiles.active_identity() == (created.profile_id, created.revision)
states = {space.collection_base: space.state for space in spaces.list_spaces()}
assert states[live.collection_base] == "retired"
assert states[stray_base] == "retired"
# ----------------------------------------------------------------------
# Nothing stood in: the SDK's EmbedService over HTTP, real Chroma, real Kuzu
# ----------------------------------------------------------------------
class _EmbeddingServer:
"""A local OpenAI-compatible `/embeddings` endpoint; counts what it was sent."""
def __init__(self, width: int) -> None:
import json
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
self.width = width
self.inputs: list[str] = []
self.dimensions: list[object] = []
server = self
class Handler(BaseHTTPRequestHandler):
def do_POST(self) -> None: # noqa: N802 - http.server's name
body = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
texts = body["input"] if isinstance(body["input"], list) else [body["input"]]
server.inputs.extend(texts)
server.dimensions.append(body.get("dimensions"))
payload = json.dumps(
{
"object": "list",
"model": body["model"],
"data": [
{
"object": "embedding",
"index": index,
"embedding": [(len(text) % 7 + 1) / 8.0] + [0.5] * (width - 1),
}
for index, text in enumerate(texts)
],
"usage": {"prompt_tokens": 1, "total_tokens": 1},
}
).encode()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(payload)))
self.end_headers()
self.wfile.write(payload)
def log_message(self, *_args: object) -> None:
return
self._server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
self.url = f"http://127.0.0.1:{self._server.server_port}/v1"
threading.Thread(target=self._server.serve_forever, daemon=True).start()
def close(self) -> None:
self._server.shutdown()
self._server.server_close()
def _home(tmp_path: Path, *, embedding_provider, embedding_url: str, api_key: str | None):
from pydantic import SecretStr
from one_memory.embedding_spaces import EmbeddingSpaceStore
from one_memory.model_profiles import (
ModelCapabilities,
ModelProfileInput,
ModelProfileStore,
ProviderKind,
)
from one_memory.secrets import InMemorySecretStore
database = tmp_path / "one-memory.sqlite3"
secrets = InMemorySecretStore()
profiles = ModelProfileStore(database)
spaces = EmbeddingSpaceStore(database)
created = profiles.create(
ModelProfileInput(
name="target",
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
base_url=embedding_url,
model="chat",
embedding_provider_kind=embedding_provider,
embedding_base_url=embedding_url,
embedding_model="stand-in",
embedding_dimensions=8,
api_key=SecretStr(api_key) if api_key else None,
),
secrets,
)
profiles.record_capabilities(
created.profile_id,
expected_revision=created.revision,
capabilities=ModelCapabilities(
endpoint_reachable=True,
model_available=True,
structured_output=True,
embedding_available=True,
),
secrets=secrets,
)
return database, secrets, profiles, spaces, created
def _service(database, data_dir, profiles, spaces, secrets, **options):
from one_memory.embedding_switch import EmbeddingSwitchService
service = EmbeddingSwitchService(
database_path=database,
data_dir=data_dir,
profiles=profiles,
spaces=spaces,
secrets=secrets,
backend=SdkReembedBackend(data_dir),
worker_enabled=False,
**options,
)
service.release_scan_hold()
return service
async def _switch(service, created) -> object:
plan = await service.plan(created.profile_id)
assert plan.needed and not plan.blockers, plan
job = await service.start_switch(
created.profile_id,
expected_revision=created.revision,
plan_token=plan.plan_token,
requested_by="cli",
)
await service.run(job.job_id)
finished = service.get_job(job.job_id)
assert finished.state == "done", (
finished.error_code,
service.jobs.get(job.job_id)["verification_json"],
service.jobs.get(job.job_id)["error_detail"],
)
return finished
def _seed_vectors(data_dir: Path, space) -> None:
import chromadb
vector = data_dir / "hy-memory" / "governed-v2" / "vector"
vector.mkdir(parents=True, exist_ok=True)
client = chromadb.PersistentClient(path=str(vector))
memory = client.get_or_create_collection(
space.collection_names()["memory"], metadata={"hnsw:space": "cosine"}
)
memory.add(
ids=["a", "b"],
embeddings=[[0.1] * 512, [0.2] * 512],
documents=["one", "two"],
metadatas=[{"user_id": "local"}, {"user_id": "hermes-a"}],
)
def test_a_switch_embeds_through_the_sdk_service_over_http(tmp_path: Path) -> None:
from one_memory.model_profiles import ProviderKind
server = _EmbeddingServer(8)
try:
database, secrets, profiles, spaces, created = _home(
tmp_path,
embedding_provider=ProviderKind.OPENAI_COMPATIBLE,
embedding_url=server.url,
api_key="sk-test",
)
_seed_vectors(tmp_path, spaces.bootstrap(None))
service = _service(database, tmp_path, profiles, spaces, secrets)
asyncio.run(_switch(service, created))
assert {"one", "two"} <= set(server.inputs)
assert set(server.dimensions) == {8}
finally:
server.close()
def test_an_ollama_embedding_needs_no_generation_key_to_re_embed(tmp_path: Path) -> None:
"""A switch only embeds: the generation model's key is not its business."""
from one_memory.embedding_backend import EmbeddingTarget
from one_memory.embedding_spaces import embedding_identity
from one_memory.model_profiles import ProviderKind
server = _EmbeddingServer(8)
try:
_database, secrets, profiles, spaces, created = _home(
tmp_path,
embedding_provider=ProviderKind.OLLAMA,
embedding_url=server.url,
api_key="sk-test",
)
profile = profiles.snapshot(created.profile_id, created.revision)
identity = embedding_identity(profile)
target = EmbeddingSpace(
space_id="t",
state="building",
model=identity.model,
dims=8,
name_dims=8,
collection_base=TARGET,
naming="v2",
created_at="2026-11-01T00:00:00+00:00",
)
backend = SdkReembedBackend(tmp_path)
# `api_key=None` is what `_embedding_target` resolves for Ollama.
service = backend._embedding_service(
EmbeddingTarget(profile=profile, space=target, api_key=None)
)
assert asyncio.run(service.embed_batch(["hello"], use_cache=False))[0][1] == 0.5
finally:
server.close()
async def _seed_graph(path: Path, dims: int) -> None:
from hy_memory.data.graph_store_kuzu import KuzuGraphStore
from hy_memory.models.memory import MemoryLayer, MemoryNode
config = hy_memory.MemoryConfig()
config.graph_store.provider = "kuzu"
config.graph_store.db_path = str(path)
config.vector_store.embedding_dims = dims
store = KuzuGraphStore(config)
await store.initialize()
assert store._available
node = MemoryNode(
user_id="local",
agent_id="a",
session_id="s",
layer=MemoryLayer.L6_SCHEMA,
content="plans ahead",
custom={"retrieval_text": "plans trips weeks ahead"},
)
node._graph_embedding = [0.1] * dims
await store.upsert_memory_node(node)
await store.close()
@pytest.mark.parametrize("mode", ["ultra", "pro"])
def test_a_real_graph_moves_to_the_target_width(tmp_path: Path, mode: str) -> None:
"""In any mode: a pro home's old graph must not be lost with the old space."""
from one_memory.hy_graph_admin import forget_graph_widths
from one_memory.model_profiles import ProviderKind
server = _EmbeddingServer(8)
try:
database, secrets, profiles, spaces, created = _home(
tmp_path,
embedding_provider=ProviderKind.OPENAI_COMPATIBLE,
embedding_url=server.url,
api_key="sk-test",
)
live = spaces.bootstrap(None)
_seed_vectors(tmp_path, live)
root = tmp_path / "hy-memory" / "governed-v2"
asyncio.run(_seed_graph(root / "graph", 512))
service = _service(
database, tmp_path, profiles, spaces, secrets, graph_enabled=lambda: mode == "ultra"
)
asyncio.run(_switch(service, created))
new = spaces.require_live()
assert new.graph_dir != "graph" and (root / new.graph_dir).exists()
assert (root / "graph").exists()
assert "plans trips weeks ahead" in server.inputs
forget_graph_widths()
finally:
server.close()
def test_verifying_the_graph_closes_what_reads_reopened(tmp_path: Path) -> None:
"""Reads are served during a switch, so a client may hold the graph again."""
from one_memory.model_profiles import ProviderKind
server = _EmbeddingServer(8)
quiesced: list[str] = []
async def quiesce():
quiesced.append("closed")
return lambda: quiesced.append("released")
try:
database, secrets, profiles, spaces, created = _home(
tmp_path,
embedding_provider=ProviderKind.OPENAI_COMPATIBLE,
embedding_url=server.url,
api_key="sk-test",
)
live = spaces.bootstrap(None)
_seed_vectors(tmp_path, live)
asyncio.run(_seed_graph(tmp_path / "hy-memory" / "governed-v2" / "graph", 512))
service = _service(
database,
tmp_path,
profiles,
spaces,
secrets,
graph_enabled=lambda: True,
quiesce_clients=quiesce,
)
asyncio.run(_switch(service, created))
# Once to copy the graph, once to verify it.
assert quiesced == ["closed", "released", "closed", "released"]
finally:
server.close()
def test_the_first_chroma_open_waits_for_a_client_under_construction(tmp_path: Path) -> None:
"""Two first opens on one directory race over Chroma's unlocked System cache."""
import threading
from one_memory.hy_sdk_runtime import first_store_open
_seed(tmp_path)
backend = SdkReembedBackend(tmp_path)
finished = threading.Event()
def scan() -> None:
asyncio.run(backend.scan())
finished.set()
with first_store_open(tmp_path):
thread = threading.Thread(target=scan)
thread.start()
assert not finished.wait(0.3)
thread.join(10)
assert finished.is_set()
+1
View File
@@ -48,6 +48,7 @@ def test_health_reports_the_single_core() -> None:
# No lifespan ran, so recovery has not begun.
"recovery": "pending",
"recovery_failed_steps": [],
"embedding_switch": None,
}
@@ -111,6 +111,11 @@ ALLOWED = {
"the offline migration opens Chroma",
opened_with="?mode=ro",
),
("sdk_vector_reattach.py", "stored_collection_width"): Allowed(
"Chroma's own chroma.sqlite3, read-only, for the width it fixed on a "
"collection: a probe on the live space's profile may only record that one",
opened_with="?mode=ro",
),
("imports/sqlite_access.py", "open_sqlite_readonly"): Allowed(
"a host app's own SQLite (Cursor state.vscdb / store.db): one direct "
"read-only open when there is no WAL, and one read-only open of the "
@@ -318,6 +318,12 @@ def run_with_sdk(tmp_path, monkeypatch, scenario):
async with app.router.lifespan_context(app):
app.state.sdk_client_handle = Handle()
monkeypatch.setattr(app.state.sdk_write_coordinator, "use_client", write_client)
# Edits and deletes wait for the startup scan of embedding spaces,
# as they would behind the App's loading screen.
for _ in range(500):
if app.state.embedding_switch.writes_open():
break
await asyncio.sleep(0.01)
async with client_for(app) as client:
await scenario(app, sdk, client)
@@ -684,6 +684,12 @@ def run_app(tmp_path, monkeypatch, scenario) -> None:
async with app.router.lifespan_context(app):
monkeypatch.setattr(app.state.sdk_write_coordinator, "use_client", write_client)
# Model writes wait for the startup scan of embedding spaces, as
# they would behind the App's loading screen.
for _ in range(500):
if app.state.embedding_switch.writes_open():
break
await asyncio.sleep(0.01)
async with client_for(app) as client:
await scenario(app, sdk, client)
@@ -12,10 +12,12 @@ import time
from concurrent.futures import ThreadPoolExecutor
from datetime import UTC, datetime, timedelta
from hashlib import sha256
from types import SimpleNamespace
import pytest
from one_memory.database_access import main_database
from one_memory.embedding_spaces import EmbeddingIdentity
from one_memory.memory_server.contracts import BatchIdentity, WriteRequest
from one_memory.memory_server.durable_queue import (
CANCELLED,
@@ -1226,6 +1228,10 @@ def _coordinator(monkeypatch, opened: list[_TrackedHandle]):
return value
monkeypatch.setattr(sdk_write_processor, "SdkClientHandle", handle)
# One live embedding space the stand-in profiles all fill.
identity = EmbeddingIdentity(model="stand-in")
monkeypatch.setattr(sdk_write_processor, "embedding_identity", lambda _profile: identity)
space = SimpleNamespace(space_id="space", identity=identity)
profiles = _SwitchableProfiles()
coordinator = sdk_write_processor.SdkWriteCoordinator(
profiles=profiles, # type: ignore[arg-type]
@@ -1233,6 +1239,7 @@ def _coordinator(monkeypatch, opened: list[_TrackedHandle]):
data_dir=None,
transport_factory=lambda _profile: None,
sedimentation_enabled=lambda: True,
spaces=SimpleNamespace(live=lambda: space), # type: ignore[arg-type]
)
return coordinator, profiles
@@ -91,7 +91,7 @@ def test_bundle_contains_management_sdk_config_and_verified_manifest(tmp_path):
assert manifest["bundle_id"] == "migration-1"
assert manifest["sdk_namespace"] == "governed-v2"
assert manifest["runtime_build"] == "build-13"
assert manifest["snapshot_sdk_version"] == "2.0.9"
assert manifest["snapshot_sdk_version"] == "2.0.11"
paths = {item["path"]: item for item in manifest["files"]}
assert {
"management/one-memory.sqlite3",
@@ -141,11 +141,19 @@ class _Recorder:
self.closed: list[str] = []
def __call__(
self, profile, *, api_key, data_dir, http_transport_factory=None, mode="pro", features=()
self,
profile,
space=None,
*,
api_key,
data_dir,
http_transport_factory=None,
mode=None,
features=(),
):
from one_memory.hy_sdk_runtime import build_sdk_config
config = build_sdk_config(profile, api_key=api_key, data_dir=data_dir)
config = build_sdk_config(profile, space, mode, api_key=api_key, data_dir=data_dir)
name = config.vector_store.collection_name
self.opened.append((name, config.vector_store.embedding_dims, api_key))
return SimpleNamespace(
@@ -155,10 +163,11 @@ class _Recorder:
@pytest.mark.anyio
async def test_reads_open_the_collection_the_active_profile_writes(
async def test_reads_open_the_live_space_the_active_profile_writes(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""#242: a 2560-dim profile wrote `memhub_<sha>_2560`, reads opened `_512`."""
from one_memory.embedding_spaces import EmbeddingSpaceStore
from one_memory.hy_sdk_runtime import build_sdk_config
from one_memory.model_profiles import ModelProfileStore
from one_memory.sdk_read_handle import ActiveProfileReadHandle
@@ -168,13 +177,14 @@ async def test_reads_open_the_collection_the_active_profile_writes(
monkeypatch.setattr("one_memory.hy_sdk_runtime.acquire_client", recorder)
profiles = ModelProfileStore(tmp_path / "one-memory.sqlite3")
secrets = InMemorySecretStore()
handle = ActiveProfileReadHandle(profiles=profiles, secrets=secrets, data_dir=tmp_path)
builtin = build_sdk_config(builtin_retrieval_snapshot(), api_key=None, data_dir=tmp_path)
assert (await handle.client()).collection == builtin.vector_store.collection_name
# A home whose custom profile was active when this build first started:
# the registry pins the collection that profile has been writing.
written = _activate(profiles, secrets, dimensions=2560)
spaces = EmbeddingSpaceStore(profiles.database_path)
spaces.bootstrap(written)
handle = ActiveProfileReadHandle(profiles=profiles, secrets=secrets, data_dir=tmp_path)
writes = build_sdk_config(written, api_key="sk-test", data_dir=tmp_path)
builtin = build_sdk_config(builtin_retrieval_snapshot(), api_key=None, data_dir=tmp_path)
async def read() -> str:
return str((await handle.client()).collection)
@@ -186,9 +196,11 @@ async def test_reads_open_the_collection_the_active_profile_writes(
@pytest.mark.anyio
async def test_a_profile_change_rebuilds_reads_without_closing_one_in_flight(
async def test_an_activation_that_skips_the_switch_never_moves_reads(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Only a switch's commit moves the space; reads stay where the data is."""
from one_memory.embedding_spaces import EmbeddingSpaceStore
from one_memory.model_profiles import ModelProfileStore
from one_memory.sdk_read_handle import ActiveProfileReadHandle
from one_memory.secrets import InMemorySecretStore
@@ -197,8 +209,40 @@ async def test_a_profile_change_rebuilds_reads_without_closing_one_in_flight(
monkeypatch.setattr("one_memory.hy_sdk_runtime.acquire_client", recorder)
profiles = ModelProfileStore(tmp_path / "one-memory.sqlite3")
secrets = InMemorySecretStore()
first = _activate(profiles, secrets, dimensions=1024)
live = EmbeddingSpaceStore(profiles.database_path).bootstrap(first)
handle = ActiveProfileReadHandle(profiles=profiles, secrets=secrets, data_dir=tmp_path)
_activate(profiles, secrets, dimensions=1024)
_activate(profiles, secrets, dimensions=2560, model="chat-2")
collection = (await asyncio.create_task(handle.client())).collection
assert collection == live.collection_base
await handle.close()
@pytest.mark.anyio
async def test_a_committed_switch_rebuilds_reads_without_closing_one_in_flight(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
from one_memory.embedding_spaces import (
EmbeddingSpaceStore,
derived_space,
insert_space_in,
set_state_in,
)
from one_memory.model_profiles import ModelProfileStore
from one_memory.sdk_read_handle import ActiveProfileReadHandle
from one_memory.secrets import InMemorySecretStore
recorder = _Recorder(tmp_path)
monkeypatch.setattr("one_memory.hy_sdk_runtime.acquire_client", recorder)
profiles = ModelProfileStore(tmp_path / "one-memory.sqlite3")
secrets = InMemorySecretStore()
spaces = EmbeddingSpaceStore(profiles.database_path)
old_space = spaces.bootstrap(_activate(profiles, secrets, dimensions=1024))
handle = ActiveProfileReadHandle(
profiles=profiles, secrets=secrets, data_dir=tmp_path, spaces=spaces
)
holding = asyncio.Event()
finish = asyncio.Event()
@@ -213,7 +257,15 @@ async def test_a_profile_change_rebuilds_reads_without_closing_one_in_flight(
await holding.wait()
[(old, _, _)] = recorder.opened
_activate(profiles, secrets, dimensions=2560, model="chat-2")
# What a switch's commit does: a new live space, the old one retired, and
# the target profile active -- one transaction.
target = _activate(profiles, secrets, dimensions=2560, model="chat-2")
with spaces.transaction() as connection:
set_state_in(connection, old_space.space_id, state="retired", expected=("live",))
insert_space_in(
connection,
derived_space(target).model_copy(update={"space_id": "new-space"}),
)
async def read() -> str:
return str((await handle.client()).collection)
+331
View File
@@ -0,0 +1,331 @@
"""The App's display language, and the SDK prompt language it implies.
The owner's rule (#243): every display language, zh-Hans and en included, is
passed as `MemoryConfig.language` (hy-memory 2.0.11); only an unset language
passes none, so the SDK keeps detecting zh/en exactly as before. A change rebuilds the
handles the way a mode or feature change does, and the language never touches
the vector collection identity.
"""
from __future__ import annotations
import asyncio
import dataclasses
import json
from contextlib import asynccontextmanager
from pathlib import Path
from types import SimpleNamespace
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr, ValidationError
from one_memory import hy_sdk_runtime, sdk_read_handle, sdk_write_processor
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.memory_server.config_state import ConfigDocument, ConfigStore
from one_memory.secrets import InMemorySecretStore
from one_memory.ui_language import UI_LANGUAGES, normalize_ui_language, sdk_language
TOKEN = "ui-language-test"
#: Every code is passed to the SDK, zh-Hans included.
PASSED = ("en", "zh-Hans", "zh-Hant", "ja", "es", "de", "pt", "fr", "ko", "ru", "id")
@pytest.fixture(autouse=True)
def _clean_sdk_env(monkeypatch):
"""The SDK's own `MEMORY_LANGUAGE` feeds its default; tests start without it."""
monkeypatch.delenv("MEMORY_LANGUAGE", raising=False)
def _config(tmp_path: Path, language: str | None):
return hy_sdk_runtime.build_sdk_config(
hy_sdk_runtime.builtin_retrieval_snapshot(),
api_key=None,
data_dir=tmp_path,
language=language,
)
# --- the rule -------------------------------------------------------------------
def test_eleven_codes_and_the_legacy_chinese_code() -> None:
assert PASSED == UI_LANGUAGES
assert len(UI_LANGUAGES) == 11
# Earlier App builds stored zh-CN; it is zh-Hans now.
assert normalize_ui_language("zh-CN") == "zh-Hans"
assert normalize_ui_language("system") is None
assert normalize_ui_language("xx") is None
assert normalize_ui_language(None) is None
def test_only_an_unset_language_leaves_the_sdk_to_detect() -> None:
assert sdk_language(None) is None
assert sdk_language("system") is None
assert sdk_language("xx") is None
# zh-Hans is passed like every other code; the legacy zh-CN maps to it.
assert sdk_language("zh-Hans") == "zh-Hans"
assert sdk_language("zh-CN") == "zh-Hans"
for code in PASSED:
assert sdk_language(code) == code
def test_no_language_builds_the_config_of_before(tmp_path) -> None:
assert _config(tmp_path, None).language is None
assert _config(tmp_path, sdk_language("zh-Hans")).language == "zh-Hans"
@pytest.mark.parametrize("code", PASSED)
def test_a_passed_code_reaches_the_memory_config(tmp_path, code) -> None:
config = _config(tmp_path, sdk_language(code))
assert config.language == code
def test_the_language_never_changes_the_collection_identity(tmp_path) -> None:
def identity(config):
return (
dataclasses.asdict(config.vector_store),
config.embedder.model,
config.embedder.embedding_dims,
config.graph_store.db_path,
config.cache.db_path,
)
baseline = identity(_config(tmp_path, None))
for code in PASSED:
assert identity(_config(tmp_path, code)) == baseline
def test_acquire_client_passes_the_language_only_when_there_is_one(tmp_path, monkeypatch) -> None:
calls: list[dict] = []
monkeypatch.setattr(
hy_sdk_runtime, "_acquire_client", lambda profile, **kwargs: calls.append(kwargs)
)
profile = hy_sdk_runtime.builtin_retrieval_snapshot()
hy_sdk_runtime.acquire_client(profile, api_key=None, data_dir=tmp_path, mode="pro")
hy_sdk_runtime.acquire_client(
profile, api_key=None, data_dir=tmp_path, mode="pro", language="ja"
)
assert "language" not in calls[0]
assert calls[1]["language"] == "ja"
# --- the stored setting -----------------------------------------------------------
def test_the_setting_persists_and_an_untouched_install_writes_none(tmp_path) -> None:
store = ConfigStore(tmp_path)
store.set_sync("cursor", True)
assert "ui_language" not in json.loads((tmp_path / "config.json").read_text())
assert store.ui_language() is None
store.set_ui_language("ja")
assert json.loads((tmp_path / "config.json").read_text())["ui_language"] == "ja"
assert ConfigStore(tmp_path).ui_language() == "ja"
store.set_ui_language(None)
assert ConfigStore(tmp_path).ui_language() is None
with pytest.raises(ValueError, match="invalid_ui_language"):
store.set_ui_language("zh-CN") # type: ignore[arg-type]
with pytest.raises(ValidationError):
ConfigDocument.model_validate({"ui_language": "klingon"})
# --- rebuilding handles -------------------------------------------------------------
class _FakeHandle:
def __init__(self, language: str | None) -> None:
self.language = language
self.closed = False
async def client(self):
return self
async def close(self):
self.closed = True
def test_the_read_handle_rebuilds_its_client_when_the_language_changes(
monkeypatch, tmp_path
) -> None:
language: list[str | None] = [None]
made: list[_FakeHandle] = []
def handle_for(profile, *, api_key, data_dir, mode=None, features=(), language=None):
made.append(_FakeHandle(language))
return made[-1]
monkeypatch.setattr(sdk_read_handle, "SdkClientHandle", handle_for)
class _NoProfile:
def active_identity(self):
return None
reads = sdk_read_handle.ActiveProfileReadHandle(
profiles=_NoProfile(), # type: ignore[arg-type]
secrets=None, # type: ignore[arg-type]
data_dir=tmp_path,
memory_mode=lambda: "pro",
language=lambda: language[0],
)
async def scenario():
# A use lasts as long as the task that asked, so each read is a task.
old = await asyncio.create_task(reads.client())
assert old.language is None
assert await asyncio.create_task(reads.client()) is old
language[0] = "ja"
fresh = await asyncio.create_task(reads.client())
assert fresh is not old and fresh.language == "ja"
for _ in range(10):
await asyncio.sleep(0)
assert old.closed and not fresh.closed
await reads.close()
asyncio.run(scenario())
class _Profiles:
def active_identity(self):
return ("profile-a", 1)
def snapshot(self, profile_id, revision):
return SimpleNamespace(profile_id=profile_id, has_api_key=False)
def resolve_snapshot_secret(self, profile_id, revision, secrets):
return None
def test_a_language_change_keeps_a_busy_write_handle_until_its_write_ends(monkeypatch) -> None:
language: list[str | None] = [None]
opened: list[_FakeHandle] = []
def handle(profile, **kwargs):
opened.append(_FakeHandle(kwargs.get("language")))
return opened[-1]
monkeypatch.setattr(sdk_write_processor, "SdkClientHandle", handle)
coordinator = sdk_write_processor.SdkWriteCoordinator(
profiles=_Profiles(), # type: ignore[arg-type]
secrets=None, # type: ignore[arg-type]
data_dir=None,
transport_factory=lambda _profile: None,
sedimentation_enabled=lambda: True,
memory_mode=lambda: "pro",
sdk_language=lambda: language[0],
)
async def scenario():
async with coordinator.use_client() as client:
assert client.language is None
language[0] = "de"
await coordinator.retire_stale_handles()
# In flight: it finishes in the language it started with (#218).
assert not opened[0].closed
assert opened[0].closed
async with coordinator.use_client() as client:
assert client.language == "de"
language[0] = None
await coordinator.retire_stale_handles()
assert opened[1].closed
await coordinator.close()
asyncio.run(scenario())
# --- the HTTP surface -----------------------------------------------------------------
def run_app(tmp_path, monkeypatch, scenario) -> None:
async def main() -> None:
settings = Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr(TOKEN),
imports_enabled=False,
automatic_memory_processing=False,
live_sync_enabled=False,
)
app = create_app(settings, secret_store=InMemorySecretStore())
@asynccontextmanager
async def write_client():
yield SimpleNamespace()
async with app.router.lifespan_context(app):
monkeypatch.setattr(app.state.sdk_write_coordinator, "use_client", write_client)
async with AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": f"Bearer {TOKEN}"},
) as client:
await scenario(app, client)
asyncio.run(main())
@pytest.mark.fresh_sdk
def test_ui_language_api_get_put_and_status(tmp_path, monkeypatch) -> None:
async def scenario(app, client):
reads = app.state.sdk_client_handle
writes = app.state.sdk_write_coordinator
retired: list[str | None] = []
retire = writes.retire_stale_handles
async def counting():
retired.append(writes.sdk_language())
await retire()
monkeypatch.setattr(writes, "retire_stale_handles", counting)
current = (await client.get("/api/v1/settings/ui-language")).json()
assert current["ui_language"] is None and current["sdk_language"] == "auto"
assert current["languages"] == list(UI_LANGUAGES)
assert reads.current_language() is None and writes.sdk_language() is None
# zh-Hans is passed explicitly: Chinese prompts, handles rebuilt (#243).
body = (
await client.put("/api/v1/settings/ui-language", json={"ui_language": "zh-Hans"})
).json()
assert body == {
**current,
"ui_language": "zh-Hans",
"sdk_language": "zh-Hans",
"rebuilt": True,
}
assert reads.current_language() == "zh-Hans" and writes.sdk_language() == "zh-Hans"
assert retired == ["zh-Hans"]
processing = (await client.get("/api/v1/memory-processing")).json()
assert processing["pipeline"]["sdk_language"] == "zh-Hans"
# The same value again changes nothing.
body = (
await client.put("/api/v1/settings/ui-language", json={"ui_language": "zh-Hans"})
).json()
assert body["rebuilt"] is False and retired == ["zh-Hans"]
body = (await client.put("/api/v1/settings/ui-language", json={"ui_language": "ja"})).json()
assert body["sdk_language"] == "ja" and body["rebuilt"] is True
assert reads.current_language() == "ja" and writes.sdk_language() == "ja"
assert retired == ["zh-Hans", "ja"]
# An explicit English choice forces English prompts.
body = (await client.put("/api/v1/settings/ui-language", json={"ui_language": "en"})).json()
assert body["sdk_language"] == "en" and body["rebuilt"] is True
processing = (await client.get("/api/v1/memory-processing")).json()
assert processing["pipeline"]["ui_language"] == "en"
assert processing["pipeline"]["sdk_language"] == "en"
# Unset (CLI `settings language system`): auto again.
body = (await client.put("/api/v1/settings/ui-language", json={"ui_language": None})).json()
assert body["sdk_language"] == "auto" and body["rebuilt"] is True
assert reads.current_language() is None
saved = json.loads((tmp_path / "data" / "config.json").read_text())
assert saved["ui_language"] is None
for bad in ({"ui_language": "zh-CN"}, {"ui_language": "xx"}, {"language": "ja"}, {}):
response = await client.put("/api/v1/settings/ui-language", json=bad)
assert response.status_code == 422, bad
run_app(tmp_path, monkeypatch, scenario)
+5 -5
View File
@@ -991,7 +991,7 @@ wheels = [
[[package]]
name = "hy-memory"
version = "2.0.9"
version = "2.0.11"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "chromadb" },
@@ -1008,9 +1008,9 @@ dependencies = [
{ name = "scikit-learn" },
{ name = "tiktoken" },
]
sdist = { url = "https://files.pythonhosted.org/packages/48/28/e01fd9a92cd090c8236df0dd06dae049c5de66345e5bd9653497ed56125e/hy_memory-2.0.9.tar.gz", hash = "sha256:5f55ea92044ced19a00b4e56c565215adedd3439a1bdb9cfe9ffe42387c8f1a1", size = 858323, upload-time = "2026-11-01T00:29:49.914Z" }
sdist = { url = "https://files.pythonhosted.org/packages/01/93/aec7d243259f6a315f38e8ebc21a9eea153a48fc13648fda149be7672788/hy_memory-2.0.11.tar.gz", hash = "sha256:96d6bd606d61052f127cd274aa6f8de8e7151d6250f1e89e4ef5b90e18ec7cb0", size = 1245738, upload-time = "2026-11-01T14:15:02.474Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/d5/44/4368555a408a58daf902c81b62184a3c99f12abd45d4f1144bf79f171a6f/hy_memory-2.0.9-py3-none-any.whl", hash = "sha256:95dbea232d2041f9b0ea3d08dd751603c2a5c0e4422b97e74f664e6ca913549e", size = 923250, upload-time = "2026-11-01T00:29:47.814Z" },
{ url = "https://files.pythonhosted.org/packages/1a/85/1f5e6a13507a0cfcdce26f67c590df00d327a0f7d5c5b5483c3bd0fa2f85/hy_memory-2.0.11-py3-none-any.whl", hash = "sha256:c0173444a83c1197a49acaccb123f8286cea205b4838c1e83d106974f5bad259", size = 1315871, upload-time = "2026-11-01T14:14:58.764Z" },
]
[[package]]
@@ -1801,7 +1801,7 @@ wheels = [
[[package]]
name = "one-memory"
version = "1.0.5"
version = "1.0.6"
source = { editable = "." }
dependencies = [
{ name = "aiomysql" },
@@ -1837,7 +1837,7 @@ requires-dist = [
{ name = "fastapi", specifier = ">=0.141.1" },
{ name = "filelock", specifier = ">=3.32.4" },
{ name = "httpx", specifier = ">=0.28.1" },
{ name = "hy-memory", specifier = "==2.0.9" },
{ name = "hy-memory", specifier = "==2.0.11" },
{ name = "keyring", specifier = ">=25.7.0" },
{ name = "numpy", specifier = "==2.5.2" },
{ name = "onnxruntime", specifier = "==1.23.2" },
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/desktop",
"version": "1.0.5",
"version": "1.0.6",
"private": true,
"main": "dist/main.js",
"scripts": {
@@ -30,7 +30,7 @@ it("rebuilds translated labels from committed language updates without exporting
expect(buildMenu).toHaveBeenLastCalledWith(
expect.arrayContaining([expect.objectContaining({ label: "Ayuda" })]),
);
snapshot.locale = "zh-CN";
snapshot.locale = "zh-Hans";
change?.();
expect(read).toHaveBeenCalledTimes(2);
expect(buildMenu).toHaveBeenCalledTimes(2);
+2 -2
View File
@@ -16,14 +16,14 @@ export function installApplicationMenu(options: {
} catch {
options.reportError(
translate(
options.language?.read().locale ?? "zh-CN",
options.language?.read().locale ?? "zh-Hans",
"nativeExportError",
),
);
}
};
const refresh = () => {
const locale = options.language?.read().locale ?? "zh-CN";
const locale = options.language?.read().locale ?? "zh-Hans";
options.setMenu(
options.buildMenu([
{
@@ -28,8 +28,8 @@ function fixture(
};
let changed: (() => void) | undefined;
const languageSnapshot: LanguageSnapshot = {
preference: "zh-CN",
locale: "zh-CN",
preference: "zh-Hans",
locale: "zh-Hans",
revision: 1,
};
const app = { isPackaged: packaged, quit: vi.fn() };
@@ -121,7 +121,7 @@ describe("native macOS background tray", () => {
expect(f.setLoginPreference).not.toHaveBeenCalled();
f.click(0);
expect(f.open).toHaveBeenCalledOnce();
f.changeLanguage("zh-CN");
f.changeLanguage("zh-Hans");
await flush();
expect(f.menu()[0]?.label).toBe("打开 MemHub");
f.click(3);
+2 -2
View File
@@ -44,7 +44,7 @@ export function installBackgroundTray(options: TrayOptions): () => void {
let generation = 0;
const refresh = async () => {
if (disposed) return;
const locale = options.language?.read().locale ?? "zh-CN";
const locale = options.language?.read().locale ?? "zh-Hans";
const t = (key: MessageKey) =>
translate(locale, key, { product: productName });
const current = ++generation;
@@ -82,7 +82,7 @@ export function installBackgroundTray(options: TrayOptions): () => void {
.catch(() =>
options.reportError(
translate(
options.language?.read().locale ?? "zh-CN",
options.language?.read().locale ?? "zh-Hans",
"nativeLoginError",
),
),
@@ -15,9 +15,13 @@ it("bakes fallback release identity when the packaging pipeline omits it", async
expect(config).toMatch(
/__ONE_MEMORY_BUILD_NUMBER__: JSON\.stringify\(\s*String\(releaseBuildNumber\)/,
);
expect(config).toMatch(
/process\.env\.ONE_MEMORY_COMMIT\?\.trim\(\) \|\| readSourceCommit\(\)/,
expect(config).toContain(
"const releaseCommit = resolveReleaseCommit(process.env.ONE_MEMORY_COMMIT)",
);
expect(config).toMatch(/if \(!trimmed\) return readSourceCommit\(\);/);
// A supplied commit must be the full 40-character form the runtime accepts.
expect(config).toContain("/^[0-9a-f]{40}$/i.test(trimmed)");
expect(config).toContain("return trimmed.toLowerCase();");
expect(config).toMatch(
/execFileSync\("git", \["rev-parse", "HEAD"\],[\s\S]*?encoding: "utf8"/,
);
@@ -0,0 +1,83 @@
import type { GuiLocale, LanguageSnapshot } from "@one-memory/i18n";
import { describe, expect, it, vi } from "vitest";
import { coreUiLanguage, installCoreLanguageSync } from "./core-language-sync";
function source(initial: LanguageSnapshot) {
let snapshot = initial;
const listeners = new Set<(value: LanguageSnapshot) => void>();
return {
read: () => snapshot,
subscribe(listener: (value: LanguageSnapshot) => void) {
listeners.add(listener);
return () => listeners.delete(listener);
},
change(next: LanguageSnapshot) {
snapshot = next;
for (const listener of listeners) listener(next);
},
};
}
const settle = () => new Promise((resolve) => setTimeout(resolve, 0));
describe("Core language sync", () => {
it("tells Core the language the GUI shows, system included", () => {
const shown = (
preference: LanguageSnapshot["preference"],
locale: GuiLocale,
) => coreUiLanguage({ preference, locale, revision: 0 });
expect(shown("system", "zh-Hans")).toBe("zh-Hans");
expect(shown("system", "en")).toBe("en");
expect(shown("zh-Hans", "zh-Hans")).toBe("zh-Hans");
expect(shown("en", "en")).toBe("en");
expect(shown("ja", "ja")).toBe("ja");
});
it("sends the stored choice once at launch, then every change", async () => {
const language = source({ preference: "es", locale: "es", revision: 0 });
const send = vi.fn(async () => {});
const dispose = installCoreLanguageSync({ language, send });
await settle();
expect(send.mock.calls).toEqual([["es"]]);
language.change({ preference: "ja", locale: "ja", revision: 1 });
await settle();
language.change({ preference: "system", locale: "en", revision: 2 });
await settle();
expect(send.mock.calls).toEqual([["es"], ["ja"], ["en"]]);
dispose();
language.change({ preference: "de", locale: "de", revision: 3 });
await settle();
expect(send).toHaveBeenCalledTimes(3);
});
it("an untouched install sends the system language the GUI shows", async () => {
const language = source({
preference: "system",
locale: "zh-Hans",
revision: 0,
});
const send = vi.fn(async () => {});
installCoreLanguageSync({ language, send });
await settle();
expect(send.mock.calls).toEqual([["zh-Hans"]]);
// Choosing the language already shown changes nothing Core sees.
language.change({ preference: "zh-Hans", locale: "zh-Hans", revision: 1 });
await settle();
expect(send).toHaveBeenCalledTimes(1);
});
it("reports a failed send and retries it with the next change", async () => {
const language = source({ preference: "fr", locale: "fr", revision: 0 });
const send = vi
.fn<(value: string) => Promise<void>>()
.mockRejectedValueOnce(new Error("core_unavailable"))
.mockResolvedValue(undefined);
const reportError = vi.fn();
installCoreLanguageSync({ language, send, reportError });
await settle();
expect(reportError).toHaveBeenCalledTimes(1);
language.change({ preference: "fr", locale: "fr", revision: 1 });
await settle();
expect(send.mock.calls).toEqual([["fr"], ["fr"]]);
});
});
@@ -0,0 +1,50 @@
import type { GuiLocale, LanguageSnapshot } from "@one-memory/i18n";
import type { NativeLanguageSource } from "./native-language";
/**
* What Core is told: the language the GUI shows. "Follow the system" sends
* the language it resolved to, so the memories match the interface a person
* sees; the App never leaves the SDK to detect zh/en (#243).
*/
export function coreUiLanguage(snapshot: LanguageSnapshot): GuiLocale {
return snapshot.locale;
}
/**
* Keep Core's `ui_language` setting on the display language the GUI shows.
*
* Core builds every SDK client with it and passes it, zh-Hans included, as
* the SDK prompt language. Sent once when installed, so a choice made before
* this existed, or the system language followed at this launch, reaches
* Core; then on every committed change. A value Core already has is not sent
* again. Best effort: a failed send is reported and retried with the next
* change or launch, and never blocks the GUI.
*/
export function installCoreLanguageSync(options: {
language: NativeLanguageSource;
send(uiLanguage: GuiLocale): Promise<void>;
reportError?(error: unknown): void;
}): () => void {
let closed = false;
let sent: GuiLocale | undefined;
let queue: Promise<void> = Promise.resolve();
const push = () => {
queue = queue.then(async () => {
if (closed) return;
const value = coreUiLanguage(options.language.read());
if (value === sent) return;
try {
await options.send(value);
sent = value;
} catch (error) {
options.reportError?.(error);
}
});
};
const unsubscribe = options.language.subscribe(push);
push();
return () => {
closed = true;
unsubscribe();
};
}
@@ -23,8 +23,8 @@ const fake = vi.hoisted(() => ({
unload: undefined as unknown as EventTarget,
packaged: true,
userData: "",
preferred: vi.fn(() => ["es-MX", "zh-CN"]),
locale: vi.fn(() => "zh-CN"),
preferred: vi.fn(() => ["es-MX", "zh-Hans"]),
locale: vi.fn(() => "zh-Hans"),
menu: vi.fn(),
loadFailure: false,
reduceMotion: false as unknown,
@@ -301,10 +301,10 @@ describe("GUI language main IPC", () => {
await expect(read(eventFor(window))).rejects.toThrow();
await expect(write(eventFor(window), "es")).rejects.toThrow();
window.frame.url = "one-memory://app/index.html";
await write(eventFor(window), "zh-CN");
await write(eventFor(window), "zh-Hans");
expect(window.frame.send).toHaveBeenCalledWith(
"one-memory:language-preference-changed",
{ preference: "zh-CN", locale: "zh-CN", revision: 1 },
{ preference: "zh-Hans", locale: "zh-Hans", revision: 1 },
);
expect(foreign.frame.send).not.toHaveBeenCalled();
window.frame.send.mockImplementation(() => {
@@ -539,7 +539,7 @@ describe("language preference preload", () => {
"es",
);
fake.invoke.mockClear();
await expect(api.setLanguagePreference("fr")).rejects.toThrow();
await expect(api.setLanguagePreference("it")).rejects.toThrow();
expect(fake.invoke).not.toHaveBeenCalled();
const listener = vi.fn();
const unsubscribe = api.onLanguagePreferenceChanged(listener);
@@ -91,7 +91,7 @@ describe("early desktop startup errors", () => {
new Error("desktop_startup_failed"),
);
expect(showError).toHaveBeenCalledExactlyOnceWith(
translate(locale ?? "zh-CN", "nativeErrorTitle"),
translate(locale ?? "zh-Hans", "nativeErrorTitle"),
stableRuntimeUserMessage(new Error("desktop_startup_failed")),
);
expect(exit).toHaveBeenCalledExactlyOnceWith(1);
@@ -59,9 +59,9 @@ export function installEarlyStartupErrorHandler(
// The dialog is the only part of this a person ever reads, so it gets the
// explanation, not the code. The code stays in the diagnostic event,
// where support can still match it against a known failure.
let locale: GuiLocale = "zh-CN";
let locale: GuiLocale = "zh-Hans";
try {
locale = options.readLocale?.() ?? "zh-CN";
locale = options.readLocale?.() ?? "zh-Hans";
} catch {
// Preferences may not exist yet; do not read files or guess the locale.
}
@@ -526,7 +526,22 @@ describe("hourly isolated desktop launch", () => {
expect(resolvePackagedBuildNumber("-1")).toBeUndefined();
expect(resolvePackagedBuildNumber(undefined, "3907")).toBe(3907);
expect(resolvePackagedBuildNumber("3908", "3907")).toBe(3908);
expect(resolvePackagedCommit(undefined, "388fd0e9")).toBe("388fd0e9");
expect(resolvePackagedCommit("not-a-sha", "388fd0e9")).toBe("388fd0e9");
});
it("resolves only a full 40-character commit and lowercases it", () => {
// Every downstream check (packaged-runtime install, local install, release
// download, release scripts) requires 40 lowercase hex characters.
const embedded = "388fd0e9b3c41a2d5e6f7a8b9c0d1e2f3a4b5c6d";
expect(resolvePackagedCommit(undefined, embedded)).toBe(embedded);
expect(resolvePackagedCommit(embedded.toUpperCase())).toBe(embedded);
expect(
resolvePackagedCommit(` ${embedded.toUpperCase()} `, undefined),
).toBe(embedded);
expect(resolvePackagedCommit("388fd0e9", embedded)).toBe(embedded);
expect(resolvePackagedCommit("388fd0e9")).toBeUndefined();
expect(resolvePackagedCommit("not-a-sha", embedded)).toBe(embedded);
expect(resolvePackagedCommit(`${embedded}0`, embedded)).toBe(embedded);
expect(resolvePackagedCommit("g".repeat(40), embedded)).toBe(embedded);
expect(resolvePackagedCommit(undefined, "unknown")).toBeUndefined();
});
});
+7 -1
View File
@@ -143,6 +143,12 @@ export function resolvePackagedBuildNumber(
return undefined;
}
/**
* The packaged-runtime install, local install, release download and release
* scripts all require a full 40-character lowercase commit. Normalise here so
* that one format crosses this boundary: an uppercase full SHA is lowercased,
* and a short or malformed value is skipped so it cannot shadow the next source.
*/
export function resolvePackagedCommit(
environmentValue: unknown,
embeddedValue?: unknown,
@@ -150,7 +156,7 @@ export function resolvePackagedCommit(
for (const value of [environmentValue, embeddedValue]) {
if (typeof value !== "string") continue;
const trimmed = value.trim();
if (/^[0-9a-f]{7,40}$/i.test(trimmed)) return trimmed;
if (/^[0-9a-f]{40}$/i.test(trimmed)) return trimmed.toLowerCase();
}
return undefined;
}
@@ -33,7 +33,7 @@ describe("language preference IPC", () => {
if (!read || !write) throw Error("Missing language handlers");
await expect(read("foreign")).rejects.toThrow("untrusted");
await expect(write("iframe", "es")).rejects.toThrow("untrusted");
await expect(write("trusted", "fr")).rejects.toThrow();
await expect(write("trusted", "it")).rejects.toThrow();
await expect(write("trusted", "es", "extra")).rejects.toThrow();
expect(save).not.toHaveBeenCalled();
expect(await read("trusted")).toEqual({
@@ -29,23 +29,42 @@ describe("GUI language preferences", () => {
revision: 0,
});
expect(await readdir(root)).toEqual([]);
expect(await store.set("zh-CN")).toEqual({
preference: "zh-CN",
locale: "zh-CN",
expect(await store.set("zh-Hans")).toEqual({
preference: "zh-Hans",
locale: "zh-Hans",
revision: 1,
});
expect(
JSON.parse(await readFile(path.join(root, "gui-language.json"), "utf8")),
).toEqual({ preference: "zh-CN" });
).toEqual({ preference: "zh-Hans" });
expect((await LanguagePreferences.open(root, "es")).read().preference).toBe(
"zh-CN",
"zh-Hans",
);
expect(
(await LanguagePreferences.open(await profile(), "es")).read().preference,
).toBe("system");
expect(await readdir(root)).toEqual(["gui-language.json"]);
});
it.each(["{broken", '{"preference":"fr"}', "null", "[]"])(
it.each([
["zh-CN", "zh-Hans", "zh-Hans"],
["es", "es", "es"],
["en", "en", "en"],
["ja", "ja", "ja"],
["system", "system", "en"],
])(
"keeps an existing user's stored %s choice as %s",
async (stored, preference, locale) => {
const root = await profile();
const file = path.join(root, "gui-language.json");
const content = `${JSON.stringify({ preference: stored })}\n`;
await writeFile(file, content);
const store = await LanguagePreferences.open(root, "en-US");
expect(store.read()).toEqual({ preference, locale, revision: 0 });
// Read as today's code; the file is rewritten only by the next change.
expect(await readFile(file, "utf8")).toBe(content);
},
);
it.each(["{broken", '{"preference":"it"}', "null", "[]"])(
"does not overwrite invalid input %s while loading",
async (content) => {
const root = await profile();
@@ -69,7 +88,7 @@ describe("GUI language preferences", () => {
});
const listener = vi.fn();
store.subscribe(listener);
await expect(store.set("zh-CN")).rejects.toThrow(
await expect(store.set("zh-Hans")).rejects.toThrow(
"language_preference_save_failed",
);
expect(store.read()).toEqual({
@@ -78,7 +97,7 @@ describe("GUI language preferences", () => {
revision: 0,
});
expect(listener).not.toHaveBeenCalled();
await store.set("zh-CN");
await store.set("zh-Hans");
expect(store.read().revision).toBe(1);
expect(listener).toHaveBeenCalledTimes(1);
});
@@ -97,7 +116,7 @@ describe("GUI language preferences", () => {
});
const listener = vi.fn();
const unsubscribe = store.subscribe(listener);
const first = store.set("zh-CN");
const first = store.set("zh-Hans");
const second = store.set("es");
await vi.waitFor(() => expect(save).toHaveBeenCalledTimes(1));
expect(store.read().revision).toBe(0);
@@ -5,6 +5,7 @@ import {
isLanguagePreference,
type LanguagePreference,
type LanguageSnapshot,
normalizeLanguagePreference,
resolveGuiLocale,
} from "@one-memory/i18n";
@@ -58,14 +59,13 @@ export class LanguagePreferences {
let preference: LanguagePreference = "system";
try {
const data: unknown = JSON.parse(await io.read(file));
if (
data &&
typeof data === "object" &&
"preference" in data &&
isLanguagePreference(data.preference)
) {
preference = data.preference;
}
// An earlier build's code (zh-CN) reads as today's (zh-Hans), so a
// person keeps the language they chose; the next change writes it.
const stored =
data && typeof data === "object" && "preference" in data
? normalizeLanguagePreference(data.preference)
: null;
if (stored !== null) preference = stored;
} catch {
// Missing, unreadable or corrupt preferences never block startup or get rewritten.
}
+51 -8
View File
@@ -59,6 +59,7 @@ import { installApplicationMenu } from "./application-menu";
import { installBackgroundHealthIpc } from "./background-health-ipc";
import { installBackgroundTray } from "./background-tray";
import { installConnectionSetupIpc } from "./connection-setup-ipc";
import { installCoreLanguageSync } from "./core-language-sync";
import {
CoreSupervisor,
ensureRuntimeProcess,
@@ -151,6 +152,7 @@ let connectionSetup: ReturnType<typeof installConnectionSetupIpc> | null = null;
let disposeBackgroundHealth: (() => void) | null = null;
let stableRuntimeRepair: Promise<void> | null = null;
let disposeDiagnostics: (() => void) | null = null;
let disposeCoreLanguageSync: (() => void) | null = null;
let desktopDiagnostics: DiagnosticLogger | null = null;
let exportDiagnosticsFromMenu: (() => Promise<void>) | null = null;
const desktopStartupStartedAt = Date.now();
@@ -165,7 +167,7 @@ installEarlyStartupErrorHandler({
process.env.ONE_MEMORY_DATA_DIR?.trim() ||
path.join(app.getPath("home"), ".one-memory"),
stderr: process.stderr,
readLocale: () => languagePreferences?.read().locale ?? "zh-CN",
readLocale: () => languagePreferences?.read().locale ?? "zh-Hans",
showError: (title, message) => dialog.showErrorBox(title, message),
exit: (code) => app.exit(code),
});
@@ -296,9 +298,9 @@ function installPackagedWebProtocol(): void {
installAppProtocol(path.join(process.resourcesPath, "web"), {
// Preferences are open by now: beforeNativeUi runs before this window.
startupScreen: () => {
let locale: GuiLocale = "zh-CN";
let locale: GuiLocale = "zh-Hans";
try {
locale = languagePreferences?.read().locale ?? "zh-CN";
locale = languagePreferences?.read().locale ?? "zh-Hans";
} catch {
// An unreadable preference keeps the Chinese placeholder.
}
@@ -1404,16 +1406,16 @@ async function initialize(isQuitting: () => boolean): Promise<void> {
void dialog.showMessageBox({
type: "warning",
message: translate(
languagePreferences?.read().locale ?? "zh-CN",
languagePreferences?.read().locale ?? "zh-Hans",
"nativeMoveTitle",
),
detail: translate(
languagePreferences?.read().locale ?? "zh-CN",
languagePreferences?.read().locale ?? "zh-Hans",
"nativeMoveDetail",
),
buttons: [
translate(
languagePreferences?.read().locale ?? "zh-CN",
languagePreferences?.read().locale ?? "zh-Hans",
"nativeAcknowledge",
),
],
@@ -1518,6 +1520,43 @@ async function initialize(isQuitting: () => boolean): Promise<void> {
};
},
});
if (languagePreferences) {
// Core builds the SDK clients with the display language (`ui_language`).
disposeCoreLanguageSync = installCoreLanguageSync({
language: languagePreferences,
send: async (uiLanguage) => {
const core = coreSupervisor;
if (!core) throw new Error("core_supervisor_unavailable");
const publishedPort = await readPublishedCorePort(dataDir).catch(
() => null,
);
const response = await fetch(
`http://127.0.0.1:${publishedPort ?? core.port}/api/v1/settings/ui-language`,
{
method: "PUT",
headers: {
Authorization: `Bearer ${core.apiToken}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ ui_language: uiLanguage }),
signal: AbortSignal.timeout(5_000),
},
);
if (!response.ok)
throw new Error(`ui_language_sync_http_${response.status}`);
},
reportError: (error) =>
void diagnostics
.log({
level: "warn",
component: "desktop.language",
event: "ui_language.sync_failed",
outcome: "failed",
...diagnosticError(error),
})
.catch(() => {}),
});
}
await diagnostics.log({
level: "info",
component: "desktop.lifecycle",
@@ -1657,6 +1696,8 @@ if (desktopVersion !== null) {
...(coreSupervisor ? { instance_id: coreSupervisor.instanceId } : {}),
});
// An admitted native effect must settle and journal before Runtime detach.
disposeCoreLanguageSync?.();
disposeCoreLanguageSync = null;
disposeLanguagePreferences?.();
disposeLanguagePreferences = null;
await languagePreferences?.drain();
@@ -1708,6 +1749,8 @@ if (desktopVersion !== null) {
});
},
dispose: () => {
disposeCoreLanguageSync?.();
disposeCoreLanguageSync = null;
disposeLanguagePreferences?.();
disposeLanguagePreferences = null;
disposeApplicationMenu?.();
@@ -1738,7 +1781,7 @@ if (desktopVersion !== null) {
showMigrationFailureDialog({
error,
dataDir: runtimeDataDir,
locale: languagePreferences?.read().locale ?? "zh-CN",
locale: languagePreferences?.read().locale ?? "zh-Hans",
dependencies: {
showMessageBoxSync: (options) => dialog.showMessageBoxSync(options),
showItemInFolder: (target) => shell.showItemInFolder(target),
@@ -1766,7 +1809,7 @@ if (desktopVersion !== null) {
return;
dialog.showErrorBox(
translate(
languagePreferences?.read().locale ?? "zh-CN",
languagePreferences?.read().locale ?? "zh-Hans",
"nativeErrorTitle",
),
stableRuntimeFailureDetail(error),
@@ -7,6 +7,7 @@ import {
MAX_ISSUE_URL_LENGTH,
MIGRATION_ISSUE_URL,
type MigrationDialogDependencies,
migrationDialogText,
migrationFailureCode,
migrationIssueBody,
migrationIssueUrl,
@@ -41,7 +42,7 @@ describe("migration failure dialog", () => {
const handled = showMigrationFailureDialog({
error: new Error("sdk_migration_preflight_failed"),
dataDir,
locale: "zh-CN",
locale: "zh-Hans",
dependencies: deps,
});
@@ -176,7 +177,7 @@ describe("migration failure dialog", () => {
// Desktop's own migration throws the code the note records.
error: new Error("sdk_data_plane_dispatch_failed"),
dataDir,
locale: "zh-CN",
locale: "zh-Hans",
dependencies: deps,
environment: {
appVersion: "2.0.6",
@@ -217,6 +218,35 @@ describe("migration failure dialog", () => {
expect(body).not.toContain(accountHome);
});
it("speaks the display language in the dialog and the issue body", () => {
const en = migrationDialogText("sdk_migration_x", "/tmp/m.log", "en", true);
expect(en.detail).toBe(
"To protect your data, MemHub will not start. Please attach the migration log to a GitHub issue and send it to us. (Error code: sdk_migration_x)" +
"\n\nYour data was restored to its state before the upgrade." +
"\n\nLog: /tmp/m.log",
);
const ja = migrationDialogText("sdk_migration_x", "/tmp/m.log", "ja");
expect(ja.message).toBe(
"MemHub のアップグレード中にデータ移行が完了しませんでした。",
);
expect([ja.openLog, ja.quit]).toEqual(["ログを開く", "終了"]);
expect(ja.detail).toContain("sdk_migration_x");
// A system language code resolves like the GUI does (de-AT → de).
expect(migrationDialogText("c", "/l", "de-AT").quit).toBe("Beenden");
const body = migrationIssueBody({
code: "sdk_migration_preflight_failed",
logPath: "~/.one-memory/logs/sdk-migration.log",
dataRestored: false,
arch: "arm64",
locale: "fr",
});
expect(body).toContain("## Échec de la migration lors de la mise à jour");
expect(body).toMatch(/- Code d’erreur\s: sdk_migration_preflight_failed/);
expect(body).toContain(": non");
expect(body).toMatch(/- Système\s: inconnu \(arm64\)/);
expect(body).toContain("~/.one-memory/logs/sdk-migration.log");
});
it("says 未知 for what the log does not know instead of guessing", () => {
const body = migrationIssueBody({
code: "sdk_migration_preflight_failed",
@@ -296,7 +326,7 @@ describe("migration failure dialog", () => {
showMigrationFailureDialog({
error: new Error("installed_version_conflict"),
dataDir: await home(),
locale: "zh-CN",
locale: "zh-Hans",
dependencies: deps,
}),
).toBe(false);
@@ -8,6 +8,12 @@ import {
readSync,
} from "node:fs";
import path from "node:path";
import {
type GuiLocale,
isGuiLocale,
resolveGuiLocale,
translate,
} from "@one-memory/i18n";
/**
* What a person sees when an upgrade did not reach SDK authority.
@@ -106,51 +112,70 @@ export interface MigrationIssueFacts {
arch?: string | null;
/** The local log path, already shown with the home directory as `~`. */
logPath: string;
/** The display language; Simplified Chinese when absent. */
locale?: string;
}
const UNKNOWN = "未知";
function versionText(value: string | null | undefined): string {
return typeof value === "string" && VERSION.test(value) ? value : UNKNOWN;
/** A display language, or the one a system language code resolves to. */
function dialogLocale(locale: string | undefined): GuiLocale {
if (locale === undefined) return "zh-Hans";
return isGuiLocale(locale) ? locale : resolveGuiLocale("system", locale);
}
function buildText(value: number | null | undefined): string {
function versionText(value: string | null | undefined): string | null {
return typeof value === "string" && VERSION.test(value) ? value : null;
}
function buildText(value: number | null | undefined): string | null {
return typeof value === "number" && Number.isSafeInteger(value) && value > 0
? String(value)
: UNKNOWN;
: null;
}
function stepText(
step: string | null | undefined,
sub: string | null | undefined,
) {
if (typeof step !== "string" || !STEP.test(step)) return UNKNOWN;
): string | null {
if (typeof step !== "string" || !STEP.test(step)) return null;
return typeof sub === "string" && STEP.test(sub) ? `${step} / ${sub}` : step;
}
/** The issue body: short, content-free, and honest about what is unknown. */
export function migrationIssueBody(facts: MigrationIssueFacts): string {
const locale = dialogLocale(facts.locale);
const unknown = translate(locale, "migration.unknown");
const known = (value: string | null) => value ?? unknown;
const restored =
facts.dataRestored === true
? "是"
? translate(locale, "migration.yes")
: facts.dataRestored === false
? "否"
: UNKNOWN;
? translate(locale, "migration.no")
: unknown;
const system = [facts.osName, facts.osVersion]
.map((part) => versionText(part))
.filter((part) => part !== UNKNOWN)
.filter((part) => part !== null)
.join(" ");
const arch = versionText(facts.arch);
return [
"## 升级迁移失败",
`- 从 MemHub ${versionText(facts.fromVersion)}(build ${buildText(facts.fromBuild)})升级到 ${versionText(facts.toVersion)}(build ${buildText(facts.toBuild)}),迁移数据失败。`,
`- 错误码:${CODE.test(facts.code) ? facts.code : UNKNOWN}`,
`- 失败步骤:${stepText(facts.step, facts.subStep)}`,
`- 数据已恢复到升级前状态:${restored}`,
`- 系统:${system || UNKNOWN}${arch === UNKNOWN ? "" : `(${arch})`}`,
`- 迁移日志(本地):${facts.logPath}`,
translate(locale, "migration.issueHeading"),
translate(locale, "migration.issueUpgrade", {
fromVersion: known(versionText(facts.fromVersion)),
fromBuild: known(buildText(facts.fromBuild)),
toVersion: known(versionText(facts.toVersion)),
toBuild: known(buildText(facts.toBuild)),
}),
translate(locale, "migration.issueCode", {
code: CODE.test(facts.code) ? facts.code : unknown,
}),
translate(locale, "migration.issueStep", {
step: known(stepText(facts.step, facts.subStep)),
}),
translate(locale, "migration.issueRestored", { restored }),
translate(locale, "migration.issueSystem", { system: system || unknown }) +
(arch === null ? "" : translate(locale, "migration.issueArch", { arch })),
translate(locale, "migration.issueLog", { path: facts.logPath }),
"",
"请把上面这个日志文件拖到这里作为附件,再提交。",
translate(locale, "migration.issueAttach"),
"",
].join("\n");
}
@@ -302,31 +327,18 @@ export function migrationDialogText(
locale: string,
dataRestored = false,
): MigrationDialogText {
if (locale.toLowerCase().startsWith("en")) {
return {
title: "MemHub",
message: "The data migration for this MemHub upgrade did not finish.",
detail:
"To protect your data, MemHub will not start. Please attach the migration log to a GitHub issue and send it to us. " +
`(Error code: ${code})` +
(dataRestored
? "\n\nYour data was restored to its state before the upgrade."
: "") +
`\n\nLog: ${logPath}`,
openLog: "Open log",
quit: "Quit",
};
}
const language = dialogLocale(locale);
return {
title: "MemHub",
message: "MemHub 升级时数据迁移没有完成。",
message: translate(language, "migration.dialogMessage"),
detail:
"为保护你的数据,MemHub 不会启动。请把迁移日志附在 GitHub 反馈里发给我们。" +
`(错误码:${code})` +
(dataRestored ? "\n\n记忆数据已恢复到升级前的状态。" : "") +
`\n\n日志:${logPath}`,
openLog: "打开日志",
quit: "退出",
translate(language, "migration.dialogDetail", { code }) +
(dataRestored
? `\n\n${translate(language, "migration.dataRestored")}`
: "") +
`\n\n${translate(language, "migration.logPath", { path: logPath })}`,
openLog: translate(language, "migration.openLog"),
quit: translate(language, "migration.quit"),
};
}
@@ -434,6 +446,7 @@ export function showMigrationFailureDialog(options: {
osVersion: environment.osVersion ?? null,
arch: environment.arch ?? null,
logPath: displayPath(logPath, environment.home ?? null),
locale: options.locale,
});
void Promise.resolve(
options.dependencies.openExternal(migrationIssueUrl(code, body)),
@@ -12,7 +12,7 @@ describe("startup screen", () => {
expect(html).not.toContain("ipcRenderer");
});
it("speaks the interface language and keeps Chinese by default", () => {
expect(startupScreenHtml()).toContain('<html lang="zh-CN">');
expect(startupScreenHtml()).toContain('<html lang="zh-Hans">');
expect(startupScreenHtml()).toContain("正在启动,正在准备记忆服务…");
const spanish = startupScreenHtml("es");
expect(spanish).toContain('<html lang="es">');
+1 -1
View File
@@ -11,7 +11,7 @@ export const STARTUP_SCREEN_URL = `${APP_ORIGIN}${STARTUP_SCREEN_PATH}`;
* the gap. It deliberately makes no IPC calls: it renders before `initialize`
* has registered any handler, and anything it asked for would fail.
*/
export function startupScreenHtml(locale: GuiLocale = "zh-CN"): string {
export function startupScreenHtml(locale: GuiLocale = "zh-Hans"): string {
return `<!doctype html>
<html lang="${locale}">
<head>
+14 -2
View File
@@ -11,8 +11,20 @@ const { resolveBuildNumber, resolveBuildVersion } =
};
const releaseVersion = resolveBuildVersion(process.env);
const releaseBuildNumber = resolveBuildNumber(process.env);
const releaseCommit =
process.env.ONE_MEMORY_COMMIT?.trim() || readSourceCommit();
const releaseCommit = resolveReleaseCommit(process.env.ONE_MEMORY_COMMIT);
// The runtime accepts only a full 40-character commit, so an explicitly
// supplied malformed value fails the build instead of being baked and skipped.
function resolveReleaseCommit(explicit: string | undefined): string {
const trimmed = explicit?.trim();
if (!trimmed) return readSourceCommit();
if (!/^[0-9a-f]{40}$/i.test(trimmed)) {
throw new Error(
`ONE_MEMORY_COMMIT must be a full 40-character hex commit, got ${JSON.stringify(trimmed)}`,
);
}
return trimmed.toLowerCase();
}
function readSourceCommit(): string {
try {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/site",
"version": "1.0.5",
"version": "1.0.6",
"private": true,
"type": "module",
"scripts": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/web",
"version": "1.0.5",
"version": "1.0.6",
"private": true,
"type": "module",
"scripts": {
@@ -27,7 +27,7 @@ it("translates application detail live without triggering extra reads or writes"
).toBeTruthy();
expect(screen.getByRole("switch", { name: "Sincronización" })).toBeTruthy();
await waitFor(() => expect(refresh).toHaveBeenCalledTimes(1));
localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-CN");
localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-Hans");
await act(async () =>
window.dispatchEvent(
new StorageEvent("storage", { key: LANGUAGE_STORAGE_KEY }),
@@ -603,7 +603,7 @@ it("shows insertion still on when neither removal nor revocation succeeded", asy
});
let languageListener: ((snapshot: unknown) => void) | undefined;
function switchLanguage(locale: "es" | "zh-CN", revision: number) {
function switchLanguage(locale: "es" | "zh-Hans", revision: number) {
act(() => languageListener?.({ preference: locale, locale, revision }));
}
async function failDisconnectInSpanish() {
@@ -660,7 +660,7 @@ it("re-words a visible failure when the language changes, without new writes", a
).getRecallHosts;
const readCount = reads.mock.calls.length;
const revokes = bridge.revokeRecallPermission.mock.calls.length;
switchLanguage("zh-CN", 2);
switchLanguage("zh-Hans", 2);
expect(
await screen.findByText(
"连接未能移除,记忆权限也未能撤销,OpenCode 仍可使用记忆,请重试。原因:应用控制通道暂不可用。",
@@ -23,8 +23,8 @@ import { queryClient, resetClient } from "./query-client";
it("localizes the whole home live without repeating reads or changing imported text", async () => {
let change:
| ((snapshot: {
preference: "es" | "zh-CN";
locale: "es" | "zh-CN";
preference: "es" | "zh-Hans";
locale: "es" | "zh-Hans";
revision: number;
}) => void)
| undefined;
@@ -52,7 +52,9 @@ it("localizes the whole home live without repeating reads or changing imported t
fireEvent.click(
screen.getByRole("button", { name: "Ver las 5 aplicaciones" }),
);
act(() => change?.({ preference: "zh-CN", locale: "zh-CN", revision: 2 }));
act(() =>
change?.({ preference: "zh-Hans", locale: "zh-Hans", revision: 2 }),
);
expect(
screen.getByRole("heading", { name: "全部应用", level: 1 }),
).toBeTruthy();
+1 -1
View File
@@ -166,7 +166,7 @@ it("traduce el historial en vivo sin cambiar nombres ni contenido importado", as
).toBeTruthy();
expect(screen.getAllByText("Consulta completada")).toHaveLength(2);
expect(screen.getByText("Consulta de memorias")).toBeTruthy();
window.localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-CN");
window.localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-Hans");
fireEvent(window, new StorageEvent("storage", { key: LANGUAGE_STORAGE_KEY }));
expect(screen.getByRole("heading", { name: "这次查记忆" })).toBeTruthy();
window.localStorage.removeItem(LANGUAGE_STORAGE_KEY);
+1 -1
View File
@@ -205,7 +205,7 @@ function ConnectionMaterialFallback({
? t("connectionFallback.promptShell", { shell: material.shell })
: "";
const prompt =
locale === "zh-CN"
locale === "zh-Hans"
? material.prompt
: t("connectionFallback.promptTemplate", {
mcp: material.mcp,
+1 -1
View File
@@ -111,7 +111,7 @@ afterEach(() => {
function SwitchLanguage() {
const { setPreference } = useGuiLanguage();
return (
<button type="button" onClick={() => void setPreference("zh-CN")}>
<button type="button" onClick={() => void setPreference("zh-Hans")}>
Cambiar idioma
</button>
);
@@ -22,7 +22,7 @@ import {
function LanguageSwitch() {
const { setPreference } = useGuiLanguage();
return (
<button type="button" onClick={() => void setPreference("zh-CN")}>
<button type="button" onClick={() => void setPreference("zh-Hans")}>
Cambiar idioma
</button>
);
+1 -1
View File
@@ -33,7 +33,7 @@ vi.mock("./api", () => ({
function HealthLanguageSwitch() {
const { setPreference } = useGuiLanguage();
return (
<button type="button" onClick={() => void setPreference("zh-CN")}>
<button type="button" onClick={() => void setPreference("zh-Hans")}>
Cambiar idioma
</button>
);
@@ -95,7 +95,7 @@ describe("ImportProgressStrip", () => {
function Switch() {
const { setPreference } = useGuiLanguage();
return (
<button type="button" onClick={() => void setPreference("zh-CN")}>
<button type="button" onClick={() => void setPreference("zh-Hans")}>
Cambiar idioma
</button>
);
+36 -30
View File
@@ -1,6 +1,6 @@
// @vitest-environment jsdom
import { translate } from "@one-memory/i18n";
import { LOCALE_NAMES, SUPPORTED_LOCALES, translate } from "@one-memory/i18n";
import {
cleanup,
fireEvent,
@@ -20,7 +20,7 @@ afterEach(() => {
vi.restoreAllMocks();
});
it.each(["zh-CN", "en", "es"] as const)(
it.each(["zh-Hans", "en", "es", "ja", "de", "ru"] as const)(
"hides the compact label while preserving its accessible name in %s",
async (locale) => {
localStorage.setItem(LANGUAGE_STORAGE_KEY, locale);
@@ -42,7 +42,7 @@ it.each(["zh-CN", "en", "es"] as const)(
},
);
it.each(["zh-CN", "en", "es"] as const)(
it.each(["zh-Hans", "en", "es", "zh-Hant", "fr", "ko"] as const)(
"keeps the normal settings label visible and accessible in %s",
(locale) => {
localStorage.setItem(LANGUAGE_STORAGE_KEY, locale);
@@ -66,7 +66,7 @@ it.each(["zh-CN", "en", "es"] as const)(
},
);
it.each(["system", "zh-CN", "en", "es"] as const)(
it.each(["system", "zh-Hans", "en", "es", "pt", "id"] as const)(
"keeps one stable settings row for the %s preference",
async (preference) => {
localStorage.setItem(LANGUAGE_STORAGE_KEY, preference);
@@ -81,40 +81,46 @@ it.each(["system", "zh-CN", "en", "es"] as const)(
expect(row).toBeTruthy();
expect(row?.querySelector("label")?.hidden).toBe(false);
await userEvent.click(selector);
expect(
screen
.getByRole("option", {
name: /简体中文|Simplified Chinese|Chino simplificado/,
})
.getAttribute("lang"),
).toBe("zh-CN");
expect(
screen
.getByRole("option", { name: /英语|English|Inglés/ })
.getAttribute("lang"),
).toBe("en");
expect(
screen
.getByRole("option", { name: /西班牙语|Spanish|Español/ })
.getAttribute("lang"),
).toBe("es");
// Every language by its own name, whatever the interface language.
for (const locale of SUPPORTED_LOCALES) {
expect(
screen
.getByRole("option", { name: LOCALE_NAMES[locale] })
.getAttribute("lang"),
).toBe(locale);
}
},
);
it("offers an accessible selector, persists and updates its label", async () => {
localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-CN");
localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-Hans");
render(
<LanguageProvider>
<LanguageSelector />
</LanguageProvider>,
);
const selector = screen.getByRole("combobox", {
name: translate("zh-CN", "languageLabel"),
name: translate("zh-Hans", "languageLabel"),
});
expect(selector.getAttribute("aria-describedby")).toBeTruthy();
await userEvent.click(selector);
expect(screen.getAllByRole("option")).toHaveLength(4);
await userEvent.click(screen.getByRole("option", { name: "西班牙语" }));
expect(
screen.getAllByRole("option").map((option) => option.textContent),
).toEqual([
translate("zh-Hans", "languageSystem"),
"English",
"简体中文",
"繁體中文",
"日本語",
"Español",
"Deutsch",
"Português",
"Français",
"한국어",
"Русский",
"Bahasa Indonesia",
]);
await userEvent.click(screen.getByRole("option", { name: "Español" }));
expect(localStorage.getItem(LANGUAGE_STORAGE_KEY)).toBe("es");
expect(
screen.getByRole("combobox", { name: translate("es", "languageLabel") }),
@@ -143,12 +149,12 @@ it("supports keyboard selection without using the system select menu", async ()
expect(selector.getAttribute("aria-expanded")).toBe("true");
await userEvent.keyboard("{Escape}");
await userEvent.keyboard("{ArrowDown}{ArrowDown}{Enter}");
expect(localStorage.getItem(LANGUAGE_STORAGE_KEY)).toBe("zh-CN");
expect(localStorage.getItem(LANGUAGE_STORAGE_KEY)).toBe("en");
expect(selector.getAttribute("aria-expanded")).toBe("false");
});
it("announces storage errors without replacing the confirmed choice", async () => {
localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-CN");
localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-Hans");
render(
<LanguageProvider>
<LanguageSelector />
@@ -158,12 +164,12 @@ it("announces storage errors without replacing the confirmed choice", async () =
throw new Error("denied");
});
await userEvent.click(screen.getByRole("combobox"));
await userEvent.click(screen.getByRole("option", { name: "西班牙语" }));
await userEvent.click(screen.getByRole("option", { name: "Español" }));
expect(screen.getByRole("alert").textContent).toBe(
translate("zh-CN", "languageSaveError"),
translate("zh-Hans", "languageSaveError"),
);
expect((screen.getByRole("combobox") as HTMLButtonElement).value).toBe(
"zh-CN",
"zh-Hans",
);
});
+12 -17
View File
@@ -1,4 +1,8 @@
import { isLanguagePreference } from "@one-memory/i18n";
import {
isLanguagePreference,
LOCALE_NAMES,
SUPPORTED_LOCALES,
} from "@one-memory/i18n";
import { useId, useState } from "react";
import { useGuiLanguage } from "./gui-language";
import { OptionMenu } from "./OptionMenu";
@@ -7,24 +11,15 @@ export function LanguageSelector({ compact = false }: { compact?: boolean }) {
const id = useId();
const { preference, setPreference, t, error } = useGuiLanguage();
const [pending, setPending] = useState(false);
// Each language by its own name, so a person can find theirs in any UI.
const options = [
{ value: "system", label: t("languageSystem") },
{
value: "zh-CN",
label: compact ? "简体中文" : t("languageChinese"),
lang: "zh-CN",
},
{
value: "en",
label: compact ? "English" : t("languageEnglish"),
lang: "en",
},
{
value: "es",
label: compact ? "Español" : t("languageSpanish"),
lang: "es",
},
] as const;
...SUPPORTED_LOCALES.map((locale) => ({
value: locale,
label: LOCALE_NAMES[locale],
lang: locale,
})),
];
return (
<section
aria-labelledby={compact ? `${id}-label` : `${id}-title`}
Loaded 100 of 154 files, more files were not shown because too many files have changed in this diff. Show more