Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d0a4fd60f2 | ||
|
|
aae877b880 | ||
|
|
670c7c6cb1 | ||
|
|
aa633623da | ||
|
|
37d8662b93 | ||
|
|
9459a1107b | ||
|
|
2c6f269170 | ||
|
|
7a2ff2f4c4 | ||
|
|
35128f2c84 | ||
|
|
f12a95ce7a | ||
|
|
2b75adc718 | ||
|
|
75ac1b01f2 | ||
|
|
e786657a0b | ||
|
|
b2c91d9751 | ||
|
|
7df371bb10 | ||
|
|
f38c370fbd | ||
|
|
e87d6305be | ||
|
|
7a78fca807 |
No files matched your search
+45
-3
@@ -4,7 +4,47 @@ Hy Memory and MemHub are versioned and released independently.
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Hy Memory: LLM prompts now ship in 11 languages: `en`, `zh-Hans`,
|
||||
## MemHub 1.0.6 — 2026-11-02
|
||||
|
||||
Build 37, on `hy-memory==2.0.11`. Linux x64/arm64 Runtime and macOS arm64
|
||||
App and Runtime.
|
||||
|
||||
- Interfaces in 11 languages: English, Simplified and Traditional Chinese,
|
||||
Japanese, Spanish, German, Portuguese, French, Korean, Russian and
|
||||
Indonesian. Choosing any language other than Simplified Chinese also sets
|
||||
the SDK prompt language; Simplified Chinese keeps the automatic zh/en
|
||||
detection.
|
||||
- Memory modes: lite (no LLM, keeps raw conversations), pro (LLM extraction)
|
||||
and ultra (pro plus the knowledge graph and a nightly evolve run). Set with
|
||||
`GET`/`PUT /api/v1/settings/memory-mode` or `memhub settings mode`. Raw
|
||||
conversations are now kept permanently. `POST /api/v2/evolve` and
|
||||
`memhub evolve` run System 2 on demand.
|
||||
- Safe embedding-model switching: an embedding-space registry, a switch that
|
||||
pauses writes, re-embeds every user's vectors and the graph, verifies and
|
||||
commits atomically, and resumes after a crash or an App update. Every start
|
||||
scans the vector store and merges data left in other spaces. When the
|
||||
embedding service is unavailable, search returns `embedding_unavailable`.
|
||||
- Multi-user Runtime: memory APIs and the CLI take `user_id` (default
|
||||
`local`); records are isolated per user; Hermes profiles can be connected as
|
||||
separate users. The App keeps using `local`.
|
||||
- Configurable SDK features through the CLI and API (`memhub settings
|
||||
features`): graph rerank, episode recall, session episodes, episode
|
||||
summaries, intention and few-shot. Defaults are unchanged.
|
||||
- Search and recall pass through the SDK's search parameters with server-side
|
||||
score floors; the memory library filters by layer; `/api/v2/raw` returns
|
||||
data again; v2 source and import routes; profile version history.
|
||||
- Upgrade fixes: migration no longer stops without a model profile (#234) or
|
||||
on a pre-existing readback mismatch (#229, #240); source grants work after
|
||||
the cutover (#233); reads use the active embedding (#242); the embedding
|
||||
endpoint no longer follows the generation `base_url`.
|
||||
- Hermes import drops gateway notices and probes and unwraps relayed
|
||||
messages; `memhub logs` no longer reports valid older records as damaged.
|
||||
|
||||
## Hy Memory 2.0.11 — 2026-11-01
|
||||
|
||||
- Published
|
||||
[`hy-memory==2.0.11`](https://pypi.org/project/hy-memory/2.0.11/).
|
||||
- LLM prompts now ship in 11 languages: `en`, `zh-Hans`,
|
||||
`zh-Hant`, `ja`, `es`, `de`, `pt`, `fr`, `ko`, `ru` and `id`. They live in
|
||||
one registry, `hy_memory/prompts/`, with one module per language. Call
|
||||
sites use `get_prompt(name, language)`, so a new language only needs a new
|
||||
@@ -17,7 +57,7 @@ Hy Memory and MemHub are versioned and released independently.
|
||||
(`prompts_zh.*_ZH`, `extractor.EXTRACT_SYSTEM_PROMPT`, ...) still import.
|
||||
The nine new languages are machine translations, and native-speaker review
|
||||
is recommended.
|
||||
- Hy Memory: new optional `language` to choose the prompt language
|
||||
- New optional `language` to choose the prompt language
|
||||
explicitly. Set it on `MemoryConfig.language` (env `MEMORY_LANGUAGE`) for
|
||||
a client default, or per call on `add` / `async_add`, `add_extracted`,
|
||||
`finalize_session`, `evolve` and `sweep`. A per-call value overrides the
|
||||
@@ -30,11 +70,13 @@ Hy Memory and MemHub are versioned and released independently.
|
||||
Without a language, the existing zh/en auto-detection runs unchanged and
|
||||
memories keep the source language. `search` has no `language`: its judge
|
||||
and rewrite prompts are English-only and keep the query's language.
|
||||
- Hy Memory: the HTTP server accepts `language` on `/api/v1/add` and
|
||||
- The HTTP server accepts `language` on `/api/v1/add` and
|
||||
`/api/v1/add_extracted`.
|
||||
|
||||
## Hy Memory 2.0.10 — 2026-11-01
|
||||
|
||||
- Published
|
||||
[`hy-memory==2.0.10`](https://pypi.org/project/hy-memory/2.0.10/).
|
||||
- New `hy_memory.reembed` module for switching the embedding
|
||||
model. Writers must be paused for the whole run. Nothing is derived from
|
||||
the provider or endpoint: source and target are named explicitly, and the
|
||||
|
||||
@@ -33,6 +33,7 @@ English | [中文](README.zh.md)
|
||||
## Latest News
|
||||
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Open-sourced** quantized **FP8** (**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**) and **GGUF** (**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**) versions of **Hy-Memory-30B-A3B** on Hugging Face!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-11]**: 📦 **Released [MemHub v1.0.6](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.6)** with interfaces in 11 languages, memory modes (lite / pro / ultra with nightly evolve), safe embedding-model switching, multi-user support for the CLI and API, and more reliable upgrades, on **`hy-memory==2.0.11`**. See the release notes for details.
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **Released [MemHub v1.0.5](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.5)** with Chinese, English and Spanish interfaces, one-time pairing codes for more custom Harnesses, and smoother browsing, search, startup and recovery, on **`hy-memory==2.0.8`**. See the release notes for details.
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **Released [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** and **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**, with a signed macOS desktop app and verified macOS/Linux Runtimes!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Released** **Hy Memory 2.0 (`hy-memory==2.0.4`)** on [PyPI](https://pypi.org/project/hy-memory/2.0.4/) and open-sourced the **Hy-Memory** monorepo together with **[MemHub](memhub/README.md)**!
|
||||
|
||||
@@ -33,6 +33,7 @@
|
||||
## 最新动态
|
||||
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **模型开源** 在 Hugging Face 发布 **Hy-Memory-30B-A3B** 的 **FP8**(**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**)与 **GGUF**(**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**)量化版本模型!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-11]**: 📦 **正式发布 [MemHub v1.0.6](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.6)**:界面支持 11 种语言,新增记忆模式(lite / pro / ultra,ultra 每晚自动整理),可安全切换嵌入模型,CLI 和 API 支持多用户,升级更可靠;底层使用 **`hy-memory==2.0.11`**。详见发布说明。
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **正式发布 [MemHub v1.0.5](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.5)**:支持中文、英文和西班牙语界面,可通过一次性配对码连接更多自定义 Harness,并优化了记忆浏览、搜索、启动和异常恢复;底层使用 **`hy-memory==2.0.8`**。详见发布说明。
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **正式发布 [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** 与 **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**,提供已签名的 macOS 桌面应用及经过验证的 macOS/Linux Runtime!
|
||||
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **正式发布** **Hy Memory 2.0 (`hy-memory==2.0.4`)**(已上线 [PyPI](https://pypi.org/project/hy-memory/2.0.4/)),并开源 **Hy-Memory** monorepo 与 **[MemHub](memhub/README.md)**!
|
||||
|
||||
+4
-4
@@ -18,13 +18,13 @@ If you run into a problem, please open an Issue with the version, platform,repro
|
||||
|
||||
## 发行与平台合同
|
||||
|
||||
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.9`。MemHub 源码版本为 `1.0.5`
|
||||
(build 36),GitHub 仓库为
|
||||
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.11`。MemHub 源码版本为 `1.0.6`
|
||||
(build 37),GitHub 仓库为
|
||||
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
|
||||
`memhub-cli`。当前代码架构见
|
||||
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)。
|
||||
|
||||
`v1.0.5` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64 App 和 Runtime。
|
||||
`v1.0.6` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64 App 和 Runtime。
|
||||
Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命周期验收;macOS
|
||||
制品由发行流水线完成 Developer ID 签名、公证和验收后手动上传。Windows 不支持。
|
||||
只有同时出现在 GitHub Release 与 `release-manifest.json` 中、且通过公开校验的制品才属于
|
||||
@@ -118,7 +118,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
|
||||
|
||||
## Hy Memory
|
||||
|
||||
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.9`。SDK 配置、权限边界及 search
|
||||
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.11`。SDK 配置、权限边界及 search
|
||||
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
|
||||
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
|
||||
|
||||
|
||||
@@ -2352,7 +2352,7 @@
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-hy-memory",
|
||||
"name": "hy-memory",
|
||||
"versionInfo": "2.0.9",
|
||||
"versionInfo": "2.0.11",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
"filesAnalyzed": false,
|
||||
"licenseConcluded": "Apache-2.0",
|
||||
|
||||
@@ -9,7 +9,7 @@ Per-package license texts are not reproduced here yet; the license
|
||||
column is filled in as part of release review, and the SBOM records
|
||||
`NOASSERTION` until then rather than guessing.
|
||||
|
||||
## hy-memory 2.0.9
|
||||
## hy-memory 2.0.11
|
||||
|
||||
License: Apache-2.0
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@one-memory/cli",
|
||||
"version": "1.0.5",
|
||||
"version": "1.0.6",
|
||||
"description": "MemHub bootstrap CLI for installing and managing the local-first shared memory runtime",
|
||||
"license": "Apache-2.0",
|
||||
"repository": {
|
||||
|
||||
+243
-25
@@ -122,6 +122,14 @@ import {
|
||||
doctorExitCode,
|
||||
renderDoctorText,
|
||||
} from "./doctor-report.js";
|
||||
import {
|
||||
activateWithEmbeddingSwitch,
|
||||
cleanupSpace,
|
||||
listSpaces,
|
||||
type SwitchIo,
|
||||
switchCancel,
|
||||
switchStatus,
|
||||
} from "./embedding-switch-flow.js";
|
||||
import {
|
||||
EXIT_FAILED,
|
||||
EXIT_OK,
|
||||
@@ -195,6 +203,13 @@ import {
|
||||
renderFeaturesText,
|
||||
runFeaturesCommand,
|
||||
} from "./memory-features-flow.js";
|
||||
import {
|
||||
type LanguageDocument,
|
||||
languageExitCode,
|
||||
parseLanguageCommand,
|
||||
renderLanguageText,
|
||||
runLanguageCommand,
|
||||
} from "./memory-language-flow.js";
|
||||
import {
|
||||
type EvolveDocument,
|
||||
evolveExitCode,
|
||||
@@ -248,7 +263,6 @@ import {
|
||||
renderModelReportText,
|
||||
} from "./model-report.js";
|
||||
import {
|
||||
activateModelProfile,
|
||||
editModelProfile,
|
||||
type ModelWriteDocument,
|
||||
modelWriteExitCode,
|
||||
@@ -1703,6 +1717,18 @@ async function runCommand(
|
||||
);
|
||||
if (apiKey === undefined) return usage();
|
||||
}
|
||||
if (
|
||||
parsed.kind === "spaces" &&
|
||||
parsed.cleanup !== undefined &&
|
||||
!parsed.confirmed
|
||||
) {
|
||||
return emitConfirmPreview(asJson, {
|
||||
command: "models spaces",
|
||||
target: parsed.cleanup,
|
||||
effect: `删除已停用的向量空间 ${parsed.cleanup}(不可恢复;正在使用的空间不能删除)`,
|
||||
retry: `${PRODUCT} models spaces --cleanup ${parsed.cleanup} --yes`,
|
||||
});
|
||||
}
|
||||
if (parsed.kind === "remove" && !parsed.confirmed) {
|
||||
return emitConfirmPreview(asJson, {
|
||||
command: "models remove",
|
||||
@@ -1725,6 +1751,50 @@ async function runCommand(
|
||||
return connected.exitCode;
|
||||
}
|
||||
const { reader, operations } = connected;
|
||||
// Ctrl-C while following a switch stops watching, never the switch.
|
||||
const interrupt = new AbortController();
|
||||
const switchIo: SwitchIo = {
|
||||
reader,
|
||||
operations,
|
||||
emit: (text) => {
|
||||
emit(text);
|
||||
},
|
||||
asJson,
|
||||
interactive:
|
||||
!asJson && (controls.stdinIsTTY ?? process.stdin.isTTY) === true,
|
||||
confirm: async (question) => {
|
||||
const prompter =
|
||||
controls.wizardPrompter ?? (await createTerminalPrompter());
|
||||
try {
|
||||
return /^(y|yes)$/i.test((await prompter.ask(question)).trim());
|
||||
} finally {
|
||||
if ("close" in prompter && typeof prompter.close === "function") {
|
||||
prompter.close();
|
||||
}
|
||||
}
|
||||
},
|
||||
...(controls.watchWait ? { wait: controls.watchWait } : {}),
|
||||
signal: controls.watchSignal ?? interrupt.signal,
|
||||
};
|
||||
const withInterrupt = async (run: () => Promise<number>) => {
|
||||
const onInterrupt = () => interrupt.abort();
|
||||
process.once("SIGINT", onInterrupt);
|
||||
try {
|
||||
return await run();
|
||||
} finally {
|
||||
process.removeListener("SIGINT", onInterrupt);
|
||||
}
|
||||
};
|
||||
if (parsed.kind === "switch") {
|
||||
return parsed.action === "status"
|
||||
? await switchStatus(switchIo)
|
||||
: await switchCancel(switchIo);
|
||||
}
|
||||
if (parsed.kind === "spaces") {
|
||||
return parsed.cleanup === undefined
|
||||
? await listSpaces(switchIo)
|
||||
: await cleanupSpace(parsed.cleanup, switchIo);
|
||||
}
|
||||
if (parsed.kind === "add") {
|
||||
const created = await createModelProfile(
|
||||
config,
|
||||
@@ -1739,33 +1809,66 @@ async function runCommand(
|
||||
emitFlowDocument(created, asJson);
|
||||
return flowExitCode(created);
|
||||
}
|
||||
const activated = await activateModelProfile(
|
||||
String(created.receipt.profile_id),
|
||||
// Core activates only a probed profile: probe first, then the same
|
||||
// path as `models activate`, so an embedding change shows its plan.
|
||||
const profileId = String(created.receipt.profile_id);
|
||||
const probe = await probeUntilSettled(
|
||||
reader,
|
||||
operations,
|
||||
profileId,
|
||||
controls.watchWait,
|
||||
);
|
||||
const document = {
|
||||
...created,
|
||||
activation:
|
||||
activated.state === "committed"
|
||||
? { state: "committed" }
|
||||
: { state: activated.state, code: activated.code },
|
||||
};
|
||||
const probed =
|
||||
probe.state === "completed" &&
|
||||
Boolean(
|
||||
(
|
||||
(probe.receipt.result as Record<string, unknown> | undefined)
|
||||
?.capabilities as Record<string, unknown> | undefined
|
||||
)?.embedding_available,
|
||||
);
|
||||
emit(
|
||||
asJson
|
||||
? `${JSON.stringify(document)}\n`
|
||||
? `${JSON.stringify({
|
||||
...created,
|
||||
probe: {
|
||||
state: probe.state,
|
||||
...("operation_id" in probe && probe.operation_id
|
||||
? { operation_id: probe.operation_id }
|
||||
: {}),
|
||||
...("code" in probe ? { code: probe.code } : {}),
|
||||
},
|
||||
})}\n`
|
||||
: `${renderFlowText(created)}${
|
||||
activated.state === "committed"
|
||||
? "已设为当前使用的模型配置。\n"
|
||||
: `设为当前使用未完成:${activated.state}(${activated.code})\n`
|
||||
probed
|
||||
? "探测通过。\n"
|
||||
: `探测未通过(${"code" in probe ? probe.code : probe.state}),没有设为当前使用;用 ${PRODUCT} models test ${profileId} 查看原因。\n`
|
||||
}`,
|
||||
);
|
||||
return activated.state === "committed" ? EXIT_OK : EXIT_FAILED;
|
||||
if (!probed) return EXIT_FAILED;
|
||||
return await withInterrupt(() =>
|
||||
activateWithEmbeddingSwitch(
|
||||
profileId,
|
||||
{ confirmed: parsed.confirmed, follow: parsed.follow },
|
||||
switchIo,
|
||||
),
|
||||
);
|
||||
}
|
||||
if (parsed.kind === "edit" || parsed.kind === "activate") {
|
||||
const document =
|
||||
parsed.kind === "edit"
|
||||
? await editModelProfile(parsed.profileId, config, apiKey, operations)
|
||||
: await activateModelProfile(parsed.profileId, operations);
|
||||
if (parsed.kind === "activate") {
|
||||
return await withInterrupt(() =>
|
||||
activateWithEmbeddingSwitch(
|
||||
parsed.profileId,
|
||||
{ confirmed: parsed.confirmed, follow: parsed.follow },
|
||||
switchIo,
|
||||
),
|
||||
);
|
||||
}
|
||||
if (parsed.kind === "edit") {
|
||||
const document = await editModelProfile(
|
||||
parsed.profileId,
|
||||
config,
|
||||
apiKey,
|
||||
operations,
|
||||
);
|
||||
emitModelWriteDocument(document, asJson);
|
||||
return modelWriteExitCode(document);
|
||||
}
|
||||
@@ -2283,6 +2386,40 @@ async function runCommand(
|
||||
);
|
||||
return featuresExitCode(document);
|
||||
}
|
||||
if (
|
||||
named === "settings" &&
|
||||
arguments_.slice(1).find((value) => value !== "--json") === "language"
|
||||
) {
|
||||
const parsed = parseLanguageCommand(
|
||||
arguments_
|
||||
.slice(1)
|
||||
.filter((value, index, all) => index !== all.indexOf("language")),
|
||||
);
|
||||
if (!parsed) return usage();
|
||||
const connected = await connectRuntime(environment);
|
||||
if (!connected.ok) {
|
||||
const document: LanguageDocument = {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
command: "settings language",
|
||||
state: "unavailable",
|
||||
code: connected.code,
|
||||
};
|
||||
emit(
|
||||
asJson ? `${JSON.stringify(document)}\n` : renderLanguageText(document),
|
||||
);
|
||||
return connected.exitCode;
|
||||
}
|
||||
const document = await runLanguageCommand(
|
||||
parsed,
|
||||
connected.reader,
|
||||
connected.operations,
|
||||
);
|
||||
emit(
|
||||
asJson ? `${JSON.stringify(document)}\n` : renderLanguageText(document),
|
||||
);
|
||||
return languageExitCode(document);
|
||||
}
|
||||
if (named === "settings") {
|
||||
const parsed = parseSettingsCommand(arguments_.slice(1));
|
||||
if (!parsed) return usage();
|
||||
@@ -5289,6 +5426,8 @@ type ModelsCommand =
|
||||
configFile: string;
|
||||
apiKeyFile?: string;
|
||||
activate: boolean;
|
||||
confirmed: boolean;
|
||||
follow?: boolean;
|
||||
}
|
||||
| {
|
||||
kind: "edit";
|
||||
@@ -5296,7 +5435,14 @@ type ModelsCommand =
|
||||
configFile: string;
|
||||
apiKeyFile?: string;
|
||||
}
|
||||
| { kind: "activate"; profileId: string }
|
||||
| {
|
||||
kind: "activate";
|
||||
profileId: string;
|
||||
confirmed: boolean;
|
||||
follow?: boolean;
|
||||
}
|
||||
| { kind: "switch"; action: "status" | "cancel" }
|
||||
| { kind: "spaces"; cleanup?: string; confirmed: boolean }
|
||||
| { kind: "test"; profileId: string }
|
||||
| { kind: "receipt"; operationId: string }
|
||||
| { kind: "remove"; profileId: string; confirmed: boolean };
|
||||
@@ -5337,16 +5483,46 @@ function parseModelsCommand(rest: readonly string[]): ModelsCommand | null {
|
||||
const flags = readFlags(
|
||||
tokens.slice(1),
|
||||
["--config-file", "--api-key-file"],
|
||||
["--activate"],
|
||||
["--activate", "--yes", "--wait", "--no-wait"],
|
||||
);
|
||||
const configFile = flags?.values.get("--config-file");
|
||||
if (!flags || configFile === undefined) return null;
|
||||
const follow = followFlag(flags.switches);
|
||||
if (follow === null) return null;
|
||||
const activate = flags.switches.has("--activate");
|
||||
// The switch flags only mean something when the add also activates.
|
||||
if (
|
||||
!activate &&
|
||||
["--yes", "--wait", "--no-wait"].some((flag) => flags.switches.has(flag))
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const apiKeyFile = flags.values.get("--api-key-file");
|
||||
return {
|
||||
kind: "add",
|
||||
configFile,
|
||||
...(apiKeyFile === undefined ? {} : { apiKeyFile }),
|
||||
activate: flags.switches.has("--activate"),
|
||||
activate,
|
||||
confirmed: flags.switches.has("--yes"),
|
||||
...(follow === undefined ? {} : { follow }),
|
||||
};
|
||||
}
|
||||
if (action === "switch") {
|
||||
if (tokens.length !== 2 || (target !== "status" && target !== "cancel")) {
|
||||
return null;
|
||||
}
|
||||
return { kind: "switch", action: target };
|
||||
}
|
||||
if (action === "spaces") {
|
||||
const flags = readFlags(tokens.slice(1), ["--cleanup"], ["--yes"]);
|
||||
if (!flags) return null;
|
||||
const cleanup = flags.values.get("--cleanup");
|
||||
if (cleanup === undefined && flags.switches.has("--yes")) return null;
|
||||
if (cleanup !== undefined && !OPERATION_ID.test(cleanup)) return null;
|
||||
return {
|
||||
kind: "spaces",
|
||||
...(cleanup === undefined ? {} : { cleanup }),
|
||||
confirmed: flags.switches.has("--yes"),
|
||||
};
|
||||
}
|
||||
if (target === undefined || !OPERATION_ID.test(target)) return null;
|
||||
@@ -5367,8 +5543,21 @@ function parseModelsCommand(rest: readonly string[]): ModelsCommand | null {
|
||||
};
|
||||
}
|
||||
if (action === "activate") {
|
||||
const flags = readFlags(tokens.slice(2), [], ["--confirm", "--yes"]);
|
||||
return flags ? { kind: "activate", profileId: target } : null;
|
||||
const flags = readFlags(
|
||||
tokens.slice(2),
|
||||
[],
|
||||
["--confirm", "--yes", "--wait", "--no-wait"],
|
||||
);
|
||||
if (!flags) return null;
|
||||
const follow = followFlag(flags.switches);
|
||||
if (follow === null) return null;
|
||||
return {
|
||||
kind: "activate",
|
||||
profileId: target,
|
||||
// `--confirm` stays the hidden alias of `--yes`.
|
||||
confirmed: flags.switches.has("--yes") || flags.switches.has("--confirm"),
|
||||
...(follow === undefined ? {} : { follow }),
|
||||
};
|
||||
}
|
||||
if (action === "test") {
|
||||
return tokens.length === 2 ? { kind: "test", profileId: target } : null;
|
||||
@@ -5391,6 +5580,35 @@ function parseModelsCommand(rest: readonly string[]): ModelsCommand | null {
|
||||
return null;
|
||||
}
|
||||
|
||||
/** `--wait` / `--no-wait`: undefined when neither, null when both. */
|
||||
function followFlag(switches: ReadonlySet<string>): boolean | undefined | null {
|
||||
const wait = switches.has("--wait");
|
||||
const noWait = switches.has("--no-wait");
|
||||
if (wait && noWait) return null;
|
||||
return wait ? true : noWait ? false : undefined;
|
||||
}
|
||||
|
||||
/** Start a probe and read its receipt until it settles (about two minutes). */
|
||||
async function probeUntilSettled(
|
||||
reader: CoreReader,
|
||||
operations: CoreOperations,
|
||||
profileId: string,
|
||||
wait?: (milliseconds: number, signal: AbortSignal) => Promise<void>,
|
||||
): Promise<ModelProbeFlowDocument> {
|
||||
let document = await startModelProbe(reader, operations, profileId);
|
||||
const pause =
|
||||
wait ??
|
||||
((milliseconds: number) =>
|
||||
new Promise<void>((resolve) => setTimeout(resolve, milliseconds)));
|
||||
const signal = new AbortController().signal;
|
||||
for (let attempt = 0; attempt < 120; attempt += 1) {
|
||||
if (document.state !== "pending" || !document.operation_id) break;
|
||||
await pause(1_000, signal);
|
||||
document = await readModelProbeReceipt(reader, document.operation_id);
|
||||
}
|
||||
return document;
|
||||
}
|
||||
|
||||
type SourcesCommand =
|
||||
| {
|
||||
kind: "ids";
|
||||
|
||||
@@ -361,9 +361,9 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
{
|
||||
name: "models",
|
||||
summary:
|
||||
"管理生成模型配置。add/test 直接提交并返回操作 ID(异步,用 receipt 查结果);edit/activate 直接执行;remove 不可逆,需 --yes。<profile-id>/<operation-id> 都是 UUID;脚本模式的凭据只从受控文件读取。",
|
||||
"管理生成模型配置。add/test 直接提交并返回操作 ID(异步,用 receipt 查结果);edit 直接执行;activate 换了向量模型(模型或维度)时先显示切换计划,需 --yes 确认后重新生成全部向量,switch 查看或取消、spaces 查看或清理旧向量数据;remove 不可逆,需 --yes。<profile-id>/<operation-id> 都是 UUID;脚本模式的凭据只从受控文件读取。",
|
||||
usage:
|
||||
"memhub models list [--json]\n memhub models show <profile-id> [--json]\n memhub models add --config-file <path> [--api-key-file <path>] [--activate] [--json]\n memhub models test <profile-id> [--json]\n memhub models receipt <operation-id> [--json]\n memhub models edit <profile-id> --config-file <path> [--api-key-file <path>] [--json]\n memhub models activate <profile-id> [--json]\n memhub models remove <profile-id> [--yes] [--json]",
|
||||
"memhub models list [--json]\n memhub models show <profile-id> [--json]\n memhub models add --config-file <path> [--api-key-file <path>] [--activate [--yes] [--wait|--no-wait]] [--json]\n memhub models test <profile-id> [--json]\n memhub models receipt <operation-id> [--json]\n memhub models edit <profile-id> --config-file <path> [--api-key-file <path>] [--json]\n memhub models activate <profile-id> [--yes] [--wait|--no-wait] [--json]\n memhub models switch <status|cancel> [--json]\n memhub models spaces [--cleanup <space-id> --yes] [--json]\n memhub models remove <profile-id> [--yes] [--json]",
|
||||
alternative_usage: "memhub models setup",
|
||||
options: [
|
||||
{
|
||||
@@ -377,7 +377,17 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
},
|
||||
{
|
||||
flag: "--activate",
|
||||
summary: "add 成功后把新配置设为当前使用。",
|
||||
summary:
|
||||
"add 成功后先探测,再按 activate 的规则设为当前使用(向量模型变化时同样要确认切换)。",
|
||||
},
|
||||
{
|
||||
flag: "--wait | --no-wait",
|
||||
summary:
|
||||
"activate 开始向量模型切换后是否跟随进度;终端里默认跟随,Ctrl-C 只停止显示。",
|
||||
},
|
||||
{
|
||||
flag: "--cleanup <space-id>",
|
||||
summary: "spaces:立即删除一个已停用的向量空间(不可恢复,需 --yes)。",
|
||||
},
|
||||
DESTRUCTIVE_CONFIRM_OPTION,
|
||||
JSON_OPTION,
|
||||
@@ -451,12 +461,13 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
{
|
||||
name: "settings",
|
||||
summary:
|
||||
"查看或切换记忆模式(lite / pro / ultra)。lite 不调用模型,只在原始对话里检索;pro 由模型提取记忆(System 1);ultra 在 pro 之上每晚自动进化(System 2,使用知识图谱)。任何模式都保留全部原始对话。切换不迁移数据:已处理的不会重做,排队中的和新的按新模式处理;需 --yes,不加时只显示计划。环境变量 ONE_MEMORY_GRAPH_STORE 会强制 ultra。settings features 查看或开关 SDK 的可选检索与提取功能(全局设置,默认全部保持 SDK 默认值):graph_rerank 图谱重排(仅 ultra)、same_series_timeline 同系列时间线(需 graph_rerank)、episode_recall 片段召回(pro / ultra,需先开 session_episodes 写入片段)、session_episodes 片段写入、episode_summary 片段摘要(每个片段一次模型调用)、intention 意图提取(默认开)、few_shot 提取示例;set 需 --yes,不加时只显示计划;reset 恢复默认值。",
|
||||
"查看或切换记忆模式(lite / pro / ultra)。lite 不调用模型,只在原始对话里检索;pro 由模型提取记忆(System 1);ultra 在 pro 之上每晚自动进化(System 2,使用知识图谱)。任何模式都保留全部原始对话。切换不迁移数据:已处理的不会重做,排队中的和新的按新模式处理;需 --yes,不加时只显示计划。环境变量 ONE_MEMORY_GRAPH_STORE 会强制 ultra。settings features 查看或开关 SDK 的可选检索与提取功能(全局设置,默认全部保持 SDK 默认值):graph_rerank 图谱重排(仅 ultra)、same_series_timeline 同系列时间线(需 graph_rerank)、episode_recall 片段召回(pro / ultra,需先开 session_episodes 写入片段)、session_episodes 片段写入、episode_summary 片段摘要(每个片段一次模型调用)、intention 意图提取(默认开)、few_shot 提取示例;set 需 --yes,不加时只显示计划;reset 恢复默认值。settings language 查看或设置界面语言(App 切换语言时会自动设置):简体中文(zh-Hans)和 system 不指定 SDK 语言,SDK 照旧自动识别中文或英文;其余语言(en、zh-Hant、ja、es、de、pt、fr、ko、ru、id)会让提取等 SDK 提示使用该语言,立即生效。",
|
||||
usage:
|
||||
"memhub settings mode [lite|pro|ultra] [--yes] [--json]\n" +
|
||||
" memhub settings features [--json]\n" +
|
||||
" memhub settings features set <name>=<true|false> [...] [--yes] [--json]\n" +
|
||||
" memhub settings features reset [<name> ...] [--json]",
|
||||
" memhub settings features reset [<name> ...] [--json]\n" +
|
||||
" memhub settings language [<code>|system] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "mode [lite|pro|ultra]",
|
||||
@@ -471,6 +482,11 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
flag: "features reset [<name> ...]",
|
||||
summary: "把指定功能(不写则全部)恢复为 SDK 默认值。",
|
||||
},
|
||||
{
|
||||
flag: "language [<code>|system]",
|
||||
summary:
|
||||
"不带参数显示界面语言和 SDK 提示语言;带语言代码(en、zh-Hans、zh-Hant、ja、es、de、pt、fr、ko、ru、id)或 system 时设置。",
|
||||
},
|
||||
CONFIRM_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
|
||||
@@ -635,6 +635,8 @@ describe("documented commands", () => {
|
||||
"receipt",
|
||||
"edit",
|
||||
"activate",
|
||||
"switch",
|
||||
"spaces",
|
||||
"remove",
|
||||
],
|
||||
// Source processing is one Freeze/Thaw switch under `sync`; the obsolete
|
||||
|
||||
@@ -97,7 +97,12 @@ export function contextExitCode(document: ContextSearchDocument): number {
|
||||
}
|
||||
|
||||
export function renderContextText(document: ContextSearchDocument): string {
|
||||
if (document.state === "unavailable") return "暂时无法查记忆,请稍后重试。\n";
|
||||
if (document.state === "unavailable") {
|
||||
// No lexical fallback: without the embedding service nothing is searched.
|
||||
return document.code === "embedding_unavailable"
|
||||
? "向量服务不可用,无法检索;请检查向量模型配置。\n"
|
||||
: "暂时无法查记忆,请稍后重试。\n";
|
||||
}
|
||||
if (document.state === "denied") return "当前应用没有查记忆权限。\n";
|
||||
if (document.state === "invalid") return "查记忆返回了无法识别的结果。\n";
|
||||
if (document.state === "rejected") {
|
||||
|
||||
@@ -8,6 +8,9 @@ import axios from "axios";
|
||||
*/
|
||||
const READ_ONLY_PATHS = [
|
||||
"/api/v1/call-events",
|
||||
// Read-only: the registry plus a scan of the vector store's collection names.
|
||||
"/api/v1/embedding-spaces",
|
||||
"/api/v1/embedding-switches/current",
|
||||
"/api/v1/import-jobs",
|
||||
"/api/v1/live-sync",
|
||||
"/api/v1/memory-processing",
|
||||
@@ -20,6 +23,7 @@ const READ_ONLY_PATHS = [
|
||||
"/api/v2/user-profile/versions",
|
||||
"/api/v1/settings/memory-mode",
|
||||
"/api/v1/settings/memory-features",
|
||||
"/api/v1/settings/ui-language",
|
||||
"/api/v2/evolve",
|
||||
] as const;
|
||||
/** One evolve run's receipt (`memhub evolve --wait`). */
|
||||
@@ -163,10 +167,23 @@ export function createCoreReader(attachment: {
|
||||
code: detailCode(response.data, response.status),
|
||||
};
|
||||
}
|
||||
return { state: "unavailable" as const, code: `http_${response.status}` };
|
||||
return {
|
||||
state: "unavailable" as const,
|
||||
code: unavailableCode(response.data, response.status),
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
/** A 503 that names `embedding_unavailable` keeps it: the search could not
|
||||
* embed its query, which needs another fix than "try again later". */
|
||||
function unavailableCode(value: unknown, status: number): string {
|
||||
return status === 503 && detailCode(value, status) === EMBEDDING_UNAVAILABLE
|
||||
? EMBEDDING_UNAVAILABLE
|
||||
: `http_${status}`;
|
||||
}
|
||||
|
||||
export const EMBEDDING_UNAVAILABLE = "embedding_unavailable";
|
||||
|
||||
/** Core's stable `detail` code when it sent one, else the bare status. */
|
||||
function detailCode(value: unknown, status: number): string {
|
||||
const detail =
|
||||
@@ -218,7 +235,10 @@ export function createCoreContextQuery(attachment: {
|
||||
if (response.status === 404) {
|
||||
return { state: "not_found" as const, code: "http_404" };
|
||||
}
|
||||
return { state: "unavailable" as const, code: `http_${response.status}` };
|
||||
return {
|
||||
state: "unavailable" as const,
|
||||
code: unavailableCode(response.data, response.status),
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -31,12 +31,29 @@ const EVOLVE_PATH = "/api/v2/evolve";
|
||||
const MEMORY_MODE_PATH = "/api/v1/settings/memory-mode";
|
||||
/** `memhub settings features set|reset` sets the SDK features. */
|
||||
const MEMORY_FEATURES_PATH = "/api/v1/settings/memory-features";
|
||||
/** `memhub settings language <code>` sets the display language Core uses. */
|
||||
const UI_LANGUAGE_PATH = "/api/v1/settings/ui-language";
|
||||
const ONBOARDING_COMPLETE_PATH = "/api/v1/onboarding/complete";
|
||||
const EMBEDDING_SWITCH_PATHS = [
|
||||
"/api/v1/embedding-switches",
|
||||
"/api/v1/embedding-switches/plan",
|
||||
] as const;
|
||||
const EMBEDDING_SWITCH_CANCEL =
|
||||
/^\/api\/v1\/embedding-switches\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\/cancel$/;
|
||||
const EMBEDDING_SPACE_PATH =
|
||||
/^\/api\/v1\/embedding-spaces\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
const MAX_BYTES = 4 * 1024 * 1024;
|
||||
|
||||
export type OperationOutcome<T> =
|
||||
| { state: "committed"; data: T }
|
||||
| { state: "rejected"; code: string; http_status: number }
|
||||
| {
|
||||
state: "rejected";
|
||||
code: string;
|
||||
http_status: number;
|
||||
/** Core's refusal body, when it carries more than the code (a switch
|
||||
* plan rides along with `embedding_switch_required`). */
|
||||
body?: unknown;
|
||||
}
|
||||
| { state: "unknown"; code: string };
|
||||
|
||||
export interface CoreOperations {
|
||||
@@ -104,6 +121,8 @@ export function createCoreOperations(attachment: {
|
||||
path !== GRAPH_WIDTH_PATH &&
|
||||
path !== EVOLVE_PATH &&
|
||||
path !== ONBOARDING_COMPLETE_PATH &&
|
||||
!(EMBEDDING_SWITCH_PATHS as readonly string[]).includes(path) &&
|
||||
!EMBEDDING_SWITCH_CANCEL.test(path) &&
|
||||
!MODEL_PROFILE_ACTIVATE.test(path)
|
||||
) {
|
||||
throw new Error("core_operation_path_not_allowed");
|
||||
@@ -121,11 +140,18 @@ export function createCoreOperations(attachment: {
|
||||
if (response.status >= 200 && response.status < 300) {
|
||||
return { state: "committed", data: response.data as T };
|
||||
}
|
||||
if (response.status >= 400 && response.status < 500) {
|
||||
if (
|
||||
(response.status >= 400 && response.status < 500) ||
|
||||
// Core answers a refusal it can name with 503 (`sdk_reembed_unavailable`)
|
||||
// or 507 (`insufficient_disk_space`) on the switch routes.
|
||||
([503, 507].includes(response.status) &&
|
||||
(EMBEDDING_SWITCH_PATHS as readonly string[]).includes(path))
|
||||
) {
|
||||
return {
|
||||
state: "rejected",
|
||||
code: safeErrorCode(response.data, response.status),
|
||||
http_status: response.status,
|
||||
body: response.data,
|
||||
};
|
||||
}
|
||||
return {
|
||||
@@ -141,6 +167,7 @@ export function createCoreOperations(attachment: {
|
||||
if (
|
||||
path !== MEMORY_MODE_PATH &&
|
||||
path !== MEMORY_FEATURES_PATH &&
|
||||
path !== UI_LANGUAGE_PATH &&
|
||||
(!SDK_MEMORY_PATH.test(path) || path.includes("?"))
|
||||
) {
|
||||
throw new Error("core_operation_path_not_allowed");
|
||||
@@ -204,7 +231,11 @@ export function createCoreOperations(attachment: {
|
||||
};
|
||||
},
|
||||
async delete<T>(path: string, signal?: AbortSignal) {
|
||||
if (!SDK_MEMORY_PATH.test(path) && !MODEL_PROFILE_PATH.test(path)) {
|
||||
if (
|
||||
!SDK_MEMORY_PATH.test(path) &&
|
||||
!MODEL_PROFILE_PATH.test(path) &&
|
||||
!EMBEDDING_SPACE_PATH.test(path)
|
||||
) {
|
||||
throw new Error("core_operation_path_not_allowed");
|
||||
}
|
||||
let response: { status: number; data: unknown };
|
||||
|
||||
@@ -0,0 +1,488 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { run } from "./cli.js";
|
||||
import { renderContextText } from "./context-search.js";
|
||||
import { describeEmbeddingSwitchSummary } from "./embedding-switch-flow.js";
|
||||
import { composeStatusReport, renderStatusText } from "./status-report.js";
|
||||
|
||||
const transport = vi.hoisted(() => ({
|
||||
get: vi.fn(),
|
||||
post: vi.fn(),
|
||||
put: vi.fn(),
|
||||
patch: vi.fn(),
|
||||
delete: vi.fn(),
|
||||
}));
|
||||
vi.mock("axios", () => ({
|
||||
default: { create: vi.fn(() => transport), get: vi.fn() },
|
||||
}));
|
||||
|
||||
const runtime = vi.hoisted(() => ({
|
||||
inspectRuntimeStatus: vi.fn(),
|
||||
discoverRuntimeAttachment: vi.fn(),
|
||||
}));
|
||||
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
|
||||
...runtime,
|
||||
}));
|
||||
|
||||
const PROFILE = "44444444-4444-4444-8444-444444444444";
|
||||
const JOB = "55555555-5555-4555-8555-555555555555";
|
||||
const SPACE = "66666666-6666-4666-8666-666666666666";
|
||||
|
||||
function plan(overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
needed: true,
|
||||
reason: "embedding_identity_changed",
|
||||
plan_token: "token-1",
|
||||
source: {
|
||||
space_id: "77777777-7777-4777-8777-777777777777",
|
||||
model: "bge-small-zh-v1.5",
|
||||
dims: 512,
|
||||
collection_base: "memhub_aaaaaaaaaaaaaaaa",
|
||||
vectors: 12480,
|
||||
},
|
||||
target: {
|
||||
space_id: null,
|
||||
model: "text-embedding-v3",
|
||||
dims: 1024,
|
||||
collection_base: "",
|
||||
},
|
||||
target_profile_id: PROFILE,
|
||||
target_revision: 5,
|
||||
target_provider_kind: "openai_compatible",
|
||||
target_local: false,
|
||||
sources: [],
|
||||
total_vectors: 12480,
|
||||
users: 2,
|
||||
graph_vectors: null,
|
||||
estimate_s: 120,
|
||||
requests: 195,
|
||||
disk_bytes_needed: 70 * 1024 * 1024,
|
||||
disk_bytes_free: 10 * 1024 * 1024 * 1024,
|
||||
retention_days: 14,
|
||||
blockers: [],
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function job(
|
||||
state: string,
|
||||
percent: number,
|
||||
extra: Record<string, unknown> = {},
|
||||
) {
|
||||
return {
|
||||
job_id: JOB,
|
||||
kind: "switch",
|
||||
state,
|
||||
source: plan().source,
|
||||
target: { ...plan().target, space_id: SPACE },
|
||||
stray_space_ids: [],
|
||||
target_profile_id: PROFILE,
|
||||
target_revision: 5,
|
||||
requested_by: "cli",
|
||||
progress: {
|
||||
phase: state === "reembedding" ? "vectors" : "prepare",
|
||||
done: Math.round((12480 * percent) / 100),
|
||||
total: 12480,
|
||||
percent,
|
||||
eta_s: 60,
|
||||
},
|
||||
cancel_requested: false,
|
||||
error_code: null,
|
||||
attempt: 1,
|
||||
created_at: "T1",
|
||||
updated_at: "T1",
|
||||
finished_at: state === "done" ? "T2" : null,
|
||||
...extra,
|
||||
};
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
for (const method of Object.values(transport)) method.mockReset();
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "running",
|
||||
code: "runtime_verified",
|
||||
instanceId: "instance-1",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
version: "1.2.3",
|
||||
});
|
||||
runtime.discoverRuntimeAttachment.mockResolvedValue({
|
||||
instanceId: "instance-1",
|
||||
apiToken: "synthetic-runtime-token",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
servicePid: 4242,
|
||||
health: {
|
||||
status: "ok",
|
||||
service: "one-memory-core",
|
||||
version: "1.2.3",
|
||||
instance_id: "instance-1",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
describe("models activate with an embedding change", () => {
|
||||
it("prints the plan and exits 2 without --yes on a non-terminal", async () => {
|
||||
transport.post.mockImplementation(async (url: string) =>
|
||||
url === "/api/v1/embedding-switches/plan"
|
||||
? { status: 200, data: plan() }
|
||||
: { status: 404, data: {} },
|
||||
);
|
||||
const result = await capturedRun(["models", "activate", PROFILE]);
|
||||
expect(result.code).toBe(2);
|
||||
expect(result.stdout).toContain("将更换向量模型");
|
||||
expect(result.stdout).toContain("bge-small-zh-v1.5(512 维)");
|
||||
expect(result.stdout).toContain("text-embedding-v3(1024 维)");
|
||||
expect(result.stdout).toContain("12,480");
|
||||
expect(result.stdout).toContain("重新运行并加上 --yes 以确认");
|
||||
expect(transport.post.mock.calls.map(([url]) => url)).toEqual([
|
||||
"/api/v1/embedding-switches/plan",
|
||||
]);
|
||||
});
|
||||
|
||||
it("starts the switch with --yes and emits JSON progress lines", async () => {
|
||||
transport.post.mockImplementation(async (url: string) => {
|
||||
if (url === "/api/v1/embedding-switches/plan") {
|
||||
return { status: 200, data: plan() };
|
||||
}
|
||||
if (url === "/api/v1/embedding-switches") {
|
||||
return { status: 202, data: job("preparing", 0) };
|
||||
}
|
||||
return { status: 404, data: {} };
|
||||
});
|
||||
const answers = [
|
||||
{
|
||||
phase: "running",
|
||||
writes_paused: true,
|
||||
job: job("reembedding", 50),
|
||||
last_finished: null,
|
||||
},
|
||||
{
|
||||
phase: "idle",
|
||||
writes_paused: false,
|
||||
job: null,
|
||||
last_finished: job("done", 100),
|
||||
},
|
||||
];
|
||||
transport.get.mockImplementation(async (url: string) =>
|
||||
url === "/api/v1/embedding-switches/current"
|
||||
? { status: 200, data: { schema_version: 1, ...answers.shift() } }
|
||||
: { status: 404, data: {} },
|
||||
);
|
||||
const result = await capturedRun(
|
||||
["models", "activate", PROFILE, "--yes", "--wait", "--json"],
|
||||
{ watchWait: async () => undefined },
|
||||
);
|
||||
expect(result.code).toBe(0);
|
||||
const lines = result.stdout
|
||||
.trim()
|
||||
.split("\n")
|
||||
.map((line) => JSON.parse(line));
|
||||
expect(lines.map((line) => line.state)).toEqual([
|
||||
"preparing",
|
||||
"reembedding",
|
||||
"done",
|
||||
"done",
|
||||
]);
|
||||
expect(lines[1]).toMatchObject({ percent: 50, job_id: JOB });
|
||||
expect(lines.at(-1)).toMatchObject({ result: "switched" });
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
"/api/v1/embedding-switches",
|
||||
{
|
||||
profile_id: PROFILE,
|
||||
expected_revision: 5,
|
||||
plan_token: "token-1",
|
||||
requested_by: "cli",
|
||||
},
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
it("refuses a plan with blockers and starts nothing", async () => {
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: plan({ blockers: ["sdk_reembed_unavailable"] }),
|
||||
});
|
||||
const result = await capturedRun([
|
||||
"models",
|
||||
"activate",
|
||||
PROFILE,
|
||||
"--yes",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
state: "blocked",
|
||||
code: "sdk_reembed_unavailable",
|
||||
});
|
||||
expect(transport.post).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("says nothing is re-embedded when only generation changed", async () => {
|
||||
transport.post.mockImplementation(async (url: string) =>
|
||||
url === "/api/v1/embedding-switches/plan"
|
||||
? {
|
||||
status: 200,
|
||||
data: plan({ needed: false, reason: "same_embedding" }),
|
||||
}
|
||||
: { status: 200, data: profileResponse() },
|
||||
);
|
||||
const result = await capturedRun(["models", "activate", PROFILE]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(result.stdout).toContain("向量模型未变化,无需重新生成向量。");
|
||||
});
|
||||
|
||||
it("reads the plan carried by a 409 embedding_switch_required", async () => {
|
||||
transport.post.mockImplementation(async (url: string) =>
|
||||
url.endsWith("/activate")
|
||||
? {
|
||||
status: 409,
|
||||
data: { detail: "embedding_switch_required", plan: plan() },
|
||||
}
|
||||
: { status: 404, data: { detail: "Not Found" } },
|
||||
);
|
||||
const result = await capturedRun(["models", "activate", PROFILE, "--json"]);
|
||||
expect(result.code).toBe(2);
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
state: "confirmation_required",
|
||||
code: "embedding_switch_required",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("models switch and spaces", () => {
|
||||
it("reports no switch, then cancels the running one", async () => {
|
||||
transport.get.mockResolvedValueOnce({
|
||||
status: 200,
|
||||
data: {
|
||||
schema_version: 1,
|
||||
phase: "idle",
|
||||
writes_paused: false,
|
||||
job: null,
|
||||
last_finished: null,
|
||||
},
|
||||
});
|
||||
const idle = await capturedRun(["models", "switch", "status"]);
|
||||
expect(idle.code).toBe(0);
|
||||
expect(idle.stdout).toContain("没有进行中的向量模型切换");
|
||||
|
||||
transport.get.mockResolvedValueOnce({
|
||||
status: 200,
|
||||
data: {
|
||||
schema_version: 1,
|
||||
phase: "running",
|
||||
writes_paused: true,
|
||||
job: job("reembedding", 20),
|
||||
last_finished: null,
|
||||
},
|
||||
});
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: job("reembedding", 20, { cancel_requested: true }),
|
||||
});
|
||||
const cancelled = await capturedRun([
|
||||
"models",
|
||||
"switch",
|
||||
"cancel",
|
||||
"--json",
|
||||
]);
|
||||
expect(cancelled.code).toBe(0);
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
`/api/v1/embedding-switches/${JOB}/cancel`,
|
||||
{},
|
||||
undefined,
|
||||
);
|
||||
expect(JSON.parse(cancelled.stdout)).toMatchObject({
|
||||
cancel_requested: true,
|
||||
});
|
||||
});
|
||||
|
||||
it("lists spaces and needs --yes before cleaning one up", async () => {
|
||||
transport.get.mockResolvedValue({
|
||||
status: 200,
|
||||
data: {
|
||||
schema_version: 1,
|
||||
live_space_id: "x",
|
||||
spaces: [
|
||||
{
|
||||
space_id: SPACE,
|
||||
state: "retired",
|
||||
model: "bge-small-zh-v1.5",
|
||||
dims: 512,
|
||||
vectors: 10,
|
||||
keep_until: "2026-11-15T00:00:00+00:00",
|
||||
},
|
||||
{
|
||||
space_id: "88888888-8888-4888-8888-888888888888",
|
||||
state: "dropped",
|
||||
model: "m",
|
||||
dims: 8,
|
||||
},
|
||||
],
|
||||
scanned: [],
|
||||
scan_error: null,
|
||||
},
|
||||
});
|
||||
const listed = await capturedRun(["models", "spaces"]);
|
||||
expect(listed.code).toBe(0);
|
||||
expect(listed.stdout).toContain("已停用");
|
||||
expect(listed.stdout).toContain("保留到 2026-11-15");
|
||||
expect(listed.stdout).not.toContain("88888888");
|
||||
|
||||
const preview = await capturedRun(["models", "spaces", "--cleanup", SPACE]);
|
||||
expect(preview.code).toBe(0);
|
||||
expect(preview.stdout).toContain("--yes");
|
||||
expect(transport.delete).not.toHaveBeenCalled();
|
||||
|
||||
transport.delete.mockResolvedValue({
|
||||
status: 200,
|
||||
data: { schema_version: 1 },
|
||||
});
|
||||
const dropped = await capturedRun([
|
||||
"models",
|
||||
"spaces",
|
||||
"--cleanup",
|
||||
SPACE,
|
||||
"--yes",
|
||||
]);
|
||||
expect(dropped.code).toBe(0);
|
||||
expect(transport.delete).toHaveBeenCalledWith(
|
||||
`/api/v1/embedding-spaces/${SPACE}`,
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("status and search wording", () => {
|
||||
it("shows the switch line in memhub status", async () => {
|
||||
const reader = async (path: string) =>
|
||||
path === "/api/v1/memory-processing"
|
||||
? {
|
||||
state: "ok" as const,
|
||||
data: {
|
||||
pending: 3,
|
||||
fresh_pending: 1,
|
||||
backlog_pending: 2,
|
||||
outside_window_pending: 0,
|
||||
running: 0,
|
||||
waiting_for_model: 0,
|
||||
completed: 9,
|
||||
failed: 0,
|
||||
embedding_switch: {
|
||||
job_id: JOB,
|
||||
kind: "switch",
|
||||
state: "reembedding",
|
||||
percent: 50,
|
||||
eta_s: 60,
|
||||
},
|
||||
},
|
||||
}
|
||||
: { state: "unavailable" as const, code: "http_503" };
|
||||
const report = await composeStatusReport({
|
||||
dataDir: "/synthetic/data",
|
||||
runtime: {
|
||||
state: "running",
|
||||
code: "runtime_verified",
|
||||
instanceId: "instance-1",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
} as never,
|
||||
background: null,
|
||||
reader: reader as never,
|
||||
});
|
||||
const text = renderStatusText(report, "MemHub");
|
||||
expect(text).toContain("向量模型:切换中 50%,记忆整理已暂停");
|
||||
expect(
|
||||
describeEmbeddingSwitchSummary({
|
||||
kind: "merge",
|
||||
state: "reembedding",
|
||||
percent: 30,
|
||||
}),
|
||||
).toBe("正在合并旧向量数据 30%,记忆整理已暂停");
|
||||
expect(
|
||||
describeEmbeddingSwitchSummary({
|
||||
kind: "merge",
|
||||
state: "scanning",
|
||||
percent: 0,
|
||||
}),
|
||||
).toBe("正在检查向量数据…");
|
||||
});
|
||||
|
||||
it("names an unavailable embedding service instead of 'try later'", () => {
|
||||
expect(
|
||||
renderContextText({
|
||||
schema_version: 2,
|
||||
product: "memhub",
|
||||
state: "unavailable",
|
||||
code: "embedding_unavailable",
|
||||
items: [],
|
||||
}),
|
||||
).toContain("向量服务不可用");
|
||||
});
|
||||
});
|
||||
|
||||
function profileResponse() {
|
||||
return {
|
||||
profile_id: PROFILE,
|
||||
name: "Synthetic",
|
||||
provider_kind: "openai_compatible",
|
||||
base_url: "https://example.invalid/v1",
|
||||
model: "synthetic-model-2",
|
||||
embedding_model: "text-embedding-v3",
|
||||
embedding_provider_kind: "openai_compatible",
|
||||
embedding_base_url: "https://example.invalid/v1",
|
||||
embedding_dimensions: 1024,
|
||||
has_api_key: true,
|
||||
credential_state: "available",
|
||||
timeout_seconds: 30,
|
||||
max_concurrency: 1,
|
||||
context_window: null,
|
||||
disable_thinking: true,
|
||||
capabilities: {
|
||||
endpoint_reachable: true,
|
||||
model_available: true,
|
||||
structured_output: true,
|
||||
embedding_available: true,
|
||||
},
|
||||
revision: 5,
|
||||
is_active: true,
|
||||
created_at: "2026-09-19T00:00:00Z",
|
||||
updated_at: "2026-09-19T00:00:00Z",
|
||||
};
|
||||
}
|
||||
|
||||
async function capturedRun(
|
||||
arguments_: string[],
|
||||
controls: Parameters<typeof run>[2] = {},
|
||||
) {
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
const stdoutSpy = vi
|
||||
.spyOn(process.stdout, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stdout += String(chunk);
|
||||
return true;
|
||||
});
|
||||
const stderrSpy = vi
|
||||
.spyOn(process.stderr, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stderr += String(chunk);
|
||||
return true;
|
||||
});
|
||||
try {
|
||||
return {
|
||||
code: await run(
|
||||
arguments_,
|
||||
{ ONE_MEMORY_DATA_DIR: "/synthetic/data" },
|
||||
{ stdinIsTTY: false, ...controls },
|
||||
),
|
||||
stdout,
|
||||
stderr,
|
||||
};
|
||||
} finally {
|
||||
stdoutSpy.mockRestore();
|
||||
stderrSpy.mockRestore();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,847 @@
|
||||
/**
|
||||
* Switching the embedding model from the CLI: `models activate` when the
|
||||
* embedding changes, `models switch status|cancel`, `models spaces`.
|
||||
*
|
||||
* Core owns every decision (`/api/v1/embedding-switches*`, contract
|
||||
* `docs/contracts/embedding-switch-v1.md`). A change of embedding model or
|
||||
* width is never applied by activation alone: Core plans a switch, the user
|
||||
* confirms it, Core re-embeds every memory into a new space and only then
|
||||
* makes it live. The CLI shows the plan, asks, starts the job and follows it.
|
||||
*/
|
||||
|
||||
import type { CoreReader } from "./core-client.js";
|
||||
import type {
|
||||
CoreDeleteOperations,
|
||||
CoreOperations,
|
||||
OperationOutcome,
|
||||
} from "./core-operations.js";
|
||||
import { EXIT_FAILED, EXIT_OK, EXIT_USAGE } from "./exit-codes.js";
|
||||
import {
|
||||
activateModelProfile,
|
||||
type ModelWriteDocument,
|
||||
modelProfileResult,
|
||||
renderModelWriteText,
|
||||
} from "./model-write-flows.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
const CODE = /^[a-z0-9_.:-]{1,128}$/;
|
||||
const TERMINAL = new Set(["done", "rolled_back"]);
|
||||
const POLL_MS = 1_000;
|
||||
|
||||
export interface SpaceSummary {
|
||||
space_id: string | null;
|
||||
model: string | null;
|
||||
dims: number | null;
|
||||
collection_base: string;
|
||||
vectors?: number | null;
|
||||
}
|
||||
|
||||
export interface EmbeddingSwitchPlan {
|
||||
needed: boolean;
|
||||
reason: string;
|
||||
plan_token: string;
|
||||
source: SpaceSummary;
|
||||
target: SpaceSummary;
|
||||
target_profile_id: string;
|
||||
target_revision: number;
|
||||
target_provider_kind: string;
|
||||
target_local: boolean;
|
||||
total_vectors: number;
|
||||
users: number | null;
|
||||
graph_vectors: number | null;
|
||||
estimate_s: number | null;
|
||||
requests: number | null;
|
||||
disk_bytes_needed: number;
|
||||
disk_bytes_free: number | null;
|
||||
retention_days: number;
|
||||
blockers: string[];
|
||||
}
|
||||
|
||||
export interface SwitchProgress {
|
||||
phase: string;
|
||||
done: number;
|
||||
total: number;
|
||||
percent: number;
|
||||
eta_s: number | null;
|
||||
}
|
||||
|
||||
export interface EmbeddingSwitchJob {
|
||||
job_id: string;
|
||||
kind: "switch" | "merge";
|
||||
state: string;
|
||||
source: SpaceSummary;
|
||||
target: SpaceSummary;
|
||||
progress: SwitchProgress;
|
||||
cancel_requested: boolean;
|
||||
error_code: string | null;
|
||||
finished_at: string | null;
|
||||
}
|
||||
|
||||
export interface SwitchCurrent {
|
||||
phase: "idle" | "scanning" | "running";
|
||||
writes_paused: boolean;
|
||||
job: EmbeddingSwitchJob | null;
|
||||
last_finished: EmbeddingSwitchJob | null;
|
||||
}
|
||||
|
||||
export interface SwitchIo {
|
||||
reader: CoreReader;
|
||||
operations: CoreOperations & CoreDeleteOperations;
|
||||
emit: (text: string) => void;
|
||||
asJson: boolean;
|
||||
/** A terminal is attached: may prompt, and follows progress by default. */
|
||||
interactive: boolean;
|
||||
confirm?: (question: string) => Promise<boolean>;
|
||||
wait?: (milliseconds: number, signal: AbortSignal) => Promise<void>;
|
||||
signal?: AbortSignal;
|
||||
}
|
||||
|
||||
export interface ActivateOptions {
|
||||
confirmed: boolean;
|
||||
/** `--wait` / `--no-wait`; undefined follows on a terminal only. */
|
||||
follow?: boolean;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Parsing: bounded projections of Core's answers, never passthrough.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function record(value: unknown): Record<string, unknown> | null {
|
||||
return value && typeof value === "object" && !Array.isArray(value)
|
||||
? (value as Record<string, unknown>)
|
||||
: null;
|
||||
}
|
||||
|
||||
function int(value: unknown): number | null {
|
||||
return typeof value === "number" && Number.isSafeInteger(value) && value >= 0
|
||||
? value
|
||||
: null;
|
||||
}
|
||||
|
||||
function str(value: unknown, max = 256): string | null {
|
||||
return typeof value === "string" && value.length <= max ? value : null;
|
||||
}
|
||||
|
||||
function space(value: unknown): SpaceSummary | null {
|
||||
const item = record(value);
|
||||
if (!item) return null;
|
||||
return {
|
||||
space_id: str(item.space_id, 64),
|
||||
model: str(item.model),
|
||||
dims: int(item.dims),
|
||||
collection_base: str(item.collection_base) ?? "",
|
||||
vectors: int(item.vectors),
|
||||
};
|
||||
}
|
||||
|
||||
export function parsePlan(value: unknown): EmbeddingSwitchPlan | null {
|
||||
const plan = record(value);
|
||||
if (!plan || typeof plan.needed !== "boolean") return null;
|
||||
const source = space(plan.source);
|
||||
const target = space(plan.target);
|
||||
const token = str(plan.plan_token, 128);
|
||||
const profile = str(plan.target_profile_id, 128);
|
||||
const revision = int(plan.target_revision);
|
||||
if (!source || !target || !token || !profile || revision === null) {
|
||||
return null;
|
||||
}
|
||||
const blockers = Array.isArray(plan.blockers)
|
||||
? plan.blockers.filter(
|
||||
(item): item is string => typeof item === "string" && CODE.test(item),
|
||||
)
|
||||
: [];
|
||||
return {
|
||||
needed: plan.needed,
|
||||
reason: str(plan.reason, 64) ?? "",
|
||||
plan_token: token,
|
||||
source,
|
||||
target,
|
||||
target_profile_id: profile,
|
||||
target_revision: revision,
|
||||
target_provider_kind: str(plan.target_provider_kind, 64) ?? "",
|
||||
target_local: plan.target_local === true,
|
||||
total_vectors: int(plan.total_vectors) ?? 0,
|
||||
users: int(plan.users),
|
||||
graph_vectors: int(plan.graph_vectors),
|
||||
estimate_s: int(plan.estimate_s),
|
||||
requests: int(plan.requests),
|
||||
disk_bytes_needed: int(plan.disk_bytes_needed) ?? 0,
|
||||
disk_bytes_free: int(plan.disk_bytes_free),
|
||||
retention_days: int(plan.retention_days) ?? 14,
|
||||
blockers,
|
||||
};
|
||||
}
|
||||
|
||||
export function parseJob(value: unknown): EmbeddingSwitchJob | null {
|
||||
const job = record(value);
|
||||
if (!job) return null;
|
||||
const jobId = str(job.job_id, 64);
|
||||
const state = str(job.state, 32);
|
||||
const source = space(job.source);
|
||||
const target = space(job.target);
|
||||
const progress = record(job.progress) ?? {};
|
||||
if (!jobId || !state || !source || !target) return null;
|
||||
if (job.kind !== "switch" && job.kind !== "merge") return null;
|
||||
const percent =
|
||||
typeof progress.percent === "number" && Number.isFinite(progress.percent)
|
||||
? Math.min(100, Math.max(0, progress.percent))
|
||||
: 0;
|
||||
return {
|
||||
job_id: jobId,
|
||||
kind: job.kind,
|
||||
state,
|
||||
source,
|
||||
target,
|
||||
progress: {
|
||||
phase: str(progress.phase, 32) ?? "prepare",
|
||||
done: int(progress.done) ?? 0,
|
||||
total: int(progress.total) ?? 0,
|
||||
percent,
|
||||
eta_s: int(progress.eta_s),
|
||||
},
|
||||
cancel_requested: job.cancel_requested === true,
|
||||
error_code:
|
||||
typeof job.error_code === "string" && CODE.test(job.error_code)
|
||||
? job.error_code
|
||||
: null,
|
||||
finished_at: str(job.finished_at, 64),
|
||||
};
|
||||
}
|
||||
|
||||
export function parseCurrent(value: unknown): SwitchCurrent | null {
|
||||
const current = record(value);
|
||||
if (!current) return null;
|
||||
const phase = current.phase;
|
||||
if (phase !== "idle" && phase !== "scanning" && phase !== "running") {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
phase,
|
||||
writes_paused: current.writes_paused === true,
|
||||
job: current.job == null ? null : parseJob(current.job),
|
||||
last_finished:
|
||||
current.last_finished == null ? null : parseJob(current.last_finished),
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Wording
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const BLOCKER_TEXT: Record<string, string> = {
|
||||
embedding_switch_running: "上一次向量模型切换还在进行,请等待完成或先取消。",
|
||||
embedding_probe_required: "新配置还没有通过探测;先运行 memhub models test。",
|
||||
sdk_reembed_unavailable:
|
||||
"当前运行时的 SDK 不支持重新生成向量,请先升级 MemHub。",
|
||||
embedding_switch_requires_migration:
|
||||
"还有旧版本的向量数据没有迁移完,请先完成升级迁移。",
|
||||
embedding_dimensions_unknown:
|
||||
"新配置的向量维度未知;先探测或在配置里写明 embedding_dimensions。",
|
||||
insufficient_disk_space: "磁盘空间不足,无法容纳新旧两份向量数据。",
|
||||
};
|
||||
|
||||
const FAILURE_TEXT: Record<string, string> = {
|
||||
writes_did_not_drain: "等待正在进行的记忆整理结束超时",
|
||||
embedding_service_failed: "向量服务连续请求失败",
|
||||
embedding_dimensions_mismatch: "向量服务返回的维度与探测到的不一致",
|
||||
embedding_verification_failed: "新向量没有通过校验",
|
||||
target_profile_changed: "切换期间目标配置被修改",
|
||||
embedding_switch_cancelled: "已取消",
|
||||
model_secret_unavailable: "读不到向量服务的 API Key",
|
||||
secure_storage_unavailable: "系统钥匙串不可用",
|
||||
embedding_switch_clients_busy: "记忆图谱一直被占用,无法重新生成",
|
||||
};
|
||||
|
||||
const STATE_TEXT: Record<string, string> = {
|
||||
preparing: "准备中(等待记忆整理暂停)",
|
||||
reembedding: "重新生成向量",
|
||||
verifying: "校验新向量",
|
||||
committing: "正式切换",
|
||||
failed: "正在回滚",
|
||||
done: "已完成",
|
||||
rolled_back: "已回滚",
|
||||
};
|
||||
|
||||
export function blockerText(code: string): string {
|
||||
return BLOCKER_TEXT[code] ?? code;
|
||||
}
|
||||
|
||||
function spaceLabel(item: SpaceSummary, local?: boolean): string {
|
||||
const dims = item.dims === null ? "维度未知" : `${item.dims} 维`;
|
||||
return `${item.model ?? "未知模型"}(${dims}${local ? ",本地" : ""})`;
|
||||
}
|
||||
|
||||
function duration(seconds: number | null): string {
|
||||
if (seconds === null) return "未知";
|
||||
if (seconds < 60) return `${Math.max(1, seconds)} 秒`;
|
||||
return `${Math.round(seconds / 60)} 分钟`;
|
||||
}
|
||||
|
||||
function megabytes(bytes: number | null): string {
|
||||
return bytes === null ? "未知" : `${Math.ceil(bytes / (1024 * 1024))} MB`;
|
||||
}
|
||||
|
||||
export function renderPlanText(plan: EmbeddingSwitchPlan): string {
|
||||
const lines = [
|
||||
`将更换向量模型:${spaceLabel(plan.source)} → ${spaceLabel(plan.target, plan.target_local)}`,
|
||||
`需要用新模型重新生成 ${plan.total_vectors.toLocaleString("en-US")} 条向量` +
|
||||
`${plan.users === null ? "" : `(${plan.users} 个用户)`}` +
|
||||
`,预计约 ${duration(plan.estimate_s)}` +
|
||||
`${plan.requests === null ? ",在本机计算,不产生费用" : `,约 ${plan.requests} 次请求`}` +
|
||||
`,额外磁盘约 ${megabytes(plan.disk_bytes_needed)}。`,
|
||||
];
|
||||
if (plan.graph_vectors) {
|
||||
lines.push(`记忆图谱另有 ${plan.graph_vectors} 条向量一并重新生成。`);
|
||||
}
|
||||
if (
|
||||
plan.source.dims !== null &&
|
||||
plan.source.dims === plan.target.dims &&
|
||||
plan.source.model !== plan.target.model
|
||||
) {
|
||||
lines.push("新旧模型维度相同,但向量不能通用,仍需重新生成。");
|
||||
}
|
||||
lines.push(
|
||||
"切换期间记忆整理暂停,检索继续使用旧模型;完成并校验通过后才会生效。",
|
||||
`旧的向量数据保留 ${plan.retention_days} 天,可用 memhub models spaces 清理。`,
|
||||
);
|
||||
for (const blocker of plan.blockers) {
|
||||
lines.push(`无法开始:${blockerText(blocker)}`);
|
||||
}
|
||||
return `${lines.join("\n")}\n`;
|
||||
}
|
||||
|
||||
export function renderProgressLine(job: EmbeddingSwitchJob): string {
|
||||
const what = job.kind === "merge" ? "正在合并旧向量数据" : "正在切换向量模型";
|
||||
const state = STATE_TEXT[job.state] ?? job.state;
|
||||
if (job.state === "reembedding" && job.progress.total > 0) {
|
||||
const eta =
|
||||
job.progress.eta_s === null
|
||||
? ""
|
||||
: ` 预计还需 ${duration(job.progress.eta_s)}`;
|
||||
return `${what}:${state} ${job.progress.done.toLocaleString("en-US")}/${job.progress.total.toLocaleString("en-US")} ${Math.floor(job.progress.percent)}%${eta}`;
|
||||
}
|
||||
return `${what}:${state}`;
|
||||
}
|
||||
|
||||
export function renderOutcomeText(job: EmbeddingSwitchJob): string {
|
||||
if (job.state === "done") {
|
||||
return job.kind === "merge"
|
||||
? "旧向量数据已合并,记忆整理已恢复。\n"
|
||||
: `已切换到 ${job.target.model ?? "新模型"}。\n`;
|
||||
}
|
||||
const code = job.error_code ?? "embedding_switch_failed";
|
||||
if (code === "embedding_switch_cancelled") {
|
||||
return `已取消切换,继续使用 ${job.source.model ?? "旧模型"}。\n`;
|
||||
}
|
||||
return `切换未完成,已恢复使用 ${job.source.model ?? "旧模型"},数据未受影响。原因:${FAILURE_TEXT[code] ?? code}(${code})\n`;
|
||||
}
|
||||
|
||||
/** One line for `memhub status`, from `memory-processing.embedding_switch`. */
|
||||
export function describeEmbeddingSwitchSummary(value: unknown): string | null {
|
||||
const summary = record(value);
|
||||
if (!summary) return null;
|
||||
const percent =
|
||||
typeof summary.percent === "number" && Number.isFinite(summary.percent)
|
||||
? Math.floor(summary.percent)
|
||||
: 0;
|
||||
if (summary.state === "scanning") return "正在检查向量数据…";
|
||||
if (summary.kind === "merge") {
|
||||
return `正在合并旧向量数据 ${percent}%,记忆整理已暂停`;
|
||||
}
|
||||
if (summary.state === "failed") return "切换未完成,正在恢复旧向量模型";
|
||||
return `切换中 ${percent}%,记忆整理已暂停`;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Flows
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function document(
|
||||
command: string,
|
||||
fields: Record<string, unknown>,
|
||||
): Record<string, unknown> {
|
||||
return { schema_version: 1, product: PRODUCT, command, ...fields };
|
||||
}
|
||||
|
||||
function emitDocument(io: SwitchIo, value: Record<string, unknown>): void {
|
||||
io.emit(`${JSON.stringify(value)}\n`);
|
||||
}
|
||||
|
||||
async function requestPlan(
|
||||
profileId: string,
|
||||
io: SwitchIo,
|
||||
): Promise<
|
||||
| { state: "plan"; plan: EmbeddingSwitchPlan }
|
||||
| { state: "unsupported" }
|
||||
| { state: "failed"; code: string }
|
||||
> {
|
||||
const outcome = await io.operations.post<unknown>(
|
||||
"/api/v1/embedding-switches/plan",
|
||||
{ profile_id: profileId },
|
||||
);
|
||||
if (outcome.state === "committed") {
|
||||
const plan = parsePlan(outcome.data);
|
||||
// Not a plan: activation itself still refuses an embedding change.
|
||||
return plan ? { state: "plan", plan } : { state: "unsupported" };
|
||||
}
|
||||
if (outcome.state === "rejected") {
|
||||
// A Core before the registry has no plan route: activation as before.
|
||||
if (outcome.http_status === 404 && outcome.code === "http_404") {
|
||||
return { state: "unsupported" };
|
||||
}
|
||||
const plan = parsePlan(record(outcome.body)?.plan);
|
||||
if (plan) return { state: "plan", plan };
|
||||
return { state: "failed", code: outcome.code };
|
||||
}
|
||||
return { state: "failed", code: outcome.code };
|
||||
}
|
||||
|
||||
function emitActivation(io: SwitchIo, result: ModelWriteDocument): number {
|
||||
io.emit(
|
||||
io.asJson ? `${JSON.stringify(result)}\n` : renderModelWriteText(result),
|
||||
);
|
||||
return result.state === "committed" ? EXIT_OK : EXIT_FAILED;
|
||||
}
|
||||
|
||||
/**
|
||||
* `memhub models activate <id> [--yes] [--wait|--no-wait] [--json]`.
|
||||
*
|
||||
* Asks Core for the plan first. No embedding change: activate as before.
|
||||
* An embedding change: show the plan, confirm (`--yes`, or a prompt on a
|
||||
* terminal; exit 2 otherwise), start the switch and follow it.
|
||||
*/
|
||||
export async function activateWithEmbeddingSwitch(
|
||||
profileId: string,
|
||||
options: ActivateOptions,
|
||||
io: SwitchIo,
|
||||
): Promise<number> {
|
||||
if (!UUID.test(profileId)) {
|
||||
return emitActivation(
|
||||
io,
|
||||
await activateModelProfile(profileId, io.operations),
|
||||
);
|
||||
}
|
||||
const planned = await requestPlan(profileId, io);
|
||||
if (planned.state === "unsupported") {
|
||||
const activated = await activateOrPlan(profileId, io);
|
||||
return activated.state === "plan"
|
||||
? confirmAndSwitch(activated.plan, options, io)
|
||||
: emitActivation(io, activated.result);
|
||||
}
|
||||
if (planned.state === "failed") {
|
||||
return reject(io, "models activate", planned.code);
|
||||
}
|
||||
const plan = planned.plan;
|
||||
if (!plan.needed) {
|
||||
if (!io.asJson && plan.reason === "same_embedding") {
|
||||
io.emit("向量模型未变化,无需重新生成向量。\n");
|
||||
}
|
||||
const activated = await activateOrPlan(profileId, io);
|
||||
if (activated.state === "plan") {
|
||||
return confirmAndSwitch(activated.plan, options, io);
|
||||
}
|
||||
return emitActivation(io, activated.result);
|
||||
}
|
||||
return confirmAndSwitch(plan, options, io);
|
||||
}
|
||||
|
||||
async function activateOrPlan(
|
||||
profileId: string,
|
||||
io: SwitchIo,
|
||||
): Promise<
|
||||
| { state: "plan"; plan: EmbeddingSwitchPlan }
|
||||
| { state: "activated"; result: ModelWriteDocument }
|
||||
> {
|
||||
const result: OperationOutcome<unknown> = await io.operations.post<unknown>(
|
||||
`/api/v1/model-profiles/${encodeURIComponent(profileId)}/activate`,
|
||||
{},
|
||||
);
|
||||
if (
|
||||
result.state === "rejected" &&
|
||||
result.code === "embedding_switch_required"
|
||||
) {
|
||||
const plan = parsePlan(record(result.body)?.plan);
|
||||
if (plan) return { state: "plan", plan };
|
||||
}
|
||||
return { state: "activated", result: modelProfileResult(profileId, result) };
|
||||
}
|
||||
|
||||
function reject(io: SwitchIo, command: string, code: string): number {
|
||||
if (io.asJson) {
|
||||
emitDocument(io, document(command, { state: "rejected", code }));
|
||||
} else {
|
||||
io.emit(`${command} 未完成:${blockerText(code)}(${code})\n`);
|
||||
}
|
||||
return EXIT_FAILED;
|
||||
}
|
||||
|
||||
async function confirmAndSwitch(
|
||||
plan: EmbeddingSwitchPlan,
|
||||
options: ActivateOptions,
|
||||
io: SwitchIo,
|
||||
): Promise<number> {
|
||||
if (plan.blockers.length > 0) {
|
||||
if (io.asJson) {
|
||||
emitDocument(
|
||||
io,
|
||||
document("models activate", {
|
||||
state: "blocked",
|
||||
code: plan.blockers[0],
|
||||
plan,
|
||||
}),
|
||||
);
|
||||
} else {
|
||||
io.emit(renderPlanText(plan));
|
||||
}
|
||||
return EXIT_FAILED;
|
||||
}
|
||||
if (!options.confirmed) {
|
||||
const ask = io.interactive && !io.asJson ? io.confirm : undefined;
|
||||
if (ask === undefined) {
|
||||
if (io.asJson) {
|
||||
emitDocument(
|
||||
io,
|
||||
document("models activate", {
|
||||
state: "confirmation_required",
|
||||
code: "embedding_switch_required",
|
||||
plan,
|
||||
next_step: `${PRODUCT} models activate ${plan.target_profile_id} --yes`,
|
||||
}),
|
||||
);
|
||||
} else {
|
||||
io.emit(`${renderPlanText(plan)}重新运行并加上 --yes 以确认\n`);
|
||||
}
|
||||
return EXIT_USAGE;
|
||||
}
|
||||
io.emit(renderPlanText(plan));
|
||||
if (!(await ask("继续?[y/N] "))) {
|
||||
io.emit("未开始切换,仍在使用原来的向量模型。\n");
|
||||
return EXIT_OK;
|
||||
}
|
||||
}
|
||||
const started = await io.operations.post<unknown>(
|
||||
"/api/v1/embedding-switches",
|
||||
{
|
||||
profile_id: plan.target_profile_id,
|
||||
expected_revision: plan.target_revision,
|
||||
plan_token: plan.plan_token,
|
||||
requested_by: "cli",
|
||||
},
|
||||
);
|
||||
if (started.state !== "committed") {
|
||||
return reject(io, "models activate", started.code);
|
||||
}
|
||||
const job = parseJob(started.data);
|
||||
if (!job) return reject(io, "models activate", "embedding_switch_invalid");
|
||||
const follow = options.follow ?? io.interactive;
|
||||
if (!follow) {
|
||||
if (io.asJson) {
|
||||
emitDocument(io, document("models activate", progressFields(job)));
|
||||
} else {
|
||||
io.emit(
|
||||
`已开始切换向量模型(${job.job_id})。用 memhub models switch status 查看进度。\n`,
|
||||
);
|
||||
}
|
||||
return EXIT_OK;
|
||||
}
|
||||
return followJob(job, io, "models activate");
|
||||
}
|
||||
|
||||
function progressFields(job: EmbeddingSwitchJob): Record<string, unknown> {
|
||||
return {
|
||||
state: job.state,
|
||||
job_id: job.job_id,
|
||||
kind: job.kind,
|
||||
phase: job.progress.phase,
|
||||
percent: job.progress.percent,
|
||||
done: job.progress.done,
|
||||
total: job.progress.total,
|
||||
eta_s: job.progress.eta_s,
|
||||
...(job.error_code ? { code: job.error_code } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
async function readCurrent(io: SwitchIo): Promise<SwitchCurrent | null> {
|
||||
const outcome = await io.reader<unknown>(
|
||||
"/api/v1/embedding-switches/current",
|
||||
);
|
||||
return outcome.state === "ok" ? parseCurrent(outcome.data) : null;
|
||||
}
|
||||
|
||||
/** Poll `current` until the job ends or the user stops watching. */
|
||||
async function followJob(
|
||||
started: EmbeddingSwitchJob,
|
||||
io: SwitchIo,
|
||||
command: string,
|
||||
): Promise<number> {
|
||||
const wait = io.wait ?? defaultWait;
|
||||
const signal = io.signal ?? new AbortController().signal;
|
||||
let last = "";
|
||||
let job = started;
|
||||
const show = (value: EmbeddingSwitchJob) => {
|
||||
const line = io.asJson
|
||||
? JSON.stringify(document(command, progressFields(value)))
|
||||
: renderProgressLine(value);
|
||||
if (line !== last) {
|
||||
io.emit(`${line}\n`);
|
||||
last = line;
|
||||
}
|
||||
};
|
||||
show(job);
|
||||
if (!io.asJson) {
|
||||
io.emit(
|
||||
"(Ctrl-C 只停止显示;用 `memhub models switch cancel` 取消切换)\n",
|
||||
);
|
||||
}
|
||||
while (!TERMINAL.has(job.state)) {
|
||||
try {
|
||||
await wait(POLL_MS, signal);
|
||||
} catch {
|
||||
// Aborted below.
|
||||
}
|
||||
if (signal.aborted) {
|
||||
if (!io.asJson) {
|
||||
io.emit(
|
||||
"已停止显示进度;切换仍在后台进行。用 `memhub models switch cancel` 取消。\n",
|
||||
);
|
||||
}
|
||||
return EXIT_OK;
|
||||
}
|
||||
const current = await readCurrent(io);
|
||||
if (current === null) continue;
|
||||
const next =
|
||||
current.job?.job_id === job.job_id
|
||||
? current.job
|
||||
: current.last_finished?.job_id === job.job_id
|
||||
? current.last_finished
|
||||
: null;
|
||||
if (next === null) continue;
|
||||
job = next;
|
||||
show(job);
|
||||
}
|
||||
if (io.asJson) {
|
||||
emitDocument(
|
||||
io,
|
||||
document(command, {
|
||||
...progressFields(job),
|
||||
result: job.state === "done" ? "switched" : "rolled_back",
|
||||
}),
|
||||
);
|
||||
} else {
|
||||
io.emit(renderOutcomeText(job));
|
||||
}
|
||||
return job.state === "done" ? EXIT_OK : EXIT_FAILED;
|
||||
}
|
||||
|
||||
function defaultWait(milliseconds: number, signal: AbortSignal): Promise<void> {
|
||||
return new Promise((resolve, reject) => {
|
||||
if (signal.aborted) {
|
||||
reject(new Error("aborted"));
|
||||
return;
|
||||
}
|
||||
const timer = setTimeout(() => {
|
||||
signal.removeEventListener("abort", onAbort);
|
||||
resolve();
|
||||
}, milliseconds);
|
||||
const onAbort = () => {
|
||||
clearTimeout(timer);
|
||||
reject(new Error("aborted"));
|
||||
};
|
||||
signal.addEventListener("abort", onAbort, { once: true });
|
||||
});
|
||||
}
|
||||
|
||||
/** `memhub models switch status [--json]`. */
|
||||
export async function switchStatus(io: SwitchIo): Promise<number> {
|
||||
const outcome = await io.reader<unknown>(
|
||||
"/api/v1/embedding-switches/current",
|
||||
);
|
||||
if (outcome.state !== "ok")
|
||||
return reject(io, "models switch status", outcome.code);
|
||||
const current = parseCurrent(outcome.data);
|
||||
if (!current) {
|
||||
return reject(io, "models switch status", "embedding_switch_invalid");
|
||||
}
|
||||
if (io.asJson) {
|
||||
emitDocument(
|
||||
io,
|
||||
document("models switch status", {
|
||||
state: current.phase,
|
||||
writes_paused: current.writes_paused,
|
||||
job: current.job ? progressFields(current.job) : null,
|
||||
last_finished: current.last_finished
|
||||
? progressFields(current.last_finished)
|
||||
: null,
|
||||
}),
|
||||
);
|
||||
return EXIT_OK;
|
||||
}
|
||||
if (current.job) {
|
||||
io.emit(
|
||||
`${renderProgressLine(current.job)}\n记忆整理已暂停,检索不受影响。\n`,
|
||||
);
|
||||
} else if (current.phase === "scanning") {
|
||||
io.emit("正在检查向量数据…\n");
|
||||
} else if (current.last_finished) {
|
||||
io.emit(
|
||||
`没有进行中的向量模型切换。上一次:${renderOutcomeText(current.last_finished)}`,
|
||||
);
|
||||
} else {
|
||||
io.emit("没有进行中的向量模型切换。\n");
|
||||
}
|
||||
return EXIT_OK;
|
||||
}
|
||||
|
||||
/** `memhub models switch cancel [--json]`: cancel the running job, if any. */
|
||||
export async function switchCancel(io: SwitchIo): Promise<number> {
|
||||
const current = await readCurrent(io);
|
||||
if (current === null) {
|
||||
return reject(io, "models switch cancel", "embedding_switch_unavailable");
|
||||
}
|
||||
if (!current.job) {
|
||||
return reject(io, "models switch cancel", "embedding_switch_not_found");
|
||||
}
|
||||
const outcome = await io.operations.post<unknown>(
|
||||
`/api/v1/embedding-switches/${current.job.job_id}/cancel`,
|
||||
{},
|
||||
);
|
||||
if (outcome.state !== "committed") {
|
||||
return reject(io, "models switch cancel", outcome.code);
|
||||
}
|
||||
const job = parseJob(outcome.data) ?? current.job;
|
||||
if (io.asJson) {
|
||||
emitDocument(
|
||||
io,
|
||||
document("models switch cancel", {
|
||||
...progressFields(job),
|
||||
cancel_requested: true,
|
||||
}),
|
||||
);
|
||||
} else {
|
||||
io.emit("已请求取消;当前批次完成后回滚,继续使用原来的向量模型。\n");
|
||||
}
|
||||
return EXIT_OK;
|
||||
}
|
||||
|
||||
interface SpaceRow {
|
||||
space_id: string;
|
||||
state: string;
|
||||
model: string | null;
|
||||
dims: number | null;
|
||||
vectors: number | null;
|
||||
keep_until: string | null;
|
||||
}
|
||||
|
||||
function spaceRows(value: unknown): SpaceRow[] | null {
|
||||
const body = record(value);
|
||||
if (!body || !Array.isArray(body.spaces)) return null;
|
||||
const rows: SpaceRow[] = [];
|
||||
for (const item of body.spaces) {
|
||||
const row = record(item);
|
||||
const spaceId = str(row?.space_id, 64);
|
||||
const state = str(row?.state, 16);
|
||||
if (!row || !spaceId || !state) return null;
|
||||
if (state === "dropped") continue;
|
||||
rows.push({
|
||||
space_id: spaceId,
|
||||
state,
|
||||
model: str(row.model),
|
||||
dims: int(row.dims) ?? int(row.name_dims),
|
||||
vectors: int(row.vectors),
|
||||
keep_until: str(row.keep_until, 64),
|
||||
});
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
const SPACE_STATE: Record<string, string> = {
|
||||
live: "使用中",
|
||||
building: "正在生成",
|
||||
stray: "待合并",
|
||||
retired: "已停用",
|
||||
};
|
||||
|
||||
/** `memhub models spaces [--json]`. */
|
||||
export async function listSpaces(io: SwitchIo): Promise<number> {
|
||||
const outcome = await io.reader<unknown>("/api/v1/embedding-spaces");
|
||||
if (outcome.state !== "ok") return reject(io, "models spaces", outcome.code);
|
||||
const rows = spaceRows(outcome.data);
|
||||
if (!rows) return reject(io, "models spaces", "embedding_spaces_invalid");
|
||||
const body = record(outcome.data) ?? {};
|
||||
const unregistered = Array.isArray(body.scanned)
|
||||
? body.scanned
|
||||
.map(record)
|
||||
.filter(
|
||||
(item): item is Record<string, unknown> =>
|
||||
item !== null && item.state === "unregistered",
|
||||
)
|
||||
.map((item) => ({
|
||||
collection_base: str(item.collection_base) ?? "",
|
||||
vectors: int(item.vectors) ?? 0,
|
||||
}))
|
||||
: [];
|
||||
if (io.asJson) {
|
||||
emitDocument(
|
||||
io,
|
||||
document("models spaces", {
|
||||
state: "ok",
|
||||
spaces: rows,
|
||||
unregistered,
|
||||
scan_error:
|
||||
typeof body.scan_error === "string" && CODE.test(body.scan_error)
|
||||
? body.scan_error
|
||||
: null,
|
||||
}),
|
||||
);
|
||||
return EXIT_OK;
|
||||
}
|
||||
const lines = ["向量空间:"];
|
||||
for (const row of rows) {
|
||||
const dims = row.dims === null ? "维度未知" : `${row.dims} 维`;
|
||||
const keep =
|
||||
row.state === "retired" && row.keep_until
|
||||
? `,保留到 ${row.keep_until.slice(0, 10)}`
|
||||
: "";
|
||||
lines.push(
|
||||
` ${row.space_id} ${SPACE_STATE[row.state] ?? row.state} ${row.model ?? "未知模型"}(${dims}) ${row.vectors ?? "?"} 条${keep}`,
|
||||
);
|
||||
}
|
||||
for (const item of unregistered) {
|
||||
lines.push(
|
||||
` (未登记)${item.collection_base} ${item.vectors} 条,下次启动时合并`,
|
||||
);
|
||||
}
|
||||
if (rows.some((row) => row.state === "retired")) {
|
||||
lines.push(
|
||||
"清理已停用的空间:memhub models spaces --cleanup <space-id> --yes",
|
||||
);
|
||||
}
|
||||
io.emit(`${lines.join("\n")}\n`);
|
||||
return EXIT_OK;
|
||||
}
|
||||
|
||||
/** `memhub models spaces --cleanup <space-id> --yes`: drop a retired space now. */
|
||||
export async function cleanupSpace(
|
||||
spaceId: string,
|
||||
io: SwitchIo,
|
||||
): Promise<number> {
|
||||
if (!UUID.test(spaceId)) {
|
||||
return reject(io, "models spaces", "embedding_space_not_found");
|
||||
}
|
||||
const outcome = await io.operations.delete<unknown>(
|
||||
`/api/v1/embedding-spaces/${spaceId}`,
|
||||
);
|
||||
if (outcome.state !== "committed") {
|
||||
return reject(io, "models spaces", outcome.code);
|
||||
}
|
||||
if (io.asJson) {
|
||||
emitDocument(
|
||||
io,
|
||||
document("models spaces", { state: "dropped", space_id: spaceId }),
|
||||
);
|
||||
} else {
|
||||
io.emit(`已清理旧向量数据:${spaceId}\n`);
|
||||
}
|
||||
return EXIT_OK;
|
||||
}
|
||||
@@ -510,15 +510,98 @@ describe("models add", () => {
|
||||
expect(result.stdout + result.stderr).not.toContain("SYNTHETIC_LOOSE_KEY");
|
||||
});
|
||||
|
||||
it("activates the new profile with --activate", async () => {
|
||||
it("probes the new profile before activating it with --activate", async () => {
|
||||
const configFile = await modelConfigFile();
|
||||
transport.post.mockImplementation(async (url: string) =>
|
||||
url === "/api/v1/model-profiles"
|
||||
? { status: 200, data: PROFILE }
|
||||
: url === "/api/v1/model-profiles/profile-1/activate"
|
||||
? { status: 200, data: { ...PROFILE, is_active: true } }
|
||||
: { status: 404, data: {} },
|
||||
let probeId: string | undefined;
|
||||
transport.get.mockImplementation(async (url: string) => {
|
||||
if (url === "/api/v1/model-profiles") {
|
||||
return { status: 200, data: [PROBE_PROFILE] };
|
||||
}
|
||||
const match = /^\/api\/v1\/model-probe-operations\/(.+)$/.exec(url);
|
||||
if (match) {
|
||||
return {
|
||||
status: 200,
|
||||
data:
|
||||
probeId === undefined
|
||||
? probeNotFound(match[1])
|
||||
: probeReceiptPassing(match[1]),
|
||||
};
|
||||
}
|
||||
return { status: 404, data: {} };
|
||||
});
|
||||
transport.post.mockImplementation(async (url: string, body: unknown) => {
|
||||
if (url === "/api/v1/model-profiles") {
|
||||
return { status: 200, data: PROBE_PROFILE };
|
||||
}
|
||||
if (url === `/api/v1/model-profiles/${PROBE_PROFILE.profile_id}/probe`) {
|
||||
probeId = (body as { operation_id: string }).operation_id;
|
||||
return { status: 200, data: probePending(probeId) };
|
||||
}
|
||||
if (url === "/api/v1/embedding-switches/plan") {
|
||||
return { status: 200, data: samePlan() };
|
||||
}
|
||||
if (
|
||||
url === `/api/v1/model-profiles/${PROBE_PROFILE.profile_id}/activate`
|
||||
) {
|
||||
return {
|
||||
status: 200,
|
||||
data: {
|
||||
...PROBE_PROFILE,
|
||||
is_active: true,
|
||||
embedding_model: "synthetic-embedding",
|
||||
},
|
||||
};
|
||||
}
|
||||
return { status: 404, data: {} };
|
||||
});
|
||||
const result = await capturedRun(
|
||||
["models", "add", "--config-file", configFile, "--activate", "--json"],
|
||||
{ watchWait: async () => undefined },
|
||||
);
|
||||
expect(result.code).toBe(0);
|
||||
const lines = result.stdout
|
||||
.trim()
|
||||
.split("\n")
|
||||
.map((line) => JSON.parse(line));
|
||||
expect(lines[0].probe).toMatchObject({ state: "completed" });
|
||||
expect(lines.at(-1)).toMatchObject({ state: "committed" });
|
||||
expect(transport.post.mock.calls.map(([url]) => url)).toEqual([
|
||||
"/api/v1/model-profiles",
|
||||
`/api/v1/model-profiles/${PROBE_PROFILE.profile_id}/probe`,
|
||||
"/api/v1/embedding-switches/plan",
|
||||
`/api/v1/model-profiles/${PROBE_PROFILE.profile_id}/activate`,
|
||||
]);
|
||||
});
|
||||
|
||||
it("does not activate a profile whose probe fails", async () => {
|
||||
const configFile = await modelConfigFile();
|
||||
let probeId: string | undefined;
|
||||
transport.get.mockImplementation(async (url: string) => {
|
||||
if (url === "/api/v1/model-profiles") {
|
||||
return { status: 200, data: [PROBE_PROFILE] };
|
||||
}
|
||||
const match = /^\/api\/v1\/model-probe-operations\/(.+)$/.exec(url);
|
||||
if (match) {
|
||||
return {
|
||||
status: 200,
|
||||
data:
|
||||
probeId === undefined
|
||||
? probeNotFound(match[1])
|
||||
: probeReceipt(match[1]),
|
||||
};
|
||||
}
|
||||
return { status: 404, data: {} };
|
||||
});
|
||||
transport.post.mockImplementation(async (url: string, body: unknown) => {
|
||||
if (url === "/api/v1/model-profiles") {
|
||||
return { status: 200, data: PROBE_PROFILE };
|
||||
}
|
||||
if (url.endsWith("/probe")) {
|
||||
probeId = (body as { operation_id: string }).operation_id;
|
||||
return { status: 200, data: probeReceipt(probeId) };
|
||||
}
|
||||
return { status: 404, data: {} };
|
||||
});
|
||||
const result = await capturedRun([
|
||||
"models",
|
||||
"add",
|
||||
@@ -527,14 +610,12 @@ describe("models add", () => {
|
||||
"--activate",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(JSON.parse(result.stdout).activation).toEqual({
|
||||
state: "committed",
|
||||
});
|
||||
expect(transport.post.mock.calls.map(([url]) => url)).toEqual([
|
||||
"/api/v1/model-profiles",
|
||||
"/api/v1/model-profiles/profile-1/activate",
|
||||
]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(
|
||||
transport.post.mock.calls.some(([url]) =>
|
||||
String(url).endsWith("/activate"),
|
||||
),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("recovers an unknown create through its operation receipt", async () => {
|
||||
@@ -752,6 +833,54 @@ function probePending(operationId: string) {
|
||||
};
|
||||
}
|
||||
|
||||
function samePlan() {
|
||||
const space = {
|
||||
space_id: null,
|
||||
model: "synthetic-embedding",
|
||||
dims: 8,
|
||||
collection_base: "",
|
||||
};
|
||||
return {
|
||||
needed: false,
|
||||
reason: "same_embedding",
|
||||
plan_token: "token-1",
|
||||
source: space,
|
||||
target: space,
|
||||
target_profile_id: PROBE_PROFILE.profile_id,
|
||||
target_revision: 3,
|
||||
target_provider_kind: "openai_compatible",
|
||||
target_local: false,
|
||||
blockers: [],
|
||||
};
|
||||
}
|
||||
|
||||
function probeReceiptPassing(operationId: string) {
|
||||
const receipt = probeReceipt(operationId);
|
||||
return {
|
||||
...receipt,
|
||||
result: {
|
||||
...receipt.result,
|
||||
stages: [
|
||||
{
|
||||
stage: "embedding",
|
||||
ok: true,
|
||||
code: "ok",
|
||||
latency_ms: 1,
|
||||
embedding_dimensions: 8,
|
||||
},
|
||||
],
|
||||
capabilities: {
|
||||
endpoint_reachable: true,
|
||||
model_available: true,
|
||||
structured_output: true,
|
||||
embedding_available: true,
|
||||
},
|
||||
embedding_width: 8,
|
||||
embedding_switch_required: false,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function probeNotFound(operationId: string) {
|
||||
return {
|
||||
schema_version: 1,
|
||||
@@ -819,7 +948,10 @@ function importJob() {
|
||||
};
|
||||
}
|
||||
|
||||
async function capturedRun(argv: string[]) {
|
||||
async function capturedRun(
|
||||
argv: string[],
|
||||
controls: Parameters<typeof run>[2] = {},
|
||||
) {
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
const originalOut = process.stdout.write;
|
||||
@@ -833,7 +965,7 @@ async function capturedRun(argv: string[]) {
|
||||
return true;
|
||||
}) as typeof process.stderr.write;
|
||||
try {
|
||||
const code = await run(argv, {});
|
||||
const code = await run(argv, {}, controls);
|
||||
return { code, stdout, stderr };
|
||||
} finally {
|
||||
process.stdout.write = originalOut;
|
||||
|
||||
@@ -0,0 +1,273 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { run } from "./cli.js";
|
||||
import { parseLanguageCommand } from "./memory-language-flow.js";
|
||||
import { composeStatusReport, renderStatusText } from "./status-report.js";
|
||||
|
||||
const transport = vi.hoisted(() => ({
|
||||
get: vi.fn(),
|
||||
post: vi.fn(),
|
||||
put: vi.fn(),
|
||||
patch: vi.fn(),
|
||||
delete: vi.fn(),
|
||||
}));
|
||||
vi.mock("axios", () => ({
|
||||
default: {
|
||||
create: vi.fn(() => transport),
|
||||
get: vi.fn(),
|
||||
post: vi.fn(),
|
||||
isAxiosError: () => false,
|
||||
},
|
||||
}));
|
||||
|
||||
const runtime = vi.hoisted(() => ({
|
||||
inspectRuntimeStatus: vi.fn(),
|
||||
discoverRuntimeAttachment: vi.fn(),
|
||||
}));
|
||||
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
|
||||
...runtime,
|
||||
}));
|
||||
|
||||
function languageState(uiLanguage: string | null, extra = {}) {
|
||||
const sdk = uiLanguage === null ? "auto" : uiLanguage;
|
||||
return {
|
||||
ui_language: uiLanguage,
|
||||
sdk_language: sdk,
|
||||
languages: [
|
||||
"en",
|
||||
"zh-Hans",
|
||||
"zh-Hant",
|
||||
"ja",
|
||||
"es",
|
||||
"de",
|
||||
"pt",
|
||||
"fr",
|
||||
"ko",
|
||||
"ru",
|
||||
"id",
|
||||
],
|
||||
...extra,
|
||||
};
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
for (const method of Object.values(transport)) method.mockReset();
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "running",
|
||||
code: "runtime_verified",
|
||||
instanceId: "instance-1",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
version: "1.2.3",
|
||||
});
|
||||
runtime.discoverRuntimeAttachment.mockResolvedValue({
|
||||
instanceId: "instance-1",
|
||||
apiToken: "synthetic-runtime-token",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
servicePid: 4242,
|
||||
health: {
|
||||
status: "ok",
|
||||
service: "one-memory-core",
|
||||
version: "1.2.3",
|
||||
instance_id: "instance-1",
|
||||
},
|
||||
});
|
||||
transport.get.mockImplementation(async (url: string) =>
|
||||
url === "/api/v1/settings/ui-language"
|
||||
? { status: 200, data: languageState(null) }
|
||||
: { status: 404, data: {} },
|
||||
);
|
||||
});
|
||||
|
||||
describe("memhub settings language", () => {
|
||||
it("shows the display language and the SDK prompt language", async () => {
|
||||
const result = await capturedRun(["settings", "language", "--json"]);
|
||||
expect(result.code).toBe(0);
|
||||
const json = JSON.parse(result.stdout);
|
||||
expect(json).toMatchObject({
|
||||
command: "settings language",
|
||||
state: "ok",
|
||||
language: { ui_language: null, sdk_language: "auto" },
|
||||
});
|
||||
expect(transport.put).not.toHaveBeenCalled();
|
||||
const text = (await capturedRun(["settings", "language"])).stdout;
|
||||
expect(text).toContain("界面语言:未设置(SDK 自动识别)");
|
||||
expect(text).toContain("SDK 提示语言:auto");
|
||||
});
|
||||
|
||||
it("sets a language at once and reports the SDK language", async () => {
|
||||
transport.put.mockResolvedValue({
|
||||
status: 200,
|
||||
data: languageState("ja", { rebuilt: true }),
|
||||
});
|
||||
const result = await capturedRun(["settings", "language", "ja", "--json"]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.put).toHaveBeenCalledWith(
|
||||
"/api/v1/settings/ui-language",
|
||||
{ ui_language: "ja" },
|
||||
undefined,
|
||||
);
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
state: "committed",
|
||||
rebuilt: true,
|
||||
language: { ui_language: "ja", sdk_language: "ja" },
|
||||
});
|
||||
const text = (await capturedRun(["settings", "language", "ja"])).stdout;
|
||||
expect(text).toContain("界面语言:ja(日本語)");
|
||||
});
|
||||
|
||||
it("sends null for system and maps the legacy zh-CN code", async () => {
|
||||
transport.put.mockResolvedValue({
|
||||
status: 200,
|
||||
data: languageState("zh-Hans", { rebuilt: false }),
|
||||
});
|
||||
await capturedRun(["settings", "language", "zh-CN"]);
|
||||
expect(transport.put).toHaveBeenLastCalledWith(
|
||||
"/api/v1/settings/ui-language",
|
||||
{ ui_language: "zh-Hans" },
|
||||
undefined,
|
||||
);
|
||||
transport.get.mockResolvedValue({
|
||||
status: 200,
|
||||
data: languageState("en"),
|
||||
});
|
||||
transport.put.mockResolvedValue({
|
||||
status: 200,
|
||||
data: languageState(null, { rebuilt: true }),
|
||||
});
|
||||
await capturedRun(["settings", "language", "system"]);
|
||||
expect(transport.put).toHaveBeenLastCalledWith(
|
||||
"/api/v1/settings/ui-language",
|
||||
{ ui_language: null },
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
it("does not write a language Core already has", async () => {
|
||||
const result = await capturedRun(["settings", "language", "system"]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(
|
||||
JSON.parse(
|
||||
(await capturedRun(["settings", "language", "system", "--json"]))
|
||||
.stdout,
|
||||
).state,
|
||||
).toBe("unchanged");
|
||||
expect(transport.put).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("refuses unknown codes before the Runtime", async () => {
|
||||
for (const bad of [
|
||||
["settings", "language", "it"],
|
||||
["settings", "language", "EN"],
|
||||
["settings", "language", "ja", "ko"],
|
||||
]) {
|
||||
expect((await capturedRun(bad)).code, bad.join(" ")).toBe(2);
|
||||
}
|
||||
expect(transport.put).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("parses the forms the help shows", () => {
|
||||
expect(parseLanguageCommand([])).toEqual({ kind: "show" });
|
||||
expect(parseLanguageCommand(["system"])).toEqual({
|
||||
kind: "set",
|
||||
language: null,
|
||||
});
|
||||
expect(parseLanguageCommand(["zh-Hant", "--json"])).toEqual({
|
||||
kind: "set",
|
||||
language: "zh-Hant",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("status language", () => {
|
||||
it.each([
|
||||
[null, "auto", "跟随系统", "auto(自动识别中/英)"],
|
||||
["ja", "ja", "ja", "SDK 提示语言 ja"],
|
||||
])(
|
||||
"shows the UI language %s and the SDK language %s",
|
||||
async (uiLanguage, sdkLanguage, uiText, sdkText) => {
|
||||
const processing = {
|
||||
pending: 0,
|
||||
completed: 0,
|
||||
failed: 0,
|
||||
blocked_stages: [],
|
||||
pipeline: {
|
||||
mode: "pro",
|
||||
configured_mode: "pro",
|
||||
forced_by_env: false,
|
||||
last_evolve_at: null,
|
||||
graph_build: { state: "idle", users: {} },
|
||||
evolve: { mode: "pro", schedule: null, running: [], users: {} },
|
||||
features: {},
|
||||
ui_language: uiLanguage,
|
||||
sdk_language: sdkLanguage,
|
||||
},
|
||||
};
|
||||
const report = await composeStatusReport({
|
||||
dataDir: "/tmp/isolated",
|
||||
runtime: { state: "running", code: "ok", host: "127.0.0.1", port: 1 },
|
||||
background: null,
|
||||
reader: (async (path: string) =>
|
||||
path === "/api/v1/memory-processing"
|
||||
? { state: "ok", data: processing }
|
||||
: { state: "unavailable", code: "x" }) as never,
|
||||
});
|
||||
expect(report.pipeline?.ui_language).toBe(uiLanguage);
|
||||
expect(report.pipeline?.sdk_language).toBe(sdkLanguage);
|
||||
const text = renderStatusText(report, "MemHub");
|
||||
expect(text).toContain(uiText);
|
||||
expect(text).toContain(sdkText);
|
||||
},
|
||||
);
|
||||
|
||||
it("reads an older Runtime without the setting as auto", async () => {
|
||||
const report = await composeStatusReport({
|
||||
dataDir: "/tmp/isolated",
|
||||
runtime: { state: "running", code: "ok", host: "127.0.0.1", port: 1 },
|
||||
background: null,
|
||||
reader: (async () => ({
|
||||
state: "ok",
|
||||
data: {
|
||||
pipeline: {
|
||||
mode: "pro",
|
||||
configured_mode: "pro",
|
||||
evolve: {},
|
||||
graph_build: {},
|
||||
},
|
||||
},
|
||||
})) as never,
|
||||
});
|
||||
expect(report.pipeline?.sdk_language).toBe("auto");
|
||||
expect(report.pipeline?.ui_language).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
async function capturedRun(arguments_: string[]) {
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
const stdoutSpy = vi
|
||||
.spyOn(process.stdout, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stdout += String(chunk);
|
||||
return true;
|
||||
});
|
||||
const stderrSpy = vi
|
||||
.spyOn(process.stderr, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stderr += String(chunk);
|
||||
return true;
|
||||
});
|
||||
try {
|
||||
return {
|
||||
code: await run(arguments_, { ONE_MEMORY_DATA_DIR: "/synthetic/data" }),
|
||||
stdout,
|
||||
stderr,
|
||||
};
|
||||
} finally {
|
||||
stdoutSpy.mockRestore();
|
||||
stderrSpy.mockRestore();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,185 @@
|
||||
/**
|
||||
* `memhub settings language [<code>|system]`.
|
||||
*
|
||||
* The display language the App reports to the Runtime (`ui_language`), and
|
||||
* the SDK prompt language it implies. Every code, zh-Hans and en included,
|
||||
* makes extraction and the other SDK prompts use that language; "system"
|
||||
* (no language set) passes none, so the SDK keeps detecting Chinese or
|
||||
* English as before. The App sends the language it shows, so it is only
|
||||
* unset on an install without the App. A change applies at once: a write in flight
|
||||
* finishes in the old language, the next ones use the new one. The App sends
|
||||
* its own choice again whenever it changes there.
|
||||
*/
|
||||
|
||||
import type { CoreReader } from "./core-client.js";
|
||||
import type { CorePutOperations } from "./core-operations.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
export const UI_LANGUAGES = [
|
||||
"en",
|
||||
"zh-Hans",
|
||||
"zh-Hant",
|
||||
"ja",
|
||||
"es",
|
||||
"de",
|
||||
"pt",
|
||||
"fr",
|
||||
"ko",
|
||||
"ru",
|
||||
"id",
|
||||
] as const;
|
||||
export type UiLanguage = (typeof UI_LANGUAGES)[number];
|
||||
|
||||
/** Codes earlier builds used, and the code each one is now. */
|
||||
const LEGACY: Record<string, UiLanguage> = { "zh-CN": "zh-Hans" };
|
||||
|
||||
const NAMES: Record<UiLanguage, string> = {
|
||||
en: "English",
|
||||
"zh-Hans": "简体中文",
|
||||
"zh-Hant": "繁體中文",
|
||||
ja: "日本語",
|
||||
es: "Español",
|
||||
de: "Deutsch",
|
||||
pt: "Português",
|
||||
fr: "Français",
|
||||
ko: "한국어",
|
||||
ru: "Русский",
|
||||
id: "Bahasa Indonesia",
|
||||
};
|
||||
|
||||
export type LanguageCommand =
|
||||
| { kind: "show" }
|
||||
| { kind: "set"; language: UiLanguage | null };
|
||||
|
||||
function isUiLanguage(value: string): value is UiLanguage {
|
||||
return (UI_LANGUAGES as readonly string[]).includes(value);
|
||||
}
|
||||
|
||||
/** Arguments after `settings language`; null = usage. */
|
||||
export function parseLanguageCommand(
|
||||
values: readonly string[],
|
||||
): LanguageCommand | null {
|
||||
const rest = values.filter((value) => value !== "--json");
|
||||
if (rest.length === 0) return { kind: "show" };
|
||||
if (rest.length !== 1) return null;
|
||||
const raw = rest[0] ?? "";
|
||||
const value: string = Object.hasOwn(LEGACY, raw) ? LEGACY[raw] : raw;
|
||||
if (value === "system") return { kind: "set", language: null };
|
||||
return isUiLanguage(value) ? { kind: "set", language: value } : null;
|
||||
}
|
||||
|
||||
export interface LanguageState {
|
||||
ui_language: UiLanguage | null;
|
||||
sdk_language: string;
|
||||
rebuilt?: boolean;
|
||||
}
|
||||
|
||||
function languageState(value: unknown): LanguageState | null {
|
||||
if (!value || typeof value !== "object") return null;
|
||||
const item = value as Record<string, unknown>;
|
||||
if (
|
||||
!(item.ui_language === null || isUiLanguage(String(item.ui_language))) ||
|
||||
typeof item.sdk_language !== "string"
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
ui_language: item.ui_language as UiLanguage | null,
|
||||
sdk_language: item.sdk_language,
|
||||
...(typeof item.rebuilt === "boolean" ? { rebuilt: item.rebuilt } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
export type LanguageDocument = {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
command: "settings language";
|
||||
} & (
|
||||
| { state: "ok" | "unchanged"; language: LanguageState }
|
||||
| { state: "committed"; language: LanguageState; rebuilt: boolean }
|
||||
| { state: "rejected" | "unknown" | "unavailable"; code: string }
|
||||
);
|
||||
|
||||
const BASE = {
|
||||
schema_version: 1 as const,
|
||||
product: PRODUCT,
|
||||
command: "settings language" as const,
|
||||
};
|
||||
|
||||
const PATH = "/api/v1/settings/ui-language";
|
||||
|
||||
export async function runLanguageCommand(
|
||||
command: LanguageCommand,
|
||||
reader: CoreReader,
|
||||
operations: CorePutOperations,
|
||||
): Promise<LanguageDocument> {
|
||||
const read = await reader<unknown>(PATH);
|
||||
if (read.state !== "ok") {
|
||||
return { ...BASE, state: "unavailable", code: read.code };
|
||||
}
|
||||
const current = languageState(read.data);
|
||||
if (current === null) {
|
||||
return { ...BASE, state: "unavailable", code: "response_invalid" };
|
||||
}
|
||||
if (command.kind === "show") {
|
||||
return { ...BASE, state: "ok", language: current };
|
||||
}
|
||||
if (current.ui_language === command.language) {
|
||||
return { ...BASE, state: "unchanged", language: current };
|
||||
}
|
||||
const result = await operations.put<unknown>(PATH, {
|
||||
ui_language: command.language,
|
||||
});
|
||||
if (result.state !== "committed") {
|
||||
return { ...BASE, state: result.state, code: result.code };
|
||||
}
|
||||
const changed = languageState(result.data);
|
||||
if (changed === null) {
|
||||
return { ...BASE, state: "unknown", code: "response_invalid" };
|
||||
}
|
||||
return {
|
||||
...BASE,
|
||||
state: "committed",
|
||||
language: changed,
|
||||
rebuilt: changed.rebuilt === true,
|
||||
};
|
||||
}
|
||||
|
||||
export function languageExitCode(document: LanguageDocument): number {
|
||||
return ["ok", "unchanged", "committed"].includes(document.state) ? 0 : 1;
|
||||
}
|
||||
|
||||
function sdkText(sdkLanguage: string): string {
|
||||
return sdkLanguage === "auto"
|
||||
? "auto(SDK 自动识别中文或英文)"
|
||||
: `${sdkLanguage}(提取等 SDK 提示使用该语言)`;
|
||||
}
|
||||
|
||||
export function renderLanguageText(document: LanguageDocument): string {
|
||||
if ("code" in document) {
|
||||
return `界面语言未能${document.state === "unavailable" ? "读取" : "修改"}:${document.code}\n`;
|
||||
}
|
||||
const state = document.language;
|
||||
const lines = [
|
||||
`界面语言:${
|
||||
state.ui_language === null
|
||||
? "未设置(SDK 自动识别)"
|
||||
: `${state.ui_language}(${NAMES[state.ui_language]})`
|
||||
}`,
|
||||
`SDK 提示语言:${sdkText(state.sdk_language)}`,
|
||||
];
|
||||
if (document.state === "ok") {
|
||||
lines.push(
|
||||
`可选:${UI_LANGUAGES.join("、")}、system。任一语言(包括 zh-Hans、en)都会让 SDK 使用该语言;system 不指定语言,SDK 自动识别中文或英文。`,
|
||||
);
|
||||
}
|
||||
if (document.state === "unchanged") lines.push("没有需要改动的设置。");
|
||||
if (document.state === "committed") {
|
||||
lines.push(
|
||||
document.rebuilt
|
||||
? "已更新。正在进行的写入按原语言完成,之后的写入和检索使用新语言。"
|
||||
: "已保存,SDK 提示语言没有变化。",
|
||||
);
|
||||
}
|
||||
return `${lines.join("\n")}\n`;
|
||||
}
|
||||
@@ -142,7 +142,7 @@ describe("memhub settings mode", () => {
|
||||
["settings", "mode", "max"],
|
||||
["settings", "mode", "--yes"],
|
||||
["settings", "mode", "lite", "pro"],
|
||||
["settings", "language", "en"],
|
||||
["settings", "theme", "dark"],
|
||||
]) {
|
||||
expect((await capturedRun(bad)).code, bad.join(" ")).toBe(2);
|
||||
}
|
||||
|
||||
@@ -372,6 +372,9 @@ export function renderMemoryAddText(document: MemoryAddDocument): string {
|
||||
export function renderMemoryPageText(document: MemoryPageDocument): string {
|
||||
const page = document.page;
|
||||
if (page.state !== "ok") {
|
||||
if (page.state === "unavailable" && page.code === "embedding_unavailable") {
|
||||
return "向量服务不可用,无法检索;请检查向量模型配置。\n";
|
||||
}
|
||||
if (page.state === "unavailable") {
|
||||
return "后台正忙,记忆列表这次没查完,不代表没有记忆,请稍后再试\n";
|
||||
}
|
||||
|
||||
@@ -22,6 +22,10 @@ const RESULT_KEYS = new Set([
|
||||
"stages",
|
||||
"capabilities",
|
||||
]);
|
||||
const OPTIONAL_RESULT_KEYS = [
|
||||
"embedding_width",
|
||||
"embedding_switch_required",
|
||||
] as const;
|
||||
const STAGE_KEYS = new Set([
|
||||
"stage",
|
||||
"ok",
|
||||
@@ -272,9 +276,34 @@ function validateResult(
|
||||
inputRevision: number,
|
||||
): Record<string, unknown> | null {
|
||||
const result = record(value);
|
||||
// A Core with the embedding registry adds what the embedding stage measured
|
||||
// and whether using it needs an embedding switch; older Cores send neither.
|
||||
const optional = Object.fromEntries(
|
||||
OPTIONAL_RESULT_KEYS.filter((key) => key in (result ?? {})).map((key) => [
|
||||
key,
|
||||
result?.[key],
|
||||
]),
|
||||
);
|
||||
const required = result
|
||||
? Object.fromEntries(
|
||||
Object.entries(result).filter(
|
||||
([key]) => !(OPTIONAL_RESULT_KEYS as readonly string[]).includes(key),
|
||||
),
|
||||
)
|
||||
: null;
|
||||
if (
|
||||
!result ||
|
||||
!exactKeys(result, RESULT_KEYS) ||
|
||||
!required ||
|
||||
!exactKeys(required, RESULT_KEYS) ||
|
||||
!(
|
||||
optional.embedding_width === undefined ||
|
||||
optional.embedding_width === null ||
|
||||
positiveInteger(optional.embedding_width)
|
||||
) ||
|
||||
!(
|
||||
optional.embedding_switch_required === undefined ||
|
||||
typeof optional.embedding_switch_required === "boolean"
|
||||
) ||
|
||||
result.profile_id !== profileId ||
|
||||
result.input_profile_revision !== inputRevision ||
|
||||
!positiveInteger(result.profile_revision) ||
|
||||
|
||||
@@ -117,7 +117,7 @@ export async function editModelProfile(
|
||||
return modelProfileResult(profileId, result);
|
||||
}
|
||||
|
||||
function modelProfileResult(
|
||||
export function modelProfileResult(
|
||||
operationId: string,
|
||||
result:
|
||||
| Awaited<ReturnType<CoreOperations["post"]>>
|
||||
|
||||
@@ -18,6 +18,7 @@ import {
|
||||
validateProcessing,
|
||||
validateSources,
|
||||
} from "./core-responses.js";
|
||||
import { describeEmbeddingSwitchSummary } from "./embedding-switch-flow.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
const MIGRATION_ISSUES_URL =
|
||||
@@ -85,6 +86,10 @@ export interface PipelineSummary {
|
||||
graph_build: string | null;
|
||||
/** SDK features (`memory_features`) whose value differs from the default. */
|
||||
features: Record<string, boolean>;
|
||||
/** The App's display language; null = never chosen or "follow the system". */
|
||||
ui_language: string | null;
|
||||
/** The SDK prompt language: `auto` (zh/en detection) or the code passed. */
|
||||
sdk_language: string;
|
||||
}
|
||||
|
||||
const MODES = new Set(["lite", "pro", "ultra"]);
|
||||
@@ -157,6 +162,10 @@ export function describePipeline(value: unknown): PipelineSummary | null {
|
||||
evolve_users: users,
|
||||
graph_build: typeof graph.state === "string" ? graph.state : null,
|
||||
features,
|
||||
ui_language: typeof item.ui_language === "string" ? item.ui_language : null,
|
||||
// An older Runtime has no setting: it always left the SDK on auto.
|
||||
sdk_language:
|
||||
typeof item.sdk_language === "string" ? item.sdk_language : "auto",
|
||||
};
|
||||
}
|
||||
|
||||
@@ -306,6 +315,10 @@ export function renderStatusText(
|
||||
);
|
||||
}
|
||||
if (report.pipeline) lines.push(`记忆模式:${pipelineLine(report.pipeline)}`);
|
||||
const embeddingSwitch = describeEmbeddingSwitchSummary(
|
||||
report.sections.processing?.embedding_switch,
|
||||
);
|
||||
if (embeddingSwitch !== null) lines.push(`向量模型:${embeddingSwitch}`);
|
||||
if (report.summary.state === "partial") {
|
||||
const notes = [
|
||||
report.summary.degraded.length > 0
|
||||
@@ -348,6 +361,13 @@ export function pipelineLine(pipeline: PipelineSummary): string {
|
||||
.join("、")}`,
|
||||
);
|
||||
}
|
||||
parts.push(
|
||||
`;界面语言 ${pipeline.ui_language ?? "跟随系统"},SDK 提示语言 ${
|
||||
pipeline.sdk_language === "auto"
|
||||
? "auto(自动识别中/英)"
|
||||
: pipeline.sdk_language
|
||||
}`,
|
||||
);
|
||||
return parts.join("");
|
||||
}
|
||||
|
||||
@@ -365,7 +385,10 @@ function sectionLine(section: Section | undefined): string {
|
||||
if (section.state === "not_checked") return "未检查";
|
||||
if (section.state !== "ok") return "暂时不可用,请稍后重试";
|
||||
return Object.entries(section)
|
||||
.filter(([key]) => key !== "state" && key !== "code")
|
||||
.filter(
|
||||
([key]) =>
|
||||
key !== "state" && key !== "code" && key !== "embedding_switch",
|
||||
)
|
||||
.map(([key, value]) => `${fieldLabel(key)} ${formatValue(value)}`)
|
||||
.join(" ");
|
||||
}
|
||||
@@ -481,9 +504,30 @@ function describeProcessing(
|
||||
reason: block.reason,
|
||||
count: block.count,
|
||||
})),
|
||||
...embeddingSwitchField(summary),
|
||||
};
|
||||
}
|
||||
|
||||
/** The running switch or merge, projected; absent when none runs. */
|
||||
function embeddingSwitchField(
|
||||
summary: MemoryProcessingSummary,
|
||||
): Record<string, unknown> {
|
||||
const value = (summary as unknown as Record<string, unknown>)
|
||||
.embedding_switch;
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) return {};
|
||||
const item = value as Record<string, unknown>;
|
||||
const kind = item.kind === "merge" ? "merge" : "switch";
|
||||
const state =
|
||||
typeof item.state === "string" && /^[a-z_]{1,32}$/.test(item.state)
|
||||
? item.state
|
||||
: "unknown";
|
||||
const percent =
|
||||
typeof item.percent === "number" && Number.isFinite(item.percent)
|
||||
? Math.min(100, Math.max(0, item.percent))
|
||||
: 0;
|
||||
return { embedding_switch: { kind, state, percent } };
|
||||
}
|
||||
|
||||
/** Projection, not passthrough: a Core that leaks a key must not make us leak. */
|
||||
function describeModel(profiles: ModelProfile[]): Record<string, unknown> {
|
||||
const active = profiles.find((profile) => profile.is_active);
|
||||
|
||||
@@ -4,7 +4,7 @@ build-backend = "uv_build"
|
||||
|
||||
[project]
|
||||
name = "one-memory"
|
||||
version = "1.0.5"
|
||||
version = "1.0.6"
|
||||
description = "MemHub local-first memory core"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
@@ -12,7 +12,7 @@ dependencies = [
|
||||
"fastapi>=0.141.1",
|
||||
"filelock>=3.32.4",
|
||||
# Public SDK version validated against MemHub's permission and recall contracts.
|
||||
"hy-memory==2.0.9",
|
||||
"hy-memory==2.0.11",
|
||||
# chromadb arrives through the HY SDK; locked here so an unrelated upgrade
|
||||
# cannot move the SDK's store. Built-in embedding consumes numpy directly.
|
||||
"chromadb==1.5.9",
|
||||
|
||||
@@ -71,6 +71,9 @@ from one_memory.diagnostic_log import (
|
||||
reset_trace_id,
|
||||
resource_snapshot,
|
||||
)
|
||||
from one_memory.embedding_health import EMBEDDING_UNAVAILABLE, EmbeddingUnavailableError
|
||||
from one_memory.embedding_spaces import EmbeddingSpaceStore
|
||||
from one_memory.embedding_switch import EmbeddingSwitchJob, EmbeddingSwitchService
|
||||
from one_memory.exception_reporting import (
|
||||
install_asyncio_exception_handler,
|
||||
install_threading_excepthook,
|
||||
@@ -237,6 +240,7 @@ from one_memory.model_profiles import (
|
||||
ModelProfilePatch,
|
||||
ModelProfileStore,
|
||||
ModelProfileUnavailableError,
|
||||
ProviderKind,
|
||||
)
|
||||
from one_memory.model_request_audit import (
|
||||
AuditedModelTransport,
|
||||
@@ -296,6 +300,12 @@ from one_memory.source_read_access import SourceReadAccess
|
||||
from one_memory.source_totals import SourceTotals
|
||||
from one_memory.strict_query import forbid_unknown_query_params
|
||||
from one_memory.target_schema import bootstrap_fresh_management_schema
|
||||
from one_memory.ui_language import (
|
||||
SDK_LANGUAGE_AUTO,
|
||||
UI_LANGUAGES,
|
||||
UiLanguage,
|
||||
sdk_language,
|
||||
)
|
||||
from one_memory.user_scope import (
|
||||
DEFAULT_USER_ID,
|
||||
MULTI_USER_SOURCES,
|
||||
@@ -330,6 +340,10 @@ class HealthResponse(BaseModel):
|
||||
# failed steps are named so a stuck recovery is visible without a log.
|
||||
recovery: Literal["pending", "running", "done", "failed"] = "done"
|
||||
recovery_failed_steps: tuple[str, ...] = ()
|
||||
# An embedding switch or a startup merge of stray spaces, while one runs
|
||||
# (`{kind, state, percent, eta_s}`): the loading screen and the status
|
||||
# line read it. Reads are served throughout; model writes are paused.
|
||||
embedding_switch: dict[str, object] | None = None
|
||||
|
||||
|
||||
class BackupRequest(BaseModel):
|
||||
@@ -666,6 +680,9 @@ class MemoryProcessingSummary(BaseModel):
|
||||
#: R10: the SDK mode and System 2 state. Optional so older readers and
|
||||
#: test doubles stay valid; Core always sends it.
|
||||
pipeline: "PipelineStatus | None" = None
|
||||
#: The running embedding switch or merge, `{job_id, kind, state, percent,
|
||||
#: eta_s}`; None when none runs. Model writes wait while it is set.
|
||||
embedding_switch: dict[str, object] | None = None
|
||||
|
||||
|
||||
class MemoryModeState(BaseModel):
|
||||
@@ -748,6 +765,38 @@ class PipelineStatus(BaseModel):
|
||||
#: The SDK features (`memory_features`) whose value differs from the SDK
|
||||
#: default; empty while every feature is at its default.
|
||||
features: dict[MemoryFeatureName, bool] = Field(default_factory=dict)
|
||||
#: The display language the App last reported (`ui_language`); null =
|
||||
#: none set (the App sends the language it shows, even following the system).
|
||||
ui_language: UiLanguage | None = None
|
||||
#: The SDK prompt language: `auto` (the SDK detects zh/en) or a code.
|
||||
sdk_language: str = SDK_LANGUAGE_AUTO
|
||||
|
||||
|
||||
class UiLanguageState(BaseModel):
|
||||
"""`GET /api/v1/settings/ui-language`."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
#: What the App reported; null = none set (CLI/API-only, or `system`).
|
||||
ui_language: UiLanguage | None
|
||||
#: What SDK clients are built with: `auto` (no language set: the SDK
|
||||
#: detects zh/en, as before) or the code passed as `MemoryConfig.language`.
|
||||
sdk_language: str
|
||||
languages: tuple[UiLanguage, ...] = UI_LANGUAGES
|
||||
|
||||
|
||||
class UiLanguageUpdate(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
#: A supported code, or null for "follow the system" (SDK auto-detect).
|
||||
ui_language: UiLanguage | None
|
||||
|
||||
|
||||
class UiLanguageChange(UiLanguageState):
|
||||
"""`PUT /api/v1/settings/ui-language`."""
|
||||
|
||||
#: Whether the SDK language changed, so the SDK clients were rebuilt.
|
||||
rebuilt: bool
|
||||
|
||||
|
||||
class MemoryFeatureState(BaseModel):
|
||||
@@ -1044,6 +1093,16 @@ def create_app(
|
||||
# state, so initialize it before authority selection.
|
||||
app.state.secret_store = secret_store or KeyringSecretStore()
|
||||
app.state.model_profile_store = ModelProfileStore(database_path)
|
||||
# The live embedding space, pinned before any SDK client opens.
|
||||
# A home this build has not seen registers the space it already
|
||||
# uses, under the name it already has (`EmbeddingSpaceStore.bootstrap`).
|
||||
app.state.embedding_spaces = EmbeddingSpaceStore(database_path)
|
||||
active_identity = app.state.model_profile_store.active_identity()
|
||||
app.state.embedding_spaces.bootstrap(
|
||||
app.state.model_profile_store.snapshot(*active_identity)
|
||||
if active_identity is not None
|
||||
else None
|
||||
)
|
||||
app.state.sdk_data_plane_migrations = SdkDataPlaneMigrationStore(database_path)
|
||||
app.state.memory_authority = app.state.sdk_data_plane_migrations.authority()
|
||||
_record_startup_authority(
|
||||
@@ -1280,6 +1339,9 @@ def create_app(
|
||||
sedimentation_enabled=lambda: _config_store(app).sedimentation_enabled(),
|
||||
memory_mode=lambda: _memory_mode(app),
|
||||
memory_features=lambda: _memory_features(app),
|
||||
spaces=app.state.embedding_spaces,
|
||||
writes_open=lambda: _embedding_writes_open(app),
|
||||
sdk_language=lambda: _sdk_language(app),
|
||||
)
|
||||
app.state.sdk_memory_runtime = SdkMemoryRuntime(
|
||||
coordinator=app.state.sdk_write_coordinator,
|
||||
@@ -1288,6 +1350,7 @@ def create_app(
|
||||
extraction_settings=app.state.extraction_settings_store,
|
||||
data_dir=runtime_settings.data_dir,
|
||||
memory_mode=lambda: _memory_mode(app),
|
||||
writes_open=lambda: _embedding_writes_open(app),
|
||||
)
|
||||
app.state.evolve_state_store = EvolveStateStore(runtime_settings.data_dir)
|
||||
app.state.evolve_service = EvolveService(
|
||||
@@ -1342,6 +1405,10 @@ def create_app(
|
||||
async with app.state.sedimentation_lock, source_lock:
|
||||
if not _config_store(app).sedimentation_enabled():
|
||||
raise RuntimeError("model_sedimentation_disabled")
|
||||
if not _embedding_writes_open(app):
|
||||
# Re-checked at the last boundary: a switch that began
|
||||
# after the claim keeps this job queued, unsent.
|
||||
raise RuntimeError("embedding_switch_in_progress")
|
||||
if not _source_writes_on(app, source_id):
|
||||
raise RuntimeError("model_content_source_disabled")
|
||||
return await mark_invoked()
|
||||
@@ -1393,6 +1460,64 @@ def create_app(
|
||||
source_claim_enabled=lambda source_id: _sedimentation_claim_enabled(app, source_id),
|
||||
)
|
||||
app.state.memory_write_drain = memory_write_drain
|
||||
|
||||
async def embedding_writes_drained() -> bool:
|
||||
"""No model write runs: none past its claim, none holding a client."""
|
||||
await asyncio.to_thread(memory_write_queue.reclaim_expired)
|
||||
running, _ = await asyncio.to_thread(app.state.sdk_memory_runtime.active_depths)
|
||||
in_use = cast(SdkWriteCoordinator, app.state.sdk_write_coordinator).in_use()
|
||||
return running == 0 and in_use == 0
|
||||
|
||||
async def quiesce_sdk_clients() -> Callable[[], None]:
|
||||
"""Close every SDK client and hold the stores, for the graph re-embed."""
|
||||
deadline = time.monotonic() + 120
|
||||
while True:
|
||||
await _read_handle(app).close_idle()
|
||||
await app.state.sdk_write_coordinator.close_idle()
|
||||
if hold_stores_offline(runtime_settings.data_dir):
|
||||
return lambda: release_stores(runtime_settings.data_dir)
|
||||
if time.monotonic() >= deadline:
|
||||
raise RuntimeError("embedding_switch_clients_busy")
|
||||
await asyncio.sleep(0.2)
|
||||
|
||||
async def after_embedding_commit(job: EmbeddingSwitchJob) -> None:
|
||||
from one_memory.hy_graph_admin import forget_graph_widths
|
||||
|
||||
forget_graph_widths()
|
||||
_read_handle(app).health.forget()
|
||||
if job.kind == "switch" and job.target_profile_id is not None:
|
||||
profile = await asyncio.to_thread(
|
||||
_model_profile_store(app).require,
|
||||
job.target_profile_id,
|
||||
_secret_store(app),
|
||||
)
|
||||
await asyncio.to_thread(
|
||||
app.state.memory_task_store.rebind_unstarted_profile, profile
|
||||
)
|
||||
await asyncio.to_thread(memory_write_queue.readmit_model_profile_waiters)
|
||||
_resume_sedimentation(app)
|
||||
_start_backfill(app)
|
||||
|
||||
from one_memory.embedding_switch import default_reembed_backend
|
||||
|
||||
app.state.embedding_switch = EmbeddingSwitchService(
|
||||
database_path=database_path,
|
||||
data_dir=runtime_settings.data_dir,
|
||||
profiles=app.state.model_profile_store,
|
||||
spaces=app.state.embedding_spaces,
|
||||
secrets=app.state.secret_store,
|
||||
backend=default_reembed_backend(runtime_settings.data_dir),
|
||||
drain_idle=embedding_writes_drained,
|
||||
on_commit=after_embedding_commit,
|
||||
quiesce_clients=quiesce_sdk_clients,
|
||||
# Only ultra keeps a graph; its width follows the space.
|
||||
graph_enabled=lambda: _memory_mode(app) == "ultra",
|
||||
worker_enabled=runtime_settings.embedding_switch_worker,
|
||||
drain_timeout_s=runtime_settings.embedding_switch_drain_timeout_seconds,
|
||||
)
|
||||
if app.state.memory_authority is not MemoryAuthority.SDK:
|
||||
# Only an SDK-authority home has SDK spaces to scan and merge.
|
||||
app.state.embedding_switch.release_scan_hold()
|
||||
# Started by `recover_after_ready`: a running drain claims jobs and
|
||||
# holds the single writer, which the rest of startup still needs.
|
||||
app.state.memory_write_drain_task = None
|
||||
@@ -1631,6 +1756,12 @@ def create_app(
|
||||
repair.coding_rows,
|
||||
)
|
||||
|
||||
async def resume_embedding_spaces() -> None:
|
||||
try:
|
||||
await app.state.embedding_switch.recover()
|
||||
finally:
|
||||
app.state.embedding_switch.start_retention()
|
||||
|
||||
async def start_write_drain() -> None:
|
||||
app.state.memory_write_drain_task = watched_task(
|
||||
asyncio.get_running_loop(),
|
||||
@@ -1719,6 +1850,13 @@ def create_app(
|
||||
# Before the drain: the repair restores metadata the
|
||||
# queued writes are read back against.
|
||||
await step("sdk_metadata_repair", repair_sdk_metadata)
|
||||
# Before the drain too: resume an unfinished embedding
|
||||
# switch (an App update may have stopped it), or scan for
|
||||
# stray spaces and start merging them. Until this decides,
|
||||
# the write gate holds every model write back; reads are
|
||||
# served from the live space throughout.
|
||||
if sdk:
|
||||
await step("embedding_spaces", resume_embedding_spaces)
|
||||
await step("write_drain", start_write_drain)
|
||||
if sdk:
|
||||
await step("evolve_schedule", start_evolve_schedule)
|
||||
@@ -1821,6 +1959,8 @@ def create_app(
|
||||
await app.state.evolve_scheduler.shutdown()
|
||||
await app.state.evolve_service.shutdown()
|
||||
await app.state.memory_graph_backfill.shutdown()
|
||||
# The job stays where it is; the next start resumes it.
|
||||
await app.state.embedding_switch.shutdown()
|
||||
await app.state.sdk_memory_runtime.shutdown()
|
||||
await app.state.sdk_write_coordinator.close()
|
||||
if app.state.sdk_client_handle is not None:
|
||||
@@ -1906,6 +2046,9 @@ def create_app(
|
||||
from one_memory.call_event_routes import call_event_router
|
||||
|
||||
app.include_router(call_event_router(protected))
|
||||
from one_memory.embedding_switch_routes import embedding_switch_router
|
||||
|
||||
app.include_router(embedding_switch_router(protected))
|
||||
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
@@ -1999,6 +2142,12 @@ def create_app(
|
||||
},
|
||||
)
|
||||
|
||||
@app.exception_handler(EmbeddingUnavailableError)
|
||||
async def embedding_unavailable(_: Request, _error: EmbeddingUnavailableError) -> JSONResponse:
|
||||
# No lexical fallback: a search that cannot embed its query fails,
|
||||
# and says why, so the App can show it (top-right toast).
|
||||
return JSONResponse(status_code=503, content={"detail": EMBEDDING_UNAVAILABLE})
|
||||
|
||||
@app.exception_handler(sqlite3.OperationalError)
|
||||
async def sqlite_error(request: Request, error: sqlite3.OperationalError) -> JSONResponse:
|
||||
record_database_error("core.api", error, route_path=request.url.path)
|
||||
@@ -2044,6 +2193,7 @@ def create_app(
|
||||
),
|
||||
recovery=getattr(app.state, "recovery", "pending"),
|
||||
recovery_failed_steps=getattr(app.state, "recovery_failed_steps", ()),
|
||||
embedding_switch=_embedding_switch_summary(app),
|
||||
)
|
||||
|
||||
@app.get("/health", response_model=HealthResponse, include_in_schema=False)
|
||||
@@ -3322,6 +3472,7 @@ def create_app(
|
||||
profile_id: str,
|
||||
request: ModelProfilePatch,
|
||||
) -> ModelProfile:
|
||||
_refuse_embedding_edit_during_switch(app, profile_id, request)
|
||||
try:
|
||||
return _model_profile_store(app).update(
|
||||
profile_id,
|
||||
@@ -3342,25 +3493,62 @@ def create_app(
|
||||
|
||||
@app.post(
|
||||
"/api/v1/model-profiles/{profile_id}/activate",
|
||||
response_model=ModelProfile,
|
||||
response_model=None,
|
||||
dependencies=protected,
|
||||
)
|
||||
async def activate_model_profile(profile_id: str) -> ModelProfile:
|
||||
async def activate_model_profile(profile_id: str) -> Any:
|
||||
# Asynchronous for the sake of what follows the activation: scheduling
|
||||
# work needs the event loop this coroutine runs on, and a synchronous
|
||||
# path operation is handed to a worker thread that has none. Activating
|
||||
# with any evidence already imported — the ordinary order, since
|
||||
# importing comes first — used to fail here outright.
|
||||
#
|
||||
# Activation never changes the embedding space by itself. A profile
|
||||
# with the live space's model and width (a generation-only change, or
|
||||
# another endpoint for the same model) activates as before. Another
|
||||
# embedding on a home with no vector at all moves the space in the
|
||||
# same transaction. Anything else answers 409 `embedding_switch_required`
|
||||
# with the plan: `POST /api/v1/embedding-switches` carries it out.
|
||||
from one_memory.embedding_switch import EmbeddingSwitchError
|
||||
from one_memory.embedding_switch_routes import refusal
|
||||
|
||||
switch = _embedding_switch(app)
|
||||
try:
|
||||
profile = await asyncio.to_thread(
|
||||
_model_profile_store(app).activate,
|
||||
profile_id,
|
||||
_secret_store(app),
|
||||
)
|
||||
# The refusals activation always made come first, with their codes.
|
||||
await asyncio.to_thread(_store_activation_preflight, app, profile_id)
|
||||
plan = await switch.plan(profile_id)
|
||||
except KeyError:
|
||||
raise HTTPException(status_code=404, detail="model_profile_not_found") from None
|
||||
except ModelProfileUnavailableError as error:
|
||||
raise HTTPException(status_code=409, detail=str(error)) from None
|
||||
except EmbeddingSwitchError as error:
|
||||
if error.code == "model_profile_not_found":
|
||||
raise HTTPException(status_code=404, detail="model_profile_not_found") from None
|
||||
return refusal(error.code, plan=error.plan)
|
||||
if "embedding_switch_running" in plan.blockers:
|
||||
return refusal("embedding_switch_running", plan=plan)
|
||||
try:
|
||||
if plan.needed:
|
||||
return refusal("embedding_switch_required", plan=plan)
|
||||
if plan.reason == "empty_home":
|
||||
await asyncio.to_thread(switch.activate_empty_home, profile_id)
|
||||
profile = await asyncio.to_thread(
|
||||
_model_profile_store(app).require, profile_id, _secret_store(app)
|
||||
)
|
||||
else:
|
||||
profile = await asyncio.to_thread(
|
||||
_model_profile_store(app).activate,
|
||||
profile_id,
|
||||
_secret_store(app),
|
||||
)
|
||||
except KeyError:
|
||||
raise HTTPException(status_code=404, detail="model_profile_not_found") from None
|
||||
except ModelProfileUnavailableError as error:
|
||||
raise HTTPException(status_code=409, detail=str(error)) from None
|
||||
except ModelProfileConflictError:
|
||||
raise HTTPException(status_code=409, detail="model_profile_conflict") from None
|
||||
except EmbeddingSwitchError as error:
|
||||
return refusal(error.code)
|
||||
# D75: work queued for the previous model continues with this one.
|
||||
# Only tasks that never reached the model are re-bound.
|
||||
await asyncio.to_thread(app.state.memory_task_store.rebind_unstarted_profile, profile)
|
||||
@@ -3933,6 +4121,7 @@ def create_app(
|
||||
if request.task is not None and request.kind in {"chat", "fact"}:
|
||||
raise HTTPException(status_code=422, detail="memory_task_requires_experience")
|
||||
namespace = _namespace_kwargs(request.kind, request.variant)
|
||||
_refuse_write_during_embedding_switch(app)
|
||||
try:
|
||||
async with app.state.sdk_write_coordinator.use_client() as client:
|
||||
# `update` takes no user: read the memory first and refuse
|
||||
@@ -3988,6 +4177,7 @@ def create_app(
|
||||
user_id: str = Query(default=DEFAULT_USER_ID, pattern=USER_ID_PATTERN),
|
||||
) -> Response:
|
||||
_check_memory_namespace(kind, variant)
|
||||
_refuse_write_during_embedding_switch(app)
|
||||
try:
|
||||
async with app.state.sdk_write_coordinator.use_client() as client:
|
||||
# `delete` takes no user. Another user's memory is answered
|
||||
@@ -4296,6 +4486,7 @@ def create_app(
|
||||
in_flight_count=in_flight_count,
|
||||
metrics=ProcessingMetrics.model_validate(store.processing_metrics()),
|
||||
pipeline=_pipeline_status(app),
|
||||
embedding_switch=_embedding_switch_summary(app),
|
||||
)
|
||||
|
||||
@app.get(
|
||||
@@ -4408,6 +4599,53 @@ def create_app(
|
||||
changed=changed,
|
||||
)
|
||||
|
||||
@app.get(
|
||||
"/api/v1/settings/ui-language",
|
||||
response_model=UiLanguageState,
|
||||
dependencies=strict,
|
||||
)
|
||||
def get_ui_language() -> UiLanguageState:
|
||||
"""The App's display language and the SDK prompt language it implies."""
|
||||
return _ui_language_state(app)
|
||||
|
||||
@app.put(
|
||||
"/api/v1/settings/ui-language",
|
||||
response_model=UiLanguageChange,
|
||||
dependencies=strict,
|
||||
)
|
||||
async def put_ui_language(request: UiLanguageUpdate) -> UiLanguageChange:
|
||||
"""Record the App's display language; one global setting, not per user.
|
||||
|
||||
Every code (zh-Hans and en included) is passed as
|
||||
`MemoryConfig.language`; only null passes no language, so the SDK
|
||||
keeps detecting zh/en as before. A change of that SDK language rebuilds the
|
||||
SDK clients like a feature change: a write in flight finishes in the
|
||||
old language. Nothing stored is migrated and no collection is renamed.
|
||||
"""
|
||||
store = _config_store(app)
|
||||
async with app.state.memory_mode_lock:
|
||||
previous = _sdk_language(app)
|
||||
if store.ui_language() != request.ui_language:
|
||||
await asyncio.to_thread(store.set_ui_language, request.ui_language)
|
||||
current = _sdk_language(app)
|
||||
rebuilt = current != previous
|
||||
if rebuilt:
|
||||
coordinator = getattr(app.state, "sdk_write_coordinator", None)
|
||||
if coordinator is not None:
|
||||
await coordinator.retire_stale_handles()
|
||||
# Reads rebuild on their next request (`ActiveProfileReadHandle`).
|
||||
diagnostic_event(
|
||||
level="info",
|
||||
component="memory.language",
|
||||
event="sdk_language.changed",
|
||||
outcome="succeeded",
|
||||
details={
|
||||
"sdk_language": current or SDK_LANGUAGE_AUTO,
|
||||
"previous": previous or SDK_LANGUAGE_AUTO,
|
||||
},
|
||||
)
|
||||
return UiLanguageChange(**_ui_language_state(app).model_dump(), rebuilt=rebuilt)
|
||||
|
||||
@app.get(
|
||||
"/api/v1/settings/evolve-schedule",
|
||||
response_model=EvolveSchedule,
|
||||
@@ -4545,6 +4783,7 @@ def create_app(
|
||||
assumption that cost a live graph once was made in code, not by a user.
|
||||
"""
|
||||
supervisor = _sdk_memory_runtime(app)
|
||||
_refuse_write_during_embedding_switch(app)
|
||||
try:
|
||||
outcome = await supervisor.decide_graph_width(request.decision)
|
||||
except HySdkError as error:
|
||||
@@ -5768,6 +6007,8 @@ def _sdk_client_handle(app: FastAPI) -> SdkClientHandle:
|
||||
data_dir=cast(Settings, app.state.settings).data_dir,
|
||||
memory_mode=lambda: _memory_mode(app),
|
||||
features=lambda: _memory_features(app),
|
||||
spaces=getattr(app.state, "embedding_spaces", None),
|
||||
language=lambda: _sdk_language(app),
|
||||
)
|
||||
app.state.sdk_client_handle = handle
|
||||
return handle
|
||||
@@ -5810,6 +6051,20 @@ def _memory_features(app: FastAPI) -> FeatureOverrides:
|
||||
return store.memory_features().overrides() if store is not None else ()
|
||||
|
||||
|
||||
def _sdk_language(app: FastAPI) -> str | None:
|
||||
"""The SDK prompt language every client is built with; None = auto-detect."""
|
||||
store = getattr(app.state, "config_store", None)
|
||||
return sdk_language(store.ui_language()) if store is not None else None
|
||||
|
||||
|
||||
def _ui_language_state(app: FastAPI) -> UiLanguageState:
|
||||
store = getattr(app.state, "config_store", None)
|
||||
return UiLanguageState(
|
||||
ui_language=store.ui_language() if store is not None else None,
|
||||
sdk_language=_sdk_language(app) or SDK_LANGUAGE_AUTO,
|
||||
)
|
||||
|
||||
|
||||
def _memory_features_state(
|
||||
app: FastAPI, defaults: dict[str, bool] | None = None
|
||||
) -> MemoryFeaturesState:
|
||||
@@ -5865,9 +6120,14 @@ def _pipeline_status(app: FastAPI) -> PipelineStatus | None:
|
||||
dict[MemoryFeatureName, bool],
|
||||
non_default_features(_memory_features(app), sdk_feature_defaults()),
|
||||
),
|
||||
**_ui_language_state(app).model_dump(exclude={"languages"}),
|
||||
)
|
||||
|
||||
|
||||
def _read_handle(app: FastAPI) -> ActiveProfileReadHandle:
|
||||
return cast(ActiveProfileReadHandle, _sdk_client_handle(app))
|
||||
|
||||
|
||||
def _log_skipped_sdk_memories(count: int) -> None:
|
||||
if count:
|
||||
logger.warning("sdk_memory_invalid_rows_skipped count=%d", count)
|
||||
@@ -6123,6 +6383,7 @@ def _sedimentation_claim_enabled(app: FastAPI, source_id: str) -> bool:
|
||||
|
||||
return (
|
||||
_config_store(app).sedimentation_enabled()
|
||||
and _embedding_writes_open(app)
|
||||
and (_memory_mode(app) == "lite" or _model_profile_store(app).active_identity() is not None)
|
||||
and _history_import_gate_released(app)
|
||||
and _source_writes_on(app, source_id)
|
||||
@@ -6137,6 +6398,74 @@ def _user_evolving(app: FastAPI, user_id: str) -> bool:
|
||||
return bool(callable(evolving) and evolving(user_id))
|
||||
|
||||
|
||||
def _embedding_switch(app: FastAPI) -> EmbeddingSwitchService:
|
||||
return cast(EmbeddingSwitchService, app.state.embedding_switch)
|
||||
|
||||
|
||||
def _embedding_writes_open(app: FastAPI) -> bool:
|
||||
"""False while an embedding switch or merge runs, or the startup scan has not decided.
|
||||
|
||||
Intake is not affected: conversations keep being queued, and sediment
|
||||
into the live space once the gate opens again.
|
||||
"""
|
||||
switch = getattr(app.state, "embedding_switch", None)
|
||||
return switch is None or switch.writes_open()
|
||||
|
||||
|
||||
def _embedding_switch_summary(app: FastAPI) -> dict[str, object] | None:
|
||||
switch = getattr(app.state, "embedding_switch", None)
|
||||
if switch is None:
|
||||
return None
|
||||
try:
|
||||
return cast(EmbeddingSwitchService, switch).summary()
|
||||
except sqlite3.Error:
|
||||
return None
|
||||
|
||||
|
||||
def _refuse_write_during_embedding_switch(app: FastAPI) -> None:
|
||||
"""Direct edits and deletes wait for the switch: 409, never a write into the old space."""
|
||||
if not _embedding_writes_open(app):
|
||||
raise HTTPException(status_code=409, detail="embedding_switch_in_progress")
|
||||
|
||||
|
||||
_EMBEDDING_PATCH_FIELDS = frozenset(
|
||||
{"embedding_model", "embedding_provider_kind", "embedding_base_url", "embedding_dimensions"}
|
||||
)
|
||||
|
||||
|
||||
def _refuse_embedding_edit_during_switch(
|
||||
app: FastAPI, profile_id: str, patch: ModelProfilePatch
|
||||
) -> None:
|
||||
"""The profiles a running switch reads from or commits to keep their embedding."""
|
||||
if not (_EMBEDDING_PATCH_FIELDS & patch.model_fields_set):
|
||||
return
|
||||
switch = getattr(app.state, "embedding_switch", None)
|
||||
if switch is None:
|
||||
return
|
||||
row = cast(EmbeddingSwitchService, switch).jobs.open_job()
|
||||
if row is None:
|
||||
return
|
||||
active = _model_profile_store(app).active_identity()
|
||||
if profile_id == row["target_profile_id"] or (active is not None and active[0] == profile_id):
|
||||
raise HTTPException(status_code=409, detail="embedding_switch_running")
|
||||
|
||||
|
||||
def _store_activation_preflight(app: FastAPI, profile_id: str) -> None:
|
||||
"""The checks `ModelProfileStore.activate` makes, for the empty-home path."""
|
||||
profile = _model_profile_store(app).require(profile_id, _secret_store(app))
|
||||
if profile.provider_kind in {ProviderKind.HY_DEFAULT, ProviderKind.ANTHROPIC}:
|
||||
raise ModelProfileUnavailableError("model_provider_not_supported")
|
||||
if not profile.embedding_model:
|
||||
raise ModelProfileUnavailableError("embedding_model_required")
|
||||
capabilities = profile.capabilities
|
||||
if not (
|
||||
capabilities.endpoint_reachable
|
||||
and capabilities.model_available
|
||||
and capabilities.embedding_available
|
||||
):
|
||||
raise ModelProfileUnavailableError("model_profile_probe_required")
|
||||
|
||||
|
||||
def _history_import_gate_released(app: FastAPI) -> bool:
|
||||
"""Hold first sedimentation until every accepted history import is terminal."""
|
||||
|
||||
|
||||
@@ -35,7 +35,7 @@ import re # noqa: E402
|
||||
import time # noqa: E402
|
||||
from collections.abc import Iterator, Sequence # noqa: E402
|
||||
from pathlib import Path # noqa: E402
|
||||
from typing import TextIO # noqa: E402
|
||||
from typing import Any, TextIO # noqa: E402
|
||||
|
||||
from one_memory.config import load_settings # noqa: E402
|
||||
from one_memory.runtime_client import RuntimeStatus, inspect_runtime # noqa: E402
|
||||
@@ -609,6 +609,14 @@ def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
|
||||
else:
|
||||
state = "required"
|
||||
document = {"schema_version": 1, "authority": authority.value, "state": state}
|
||||
from one_memory.embedding_switch import read_open_job
|
||||
|
||||
# Informational. An unfinished embedding switch never makes a
|
||||
# migration required or blocks it: the upgrade installs, `sdk-migrate`
|
||||
# leaves the switch's tables alone, and the new Core resumes the job.
|
||||
switch = read_open_job(database_path)
|
||||
if switch is not None:
|
||||
document["embedding_switch"] = switch
|
||||
from one_memory.sdk_migration_log import write_migration_log
|
||||
|
||||
write_migration_log(
|
||||
@@ -616,7 +624,15 @@ def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
|
||||
writer="core-sdk-migration-status",
|
||||
event="status.probed",
|
||||
outcome="succeeded",
|
||||
details={"authority": document["authority"], "state": document["state"]},
|
||||
details={
|
||||
"authority": document["authority"],
|
||||
"state": document["state"],
|
||||
**(
|
||||
{"embedding_switch": document["embedding_switch"]}
|
||||
if "embedding_switch" in document
|
||||
else {}
|
||||
),
|
||||
},
|
||||
)
|
||||
if as_json:
|
||||
stream.write(f"{json.dumps(document, ensure_ascii=False)}\n")
|
||||
@@ -696,6 +712,12 @@ def _refuse_unmigrated_home(data_dir: Path) -> None:
|
||||
raise SystemExit(SDK_MIGRATION_REQUIRED_EXIT)
|
||||
|
||||
|
||||
def _without_embedding_switch(settings: Any) -> Any:
|
||||
"""The maintenance Core's settings: it never resumes or runs a switch."""
|
||||
copy = getattr(settings, "model_copy", None)
|
||||
return copy(update={"embedding_switch_worker": False}) if callable(copy) else settings
|
||||
|
||||
|
||||
def serve(*, maintenance: bool = False) -> None:
|
||||
import uvicorn
|
||||
|
||||
@@ -740,7 +762,8 @@ def serve(*, maintenance: bool = False) -> None:
|
||||
if launcher is not None:
|
||||
ParentWatchdog(launcher).start()
|
||||
application = create_app(
|
||||
settings,
|
||||
# Only the serving Core runs or resumes an embedding switch.
|
||||
_without_embedding_switch(settings) if maintenance else settings,
|
||||
manage_runtime=not maintenance,
|
||||
)
|
||||
if maintenance:
|
||||
|
||||
@@ -45,6 +45,11 @@ class Settings(BaseSettings):
|
||||
model_request_limit: int | None = Field(default=None, ge=1, le=100_000)
|
||||
model_request_body_byte_limit: int = Field(default=24_576, ge=1)
|
||||
model_request_output_token_limit: int = Field(default=16_384, ge=1)
|
||||
# The embedding switch worker (and the startup merge). The maintenance
|
||||
# Core turns it off: only the serving Core may resume or run a switch.
|
||||
embedding_switch_worker: bool = True
|
||||
# How long a switch waits for model writes already past their claim.
|
||||
embedding_switch_drain_timeout_seconds: float = Field(default=600, gt=0)
|
||||
|
||||
@field_validator("development_origin")
|
||||
@classmethod
|
||||
|
||||
@@ -19,6 +19,7 @@ from one_memory.context_models import (
|
||||
SdkMemoryContextItem,
|
||||
SdkRawContextItem,
|
||||
)
|
||||
from one_memory.embedding_health import EMBEDDING_UNAVAILABLE, EmbeddingUnavailableError
|
||||
from one_memory.external_memory_access import external_access
|
||||
from one_memory.hy_runtime_config import HY_MEMORY_READER
|
||||
from one_memory.hy_sdk_runtime import sdk_relevance_floor
|
||||
@@ -263,6 +264,10 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
)
|
||||
await record(final, timed_out=True)
|
||||
return final
|
||||
for lane in (raw_items, memory_items):
|
||||
if isinstance(lane, EmbeddingUnavailableError):
|
||||
# No lexical fallback: the query could not be embedded.
|
||||
raise HTTPException(503, EMBEDDING_UNAVAILABLE)
|
||||
partial = isinstance(raw_items, Exception) or isinstance(memory_items, Exception)
|
||||
raw_candidates = (
|
||||
() if isinstance(raw_items, Exception) else cast(tuple[SdkContextItem, ...], raw_items)
|
||||
|
||||
@@ -0,0 +1,180 @@
|
||||
"""What the embedding switch needs from the SDK's re-embed module, in Core terms.
|
||||
|
||||
The switch worker, the startup merge and the retention sweep only talk to a
|
||||
`ReembedBackend`. The production one (`hy_sdk_migration.SdkReembedBackend`)
|
||||
is the single place that calls `hy_memory`'s re-embed functions; tests use
|
||||
an in-memory one. Every space is named explicitly -- collection names and
|
||||
graph paths -- and nothing here derives a name from a provider or endpoint.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Mapping
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Protocol
|
||||
|
||||
from one_memory.embedding_spaces import EmbeddingSpace, space_collection_names
|
||||
from one_memory.model_profiles import ModelProfileSnapshot
|
||||
|
||||
|
||||
class ReembedUnavailableError(RuntimeError):
|
||||
"""The installed SDK has no re-embed module (before hy-memory 2.0.10)."""
|
||||
|
||||
def __init__(self, code: str = "sdk_reembed_unavailable") -> None:
|
||||
super().__init__(code)
|
||||
self.code = code
|
||||
|
||||
|
||||
class ReembedCancelledError(RuntimeError):
|
||||
"""A cancel was seen at a batch boundary; every finished batch is written."""
|
||||
|
||||
|
||||
class EmbeddingWidthError(RuntimeError):
|
||||
"""The embedding service returned vectors of another width than the target's."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PhysicalSpace:
|
||||
"""One space's collections, named explicitly."""
|
||||
|
||||
collection_base: str
|
||||
name_dims: int | None
|
||||
#: The width of the vectors in it, or to be written to it.
|
||||
dims: int | None
|
||||
model: str | None = None
|
||||
#: Explicit role -> collection name; None derives them from the base.
|
||||
collections: Mapping[str, str] | None = None
|
||||
|
||||
def names(self) -> dict[str, str]:
|
||||
if self.collections is not None:
|
||||
return dict(self.collections)
|
||||
return space_collection_names(self.collection_base, self.name_dims)
|
||||
|
||||
@classmethod
|
||||
def of(cls, space: EmbeddingSpace) -> PhysicalSpace:
|
||||
return cls(
|
||||
collection_base=space.collection_base,
|
||||
name_dims=space.name_dims,
|
||||
dims=space.dims if space.dims is not None else space.name_dims,
|
||||
model=space.model,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ScannedSpace:
|
||||
"""A space found in the vector store, grouped from its collection names."""
|
||||
|
||||
collection_base: str
|
||||
name_dims: int | None
|
||||
#: The stored width when every non-empty role agrees.
|
||||
width: int | None
|
||||
counts: Mapping[str, int]
|
||||
collections: Mapping[str, str]
|
||||
per_user: Mapping[str, int] = field(default_factory=dict)
|
||||
|
||||
@property
|
||||
def vectors(self) -> int:
|
||||
return sum(self.counts.values())
|
||||
|
||||
@property
|
||||
def key(self) -> tuple[str, int]:
|
||||
return (self.collection_base, self.name_dims or 0)
|
||||
|
||||
def physical(self) -> PhysicalSpace:
|
||||
return PhysicalSpace(
|
||||
collection_base=self.collection_base,
|
||||
name_dims=self.name_dims,
|
||||
dims=self.width if self.width is not None else self.name_dims,
|
||||
collections=dict(self.collections),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ReembedStep:
|
||||
"""Progress of one role (vectors) or table (graph)."""
|
||||
|
||||
phase: str
|
||||
role: str
|
||||
done: int
|
||||
total: int
|
||||
embedded: int = 0
|
||||
reused: int = 0
|
||||
kept: int = 0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VectorCheck:
|
||||
per_role: Mapping[str, Mapping[str, int]]
|
||||
#: Target records per metadata `user_id`.
|
||||
per_user: Mapping[str, int]
|
||||
problems: tuple[str, ...]
|
||||
warnings: tuple[str, ...] = ()
|
||||
self_recall_hits: int = 0
|
||||
self_recall_samples: int = 0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GraphCheck:
|
||||
ok: bool
|
||||
problems: tuple[str, ...] = ()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EmbeddingTarget:
|
||||
"""The embedding that fills a space: the profile to reach it, and its key."""
|
||||
|
||||
profile: ModelProfileSnapshot
|
||||
space: EmbeddingSpace
|
||||
api_key: str | None
|
||||
|
||||
|
||||
Cancel = Callable[[], bool]
|
||||
OnStep = Callable[[ReembedStep], None]
|
||||
|
||||
|
||||
class ReembedBackend(Protocol):
|
||||
def unavailable_reason(self) -> str | None:
|
||||
"""None when re-embedding works; otherwise the code that says why not."""
|
||||
|
||||
async def scan(self, *, per_user: bool = False) -> tuple[ScannedSpace, ...]:
|
||||
"""Every space with collections in the home's vector store. Read-only."""
|
||||
|
||||
async def diff(self, source: PhysicalSpace, target: PhysicalSpace) -> dict[str, int]:
|
||||
"""Per role, how many ids the source holds that the target does not."""
|
||||
|
||||
async def reembed(
|
||||
self,
|
||||
source: PhysicalSpace,
|
||||
target: PhysicalSpace,
|
||||
*,
|
||||
embedding: EmbeddingTarget,
|
||||
cancel: Cancel,
|
||||
on_step: OnStep,
|
||||
batch_size: int = 64,
|
||||
) -> None:
|
||||
"""Embed every source record the target lacks into the target. Target wins."""
|
||||
|
||||
async def verify(
|
||||
self, source: PhysicalSpace, target: PhysicalSpace, *, embedding: EmbeddingTarget
|
||||
) -> VectorCheck: ...
|
||||
|
||||
async def graph_vectors(self, path: Path) -> int | None:
|
||||
"""Embeddings a graph holds, or None when there is no graph there."""
|
||||
|
||||
async def reembed_graph(
|
||||
self,
|
||||
source: Path,
|
||||
target: Path,
|
||||
*,
|
||||
embedding: EmbeddingTarget,
|
||||
target_dims: int,
|
||||
cancel: Cancel,
|
||||
on_step: OnStep,
|
||||
) -> None: ...
|
||||
|
||||
async def verify_graph(self, source: Path, target: Path, *, target_dims: int) -> GraphCheck: ...
|
||||
|
||||
async def drop(self, space: PhysicalSpace) -> None: ...
|
||||
|
||||
async def drop_graph(self, path: Path) -> None: ...
|
||||
@@ -0,0 +1,147 @@
|
||||
"""`embedding_unavailable`: a search that could not embed its query says so.
|
||||
|
||||
With a remote embedding model, every search embeds the query through that
|
||||
service. When the service is down, the SDK swallows the failure and answers
|
||||
with no hits, which reads as "you have no such memory". There is no lexical
|
||||
fallback (owner decision): the search fails with the explicit code
|
||||
`embedding_unavailable`, the API answers 503 with that code, and the App
|
||||
shows a toast.
|
||||
|
||||
The check is only made when a search came back empty or raised, so a working
|
||||
service costs nothing on searches that find something. It sends one tiny
|
||||
embedding request to the same endpoint, model and width the read client
|
||||
uses; the answer is remembered briefly so a burst of searches sends one.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
import time
|
||||
from collections.abc import Callable, Mapping
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from one_memory.http_client import create_client
|
||||
from one_memory.hy_sdk_types import HySdkError
|
||||
from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind
|
||||
|
||||
EMBEDDING_UNAVAILABLE = "embedding_unavailable"
|
||||
_OK_TTL_S = 30.0
|
||||
_FAILED_TTL_S = 10.0
|
||||
_TIMEOUT_S = 8.0
|
||||
|
||||
|
||||
def _no_transport() -> httpx.BaseTransport | None:
|
||||
return None
|
||||
|
||||
|
||||
#: Tests replace this to answer the check without a network.
|
||||
transport_factory: Callable[[], httpx.BaseTransport | None] = _no_transport
|
||||
|
||||
|
||||
class EmbeddingUnavailableError(HySdkError):
|
||||
"""The live space's embedding service did not answer; search cannot run."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__(EMBEDDING_UNAVAILABLE)
|
||||
|
||||
|
||||
class EmbeddingHealth:
|
||||
"""Whether a remote embedding answers, remembered for a few seconds."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._lock = threading.Lock()
|
||||
self._known: dict[tuple[str, str, int | None], tuple[bool, float]] = {}
|
||||
|
||||
def available(
|
||||
self,
|
||||
profile: ModelProfileSnapshot,
|
||||
*,
|
||||
endpoint: str | None,
|
||||
name_dims: int | None,
|
||||
api_key: str | None,
|
||||
) -> bool:
|
||||
provider = profile.embedding_provider_kind or profile.provider_kind
|
||||
if provider is ProviderKind.LOCAL_ONNX or endpoint is None:
|
||||
return True
|
||||
model = profile.embedding_model or ""
|
||||
key = (endpoint, model, name_dims)
|
||||
now = time.monotonic()
|
||||
with self._lock:
|
||||
known = self._known.get(key)
|
||||
if known is not None and known[1] > now:
|
||||
return known[0]
|
||||
ok = _answers(
|
||||
endpoint,
|
||||
model=model,
|
||||
name_dims=name_dims,
|
||||
api_key="ollama" if provider is ProviderKind.OLLAMA else api_key,
|
||||
)
|
||||
with self._lock:
|
||||
self._known[key] = (ok, now + (_OK_TTL_S if ok else _FAILED_TTL_S))
|
||||
return ok
|
||||
|
||||
def forget(self) -> None:
|
||||
with self._lock:
|
||||
self._known.clear()
|
||||
|
||||
|
||||
def _answers(endpoint: str, *, model: str, name_dims: int | None, api_key: str | None) -> bool:
|
||||
body: dict[str, Any] = {"model": model, "input": ["ping"]}
|
||||
if name_dims:
|
||||
body["dimensions"] = name_dims
|
||||
headers = {"Authorization": f"Bearer {api_key}"} if api_key else {}
|
||||
transport = transport_factory()
|
||||
try:
|
||||
with create_client(
|
||||
timeout=_TIMEOUT_S,
|
||||
**({"transport": transport} if transport is not None else {}),
|
||||
) as client:
|
||||
response = client.post(f"{endpoint.rstrip('/')}/embeddings", json=body, headers=headers)
|
||||
if response.status_code != 200:
|
||||
return False
|
||||
data = response.json().get("data")
|
||||
return bool(isinstance(data, list) and data and data[0].get("embedding"))
|
||||
except (httpx.HTTPError, ValueError, AttributeError):
|
||||
return False
|
||||
|
||||
|
||||
def has_hits(result: object) -> bool:
|
||||
"""Whether an SDK search result holds any hit, whatever its scene's shape."""
|
||||
if not isinstance(result, Mapping):
|
||||
return bool(result)
|
||||
for key in ("memories", "facts", "experiences"):
|
||||
value = result.get(key)
|
||||
if isinstance(value, list) and value:
|
||||
return True
|
||||
if isinstance(value, Mapping) and any(
|
||||
isinstance(item, list) and item for item in value.values()
|
||||
):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
class EmbeddingGuardedClient:
|
||||
"""An SDK client whose `search` reports a dead embedding instead of nothing.
|
||||
|
||||
Everything else is the client's own attribute, unchanged.
|
||||
"""
|
||||
|
||||
def __init__(self, client: Any, available: Callable[[], bool]) -> None:
|
||||
self._client = client
|
||||
self._available = available
|
||||
|
||||
def search(self, *args: Any, **kwargs: Any) -> Any:
|
||||
try:
|
||||
result = self._client.search(*args, **kwargs)
|
||||
except Exception:
|
||||
if not self._available():
|
||||
raise EmbeddingUnavailableError() from None
|
||||
raise
|
||||
if not has_hits(result) and not self._available():
|
||||
raise EmbeddingUnavailableError()
|
||||
return result
|
||||
|
||||
def __getattr__(self, name: str) -> Any:
|
||||
return getattr(self._client, name)
|
||||
@@ -0,0 +1,554 @@
|
||||
"""Which embedding space the home reads and writes: an explicit registry.
|
||||
|
||||
The SDK names its vector collections after `vector_store.collection_name`
|
||||
and the configured width, so the space used to follow whatever the active
|
||||
profile hashed to (`_collection_base`). Any change to that identity -- a
|
||||
probe recording the measured width, a new endpoint -- silently opened new,
|
||||
empty collections beside the old ones, and nothing carried the memories over.
|
||||
|
||||
The live space is pinned here instead, in the main Core database beside the
|
||||
model profiles, so a switch commits the new space and the profile that fills
|
||||
it in one transaction. The identity of a space is its embedding model and
|
||||
width only: a new endpoint or provider serving the same model at the same
|
||||
width keeps the space.
|
||||
|
||||
Existing homes keep their collections byte-for-byte. The first start registers
|
||||
the space the active profile (or the built-in model) already opened under its
|
||||
existing four-attribute name; only spaces created by a switch use the
|
||||
model-and-width name. A space whose width was never declared ("auto") keeps
|
||||
its unsuffixed name and only records the width it was measured at.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import sqlite3
|
||||
from collections.abc import Iterator, Mapping
|
||||
from contextlib import contextmanager
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
from uuid import uuid4
|
||||
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
from one_memory.database import take_the_write_lock_first
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.embedding_assets import BUILTIN_EMBEDDING_DIMENSIONS, BUILTIN_EMBEDDING_MODEL
|
||||
from one_memory.memory_server.migrations import embedding_spaces_ddl
|
||||
from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind
|
||||
|
||||
#: How long a space replaced by a switch or folded in by a merge is kept.
|
||||
RETIRED_SPACE_RETENTION = timedelta(days=14)
|
||||
|
||||
SpaceState = Literal["live", "building", "stray", "retired", "dropped"]
|
||||
SpaceNaming = Literal["legacy", "v2", "discovered"]
|
||||
|
||||
#: Every vector collection one space consists of, as the SDK names them.
|
||||
VECTOR_ROLES = (
|
||||
"memory",
|
||||
"entities",
|
||||
"coding_keys",
|
||||
"coding_keys_cv",
|
||||
"working_facts",
|
||||
"working_facts_cv",
|
||||
)
|
||||
|
||||
#: The graph directory of the first space; later spaces get their own.
|
||||
DEFAULT_GRAPH_DIR = "graph"
|
||||
|
||||
EMBEDDING_SPACES_SCHEMA = embedding_spaces_ddl()
|
||||
|
||||
|
||||
#: Bumped by every registry write in this process, so cached live rows are
|
||||
#: reread after a change made through any connection (a probe's, a commit's).
|
||||
_CHANGES = [0]
|
||||
|
||||
|
||||
def _changed() -> None:
|
||||
_CHANGES[0] += 1
|
||||
|
||||
|
||||
class EmbeddingSpaceError(RuntimeError):
|
||||
"""A registry refusal with a stable, product-facing code."""
|
||||
|
||||
def __init__(self, code: str) -> None:
|
||||
super().__init__(code)
|
||||
self.code = code
|
||||
|
||||
|
||||
class EmbeddingIdentity(BaseModel):
|
||||
"""What makes two vectors comparable: the model and the width.
|
||||
|
||||
`dims=None` is a width nobody declared or measured yet; it matches any
|
||||
width, because the collection it names was opened without one.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
model: str
|
||||
dims: int | None = None
|
||||
|
||||
def matches(self, other: EmbeddingIdentity) -> bool:
|
||||
if self.model != other.model:
|
||||
return False
|
||||
return self.dims is None or other.dims is None or self.dims == other.dims
|
||||
|
||||
|
||||
class EmbeddingSpace(BaseModel):
|
||||
"""One embedding space: the physical collections and graph one model fills."""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
space_id: str
|
||||
state: SpaceState
|
||||
#: None only for a space found on disk that no registry row named.
|
||||
model: str | None = None
|
||||
#: The vector width stored in this space, once known.
|
||||
dims: int | None = None
|
||||
#: The width the collection names and embedding requests carry. None is
|
||||
#: an "auto" space: unsuffixed names and no `dimensions` request field.
|
||||
name_dims: int | None = None
|
||||
collection_base: str
|
||||
naming: SpaceNaming = "legacy"
|
||||
graph_dir: str | None = DEFAULT_GRAPH_DIR
|
||||
provider_kind: str | None = None
|
||||
#: The profile version whose embedding fills this space; None = built-in.
|
||||
profile_id: str | None = None
|
||||
profile_revision: int | None = None
|
||||
#: Vectors counted the last time the space was scanned.
|
||||
vectors: int | None = None
|
||||
created_at: str
|
||||
live_since: str | None = None
|
||||
retired_at: str | None = None
|
||||
keep_until: str | None = None
|
||||
dropped_at: str | None = None
|
||||
|
||||
@property
|
||||
def identity(self) -> EmbeddingIdentity | None:
|
||||
if self.model is None:
|
||||
return None
|
||||
return EmbeddingIdentity(model=self.model, dims=self.dims)
|
||||
|
||||
@property
|
||||
def builtin(self) -> bool:
|
||||
return self.provider_kind == ProviderKind.LOCAL_ONNX.value and self.profile_id is None
|
||||
|
||||
def collection_names(self) -> dict[str, str]:
|
||||
return space_collection_names(self.collection_base, self.name_dims)
|
||||
|
||||
|
||||
def space_collection_names(base: str, name_dims: int | None) -> dict[str, str]:
|
||||
"""The SDK's physical collection name per role for one space.
|
||||
|
||||
Mirrors `VectorStoreBase` (the `_<dims>` suffix), the entity store and the
|
||||
coding stores. The SDK's own `vector_space_names` is checked against this
|
||||
in the migration adapter when the installed SDK provides it.
|
||||
"""
|
||||
suffix = f"_{name_dims}" if name_dims else ""
|
||||
return {
|
||||
"memory": f"{base}{suffix}",
|
||||
"entities": f"{base}{suffix}_entities",
|
||||
"coding_keys": f"{base}_coding_keys{suffix}",
|
||||
"coding_keys_cv": f"{base}_coding_keys_cv{suffix}",
|
||||
"working_facts": f"{base}_working_facts{suffix}",
|
||||
"working_facts_cv": f"{base}_working_facts_cv{suffix}",
|
||||
}
|
||||
|
||||
|
||||
def embedding_identity(profile: ModelProfileSnapshot) -> EmbeddingIdentity:
|
||||
"""The embedding identity a profile asks for: model name and width only."""
|
||||
provider = profile.embedding_provider_kind or profile.provider_kind
|
||||
local = provider is ProviderKind.LOCAL_ONNX
|
||||
model = profile.embedding_model or (BUILTIN_EMBEDDING_MODEL if local else None)
|
||||
if not model:
|
||||
raise EmbeddingSpaceError("hy_sdk_embedding_model_required")
|
||||
dims = profile.embedding_dimensions
|
||||
if local and dims is None:
|
||||
dims = BUILTIN_EMBEDDING_DIMENSIONS
|
||||
return EmbeddingIdentity(model=model, dims=dims)
|
||||
|
||||
|
||||
def v2_collection_base(identity: EmbeddingIdentity, generation: int = 0) -> str:
|
||||
"""The base name a switch gives a new space: model and width, nothing else."""
|
||||
digest = hashlib.sha256(f"{identity.model}\0{identity.dims or 'auto'}".encode()).hexdigest()
|
||||
base = f"memhub_e_{digest[:16]}"
|
||||
return f"{base}_g{generation}" if generation else base
|
||||
|
||||
|
||||
def graph_dir_for(space_id: str) -> str:
|
||||
return f"graph-{space_id[:8]}"
|
||||
|
||||
|
||||
def derived_space(profile: ModelProfileSnapshot) -> EmbeddingSpace:
|
||||
"""The space `profile` opened before the registry existed, under its old name.
|
||||
|
||||
Never stored as is: bootstrap stores it, and offline tools that run beside
|
||||
a home with no registry yet read through it.
|
||||
"""
|
||||
from one_memory.hy_sdk_runtime import legacy_collection_base
|
||||
|
||||
provider = profile.embedding_provider_kind or profile.provider_kind
|
||||
identity = embedding_identity(profile)
|
||||
builtin = profile.profile_id == _BUILTIN_PROFILE_ID
|
||||
return EmbeddingSpace(
|
||||
space_id="derived",
|
||||
state="live",
|
||||
model=identity.model,
|
||||
dims=identity.dims,
|
||||
name_dims=identity.dims,
|
||||
collection_base=legacy_collection_base(profile),
|
||||
naming="legacy",
|
||||
graph_dir=DEFAULT_GRAPH_DIR,
|
||||
provider_kind=provider.value,
|
||||
profile_id=None if builtin else profile.profile_id,
|
||||
profile_revision=None if builtin else profile.revision,
|
||||
created_at=_now(),
|
||||
)
|
||||
|
||||
|
||||
_BUILTIN_PROFILE_ID = "builtin-retrieval"
|
||||
|
||||
|
||||
def _now() -> str:
|
||||
return datetime.now(UTC).isoformat()
|
||||
|
||||
|
||||
def _row(row: sqlite3.Row | Mapping[str, object]) -> EmbeddingSpace:
|
||||
return EmbeddingSpace.model_validate(dict(row))
|
||||
|
||||
|
||||
def ensure_schema(connection: sqlite3.Connection) -> None:
|
||||
connection.executescript(EMBEDDING_SPACES_SCHEMA)
|
||||
|
||||
|
||||
def registry_present(connection: sqlite3.Connection) -> bool:
|
||||
return (
|
||||
connection.execute(
|
||||
"SELECT 1 FROM sqlite_master WHERE type='table' AND name='embedding_spaces'"
|
||||
).fetchone()
|
||||
is not None
|
||||
)
|
||||
|
||||
|
||||
def live_space_in(connection: sqlite3.Connection) -> EmbeddingSpace | None:
|
||||
if not registry_present(connection):
|
||||
return None
|
||||
previous = connection.row_factory
|
||||
connection.row_factory = sqlite3.Row
|
||||
try:
|
||||
row = connection.execute("SELECT * FROM embedding_spaces WHERE state='live'").fetchone()
|
||||
finally:
|
||||
connection.row_factory = previous
|
||||
return _row(row) if row is not None else None
|
||||
|
||||
|
||||
def read_live_space(database_path: Path) -> EmbeddingSpace | None:
|
||||
"""The live space, read only, or None for a home with no registry yet."""
|
||||
if not database_path.is_file():
|
||||
return None
|
||||
with main_database(database_path).read(row_factory=sqlite3.Row) as connection:
|
||||
return live_space_in(connection)
|
||||
|
||||
|
||||
class EmbeddingSpaceStore:
|
||||
"""The `embedding_spaces` table of the main Core database."""
|
||||
|
||||
def __init__(self, database_path: Path) -> None:
|
||||
self.database_path = database_path
|
||||
#: The live row, read once per change: every read and write asks for
|
||||
#: it. Only this process writes the registry while Core serves.
|
||||
self._live_cache: tuple[int, EmbeddingSpace] | None = None
|
||||
with self._connect(writing=True) as connection:
|
||||
ensure_schema(connection)
|
||||
|
||||
def invalidate(self) -> None:
|
||||
_changed()
|
||||
|
||||
@contextmanager
|
||||
def _connect(
|
||||
self, *, writing: bool = False, lock_first: bool = True
|
||||
) -> Iterator[sqlite3.Connection]:
|
||||
database = main_database(self.database_path)
|
||||
checkout = (
|
||||
database.write(row_factory=sqlite3.Row)
|
||||
if writing
|
||||
else database.read(row_factory=sqlite3.Row)
|
||||
)
|
||||
with checkout as connection, connection:
|
||||
if writing and lock_first:
|
||||
take_the_write_lock_first(connection)
|
||||
yield connection
|
||||
|
||||
@contextmanager
|
||||
def transaction(self) -> Iterator[sqlite3.Connection]:
|
||||
"""One `BEGIN IMMEDIATE` on the main database, committed on success."""
|
||||
_changed()
|
||||
try:
|
||||
with self._connect(writing=True, lock_first=False) as connection:
|
||||
connection.execute("BEGIN IMMEDIATE")
|
||||
yield connection
|
||||
finally:
|
||||
_changed()
|
||||
|
||||
def live(self) -> EmbeddingSpace | None:
|
||||
generation = _CHANGES[0]
|
||||
cached = self._live_cache
|
||||
if cached is not None and cached[0] == generation:
|
||||
return cached[1]
|
||||
with self._connect() as connection:
|
||||
space = live_space_in(connection)
|
||||
self._live_cache = (generation, space) if space is not None else None
|
||||
return space
|
||||
|
||||
def require_live(self) -> EmbeddingSpace:
|
||||
space = self.live()
|
||||
if space is None:
|
||||
raise EmbeddingSpaceError("embedding_space_not_bootstrapped")
|
||||
return space
|
||||
|
||||
def get(self, space_id: str) -> EmbeddingSpace | None:
|
||||
with self._connect() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT * FROM embedding_spaces WHERE space_id=?", (space_id,)
|
||||
).fetchone()
|
||||
return _row(row) if row is not None else None
|
||||
|
||||
def list_spaces(self, *, include_dropped: bool = False) -> list[EmbeddingSpace]:
|
||||
with self._connect() as connection:
|
||||
rows = connection.execute(
|
||||
"SELECT * FROM embedding_spaces "
|
||||
+ ("" if include_dropped else "WHERE state != 'dropped' ")
|
||||
+ "ORDER BY created_at, space_id"
|
||||
).fetchall()
|
||||
return [_row(row) for row in rows]
|
||||
|
||||
def bootstrap(self, active: ModelProfileSnapshot | None) -> EmbeddingSpace:
|
||||
"""Register the space this home already uses, once, under its existing name.
|
||||
|
||||
`active` is the active profile, or None for the built-in model. The
|
||||
collection name is derived exactly as before the registry, so nothing
|
||||
on disk moves.
|
||||
"""
|
||||
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot
|
||||
|
||||
with self.transaction() as connection:
|
||||
existing = live_space_in(connection)
|
||||
if existing is not None:
|
||||
return existing
|
||||
derived = derived_space(active or builtin_retrieval_snapshot())
|
||||
space = derived.model_copy(
|
||||
update={"space_id": str(uuid4()), "live_since": derived.created_at}
|
||||
)
|
||||
_insert(connection, space)
|
||||
return space
|
||||
|
||||
def record_width(self, space_id: str, dims: int) -> EmbeddingSpace:
|
||||
"""A relabel: the space's stored width is now known. Names never change."""
|
||||
with self.transaction() as connection:
|
||||
record_width_in(connection, space_id, dims)
|
||||
space = self.get(space_id)
|
||||
if space is None: # pragma: no cover - the row was just updated
|
||||
raise EmbeddingSpaceError("embedding_space_not_found")
|
||||
return space
|
||||
|
||||
def record_vectors(self, counts: Mapping[str, int]) -> None:
|
||||
"""Cache the vector count per space id from the last scan (display only)."""
|
||||
if not counts:
|
||||
return
|
||||
_changed()
|
||||
with self._connect(writing=True) as connection:
|
||||
connection.executemany(
|
||||
"UPDATE embedding_spaces SET vectors=? WHERE space_id=?",
|
||||
[(count, space_id) for space_id, count in counts.items()],
|
||||
)
|
||||
|
||||
def register_stray(
|
||||
self,
|
||||
*,
|
||||
collection_base: str,
|
||||
name_dims: int | None,
|
||||
dims: int | None,
|
||||
vectors: int,
|
||||
) -> EmbeddingSpace:
|
||||
"""Name a non-empty space the scan found that no row described."""
|
||||
with self.transaction() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT * FROM embedding_spaces WHERE collection_base=? "
|
||||
"AND COALESCE(name_dims,0)=? AND state != 'dropped'",
|
||||
(collection_base, name_dims or 0),
|
||||
).fetchone()
|
||||
if row is not None:
|
||||
return _row(row)
|
||||
space = EmbeddingSpace(
|
||||
space_id=str(uuid4()),
|
||||
state="stray",
|
||||
dims=dims,
|
||||
name_dims=name_dims,
|
||||
collection_base=collection_base,
|
||||
naming="discovered",
|
||||
graph_dir=None,
|
||||
vectors=vectors,
|
||||
created_at=_now(),
|
||||
)
|
||||
_insert(connection, space)
|
||||
return space
|
||||
|
||||
def mark_dropped(self, space_id: str) -> None:
|
||||
_changed()
|
||||
with self._connect(writing=True) as connection:
|
||||
connection.execute(
|
||||
"UPDATE embedding_spaces SET state='dropped', dropped_at=? "
|
||||
"WHERE space_id=? AND state IN ('retired','building')",
|
||||
(_now(), space_id),
|
||||
)
|
||||
|
||||
def expired(self, *, now: datetime | None = None) -> list[EmbeddingSpace]:
|
||||
moment = (now or datetime.now(UTC)).isoformat()
|
||||
with self._connect() as connection:
|
||||
rows = connection.execute(
|
||||
"SELECT * FROM embedding_spaces WHERE state='retired' "
|
||||
"AND keep_until IS NOT NULL AND keep_until <= ?",
|
||||
(moment,),
|
||||
).fetchall()
|
||||
return [_row(row) for row in rows]
|
||||
|
||||
|
||||
def registry_for(profiles: object) -> EmbeddingSpaceStore | None:
|
||||
"""The registry beside a profile store, or None for a store with no database."""
|
||||
path = getattr(profiles, "database_path", None)
|
||||
return EmbeddingSpaceStore(path) if isinstance(path, Path) else None
|
||||
|
||||
|
||||
def _insert(connection: sqlite3.Connection, space: EmbeddingSpace) -> None:
|
||||
values = space.model_dump()
|
||||
columns = ", ".join(values)
|
||||
marks = ", ".join("?" for _ in values)
|
||||
connection.execute(
|
||||
f"INSERT INTO embedding_spaces ({columns}) VALUES ({marks})",
|
||||
tuple(values.values()),
|
||||
)
|
||||
|
||||
|
||||
def insert_space_in(connection: sqlite3.Connection, space: EmbeddingSpace) -> None:
|
||||
_changed()
|
||||
_insert(connection, space)
|
||||
|
||||
|
||||
def record_width_in(connection: sqlite3.Connection, space_id: str, dims: int) -> None:
|
||||
_changed()
|
||||
changed = connection.execute(
|
||||
"UPDATE embedding_spaces SET dims=? WHERE space_id=? AND (dims IS NULL OR dims=?)",
|
||||
(dims, space_id, dims),
|
||||
)
|
||||
if changed.rowcount != 1:
|
||||
raise EmbeddingSpaceError("embedding_space_width_conflict")
|
||||
|
||||
|
||||
def space_in(connection: sqlite3.Connection, space_id: str) -> EmbeddingSpace | None:
|
||||
previous = connection.row_factory
|
||||
connection.row_factory = sqlite3.Row
|
||||
try:
|
||||
row = connection.execute(
|
||||
"SELECT * FROM embedding_spaces WHERE space_id=?", (space_id,)
|
||||
).fetchone()
|
||||
finally:
|
||||
connection.row_factory = previous
|
||||
return _row(row) if row is not None else None
|
||||
|
||||
|
||||
def spaces_in(connection: sqlite3.Connection) -> list[EmbeddingSpace]:
|
||||
previous = connection.row_factory
|
||||
connection.row_factory = sqlite3.Row
|
||||
try:
|
||||
rows = connection.execute("SELECT * FROM embedding_spaces").fetchall()
|
||||
finally:
|
||||
connection.row_factory = previous
|
||||
return [_row(row) for row in rows]
|
||||
|
||||
|
||||
def set_state_in(
|
||||
connection: sqlite3.Connection,
|
||||
space_id: str,
|
||||
*,
|
||||
state: SpaceState,
|
||||
expected: tuple[SpaceState, ...],
|
||||
now: datetime | None = None,
|
||||
) -> None:
|
||||
"""Move one space between states, refusing any it was not expected in."""
|
||||
_changed()
|
||||
moment = now or datetime.now(UTC)
|
||||
updates: dict[str, object] = {"state": state}
|
||||
if state == "live":
|
||||
updates["live_since"] = moment.isoformat()
|
||||
elif state == "retired":
|
||||
updates["retired_at"] = moment.isoformat()
|
||||
updates["keep_until"] = (moment + RETIRED_SPACE_RETENTION).isoformat()
|
||||
elif state == "dropped":
|
||||
updates["dropped_at"] = moment.isoformat()
|
||||
assignments = ", ".join(f"{column}=?" for column in updates)
|
||||
marks = ",".join("?" for _ in expected)
|
||||
changed = connection.execute(
|
||||
f"UPDATE embedding_spaces SET {assignments} WHERE space_id=? AND state IN ({marks})",
|
||||
(*updates.values(), space_id, *expected),
|
||||
)
|
||||
if changed.rowcount != 1:
|
||||
raise EmbeddingSpaceError("embedding_space_state_changed")
|
||||
|
||||
|
||||
def next_v2_base(
|
||||
connection: sqlite3.Connection,
|
||||
identity: EmbeddingIdentity,
|
||||
*,
|
||||
stored_names: frozenset[str],
|
||||
) -> str:
|
||||
"""A base for a new space no registry row and no stored collection uses.
|
||||
|
||||
Switching back to a model whose retired space is still kept gets a fresh
|
||||
`_g<n>` base, so stale retired vectors are never mixed with fresh ones.
|
||||
"""
|
||||
used = {space.collection_base for space in spaces_in(connection)}
|
||||
generation = 0
|
||||
while True:
|
||||
base = v2_collection_base(identity, generation)
|
||||
names = space_collection_names(base, identity.dims).values()
|
||||
if base not in used and not any(name in stored_names for name in names):
|
||||
return base
|
||||
generation += 1
|
||||
|
||||
|
||||
def probe_width_allowed(
|
||||
connection: sqlite3.Connection,
|
||||
profile_id: str,
|
||||
width: int,
|
||||
*,
|
||||
data_dir: Path,
|
||||
) -> bool:
|
||||
"""Whether a probe may record `width` on a profile without moving the space.
|
||||
|
||||
A profile that fills the live space (the active one, or the one the space
|
||||
was built with) may record its measured width only when it is the
|
||||
space's: equal to the width already known, or -- for an "auto" space
|
||||
whose width was never recorded -- equal to the width Chroma stored, or
|
||||
any width when nothing is stored yet. That is a relabel; the names never
|
||||
change. Any other width needs a switch (B2: it used to silently open a
|
||||
new, empty collection). Every other profile records what it measured.
|
||||
"""
|
||||
live = live_space_in(connection)
|
||||
if live is None:
|
||||
return True
|
||||
active = connection.execute(
|
||||
"SELECT profile_id FROM model_profiles WHERE is_active=1"
|
||||
).fetchone()
|
||||
fills_live = live.profile_id == profile_id or (active is not None and active[0] == profile_id)
|
||||
if not fills_live:
|
||||
return True
|
||||
if live.dims is not None:
|
||||
return live.dims == width
|
||||
from one_memory.sdk_vector_reattach import stored_collection_width
|
||||
|
||||
stored = stored_collection_width(data_dir, live.collection_names()["memory"])
|
||||
if stored is not None and stored != width:
|
||||
return False
|
||||
record_width_in(connection, live.space_id, width)
|
||||
return True
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,161 @@
|
||||
"""HTTP surface of the embedding switch: spaces, plans, jobs, cleanup.
|
||||
|
||||
The contract the App and the CLI build on is
|
||||
`memhub/docs/contracts/embedding-switch-v1.md`. Errors keep Core's shape,
|
||||
`{"detail": "<code>"}`; a refusal that computed a plan also carries it as
|
||||
`"plan"`, so a client can show the confirmation without asking again.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from collections.abc import Sequence
|
||||
from typing import Any, cast
|
||||
|
||||
from fastapi import APIRouter, Request, Response
|
||||
from fastapi.params import Depends
|
||||
from fastapi.responses import JSONResponse
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from one_memory.embedding_backend import ReembedUnavailableError
|
||||
from one_memory.embedding_spaces import EmbeddingSpaceError
|
||||
from one_memory.embedding_switch import (
|
||||
EmbeddingSwitchError,
|
||||
EmbeddingSwitchPlan,
|
||||
EmbeddingSwitchService,
|
||||
)
|
||||
|
||||
#: HTTP status per refusal code; anything not listed is a 409.
|
||||
_STATUS = {
|
||||
"model_profile_not_found": 404,
|
||||
"embedding_switch_not_found": 404,
|
||||
"embedding_space_not_found": 404,
|
||||
"sdk_reembed_unavailable": 503,
|
||||
"secure_storage_unavailable": 503,
|
||||
"insufficient_disk_space": 507,
|
||||
}
|
||||
|
||||
|
||||
class EmbeddingSwitchPlanRequest(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
profile_id: str = Field(min_length=1, max_length=128)
|
||||
expected_revision: int | None = Field(default=None, ge=1)
|
||||
|
||||
|
||||
class EmbeddingSwitchStartRequest(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
profile_id: str = Field(min_length=1, max_length=128)
|
||||
expected_revision: int = Field(ge=1)
|
||||
plan_token: str = Field(min_length=1, max_length=128)
|
||||
requested_by: str = Field(default="app", pattern=r"^(app|cli)$")
|
||||
|
||||
|
||||
def refusal(code: str, *, plan: EmbeddingSwitchPlan | None = None) -> JSONResponse:
|
||||
content: dict[str, Any] = {"detail": code}
|
||||
if plan is not None:
|
||||
content["plan"] = plan.model_dump(mode="json")
|
||||
return JSONResponse(status_code=_STATUS.get(code, 409), content=content)
|
||||
|
||||
|
||||
def _service(request: Request) -> EmbeddingSwitchService:
|
||||
return cast(EmbeddingSwitchService, request.app.state.embedding_switch)
|
||||
|
||||
|
||||
def embedding_switch_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
router = APIRouter(dependencies=dependencies)
|
||||
|
||||
@router.get("/api/v1/embedding-spaces")
|
||||
async def list_embedding_spaces(request: Request) -> dict[str, object]:
|
||||
"""The registry and what the vector store holds, space by space."""
|
||||
service = _service(request)
|
||||
spaces = await asyncio.to_thread(service.spaces.list_spaces)
|
||||
live = next((space for space in spaces if space.state == "live"), None)
|
||||
scanned: list[dict[str, object]] | None = None
|
||||
scan_error = service.backend.unavailable_reason()
|
||||
if scan_error is None:
|
||||
try:
|
||||
found = await service.backend.scan()
|
||||
except Exception as error: # noqa: BLE001 - the registry still answers
|
||||
scan_error = getattr(error, "code", None) or "embedding_space_scan_failed"
|
||||
else:
|
||||
known = {(space.collection_base, space.name_dims or 0): space for space in spaces}
|
||||
scanned = [
|
||||
{
|
||||
"collection_base": item.collection_base,
|
||||
"name_dims": item.name_dims,
|
||||
"width": item.width,
|
||||
"vectors": item.vectors,
|
||||
"counts": dict(item.counts),
|
||||
"space_id": known[item.key].space_id if item.key in known else None,
|
||||
"state": known[item.key].state if item.key in known else "unregistered",
|
||||
}
|
||||
for item in found
|
||||
]
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"live_space_id": live.space_id if live is not None else None,
|
||||
"spaces": [space.model_dump(mode="json") for space in spaces],
|
||||
"scanned": scanned,
|
||||
"scan_error": scan_error,
|
||||
}
|
||||
|
||||
@router.delete("/api/v1/embedding-spaces/{space_id}", response_model=None)
|
||||
async def delete_embedding_space(space_id: str, request: Request) -> Any:
|
||||
"""Clean up a retired space now instead of after its 14 days."""
|
||||
try:
|
||||
space = await _service(request).drop_space(space_id)
|
||||
except (EmbeddingSwitchError, EmbeddingSpaceError, ReembedUnavailableError) as error:
|
||||
return refusal(error.code)
|
||||
except RuntimeError as error:
|
||||
if "held open" in str(error):
|
||||
return refusal("embedding_space_in_use")
|
||||
raise
|
||||
return {"schema_version": 1, "space": space.model_dump(mode="json")}
|
||||
|
||||
@router.post("/api/v1/embedding-switches/plan", response_model=None)
|
||||
async def plan_embedding_switch(value: EmbeddingSwitchPlanRequest, request: Request) -> Any:
|
||||
try:
|
||||
plan = await _service(request).plan(
|
||||
value.profile_id, expected_revision=value.expected_revision
|
||||
)
|
||||
except EmbeddingSwitchError as error:
|
||||
return refusal(error.code, plan=error.plan)
|
||||
return plan.model_dump(mode="json")
|
||||
|
||||
@router.post("/api/v1/embedding-switches", status_code=202, response_model=None)
|
||||
async def start_embedding_switch(value: EmbeddingSwitchStartRequest, request: Request) -> Any:
|
||||
try:
|
||||
job = await _service(request).start_switch(
|
||||
value.profile_id,
|
||||
expected_revision=value.expected_revision,
|
||||
plan_token=value.plan_token,
|
||||
requested_by=value.requested_by,
|
||||
)
|
||||
except EmbeddingSwitchError as error:
|
||||
return refusal(error.code, plan=error.plan)
|
||||
return job.model_dump(mode="json")
|
||||
|
||||
@router.get("/api/v1/embedding-switches/current")
|
||||
async def current_embedding_switch(request: Request, response: Response) -> dict[str, object]:
|
||||
"""Polled by the loading screen, the banner and `memhub models switch status`."""
|
||||
response.headers["Cache-Control"] = "no-store"
|
||||
return await asyncio.to_thread(_service(request).current)
|
||||
|
||||
@router.get("/api/v1/embedding-switches/{job_id}", response_model=None)
|
||||
async def get_embedding_switch(job_id: str, request: Request) -> Any:
|
||||
job = await asyncio.to_thread(_service(request).get_job, job_id)
|
||||
if job is None:
|
||||
return refusal("embedding_switch_not_found")
|
||||
return job.model_dump(mode="json")
|
||||
|
||||
@router.post("/api/v1/embedding-switches/{job_id}/cancel", response_model=None)
|
||||
async def cancel_embedding_switch(job_id: str, request: Request) -> Any:
|
||||
try:
|
||||
job = await asyncio.to_thread(_service(request).cancel, job_id)
|
||||
except EmbeddingSwitchError as error:
|
||||
return refusal(error.code)
|
||||
return job.model_dump(mode="json")
|
||||
|
||||
return router
|
||||
@@ -94,11 +94,15 @@ def inspect_profile_graph_width(
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
mode: str | None = None,
|
||||
graph_dir: str = "graph",
|
||||
) -> GraphWidthMismatch | None:
|
||||
"""Inspect the graph path and width selected by the active profile."""
|
||||
"""Inspect the graph path and width selected by the active profile.
|
||||
|
||||
`graph_dir` is the live embedding space's graph (`embedding_spaces`).
|
||||
"""
|
||||
del api_key
|
||||
return _inspect_graph_width(
|
||||
_profile_graph_path(data_dir, mode),
|
||||
_profile_graph_path(data_dir, mode, graph_dir),
|
||||
_profile_embedding_width(profile),
|
||||
)
|
||||
|
||||
@@ -110,11 +114,12 @@ def decide_profile_graph_width(
|
||||
data_dir: Path,
|
||||
decision: GraphWidthDecision,
|
||||
mode: str | None = None,
|
||||
graph_dir: str = "graph",
|
||||
) -> GraphWidthOutcome:
|
||||
"""Apply the user's decision to the active profile's graph."""
|
||||
del api_key
|
||||
config = _GraphAdminConfig(
|
||||
graph_store=_GraphStoreConfig(str(_profile_graph_path(data_dir, mode))),
|
||||
graph_store=_GraphStoreConfig(str(_profile_graph_path(data_dir, mode, graph_dir))),
|
||||
embedder=_EmbeddingWidthConfig(_profile_embedding_width(profile)),
|
||||
)
|
||||
return apply_graph_width_decision(
|
||||
@@ -204,6 +209,12 @@ _GRAPH_WIDTH_CACHE: dict[str, int | None] = {}
|
||||
_GRAPH_WIDTH_LOCK = threading.Lock()
|
||||
|
||||
|
||||
def forget_graph_widths() -> None:
|
||||
"""Drop every remembered width: an embedding switch moved the graph."""
|
||||
with _GRAPH_WIDTH_LOCK:
|
||||
_GRAPH_WIDTH_CACHE.clear()
|
||||
|
||||
|
||||
def _graph_embedding_width(db_path: Path) -> int | None:
|
||||
"""Read the Memory embedding width directly from Kuzu's schema."""
|
||||
try:
|
||||
@@ -252,9 +263,9 @@ def _inspect_graph_width(
|
||||
)
|
||||
|
||||
|
||||
def _profile_graph_path(data_dir: Path, mode: str | None = None) -> Path:
|
||||
def _profile_graph_path(data_dir: Path, mode: str | None = None, graph_dir: str = "graph") -> Path:
|
||||
ultra = mode == "ultra" if mode is not None else graph_store_enabled()
|
||||
name = "graph" if ultra else "graph-disabled"
|
||||
name = graph_dir if ultra else "graph-disabled"
|
||||
return data_dir / "hy-memory" / HY_MEMORY_NAMESPACE / name
|
||||
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ from pathlib import Path
|
||||
from typing import Literal, cast
|
||||
|
||||
HY_MEMORY_DISTRIBUTION = "hy-memory"
|
||||
HY_MEMORY_VERSION = "2.0.9"
|
||||
HY_MEMORY_VERSION = "2.0.11"
|
||||
HY_MEMORY_NAMESPACE = "governed-v2"
|
||||
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
|
||||
|
||||
|
||||
@@ -3,20 +3,26 @@
|
||||
Runtime product code must use public ``HyMemoryClient`` methods. The startup
|
||||
migration boundary is the isolated exception: all writers are stopped, so it
|
||||
may copy, restore and repair SDK-owned local stores.
|
||||
|
||||
The embedding switch is the second such boundary: model writes are paused
|
||||
while it re-embeds one space into another (`SdkReembedBackend`), so it may
|
||||
read and fill SDK collections through the SDK's re-embed module.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import sqlite3
|
||||
from collections.abc import Iterator
|
||||
from contextlib import closing
|
||||
from contextlib import closing, contextmanager
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from hy_memory import ( # type: ignore[import-untyped]
|
||||
import hy_memory # type: ignore[import-untyped]
|
||||
from hy_memory import (
|
||||
DataPlaneRestore,
|
||||
DataPlaneSnapshot,
|
||||
restore_local_data_plane,
|
||||
@@ -497,7 +503,7 @@ def _ensure_coding_schema(path: Path) -> None:
|
||||
CodingMemoryStore,
|
||||
)
|
||||
|
||||
# Relies on the pinned SDK's internals (hy-memory==2.0.9 is pinned exactly).
|
||||
# Relies on the pinned SDK's internals (hy-memory==2.0.11 is pinned exactly).
|
||||
store = CodingMemoryStore.__new__(CodingMemoryStore)
|
||||
store._db_path = str(path)
|
||||
store._conn = None
|
||||
@@ -511,3 +517,256 @@ def _ensure_coding_schema(path: Path) -> None:
|
||||
def _batches[T](values: list[T] | tuple[T, ...]) -> Iterator[tuple[T, ...]]:
|
||||
for start in range(0, len(values), _BATCH):
|
||||
yield tuple(values[start : start + _BATCH])
|
||||
|
||||
|
||||
# The SDK functions the embedding switch needs. They arrived in hy-memory
|
||||
# 2.0.10; an older SDK reports `sdk_reembed_unavailable` instead of failing.
|
||||
_REEMBED_API = (
|
||||
"VectorSpace",
|
||||
"scan_vector_spaces",
|
||||
"diff_spaces",
|
||||
"reembed_vector_space",
|
||||
"verify_reembed",
|
||||
"drop_vector_space",
|
||||
"reembed_graph",
|
||||
"verify_graph_reembed",
|
||||
"drop_graph",
|
||||
"ReembedCancelled",
|
||||
)
|
||||
|
||||
|
||||
class SdkReembedBackend:
|
||||
"""`ReembedBackend` over `hy_memory`'s re-embed module, for one home.
|
||||
|
||||
Runs beside live SDK clients: the vector functions open Chroma with the
|
||||
SDK's own settings, so they share its client system. The graph functions
|
||||
refuse a graph a live client holds open; the switch closes them first.
|
||||
"""
|
||||
|
||||
def __init__(self, data_dir: Path) -> None:
|
||||
self.data_dir = data_dir
|
||||
self.root = data_dir.resolve() / "hy-memory" / HY_MEMORY_NAMESPACE
|
||||
self.persist_directory = self.root / "vector"
|
||||
self._chroma_opened = False
|
||||
|
||||
def unavailable_reason(self) -> str | None:
|
||||
missing = [name for name in _REEMBED_API if not hasattr(hy_memory, name)]
|
||||
return "sdk_reembed_unavailable" if missing else None
|
||||
|
||||
def _sdk(self) -> Any:
|
||||
from one_memory.embedding_backend import ReembedUnavailableError
|
||||
|
||||
reason = self.unavailable_reason()
|
||||
if reason is not None:
|
||||
raise ReembedUnavailableError(reason)
|
||||
return hy_memory
|
||||
|
||||
def _open_chroma_once(self) -> None:
|
||||
"""Open Chroma once under the client-construction lock.
|
||||
|
||||
Every later open beside the Runtime finds that system. Two first
|
||||
opens race over Chroma's unlocked cache, and a client under
|
||||
construction lost its stores that way (see `hy_sdk_runtime`).
|
||||
"""
|
||||
if self._chroma_opened or not self.persist_directory.is_dir():
|
||||
return
|
||||
import chromadb
|
||||
|
||||
from one_memory.hy_sdk_runtime import first_store_open
|
||||
|
||||
with first_store_open(self.data_dir):
|
||||
chromadb.PersistentClient(path=str(self.persist_directory))
|
||||
self._chroma_opened = True
|
||||
|
||||
def _vector_space(self, space: Any) -> Any:
|
||||
return self._sdk().VectorSpace(
|
||||
base_name=space.collection_base,
|
||||
dims=space.dims,
|
||||
persist_directory=str(self.persist_directory),
|
||||
model=space.model,
|
||||
collections=space.names(),
|
||||
)
|
||||
|
||||
async def scan(self, *, per_user: bool = False) -> tuple[Any, ...]:
|
||||
from one_memory.embedding_backend import ScannedSpace
|
||||
|
||||
if not self.persist_directory.is_dir():
|
||||
return ()
|
||||
await asyncio.to_thread(self._open_chroma_once)
|
||||
found = await self._sdk().scan_vector_spaces(str(self.persist_directory), per_user=per_user)
|
||||
return tuple(
|
||||
ScannedSpace(
|
||||
collection_base=str(item.base_name),
|
||||
name_dims=item.dims,
|
||||
width=item.width,
|
||||
counts={role: int(entry.count) for role, entry in item.roles.items()},
|
||||
collections={role: str(entry.name) for role, entry in item.roles.items()},
|
||||
per_user=dict(item.per_user) if per_user else {},
|
||||
)
|
||||
for item in found
|
||||
)
|
||||
|
||||
async def diff(self, source: Any, target: Any) -> dict[str, int]:
|
||||
await asyncio.to_thread(self._open_chroma_once)
|
||||
result = await self._sdk().diff_spaces(
|
||||
self._vector_space(source), self._vector_space(target)
|
||||
)
|
||||
return {role: len(ids) for role, ids in result.missing.items()}
|
||||
|
||||
async def reembed(
|
||||
self,
|
||||
source: Any,
|
||||
target: Any,
|
||||
*,
|
||||
embedding: Any,
|
||||
cancel: Any,
|
||||
on_step: Any,
|
||||
batch_size: int = 64,
|
||||
) -> None:
|
||||
from one_memory.embedding_backend import ReembedStep
|
||||
|
||||
await asyncio.to_thread(self._open_chroma_once)
|
||||
sdk = self._sdk()
|
||||
|
||||
def progress(item: Any) -> None:
|
||||
on_step(
|
||||
ReembedStep(
|
||||
phase="vectors",
|
||||
role=str(item.role),
|
||||
done=int(item.done),
|
||||
total=int(item.total),
|
||||
embedded=int(item.embedded),
|
||||
reused=int(item.reused),
|
||||
kept=int(getattr(item, "kept", 0)),
|
||||
)
|
||||
)
|
||||
|
||||
with self._translated():
|
||||
await sdk.reembed_vector_space(
|
||||
self._vector_space(source),
|
||||
self._vector_space(target),
|
||||
embedding=self._embedding_service(embedding),
|
||||
batch_size=batch_size,
|
||||
cancel=cancel,
|
||||
on_progress=progress,
|
||||
# The target wins: an id it already holds is never touched, so
|
||||
# a resumed run continues and a merge never overwrites live data.
|
||||
skip_existing=True,
|
||||
)
|
||||
|
||||
async def verify(self, source: Any, target: Any, *, embedding: Any) -> Any:
|
||||
from one_memory.embedding_backend import VectorCheck
|
||||
|
||||
await asyncio.to_thread(self._open_chroma_once)
|
||||
with self._translated():
|
||||
result = await self._sdk().verify_reembed(
|
||||
self._vector_space(source),
|
||||
self._vector_space(target),
|
||||
embedding=self._embedding_service(embedding),
|
||||
)
|
||||
return VectorCheck(
|
||||
per_role={role: dict(values) for role, values in result.per_role.items()},
|
||||
per_user=dict(result.per_user),
|
||||
problems=tuple(result.problems),
|
||||
warnings=tuple(result.warnings),
|
||||
self_recall_hits=int(result.self_recall_hits),
|
||||
self_recall_samples=int(result.self_recall_samples),
|
||||
)
|
||||
|
||||
async def graph_vectors(self, path: Path) -> int | None:
|
||||
if not path.exists():
|
||||
return None
|
||||
# Not re-exported at the package root; the module is the SDK's own.
|
||||
plan = await self._sdk().reembed.plan_graph_reembed(path)
|
||||
return int(plan.vectors)
|
||||
|
||||
async def reembed_graph(
|
||||
self,
|
||||
source: Path,
|
||||
target: Path,
|
||||
*,
|
||||
embedding: Any,
|
||||
target_dims: int,
|
||||
cancel: Any,
|
||||
on_step: Any,
|
||||
) -> None:
|
||||
from one_memory.embedding_backend import ReembedStep
|
||||
|
||||
def progress(item: Any) -> None:
|
||||
on_step(
|
||||
ReembedStep(
|
||||
phase="graph",
|
||||
role=str(item.table),
|
||||
done=int(item.done),
|
||||
total=int(item.total),
|
||||
embedded=int(item.embedded),
|
||||
)
|
||||
)
|
||||
|
||||
with self._translated():
|
||||
await self._sdk().reembed_graph(
|
||||
source,
|
||||
target,
|
||||
embedding=self._embedding_service(embedding),
|
||||
target_dims=target_dims,
|
||||
cancel=cancel,
|
||||
on_progress=progress,
|
||||
# A resumed switch rebuilds the copy from the untouched source.
|
||||
overwrite=True,
|
||||
)
|
||||
|
||||
async def verify_graph(self, source: Path, target: Path, *, target_dims: int) -> Any:
|
||||
from one_memory.embedding_backend import GraphCheck
|
||||
|
||||
result = await self._sdk().verify_graph_reembed(source, target, target_dims=target_dims)
|
||||
return GraphCheck(ok=bool(result.ok), problems=tuple(result.problems))
|
||||
|
||||
async def drop(self, space: Any) -> None:
|
||||
await asyncio.to_thread(self._open_chroma_once)
|
||||
await self._sdk().drop_vector_space(self._vector_space(space))
|
||||
|
||||
async def drop_graph(self, path: Path) -> None:
|
||||
await self._sdk().drop_graph(path)
|
||||
|
||||
def _embedding_service(self, embedding: Any) -> Any:
|
||||
"""The service that computes a target space's vectors.
|
||||
|
||||
The built-in model is MemHub's ONNX service, exactly what the Runtime
|
||||
client embeds with. A remote model gets the SDK's own service, built
|
||||
from the same configuration the Runtime client would use.
|
||||
"""
|
||||
from one_memory.hy_sdk_runtime import (
|
||||
LocalOnnxEmbeddingService,
|
||||
build_sdk_config,
|
||||
retrieval_view,
|
||||
)
|
||||
from one_memory.model_profiles import ProviderKind
|
||||
|
||||
profile = embedding.profile
|
||||
provider = profile.embedding_provider_kind or profile.provider_kind
|
||||
if provider is ProviderKind.LOCAL_ONNX:
|
||||
return LocalOnnxEmbeddingService()
|
||||
from hy_memory.core.embed_service import EmbedService # type: ignore[import-untyped]
|
||||
|
||||
return EmbedService(
|
||||
# The retrieval view: re-embedding never needs the generation model.
|
||||
build_sdk_config(
|
||||
retrieval_view(profile),
|
||||
embedding.space,
|
||||
api_key=embedding.api_key,
|
||||
data_dir=self.data_dir,
|
||||
)
|
||||
)
|
||||
|
||||
@contextmanager
|
||||
def _translated(self) -> Iterator[None]:
|
||||
"""SDK refusals as the Core errors the switch classifies."""
|
||||
from one_memory.embedding_backend import EmbeddingWidthError, ReembedCancelledError
|
||||
|
||||
sdk = self._sdk()
|
||||
try:
|
||||
yield
|
||||
except sdk.ReembedCancelled:
|
||||
raise ReembedCancelledError("embedding_switch_cancelled") from None
|
||||
except hy_memory.EmbeddingDimensionMismatchError as error:
|
||||
raise EmbeddingWidthError("embedding_dimensions_mismatch") from error
|
||||
@@ -11,10 +11,11 @@ import asyncio
|
||||
import hashlib
|
||||
import os
|
||||
import threading
|
||||
from collections.abc import Callable, Coroutine, Iterable, Mapping
|
||||
from collections.abc import Callable, Coroutine, Iterable, Iterator, Mapping
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, ClassVar
|
||||
from typing import Any, ClassVar, Literal, TypedDict, cast
|
||||
|
||||
from hy_memory import ( # type: ignore[import-untyped]
|
||||
AmbiguousMemoryIdError as AmbiguousMemoryIdError, # re-export for api.py
|
||||
@@ -29,6 +30,13 @@ from one_memory.embedding_assets import (
|
||||
BUILTIN_EMBEDDING_DIMENSIONS,
|
||||
BUILTIN_EMBEDDING_MODEL,
|
||||
)
|
||||
from one_memory.embedding_spaces import (
|
||||
DEFAULT_GRAPH_DIR,
|
||||
EmbeddingSpace,
|
||||
EmbeddingSpaceError,
|
||||
derived_space,
|
||||
embedding_identity,
|
||||
)
|
||||
from one_memory.hy_graph_admin import (
|
||||
GraphWidthMismatchError,
|
||||
effective_memory_mode,
|
||||
@@ -54,6 +62,10 @@ BUILTIN_RETRIEVAL_PROFILE_ID = "builtin-retrieval"
|
||||
_LOCAL_ONNX_PLACEHOLDER_URL = "http://127.0.0.1/local-onnx"
|
||||
DEFAULT_RELEVANCE_FLOOR = 0.35
|
||||
|
||||
#: The SDK's memory modes. A mode never changes a collection name: the space
|
||||
#: names the collections, the mode only which stores the client opens.
|
||||
MemoryMode = Literal["lite", "pro", "ultra"]
|
||||
|
||||
|
||||
@dataclass
|
||||
class _RuntimeEntry:
|
||||
@@ -176,37 +188,77 @@ class SdkClientHandle:
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
http_transport_factory: Callable[[], Any] | None = None,
|
||||
space: EmbeddingSpace | None = None,
|
||||
mode: str | None = None,
|
||||
features: FeatureOverrides = (),
|
||||
language: str | None = None,
|
||||
) -> None:
|
||||
self._profile = profile
|
||||
self._api_key = api_key
|
||||
self._data_dir = data_dir
|
||||
self._http_transport_factory = http_transport_factory
|
||||
self._space = space
|
||||
#: The SDK mode this handle's client is built with, fixed for its life.
|
||||
self.mode = mode or effective_memory_mode()
|
||||
#: The SDK features set on its config (`memory_features`), fixed likewise.
|
||||
self.features = features
|
||||
#: The SDK prompt language (`ui_language`); None = SDK auto-detect.
|
||||
self.language = language
|
||||
self._lease: ClientLease | None = None
|
||||
self._constructing: asyncio.Future[ClientLease] | None = None
|
||||
self._lock = asyncio.Lock()
|
||||
|
||||
@property
|
||||
def profile(self) -> ModelProfileSnapshot:
|
||||
return self._profile
|
||||
|
||||
@property
|
||||
def space(self) -> EmbeddingSpace | None:
|
||||
return self._space
|
||||
|
||||
@property
|
||||
def api_key(self) -> str | None:
|
||||
return self._api_key
|
||||
|
||||
async def client(self) -> HyMemoryClient:
|
||||
if self._lease is not None:
|
||||
return self._lease.client
|
||||
async with self._lock:
|
||||
if self._lease is None:
|
||||
self._lease = await asyncio.to_thread(
|
||||
acquire_client,
|
||||
self._profile,
|
||||
api_key=self._api_key,
|
||||
data_dir=self._data_dir,
|
||||
http_transport_factory=self._http_transport_factory,
|
||||
mode=self.mode,
|
||||
features=self.features,
|
||||
constructing = asyncio.ensure_future(
|
||||
asyncio.to_thread(
|
||||
_acquire_for_handle,
|
||||
self._profile,
|
||||
self._space,
|
||||
api_key=self._api_key,
|
||||
data_dir=self._data_dir,
|
||||
http_transport_factory=self._http_transport_factory,
|
||||
mode=self.mode,
|
||||
features=self.features,
|
||||
**language_kwargs(self.language),
|
||||
)
|
||||
)
|
||||
self._constructing = constructing
|
||||
try:
|
||||
self._lease = await asyncio.shield(constructing)
|
||||
except asyncio.CancelledError:
|
||||
# The construction thread runs on. Keep what it builds, so
|
||||
# `close` releases it instead of leaving its runtime open.
|
||||
constructing.add_done_callback(self._keep)
|
||||
raise
|
||||
return self._lease.client
|
||||
|
||||
def _keep(self, constructing: asyncio.Future[ClientLease]) -> None:
|
||||
if constructing.cancelled() or constructing.exception() is not None:
|
||||
return
|
||||
if self._lease is None:
|
||||
self._lease = constructing.result()
|
||||
|
||||
async def close(self) -> None:
|
||||
constructing = self._constructing
|
||||
if constructing is not None and not constructing.done():
|
||||
# A construction a cancelled caller left behind: wait for its lease.
|
||||
await asyncio.gather(constructing, return_exceptions=True)
|
||||
async with self._lock:
|
||||
lease, self._lease = self._lease, None
|
||||
if lease is not None:
|
||||
@@ -299,33 +351,67 @@ def apply_sdk_features(
|
||||
return config
|
||||
|
||||
|
||||
class LanguageKwargs(TypedDict, total=False):
|
||||
language: str
|
||||
|
||||
|
||||
def language_kwargs(language: str | None) -> LanguageKwargs:
|
||||
"""`language=` for a client constructor, only when there is one.
|
||||
|
||||
Without a language every call is the one it was before the setting
|
||||
existed, so stand-ins written for that call keep working.
|
||||
"""
|
||||
return {"language": language} if language is not None else {}
|
||||
|
||||
|
||||
def apply_sdk_language(config: MemoryConfig, language: str | None) -> MemoryConfig:
|
||||
"""Set the SDK prompt language (`MemoryConfig.language`), when there is one.
|
||||
|
||||
None (no language set) sets nothing: the SDK detects zh/en as it always
|
||||
did. Every display language, zh-Hans included, is set explicitly. Like a feature, the language never touches `vector_store`, so
|
||||
the collection identity never depends on it.
|
||||
"""
|
||||
if language is not None:
|
||||
config.language = language
|
||||
return config
|
||||
|
||||
|
||||
def build_sdk_config(
|
||||
profile: ModelProfileSnapshot,
|
||||
space: EmbeddingSpace | None = None,
|
||||
mode: MemoryMode | None = None,
|
||||
*,
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
features: Mapping[str, bool] | Iterable[tuple[str, bool]] = (),
|
||||
language: str | None = None,
|
||||
) -> MemoryConfig:
|
||||
"""Map a model snapshot to SDK models, endpoints, backends, and local paths.
|
||||
"""Map a model snapshot and its embedding space to an SDK configuration.
|
||||
|
||||
`features` (`memory_features`) are set on top; with none the config is
|
||||
exactly what it was before the setting existed.
|
||||
`space` names the collections and the graph; the profile only says how to
|
||||
reach the models. A profile whose embedding identity (model and width) is
|
||||
not the space's raises `embedding_space_mismatch` instead of opening new,
|
||||
empty collections. Without `space`, the space the profile opened before
|
||||
the registry existed is used: offline tools on a home with no registry.
|
||||
`mode` never changes a name, and neither do `features`
|
||||
(`memory_features`), which are set on top; with none the config is
|
||||
exactly what it was before the setting existed. So is `language`
|
||||
(`apply_sdk_language`); None leaves the SDK's own zh/en detection.
|
||||
"""
|
||||
if mode is not None and mode not in {"lite", "pro", "ultra"}:
|
||||
raise HySdkRuntimeError("hy_sdk_memory_mode_invalid")
|
||||
generation_endpoint, generation_key = _generation_access(profile, api_key)
|
||||
embedding_provider = profile.embedding_provider_kind or profile.provider_kind
|
||||
embedding_service_is_local = embedding_provider is ProviderKind.LOCAL_ONNX
|
||||
embedding_model = profile.embedding_model or (
|
||||
BUILTIN_EMBEDDING_MODEL if embedding_service_is_local else None
|
||||
)
|
||||
if not embedding_model:
|
||||
raise HySdkRuntimeError("hy_sdk_embedding_model_required")
|
||||
dimensions: int | None
|
||||
try:
|
||||
identity = embedding_identity(profile)
|
||||
except EmbeddingSpaceError as error:
|
||||
raise HySdkRuntimeError(error.code) from None
|
||||
if embedding_service_is_local:
|
||||
embedding_endpoint = None
|
||||
embedding_key = None
|
||||
dimensions = profile.embedding_dimensions or BUILTIN_EMBEDDING_DIMENSIONS
|
||||
else:
|
||||
# Never the generation endpoint: the collection name hashes this one.
|
||||
# Never the generation endpoint: a remote embedding has its own.
|
||||
embedding_endpoint = _openai_endpoint(embedding_provider, profile.embedding_base_url)
|
||||
embedding_key = "ollama" if embedding_provider is ProviderKind.OLLAMA else api_key
|
||||
if embedding_provider is ProviderKind.OPENAI_COMPATIBLE and not embedding_key:
|
||||
@@ -335,13 +421,18 @@ def build_sdk_config(
|
||||
ProviderKind.OLLAMA,
|
||||
}:
|
||||
raise HySdkRuntimeError("hy_sdk_embedding_provider_unsupported")
|
||||
dimensions = profile.embedding_dimensions
|
||||
if space is None:
|
||||
space = derived_space(profile)
|
||||
elif space.identity is None or not identity.matches(space.identity):
|
||||
raise HySdkRuntimeError("embedding_space_mismatch")
|
||||
# The width the space was opened with: its names carry it, and a remote
|
||||
# request sends it as `dimensions`. An "auto" space sends none, as before.
|
||||
dimensions = space.name_dims
|
||||
embedding_model = identity.model
|
||||
|
||||
root = data_dir.resolve() / "hy-memory" / "governed-v2"
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
collection = _collection_base(
|
||||
embedding_provider, embedding_endpoint, embedding_model, dimensions
|
||||
)
|
||||
collection = space.collection_base
|
||||
|
||||
llm: dict[str, Any] = {
|
||||
"provider": "openai",
|
||||
@@ -379,7 +470,7 @@ def build_sdk_config(
|
||||
},
|
||||
"graph_store": {
|
||||
"provider": "kuzu",
|
||||
"db_path": str(root / "graph"),
|
||||
"db_path": str(root / (space.graph_dir or DEFAULT_GRAPH_DIR)),
|
||||
"url": None,
|
||||
"username": None,
|
||||
"password": None,
|
||||
@@ -393,25 +484,54 @@ def build_sdk_config(
|
||||
"coding": {"db_path": str(root / "coding.sqlite3")},
|
||||
}
|
||||
)
|
||||
return apply_sdk_features(config, features)
|
||||
return apply_sdk_language(apply_sdk_features(config, features), language)
|
||||
|
||||
|
||||
def _acquire_for_handle(
|
||||
profile: ModelProfileSnapshot,
|
||||
space: EmbeddingSpace | None,
|
||||
*,
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
http_transport_factory: Callable[[], Any] | None,
|
||||
mode: str | None,
|
||||
features: FeatureOverrides = (),
|
||||
language: str | None = None,
|
||||
) -> ClientLease:
|
||||
# `acquire_client` is looked up at call time, so a stand-in replaces every
|
||||
# construction. A handle without a space keeps the call it always made.
|
||||
named: dict[str, Any] = {"space": space} if space is not None else {}
|
||||
return acquire_client(
|
||||
profile,
|
||||
api_key=api_key,
|
||||
data_dir=data_dir,
|
||||
http_transport_factory=http_transport_factory,
|
||||
mode=mode,
|
||||
features=features,
|
||||
**named,
|
||||
**language_kwargs(language),
|
||||
)
|
||||
|
||||
|
||||
def acquire_client(
|
||||
profile: ModelProfileSnapshot,
|
||||
space: EmbeddingSpace | None = None,
|
||||
*,
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
http_transport_factory: Callable[[], Any] | None = None,
|
||||
mode: str | None = None,
|
||||
features: FeatureOverrides = (),
|
||||
language: str | None = None,
|
||||
) -> ClientLease:
|
||||
"""Construct a public SDK client and return its explicit runtime lease.
|
||||
|
||||
`mode` is the SDK pipeline mode (lite / pro / ultra); None keeps the
|
||||
pre-setting default (ultra when `ONE_MEMORY_GRAPH_STORE` is set, else pro).
|
||||
`features` are the `memory_features` set on its config; none keeps every
|
||||
SDK default.
|
||||
Waits while `hold_stores_offline` holds this home's stores.
|
||||
`space` names the collections (`build_sdk_config`). `mode` is the SDK
|
||||
pipeline mode (lite / pro / ultra) and never changes a name; None keeps
|
||||
the pre-setting default (ultra when `ONE_MEMORY_GRAPH_STORE` is set, else
|
||||
pro). `features` are the `memory_features` set on its config; none keeps
|
||||
every SDK default. `language` is the SDK prompt language; None
|
||||
auto-detects. Waits while `hold_stores_offline` holds this home's stores.
|
||||
"""
|
||||
key = data_dir.resolve()
|
||||
with _STORE_GATE:
|
||||
@@ -419,6 +539,7 @@ def acquire_client(
|
||||
_STORE_GATE.wait()
|
||||
_OPENING[key] = _OPENING.get(key, 0) + 1
|
||||
try:
|
||||
named: dict[str, Any] = {"space": space} if space is not None else {}
|
||||
return _acquire_client(
|
||||
profile,
|
||||
api_key=api_key,
|
||||
@@ -426,6 +547,8 @@ def acquire_client(
|
||||
http_transport_factory=http_transport_factory,
|
||||
mode=mode or effective_memory_mode(),
|
||||
features=features,
|
||||
**named,
|
||||
**language_kwargs(language),
|
||||
)
|
||||
finally:
|
||||
with _STORE_GATE:
|
||||
@@ -435,6 +558,20 @@ def acquire_client(
|
||||
_STORE_GATE.notify_all()
|
||||
|
||||
|
||||
@contextmanager
|
||||
def first_store_open(data_dir: Path) -> Iterator[None]:
|
||||
"""One opening of this home's stores at a time: Chroma's first open is unlocked.
|
||||
|
||||
The same lock client construction takes, for anything else that opens
|
||||
Chroma beside the Runtime (the re-embed functions).
|
||||
"""
|
||||
key = data_dir.resolve()
|
||||
with _STORE_GATE:
|
||||
constructing = _CONSTRUCTING.setdefault(key, threading.Lock())
|
||||
with constructing:
|
||||
yield
|
||||
|
||||
|
||||
def hold_stores_offline(data_dir: Path) -> bool:
|
||||
"""Hold this home's SDK stores for offline maintenance; False if a client is open.
|
||||
|
||||
@@ -460,14 +597,24 @@ def release_stores(data_dir: Path) -> None:
|
||||
|
||||
def _acquire_client(
|
||||
profile: ModelProfileSnapshot,
|
||||
space: EmbeddingSpace | None = None,
|
||||
*,
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
http_transport_factory: Callable[[], Any] | None,
|
||||
mode: str = "pro",
|
||||
features: FeatureOverrides = (),
|
||||
language: str | None = None,
|
||||
) -> ClientLease:
|
||||
config = build_sdk_config(profile, api_key=api_key, data_dir=data_dir, features=features)
|
||||
config = build_sdk_config(
|
||||
profile,
|
||||
space,
|
||||
cast(MemoryMode, mode),
|
||||
api_key=api_key,
|
||||
data_dir=data_dir,
|
||||
features=features,
|
||||
language=language,
|
||||
)
|
||||
config.http_transport_factory = http_transport_factory
|
||||
mismatch = inspect_graph_width_mismatch(config)
|
||||
if mismatch is not None:
|
||||
@@ -523,6 +670,33 @@ def _generation_access(
|
||||
return endpoint, api_key
|
||||
|
||||
|
||||
def embedding_endpoint(profile: ModelProfileSnapshot) -> str | None:
|
||||
"""The OpenAI-compatible base URL the SDK embeds through; None when local."""
|
||||
provider = profile.embedding_provider_kind or profile.provider_kind
|
||||
if provider is ProviderKind.LOCAL_ONNX:
|
||||
return None
|
||||
return _openai_endpoint(provider, profile.embedding_base_url)
|
||||
|
||||
|
||||
def legacy_collection_base(profile: ModelProfileSnapshot) -> str:
|
||||
"""The four-attribute name a profile's space had before the registry.
|
||||
|
||||
Existing homes are registered under exactly this name, so it must never
|
||||
change: provider, embedding endpoint, model and declared width.
|
||||
"""
|
||||
provider = profile.embedding_provider_kind or profile.provider_kind
|
||||
local = provider is ProviderKind.LOCAL_ONNX
|
||||
model = profile.embedding_model or (BUILTIN_EMBEDDING_MODEL if local else None)
|
||||
if not model:
|
||||
raise HySdkRuntimeError("hy_sdk_embedding_model_required")
|
||||
if local:
|
||||
return _collection_base(
|
||||
provider, None, model, profile.embedding_dimensions or BUILTIN_EMBEDDING_DIMENSIONS
|
||||
)
|
||||
endpoint = _openai_endpoint(provider, profile.embedding_base_url)
|
||||
return _collection_base(provider, endpoint, model, profile.embedding_dimensions)
|
||||
|
||||
|
||||
def _collection_base(
|
||||
provider: ProviderKind,
|
||||
endpoint: str | None,
|
||||
|
||||
@@ -19,6 +19,7 @@ from pydantic import BaseModel, ConfigDict, Field, StrictBool, field_validator
|
||||
|
||||
from one_memory.memory_features import MemoryFeatures
|
||||
from one_memory.memory_queue import ExtractionWindowPreset
|
||||
from one_memory.ui_language import UI_LANGUAGES, UiLanguage
|
||||
|
||||
_SOURCE_ID = re.compile(r"^[a-z0-9][a-z0-9-]{0,127}$")
|
||||
|
||||
@@ -73,6 +74,10 @@ class ConfigDocument(BaseModel):
|
||||
#: SDK retrieval and extraction features a person set; an unset one keeps
|
||||
#: the SDK default. See `one_memory.memory_features`.
|
||||
memory_features: MemoryFeatures = Field(default_factory=MemoryFeatures)
|
||||
#: The display language the App shows (its choice, or the system language
|
||||
#: it follows); None = none set. See `one_memory.ui_language` for what it
|
||||
#: does to the SDK.
|
||||
ui_language: UiLanguage | None = None
|
||||
|
||||
@field_validator("sources")
|
||||
@classmethod
|
||||
@@ -95,7 +100,7 @@ class ConfigFileError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
_WRITTEN_WHEN_SET = frozenset({"memory_mode", "evolve_schedule", "memory_features"})
|
||||
_WRITTEN_WHEN_SET = frozenset({"memory_mode", "evolve_schedule", "memory_features", "ui_language"})
|
||||
|
||||
|
||||
class ConfigStore:
|
||||
@@ -183,6 +188,19 @@ class ConfigStore:
|
||||
self._document = updated
|
||||
return updated
|
||||
|
||||
def ui_language(self) -> UiLanguage | None:
|
||||
with self._lock:
|
||||
return self._document.ui_language
|
||||
|
||||
def set_ui_language(self, language: UiLanguage | None) -> ConfigDocument:
|
||||
if language is not None and language not in UI_LANGUAGES:
|
||||
raise ValueError("invalid_ui_language")
|
||||
with self._lock:
|
||||
updated = self._document.model_copy(update={"ui_language": language})
|
||||
self._write(updated)
|
||||
self._document = updated
|
||||
return updated
|
||||
|
||||
def migrate_extraction_window_preset(self, preset: ExtractionWindowPreset) -> ConfigDocument:
|
||||
"""Adopt the legacy SQL value only when config.json has no value."""
|
||||
if not isinstance(preset, ExtractionWindowPreset):
|
||||
|
||||
@@ -253,6 +253,9 @@ MODEL_NOT_READY_CODES = frozenset(
|
||||
"model_secret_unavailable",
|
||||
"secure_storage_unavailable",
|
||||
"hy_memory_client_unavailable",
|
||||
# The active profile's embedding is not the live space's: activation
|
||||
# must go through an embedding switch, so the write waits for it.
|
||||
"embedding_space_mismatch",
|
||||
}
|
||||
)
|
||||
|
||||
@@ -2565,6 +2568,9 @@ async def execute_job(
|
||||
"model_sedimentation_disabled",
|
||||
# Evolve holds this user; the job runs once it is done.
|
||||
"memory_evolve_in_progress",
|
||||
# An embedding switch paused writes: the job stays queued and
|
||||
# sediments into the new space once the switch commits.
|
||||
"embedding_switch_in_progress",
|
||||
}
|
||||
and not hy_invoked
|
||||
):
|
||||
|
||||
@@ -1383,6 +1383,83 @@ async def _050_model_probe_operations_reserved_at(cursor: Any) -> None:
|
||||
)
|
||||
|
||||
|
||||
def embedding_spaces_ddl() -> str:
|
||||
"""The embedding space registry (`embedding_spaces.py`); the main Core SQLite only.
|
||||
|
||||
Unversioned on purpose, like the visibility tables: a migration raises
|
||||
the build's schema version, and a library this build opened would then
|
||||
refuse an older package. The table is additive and created idempotently;
|
||||
an older build ignores it and opens the space its active profile names,
|
||||
which is the registered space on every home that never switched. SQLite
|
||||
only (partial unique indexes): it lives in the main Core database.
|
||||
"""
|
||||
return """
|
||||
CREATE TABLE IF NOT EXISTS embedding_spaces (
|
||||
space_id TEXT PRIMARY KEY NOT NULL,
|
||||
state TEXT NOT NULL CHECK(state IN ('live','building','stray','retired','dropped')),
|
||||
model TEXT,
|
||||
dims INTEGER,
|
||||
name_dims INTEGER,
|
||||
collection_base TEXT NOT NULL,
|
||||
naming TEXT NOT NULL CHECK(naming IN ('legacy','v2','discovered')),
|
||||
graph_dir TEXT,
|
||||
provider_kind TEXT,
|
||||
profile_id TEXT,
|
||||
profile_revision INTEGER,
|
||||
vectors INTEGER,
|
||||
created_at TEXT NOT NULL,
|
||||
live_since TEXT,
|
||||
retired_at TEXT,
|
||||
keep_until TEXT,
|
||||
dropped_at TEXT
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS embedding_spaces_one_live
|
||||
ON embedding_spaces(state) WHERE state = 'live';
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS embedding_spaces_one_physical
|
||||
ON embedding_spaces(collection_base, COALESCE(name_dims, 0))
|
||||
WHERE state != 'dropped';
|
||||
"""
|
||||
|
||||
|
||||
def embedding_switch_jobs_ddl() -> str:
|
||||
"""Embedding switch and merge jobs (`embedding_switch.py`); unversioned, as above.
|
||||
|
||||
At most one job is open at a time (the partial unique index), and it
|
||||
lives beside the profiles and the registry so the commit is one
|
||||
transaction over all three.
|
||||
"""
|
||||
return """
|
||||
CREATE TABLE IF NOT EXISTS embedding_switch_jobs (
|
||||
job_id TEXT PRIMARY KEY NOT NULL,
|
||||
kind TEXT NOT NULL CHECK(kind IN ('switch','merge')),
|
||||
state TEXT NOT NULL CHECK(state IN (
|
||||
'preparing','reembedding','verifying','committing','done','failed','rolled_back'
|
||||
)),
|
||||
source_space_id TEXT NOT NULL,
|
||||
target_space_id TEXT NOT NULL,
|
||||
stray_space_ids TEXT NOT NULL DEFAULT '[]',
|
||||
target_profile_id TEXT,
|
||||
target_revision INTEGER,
|
||||
prior_profile_id TEXT,
|
||||
requested_by TEXT NOT NULL,
|
||||
plan_json TEXT NOT NULL,
|
||||
progress_json TEXT NOT NULL DEFAULT '{}',
|
||||
verification_json TEXT,
|
||||
error_code TEXT,
|
||||
error_detail TEXT,
|
||||
cancel_requested INTEGER NOT NULL DEFAULT 0,
|
||||
owner_pid INTEGER,
|
||||
heartbeat_at TEXT,
|
||||
attempt INTEGER NOT NULL DEFAULT 1,
|
||||
created_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL,
|
||||
finished_at TEXT
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS embedding_switch_one_open
|
||||
ON embedding_switch_jobs((1)) WHERE state NOT IN ('done','rolled_back');
|
||||
"""
|
||||
|
||||
|
||||
def model_profile_seeds_ddl(dialect: str = "mysql") -> str:
|
||||
"""Which presets have been seeded, so seeding happens once."""
|
||||
if dialect == "sqlite":
|
||||
|
||||
@@ -423,7 +423,7 @@ class ModelProber:
|
||||
ModelProbeOperations(self.store).assert_no_pending(profile.profile_id)
|
||||
# Measuring embedding width may create a new profile version of its own.
|
||||
input_revision = profile.revision
|
||||
profile = self._record_embedding_width(profile, stages)
|
||||
profile, _switch_required = self._record_embedding_width(profile, stages)
|
||||
self.store.record_capabilities(
|
||||
profile.profile_id,
|
||||
expected_revision=profile.revision,
|
||||
@@ -442,32 +442,55 @@ class ModelProber:
|
||||
self,
|
||||
profile: ModelProfile,
|
||||
stages: tuple[ProbeStage, ...],
|
||||
) -> ModelProfile:
|
||||
"""Store the measured vector width on the profile.
|
||||
) -> tuple[ModelProfile, bool]:
|
||||
"""Store the measured vector width on the profile, unless it moves the space.
|
||||
|
||||
The graph store fixes its embedding column width when it is first
|
||||
created, so guessing this leaves every relation write failing later.
|
||||
On the profile that fills the live embedding space a different width
|
||||
is not recorded: it would name other collections (B2). True when the
|
||||
width was refused that way: using it takes an embedding switch, which
|
||||
the switch plan reports (the probe result keeps its shape).
|
||||
"""
|
||||
width = next(
|
||||
(
|
||||
stage.embedding_dimensions
|
||||
for stage in stages
|
||||
if stage.stage == "embedding" and stage.embedding_dimensions
|
||||
),
|
||||
None,
|
||||
)
|
||||
width = measured_width(stages)
|
||||
if width is None or width == profile.embedding_dimensions:
|
||||
return profile
|
||||
return self.store.update(
|
||||
profile.profile_id,
|
||||
ModelProfilePatch(
|
||||
expected_revision=profile.revision,
|
||||
embedding_dimensions=width,
|
||||
return profile, False
|
||||
from one_memory.embedding_spaces import EmbeddingSpaceStore, probe_width_allowed
|
||||
|
||||
spaces = EmbeddingSpaceStore(self.store.database_path)
|
||||
with spaces.transaction() as connection:
|
||||
allowed = probe_width_allowed(
|
||||
connection,
|
||||
profile.profile_id,
|
||||
width,
|
||||
data_dir=self.store.database_path.parent,
|
||||
)
|
||||
if not allowed:
|
||||
return profile, True
|
||||
return (
|
||||
self.store.update(
|
||||
profile.profile_id,
|
||||
ModelProfilePatch(
|
||||
expected_revision=profile.revision,
|
||||
embedding_dimensions=width,
|
||||
),
|
||||
self.secrets,
|
||||
),
|
||||
self.secrets,
|
||||
False,
|
||||
)
|
||||
|
||||
|
||||
def measured_width(stages: tuple[ProbeStage, ...]) -> int | None:
|
||||
return next(
|
||||
(
|
||||
stage.embedding_dimensions
|
||||
for stage in stages
|
||||
if stage.stage == "embedding" and stage.embedding_dimensions
|
||||
),
|
||||
None,
|
||||
)
|
||||
|
||||
|
||||
def _models_request(
|
||||
profile: ModelProfile,
|
||||
secret: str | None,
|
||||
|
||||
@@ -13,6 +13,7 @@ from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
|
||||
from one_memory.database import take_the_write_lock_first
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.embedding_spaces import probe_width_allowed
|
||||
from one_memory.memory_server.migrations import model_probe_operations_ddl
|
||||
from one_memory.model_probe import ModelProbeResult, ProbeStage
|
||||
from one_memory.model_profiles import ModelCapabilities, ModelProfileStore
|
||||
@@ -349,7 +350,19 @@ class ModelProbeOperations:
|
||||
)
|
||||
revision = profile["revision"]
|
||||
now = datetime.now(UTC).isoformat()
|
||||
if width is not None and width != profile["embedding_dimensions"]:
|
||||
# The profile that fills the live embedding space records only the
|
||||
# space's own width: another one would move writes to new, empty
|
||||
# collections (B2). Using it takes an embedding switch.
|
||||
if (
|
||||
width is not None
|
||||
and width != profile["embedding_dimensions"]
|
||||
and probe_width_allowed(
|
||||
db,
|
||||
profile["profile_id"],
|
||||
width,
|
||||
data_dir=self.store.database_path.parent,
|
||||
)
|
||||
):
|
||||
revision += 1
|
||||
db.execute(
|
||||
"UPDATE model_profiles SET embedding_dimensions=?,revision=? "
|
||||
|
||||
@@ -703,10 +703,16 @@ class ModelProfileStore:
|
||||
}
|
||||
)
|
||||
# User changes to service/model/credentials need a new explicit activation.
|
||||
# Probe-discovered embedding width alone retains its existing behavior.
|
||||
# Probe-discovered embedding width alone retains its existing behavior;
|
||||
# replacing one declared width with another is a new embedding
|
||||
# identity, and activating it goes through an embedding switch.
|
||||
deactivate = (
|
||||
patch.api_key is not None
|
||||
or patch.clear_api_key
|
||||
or (
|
||||
row["embedding_dimensions"] is not None
|
||||
and next_values["embedding_dimensions"] != row["embedding_dimensions"]
|
||||
)
|
||||
or any(
|
||||
row[key] != next_values[key]
|
||||
for key in (
|
||||
|
||||
@@ -47,10 +47,13 @@ class SdkMemoryRuntime:
|
||||
extraction_settings: ExtractionSettingsStore,
|
||||
data_dir: Path,
|
||||
memory_mode: Callable[[], str] = lambda: "pro",
|
||||
writes_open: Callable[[], bool] = lambda: True,
|
||||
) -> None:
|
||||
self.coordinator = coordinator
|
||||
#: The SDK mode, for the graph store's path (`hy_graph_admin`).
|
||||
self.memory_mode = memory_mode
|
||||
#: False while an embedding switch pauses model writes.
|
||||
self.writes_open = writes_open
|
||||
self.read_handle = read_handle
|
||||
self.write_queue = write_queue
|
||||
self.extraction_settings = extraction_settings
|
||||
@@ -111,6 +114,8 @@ class SdkMemoryRuntime:
|
||||
)
|
||||
|
||||
async def digest(self) -> dict[str, object]:
|
||||
if not self.writes_open():
|
||||
raise HySdkError("embedding_switch_in_progress")
|
||||
async with self.coordinator.use_client() as client:
|
||||
try:
|
||||
raw = await asyncio.to_thread(
|
||||
@@ -151,12 +156,14 @@ class SdkMemoryRuntime:
|
||||
if str(error) == "model_profile_not_active":
|
||||
return None
|
||||
raise
|
||||
space = await self.coordinator.live_space()
|
||||
return await asyncio.to_thread(
|
||||
inspect_profile_graph_width,
|
||||
profile,
|
||||
api_key=secret,
|
||||
data_dir=self.data_dir,
|
||||
mode=self.memory_mode(),
|
||||
graph_dir=space.graph_dir if space is not None and space.graph_dir else "graph",
|
||||
)
|
||||
|
||||
async def decide_graph_width(
|
||||
@@ -167,6 +174,7 @@ class SdkMemoryRuntime:
|
||||
profile, secret = await self.coordinator.active_profile_credentials()
|
||||
except RuntimeError as error:
|
||||
raise HySdkError(str(error)) from error
|
||||
space = await self.coordinator.live_space()
|
||||
return await asyncio.to_thread(
|
||||
decide_profile_graph_width,
|
||||
profile,
|
||||
@@ -174,6 +182,7 @@ class SdkMemoryRuntime:
|
||||
data_dir=self.data_dir,
|
||||
decision=decision,
|
||||
mode=self.memory_mode(),
|
||||
graph_dir=space.graph_dir if space is not None and space.graph_dir else "graph",
|
||||
)
|
||||
|
||||
@asynccontextmanager
|
||||
@@ -183,6 +192,8 @@ class SdkMemoryRuntime:
|
||||
Callers record these against the edit or deletion instead of failing
|
||||
the request that already committed it.
|
||||
"""
|
||||
if not self.writes_open():
|
||||
raise HySdkError("embedding_switch_in_progress")
|
||||
async with AsyncExitStack() as stack:
|
||||
try:
|
||||
client = await stack.enter_async_context(self.coordinator.use_client())
|
||||
|
||||
@@ -222,13 +222,34 @@ def runtime_sdk_client(
|
||||
raise DataPlaneMigrationError("sdk_migration_secure_storage_unavailable") from None
|
||||
if secret is None:
|
||||
raise DataPlaneMigrationError("sdk_migration_model_secret_unavailable")
|
||||
lease = acquire_client(profile, api_key=secret, data_dir=data_dir)
|
||||
# The registry's live space when this home has one and the profile fills
|
||||
# it; otherwise the space the profile named before the registry existed.
|
||||
# A switch in progress never changes it: the profile and the live space
|
||||
# change together, at the switch's commit.
|
||||
space = live_space_for(database_path, profile)
|
||||
lease = (
|
||||
acquire_client(profile, space, api_key=secret, data_dir=data_dir)
|
||||
if space is not None
|
||||
else acquire_client(profile, api_key=secret, data_dir=data_dir)
|
||||
)
|
||||
try:
|
||||
yield lease.client
|
||||
finally:
|
||||
lease.close()
|
||||
|
||||
|
||||
def live_space_for(database_path: Path, profile: ModelProfileSnapshot) -> Any:
|
||||
"""The live embedding space `profile` reads, or None to derive it from the profile."""
|
||||
from one_memory.embedding_spaces import embedding_identity, read_live_space
|
||||
|
||||
space = read_live_space(database_path)
|
||||
if space is None or space.identity is None:
|
||||
return None
|
||||
if not embedding_identity(profile).matches(space.identity):
|
||||
return None
|
||||
return space
|
||||
|
||||
|
||||
def verify_readback(
|
||||
client: Any, legacy: LegacyMemoryIds, *, chat_missing_before_migration: int = 0
|
||||
) -> ReadbackReport:
|
||||
|
||||
@@ -1,20 +1,25 @@
|
||||
"""The read client, kept in the embedding space the active profile writes to.
|
||||
"""The read client, kept in the live embedding space.
|
||||
|
||||
Every read used to open the built-in 512-dim collection while writes followed
|
||||
the active profile, so a custom embedding read an empty space (#242). This
|
||||
handle opens the active profile's `retrieval_view` instead, and the built-in
|
||||
space only when no profile is active.
|
||||
the active profile, so a custom embedding read an empty space (#242). Reads
|
||||
now open the space the registry pins as live (`embedding_spaces`), which is
|
||||
the space writes fill. The embedding that reaches it is the active profile's
|
||||
when its model and width are the space's -- a new endpoint for the same model
|
||||
keeps the space -- and otherwise the profile version that filled the space
|
||||
(the built-in model for the built-in space). A switch that is still running
|
||||
never changes this: reads keep serving the old space until its commit.
|
||||
|
||||
A profile change builds a new client and retires the old one; a retired client
|
||||
is closed once no task still uses it, as `SdkWriteCoordinator` does for writes
|
||||
(#218). A use is the task that asked for the client: callers keep their
|
||||
`client = await handle.client()` shape, and the use ends with that task (one
|
||||
HTTP request, one retention pass).
|
||||
A change of space or profile builds a new client and retires the old one; a
|
||||
retired client is closed once no task still uses it, as `SdkWriteCoordinator`
|
||||
does for writes (#218). A use is the task that asked for the client: callers
|
||||
keep their `client = await handle.client()` shape, and the use ends with that
|
||||
task (one HTTP request, one retention pass).
|
||||
|
||||
The memory mode (lite / pro / ultra) is fixed per SDK client, so a mode change
|
||||
rebuilds and retires the same way, and so do the SDK features
|
||||
(`memory_features`): handles are keyed by (profile, mode, features). Neither
|
||||
mode nor features names a collection; both are separate client parameters.
|
||||
(`memory_features`) and the SDK prompt language (`ui_language`): handles are
|
||||
keyed by (space, profile, mode, features, language). None of mode, features
|
||||
or language names a collection; each is a separate client parameter.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -24,23 +29,34 @@ from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from one_memory.embedding_health import EmbeddingGuardedClient, EmbeddingHealth
|
||||
from one_memory.embedding_spaces import (
|
||||
EmbeddingSpace,
|
||||
EmbeddingSpaceStore,
|
||||
embedding_identity,
|
||||
registry_for,
|
||||
)
|
||||
from one_memory.hy_graph_admin import effective_memory_mode
|
||||
from one_memory.hy_sdk_runtime import (
|
||||
BUILTIN_RETRIEVAL_PROFILE_ID,
|
||||
SdkClientHandle,
|
||||
builtin_retrieval_snapshot,
|
||||
embedding_endpoint,
|
||||
language_kwargs,
|
||||
retrieval_view,
|
||||
)
|
||||
from one_memory.hy_sdk_types import HySdkRuntimeError
|
||||
from one_memory.memory_features import FeatureOverrides
|
||||
from one_memory.model_profiles import ModelProfileStore, ProviderKind
|
||||
from one_memory.model_profiles import ModelProfileSnapshot, ModelProfileStore, ProviderKind
|
||||
from one_memory.secrets import SecretStore, SecretStoreUnavailableError
|
||||
|
||||
_BUILTIN = (BUILTIN_RETRIEVAL_PROFILE_ID, 0)
|
||||
#: (space id, profile id, revision, memory mode, features, SDK language)
|
||||
_HandleKey = tuple[str, str, int, str, FeatureOverrides, str | None]
|
||||
|
||||
|
||||
class ActiveProfileReadHandle(SdkClientHandle):
|
||||
"""An `SdkClientHandle` whose profile follows the active one."""
|
||||
"""An `SdkClientHandle` that follows the live space and its embedding."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
@@ -50,6 +66,9 @@ class ActiveProfileReadHandle(SdkClientHandle):
|
||||
data_dir: Path,
|
||||
memory_mode: Callable[[], str] = effective_memory_mode,
|
||||
features: Callable[[], FeatureOverrides] = lambda: (),
|
||||
spaces: EmbeddingSpaceStore | None = None,
|
||||
health: EmbeddingHealth | None = None,
|
||||
language: Callable[[], str | None] = lambda: None,
|
||||
) -> None:
|
||||
super().__init__(builtin_retrieval_snapshot(), api_key=None, data_dir=data_dir)
|
||||
self.profiles = profiles
|
||||
@@ -57,7 +76,13 @@ class ActiveProfileReadHandle(SdkClientHandle):
|
||||
self.data_dir = data_dir
|
||||
self.memory_mode = memory_mode
|
||||
self.current_features = features
|
||||
self._handles: dict[tuple[str, int, str, FeatureOverrides], SdkClientHandle] = {}
|
||||
self.current_language = language
|
||||
#: None only for a profile store without a database (test doubles).
|
||||
self.spaces = spaces if spaces is not None else registry_for(profiles)
|
||||
self.health = health or EmbeddingHealth()
|
||||
self._handles: dict[_HandleKey, SdkClientHandle] = {}
|
||||
#: Whether an active profile version reads a space, per (profile, space).
|
||||
self._matches: dict[tuple[str, int, str], bool] = {}
|
||||
#: Tasks in flight per handle; a retired handle closes at zero.
|
||||
self._uses: dict[SdkClientHandle, int] = {}
|
||||
self._retired: set[SdkClientHandle] = set()
|
||||
@@ -65,22 +90,54 @@ class ActiveProfileReadHandle(SdkClientHandle):
|
||||
self._handle_lock = asyncio.Lock()
|
||||
|
||||
async def client(self) -> Any:
|
||||
profile = await asyncio.to_thread(self.profiles.active_identity) or _BUILTIN
|
||||
identity = (*profile, self.memory_mode(), self.current_features())
|
||||
handle = await self.current_handle()
|
||||
return self._guarded(handle, await handle.client())
|
||||
|
||||
def _guarded(self, handle: SdkClientHandle, client: Any) -> Any:
|
||||
"""A remote embedding's searches report `embedding_unavailable` (no fallback)."""
|
||||
profile = getattr(handle, "profile", None)
|
||||
if not isinstance(profile, ModelProfileSnapshot):
|
||||
return client
|
||||
endpoint = embedding_endpoint(profile)
|
||||
if endpoint is None:
|
||||
return client
|
||||
space = handle.space
|
||||
name_dims = space.name_dims if space is not None else profile.embedding_dimensions
|
||||
return EmbeddingGuardedClient(
|
||||
client,
|
||||
lambda: self.health.available(
|
||||
profile, endpoint=endpoint, name_dims=name_dims, api_key=handle.api_key
|
||||
),
|
||||
)
|
||||
|
||||
async def current_handle(self) -> SdkClientHandle:
|
||||
"""The handle of the live space, held for the asking task."""
|
||||
space, identity = await asyncio.to_thread(self._resolve)
|
||||
mode, features = self.memory_mode(), self.current_features()
|
||||
language = self.current_language()
|
||||
key = (
|
||||
space.space_id if space is not None else "",
|
||||
*identity,
|
||||
mode,
|
||||
features,
|
||||
language,
|
||||
)
|
||||
async with self._handle_lock:
|
||||
handle = self._handles.get(identity)
|
||||
handle = self._handles.get(key)
|
||||
if handle is None:
|
||||
# Only on a change of profile: resolving a key can reach the
|
||||
# system keychain, so it is not done on every read.
|
||||
handle = await asyncio.to_thread(self._build, identity)
|
||||
self._handles[identity] = handle
|
||||
self._retired.update(value for key, value in self._handles.items() if key != identity)
|
||||
self._handles = {identity: handle}
|
||||
# Only on a change of space or profile: resolving a key can
|
||||
# reach the system keychain, so it is not done on every read.
|
||||
handle = await asyncio.to_thread(
|
||||
self._build, space, identity, mode, features, language
|
||||
)
|
||||
self._handles[key] = handle
|
||||
self._retired.update(value for name, value in self._handles.items() if name != key)
|
||||
self._handles = {key: handle}
|
||||
unused = {value for value in self._retired if not self._uses.get(value)}
|
||||
self._retired -= unused
|
||||
self._hold(handle)
|
||||
await asyncio.gather(*(value.close() for value in unused))
|
||||
return await handle.client()
|
||||
return handle
|
||||
|
||||
async def close(self) -> None:
|
||||
async with self._handle_lock:
|
||||
@@ -88,6 +145,51 @@ class ActiveProfileReadHandle(SdkClientHandle):
|
||||
self._handles, self._retired, self._uses = {}, set(), {}
|
||||
await asyncio.gather(*(handle.close() for handle in handles), *self._closing)
|
||||
|
||||
async def close_idle(self) -> None:
|
||||
"""Retire every client; the unused close now, the rest when their task ends."""
|
||||
async with self._handle_lock:
|
||||
self._retired.update(self._handles.values())
|
||||
self._handles = {}
|
||||
unused = {value for value in self._retired if not self._uses.get(value)}
|
||||
self._retired -= unused
|
||||
await asyncio.gather(*(value.close() for value in unused))
|
||||
|
||||
def _resolve(self) -> tuple[EmbeddingSpace | None, tuple[str, int]]:
|
||||
active = self.profiles.active_identity()
|
||||
if self.spaces is None:
|
||||
return None, active or _BUILTIN
|
||||
space = self.spaces.live()
|
||||
if space is None:
|
||||
# A home this process has not bootstrapped yet: register what it
|
||||
# already uses, under the name it already has.
|
||||
space = self.spaces.bootstrap(
|
||||
self.profiles.snapshot(*active) if active is not None else None
|
||||
)
|
||||
if active is not None and self._reads(active, space):
|
||||
return space, active
|
||||
if space.profile_id is not None and space.profile_revision is not None:
|
||||
identity = (space.profile_id, space.profile_revision)
|
||||
try:
|
||||
self.profiles.snapshot(*identity)
|
||||
except KeyError:
|
||||
# Its endpoint and key went with the profile; the built-in
|
||||
# model cannot read these vectors. Say so, never read wrong.
|
||||
raise HySdkRuntimeError("embedding_space_profile_missing") from None
|
||||
return space, identity
|
||||
return space, _BUILTIN
|
||||
|
||||
def _reads(self, identity: tuple[str, int], space: EmbeddingSpace) -> bool:
|
||||
key = (*identity, space.space_id)
|
||||
known = self._matches.get(key)
|
||||
if known is None:
|
||||
if space.identity is None:
|
||||
known = False
|
||||
else:
|
||||
profile = self.profiles.snapshot(*identity)
|
||||
known = embedding_identity(profile).matches(space.identity)
|
||||
self._matches[key] = known
|
||||
return known
|
||||
|
||||
def _hold(self, handle: SdkClientHandle) -> None:
|
||||
"""Count a use of `handle` until the asking task finishes."""
|
||||
task = asyncio.current_task()
|
||||
@@ -108,9 +210,18 @@ class ActiveProfileReadHandle(SdkClientHandle):
|
||||
self._closing.add(closing)
|
||||
closing.add_done_callback(self._closing.discard)
|
||||
|
||||
def _build(self, key: tuple[str, int, str, FeatureOverrides]) -> SdkClientHandle:
|
||||
profile_id, revision, mode, features = key
|
||||
identity = (profile_id, revision)
|
||||
def _build(
|
||||
self,
|
||||
space: EmbeddingSpace | None,
|
||||
identity: tuple[str, int],
|
||||
mode: str,
|
||||
features: FeatureOverrides = (),
|
||||
language: str | None = None,
|
||||
) -> SdkClientHandle:
|
||||
# Without a registry (a profile store with no database) the handle
|
||||
# opens the space its profile names, exactly as before.
|
||||
named: dict[str, Any] = {"space": space} if space is not None else {}
|
||||
named.update(language_kwargs(language))
|
||||
if identity == _BUILTIN:
|
||||
return SdkClientHandle(
|
||||
builtin_retrieval_snapshot(),
|
||||
@@ -118,8 +229,9 @@ class ActiveProfileReadHandle(SdkClientHandle):
|
||||
data_dir=self.data_dir,
|
||||
mode=mode,
|
||||
features=features,
|
||||
**named,
|
||||
)
|
||||
profile = self.profiles.snapshot(*identity)
|
||||
profile: ModelProfileSnapshot = self.profiles.snapshot(*identity)
|
||||
view = retrieval_view(profile)
|
||||
secret: str | None = None
|
||||
# Only a remote OpenAI-compatible embedding sends the shared key.
|
||||
@@ -131,5 +243,5 @@ class ActiveProfileReadHandle(SdkClientHandle):
|
||||
if secret is None:
|
||||
raise HySdkRuntimeError("model_secret_unavailable")
|
||||
return SdkClientHandle(
|
||||
view, api_key=secret, data_dir=self.data_dir, mode=mode, features=features
|
||||
view, api_key=secret, data_dir=self.data_dir, mode=mode, features=features, **named
|
||||
)
|
||||
@@ -73,6 +73,30 @@ def _stored_names(data_dir: Path, *, filled: bool = False) -> tuple[str, ...]:
|
||||
return tuple(sorted(str(row[0]) for row in rows))
|
||||
|
||||
|
||||
def filled_collection_names(data_dir: Path) -> tuple[str, ...]:
|
||||
"""Every vector collection holding at least one record. Read-only.
|
||||
|
||||
Safe beside a serving Core: it reads Chroma's own SQLite file, never the
|
||||
client system the SDK shares.
|
||||
"""
|
||||
return _stored_names(data_dir, filled=True)
|
||||
|
||||
|
||||
def stored_collection_width(data_dir: Path, name: str) -> int | None:
|
||||
"""The vector width Chroma fixed for `name` on its first add; None if unset."""
|
||||
database = _vector_directory(data_dir) / "chroma.sqlite3"
|
||||
if not database.is_file():
|
||||
return None
|
||||
with closing(sqlite3.connect(f"file:{database}?mode=ro", uri=True)) as connection:
|
||||
try:
|
||||
row = connection.execute(
|
||||
"SELECT dimension FROM collections WHERE name=?", (name,)
|
||||
).fetchone()
|
||||
except sqlite3.Error:
|
||||
return None
|
||||
return int(row[0]) if row is not None and row[0] is not None else None
|
||||
|
||||
|
||||
def legacy_vector_collections(
|
||||
data_dir: Path, profile: ModelProfileSnapshot
|
||||
) -> tuple[tuple[str, str], ...]:
|
||||
|
||||
@@ -9,8 +9,18 @@ from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from one_memory.embedding_spaces import (
|
||||
EmbeddingSpace,
|
||||
EmbeddingSpaceStore,
|
||||
embedding_identity,
|
||||
registry_for,
|
||||
)
|
||||
from one_memory.hy_graph_admin import effective_memory_mode
|
||||
from one_memory.hy_sdk_runtime import SdkClientHandle, builtin_retrieval_snapshot
|
||||
from one_memory.hy_sdk_runtime import (
|
||||
SdkClientHandle,
|
||||
builtin_retrieval_snapshot,
|
||||
language_kwargs,
|
||||
)
|
||||
from one_memory.imports.models import EvidenceContentKind, EvidenceOrigin, EvidenceRecord
|
||||
from one_memory.input_sanitization import sanitize_model_input
|
||||
from one_memory.memory_features import FeatureOverrides
|
||||
@@ -147,15 +157,22 @@ class SdkWriteCoordinator:
|
||||
the SharedRuntime closed mid-`add`, the job failed with AttributeError
|
||||
after HY was invoked, and it was left `unknown`.
|
||||
|
||||
Handles are keyed by profile, memory mode and the SDK features
|
||||
(`memory_features`), so a mode or feature change retires handles the same
|
||||
way. In lite no generation model is needed: without an
|
||||
active profile the built-in local embedder writes raw turns.
|
||||
Handles are keyed by profile, memory mode, the SDK features
|
||||
(`memory_features`) and the SDK language (`ui_language`), so a mode,
|
||||
feature or language change retires handles the same way. In lite no
|
||||
generation model is needed: without an active profile the built-in
|
||||
local embedder writes raw turns.
|
||||
|
||||
`exclusive_for_evolve(user_id)` keeps evolve and that user's writes
|
||||
apart: it waits for the user's write in flight, and while it runs that
|
||||
user's jobs are refused before the SDK is called
|
||||
(`memory_evolve_in_progress`), so the drain puts them back.
|
||||
|
||||
Writes go to the live embedding space only. An active profile whose
|
||||
embedding (model and width) is not the live space's is refused with
|
||||
`embedding_space_mismatch`: it used to open a new, empty collection.
|
||||
While an embedding switch runs, `writes_open` is false and no write
|
||||
starts (`embedding_switch_in_progress`); the job waits in the queue.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
@@ -168,6 +185,9 @@ class SdkWriteCoordinator:
|
||||
sedimentation_enabled: Callable[[], bool],
|
||||
memory_mode: Callable[[], str] = effective_memory_mode,
|
||||
memory_features: Callable[[], FeatureOverrides] = lambda: (),
|
||||
spaces: EmbeddingSpaceStore | None = None,
|
||||
writes_open: Callable[[], bool] = lambda: True,
|
||||
sdk_language: Callable[[], str | None] = lambda: None,
|
||||
) -> None:
|
||||
self.profiles = profiles
|
||||
self.secrets = secrets
|
||||
@@ -176,7 +196,12 @@ class SdkWriteCoordinator:
|
||||
self.sedimentation_enabled = sedimentation_enabled
|
||||
self.memory_mode = memory_mode
|
||||
self.memory_features = memory_features
|
||||
self._handles: dict[tuple[str, int, str, FeatureOverrides], SdkClientHandle] = {}
|
||||
self.sdk_language = sdk_language
|
||||
#: Keyed by (profile id, revision, space id, memory mode, features,
|
||||
#: SDK language).
|
||||
self._handles: dict[
|
||||
tuple[str, int, str, str, FeatureOverrides, str | None], SdkClientHandle
|
||||
] = {}
|
||||
#: Users evolve (or the graph backfill) is running for, the writes in
|
||||
#: flight per user, and one exclusive run per user at a time. Writes
|
||||
#: of one user still run side by side; only evolve waits for them.
|
||||
@@ -184,6 +209,10 @@ class SdkWriteCoordinator:
|
||||
self._writing: dict[str, int] = {}
|
||||
self._writes_idle = asyncio.Condition()
|
||||
self._exclusive: dict[str, asyncio.Lock] = {}
|
||||
#: None only for a profile store without a database (test doubles):
|
||||
#: then every client opens the space its profile names, as before.
|
||||
self.spaces = spaces if spaces is not None else registry_for(profiles)
|
||||
self.writes_open = writes_open
|
||||
#: Uses in flight per handle; a retired handle closes at zero.
|
||||
self._uses: dict[SdkClientHandle, int] = {}
|
||||
self._retired: set[SdkClientHandle] = set()
|
||||
@@ -198,6 +227,9 @@ class SdkWriteCoordinator:
|
||||
) -> SdkWriteReceipt:
|
||||
if not self.sedimentation_enabled():
|
||||
raise RuntimeError("model_sedimentation_disabled")
|
||||
if not self.writes_open():
|
||||
# An embedding switch paused writes: the drain puts the job back.
|
||||
raise RuntimeError("embedding_switch_in_progress")
|
||||
user_id = records[0].user_id if records else "local"
|
||||
if user_id in self._evolving:
|
||||
# Before any SDK call: the drain returns the job to pending.
|
||||
@@ -222,15 +254,15 @@ class SdkWriteCoordinator:
|
||||
self._writes_idle.notify_all()
|
||||
|
||||
async def retire_stale_handles(self) -> None:
|
||||
"""After a mode or feature change: retire handles built otherwise.
|
||||
"""After a mode, feature or language change: retire handles built otherwise.
|
||||
|
||||
The idle ones close now. A handle still writing is closed when its
|
||||
last use ends (#218), so a write that started in the old mode, or
|
||||
with the old features, finishes in it.
|
||||
"""
|
||||
build = (self.memory_mode(), self.memory_features())
|
||||
build = (self.memory_mode(), self.memory_features(), self.sdk_language())
|
||||
async with self._lock:
|
||||
stale = {key: handle for key, handle in self._handles.items() if key[2:] != build}
|
||||
stale = {key: handle for key, handle in self._handles.items() if key[3:] != build}
|
||||
for key in stale:
|
||||
del self._handles[key]
|
||||
self._retired.update(stale.values())
|
||||
@@ -250,8 +282,11 @@ class SdkWriteCoordinator:
|
||||
"""Hold one user's writes off while evolve runs for them.
|
||||
|
||||
New writes of the user are refused from here on; the ones already in
|
||||
flight are waited for before the body runs.
|
||||
flight are waited for before the body runs. Evolve and the graph
|
||||
backfill write too, so an embedding switch refuses them as well.
|
||||
"""
|
||||
if not self.writes_open():
|
||||
raise RuntimeError("embedding_switch_in_progress")
|
||||
async with self._exclusive.setdefault(user_id, asyncio.Lock()):
|
||||
self._evolving.add(user_id)
|
||||
try:
|
||||
@@ -271,6 +306,19 @@ class SdkWriteCoordinator:
|
||||
self._handles, self._retired, self._uses = {}, set(), {}
|
||||
await asyncio.gather(*(handle.close() for handle in handles))
|
||||
|
||||
def in_use(self) -> int:
|
||||
"""Writes and reads holding a client right now."""
|
||||
return sum(self._uses.values())
|
||||
|
||||
async def close_idle(self) -> None:
|
||||
"""Retire every handle; the unused close now, the rest when released."""
|
||||
async with self._lock:
|
||||
self._retired.update(self._handles.values())
|
||||
self._handles = {}
|
||||
unused = {value for value in self._retired if not self._uses.get(value)}
|
||||
self._retired -= unused
|
||||
await asyncio.gather(*(value.close() for value in unused))
|
||||
|
||||
@asynccontextmanager
|
||||
async def use_client(self) -> AsyncIterator[Any]:
|
||||
"""The public SDK client of the active write profile, kept open while used."""
|
||||
@@ -281,20 +329,32 @@ class SdkWriteCoordinator:
|
||||
self,
|
||||
) -> tuple[ModelProfileSnapshot, str | None]:
|
||||
"""Resolve one stable profile snapshot for graph administration."""
|
||||
_identity, profile, secret = await asyncio.to_thread(self._read_active_credentials)
|
||||
_identity, profile, secret, _space = await asyncio.to_thread(self._read_active_credentials)
|
||||
return profile, secret
|
||||
|
||||
async def live_space(self) -> EmbeddingSpace | None:
|
||||
if self.spaces is None:
|
||||
return None
|
||||
return await asyncio.to_thread(self._live_space)
|
||||
|
||||
@asynccontextmanager
|
||||
async def _active_handle(self) -> AsyncIterator[SdkClientHandle]:
|
||||
"""Use the active profile's handle; retire the others, closing the unused."""
|
||||
mode = self.memory_mode()
|
||||
features = self.memory_features()
|
||||
profile_identity, profile, secret = await asyncio.to_thread(
|
||||
language = self.sdk_language()
|
||||
profile_identity, profile, secret, space = await asyncio.to_thread(
|
||||
self._read_active_credentials, mode
|
||||
)
|
||||
identity = (*profile_identity, mode, features)
|
||||
key = (
|
||||
*profile_identity,
|
||||
space.space_id if space is not None else "",
|
||||
mode,
|
||||
features,
|
||||
language,
|
||||
)
|
||||
async with self._lock:
|
||||
handle = self._handles.get(identity)
|
||||
handle = self._handles.get(key)
|
||||
if handle is None:
|
||||
handle = SdkClientHandle(
|
||||
profile,
|
||||
@@ -303,10 +363,12 @@ class SdkWriteCoordinator:
|
||||
http_transport_factory=self.transport_factory(profile),
|
||||
mode=mode,
|
||||
features=features,
|
||||
space=space,
|
||||
**language_kwargs(language),
|
||||
)
|
||||
self._handles[identity] = handle
|
||||
self._retired.update(value for key, value in self._handles.items() if key != identity)
|
||||
self._handles = {identity: handle}
|
||||
self._handles[key] = handle
|
||||
self._retired.update(value for name, value in self._handles.items() if name != key)
|
||||
self._handles = {key: handle}
|
||||
self._uses[handle] = self._uses.get(handle, 0) + 1
|
||||
unused = {value for value in self._retired if not self._uses.get(value)}
|
||||
self._retired -= unused
|
||||
@@ -325,18 +387,51 @@ class SdkWriteCoordinator:
|
||||
self._retired.discard(handle)
|
||||
await handle.close()
|
||||
|
||||
def _live_space(self) -> EmbeddingSpace:
|
||||
assert self.spaces is not None
|
||||
space = self.spaces.live()
|
||||
if space is None:
|
||||
active = self.profiles.active_identity()
|
||||
space = self.spaces.bootstrap(
|
||||
self.profiles.snapshot(*active) if active is not None else None
|
||||
)
|
||||
return space
|
||||
|
||||
def _lite_credentials(
|
||||
self,
|
||||
) -> tuple[tuple[str, int], ModelProfileSnapshot, str | None, EmbeddingSpace | None]:
|
||||
space = self._live_space() if self.spaces is not None else None
|
||||
if space is None or space.profile_id is None or space.profile_revision is None:
|
||||
builtin = builtin_retrieval_snapshot()
|
||||
return (builtin.profile_id, builtin.revision), builtin, None, space
|
||||
identity = (space.profile_id, space.profile_revision)
|
||||
profile = self.profiles.snapshot(*identity)
|
||||
try:
|
||||
secret = self.profiles.resolve_snapshot_secret(*identity, self.secrets)
|
||||
except SecretStoreUnavailableError:
|
||||
raise RuntimeError("secure_storage_unavailable") from None
|
||||
if profile.has_api_key and secret is None:
|
||||
raise RuntimeError("model_secret_unavailable")
|
||||
return identity, profile, secret, space
|
||||
|
||||
def _read_active_credentials(
|
||||
self,
|
||||
mode: str = "pro",
|
||||
) -> tuple[tuple[str, int], ModelProfileSnapshot, str | None]:
|
||||
) -> tuple[tuple[str, int], ModelProfileSnapshot, str | None, EmbeddingSpace | None]:
|
||||
identity = self.profiles.active_identity()
|
||||
if identity is None:
|
||||
if mode == "lite":
|
||||
# lite stores raw turns with one embedding and calls no model.
|
||||
builtin = builtin_retrieval_snapshot()
|
||||
return (builtin.profile_id, builtin.revision), builtin, None
|
||||
# lite stores raw turns with one embedding and calls no model:
|
||||
# the live space's own, so they land where reads look.
|
||||
return self._lite_credentials()
|
||||
raise RuntimeError("model_profile_not_active")
|
||||
profile = self.profiles.snapshot(*identity)
|
||||
space = self._live_space() if self.spaces is not None else None
|
||||
if space is not None and (
|
||||
space.identity is None or not embedding_identity(profile).matches(space.identity)
|
||||
):
|
||||
# Never a new, empty collection: the switch moves the space.
|
||||
raise RuntimeError("embedding_space_mismatch")
|
||||
try:
|
||||
secret = self.profiles.resolve_snapshot_secret(
|
||||
*identity,
|
||||
@@ -348,7 +443,7 @@ class SdkWriteCoordinator:
|
||||
raise RuntimeError("model_profile_changed")
|
||||
if profile.has_api_key and secret is None:
|
||||
raise RuntimeError("model_secret_unavailable")
|
||||
return identity, profile, secret
|
||||
return identity, profile, secret, space
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
|
||||
@@ -145,6 +145,10 @@ async def search_within(
|
||||
return (), "failed"
|
||||
error = pending.exception()
|
||||
if error is not None:
|
||||
if getattr(error, "code", None) == "embedding_unavailable":
|
||||
# Not "HY found nothing": the query could not be embedded, and
|
||||
# the caller must say so instead of answering empty.
|
||||
raise error
|
||||
return (), "failed"
|
||||
found = pending.result()
|
||||
normalized = normalize_semantic_hits(found)
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
"""The display language a person chose, and the SDK prompt language it implies.
|
||||
|
||||
The App owns the choice (its language selector) and tells Core whenever it
|
||||
changes, so Core can build every SDK client with it. It is one global value in
|
||||
`config.json` (`ui_language`), beside `memory_mode` and `memory_features`;
|
||||
absent means no language was set (a CLI/API-only install, or
|
||||
`settings language system`). An App that follows the system sends the
|
||||
language it resolved and shows, not null.
|
||||
|
||||
The owner's rule for the SDK:
|
||||
|
||||
- Any supported code, zh-Hans and en included: pass it as
|
||||
`MemoryConfig.language`, so extraction and the other prompts use that
|
||||
language. Every display language is passed explicitly (#243).
|
||||
- No language at all (never set, as on a CLI/API-only install): pass no
|
||||
language. The SDK keeps detecting Chinese or English from the text, exactly
|
||||
as before this setting existed. The App does not leave it unset: it sends
|
||||
the language it shows, including when it follows the system.
|
||||
|
||||
The language never names a collection; a change rebuilds the SDK handles the
|
||||
way a mode or feature change does (#218 use count).
|
||||
|
||||
No SDK import here: only `hy_sdk_runtime` and `hy_sdk_migration` import it.
|
||||
`config_state` imports this module, so it imports nothing from there.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Literal
|
||||
|
||||
UiLanguage = Literal[
|
||||
"en",
|
||||
"zh-Hans",
|
||||
"zh-Hant",
|
||||
"ja",
|
||||
"es",
|
||||
"de",
|
||||
"pt",
|
||||
"fr",
|
||||
"ko",
|
||||
"ru",
|
||||
"id",
|
||||
]
|
||||
UI_LANGUAGES: tuple[UiLanguage, ...] = (
|
||||
"en",
|
||||
"zh-Hans",
|
||||
"zh-Hant",
|
||||
"ja",
|
||||
"es",
|
||||
"de",
|
||||
"pt",
|
||||
"fr",
|
||||
"ko",
|
||||
"ru",
|
||||
"id",
|
||||
)
|
||||
#: Codes earlier App builds stored, and the code each one is now.
|
||||
LEGACY_UI_LANGUAGES: dict[str, UiLanguage] = {"zh-CN": "zh-Hans"}
|
||||
#: What status shows when no language is passed to the SDK.
|
||||
SDK_LANGUAGE_AUTO = "auto"
|
||||
|
||||
|
||||
def normalize_ui_language(value: object) -> UiLanguage | None:
|
||||
"""A supported code (legacy ones mapped), or None for anything else."""
|
||||
if not isinstance(value, str):
|
||||
return None
|
||||
value = LEGACY_UI_LANGUAGES.get(value, value)
|
||||
return value if value in UI_LANGUAGES else None
|
||||
|
||||
|
||||
def sdk_language(ui_language: str | None) -> str | None:
|
||||
"""The `MemoryConfig.language` for a display language; None = SDK auto-detect.
|
||||
|
||||
Every supported code is passed as is; only an unset (or unknown) value
|
||||
leaves the SDK to detect zh/en.
|
||||
"""
|
||||
return normalize_ui_language(ui_language)
|
||||
@@ -88,6 +88,9 @@
|
||||
| `test_diagnostic_trace_api.py` | `integration` | 覆盖 trace lookup is disabled until detailed logging is enabled、trace lookup returns only safe structured steps、log level route applies current process level 等 4 个场景。 |
|
||||
| `test_disable_thinking.py` | `contract` | 运行期生成请求须按档案可选关闭思考;默认不向厂商下发未知字段。 |
|
||||
| `test_dual_engine_helper.py` | `contract` | Tests for the dual-engine harness itself. |
|
||||
| `test_embedding_switch.py` | `contract` | Switching the embedding model, and merging stray spaces, without losing a memory. |
|
||||
| `test_embedding_switch_api.py` | `integration` | The embedding switch through Core's HTTP surface and lifespan. |
|
||||
| `test_embedding_switch_sdk_adapter.py` | `contract` | The re-embed adapter against the SDK's real re-embed module and Chroma. |
|
||||
| `test_event_loop_blocking.py` | `contract` | Proof, as a test, that blocking work on the loop freezes it — and the fix works. |
|
||||
| `test_evidence_store.py` | `contract` | 覆盖 same source message merges origins without duplication、identical text from different sources is not deduplicated、rescanning an unchanged file does not write the message again 等 26 个场景。 |
|
||||
| `test_export_probe.py` | `contract` | What a probe may say about a stranger's chat history, and what it may not. |
|
||||
@@ -301,6 +304,7 @@
|
||||
| `test_sync_facade.py` | `contract` | A2: the sync facade must keep the sync shape call sites already use. |
|
||||
| `test_tail_metadata_lane.py` | `contract` | Post-response metadata retries must be bounded and never block the route. |
|
||||
| `test_two_level_discovery.py` | `contract` | 3925a kernel item 1: two-level discovery. |
|
||||
| `test_ui_language.py` | `contract` | The App's display language, and the SDK prompt language it implies. |
|
||||
| `test_ui_session.py` | `contract` | 覆盖 exchange is single use and creates a session、expired exchange cannot create a session、exchange is bound to the core instance 等 5 个场景。 |
|
||||
| `test_ui_session_api.py` | `integration` | 覆盖 ui session exchange sets httponly strict cookie、ui session requires bearer for exchange、ui session reports missing web assets 场景。 |
|
||||
| `test_unanswered_turn.py` | `contract` | BUG-06: which conversations end on a question nobody answered. |
|
||||
@@ -314,4 +318,4 @@
|
||||
| `test_workspace_recall.py` | `contract` | Workspace memories were unreachable from every host; these pin the way in. |
|
||||
| `test_write_queue_lease_renewal.py` | `contract` | Rule 8: a lease must outlive the work it covers, and be renewed while it runs. |
|
||||
|
||||
共 309 个测试文件。
|
||||
共 313 个测试文件。
|
||||
@@ -14,8 +14,8 @@ from pathlib import Path
|
||||
import pytest
|
||||
|
||||
PRIMARY_MARKERS = ("unit", "contract", "integration", "performance", "live")
|
||||
EXPECTED_TEST_FILE_COUNT = 309
|
||||
EXPECTED_TEST_FILES_SHA256 = "a7d6ed33175f618d2d40f401d447a5553e1347ffd51dd878370c0fc84d13396d"
|
||||
EXPECTED_TEST_FILE_COUNT = 313
|
||||
EXPECTED_TEST_FILES_SHA256 = "dcada5d5421767196a79e415dd47c2b6c6d67681972dbe81481b31867d318928"
|
||||
|
||||
# Pure transformations, validation models, parity checks and state machines.
|
||||
# This list is intentionally small: uncertain files default to contract.
|
||||
@@ -54,6 +54,7 @@ INTEGRATION_FILES = frozenset(
|
||||
"test_backup_endpoint.py",
|
||||
"test_recall_raw_fallback.py",
|
||||
"test_core_ready_before_recovery.py",
|
||||
"test_embedding_switch_api.py",
|
||||
"test_backup_restore_api.py",
|
||||
"test_c1_import_via_api_path.py",
|
||||
"test_capture_deadline.py",
|
||||
|
||||
@@ -0,0 +1,301 @@
|
||||
"""An in-memory `ReembedBackend` for embedding switch tests.
|
||||
|
||||
It keeps collections as plain dicts named exactly as the SDK names them, so a
|
||||
test can lay out a home -- a live space, a stray 512 space left by an older
|
||||
build, a retired space -- and check what a switch or merge did to each. The
|
||||
vectors themselves are not computed: a record remembers the model and width
|
||||
it was "embedded" with, which is what the assertions need.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import re
|
||||
import shutil
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
from one_memory.embedding_backend import (
|
||||
EmbeddingTarget,
|
||||
EmbeddingWidthError,
|
||||
GraphCheck,
|
||||
PhysicalSpace,
|
||||
ReembedCancelledError,
|
||||
ReembedStep,
|
||||
ScannedSpace,
|
||||
VectorCheck,
|
||||
)
|
||||
from one_memory.embedding_spaces import VECTOR_ROLES, space_collection_names
|
||||
|
||||
_ROLE_SUFFIX = re.compile(
|
||||
r"^(?P<base>.+?)_(?P<role>coding_keys_cv|coding_keys|working_facts_cv|working_facts)"
|
||||
r"(?:_(?P<dims>\d+))?$"
|
||||
)
|
||||
_DIMS_SUFFIX = re.compile(r"^(?P<base>.+)_(?P<dims>\d+)$")
|
||||
|
||||
|
||||
class Crash(BaseException):
|
||||
"""Stands in for the process dying: nothing in the worker catches it."""
|
||||
|
||||
|
||||
@dataclass
|
||||
class Record:
|
||||
document: str
|
||||
metadata: dict[str, object]
|
||||
width: int
|
||||
model: str
|
||||
|
||||
|
||||
@dataclass
|
||||
class FakeVectorStore:
|
||||
collections: dict[str, dict[str, Record]] = field(default_factory=dict)
|
||||
|
||||
def put(
|
||||
self,
|
||||
base: str,
|
||||
name_dims: int | None,
|
||||
records: dict[str, tuple[str, str]],
|
||||
*,
|
||||
role: str = "memory",
|
||||
width: int | None = None,
|
||||
model: str = "old-model",
|
||||
) -> None:
|
||||
"""`records` maps id -> (document, user_id)."""
|
||||
name = space_collection_names(base, name_dims)[role]
|
||||
target = self.collections.setdefault(name, {})
|
||||
for record_id, (document, user) in records.items():
|
||||
target[record_id] = Record(
|
||||
document=document,
|
||||
metadata={"user_id": user, "content": document},
|
||||
width=width or name_dims or 0,
|
||||
model=model,
|
||||
)
|
||||
|
||||
def space(self, base: str, name_dims: int | None) -> dict[str, dict[str, Record]]:
|
||||
return {
|
||||
role: dict(self.collections.get(name, {}))
|
||||
for role, name in space_collection_names(base, name_dims).items()
|
||||
if name in self.collections
|
||||
}
|
||||
|
||||
def ids(self, base: str, name_dims: int | None, role: str = "memory") -> set[str]:
|
||||
return set(self.collections.get(space_collection_names(base, name_dims)[role], {}))
|
||||
|
||||
|
||||
def parse(name: str) -> tuple[str, int | None, str] | None:
|
||||
if name.endswith("_tag_index"):
|
||||
return None
|
||||
match = _ROLE_SUFFIX.match(name)
|
||||
if match:
|
||||
dims = match.group("dims")
|
||||
return match.group("base"), int(dims) if dims else None, match.group("role")
|
||||
role = "memory"
|
||||
if name.endswith("_entities"):
|
||||
name, role = name[: -len("_entities")], "entities"
|
||||
match = _DIMS_SUFFIX.match(name)
|
||||
if match:
|
||||
return match.group("base"), int(match.group("dims")), role
|
||||
return name, None, role
|
||||
|
||||
|
||||
class FakeReembedBackend:
|
||||
"""The backend contract, over `FakeVectorStore`."""
|
||||
|
||||
def __init__(self, store: FakeVectorStore | None = None) -> None:
|
||||
self.store = store or FakeVectorStore()
|
||||
self.unavailable: str | None = None
|
||||
#: Called before every batch; a test raises `Crash` or flips a cancel here.
|
||||
self.before_batch: Callable[[str, int], None] = lambda _role, _done: None
|
||||
#: Width the "embedding service" returns; None returns the target's.
|
||||
self.returned_width: int | None = None
|
||||
#: Ids the re-embed silently loses, to prove verification catches it.
|
||||
self.lose_ids: set[str] = set()
|
||||
self.batch_size = 2
|
||||
self.reembed_calls: list[tuple[str, str]] = []
|
||||
self.dropped: list[str] = []
|
||||
self.graphs_built: list[Path] = []
|
||||
#: When set, every batch waits for it: a switch a test holds mid-way.
|
||||
self.pause: asyncio.Event | None = None
|
||||
|
||||
def unavailable_reason(self) -> str | None:
|
||||
return self.unavailable
|
||||
|
||||
async def scan(self, *, per_user: bool = False) -> tuple[ScannedSpace, ...]:
|
||||
grouped: dict[tuple[str, int | None], dict[str, str]] = {}
|
||||
for name in self.store.collections:
|
||||
parsed = parse(name)
|
||||
if parsed is None:
|
||||
continue
|
||||
base, dims, role = parsed
|
||||
grouped.setdefault((base, dims), {})[role] = name
|
||||
out = []
|
||||
for (base, dims), roles in sorted(
|
||||
grouped.items(), key=lambda kv: (kv[0][0], kv[0][1] or 0)
|
||||
):
|
||||
counts = {role: len(self.store.collections[name]) for role, name in roles.items()}
|
||||
widths = {
|
||||
record.width
|
||||
for name in roles.values()
|
||||
for record in self.store.collections[name].values()
|
||||
}
|
||||
users: dict[str, int] = {}
|
||||
if per_user:
|
||||
for name in roles.values():
|
||||
for record in self.store.collections[name].values():
|
||||
user = str(record.metadata.get("user_id") or "")
|
||||
users[user] = users.get(user, 0) + 1
|
||||
out.append(
|
||||
ScannedSpace(
|
||||
collection_base=base,
|
||||
name_dims=dims,
|
||||
width=widths.pop() if len(widths) == 1 else None,
|
||||
counts=counts,
|
||||
collections=roles,
|
||||
per_user=users,
|
||||
)
|
||||
)
|
||||
return tuple(out)
|
||||
|
||||
async def diff(self, source: PhysicalSpace, target: PhysicalSpace) -> dict[str, int]:
|
||||
target_names = target.names()
|
||||
out = {}
|
||||
for role, name in source.names().items():
|
||||
have = set(self.store.collections.get(target_names.get(role, ""), {}))
|
||||
out[role] = len(set(self.store.collections.get(name, {})) - have)
|
||||
return out
|
||||
|
||||
async def reembed(
|
||||
self,
|
||||
source: PhysicalSpace,
|
||||
target: PhysicalSpace,
|
||||
*,
|
||||
embedding: EmbeddingTarget,
|
||||
cancel: Callable[[], bool],
|
||||
on_step: Callable[[ReembedStep], None],
|
||||
batch_size: int = 64,
|
||||
) -> None:
|
||||
del batch_size
|
||||
if target.dims is None:
|
||||
raise ValueError("the target space needs an explicit width")
|
||||
self.reembed_calls.append((source.collection_base, target.collection_base))
|
||||
target_names = target.names()
|
||||
model = embedding.space.model or "?"
|
||||
for role in VECTOR_ROLES:
|
||||
name = source.names().get(role)
|
||||
if name is None or name not in self.store.collections:
|
||||
continue
|
||||
records = sorted(self.store.collections[name].items())
|
||||
destination = self.store.collections.setdefault(target_names[role], {})
|
||||
done = embedded = kept = 0
|
||||
for start in range(0, len(records), self.batch_size):
|
||||
if cancel():
|
||||
raise ReembedCancelledError("embedding_switch_cancelled")
|
||||
self.before_batch(role, done)
|
||||
if self.pause is not None:
|
||||
await self.pause.wait()
|
||||
batch = records[start : start + self.batch_size]
|
||||
width = self.returned_width or target.dims
|
||||
if width != target.dims:
|
||||
raise EmbeddingWidthError("embedding_dimensions_mismatch")
|
||||
for record_id, record in batch:
|
||||
if record_id in destination:
|
||||
kept += 1
|
||||
continue
|
||||
if record_id in self.lose_ids:
|
||||
continue
|
||||
destination[record_id] = Record(
|
||||
document=record.document,
|
||||
metadata=dict(record.metadata),
|
||||
width=width,
|
||||
model=model,
|
||||
)
|
||||
embedded += 1
|
||||
done += len(batch)
|
||||
on_step(
|
||||
ReembedStep(
|
||||
phase="vectors",
|
||||
role=role,
|
||||
done=done,
|
||||
total=len(records),
|
||||
embedded=embedded,
|
||||
kept=kept,
|
||||
)
|
||||
)
|
||||
|
||||
async def verify(
|
||||
self, source: PhysicalSpace, target: PhysicalSpace, *, embedding: EmbeddingTarget
|
||||
) -> VectorCheck:
|
||||
del embedding
|
||||
problems: list[str] = []
|
||||
per_role: dict[str, dict[str, int]] = {}
|
||||
users_source: dict[str, int] = {}
|
||||
users_target: dict[str, int] = {}
|
||||
target_names = target.names()
|
||||
for role, name in source.names().items():
|
||||
src = self.store.collections.get(name, {})
|
||||
tgt = self.store.collections.get(target_names.get(role, ""), {})
|
||||
missing = set(src) - set(tgt)
|
||||
extra = set(tgt) - set(src)
|
||||
mismatched = {
|
||||
key for key in set(src) & set(tgt) if src[key].document != tgt[key].document
|
||||
}
|
||||
bad = sum(1 for record in tgt.values() if record.width != target.dims)
|
||||
per_role[role] = {
|
||||
"source": len(src),
|
||||
"target": len(tgt),
|
||||
"missing": len(missing),
|
||||
"extra": len(extra),
|
||||
"mismatched": len(mismatched),
|
||||
"bad_dims": bad,
|
||||
}
|
||||
for label, value in (
|
||||
("missing", missing),
|
||||
("extra", extra),
|
||||
("mismatched", mismatched),
|
||||
):
|
||||
if value:
|
||||
problems.append(f"{role}: {len(value)} {label}")
|
||||
if bad:
|
||||
problems.append(f"{role}: {bad} bad dims")
|
||||
for record in src.values():
|
||||
user = str(record.metadata.get("user_id") or "")
|
||||
users_source[user] = users_source.get(user, 0) + 1
|
||||
for record in tgt.values():
|
||||
user = str(record.metadata.get("user_id") or "")
|
||||
users_target[user] = users_target.get(user, 0) + 1
|
||||
for user in sorted(set(users_source) | set(users_target)):
|
||||
if users_source.get(user, 0) != users_target.get(user, 0):
|
||||
problems.append(f"user {user!r}: counts differ")
|
||||
return VectorCheck(per_role=per_role, per_user=users_target, problems=tuple(problems))
|
||||
|
||||
async def graph_vectors(self, path: Path) -> int | None:
|
||||
return 3 if path.exists() else None
|
||||
|
||||
async def reembed_graph(
|
||||
self,
|
||||
source: Path,
|
||||
target: Path,
|
||||
*,
|
||||
embedding: EmbeddingTarget,
|
||||
target_dims: int,
|
||||
cancel: Callable[[], bool],
|
||||
on_step: Callable[[ReembedStep], None],
|
||||
) -> None:
|
||||
del embedding, cancel
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
(target / "width").write_text(str(target_dims))
|
||||
self.graphs_built.append(target)
|
||||
on_step(ReembedStep(phase="graph", role="Memory", done=3, total=3, embedded=3))
|
||||
|
||||
async def verify_graph(self, source: Path, target: Path, *, target_dims: int) -> GraphCheck:
|
||||
ok = (target / "width").is_file() and (target / "width").read_text() == str(target_dims)
|
||||
return GraphCheck(ok=ok, problems=() if ok else ("graph width",))
|
||||
|
||||
async def drop(self, space: PhysicalSpace) -> None:
|
||||
for name in space.names().values():
|
||||
if self.store.collections.pop(name, None) is not None:
|
||||
self.dropped.append(name)
|
||||
|
||||
async def drop_graph(self, path: Path) -> None:
|
||||
shutil.rmtree(path, ignore_errors=True)
|
||||
@@ -164,7 +164,16 @@ def test_metadata_repair_never_overlaps_an_sdk_client(tmp_path, monkeypatch):
|
||||
events.append("repair_finished")
|
||||
return MetadataRepairResult(None, 0, 0, 0)
|
||||
|
||||
def open_client(profile, *, api_key, data_dir, http_transport_factory, mode="pro", features=()):
|
||||
def open_client(
|
||||
profile,
|
||||
space=None,
|
||||
*,
|
||||
api_key,
|
||||
data_dir,
|
||||
http_transport_factory,
|
||||
mode="pro",
|
||||
features=(),
|
||||
):
|
||||
events.append("client_opened")
|
||||
return ClientLease(object(), data_dir.resolve())
|
||||
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,383 @@
|
||||
"""The embedding switch through Core's HTTP surface and lifespan.
|
||||
|
||||
Activation refusals and the empty-home path, a whole switch driven over the
|
||||
API with writes paused and intake still queueing, the startup merge of a
|
||||
stray space before the drain starts, and `embedding_unavailable` on recall
|
||||
and context search when the remote embedding does not answer.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import time
|
||||
from types import SimpleNamespace
|
||||
from uuid import uuid4
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
from pydantic import SecretStr
|
||||
|
||||
from one_memory import embedding_health
|
||||
from one_memory.api import create_app
|
||||
from one_memory.config import Settings
|
||||
from one_memory.model_profiles import ModelCapabilities, ModelProfileInput, ProviderKind
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
from tests.embedding_fakes import FakeReembedBackend
|
||||
|
||||
pytestmark = pytest.mark.fresh_sdk
|
||||
|
||||
ENDPOINT = "https://models.example/v1"
|
||||
|
||||
|
||||
def builtin_base() -> str:
|
||||
identity = "\0".join(("local_onnx", "local", "bge-small-zh-v1.5", "512"))
|
||||
return "memhub_" + hashlib.sha256(identity.encode()).hexdigest()[:16]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def backend(monkeypatch) -> FakeReembedBackend:
|
||||
fake = FakeReembedBackend()
|
||||
monkeypatch.setattr(
|
||||
"one_memory.embedding_switch.default_reembed_backend", lambda _data_dir: fake
|
||||
)
|
||||
return fake
|
||||
|
||||
|
||||
def application(tmp_path):
|
||||
return create_app(
|
||||
Settings(
|
||||
data_dir=tmp_path / "data",
|
||||
source_home=tmp_path,
|
||||
api_token=SecretStr("switch-test"),
|
||||
imports_enabled=False,
|
||||
automatic_memory_processing=False,
|
||||
live_sync_enabled=False,
|
||||
),
|
||||
secret_store=InMemorySecretStore(),
|
||||
)
|
||||
|
||||
|
||||
def client_for(app):
|
||||
return AsyncClient(
|
||||
transport=ASGITransport(app=app),
|
||||
base_url="http://test",
|
||||
headers={"Authorization": "Bearer switch-test"},
|
||||
)
|
||||
|
||||
|
||||
async def recovered(client) -> dict:
|
||||
deadline = time.monotonic() + 10
|
||||
while time.monotonic() < deadline:
|
||||
body = (await client.get("/api/v1/health")).json()
|
||||
if body["recovery"] in {"done", "failed"}:
|
||||
return body
|
||||
await asyncio.sleep(0.01)
|
||||
raise AssertionError("recovery did not finish")
|
||||
|
||||
|
||||
async def until(client, predicate, *, seconds: float = 10.0) -> dict:
|
||||
deadline = time.monotonic() + seconds
|
||||
while time.monotonic() < deadline:
|
||||
body = (await client.get("/api/v1/embedding-switches/current")).json()
|
||||
if predicate(body):
|
||||
return body
|
||||
await asyncio.sleep(0.01)
|
||||
raise AssertionError(f"switch never reached the expected state: {body}")
|
||||
|
||||
|
||||
def remote_profile(app, *, model="bge-m3", dims=1024, name="custom", endpoint=ENDPOINT):
|
||||
store, secrets = app.state.model_profile_store, app.state.secret_store
|
||||
created = store.create(
|
||||
ModelProfileInput(
|
||||
name=name,
|
||||
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
|
||||
base_url=endpoint,
|
||||
model="chat",
|
||||
embedding_provider_kind=ProviderKind.OPENAI_COMPATIBLE,
|
||||
embedding_base_url=endpoint,
|
||||
embedding_model=model,
|
||||
embedding_dimensions=dims,
|
||||
api_key=SecretStr("sk-test"),
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
store.record_capabilities(
|
||||
created.profile_id,
|
||||
expected_revision=created.revision,
|
||||
capabilities=ModelCapabilities(
|
||||
endpoint_reachable=True,
|
||||
model_available=True,
|
||||
structured_output=True,
|
||||
embedding_available=True,
|
||||
),
|
||||
secrets=secrets,
|
||||
)
|
||||
return created
|
||||
|
||||
|
||||
def test_activating_another_embedding_answers_with_the_plan(tmp_path, backend):
|
||||
backend.store.put(builtin_base(), 512, {"a": ("one", "local"), "b": ("two", "bob")})
|
||||
|
||||
async def scenario():
|
||||
app = application(tmp_path)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
await recovered(client)
|
||||
profile = remote_profile(app)
|
||||
|
||||
response = await client.post(f"/api/v1/model-profiles/{profile.profile_id}/activate")
|
||||
|
||||
assert response.status_code == 409
|
||||
body = response.json()
|
||||
assert body["detail"] == "embedding_switch_required"
|
||||
plan = body["plan"]
|
||||
assert plan["needed"] is True and plan["total_vectors"] == 2 and plan["users"] == 2
|
||||
assert plan["source"]["model"] == "bge-small-zh-v1.5"
|
||||
assert (plan["target"]["model"], plan["target"]["dims"]) == ("bge-m3", 1024)
|
||||
assert app.state.model_profile_store.active_identity() is None
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_an_empty_home_activates_another_embedding_directly(tmp_path, backend):
|
||||
async def scenario():
|
||||
app = application(tmp_path)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
await recovered(client)
|
||||
builtin = app.state.embedding_spaces.require_live()
|
||||
profile = remote_profile(app)
|
||||
|
||||
response = await client.post(f"/api/v1/model-profiles/{profile.profile_id}/activate")
|
||||
|
||||
assert response.status_code == 200, response.text
|
||||
assert response.json()["is_active"] is True
|
||||
live = app.state.embedding_spaces.require_live()
|
||||
assert (live.model, live.dims) == ("bge-m3", 1024)
|
||||
assert app.state.embedding_spaces.get(builtin.space_id).state == "dropped"
|
||||
# The same embedding again: a plain activation, no plan.
|
||||
again = remote_profile(app, name="same model, other chat model")
|
||||
assert (
|
||||
await client.post(f"/api/v1/model-profiles/{again.profile_id}/activate")
|
||||
).status_code == 200
|
||||
assert app.state.embedding_spaces.require_live().space_id == live.space_id
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_a_switch_over_the_api_pauses_writes_and_keeps_intake(tmp_path, backend):
|
||||
backend.store.put(
|
||||
builtin_base(),
|
||||
512,
|
||||
{f"m{i}": (f"memory {i}", "local") for i in range(6)},
|
||||
model="bge-small-zh-v1.5",
|
||||
)
|
||||
|
||||
async def scenario():
|
||||
app = application(tmp_path)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
await recovered(client)
|
||||
old = app.state.embedding_spaces.require_live()
|
||||
profile = remote_profile(app)
|
||||
planned = await client.post(
|
||||
"/api/v1/embedding-switches/plan", json={"profile_id": profile.profile_id}
|
||||
)
|
||||
assert planned.status_code == 200
|
||||
plan = planned.json()
|
||||
stale = await client.post(
|
||||
"/api/v1/embedding-switches",
|
||||
json={
|
||||
"profile_id": profile.profile_id,
|
||||
"expected_revision": profile.revision,
|
||||
"plan_token": "stale",
|
||||
},
|
||||
)
|
||||
assert stale.status_code == 409
|
||||
assert stale.json()["detail"] == "embedding_switch_plan_stale"
|
||||
|
||||
backend.pause = asyncio.Event()
|
||||
started = await client.post(
|
||||
"/api/v1/embedding-switches",
|
||||
json={
|
||||
"profile_id": profile.profile_id,
|
||||
"expected_revision": profile.revision,
|
||||
"plan_token": plan["plan_token"],
|
||||
"requested_by": "cli",
|
||||
},
|
||||
)
|
||||
assert started.status_code == 202, started.text
|
||||
job_id = started.json()["job_id"]
|
||||
running = await until(
|
||||
client, lambda body: (body["job"] or {}).get("state") == "reembedding"
|
||||
)
|
||||
assert running["writes_paused"] is True and running["phase"] == "running"
|
||||
|
||||
# Direct writers wait; intake keeps queueing; status shows the switch.
|
||||
edit = await client.put(
|
||||
f"/api/v2/memories/{uuid4()}", json={"content": "changed", "user_id": "local"}
|
||||
)
|
||||
assert edit.status_code == 409
|
||||
assert edit.json()["detail"] == "embedding_switch_in_progress"
|
||||
delete = await client.delete(f"/api/v2/memories/{uuid4()}")
|
||||
assert delete.status_code == 409
|
||||
added = await client.post(
|
||||
"/api/v2/memories",
|
||||
json={"request_id": str(uuid4()), "text": "请记住这件事", "scope": "all-apps"},
|
||||
)
|
||||
assert added.status_code == 201
|
||||
processing = (await client.get("/api/v1/memory-processing")).json()
|
||||
assert processing["embedding_switch"]["job_id"] == job_id
|
||||
assert (await client.get("/api/v1/health")).json()["embedding_switch"]["kind"] == (
|
||||
"switch"
|
||||
)
|
||||
other = remote_profile(app, model="text-embedding-v3", dims=768, name="other")
|
||||
refused = await client.post(f"/api/v1/model-profiles/{other.profile_id}/activate")
|
||||
assert refused.json()["detail"] == "embedding_switch_running"
|
||||
|
||||
backend.pause.set()
|
||||
done = await until(
|
||||
client, lambda body: (body["last_finished"] or {}).get("job_id") == job_id
|
||||
)
|
||||
assert done["last_finished"]["state"] == "done"
|
||||
assert done["writes_paused"] is False
|
||||
assert app.state.model_profile_store.active_identity()[0] == profile.profile_id
|
||||
|
||||
spaces = (await client.get("/api/v1/embedding-spaces")).json()
|
||||
states = {space["space_id"]: space["state"] for space in spaces["spaces"]}
|
||||
assert states[old.space_id] == "retired"
|
||||
assert spaces["live_space_id"] == done["last_finished"]["target"]["space_id"]
|
||||
cleaned = await client.delete(f"/api/v1/embedding-spaces/{old.space_id}")
|
||||
assert cleaned.status_code == 200
|
||||
assert cleaned.json()["space"]["state"] == "dropped"
|
||||
live_cleanup = await client.delete(
|
||||
f"/api/v1/embedding-spaces/{spaces['live_space_id']}"
|
||||
)
|
||||
assert live_cleanup.json()["detail"] == "embedding_space_not_retired"
|
||||
|
||||
# Same model and width behind another endpoint: no re-embed.
|
||||
proxied = remote_profile(app, name="proxy", endpoint="https://proxy.example/v1")
|
||||
moved = await client.post(f"/api/v1/model-profiles/{proxied.profile_id}/activate")
|
||||
assert moved.status_code == 200, moved.text
|
||||
assert app.state.embedding_spaces.require_live().space_id == spaces["live_space_id"]
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_cancel_over_the_api(tmp_path, backend):
|
||||
backend.store.put(builtin_base(), 512, {f"m{i}": (f"m {i}", "local") for i in range(4)})
|
||||
|
||||
async def scenario():
|
||||
app = application(tmp_path)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
await recovered(client)
|
||||
profile = remote_profile(app)
|
||||
plan = (
|
||||
await client.post(
|
||||
"/api/v1/embedding-switches/plan", json={"profile_id": profile.profile_id}
|
||||
)
|
||||
).json()
|
||||
backend.pause = asyncio.Event()
|
||||
job_id = (
|
||||
await client.post(
|
||||
"/api/v1/embedding-switches",
|
||||
json={
|
||||
"profile_id": profile.profile_id,
|
||||
"expected_revision": profile.revision,
|
||||
"plan_token": plan["plan_token"],
|
||||
},
|
||||
)
|
||||
).json()["job_id"]
|
||||
await until(client, lambda body: (body["job"] or {}).get("state") == "reembedding")
|
||||
|
||||
cancelled = await client.post(f"/api/v1/embedding-switches/{job_id}/cancel")
|
||||
assert cancelled.status_code == 200 and cancelled.json()["cancel_requested"]
|
||||
backend.pause.set()
|
||||
done = await until(client, lambda body: body["last_finished"] is not None)
|
||||
|
||||
assert done["last_finished"]["state"] == "rolled_back"
|
||||
assert done["last_finished"]["error_code"] == "embedding_switch_cancelled"
|
||||
missing = await client.post(f"/api/v1/embedding-switches/{uuid4()}/cancel")
|
||||
assert missing.status_code == 404
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_startup_merges_a_stray_space_before_writes_resume(tmp_path, backend):
|
||||
backend.store.put(builtin_base(), 512, {"live-1": ("live", "local")})
|
||||
stray = "memhub_" + "a" * 16
|
||||
backend.store.put(stray, 1024, {"stray-1": ("left by an older build", "hermes-1")})
|
||||
|
||||
async def scenario():
|
||||
app = application(tmp_path)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
# Core answers at once; the merge runs behind the loading screen.
|
||||
assert (await client.get("/api/v1/health")).status_code == 200
|
||||
await recovered(client)
|
||||
done = await until(client, lambda body: body["last_finished"] is not None)
|
||||
|
||||
assert done["last_finished"]["kind"] == "merge"
|
||||
assert done["last_finished"]["state"] == "done"
|
||||
assert backend.store.ids(builtin_base(), 512) == {"live-1", "stray-1"}
|
||||
spaces = (await client.get("/api/v1/embedding-spaces")).json()
|
||||
stray_row = next(row for row in spaces["spaces"] if row["collection_base"] == stray)
|
||||
assert stray_row["state"] == "retired" and stray_row["keep_until"]
|
||||
assert app.state.embedding_switch.writes_open()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
class _EmptySearchClient:
|
||||
def search(self, *_args, **_kwargs):
|
||||
return {"request_id": "r", "memories": {"schema": [], "proactive": [], "normal": []}}
|
||||
|
||||
def get(self, *_args, **_kwargs):
|
||||
return None
|
||||
|
||||
|
||||
def test_a_dead_remote_embedding_answers_embedding_unavailable(tmp_path, backend, monkeypatch):
|
||||
status = {"code": 503}
|
||||
|
||||
def transport():
|
||||
return httpx.MockTransport(
|
||||
lambda request: httpx.Response(
|
||||
status["code"], json={"data": [{"embedding": [0.1] * 4}]}
|
||||
)
|
||||
)
|
||||
|
||||
monkeypatch.setattr(embedding_health, "transport_factory", transport)
|
||||
monkeypatch.setattr(
|
||||
"one_memory.hy_sdk_runtime.acquire_client",
|
||||
lambda *args, **kwargs: SimpleNamespace(client=_EmptySearchClient(), close=lambda: None),
|
||||
)
|
||||
|
||||
async def ready(_app):
|
||||
return True
|
||||
|
||||
monkeypatch.setattr("one_memory.api._recall_model_ready", ready)
|
||||
|
||||
async def scenario():
|
||||
app = application(tmp_path)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
await recovered(client)
|
||||
profile = remote_profile(app)
|
||||
assert (
|
||||
await client.post(f"/api/v1/model-profiles/{profile.profile_id}/activate")
|
||||
).status_code == 200
|
||||
|
||||
recall = await client.post("/api/v1/recall", json={"query": "数据库"})
|
||||
assert recall.status_code == 503
|
||||
assert recall.json()["detail"] == "embedding_unavailable"
|
||||
context = await client.post(
|
||||
"/api/v2/context/search",
|
||||
json={"query": "数据库", "user_id": "local", "kinds": ["memory"]},
|
||||
)
|
||||
assert context.status_code == 503
|
||||
assert context.json()["detail"] == "embedding_unavailable"
|
||||
|
||||
# The service answers again: an empty result is just empty.
|
||||
status["code"] = 200
|
||||
app.state.sdk_client_handle.health.forget()
|
||||
recall = await client.post("/api/v1/recall", json={"query": "数据库"})
|
||||
assert recall.status_code == 200
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -0,0 +1,604 @@
|
||||
"""The re-embed adapter against the SDK's real re-embed module and Chroma.
|
||||
|
||||
Skipped on an SDK without `hy_memory.reembed` (before 2.0.10): there the
|
||||
switch is unavailable and the plan says `sdk_reembed_unavailable`, which
|
||||
`test_embedding_switch.py` covers.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
|
||||
import hy_memory
|
||||
import pytest
|
||||
|
||||
from one_memory.embedding_backend import EmbeddingTarget, PhysicalSpace
|
||||
from one_memory.embedding_spaces import EmbeddingSpace, space_collection_names
|
||||
from one_memory.hy_sdk_migration import SdkReembedBackend
|
||||
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot
|
||||
|
||||
pytestmark = pytest.mark.skipif(
|
||||
SdkReembedBackend(Path("/nonexistent")).unavailable_reason() is not None,
|
||||
reason="the installed SDK has no re-embed module (hy-memory < 2.0.10)",
|
||||
)
|
||||
|
||||
SOURCE = "memhub_" + "1" * 16
|
||||
TARGET = "memhub_e_" + "2" * 16
|
||||
|
||||
|
||||
class _Embedder:
|
||||
"""A deterministic 8-wide stand-in for the target model."""
|
||||
|
||||
def __init__(self, width: int = 8) -> None:
|
||||
self.width = width
|
||||
self.texts: list[str] = []
|
||||
|
||||
async def embed_batch(self, texts: list[str], use_cache: bool = True) -> list[list[float]]:
|
||||
self.texts.extend(texts)
|
||||
return [[float(len(text) % 7 + 1)] + [0.5] * (self.width - 1) for text in texts]
|
||||
|
||||
async def embed(self, text: str, use_cache: bool = True) -> list[float]:
|
||||
return (await self.embed_batch([text]))[0]
|
||||
|
||||
def get_embedding_dims(self) -> int:
|
||||
return self.width
|
||||
|
||||
|
||||
def _seed(data_dir: Path) -> None:
|
||||
import chromadb
|
||||
|
||||
vector = data_dir / "hy-memory" / "governed-v2" / "vector"
|
||||
vector.mkdir(parents=True)
|
||||
client = chromadb.PersistentClient(path=str(vector))
|
||||
names = space_collection_names(SOURCE, 4)
|
||||
memory = client.get_or_create_collection(names["memory"], metadata={"hnsw:space": "cosine"})
|
||||
memory.add(
|
||||
ids=["m1", "m2", "m3"],
|
||||
embeddings=[[0.1, 0.2, 0.3, 0.4]] * 3,
|
||||
documents=["one", "two", "three"],
|
||||
metadatas=[{"user_id": "local"}, {"user_id": "hermes-a"}, {"user_id": "hermes-b"}],
|
||||
)
|
||||
keys = client.get_or_create_collection(names["coding_keys"], metadata={"hnsw:space": "cosine"})
|
||||
keys.add(
|
||||
ids=["e1:0"],
|
||||
embeddings=[[0.4, 0.3, 0.2, 0.1]],
|
||||
documents=["task"],
|
||||
metadatas=[{"user_id": "local"}],
|
||||
)
|
||||
|
||||
|
||||
def _target() -> EmbeddingTarget:
|
||||
space = EmbeddingSpace(
|
||||
space_id="target",
|
||||
state="building",
|
||||
model="stand-in",
|
||||
dims=8,
|
||||
name_dims=8,
|
||||
collection_base=TARGET,
|
||||
naming="v2",
|
||||
created_at="2026-11-01T00:00:00+00:00",
|
||||
)
|
||||
return EmbeddingTarget(profile=builtin_retrieval_snapshot(), space=space, api_key=None)
|
||||
|
||||
|
||||
def test_core_names_a_space_exactly_as_the_sdk_does() -> None:
|
||||
import hy_memory
|
||||
|
||||
for dims in (None, 512, 1024):
|
||||
sdk = hy_memory.vector_space_names(hy_memory.VectorSpace(base_name=SOURCE, dims=dims))
|
||||
assert sdk == space_collection_names(SOURCE, dims)
|
||||
|
||||
|
||||
def test_scan_reembed_verify_and_drop_through_the_sdk(tmp_path: Path, monkeypatch) -> None:
|
||||
_seed(tmp_path)
|
||||
backend = SdkReembedBackend(tmp_path)
|
||||
embedder = _Embedder()
|
||||
monkeypatch.setattr(backend, "_embedding_service", lambda _target: embedder)
|
||||
source = PhysicalSpace(collection_base=SOURCE, name_dims=4, dims=4)
|
||||
target = PhysicalSpace(collection_base=TARGET, name_dims=8, dims=8, model="stand-in")
|
||||
steps = []
|
||||
|
||||
async def scenario() -> None:
|
||||
scanned = await backend.scan(per_user=True)
|
||||
found = {item.collection_base: item for item in scanned}
|
||||
assert found[SOURCE].vectors == 4
|
||||
assert found[SOURCE].per_user == {"local": 2, "hermes-a": 1, "hermes-b": 1}
|
||||
|
||||
await backend.reembed(
|
||||
source, target, embedding=_target(), cancel=lambda: False, on_step=steps.append
|
||||
)
|
||||
assert not any((await backend.diff(source, target)).values())
|
||||
check = await backend.verify(source, target, embedding=_target())
|
||||
assert check.problems == ()
|
||||
assert check.per_user == {"local": 2, "hermes-a": 1, "hermes-b": 1}
|
||||
|
||||
# Resumable: a second run embeds nothing more.
|
||||
before = len(embedder.texts)
|
||||
await backend.reembed(
|
||||
source, target, embedding=_target(), cancel=lambda: False, on_step=steps.append
|
||||
)
|
||||
assert len(embedder.texts) == before
|
||||
|
||||
await backend.drop(target)
|
||||
remaining = {item.collection_base for item in await backend.scan()}
|
||||
assert TARGET not in remaining and SOURCE in remaining
|
||||
|
||||
asyncio.run(scenario())
|
||||
assert {step.role for step in steps} >= {"memory", "coding_keys"}
|
||||
|
||||
|
||||
def test_a_wrong_width_fails_before_anything_is_written(tmp_path: Path, monkeypatch) -> None:
|
||||
from one_memory.embedding_backend import EmbeddingWidthError
|
||||
|
||||
_seed(tmp_path)
|
||||
backend = SdkReembedBackend(tmp_path)
|
||||
monkeypatch.setattr(backend, "_embedding_service", lambda _target: _Embedder(width=6))
|
||||
source = PhysicalSpace(collection_base=SOURCE, name_dims=4, dims=4)
|
||||
target = PhysicalSpace(collection_base=TARGET, name_dims=8, dims=8)
|
||||
|
||||
async def scenario() -> None:
|
||||
with pytest.raises(EmbeddingWidthError):
|
||||
await backend.reembed(
|
||||
source, target, embedding=_target(), cancel=lambda: False, on_step=lambda _: None
|
||||
)
|
||||
found = {item.collection_base: item for item in await backend.scan()}
|
||||
assert TARGET not in found or found[TARGET].vectors == 0
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_cancel_stops_at_a_batch_boundary(tmp_path: Path, monkeypatch) -> None:
|
||||
from one_memory.embedding_backend import ReembedCancelledError
|
||||
|
||||
_seed(tmp_path)
|
||||
backend = SdkReembedBackend(tmp_path)
|
||||
monkeypatch.setattr(backend, "_embedding_service", lambda _target: _Embedder())
|
||||
source = PhysicalSpace(collection_base=SOURCE, name_dims=4, dims=4)
|
||||
target = PhysicalSpace(collection_base=TARGET, name_dims=8, dims=8)
|
||||
|
||||
async def scenario() -> None:
|
||||
with pytest.raises(ReembedCancelledError):
|
||||
await backend.reembed(
|
||||
source, target, embedding=_target(), cancel=lambda: True, on_step=lambda _: None
|
||||
)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_a_whole_switch_through_the_real_sdk(tmp_path: Path, monkeypatch) -> None:
|
||||
"""Registry, worker and commit over real Chroma collections and the SDK."""
|
||||
import chromadb
|
||||
from pydantic import SecretStr
|
||||
|
||||
from one_memory.embedding_spaces import EmbeddingSpaceStore
|
||||
from one_memory.embedding_switch import EmbeddingSwitchService
|
||||
from one_memory.model_profiles import (
|
||||
ModelCapabilities,
|
||||
ModelProfileInput,
|
||||
ModelProfileStore,
|
||||
ProviderKind,
|
||||
)
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
database = tmp_path / "one-memory.sqlite3"
|
||||
secrets = InMemorySecretStore()
|
||||
profiles = ModelProfileStore(database)
|
||||
spaces = EmbeddingSpaceStore(database)
|
||||
live = spaces.bootstrap(None)
|
||||
vector = tmp_path / "hy-memory" / "governed-v2" / "vector"
|
||||
vector.mkdir(parents=True)
|
||||
client = chromadb.PersistentClient(path=str(vector))
|
||||
memory = client.get_or_create_collection(
|
||||
live.collection_names()["memory"], metadata={"hnsw:space": "cosine"}
|
||||
)
|
||||
memory.add(
|
||||
ids=["a", "b", "c"],
|
||||
embeddings=[[0.1] * 512, [0.2] * 512, [0.3] * 512],
|
||||
documents=["one", "two", "three"],
|
||||
metadatas=[{"user_id": "local"}, {"user_id": "hermes-a"}, {"user_id": "local"}],
|
||||
)
|
||||
stray_base = "memhub_" + "9" * 16
|
||||
stray = client.get_or_create_collection(
|
||||
space_collection_names(stray_base, 768)["memory"], metadata={"hnsw:space": "cosine"}
|
||||
)
|
||||
stray.add(
|
||||
ids=["a", "z"],
|
||||
embeddings=[[0.5] * 768, [0.6] * 768],
|
||||
documents=["stale", "only in the stray"],
|
||||
metadatas=[{"user_id": "local"}, {"user_id": "hermes-b"}],
|
||||
)
|
||||
created = profiles.create(
|
||||
ModelProfileInput(
|
||||
name="remote",
|
||||
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
|
||||
base_url="https://models.example/v1",
|
||||
model="chat",
|
||||
embedding_provider_kind=ProviderKind.OPENAI_COMPATIBLE,
|
||||
embedding_base_url="https://models.example/v1",
|
||||
embedding_model="stand-in",
|
||||
embedding_dimensions=8,
|
||||
api_key=SecretStr("sk-test"),
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
profiles.record_capabilities(
|
||||
created.profile_id,
|
||||
expected_revision=created.revision,
|
||||
capabilities=ModelCapabilities(
|
||||
endpoint_reachable=True,
|
||||
model_available=True,
|
||||
structured_output=True,
|
||||
embedding_available=True,
|
||||
),
|
||||
secrets=secrets,
|
||||
)
|
||||
backend = SdkReembedBackend(tmp_path)
|
||||
monkeypatch.setattr(backend, "_embedding_service", lambda _target: _Embedder())
|
||||
service = EmbeddingSwitchService(
|
||||
database_path=database,
|
||||
data_dir=tmp_path,
|
||||
profiles=profiles,
|
||||
spaces=spaces,
|
||||
secrets=secrets,
|
||||
backend=backend,
|
||||
worker_enabled=False,
|
||||
)
|
||||
service.release_scan_hold()
|
||||
|
||||
async def scenario() -> None:
|
||||
plan = await service.plan(created.profile_id)
|
||||
assert plan.needed and plan.total_vectors == 5 and plan.users == 3
|
||||
job = await service.start_switch(
|
||||
created.profile_id,
|
||||
expected_revision=created.revision,
|
||||
plan_token=plan.plan_token,
|
||||
requested_by="cli",
|
||||
)
|
||||
await service.run(job.job_id)
|
||||
finished = service.get_job(job.job_id)
|
||||
assert finished.state == "done", service.jobs.get(job.job_id)["verification_json"]
|
||||
|
||||
asyncio.run(scenario())
|
||||
new = spaces.require_live()
|
||||
target = client.get_collection(new.collection_names()["memory"])
|
||||
got = target.get(include=["documents", "embeddings"])
|
||||
documents = dict(zip(got["ids"], got["documents"], strict=True))
|
||||
assert documents == {"a": "one", "b": "two", "c": "three", "z": "only in the stray"}
|
||||
assert {len(vector) for vector in got["embeddings"]} == {8}
|
||||
assert profiles.active_identity() == (created.profile_id, created.revision)
|
||||
states = {space.collection_base: space.state for space in spaces.list_spaces()}
|
||||
assert states[live.collection_base] == "retired"
|
||||
assert states[stray_base] == "retired"
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# Nothing stood in: the SDK's EmbedService over HTTP, real Chroma, real Kuzu
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
|
||||
class _EmbeddingServer:
|
||||
"""A local OpenAI-compatible `/embeddings` endpoint; counts what it was sent."""
|
||||
|
||||
def __init__(self, width: int) -> None:
|
||||
import json
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
|
||||
self.width = width
|
||||
self.inputs: list[str] = []
|
||||
self.dimensions: list[object] = []
|
||||
server = self
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def do_POST(self) -> None: # noqa: N802 - http.server's name
|
||||
body = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
|
||||
texts = body["input"] if isinstance(body["input"], list) else [body["input"]]
|
||||
server.inputs.extend(texts)
|
||||
server.dimensions.append(body.get("dimensions"))
|
||||
payload = json.dumps(
|
||||
{
|
||||
"object": "list",
|
||||
"model": body["model"],
|
||||
"data": [
|
||||
{
|
||||
"object": "embedding",
|
||||
"index": index,
|
||||
"embedding": [(len(text) % 7 + 1) / 8.0] + [0.5] * (width - 1),
|
||||
}
|
||||
for index, text in enumerate(texts)
|
||||
],
|
||||
"usage": {"prompt_tokens": 1, "total_tokens": 1},
|
||||
}
|
||||
).encode()
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(payload)))
|
||||
self.end_headers()
|
||||
self.wfile.write(payload)
|
||||
|
||||
def log_message(self, *_args: object) -> None:
|
||||
return
|
||||
|
||||
self._server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||
self.url = f"http://127.0.0.1:{self._server.server_port}/v1"
|
||||
threading.Thread(target=self._server.serve_forever, daemon=True).start()
|
||||
|
||||
def close(self) -> None:
|
||||
self._server.shutdown()
|
||||
self._server.server_close()
|
||||
|
||||
|
||||
def _home(tmp_path: Path, *, embedding_provider, embedding_url: str, api_key: str | None):
|
||||
from pydantic import SecretStr
|
||||
|
||||
from one_memory.embedding_spaces import EmbeddingSpaceStore
|
||||
from one_memory.model_profiles import (
|
||||
ModelCapabilities,
|
||||
ModelProfileInput,
|
||||
ModelProfileStore,
|
||||
ProviderKind,
|
||||
)
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
database = tmp_path / "one-memory.sqlite3"
|
||||
secrets = InMemorySecretStore()
|
||||
profiles = ModelProfileStore(database)
|
||||
spaces = EmbeddingSpaceStore(database)
|
||||
created = profiles.create(
|
||||
ModelProfileInput(
|
||||
name="target",
|
||||
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
|
||||
base_url=embedding_url,
|
||||
model="chat",
|
||||
embedding_provider_kind=embedding_provider,
|
||||
embedding_base_url=embedding_url,
|
||||
embedding_model="stand-in",
|
||||
embedding_dimensions=8,
|
||||
api_key=SecretStr(api_key) if api_key else None,
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
profiles.record_capabilities(
|
||||
created.profile_id,
|
||||
expected_revision=created.revision,
|
||||
capabilities=ModelCapabilities(
|
||||
endpoint_reachable=True,
|
||||
model_available=True,
|
||||
structured_output=True,
|
||||
embedding_available=True,
|
||||
),
|
||||
secrets=secrets,
|
||||
)
|
||||
return database, secrets, profiles, spaces, created
|
||||
|
||||
|
||||
def _service(database, data_dir, profiles, spaces, secrets, **options):
|
||||
from one_memory.embedding_switch import EmbeddingSwitchService
|
||||
|
||||
service = EmbeddingSwitchService(
|
||||
database_path=database,
|
||||
data_dir=data_dir,
|
||||
profiles=profiles,
|
||||
spaces=spaces,
|
||||
secrets=secrets,
|
||||
backend=SdkReembedBackend(data_dir),
|
||||
worker_enabled=False,
|
||||
**options,
|
||||
)
|
||||
service.release_scan_hold()
|
||||
return service
|
||||
|
||||
|
||||
async def _switch(service, created) -> object:
|
||||
plan = await service.plan(created.profile_id)
|
||||
assert plan.needed and not plan.blockers, plan
|
||||
job = await service.start_switch(
|
||||
created.profile_id,
|
||||
expected_revision=created.revision,
|
||||
plan_token=plan.plan_token,
|
||||
requested_by="cli",
|
||||
)
|
||||
await service.run(job.job_id)
|
||||
finished = service.get_job(job.job_id)
|
||||
assert finished.state == "done", (
|
||||
finished.error_code,
|
||||
service.jobs.get(job.job_id)["verification_json"],
|
||||
service.jobs.get(job.job_id)["error_detail"],
|
||||
)
|
||||
return finished
|
||||
|
||||
|
||||
def _seed_vectors(data_dir: Path, space) -> None:
|
||||
import chromadb
|
||||
|
||||
vector = data_dir / "hy-memory" / "governed-v2" / "vector"
|
||||
vector.mkdir(parents=True, exist_ok=True)
|
||||
client = chromadb.PersistentClient(path=str(vector))
|
||||
memory = client.get_or_create_collection(
|
||||
space.collection_names()["memory"], metadata={"hnsw:space": "cosine"}
|
||||
)
|
||||
memory.add(
|
||||
ids=["a", "b"],
|
||||
embeddings=[[0.1] * 512, [0.2] * 512],
|
||||
documents=["one", "two"],
|
||||
metadatas=[{"user_id": "local"}, {"user_id": "hermes-a"}],
|
||||
)
|
||||
|
||||
|
||||
def test_a_switch_embeds_through_the_sdk_service_over_http(tmp_path: Path) -> None:
|
||||
from one_memory.model_profiles import ProviderKind
|
||||
|
||||
server = _EmbeddingServer(8)
|
||||
try:
|
||||
database, secrets, profiles, spaces, created = _home(
|
||||
tmp_path,
|
||||
embedding_provider=ProviderKind.OPENAI_COMPATIBLE,
|
||||
embedding_url=server.url,
|
||||
api_key="sk-test",
|
||||
)
|
||||
_seed_vectors(tmp_path, spaces.bootstrap(None))
|
||||
service = _service(database, tmp_path, profiles, spaces, secrets)
|
||||
|
||||
asyncio.run(_switch(service, created))
|
||||
|
||||
assert {"one", "two"} <= set(server.inputs)
|
||||
assert set(server.dimensions) == {8}
|
||||
finally:
|
||||
server.close()
|
||||
|
||||
|
||||
def test_an_ollama_embedding_needs_no_generation_key_to_re_embed(tmp_path: Path) -> None:
|
||||
"""A switch only embeds: the generation model's key is not its business."""
|
||||
from one_memory.embedding_backend import EmbeddingTarget
|
||||
from one_memory.embedding_spaces import embedding_identity
|
||||
from one_memory.model_profiles import ProviderKind
|
||||
|
||||
server = _EmbeddingServer(8)
|
||||
try:
|
||||
_database, secrets, profiles, spaces, created = _home(
|
||||
tmp_path,
|
||||
embedding_provider=ProviderKind.OLLAMA,
|
||||
embedding_url=server.url,
|
||||
api_key="sk-test",
|
||||
)
|
||||
profile = profiles.snapshot(created.profile_id, created.revision)
|
||||
identity = embedding_identity(profile)
|
||||
target = EmbeddingSpace(
|
||||
space_id="t",
|
||||
state="building",
|
||||
model=identity.model,
|
||||
dims=8,
|
||||
name_dims=8,
|
||||
collection_base=TARGET,
|
||||
naming="v2",
|
||||
created_at="2026-11-01T00:00:00+00:00",
|
||||
)
|
||||
backend = SdkReembedBackend(tmp_path)
|
||||
|
||||
# `api_key=None` is what `_embedding_target` resolves for Ollama.
|
||||
service = backend._embedding_service(
|
||||
EmbeddingTarget(profile=profile, space=target, api_key=None)
|
||||
)
|
||||
|
||||
assert asyncio.run(service.embed_batch(["hello"], use_cache=False))[0][1] == 0.5
|
||||
finally:
|
||||
server.close()
|
||||
|
||||
|
||||
async def _seed_graph(path: Path, dims: int) -> None:
|
||||
from hy_memory.data.graph_store_kuzu import KuzuGraphStore
|
||||
from hy_memory.models.memory import MemoryLayer, MemoryNode
|
||||
|
||||
config = hy_memory.MemoryConfig()
|
||||
config.graph_store.provider = "kuzu"
|
||||
config.graph_store.db_path = str(path)
|
||||
config.vector_store.embedding_dims = dims
|
||||
store = KuzuGraphStore(config)
|
||||
await store.initialize()
|
||||
assert store._available
|
||||
node = MemoryNode(
|
||||
user_id="local",
|
||||
agent_id="a",
|
||||
session_id="s",
|
||||
layer=MemoryLayer.L6_SCHEMA,
|
||||
content="plans ahead",
|
||||
custom={"retrieval_text": "plans trips weeks ahead"},
|
||||
)
|
||||
node._graph_embedding = [0.1] * dims
|
||||
await store.upsert_memory_node(node)
|
||||
await store.close()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("mode", ["ultra", "pro"])
|
||||
def test_a_real_graph_moves_to_the_target_width(tmp_path: Path, mode: str) -> None:
|
||||
"""In any mode: a pro home's old graph must not be lost with the old space."""
|
||||
from one_memory.hy_graph_admin import forget_graph_widths
|
||||
from one_memory.model_profiles import ProviderKind
|
||||
|
||||
server = _EmbeddingServer(8)
|
||||
try:
|
||||
database, secrets, profiles, spaces, created = _home(
|
||||
tmp_path,
|
||||
embedding_provider=ProviderKind.OPENAI_COMPATIBLE,
|
||||
embedding_url=server.url,
|
||||
api_key="sk-test",
|
||||
)
|
||||
live = spaces.bootstrap(None)
|
||||
_seed_vectors(tmp_path, live)
|
||||
root = tmp_path / "hy-memory" / "governed-v2"
|
||||
asyncio.run(_seed_graph(root / "graph", 512))
|
||||
service = _service(
|
||||
database, tmp_path, profiles, spaces, secrets, graph_enabled=lambda: mode == "ultra"
|
||||
)
|
||||
|
||||
asyncio.run(_switch(service, created))
|
||||
|
||||
new = spaces.require_live()
|
||||
assert new.graph_dir != "graph" and (root / new.graph_dir).exists()
|
||||
assert (root / "graph").exists()
|
||||
assert "plans trips weeks ahead" in server.inputs
|
||||
forget_graph_widths()
|
||||
finally:
|
||||
server.close()
|
||||
|
||||
|
||||
def test_verifying_the_graph_closes_what_reads_reopened(tmp_path: Path) -> None:
|
||||
"""Reads are served during a switch, so a client may hold the graph again."""
|
||||
from one_memory.model_profiles import ProviderKind
|
||||
|
||||
server = _EmbeddingServer(8)
|
||||
quiesced: list[str] = []
|
||||
|
||||
async def quiesce():
|
||||
quiesced.append("closed")
|
||||
return lambda: quiesced.append("released")
|
||||
|
||||
try:
|
||||
database, secrets, profiles, spaces, created = _home(
|
||||
tmp_path,
|
||||
embedding_provider=ProviderKind.OPENAI_COMPATIBLE,
|
||||
embedding_url=server.url,
|
||||
api_key="sk-test",
|
||||
)
|
||||
live = spaces.bootstrap(None)
|
||||
_seed_vectors(tmp_path, live)
|
||||
asyncio.run(_seed_graph(tmp_path / "hy-memory" / "governed-v2" / "graph", 512))
|
||||
service = _service(
|
||||
database,
|
||||
tmp_path,
|
||||
profiles,
|
||||
spaces,
|
||||
secrets,
|
||||
graph_enabled=lambda: True,
|
||||
quiesce_clients=quiesce,
|
||||
)
|
||||
|
||||
asyncio.run(_switch(service, created))
|
||||
|
||||
# Once to copy the graph, once to verify it.
|
||||
assert quiesced == ["closed", "released", "closed", "released"]
|
||||
finally:
|
||||
server.close()
|
||||
|
||||
|
||||
def test_the_first_chroma_open_waits_for_a_client_under_construction(tmp_path: Path) -> None:
|
||||
"""Two first opens on one directory race over Chroma's unlocked System cache."""
|
||||
import threading
|
||||
|
||||
from one_memory.hy_sdk_runtime import first_store_open
|
||||
|
||||
_seed(tmp_path)
|
||||
backend = SdkReembedBackend(tmp_path)
|
||||
finished = threading.Event()
|
||||
|
||||
def scan() -> None:
|
||||
asyncio.run(backend.scan())
|
||||
finished.set()
|
||||
|
||||
with first_store_open(tmp_path):
|
||||
thread = threading.Thread(target=scan)
|
||||
thread.start()
|
||||
assert not finished.wait(0.3)
|
||||
thread.join(10)
|
||||
assert finished.is_set()
|
||||
@@ -48,6 +48,7 @@ def test_health_reports_the_single_core() -> None:
|
||||
# No lifespan ran, so recovery has not begun.
|
||||
"recovery": "pending",
|
||||
"recovery_failed_steps": [],
|
||||
"embedding_switch": None,
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -111,6 +111,11 @@ ALLOWED = {
|
||||
"the offline migration opens Chroma",
|
||||
opened_with="?mode=ro",
|
||||
),
|
||||
("sdk_vector_reattach.py", "stored_collection_width"): Allowed(
|
||||
"Chroma's own chroma.sqlite3, read-only, for the width it fixed on a "
|
||||
"collection: a probe on the live space's profile may only record that one",
|
||||
opened_with="?mode=ro",
|
||||
),
|
||||
("imports/sqlite_access.py", "open_sqlite_readonly"): Allowed(
|
||||
"a host app's own SQLite (Cursor state.vscdb / store.db): one direct "
|
||||
"read-only open when there is no WAL, and one read-only open of the "
|
||||
|
||||
@@ -318,6 +318,12 @@ def run_with_sdk(tmp_path, monkeypatch, scenario):
|
||||
async with app.router.lifespan_context(app):
|
||||
app.state.sdk_client_handle = Handle()
|
||||
monkeypatch.setattr(app.state.sdk_write_coordinator, "use_client", write_client)
|
||||
# Edits and deletes wait for the startup scan of embedding spaces,
|
||||
# as they would behind the App's loading screen.
|
||||
for _ in range(500):
|
||||
if app.state.embedding_switch.writes_open():
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
async with client_for(app) as client:
|
||||
await scenario(app, sdk, client)
|
||||
|
||||
|
||||
@@ -684,6 +684,12 @@ def run_app(tmp_path, monkeypatch, scenario) -> None:
|
||||
|
||||
async with app.router.lifespan_context(app):
|
||||
monkeypatch.setattr(app.state.sdk_write_coordinator, "use_client", write_client)
|
||||
# Model writes wait for the startup scan of embedding spaces, as
|
||||
# they would behind the App's loading screen.
|
||||
for _ in range(500):
|
||||
if app.state.embedding_switch.writes_open():
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
async with client_for(app) as client:
|
||||
await scenario(app, sdk, client)
|
||||
|
||||
|
||||
@@ -12,10 +12,12 @@ import time
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from hashlib import sha256
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.embedding_spaces import EmbeddingIdentity
|
||||
from one_memory.memory_server.contracts import BatchIdentity, WriteRequest
|
||||
from one_memory.memory_server.durable_queue import (
|
||||
CANCELLED,
|
||||
@@ -1226,6 +1228,10 @@ def _coordinator(monkeypatch, opened: list[_TrackedHandle]):
|
||||
return value
|
||||
|
||||
monkeypatch.setattr(sdk_write_processor, "SdkClientHandle", handle)
|
||||
# One live embedding space the stand-in profiles all fill.
|
||||
identity = EmbeddingIdentity(model="stand-in")
|
||||
monkeypatch.setattr(sdk_write_processor, "embedding_identity", lambda _profile: identity)
|
||||
space = SimpleNamespace(space_id="space", identity=identity)
|
||||
profiles = _SwitchableProfiles()
|
||||
coordinator = sdk_write_processor.SdkWriteCoordinator(
|
||||
profiles=profiles, # type: ignore[arg-type]
|
||||
@@ -1233,6 +1239,7 @@ def _coordinator(monkeypatch, opened: list[_TrackedHandle]):
|
||||
data_dir=None,
|
||||
transport_factory=lambda _profile: None,
|
||||
sedimentation_enabled=lambda: True,
|
||||
spaces=SimpleNamespace(live=lambda: space), # type: ignore[arg-type]
|
||||
)
|
||||
return coordinator, profiles
|
||||
|
||||
|
||||
@@ -91,7 +91,7 @@ def test_bundle_contains_management_sdk_config_and_verified_manifest(tmp_path):
|
||||
assert manifest["bundle_id"] == "migration-1"
|
||||
assert manifest["sdk_namespace"] == "governed-v2"
|
||||
assert manifest["runtime_build"] == "build-13"
|
||||
assert manifest["snapshot_sdk_version"] == "2.0.9"
|
||||
assert manifest["snapshot_sdk_version"] == "2.0.11"
|
||||
paths = {item["path"]: item for item in manifest["files"]}
|
||||
assert {
|
||||
"management/one-memory.sqlite3",
|
||||
|
||||
@@ -141,11 +141,19 @@ class _Recorder:
|
||||
self.closed: list[str] = []
|
||||
|
||||
def __call__(
|
||||
self, profile, *, api_key, data_dir, http_transport_factory=None, mode="pro", features=()
|
||||
self,
|
||||
profile,
|
||||
space=None,
|
||||
*,
|
||||
api_key,
|
||||
data_dir,
|
||||
http_transport_factory=None,
|
||||
mode=None,
|
||||
features=(),
|
||||
):
|
||||
from one_memory.hy_sdk_runtime import build_sdk_config
|
||||
|
||||
config = build_sdk_config(profile, api_key=api_key, data_dir=data_dir)
|
||||
config = build_sdk_config(profile, space, mode, api_key=api_key, data_dir=data_dir)
|
||||
name = config.vector_store.collection_name
|
||||
self.opened.append((name, config.vector_store.embedding_dims, api_key))
|
||||
return SimpleNamespace(
|
||||
@@ -155,10 +163,11 @@ class _Recorder:
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_reads_open_the_collection_the_active_profile_writes(
|
||||
async def test_reads_open_the_live_space_the_active_profile_writes(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""#242: a 2560-dim profile wrote `memhub_<sha>_2560`, reads opened `_512`."""
|
||||
from one_memory.embedding_spaces import EmbeddingSpaceStore
|
||||
from one_memory.hy_sdk_runtime import build_sdk_config
|
||||
from one_memory.model_profiles import ModelProfileStore
|
||||
from one_memory.sdk_read_handle import ActiveProfileReadHandle
|
||||
@@ -168,13 +177,14 @@ async def test_reads_open_the_collection_the_active_profile_writes(
|
||||
monkeypatch.setattr("one_memory.hy_sdk_runtime.acquire_client", recorder)
|
||||
profiles = ModelProfileStore(tmp_path / "one-memory.sqlite3")
|
||||
secrets = InMemorySecretStore()
|
||||
handle = ActiveProfileReadHandle(profiles=profiles, secrets=secrets, data_dir=tmp_path)
|
||||
|
||||
builtin = build_sdk_config(builtin_retrieval_snapshot(), api_key=None, data_dir=tmp_path)
|
||||
assert (await handle.client()).collection == builtin.vector_store.collection_name
|
||||
|
||||
# A home whose custom profile was active when this build first started:
|
||||
# the registry pins the collection that profile has been writing.
|
||||
written = _activate(profiles, secrets, dimensions=2560)
|
||||
spaces = EmbeddingSpaceStore(profiles.database_path)
|
||||
spaces.bootstrap(written)
|
||||
handle = ActiveProfileReadHandle(profiles=profiles, secrets=secrets, data_dir=tmp_path)
|
||||
writes = build_sdk_config(written, api_key="sk-test", data_dir=tmp_path)
|
||||
builtin = build_sdk_config(builtin_retrieval_snapshot(), api_key=None, data_dir=tmp_path)
|
||||
|
||||
async def read() -> str:
|
||||
return str((await handle.client()).collection)
|
||||
@@ -186,9 +196,11 @@ async def test_reads_open_the_collection_the_active_profile_writes(
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_a_profile_change_rebuilds_reads_without_closing_one_in_flight(
|
||||
async def test_an_activation_that_skips_the_switch_never_moves_reads(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""Only a switch's commit moves the space; reads stay where the data is."""
|
||||
from one_memory.embedding_spaces import EmbeddingSpaceStore
|
||||
from one_memory.model_profiles import ModelProfileStore
|
||||
from one_memory.sdk_read_handle import ActiveProfileReadHandle
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
@@ -197,8 +209,40 @@ async def test_a_profile_change_rebuilds_reads_without_closing_one_in_flight(
|
||||
monkeypatch.setattr("one_memory.hy_sdk_runtime.acquire_client", recorder)
|
||||
profiles = ModelProfileStore(tmp_path / "one-memory.sqlite3")
|
||||
secrets = InMemorySecretStore()
|
||||
first = _activate(profiles, secrets, dimensions=1024)
|
||||
live = EmbeddingSpaceStore(profiles.database_path).bootstrap(first)
|
||||
handle = ActiveProfileReadHandle(profiles=profiles, secrets=secrets, data_dir=tmp_path)
|
||||
_activate(profiles, secrets, dimensions=1024)
|
||||
_activate(profiles, secrets, dimensions=2560, model="chat-2")
|
||||
|
||||
collection = (await asyncio.create_task(handle.client())).collection
|
||||
|
||||
assert collection == live.collection_base
|
||||
await handle.close()
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_a_committed_switch_rebuilds_reads_without_closing_one_in_flight(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
from one_memory.embedding_spaces import (
|
||||
EmbeddingSpaceStore,
|
||||
derived_space,
|
||||
insert_space_in,
|
||||
set_state_in,
|
||||
)
|
||||
from one_memory.model_profiles import ModelProfileStore
|
||||
from one_memory.sdk_read_handle import ActiveProfileReadHandle
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
recorder = _Recorder(tmp_path)
|
||||
monkeypatch.setattr("one_memory.hy_sdk_runtime.acquire_client", recorder)
|
||||
profiles = ModelProfileStore(tmp_path / "one-memory.sqlite3")
|
||||
secrets = InMemorySecretStore()
|
||||
spaces = EmbeddingSpaceStore(profiles.database_path)
|
||||
old_space = spaces.bootstrap(_activate(profiles, secrets, dimensions=1024))
|
||||
handle = ActiveProfileReadHandle(
|
||||
profiles=profiles, secrets=secrets, data_dir=tmp_path, spaces=spaces
|
||||
)
|
||||
|
||||
holding = asyncio.Event()
|
||||
finish = asyncio.Event()
|
||||
@@ -213,7 +257,15 @@ async def test_a_profile_change_rebuilds_reads_without_closing_one_in_flight(
|
||||
await holding.wait()
|
||||
[(old, _, _)] = recorder.opened
|
||||
|
||||
_activate(profiles, secrets, dimensions=2560, model="chat-2")
|
||||
# What a switch's commit does: a new live space, the old one retired, and
|
||||
# the target profile active -- one transaction.
|
||||
target = _activate(profiles, secrets, dimensions=2560, model="chat-2")
|
||||
with spaces.transaction() as connection:
|
||||
set_state_in(connection, old_space.space_id, state="retired", expected=("live",))
|
||||
insert_space_in(
|
||||
connection,
|
||||
derived_space(target).model_copy(update={"space_id": "new-space"}),
|
||||
)
|
||||
|
||||
async def read() -> str:
|
||||
return str((await handle.client()).collection)
|
||||
|
||||
@@ -0,0 +1,331 @@
|
||||
"""The App's display language, and the SDK prompt language it implies.
|
||||
|
||||
The owner's rule (#243): every display language, zh-Hans and en included, is
|
||||
passed as `MemoryConfig.language` (hy-memory 2.0.11); only an unset language
|
||||
passes none, so the SDK keeps detecting zh/en exactly as before. A change rebuilds the
|
||||
handles the way a mode or feature change does, and the language never touches
|
||||
the vector collection identity.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import dataclasses
|
||||
import json
|
||||
from contextlib import asynccontextmanager
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
from pydantic import SecretStr, ValidationError
|
||||
|
||||
from one_memory import hy_sdk_runtime, sdk_read_handle, sdk_write_processor
|
||||
from one_memory.api import create_app
|
||||
from one_memory.config import Settings
|
||||
from one_memory.memory_server.config_state import ConfigDocument, ConfigStore
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
from one_memory.ui_language import UI_LANGUAGES, normalize_ui_language, sdk_language
|
||||
|
||||
TOKEN = "ui-language-test"
|
||||
#: Every code is passed to the SDK, zh-Hans included.
|
||||
PASSED = ("en", "zh-Hans", "zh-Hant", "ja", "es", "de", "pt", "fr", "ko", "ru", "id")
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean_sdk_env(monkeypatch):
|
||||
"""The SDK's own `MEMORY_LANGUAGE` feeds its default; tests start without it."""
|
||||
monkeypatch.delenv("MEMORY_LANGUAGE", raising=False)
|
||||
|
||||
|
||||
def _config(tmp_path: Path, language: str | None):
|
||||
return hy_sdk_runtime.build_sdk_config(
|
||||
hy_sdk_runtime.builtin_retrieval_snapshot(),
|
||||
api_key=None,
|
||||
data_dir=tmp_path,
|
||||
language=language,
|
||||
)
|
||||
|
||||
|
||||
# --- the rule -------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_eleven_codes_and_the_legacy_chinese_code() -> None:
|
||||
assert PASSED == UI_LANGUAGES
|
||||
assert len(UI_LANGUAGES) == 11
|
||||
# Earlier App builds stored zh-CN; it is zh-Hans now.
|
||||
assert normalize_ui_language("zh-CN") == "zh-Hans"
|
||||
assert normalize_ui_language("system") is None
|
||||
assert normalize_ui_language("xx") is None
|
||||
assert normalize_ui_language(None) is None
|
||||
|
||||
|
||||
def test_only_an_unset_language_leaves_the_sdk_to_detect() -> None:
|
||||
assert sdk_language(None) is None
|
||||
assert sdk_language("system") is None
|
||||
assert sdk_language("xx") is None
|
||||
# zh-Hans is passed like every other code; the legacy zh-CN maps to it.
|
||||
assert sdk_language("zh-Hans") == "zh-Hans"
|
||||
assert sdk_language("zh-CN") == "zh-Hans"
|
||||
for code in PASSED:
|
||||
assert sdk_language(code) == code
|
||||
|
||||
|
||||
def test_no_language_builds_the_config_of_before(tmp_path) -> None:
|
||||
assert _config(tmp_path, None).language is None
|
||||
assert _config(tmp_path, sdk_language("zh-Hans")).language == "zh-Hans"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("code", PASSED)
|
||||
def test_a_passed_code_reaches_the_memory_config(tmp_path, code) -> None:
|
||||
config = _config(tmp_path, sdk_language(code))
|
||||
assert config.language == code
|
||||
|
||||
|
||||
def test_the_language_never_changes_the_collection_identity(tmp_path) -> None:
|
||||
def identity(config):
|
||||
return (
|
||||
dataclasses.asdict(config.vector_store),
|
||||
config.embedder.model,
|
||||
config.embedder.embedding_dims,
|
||||
config.graph_store.db_path,
|
||||
config.cache.db_path,
|
||||
)
|
||||
|
||||
baseline = identity(_config(tmp_path, None))
|
||||
for code in PASSED:
|
||||
assert identity(_config(tmp_path, code)) == baseline
|
||||
|
||||
|
||||
def test_acquire_client_passes_the_language_only_when_there_is_one(tmp_path, monkeypatch) -> None:
|
||||
calls: list[dict] = []
|
||||
monkeypatch.setattr(
|
||||
hy_sdk_runtime, "_acquire_client", lambda profile, **kwargs: calls.append(kwargs)
|
||||
)
|
||||
profile = hy_sdk_runtime.builtin_retrieval_snapshot()
|
||||
hy_sdk_runtime.acquire_client(profile, api_key=None, data_dir=tmp_path, mode="pro")
|
||||
hy_sdk_runtime.acquire_client(
|
||||
profile, api_key=None, data_dir=tmp_path, mode="pro", language="ja"
|
||||
)
|
||||
assert "language" not in calls[0]
|
||||
assert calls[1]["language"] == "ja"
|
||||
|
||||
|
||||
# --- the stored setting -----------------------------------------------------------
|
||||
|
||||
|
||||
def test_the_setting_persists_and_an_untouched_install_writes_none(tmp_path) -> None:
|
||||
store = ConfigStore(tmp_path)
|
||||
store.set_sync("cursor", True)
|
||||
assert "ui_language" not in json.loads((tmp_path / "config.json").read_text())
|
||||
assert store.ui_language() is None
|
||||
store.set_ui_language("ja")
|
||||
assert json.loads((tmp_path / "config.json").read_text())["ui_language"] == "ja"
|
||||
assert ConfigStore(tmp_path).ui_language() == "ja"
|
||||
store.set_ui_language(None)
|
||||
assert ConfigStore(tmp_path).ui_language() is None
|
||||
with pytest.raises(ValueError, match="invalid_ui_language"):
|
||||
store.set_ui_language("zh-CN") # type: ignore[arg-type]
|
||||
with pytest.raises(ValidationError):
|
||||
ConfigDocument.model_validate({"ui_language": "klingon"})
|
||||
|
||||
|
||||
# --- rebuilding handles -------------------------------------------------------------
|
||||
|
||||
|
||||
class _FakeHandle:
|
||||
def __init__(self, language: str | None) -> None:
|
||||
self.language = language
|
||||
self.closed = False
|
||||
|
||||
async def client(self):
|
||||
return self
|
||||
|
||||
async def close(self):
|
||||
self.closed = True
|
||||
|
||||
|
||||
def test_the_read_handle_rebuilds_its_client_when_the_language_changes(
|
||||
monkeypatch, tmp_path
|
||||
) -> None:
|
||||
language: list[str | None] = [None]
|
||||
made: list[_FakeHandle] = []
|
||||
|
||||
def handle_for(profile, *, api_key, data_dir, mode=None, features=(), language=None):
|
||||
made.append(_FakeHandle(language))
|
||||
return made[-1]
|
||||
|
||||
monkeypatch.setattr(sdk_read_handle, "SdkClientHandle", handle_for)
|
||||
|
||||
class _NoProfile:
|
||||
def active_identity(self):
|
||||
return None
|
||||
|
||||
reads = sdk_read_handle.ActiveProfileReadHandle(
|
||||
profiles=_NoProfile(), # type: ignore[arg-type]
|
||||
secrets=None, # type: ignore[arg-type]
|
||||
data_dir=tmp_path,
|
||||
memory_mode=lambda: "pro",
|
||||
language=lambda: language[0],
|
||||
)
|
||||
|
||||
async def scenario():
|
||||
# A use lasts as long as the task that asked, so each read is a task.
|
||||
old = await asyncio.create_task(reads.client())
|
||||
assert old.language is None
|
||||
assert await asyncio.create_task(reads.client()) is old
|
||||
language[0] = "ja"
|
||||
fresh = await asyncio.create_task(reads.client())
|
||||
assert fresh is not old and fresh.language == "ja"
|
||||
for _ in range(10):
|
||||
await asyncio.sleep(0)
|
||||
assert old.closed and not fresh.closed
|
||||
await reads.close()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
class _Profiles:
|
||||
def active_identity(self):
|
||||
return ("profile-a", 1)
|
||||
|
||||
def snapshot(self, profile_id, revision):
|
||||
return SimpleNamespace(profile_id=profile_id, has_api_key=False)
|
||||
|
||||
def resolve_snapshot_secret(self, profile_id, revision, secrets):
|
||||
return None
|
||||
|
||||
|
||||
def test_a_language_change_keeps_a_busy_write_handle_until_its_write_ends(monkeypatch) -> None:
|
||||
language: list[str | None] = [None]
|
||||
opened: list[_FakeHandle] = []
|
||||
|
||||
def handle(profile, **kwargs):
|
||||
opened.append(_FakeHandle(kwargs.get("language")))
|
||||
return opened[-1]
|
||||
|
||||
monkeypatch.setattr(sdk_write_processor, "SdkClientHandle", handle)
|
||||
coordinator = sdk_write_processor.SdkWriteCoordinator(
|
||||
profiles=_Profiles(), # type: ignore[arg-type]
|
||||
secrets=None, # type: ignore[arg-type]
|
||||
data_dir=None,
|
||||
transport_factory=lambda _profile: None,
|
||||
sedimentation_enabled=lambda: True,
|
||||
memory_mode=lambda: "pro",
|
||||
sdk_language=lambda: language[0],
|
||||
)
|
||||
|
||||
async def scenario():
|
||||
async with coordinator.use_client() as client:
|
||||
assert client.language is None
|
||||
language[0] = "de"
|
||||
await coordinator.retire_stale_handles()
|
||||
# In flight: it finishes in the language it started with (#218).
|
||||
assert not opened[0].closed
|
||||
assert opened[0].closed
|
||||
async with coordinator.use_client() as client:
|
||||
assert client.language == "de"
|
||||
language[0] = None
|
||||
await coordinator.retire_stale_handles()
|
||||
assert opened[1].closed
|
||||
await coordinator.close()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
# --- the HTTP surface -----------------------------------------------------------------
|
||||
|
||||
|
||||
def run_app(tmp_path, monkeypatch, scenario) -> None:
|
||||
async def main() -> None:
|
||||
settings = Settings(
|
||||
data_dir=tmp_path / "data",
|
||||
source_home=tmp_path,
|
||||
api_token=SecretStr(TOKEN),
|
||||
imports_enabled=False,
|
||||
automatic_memory_processing=False,
|
||||
live_sync_enabled=False,
|
||||
)
|
||||
app = create_app(settings, secret_store=InMemorySecretStore())
|
||||
|
||||
@asynccontextmanager
|
||||
async def write_client():
|
||||
yield SimpleNamespace()
|
||||
|
||||
async with app.router.lifespan_context(app):
|
||||
monkeypatch.setattr(app.state.sdk_write_coordinator, "use_client", write_client)
|
||||
async with AsyncClient(
|
||||
transport=ASGITransport(app=app),
|
||||
base_url="http://test",
|
||||
headers={"Authorization": f"Bearer {TOKEN}"},
|
||||
) as client:
|
||||
await scenario(app, client)
|
||||
|
||||
asyncio.run(main())
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_ui_language_api_get_put_and_status(tmp_path, monkeypatch) -> None:
|
||||
async def scenario(app, client):
|
||||
reads = app.state.sdk_client_handle
|
||||
writes = app.state.sdk_write_coordinator
|
||||
retired: list[str | None] = []
|
||||
retire = writes.retire_stale_handles
|
||||
|
||||
async def counting():
|
||||
retired.append(writes.sdk_language())
|
||||
await retire()
|
||||
|
||||
monkeypatch.setattr(writes, "retire_stale_handles", counting)
|
||||
|
||||
current = (await client.get("/api/v1/settings/ui-language")).json()
|
||||
assert current["ui_language"] is None and current["sdk_language"] == "auto"
|
||||
assert current["languages"] == list(UI_LANGUAGES)
|
||||
assert reads.current_language() is None and writes.sdk_language() is None
|
||||
|
||||
# zh-Hans is passed explicitly: Chinese prompts, handles rebuilt (#243).
|
||||
body = (
|
||||
await client.put("/api/v1/settings/ui-language", json={"ui_language": "zh-Hans"})
|
||||
).json()
|
||||
assert body == {
|
||||
**current,
|
||||
"ui_language": "zh-Hans",
|
||||
"sdk_language": "zh-Hans",
|
||||
"rebuilt": True,
|
||||
}
|
||||
assert reads.current_language() == "zh-Hans" and writes.sdk_language() == "zh-Hans"
|
||||
assert retired == ["zh-Hans"]
|
||||
processing = (await client.get("/api/v1/memory-processing")).json()
|
||||
assert processing["pipeline"]["sdk_language"] == "zh-Hans"
|
||||
|
||||
# The same value again changes nothing.
|
||||
body = (
|
||||
await client.put("/api/v1/settings/ui-language", json={"ui_language": "zh-Hans"})
|
||||
).json()
|
||||
assert body["rebuilt"] is False and retired == ["zh-Hans"]
|
||||
|
||||
body = (await client.put("/api/v1/settings/ui-language", json={"ui_language": "ja"})).json()
|
||||
assert body["sdk_language"] == "ja" and body["rebuilt"] is True
|
||||
assert reads.current_language() == "ja" and writes.sdk_language() == "ja"
|
||||
assert retired == ["zh-Hans", "ja"]
|
||||
|
||||
# An explicit English choice forces English prompts.
|
||||
body = (await client.put("/api/v1/settings/ui-language", json={"ui_language": "en"})).json()
|
||||
assert body["sdk_language"] == "en" and body["rebuilt"] is True
|
||||
|
||||
processing = (await client.get("/api/v1/memory-processing")).json()
|
||||
assert processing["pipeline"]["ui_language"] == "en"
|
||||
assert processing["pipeline"]["sdk_language"] == "en"
|
||||
|
||||
# Unset (CLI `settings language system`): auto again.
|
||||
body = (await client.put("/api/v1/settings/ui-language", json={"ui_language": None})).json()
|
||||
assert body["sdk_language"] == "auto" and body["rebuilt"] is True
|
||||
assert reads.current_language() is None
|
||||
saved = json.loads((tmp_path / "data" / "config.json").read_text())
|
||||
assert saved["ui_language"] is None
|
||||
|
||||
for bad in ({"ui_language": "zh-CN"}, {"ui_language": "xx"}, {"language": "ja"}, {}):
|
||||
response = await client.put("/api/v1/settings/ui-language", json=bad)
|
||||
assert response.status_code == 422, bad
|
||||
|
||||
run_app(tmp_path, monkeypatch, scenario)
|
||||
Generated
+5
-5
@@ -991,7 +991,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "hy-memory"
|
||||
version = "2.0.9"
|
||||
version = "2.0.11"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "chromadb" },
|
||||
@@ -1008,9 +1008,9 @@ dependencies = [
|
||||
{ name = "scikit-learn" },
|
||||
{ name = "tiktoken" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/48/28/e01fd9a92cd090c8236df0dd06dae049c5de66345e5bd9653497ed56125e/hy_memory-2.0.9.tar.gz", hash = "sha256:5f55ea92044ced19a00b4e56c565215adedd3439a1bdb9cfe9ffe42387c8f1a1", size = 858323, upload-time = "2026-11-01T00:29:49.914Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/01/93/aec7d243259f6a315f38e8ebc21a9eea153a48fc13648fda149be7672788/hy_memory-2.0.11.tar.gz", hash = "sha256:96d6bd606d61052f127cd274aa6f8de8e7151d6250f1e89e4ef5b90e18ec7cb0", size = 1245738, upload-time = "2026-11-01T14:15:02.474Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d5/44/4368555a408a58daf902c81b62184a3c99f12abd45d4f1144bf79f171a6f/hy_memory-2.0.9-py3-none-any.whl", hash = "sha256:95dbea232d2041f9b0ea3d08dd751603c2a5c0e4422b97e74f664e6ca913549e", size = 923250, upload-time = "2026-11-01T00:29:47.814Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1a/85/1f5e6a13507a0cfcdce26f67c590df00d327a0f7d5c5b5483c3bd0fa2f85/hy_memory-2.0.11-py3-none-any.whl", hash = "sha256:c0173444a83c1197a49acaccb123f8286cea205b4838c1e83d106974f5bad259", size = 1315871, upload-time = "2026-11-01T14:14:58.764Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1801,7 +1801,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "one-memory"
|
||||
version = "1.0.5"
|
||||
version = "1.0.6"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "aiomysql" },
|
||||
@@ -1837,7 +1837,7 @@ requires-dist = [
|
||||
{ name = "fastapi", specifier = ">=0.141.1" },
|
||||
{ name = "filelock", specifier = ">=3.32.4" },
|
||||
{ name = "httpx", specifier = ">=0.28.1" },
|
||||
{ name = "hy-memory", specifier = "==2.0.9" },
|
||||
{ name = "hy-memory", specifier = "==2.0.11" },
|
||||
{ name = "keyring", specifier = ">=25.7.0" },
|
||||
{ name = "numpy", specifier = "==2.5.2" },
|
||||
{ name = "onnxruntime", specifier = "==1.23.2" },
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@one-memory/desktop",
|
||||
"version": "1.0.5",
|
||||
"version": "1.0.6",
|
||||
"private": true,
|
||||
"main": "dist/main.js",
|
||||
"scripts": {
|
||||
|
||||
@@ -30,7 +30,7 @@ it("rebuilds translated labels from committed language updates without exporting
|
||||
expect(buildMenu).toHaveBeenLastCalledWith(
|
||||
expect.arrayContaining([expect.objectContaining({ label: "Ayuda" })]),
|
||||
);
|
||||
snapshot.locale = "zh-CN";
|
||||
snapshot.locale = "zh-Hans";
|
||||
change?.();
|
||||
expect(read).toHaveBeenCalledTimes(2);
|
||||
expect(buildMenu).toHaveBeenCalledTimes(2);
|
||||
|
||||
@@ -16,14 +16,14 @@ export function installApplicationMenu(options: {
|
||||
} catch {
|
||||
options.reportError(
|
||||
translate(
|
||||
options.language?.read().locale ?? "zh-CN",
|
||||
options.language?.read().locale ?? "zh-Hans",
|
||||
"nativeExportError",
|
||||
),
|
||||
);
|
||||
}
|
||||
};
|
||||
const refresh = () => {
|
||||
const locale = options.language?.read().locale ?? "zh-CN";
|
||||
const locale = options.language?.read().locale ?? "zh-Hans";
|
||||
options.setMenu(
|
||||
options.buildMenu([
|
||||
{
|
||||
|
||||
@@ -28,8 +28,8 @@ function fixture(
|
||||
};
|
||||
let changed: (() => void) | undefined;
|
||||
const languageSnapshot: LanguageSnapshot = {
|
||||
preference: "zh-CN",
|
||||
locale: "zh-CN",
|
||||
preference: "zh-Hans",
|
||||
locale: "zh-Hans",
|
||||
revision: 1,
|
||||
};
|
||||
const app = { isPackaged: packaged, quit: vi.fn() };
|
||||
@@ -121,7 +121,7 @@ describe("native macOS background tray", () => {
|
||||
expect(f.setLoginPreference).not.toHaveBeenCalled();
|
||||
f.click(0);
|
||||
expect(f.open).toHaveBeenCalledOnce();
|
||||
f.changeLanguage("zh-CN");
|
||||
f.changeLanguage("zh-Hans");
|
||||
await flush();
|
||||
expect(f.menu()[0]?.label).toBe("打开 MemHub");
|
||||
f.click(3);
|
||||
|
||||
@@ -44,7 +44,7 @@ export function installBackgroundTray(options: TrayOptions): () => void {
|
||||
let generation = 0;
|
||||
const refresh = async () => {
|
||||
if (disposed) return;
|
||||
const locale = options.language?.read().locale ?? "zh-CN";
|
||||
const locale = options.language?.read().locale ?? "zh-Hans";
|
||||
const t = (key: MessageKey) =>
|
||||
translate(locale, key, { product: productName });
|
||||
const current = ++generation;
|
||||
@@ -82,7 +82,7 @@ export function installBackgroundTray(options: TrayOptions): () => void {
|
||||
.catch(() =>
|
||||
options.reportError(
|
||||
translate(
|
||||
options.language?.read().locale ?? "zh-CN",
|
||||
options.language?.read().locale ?? "zh-Hans",
|
||||
"nativeLoginError",
|
||||
),
|
||||
),
|
||||
|
||||
@@ -15,9 +15,13 @@ it("bakes fallback release identity when the packaging pipeline omits it", async
|
||||
expect(config).toMatch(
|
||||
/__ONE_MEMORY_BUILD_NUMBER__: JSON\.stringify\(\s*String\(releaseBuildNumber\)/,
|
||||
);
|
||||
expect(config).toMatch(
|
||||
/process\.env\.ONE_MEMORY_COMMIT\?\.trim\(\) \|\| readSourceCommit\(\)/,
|
||||
expect(config).toContain(
|
||||
"const releaseCommit = resolveReleaseCommit(process.env.ONE_MEMORY_COMMIT)",
|
||||
);
|
||||
expect(config).toMatch(/if \(!trimmed\) return readSourceCommit\(\);/);
|
||||
// A supplied commit must be the full 40-character form the runtime accepts.
|
||||
expect(config).toContain("/^[0-9a-f]{40}$/i.test(trimmed)");
|
||||
expect(config).toContain("return trimmed.toLowerCase();");
|
||||
expect(config).toMatch(
|
||||
/execFileSync\("git", \["rev-parse", "HEAD"\],[\s\S]*?encoding: "utf8"/,
|
||||
);
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
import type { GuiLocale, LanguageSnapshot } from "@one-memory/i18n";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { coreUiLanguage, installCoreLanguageSync } from "./core-language-sync";
|
||||
|
||||
function source(initial: LanguageSnapshot) {
|
||||
let snapshot = initial;
|
||||
const listeners = new Set<(value: LanguageSnapshot) => void>();
|
||||
return {
|
||||
read: () => snapshot,
|
||||
subscribe(listener: (value: LanguageSnapshot) => void) {
|
||||
listeners.add(listener);
|
||||
return () => listeners.delete(listener);
|
||||
},
|
||||
change(next: LanguageSnapshot) {
|
||||
snapshot = next;
|
||||
for (const listener of listeners) listener(next);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const settle = () => new Promise((resolve) => setTimeout(resolve, 0));
|
||||
|
||||
describe("Core language sync", () => {
|
||||
it("tells Core the language the GUI shows, system included", () => {
|
||||
const shown = (
|
||||
preference: LanguageSnapshot["preference"],
|
||||
locale: GuiLocale,
|
||||
) => coreUiLanguage({ preference, locale, revision: 0 });
|
||||
expect(shown("system", "zh-Hans")).toBe("zh-Hans");
|
||||
expect(shown("system", "en")).toBe("en");
|
||||
expect(shown("zh-Hans", "zh-Hans")).toBe("zh-Hans");
|
||||
expect(shown("en", "en")).toBe("en");
|
||||
expect(shown("ja", "ja")).toBe("ja");
|
||||
});
|
||||
|
||||
it("sends the stored choice once at launch, then every change", async () => {
|
||||
const language = source({ preference: "es", locale: "es", revision: 0 });
|
||||
const send = vi.fn(async () => {});
|
||||
const dispose = installCoreLanguageSync({ language, send });
|
||||
await settle();
|
||||
expect(send.mock.calls).toEqual([["es"]]);
|
||||
language.change({ preference: "ja", locale: "ja", revision: 1 });
|
||||
await settle();
|
||||
language.change({ preference: "system", locale: "en", revision: 2 });
|
||||
await settle();
|
||||
expect(send.mock.calls).toEqual([["es"], ["ja"], ["en"]]);
|
||||
dispose();
|
||||
language.change({ preference: "de", locale: "de", revision: 3 });
|
||||
await settle();
|
||||
expect(send).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
|
||||
it("an untouched install sends the system language the GUI shows", async () => {
|
||||
const language = source({
|
||||
preference: "system",
|
||||
locale: "zh-Hans",
|
||||
revision: 0,
|
||||
});
|
||||
const send = vi.fn(async () => {});
|
||||
installCoreLanguageSync({ language, send });
|
||||
await settle();
|
||||
expect(send.mock.calls).toEqual([["zh-Hans"]]);
|
||||
// Choosing the language already shown changes nothing Core sees.
|
||||
language.change({ preference: "zh-Hans", locale: "zh-Hans", revision: 1 });
|
||||
await settle();
|
||||
expect(send).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("reports a failed send and retries it with the next change", async () => {
|
||||
const language = source({ preference: "fr", locale: "fr", revision: 0 });
|
||||
const send = vi
|
||||
.fn<(value: string) => Promise<void>>()
|
||||
.mockRejectedValueOnce(new Error("core_unavailable"))
|
||||
.mockResolvedValue(undefined);
|
||||
const reportError = vi.fn();
|
||||
installCoreLanguageSync({ language, send, reportError });
|
||||
await settle();
|
||||
expect(reportError).toHaveBeenCalledTimes(1);
|
||||
language.change({ preference: "fr", locale: "fr", revision: 1 });
|
||||
await settle();
|
||||
expect(send.mock.calls).toEqual([["fr"], ["fr"]]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,50 @@
|
||||
import type { GuiLocale, LanguageSnapshot } from "@one-memory/i18n";
|
||||
import type { NativeLanguageSource } from "./native-language";
|
||||
|
||||
/**
|
||||
* What Core is told: the language the GUI shows. "Follow the system" sends
|
||||
* the language it resolved to, so the memories match the interface a person
|
||||
* sees; the App never leaves the SDK to detect zh/en (#243).
|
||||
*/
|
||||
export function coreUiLanguage(snapshot: LanguageSnapshot): GuiLocale {
|
||||
return snapshot.locale;
|
||||
}
|
||||
|
||||
/**
|
||||
* Keep Core's `ui_language` setting on the display language the GUI shows.
|
||||
*
|
||||
* Core builds every SDK client with it and passes it, zh-Hans included, as
|
||||
* the SDK prompt language. Sent once when installed, so a choice made before
|
||||
* this existed, or the system language followed at this launch, reaches
|
||||
* Core; then on every committed change. A value Core already has is not sent
|
||||
* again. Best effort: a failed send is reported and retried with the next
|
||||
* change or launch, and never blocks the GUI.
|
||||
*/
|
||||
export function installCoreLanguageSync(options: {
|
||||
language: NativeLanguageSource;
|
||||
send(uiLanguage: GuiLocale): Promise<void>;
|
||||
reportError?(error: unknown): void;
|
||||
}): () => void {
|
||||
let closed = false;
|
||||
let sent: GuiLocale | undefined;
|
||||
let queue: Promise<void> = Promise.resolve();
|
||||
const push = () => {
|
||||
queue = queue.then(async () => {
|
||||
if (closed) return;
|
||||
const value = coreUiLanguage(options.language.read());
|
||||
if (value === sent) return;
|
||||
try {
|
||||
await options.send(value);
|
||||
sent = value;
|
||||
} catch (error) {
|
||||
options.reportError?.(error);
|
||||
}
|
||||
});
|
||||
};
|
||||
const unsubscribe = options.language.subscribe(push);
|
||||
push();
|
||||
return () => {
|
||||
closed = true;
|
||||
unsubscribe();
|
||||
};
|
||||
}
|
||||
@@ -23,8 +23,8 @@ const fake = vi.hoisted(() => ({
|
||||
unload: undefined as unknown as EventTarget,
|
||||
packaged: true,
|
||||
userData: "",
|
||||
preferred: vi.fn(() => ["es-MX", "zh-CN"]),
|
||||
locale: vi.fn(() => "zh-CN"),
|
||||
preferred: vi.fn(() => ["es-MX", "zh-Hans"]),
|
||||
locale: vi.fn(() => "zh-Hans"),
|
||||
menu: vi.fn(),
|
||||
loadFailure: false,
|
||||
reduceMotion: false as unknown,
|
||||
@@ -301,10 +301,10 @@ describe("GUI language main IPC", () => {
|
||||
await expect(read(eventFor(window))).rejects.toThrow();
|
||||
await expect(write(eventFor(window), "es")).rejects.toThrow();
|
||||
window.frame.url = "one-memory://app/index.html";
|
||||
await write(eventFor(window), "zh-CN");
|
||||
await write(eventFor(window), "zh-Hans");
|
||||
expect(window.frame.send).toHaveBeenCalledWith(
|
||||
"one-memory:language-preference-changed",
|
||||
{ preference: "zh-CN", locale: "zh-CN", revision: 1 },
|
||||
{ preference: "zh-Hans", locale: "zh-Hans", revision: 1 },
|
||||
);
|
||||
expect(foreign.frame.send).not.toHaveBeenCalled();
|
||||
window.frame.send.mockImplementation(() => {
|
||||
@@ -539,7 +539,7 @@ describe("language preference preload", () => {
|
||||
"es",
|
||||
);
|
||||
fake.invoke.mockClear();
|
||||
await expect(api.setLanguagePreference("fr")).rejects.toThrow();
|
||||
await expect(api.setLanguagePreference("it")).rejects.toThrow();
|
||||
expect(fake.invoke).not.toHaveBeenCalled();
|
||||
const listener = vi.fn();
|
||||
const unsubscribe = api.onLanguagePreferenceChanged(listener);
|
||||
|
||||
@@ -91,7 +91,7 @@ describe("early desktop startup errors", () => {
|
||||
new Error("desktop_startup_failed"),
|
||||
);
|
||||
expect(showError).toHaveBeenCalledExactlyOnceWith(
|
||||
translate(locale ?? "zh-CN", "nativeErrorTitle"),
|
||||
translate(locale ?? "zh-Hans", "nativeErrorTitle"),
|
||||
stableRuntimeUserMessage(new Error("desktop_startup_failed")),
|
||||
);
|
||||
expect(exit).toHaveBeenCalledExactlyOnceWith(1);
|
||||
|
||||
@@ -59,9 +59,9 @@ export function installEarlyStartupErrorHandler(
|
||||
// The dialog is the only part of this a person ever reads, so it gets the
|
||||
// explanation, not the code. The code stays in the diagnostic event,
|
||||
// where support can still match it against a known failure.
|
||||
let locale: GuiLocale = "zh-CN";
|
||||
let locale: GuiLocale = "zh-Hans";
|
||||
try {
|
||||
locale = options.readLocale?.() ?? "zh-CN";
|
||||
locale = options.readLocale?.() ?? "zh-Hans";
|
||||
} catch {
|
||||
// Preferences may not exist yet; do not read files or guess the locale.
|
||||
}
|
||||
|
||||
@@ -526,7 +526,22 @@ describe("hourly isolated desktop launch", () => {
|
||||
expect(resolvePackagedBuildNumber("-1")).toBeUndefined();
|
||||
expect(resolvePackagedBuildNumber(undefined, "3907")).toBe(3907);
|
||||
expect(resolvePackagedBuildNumber("3908", "3907")).toBe(3908);
|
||||
expect(resolvePackagedCommit(undefined, "388fd0e9")).toBe("388fd0e9");
|
||||
expect(resolvePackagedCommit("not-a-sha", "388fd0e9")).toBe("388fd0e9");
|
||||
});
|
||||
|
||||
it("resolves only a full 40-character commit and lowercases it", () => {
|
||||
// Every downstream check (packaged-runtime install, local install, release
|
||||
// download, release scripts) requires 40 lowercase hex characters.
|
||||
const embedded = "388fd0e9b3c41a2d5e6f7a8b9c0d1e2f3a4b5c6d";
|
||||
expect(resolvePackagedCommit(undefined, embedded)).toBe(embedded);
|
||||
expect(resolvePackagedCommit(embedded.toUpperCase())).toBe(embedded);
|
||||
expect(
|
||||
resolvePackagedCommit(` ${embedded.toUpperCase()} `, undefined),
|
||||
).toBe(embedded);
|
||||
expect(resolvePackagedCommit("388fd0e9", embedded)).toBe(embedded);
|
||||
expect(resolvePackagedCommit("388fd0e9")).toBeUndefined();
|
||||
expect(resolvePackagedCommit("not-a-sha", embedded)).toBe(embedded);
|
||||
expect(resolvePackagedCommit(`${embedded}0`, embedded)).toBe(embedded);
|
||||
expect(resolvePackagedCommit("g".repeat(40), embedded)).toBe(embedded);
|
||||
expect(resolvePackagedCommit(undefined, "unknown")).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -143,6 +143,12 @@ export function resolvePackagedBuildNumber(
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* The packaged-runtime install, local install, release download and release
|
||||
* scripts all require a full 40-character lowercase commit. Normalise here so
|
||||
* that one format crosses this boundary: an uppercase full SHA is lowercased,
|
||||
* and a short or malformed value is skipped so it cannot shadow the next source.
|
||||
*/
|
||||
export function resolvePackagedCommit(
|
||||
environmentValue: unknown,
|
||||
embeddedValue?: unknown,
|
||||
@@ -150,7 +156,7 @@ export function resolvePackagedCommit(
|
||||
for (const value of [environmentValue, embeddedValue]) {
|
||||
if (typeof value !== "string") continue;
|
||||
const trimmed = value.trim();
|
||||
if (/^[0-9a-f]{7,40}$/i.test(trimmed)) return trimmed;
|
||||
if (/^[0-9a-f]{40}$/i.test(trimmed)) return trimmed.toLowerCase();
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
@@ -33,7 +33,7 @@ describe("language preference IPC", () => {
|
||||
if (!read || !write) throw Error("Missing language handlers");
|
||||
await expect(read("foreign")).rejects.toThrow("untrusted");
|
||||
await expect(write("iframe", "es")).rejects.toThrow("untrusted");
|
||||
await expect(write("trusted", "fr")).rejects.toThrow();
|
||||
await expect(write("trusted", "it")).rejects.toThrow();
|
||||
await expect(write("trusted", "es", "extra")).rejects.toThrow();
|
||||
expect(save).not.toHaveBeenCalled();
|
||||
expect(await read("trusted")).toEqual({
|
||||
|
||||
@@ -29,23 +29,42 @@ describe("GUI language preferences", () => {
|
||||
revision: 0,
|
||||
});
|
||||
expect(await readdir(root)).toEqual([]);
|
||||
expect(await store.set("zh-CN")).toEqual({
|
||||
preference: "zh-CN",
|
||||
locale: "zh-CN",
|
||||
expect(await store.set("zh-Hans")).toEqual({
|
||||
preference: "zh-Hans",
|
||||
locale: "zh-Hans",
|
||||
revision: 1,
|
||||
});
|
||||
expect(
|
||||
JSON.parse(await readFile(path.join(root, "gui-language.json"), "utf8")),
|
||||
).toEqual({ preference: "zh-CN" });
|
||||
).toEqual({ preference: "zh-Hans" });
|
||||
expect((await LanguagePreferences.open(root, "es")).read().preference).toBe(
|
||||
"zh-CN",
|
||||
"zh-Hans",
|
||||
);
|
||||
expect(
|
||||
(await LanguagePreferences.open(await profile(), "es")).read().preference,
|
||||
).toBe("system");
|
||||
expect(await readdir(root)).toEqual(["gui-language.json"]);
|
||||
});
|
||||
it.each(["{broken", '{"preference":"fr"}', "null", "[]"])(
|
||||
it.each([
|
||||
["zh-CN", "zh-Hans", "zh-Hans"],
|
||||
["es", "es", "es"],
|
||||
["en", "en", "en"],
|
||||
["ja", "ja", "ja"],
|
||||
["system", "system", "en"],
|
||||
])(
|
||||
"keeps an existing user's stored %s choice as %s",
|
||||
async (stored, preference, locale) => {
|
||||
const root = await profile();
|
||||
const file = path.join(root, "gui-language.json");
|
||||
const content = `${JSON.stringify({ preference: stored })}\n`;
|
||||
await writeFile(file, content);
|
||||
const store = await LanguagePreferences.open(root, "en-US");
|
||||
expect(store.read()).toEqual({ preference, locale, revision: 0 });
|
||||
// Read as today's code; the file is rewritten only by the next change.
|
||||
expect(await readFile(file, "utf8")).toBe(content);
|
||||
},
|
||||
);
|
||||
it.each(["{broken", '{"preference":"it"}', "null", "[]"])(
|
||||
"does not overwrite invalid input %s while loading",
|
||||
async (content) => {
|
||||
const root = await profile();
|
||||
@@ -69,7 +88,7 @@ describe("GUI language preferences", () => {
|
||||
});
|
||||
const listener = vi.fn();
|
||||
store.subscribe(listener);
|
||||
await expect(store.set("zh-CN")).rejects.toThrow(
|
||||
await expect(store.set("zh-Hans")).rejects.toThrow(
|
||||
"language_preference_save_failed",
|
||||
);
|
||||
expect(store.read()).toEqual({
|
||||
@@ -78,7 +97,7 @@ describe("GUI language preferences", () => {
|
||||
revision: 0,
|
||||
});
|
||||
expect(listener).not.toHaveBeenCalled();
|
||||
await store.set("zh-CN");
|
||||
await store.set("zh-Hans");
|
||||
expect(store.read().revision).toBe(1);
|
||||
expect(listener).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
@@ -97,7 +116,7 @@ describe("GUI language preferences", () => {
|
||||
});
|
||||
const listener = vi.fn();
|
||||
const unsubscribe = store.subscribe(listener);
|
||||
const first = store.set("zh-CN");
|
||||
const first = store.set("zh-Hans");
|
||||
const second = store.set("es");
|
||||
await vi.waitFor(() => expect(save).toHaveBeenCalledTimes(1));
|
||||
expect(store.read().revision).toBe(0);
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
isLanguagePreference,
|
||||
type LanguagePreference,
|
||||
type LanguageSnapshot,
|
||||
normalizeLanguagePreference,
|
||||
resolveGuiLocale,
|
||||
} from "@one-memory/i18n";
|
||||
|
||||
@@ -58,14 +59,13 @@ export class LanguagePreferences {
|
||||
let preference: LanguagePreference = "system";
|
||||
try {
|
||||
const data: unknown = JSON.parse(await io.read(file));
|
||||
if (
|
||||
data &&
|
||||
typeof data === "object" &&
|
||||
"preference" in data &&
|
||||
isLanguagePreference(data.preference)
|
||||
) {
|
||||
preference = data.preference;
|
||||
}
|
||||
// An earlier build's code (zh-CN) reads as today's (zh-Hans), so a
|
||||
// person keeps the language they chose; the next change writes it.
|
||||
const stored =
|
||||
data && typeof data === "object" && "preference" in data
|
||||
? normalizeLanguagePreference(data.preference)
|
||||
: null;
|
||||
if (stored !== null) preference = stored;
|
||||
} catch {
|
||||
// Missing, unreadable or corrupt preferences never block startup or get rewritten.
|
||||
}
|
||||
|
||||
@@ -59,6 +59,7 @@ import { installApplicationMenu } from "./application-menu";
|
||||
import { installBackgroundHealthIpc } from "./background-health-ipc";
|
||||
import { installBackgroundTray } from "./background-tray";
|
||||
import { installConnectionSetupIpc } from "./connection-setup-ipc";
|
||||
import { installCoreLanguageSync } from "./core-language-sync";
|
||||
import {
|
||||
CoreSupervisor,
|
||||
ensureRuntimeProcess,
|
||||
@@ -151,6 +152,7 @@ let connectionSetup: ReturnType<typeof installConnectionSetupIpc> | null = null;
|
||||
let disposeBackgroundHealth: (() => void) | null = null;
|
||||
let stableRuntimeRepair: Promise<void> | null = null;
|
||||
let disposeDiagnostics: (() => void) | null = null;
|
||||
let disposeCoreLanguageSync: (() => void) | null = null;
|
||||
let desktopDiagnostics: DiagnosticLogger | null = null;
|
||||
let exportDiagnosticsFromMenu: (() => Promise<void>) | null = null;
|
||||
const desktopStartupStartedAt = Date.now();
|
||||
@@ -165,7 +167,7 @@ installEarlyStartupErrorHandler({
|
||||
process.env.ONE_MEMORY_DATA_DIR?.trim() ||
|
||||
path.join(app.getPath("home"), ".one-memory"),
|
||||
stderr: process.stderr,
|
||||
readLocale: () => languagePreferences?.read().locale ?? "zh-CN",
|
||||
readLocale: () => languagePreferences?.read().locale ?? "zh-Hans",
|
||||
showError: (title, message) => dialog.showErrorBox(title, message),
|
||||
exit: (code) => app.exit(code),
|
||||
});
|
||||
@@ -296,9 +298,9 @@ function installPackagedWebProtocol(): void {
|
||||
installAppProtocol(path.join(process.resourcesPath, "web"), {
|
||||
// Preferences are open by now: beforeNativeUi runs before this window.
|
||||
startupScreen: () => {
|
||||
let locale: GuiLocale = "zh-CN";
|
||||
let locale: GuiLocale = "zh-Hans";
|
||||
try {
|
||||
locale = languagePreferences?.read().locale ?? "zh-CN";
|
||||
locale = languagePreferences?.read().locale ?? "zh-Hans";
|
||||
} catch {
|
||||
// An unreadable preference keeps the Chinese placeholder.
|
||||
}
|
||||
@@ -1404,16 +1406,16 @@ async function initialize(isQuitting: () => boolean): Promise<void> {
|
||||
void dialog.showMessageBox({
|
||||
type: "warning",
|
||||
message: translate(
|
||||
languagePreferences?.read().locale ?? "zh-CN",
|
||||
languagePreferences?.read().locale ?? "zh-Hans",
|
||||
"nativeMoveTitle",
|
||||
),
|
||||
detail: translate(
|
||||
languagePreferences?.read().locale ?? "zh-CN",
|
||||
languagePreferences?.read().locale ?? "zh-Hans",
|
||||
"nativeMoveDetail",
|
||||
),
|
||||
buttons: [
|
||||
translate(
|
||||
languagePreferences?.read().locale ?? "zh-CN",
|
||||
languagePreferences?.read().locale ?? "zh-Hans",
|
||||
"nativeAcknowledge",
|
||||
),
|
||||
],
|
||||
@@ -1518,6 +1520,43 @@ async function initialize(isQuitting: () => boolean): Promise<void> {
|
||||
};
|
||||
},
|
||||
});
|
||||
if (languagePreferences) {
|
||||
// Core builds the SDK clients with the display language (`ui_language`).
|
||||
disposeCoreLanguageSync = installCoreLanguageSync({
|
||||
language: languagePreferences,
|
||||
send: async (uiLanguage) => {
|
||||
const core = coreSupervisor;
|
||||
if (!core) throw new Error("core_supervisor_unavailable");
|
||||
const publishedPort = await readPublishedCorePort(dataDir).catch(
|
||||
() => null,
|
||||
);
|
||||
const response = await fetch(
|
||||
`http://127.0.0.1:${publishedPort ?? core.port}/api/v1/settings/ui-language`,
|
||||
{
|
||||
method: "PUT",
|
||||
headers: {
|
||||
Authorization: `Bearer ${core.apiToken}`,
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body: JSON.stringify({ ui_language: uiLanguage }),
|
||||
signal: AbortSignal.timeout(5_000),
|
||||
},
|
||||
);
|
||||
if (!response.ok)
|
||||
throw new Error(`ui_language_sync_http_${response.status}`);
|
||||
},
|
||||
reportError: (error) =>
|
||||
void diagnostics
|
||||
.log({
|
||||
level: "warn",
|
||||
component: "desktop.language",
|
||||
event: "ui_language.sync_failed",
|
||||
outcome: "failed",
|
||||
...diagnosticError(error),
|
||||
})
|
||||
.catch(() => {}),
|
||||
});
|
||||
}
|
||||
await diagnostics.log({
|
||||
level: "info",
|
||||
component: "desktop.lifecycle",
|
||||
@@ -1657,6 +1696,8 @@ if (desktopVersion !== null) {
|
||||
...(coreSupervisor ? { instance_id: coreSupervisor.instanceId } : {}),
|
||||
});
|
||||
// An admitted native effect must settle and journal before Runtime detach.
|
||||
disposeCoreLanguageSync?.();
|
||||
disposeCoreLanguageSync = null;
|
||||
disposeLanguagePreferences?.();
|
||||
disposeLanguagePreferences = null;
|
||||
await languagePreferences?.drain();
|
||||
@@ -1708,6 +1749,8 @@ if (desktopVersion !== null) {
|
||||
});
|
||||
},
|
||||
dispose: () => {
|
||||
disposeCoreLanguageSync?.();
|
||||
disposeCoreLanguageSync = null;
|
||||
disposeLanguagePreferences?.();
|
||||
disposeLanguagePreferences = null;
|
||||
disposeApplicationMenu?.();
|
||||
@@ -1738,7 +1781,7 @@ if (desktopVersion !== null) {
|
||||
showMigrationFailureDialog({
|
||||
error,
|
||||
dataDir: runtimeDataDir,
|
||||
locale: languagePreferences?.read().locale ?? "zh-CN",
|
||||
locale: languagePreferences?.read().locale ?? "zh-Hans",
|
||||
dependencies: {
|
||||
showMessageBoxSync: (options) => dialog.showMessageBoxSync(options),
|
||||
showItemInFolder: (target) => shell.showItemInFolder(target),
|
||||
@@ -1766,7 +1809,7 @@ if (desktopVersion !== null) {
|
||||
return;
|
||||
dialog.showErrorBox(
|
||||
translate(
|
||||
languagePreferences?.read().locale ?? "zh-CN",
|
||||
languagePreferences?.read().locale ?? "zh-Hans",
|
||||
"nativeErrorTitle",
|
||||
),
|
||||
stableRuntimeFailureDetail(error),
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
MAX_ISSUE_URL_LENGTH,
|
||||
MIGRATION_ISSUE_URL,
|
||||
type MigrationDialogDependencies,
|
||||
migrationDialogText,
|
||||
migrationFailureCode,
|
||||
migrationIssueBody,
|
||||
migrationIssueUrl,
|
||||
@@ -41,7 +42,7 @@ describe("migration failure dialog", () => {
|
||||
const handled = showMigrationFailureDialog({
|
||||
error: new Error("sdk_migration_preflight_failed"),
|
||||
dataDir,
|
||||
locale: "zh-CN",
|
||||
locale: "zh-Hans",
|
||||
dependencies: deps,
|
||||
});
|
||||
|
||||
@@ -176,7 +177,7 @@ describe("migration failure dialog", () => {
|
||||
// Desktop's own migration throws the code the note records.
|
||||
error: new Error("sdk_data_plane_dispatch_failed"),
|
||||
dataDir,
|
||||
locale: "zh-CN",
|
||||
locale: "zh-Hans",
|
||||
dependencies: deps,
|
||||
environment: {
|
||||
appVersion: "2.0.6",
|
||||
@@ -217,6 +218,35 @@ describe("migration failure dialog", () => {
|
||||
expect(body).not.toContain(accountHome);
|
||||
});
|
||||
|
||||
it("speaks the display language in the dialog and the issue body", () => {
|
||||
const en = migrationDialogText("sdk_migration_x", "/tmp/m.log", "en", true);
|
||||
expect(en.detail).toBe(
|
||||
"To protect your data, MemHub will not start. Please attach the migration log to a GitHub issue and send it to us. (Error code: sdk_migration_x)" +
|
||||
"\n\nYour data was restored to its state before the upgrade." +
|
||||
"\n\nLog: /tmp/m.log",
|
||||
);
|
||||
const ja = migrationDialogText("sdk_migration_x", "/tmp/m.log", "ja");
|
||||
expect(ja.message).toBe(
|
||||
"MemHub のアップグレード中にデータ移行が完了しませんでした。",
|
||||
);
|
||||
expect([ja.openLog, ja.quit]).toEqual(["ログを開く", "終了"]);
|
||||
expect(ja.detail).toContain("sdk_migration_x");
|
||||
// A system language code resolves like the GUI does (de-AT → de).
|
||||
expect(migrationDialogText("c", "/l", "de-AT").quit).toBe("Beenden");
|
||||
const body = migrationIssueBody({
|
||||
code: "sdk_migration_preflight_failed",
|
||||
logPath: "~/.one-memory/logs/sdk-migration.log",
|
||||
dataRestored: false,
|
||||
arch: "arm64",
|
||||
locale: "fr",
|
||||
});
|
||||
expect(body).toContain("## Échec de la migration lors de la mise à jour");
|
||||
expect(body).toMatch(/- Code d’erreur\s: sdk_migration_preflight_failed/);
|
||||
expect(body).toContain(": non");
|
||||
expect(body).toMatch(/- Système\s: inconnu \(arm64\)/);
|
||||
expect(body).toContain("~/.one-memory/logs/sdk-migration.log");
|
||||
});
|
||||
|
||||
it("says 未知 for what the log does not know instead of guessing", () => {
|
||||
const body = migrationIssueBody({
|
||||
code: "sdk_migration_preflight_failed",
|
||||
@@ -296,7 +326,7 @@ describe("migration failure dialog", () => {
|
||||
showMigrationFailureDialog({
|
||||
error: new Error("installed_version_conflict"),
|
||||
dataDir: await home(),
|
||||
locale: "zh-CN",
|
||||
locale: "zh-Hans",
|
||||
dependencies: deps,
|
||||
}),
|
||||
).toBe(false);
|
||||
|
||||
@@ -8,6 +8,12 @@ import {
|
||||
readSync,
|
||||
} from "node:fs";
|
||||
import path from "node:path";
|
||||
import {
|
||||
type GuiLocale,
|
||||
isGuiLocale,
|
||||
resolveGuiLocale,
|
||||
translate,
|
||||
} from "@one-memory/i18n";
|
||||
|
||||
/**
|
||||
* What a person sees when an upgrade did not reach SDK authority.
|
||||
@@ -106,51 +112,70 @@ export interface MigrationIssueFacts {
|
||||
arch?: string | null;
|
||||
/** The local log path, already shown with the home directory as `~`. */
|
||||
logPath: string;
|
||||
/** The display language; Simplified Chinese when absent. */
|
||||
locale?: string;
|
||||
}
|
||||
|
||||
const UNKNOWN = "未知";
|
||||
|
||||
function versionText(value: string | null | undefined): string {
|
||||
return typeof value === "string" && VERSION.test(value) ? value : UNKNOWN;
|
||||
/** A display language, or the one a system language code resolves to. */
|
||||
function dialogLocale(locale: string | undefined): GuiLocale {
|
||||
if (locale === undefined) return "zh-Hans";
|
||||
return isGuiLocale(locale) ? locale : resolveGuiLocale("system", locale);
|
||||
}
|
||||
|
||||
function buildText(value: number | null | undefined): string {
|
||||
function versionText(value: string | null | undefined): string | null {
|
||||
return typeof value === "string" && VERSION.test(value) ? value : null;
|
||||
}
|
||||
|
||||
function buildText(value: number | null | undefined): string | null {
|
||||
return typeof value === "number" && Number.isSafeInteger(value) && value > 0
|
||||
? String(value)
|
||||
: UNKNOWN;
|
||||
: null;
|
||||
}
|
||||
|
||||
function stepText(
|
||||
step: string | null | undefined,
|
||||
sub: string | null | undefined,
|
||||
) {
|
||||
if (typeof step !== "string" || !STEP.test(step)) return UNKNOWN;
|
||||
): string | null {
|
||||
if (typeof step !== "string" || !STEP.test(step)) return null;
|
||||
return typeof sub === "string" && STEP.test(sub) ? `${step} / ${sub}` : step;
|
||||
}
|
||||
|
||||
/** The issue body: short, content-free, and honest about what is unknown. */
|
||||
export function migrationIssueBody(facts: MigrationIssueFacts): string {
|
||||
const locale = dialogLocale(facts.locale);
|
||||
const unknown = translate(locale, "migration.unknown");
|
||||
const known = (value: string | null) => value ?? unknown;
|
||||
const restored =
|
||||
facts.dataRestored === true
|
||||
? "是"
|
||||
? translate(locale, "migration.yes")
|
||||
: facts.dataRestored === false
|
||||
? "否"
|
||||
: UNKNOWN;
|
||||
? translate(locale, "migration.no")
|
||||
: unknown;
|
||||
const system = [facts.osName, facts.osVersion]
|
||||
.map((part) => versionText(part))
|
||||
.filter((part) => part !== UNKNOWN)
|
||||
.filter((part) => part !== null)
|
||||
.join(" ");
|
||||
const arch = versionText(facts.arch);
|
||||
return [
|
||||
"## 升级迁移失败",
|
||||
`- 从 MemHub ${versionText(facts.fromVersion)}(build ${buildText(facts.fromBuild)})升级到 ${versionText(facts.toVersion)}(build ${buildText(facts.toBuild)}),迁移数据失败。`,
|
||||
`- 错误码:${CODE.test(facts.code) ? facts.code : UNKNOWN}`,
|
||||
`- 失败步骤:${stepText(facts.step, facts.subStep)}`,
|
||||
`- 数据已恢复到升级前状态:${restored}`,
|
||||
`- 系统:${system || UNKNOWN}${arch === UNKNOWN ? "" : `(${arch})`}`,
|
||||
`- 迁移日志(本地):${facts.logPath}`,
|
||||
translate(locale, "migration.issueHeading"),
|
||||
translate(locale, "migration.issueUpgrade", {
|
||||
fromVersion: known(versionText(facts.fromVersion)),
|
||||
fromBuild: known(buildText(facts.fromBuild)),
|
||||
toVersion: known(versionText(facts.toVersion)),
|
||||
toBuild: known(buildText(facts.toBuild)),
|
||||
}),
|
||||
translate(locale, "migration.issueCode", {
|
||||
code: CODE.test(facts.code) ? facts.code : unknown,
|
||||
}),
|
||||
translate(locale, "migration.issueStep", {
|
||||
step: known(stepText(facts.step, facts.subStep)),
|
||||
}),
|
||||
translate(locale, "migration.issueRestored", { restored }),
|
||||
translate(locale, "migration.issueSystem", { system: system || unknown }) +
|
||||
(arch === null ? "" : translate(locale, "migration.issueArch", { arch })),
|
||||
translate(locale, "migration.issueLog", { path: facts.logPath }),
|
||||
"",
|
||||
"请把上面这个日志文件拖到这里作为附件,再提交。",
|
||||
translate(locale, "migration.issueAttach"),
|
||||
"",
|
||||
].join("\n");
|
||||
}
|
||||
@@ -302,31 +327,18 @@ export function migrationDialogText(
|
||||
locale: string,
|
||||
dataRestored = false,
|
||||
): MigrationDialogText {
|
||||
if (locale.toLowerCase().startsWith("en")) {
|
||||
return {
|
||||
title: "MemHub",
|
||||
message: "The data migration for this MemHub upgrade did not finish.",
|
||||
detail:
|
||||
"To protect your data, MemHub will not start. Please attach the migration log to a GitHub issue and send it to us. " +
|
||||
`(Error code: ${code})` +
|
||||
(dataRestored
|
||||
? "\n\nYour data was restored to its state before the upgrade."
|
||||
: "") +
|
||||
`\n\nLog: ${logPath}`,
|
||||
openLog: "Open log",
|
||||
quit: "Quit",
|
||||
};
|
||||
}
|
||||
const language = dialogLocale(locale);
|
||||
return {
|
||||
title: "MemHub",
|
||||
message: "MemHub 升级时数据迁移没有完成。",
|
||||
message: translate(language, "migration.dialogMessage"),
|
||||
detail:
|
||||
"为保护你的数据,MemHub 不会启动。请把迁移日志附在 GitHub 反馈里发给我们。" +
|
||||
`(错误码:${code})` +
|
||||
(dataRestored ? "\n\n记忆数据已恢复到升级前的状态。" : "") +
|
||||
`\n\n日志:${logPath}`,
|
||||
openLog: "打开日志",
|
||||
quit: "退出",
|
||||
translate(language, "migration.dialogDetail", { code }) +
|
||||
(dataRestored
|
||||
? `\n\n${translate(language, "migration.dataRestored")}`
|
||||
: "") +
|
||||
`\n\n${translate(language, "migration.logPath", { path: logPath })}`,
|
||||
openLog: translate(language, "migration.openLog"),
|
||||
quit: translate(language, "migration.quit"),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -434,6 +446,7 @@ export function showMigrationFailureDialog(options: {
|
||||
osVersion: environment.osVersion ?? null,
|
||||
arch: environment.arch ?? null,
|
||||
logPath: displayPath(logPath, environment.home ?? null),
|
||||
locale: options.locale,
|
||||
});
|
||||
void Promise.resolve(
|
||||
options.dependencies.openExternal(migrationIssueUrl(code, body)),
|
||||
|
||||
@@ -12,7 +12,7 @@ describe("startup screen", () => {
|
||||
expect(html).not.toContain("ipcRenderer");
|
||||
});
|
||||
it("speaks the interface language and keeps Chinese by default", () => {
|
||||
expect(startupScreenHtml()).toContain('<html lang="zh-CN">');
|
||||
expect(startupScreenHtml()).toContain('<html lang="zh-Hans">');
|
||||
expect(startupScreenHtml()).toContain("正在启动,正在准备记忆服务…");
|
||||
const spanish = startupScreenHtml("es");
|
||||
expect(spanish).toContain('<html lang="es">');
|
||||
|
||||
@@ -11,7 +11,7 @@ export const STARTUP_SCREEN_URL = `${APP_ORIGIN}${STARTUP_SCREEN_PATH}`;
|
||||
* the gap. It deliberately makes no IPC calls: it renders before `initialize`
|
||||
* has registered any handler, and anything it asked for would fail.
|
||||
*/
|
||||
export function startupScreenHtml(locale: GuiLocale = "zh-CN"): string {
|
||||
export function startupScreenHtml(locale: GuiLocale = "zh-Hans"): string {
|
||||
return `<!doctype html>
|
||||
<html lang="${locale}">
|
||||
<head>
|
||||
|
||||
@@ -11,8 +11,20 @@ const { resolveBuildNumber, resolveBuildVersion } =
|
||||
};
|
||||
const releaseVersion = resolveBuildVersion(process.env);
|
||||
const releaseBuildNumber = resolveBuildNumber(process.env);
|
||||
const releaseCommit =
|
||||
process.env.ONE_MEMORY_COMMIT?.trim() || readSourceCommit();
|
||||
const releaseCommit = resolveReleaseCommit(process.env.ONE_MEMORY_COMMIT);
|
||||
|
||||
// The runtime accepts only a full 40-character commit, so an explicitly
|
||||
// supplied malformed value fails the build instead of being baked and skipped.
|
||||
function resolveReleaseCommit(explicit: string | undefined): string {
|
||||
const trimmed = explicit?.trim();
|
||||
if (!trimmed) return readSourceCommit();
|
||||
if (!/^[0-9a-f]{40}$/i.test(trimmed)) {
|
||||
throw new Error(
|
||||
`ONE_MEMORY_COMMIT must be a full 40-character hex commit, got ${JSON.stringify(trimmed)}`,
|
||||
);
|
||||
}
|
||||
return trimmed.toLowerCase();
|
||||
}
|
||||
|
||||
function readSourceCommit(): string {
|
||||
try {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@one-memory/site",
|
||||
"version": "1.0.5",
|
||||
"version": "1.0.6",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@one-memory/web",
|
||||
"version": "1.0.5",
|
||||
"version": "1.0.6",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
|
||||
@@ -27,7 +27,7 @@ it("translates application detail live without triggering extra reads or writes"
|
||||
).toBeTruthy();
|
||||
expect(screen.getByRole("switch", { name: "Sincronización" })).toBeTruthy();
|
||||
await waitFor(() => expect(refresh).toHaveBeenCalledTimes(1));
|
||||
localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-CN");
|
||||
localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-Hans");
|
||||
await act(async () =>
|
||||
window.dispatchEvent(
|
||||
new StorageEvent("storage", { key: LANGUAGE_STORAGE_KEY }),
|
||||
@@ -603,7 +603,7 @@ it("shows insertion still on when neither removal nor revocation succeeded", asy
|
||||
});
|
||||
|
||||
let languageListener: ((snapshot: unknown) => void) | undefined;
|
||||
function switchLanguage(locale: "es" | "zh-CN", revision: number) {
|
||||
function switchLanguage(locale: "es" | "zh-Hans", revision: number) {
|
||||
act(() => languageListener?.({ preference: locale, locale, revision }));
|
||||
}
|
||||
async function failDisconnectInSpanish() {
|
||||
@@ -660,7 +660,7 @@ it("re-words a visible failure when the language changes, without new writes", a
|
||||
).getRecallHosts;
|
||||
const readCount = reads.mock.calls.length;
|
||||
const revokes = bridge.revokeRecallPermission.mock.calls.length;
|
||||
switchLanguage("zh-CN", 2);
|
||||
switchLanguage("zh-Hans", 2);
|
||||
expect(
|
||||
await screen.findByText(
|
||||
"连接未能移除,记忆权限也未能撤销,OpenCode 仍可使用记忆,请重试。原因:应用控制通道暂不可用。",
|
||||
|
||||
@@ -23,8 +23,8 @@ import { queryClient, resetClient } from "./query-client";
|
||||
it("localizes the whole home live without repeating reads or changing imported text", async () => {
|
||||
let change:
|
||||
| ((snapshot: {
|
||||
preference: "es" | "zh-CN";
|
||||
locale: "es" | "zh-CN";
|
||||
preference: "es" | "zh-Hans";
|
||||
locale: "es" | "zh-Hans";
|
||||
revision: number;
|
||||
}) => void)
|
||||
| undefined;
|
||||
@@ -52,7 +52,9 @@ it("localizes the whole home live without repeating reads or changing imported t
|
||||
fireEvent.click(
|
||||
screen.getByRole("button", { name: "Ver las 5 aplicaciones" }),
|
||||
);
|
||||
act(() => change?.({ preference: "zh-CN", locale: "zh-CN", revision: 2 }));
|
||||
act(() =>
|
||||
change?.({ preference: "zh-Hans", locale: "zh-Hans", revision: 2 }),
|
||||
);
|
||||
expect(
|
||||
screen.getByRole("heading", { name: "全部应用", level: 1 }),
|
||||
).toBeTruthy();
|
||||
|
||||
@@ -166,7 +166,7 @@ it("traduce el historial en vivo sin cambiar nombres ni contenido importado", as
|
||||
).toBeTruthy();
|
||||
expect(screen.getAllByText("Consulta completada")).toHaveLength(2);
|
||||
expect(screen.getByText("Consulta de memorias")).toBeTruthy();
|
||||
window.localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-CN");
|
||||
window.localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-Hans");
|
||||
fireEvent(window, new StorageEvent("storage", { key: LANGUAGE_STORAGE_KEY }));
|
||||
expect(screen.getByRole("heading", { name: "这次查记忆" })).toBeTruthy();
|
||||
window.localStorage.removeItem(LANGUAGE_STORAGE_KEY);
|
||||
|
||||
@@ -205,7 +205,7 @@ function ConnectionMaterialFallback({
|
||||
? t("connectionFallback.promptShell", { shell: material.shell })
|
||||
: "";
|
||||
const prompt =
|
||||
locale === "zh-CN"
|
||||
locale === "zh-Hans"
|
||||
? material.prompt
|
||||
: t("connectionFallback.promptTemplate", {
|
||||
mcp: material.mcp,
|
||||
|
||||
@@ -111,7 +111,7 @@ afterEach(() => {
|
||||
function SwitchLanguage() {
|
||||
const { setPreference } = useGuiLanguage();
|
||||
return (
|
||||
<button type="button" onClick={() => void setPreference("zh-CN")}>
|
||||
<button type="button" onClick={() => void setPreference("zh-Hans")}>
|
||||
Cambiar idioma
|
||||
</button>
|
||||
);
|
||||
|
||||
@@ -22,7 +22,7 @@ import {
|
||||
function LanguageSwitch() {
|
||||
const { setPreference } = useGuiLanguage();
|
||||
return (
|
||||
<button type="button" onClick={() => void setPreference("zh-CN")}>
|
||||
<button type="button" onClick={() => void setPreference("zh-Hans")}>
|
||||
Cambiar idioma
|
||||
</button>
|
||||
);
|
||||
|
||||
@@ -33,7 +33,7 @@ vi.mock("./api", () => ({
|
||||
function HealthLanguageSwitch() {
|
||||
const { setPreference } = useGuiLanguage();
|
||||
return (
|
||||
<button type="button" onClick={() => void setPreference("zh-CN")}>
|
||||
<button type="button" onClick={() => void setPreference("zh-Hans")}>
|
||||
Cambiar idioma
|
||||
</button>
|
||||
);
|
||||
|
||||
@@ -95,7 +95,7 @@ describe("ImportProgressStrip", () => {
|
||||
function Switch() {
|
||||
const { setPreference } = useGuiLanguage();
|
||||
return (
|
||||
<button type="button" onClick={() => void setPreference("zh-CN")}>
|
||||
<button type="button" onClick={() => void setPreference("zh-Hans")}>
|
||||
Cambiar idioma
|
||||
</button>
|
||||
);
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
// @vitest-environment jsdom
|
||||
|
||||
import { translate } from "@one-memory/i18n";
|
||||
import { LOCALE_NAMES, SUPPORTED_LOCALES, translate } from "@one-memory/i18n";
|
||||
import {
|
||||
cleanup,
|
||||
fireEvent,
|
||||
@@ -20,7 +20,7 @@ afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it.each(["zh-CN", "en", "es"] as const)(
|
||||
it.each(["zh-Hans", "en", "es", "ja", "de", "ru"] as const)(
|
||||
"hides the compact label while preserving its accessible name in %s",
|
||||
async (locale) => {
|
||||
localStorage.setItem(LANGUAGE_STORAGE_KEY, locale);
|
||||
@@ -42,7 +42,7 @@ it.each(["zh-CN", "en", "es"] as const)(
|
||||
},
|
||||
);
|
||||
|
||||
it.each(["zh-CN", "en", "es"] as const)(
|
||||
it.each(["zh-Hans", "en", "es", "zh-Hant", "fr", "ko"] as const)(
|
||||
"keeps the normal settings label visible and accessible in %s",
|
||||
(locale) => {
|
||||
localStorage.setItem(LANGUAGE_STORAGE_KEY, locale);
|
||||
@@ -66,7 +66,7 @@ it.each(["zh-CN", "en", "es"] as const)(
|
||||
},
|
||||
);
|
||||
|
||||
it.each(["system", "zh-CN", "en", "es"] as const)(
|
||||
it.each(["system", "zh-Hans", "en", "es", "pt", "id"] as const)(
|
||||
"keeps one stable settings row for the %s preference",
|
||||
async (preference) => {
|
||||
localStorage.setItem(LANGUAGE_STORAGE_KEY, preference);
|
||||
@@ -81,40 +81,46 @@ it.each(["system", "zh-CN", "en", "es"] as const)(
|
||||
expect(row).toBeTruthy();
|
||||
expect(row?.querySelector("label")?.hidden).toBe(false);
|
||||
await userEvent.click(selector);
|
||||
expect(
|
||||
screen
|
||||
.getByRole("option", {
|
||||
name: /简体中文|Simplified Chinese|Chino simplificado/,
|
||||
})
|
||||
.getAttribute("lang"),
|
||||
).toBe("zh-CN");
|
||||
expect(
|
||||
screen
|
||||
.getByRole("option", { name: /英语|English|Inglés/ })
|
||||
.getAttribute("lang"),
|
||||
).toBe("en");
|
||||
expect(
|
||||
screen
|
||||
.getByRole("option", { name: /西班牙语|Spanish|Español/ })
|
||||
.getAttribute("lang"),
|
||||
).toBe("es");
|
||||
// Every language by its own name, whatever the interface language.
|
||||
for (const locale of SUPPORTED_LOCALES) {
|
||||
expect(
|
||||
screen
|
||||
.getByRole("option", { name: LOCALE_NAMES[locale] })
|
||||
.getAttribute("lang"),
|
||||
).toBe(locale);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it("offers an accessible selector, persists and updates its label", async () => {
|
||||
localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-CN");
|
||||
localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-Hans");
|
||||
render(
|
||||
<LanguageProvider>
|
||||
<LanguageSelector />
|
||||
</LanguageProvider>,
|
||||
);
|
||||
const selector = screen.getByRole("combobox", {
|
||||
name: translate("zh-CN", "languageLabel"),
|
||||
name: translate("zh-Hans", "languageLabel"),
|
||||
});
|
||||
expect(selector.getAttribute("aria-describedby")).toBeTruthy();
|
||||
await userEvent.click(selector);
|
||||
expect(screen.getAllByRole("option")).toHaveLength(4);
|
||||
await userEvent.click(screen.getByRole("option", { name: "西班牙语" }));
|
||||
expect(
|
||||
screen.getAllByRole("option").map((option) => option.textContent),
|
||||
).toEqual([
|
||||
translate("zh-Hans", "languageSystem"),
|
||||
"English",
|
||||
"简体中文",
|
||||
"繁體中文",
|
||||
"日本語",
|
||||
"Español",
|
||||
"Deutsch",
|
||||
"Português",
|
||||
"Français",
|
||||
"한국어",
|
||||
"Русский",
|
||||
"Bahasa Indonesia",
|
||||
]);
|
||||
await userEvent.click(screen.getByRole("option", { name: "Español" }));
|
||||
expect(localStorage.getItem(LANGUAGE_STORAGE_KEY)).toBe("es");
|
||||
expect(
|
||||
screen.getByRole("combobox", { name: translate("es", "languageLabel") }),
|
||||
@@ -143,12 +149,12 @@ it("supports keyboard selection without using the system select menu", async ()
|
||||
expect(selector.getAttribute("aria-expanded")).toBe("true");
|
||||
await userEvent.keyboard("{Escape}");
|
||||
await userEvent.keyboard("{ArrowDown}{ArrowDown}{Enter}");
|
||||
expect(localStorage.getItem(LANGUAGE_STORAGE_KEY)).toBe("zh-CN");
|
||||
expect(localStorage.getItem(LANGUAGE_STORAGE_KEY)).toBe("en");
|
||||
expect(selector.getAttribute("aria-expanded")).toBe("false");
|
||||
});
|
||||
|
||||
it("announces storage errors without replacing the confirmed choice", async () => {
|
||||
localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-CN");
|
||||
localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-Hans");
|
||||
render(
|
||||
<LanguageProvider>
|
||||
<LanguageSelector />
|
||||
@@ -158,12 +164,12 @@ it("announces storage errors without replacing the confirmed choice", async () =
|
||||
throw new Error("denied");
|
||||
});
|
||||
await userEvent.click(screen.getByRole("combobox"));
|
||||
await userEvent.click(screen.getByRole("option", { name: "西班牙语" }));
|
||||
await userEvent.click(screen.getByRole("option", { name: "Español" }));
|
||||
expect(screen.getByRole("alert").textContent).toBe(
|
||||
translate("zh-CN", "languageSaveError"),
|
||||
translate("zh-Hans", "languageSaveError"),
|
||||
);
|
||||
expect((screen.getByRole("combobox") as HTMLButtonElement).value).toBe(
|
||||
"zh-CN",
|
||||
"zh-Hans",
|
||||
);
|
||||
});
|
||||
|
||||
|
||||
@@ -1,4 +1,8 @@
|
||||
import { isLanguagePreference } from "@one-memory/i18n";
|
||||
import {
|
||||
isLanguagePreference,
|
||||
LOCALE_NAMES,
|
||||
SUPPORTED_LOCALES,
|
||||
} from "@one-memory/i18n";
|
||||
import { useId, useState } from "react";
|
||||
import { useGuiLanguage } from "./gui-language";
|
||||
import { OptionMenu } from "./OptionMenu";
|
||||
@@ -7,24 +11,15 @@ export function LanguageSelector({ compact = false }: { compact?: boolean }) {
|
||||
const id = useId();
|
||||
const { preference, setPreference, t, error } = useGuiLanguage();
|
||||
const [pending, setPending] = useState(false);
|
||||
// Each language by its own name, so a person can find theirs in any UI.
|
||||
const options = [
|
||||
{ value: "system", label: t("languageSystem") },
|
||||
{
|
||||
value: "zh-CN",
|
||||
label: compact ? "简体中文" : t("languageChinese"),
|
||||
lang: "zh-CN",
|
||||
},
|
||||
{
|
||||
value: "en",
|
||||
label: compact ? "English" : t("languageEnglish"),
|
||||
lang: "en",
|
||||
},
|
||||
{
|
||||
value: "es",
|
||||
label: compact ? "Español" : t("languageSpanish"),
|
||||
lang: "es",
|
||||
},
|
||||
] as const;
|
||||
...SUPPORTED_LOCALES.map((locale) => ({
|
||||
value: locale,
|
||||
label: LOCALE_NAMES[locale],
|
||||
lang: locale,
|
||||
})),
|
||||
];
|
||||
return (
|
||||
<section
|
||||
aria-labelledby={compact ? `${id}-label` : `${id}-title`}
|
||||
|
||||
Loaded 100 of 154 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user