Compare commits

..
Author SHA1 Message Date
kristinhe1111 51fcd32089 Merge pull request #293 from Tencent-Hunyuan/feat/memhub-model-center
feat(memhub): add safe model center
2026-11-03 19:47:45 +08:00
246 changed files with 4789 additions and 24662 deletions

No files matched your search

+9 -83
View File
@@ -4,35 +4,13 @@ Hy Memory and MemHub are versioned and released independently.
## Unreleased
- MemHub host capture hooks no longer write the database. `one-memory
capture <host>` used to open the main SQLite file from its own short-lived
process on every hook, re-create the retired `evidence` / FTS tables on an
SDK-authority home and write the write queue outside Core's single writer,
contending for the file lock with Core and with other hosts' hooks. It now
reads only the runtime receipt and token and posts the hook's JSON to the
running Core (`POST /api/v1/live-capture/{source}`, 1.5 s budget). Core
applies the same gates (grant, sync switch, rejection counting, automated
sessions, the Cursor/Claude Code rule; the migration fence stays on the hook
side) and stages the turn in-process through the live file-sync path. When
Core is not running the turn is dropped silently; nothing falls back to a
local write.
- MemHub no longer sends one turn to the SDK twice when a host's hook and live
file sync both see it. The two lanes gave the same message different
identities (the hook carries no workspace and often no message id) and
separate assembly rows, so both were written as raws. Hook input for a
source that live sync reads from its own files (every hooked host except
Qwen Work, while live sync is on) is now left to the file path; only a
source without file polling is staged from its hook. Tables an older hook
re-created on an SDK home are dropped by the existing startup cleanup.
## Hy Memory 2.0.14 — 2026-11-04
- `search()` / `async_search()` take an optional `statuses` for explicit
L1_RAW search (`layers=["l1_raw"]`, or any search in lite mode), e.g.
`statuses=["active", "shadow"]`, so raw turns a pro extraction already
shadowed can still be searched. Default stays active-only. Raw hits now
report their real `status`. Passing `statuses` to a non-raw search or to
`scene="all"` raises `ValueError` instead of being ignored.
- MemHub App now exposes the complete memory-mode lifecycle. New installs
explicitly choose lite or pro during onboarding; pro completes only after a
generation model is active, while ultra remains an advanced Settings choice.
Settings shows mode effects, graph backfill, the default 03:00 evolve
schedule and manual evolve. The memory page shows original conversations in
lite and advanced memories in pro/ultra. Existing installs remain pro unless
the user changes them, and embedding maintenance blocks mode/evolve changes.
## Hy Memory 2.0.13 — 2026-11-02
@@ -103,63 +81,11 @@ Hy Memory and MemHub are versioned and released independently.
like the other feature switches. The API answer gains `parameters`, and
`pipeline.features` can now carry the threshold as a number. Refs #241.
## MemHub 1.0.6 — 2026-11-04
## MemHub 1.0.6 — 2026-11-02
Build 44, on `hy-memory==2.0.14`. Linux x64/arm64 Runtime and macOS arm64
Build 38, on `hy-memory==2.0.13`. Linux x64/arm64 Runtime and macOS arm64
App and Runtime.
- MemHub App now exposes the complete memory-mode lifecycle. New installs
explicitly choose lite or pro during onboarding; pro completes only after a
generation model is active, while ultra remains an advanced Settings choice.
Settings shows mode effects, graph backfill, the default 03:00 evolve
schedule and manual evolve. The memory page shows original conversations in
lite and advanced memories in pro/ultra. Existing installs remain pro unless
the user changes them, and embedding maintenance blocks mode/evolve changes.
- MemHub pins `hy-memory==2.0.14`. Lite search is plain RAG over raw: the
App's lite search, CLI context search, `/api/v1/recall` and the Host Bridge
(`MemHub_recall`, context search) return every vector hit above the floor,
ranked by score, over active and shadow raws, so history pro already
processed is still found after switching to lite. The turn view no longer
drops a hit whose turns do not literally contain the query; it shows the
hit's closest turn instead. Pro and ultra raw search stays active-only.
Fixes every raw search failing in packaged builds since #316, which passed
`statuses` to an SDK (2.0.13) that did not accept it. The lite raw-turn
page also stops re-listing the whole raw layer on each poll when a live
session only adds new raws.
- MemHub Core no longer stalls writes or freezes the App for tens of seconds
on a machine with a long live-sync history. Finished live-sync import jobs
are pruned to the newest 20 per source (at startup, in bounded batches, and
every 10 minutes); history imports and every unfinished job are kept. Job
lookups are filtered in SQL with new indexes (schema migration 63) instead
of reading the whole table. The durable write queue runs its claims,
settles and lease renewals on its own threads, so import work can no longer
starve a lease. Import abandon, resume and enqueue, and the file watcher's
path attribution, no longer touch the database or the disk on the event
loop, and the WAL cadence checkpoint runs on its own thread instead of on
the writer's. The write drain now restarts itself after an unexpected exit,
reports the failure with its stack, and shows `write_drain` /
`write_drain_error` in memory-processing and `memhub status`.
- Model Center (#293, #295): Settings splits into 界面语言 / 记忆模式 /
模型配置 pages, with separate 记忆整理 (generation) and 记忆搜索
(embedding) lanes. Changing one lane sends only that lane's patch. An
embedding change shows the switch plan, confirmation, progress, cancel and
rollback, and the search model only changes after the rebuilt index is
verified. Model panels open inside the lane that launched them and close
only after Core confirms the profile is active (#310).
- Lite keeps raw conversations (#297): lite raw writes are no longer stopped
by the "pause model sedimentation" switch, the memory page lists every
imported raw conversation by role (by turn since #316), and lite adds a
manual raw input (source `one-memory-manual`) that never calls a
generation model.
- The home page follows the memory mode (#298): lite counts raw
conversations and previews the latest one; pro and ultra read memories.
Pro and ultra now require an active generation model, in both the mode
preview and the switch.
- Onboarding (#304) asks for lite or pro and resumes after a restart (schema
migration 62); pro finishes only once a generation model is active.
- Smaller App fixes: compact home status copy (#307), clearer mode-switch
effects (#312), Ultra topic nodes are hover-only (#315), and settings
controls use the content width (#317).
- Interfaces in 11 languages: English, Simplified and Traditional Chinese,
Japanese, Spanish, German, Portuguese, French, Korean, Russian and
Indonesian. Every interface language, Simplified Chinese included, sets the
+1 -1
View File
@@ -33,7 +33,7 @@ English | [中文](README.zh.md)
## Latest News
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Open-sourced** quantized **FP8** (**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**) and **GGUF** (**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**) versions of **Hy-Memory-30B-A3B** on Hugging Face!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-11]**: 📦 **Released [MemHub v1.0.6](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.6)** with interfaces in 11 languages, memory modes (lite / pro / ultra with nightly evolve), safe embedding-model switching, multi-user support for the CLI and API, memory writes that retry themselves and pause while the model provider is down, and more reliable upgrades, on **`hy-memory==2.0.14`**. See the release notes for details.
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-11]**: 📦 **Released [MemHub v1.0.6](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.6)** with interfaces in 11 languages, memory modes (lite / pro / ultra with nightly evolve), safe embedding-model switching, multi-user support for the CLI and API, memory writes that retry themselves and pause while the model provider is down, and more reliable upgrades, on **`hy-memory==2.0.13`**. See the release notes for details.
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **Released [MemHub v1.0.5](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.5)** with Chinese, English and Spanish interfaces, one-time pairing codes for more custom Harnesses, and smoother browsing, search, startup and recovery, on **`hy-memory==2.0.8`**. See the release notes for details.
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **Released [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** and **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**, with a signed macOS desktop app and verified macOS/Linux Runtimes!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **Released** **Hy Memory 2.0 (`hy-memory==2.0.4`)** on [PyPI](https://pypi.org/project/hy-memory/2.0.4/) and open-sourced the **Hy-Memory** monorepo together with **[MemHub](memhub/README.md)**!
+1 -1
View File
@@ -33,7 +33,7 @@
## 最新动态
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **模型开源** 在 Hugging Face 发布 **Hy-Memory-30B-A3B** 的 **FP8**(**[Hy-Memory-30B-A3B-FP8](https://huggingface.co/tencent/Hy-Memory-30B-A3B-FP8)**)与 **GGUF**(**[Hy-Memory-30B-A3B-GGUF](https://huggingface.co/tencent/Hy-Memory-30B-A3B-GGUF)**)量化版本模型!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-11]**: 📦 **正式发布 [MemHub v1.0.6](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.6)**:界面支持 11 种语言,新增记忆模式(lite / pro / ultra,ultra 每晚自动整理),可安全切换嵌入模型,CLI 和 API 支持多用户,记忆写入自动重试、模型服务异常时暂停排队,升级更可靠;底层使用 **`hy-memory==2.0.14`**。详见发布说明。
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-11]**: 📦 **正式发布 [MemHub v1.0.6](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.6)**:界面支持 11 种语言,新增记忆模式(lite / pro / ultra,ultra 每晚自动整理),可安全切换嵌入模型,CLI 和 API 支持多用户,记忆写入自动重试、模型服务异常时暂停排队,升级更可靠;底层使用 **`hy-memory==2.0.13`**。详见发布说明。
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **正式发布 [MemHub v1.0.5](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.5)**:支持中文、英文和西班牙语界面,可通过一次性配对码连接更多自定义 Harness,并优化了记忆浏览、搜索、启动和异常恢复;底层使用 **`hy-memory==2.0.8`**。详见发布说明。
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 📦 **正式发布 [MemHub v1.0.4](https://github.com/Tencent-Hunyuan/Hy-Memory/releases/tag/v1.0.4)** 与 **[memhub-cli 1.0.5](https://www.npmjs.com/package/memhub-cli/v/1.0.5)**,提供已签名的 macOS 桌面应用及经过验证的 macOS/Linux Runtime!
* *<mark>[new]</mark>* 🔥🔥🔥 **[2026-10]**: 🚀 **正式发布** **Hy Memory 2.0 (`hy-memory==2.0.4`)**(已上线 [PyPI](https://pypi.org/project/hy-memory/2.0.4/)),并开源 **Hy-Memory** monorepo 与 **[MemHub](memhub/README.md)**!
+5 -24
View File
@@ -1864,7 +1864,6 @@ class HyMemoryClient:
branch: Optional[str] = None,
request_id: Optional[str] = None,
layers: Optional[List[Any]] = None,
statuses: Optional[List[Any]] = None,
embed_override: Optional[Dict[str, Any]] = None,
query_embedding: Optional[List[float]] = None,
rerank: bool = True,
@@ -1957,8 +1956,6 @@ class HyMemoryClient:
layers: 可选。pro/ultra 下只召回指定层级(字符串或
MemoryLayer),在召回阶段生效。lite 下忽略该参数并
强制只召回 L1_RAW。
statuses: 仅用于 L1_RAW 定向搜索,可显式包含 active / shadow。
非 raw 搜索或 scene="all" 传入时拒绝,避免静默忽略。
rerank: 默认 True(原行为)。False 时本次调用跳过全部重排阶段:
chat 路的本地 cross-encoder 重排(recall.local_rerank_enabled)
与记忆关系图重排(graph_rerank / episode 扩展),working/
@@ -2002,7 +1999,7 @@ class HyMemoryClient:
intention_limit=intention_limit, created_after=created_after,
reader=reader,
workspace_id=workspace_id, branch=branch,
request_id=request_id, layers=layers, statuses=statuses,
request_id=request_id, layers=layers,
embed_override=embed_override,
query_embedding=query_embedding,
rerank=rerank,
@@ -3390,14 +3387,12 @@ class HyMemoryClient:
request_id: str,
t0: float,
query_embedding: Optional[List[float]],
statuses: Optional[List[Any]],
) -> Dict[str, Any]:
"""Implement `search(..., layers=['l1_raw'])` without a second API."""
from .models.memory import MemoryLayer, MemoryStatus
embedding = query_embedding or await self._embed_service.embed(query)
fetch = min(max(limit * (10 if session_ids else 2), limit), 10_000)
status_filter = self._normalize_statuses(statuses) or [MemoryStatus.ACTIVE]
hits = await self._vector_store.search(
query_embedding=embedding,
user_ids=user_ids,
@@ -3407,7 +3402,7 @@ class HyMemoryClient:
layers=[MemoryLayer.L1_RAW],
limit=fetch,
score_threshold=min_score,
status_filter=status_filter,
status_filter=[MemoryStatus.ACTIVE],
only_latest=False,
created_after=created_after,
)
@@ -3425,7 +3420,7 @@ class HyMemoryClient:
"memory_id": row.pop("raw_id"),
**row,
"layer": MemoryLayer.L1_RAW.value,
"status": node.status.value,
"status": MemoryStatus.ACTIVE.value,
"score": round(float(hit.get("score", 0.0)), 4),
}
)
@@ -3869,7 +3864,6 @@ class HyMemoryClient:
branch: Optional[str] = None,
request_id: Optional[str] = None,
layers: Optional[List[Any]] = None,
statuses: Optional[List[Any]] = None,
embed_override: Optional[Dict[str, Any]] = None,
query_embedding: Optional[List[float]] = None,
rerank: bool = True,
@@ -3899,7 +3893,7 @@ class HyMemoryClient:
schema_min_score=schema_min_score, schema_limit=schema_limit,
intention_limit=intention_limit, created_after=created_after,
reader=reader, workspace_id=workspace_id, branch=branch,
request_id=request_id, layers=layers, statuses=statuses,
request_id=request_id, layers=layers,
query_embedding=query_embedding,
rerank=rerank,
)
@@ -3914,7 +3908,7 @@ class HyMemoryClient:
schema_min_score=schema_min_score, schema_limit=schema_limit,
intention_limit=intention_limit, created_after=created_after,
reader=reader, workspace_id=workspace_id, branch=branch,
request_id=request_id, layers=layers, statuses=statuses,
request_id=request_id, layers=layers,
query_embedding=query_embedding,
rerank=rerank,
)
@@ -3944,7 +3938,6 @@ class HyMemoryClient:
branch: Optional[str] = None,
request_id: Optional[str] = None,
layers: Optional[List[Any]] = None,
statuses: Optional[List[Any]] = None,
query_embedding: Optional[List[float]] = None,
rerank: bool = True,
) -> Dict[str, Any]:
@@ -4030,17 +4023,6 @@ class HyMemoryClient:
"Use single agent_id with multiple session_ids, or multiple agent_ids without session_ids."
)
if statuses is not None:
from .models.memory import MemoryLayer
requested_layers = self._normalize_layers(layers)
if scene_all or (
not lite_mode and requested_layers != [MemoryLayer.L1_RAW]
):
raise ValueError(
"statuses is supported only for explicit L1_RAW search"
)
if scene_all:
return await self._search_all_scenes(
query, queries=queries, search_mode=search_mode,
@@ -4376,7 +4358,6 @@ class HyMemoryClient:
request_id=request_id,
t0=t0,
query_embedding=query_embedding,
statuses=statuses,
)
read_req = ReadRequest(
+3 -3
View File
@@ -18,8 +18,8 @@ If you run into a problem, please open an Issue with the version, platform,repro
## 发行与平台合同
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.14`。MemHub 源码版本为 `1.0.6`
(build 44),GitHub 仓库为
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.13`。MemHub 源码版本为 `1.0.6`
(build 38),GitHub 仓库为
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
`memhub-cli`。当前代码架构见
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)。
@@ -118,7 +118,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
## Hy Memory
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.14`。SDK 配置、权限边界及 search
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.13`。SDK 配置、权限边界及 search
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
+1 -1
View File
@@ -2352,7 +2352,7 @@
{
"SPDXID": "SPDXRef-Package-hy-memory",
"name": "hy-memory",
"versionInfo": "2.0.14",
"versionInfo": "2.0.13",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"licenseConcluded": "Apache-2.0",
+1 -1
View File
@@ -9,7 +9,7 @@ Per-package license texts are not reproduced here yet; the license
column is filled in as part of release review, and the SBOM records
`NOASSERTION` until then rather than guessing.
## hy-memory 2.0.14
## hy-memory 2.0.13
License: Apache-2.0
-3
View File
@@ -17,14 +17,11 @@ const READ_ONLY_PATHS = [
"/api/v1/model-profiles",
"/api/v1/source-grants",
"/api/v1/sources",
"/api/v2/home-projection",
"/api/v2/home-summary",
"/api/v2/raw",
"/api/v2/memory-library",
"/api/v2/user-profile",
"/api/v2/user-profile/versions",
"/api/v1/settings/memory-mode",
"/api/v1/settings/memory-mode/plan",
"/api/v1/settings/memory-features",
"/api/v1/settings/ui-language",
"/api/v2/evolve",
-175
View File
@@ -1,5 +1,4 @@
import type {
HomeProjectionV1,
ImportIssue,
ImportJob,
LiveSyncHealth,
@@ -27,9 +26,6 @@ export const MEMORY_STATUSES = [
const ID_MAX = 128;
const LABEL_MAX = 256;
const LIST_MAX = 500;
const USER_ID = /^[A-Za-z0-9._-]{1,64}$/;
const ISO_TIME =
/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:Z|[+-]\d{2}:\d{2})$/;
const LIVE_SYNC_STATES = [
"idle",
"running",
@@ -410,153 +406,6 @@ export function validateHomeSummary(value: unknown): SdkHomeSummary | null {
return summary as unknown as SdkHomeSummary;
}
/** `GET /api/v2/home-projection`: the mode-aware home content plane. */
export function validateHomeProjection(
value: unknown,
): HomeProjectionV1 | null {
const projection = exactRecord(value, [
"projection_version",
"user_id",
"effective_mode",
"capabilities",
"visible_count",
"display_state",
"latest",
"observed_at",
]);
if (!projection) return null;
const mode = projection.effective_mode;
const capabilities = exactRecord(projection.capabilities, [
"content_plane",
"generation_required",
"graph_enabled",
"evolve_available",
]);
if (
projection.projection_version !== 1 ||
typeof projection.user_id !== "string" ||
!USER_ID.test(projection.user_id) ||
!oneOf(mode, ["lite", "pro", "ultra"] as const) ||
!capabilities ||
capabilities.content_plane !== (mode === "lite" ? "raw" : "memory") ||
capabilities.generation_required !== (mode !== "lite") ||
capabilities.graph_enabled !== (mode === "ultra") ||
capabilities.evolve_available !== (mode === "ultra") ||
count(projection.visible_count) === null ||
!oneOf(projection.display_state, [
"empty",
"excerpt",
"no_user_excerpt",
"no_displayable_memory",
] as const) ||
!isoTime(projection.observed_at) ||
!validProjectionLatest(projection.latest, mode)
) {
return null;
}
const visible = projection.visible_count as number;
const latest = projection.latest as Record<string, unknown> | null;
if (
(visible === 0 &&
(latest !== null || projection.display_state !== "empty")) ||
(visible > 0 && projection.display_state === "empty") ||
(latest === null &&
["excerpt", "no_user_excerpt"].includes(
projection.display_state as string,
)) ||
(latest?.type === "raw" &&
projection.display_state !== latest.display_state) ||
(latest?.type === "memory" && projection.display_state !== "excerpt")
) {
return null;
}
return projection as unknown as HomeProjectionV1;
}
function validProjectionLatest(
value: unknown,
mode: "lite" | "pro" | "ultra",
): boolean {
if (value === null) return true;
const base = [
"type",
"time",
"time_basis",
"excerpt",
"truncated",
"source_id",
];
const candidate = record(value);
if (!candidate) return false;
const keys =
candidate.type === "raw"
? [...base, "raw_id", "session_id", "display_state"]
: [...base, "memory_id", "scene", "kind", "variant"];
const latest = exactRecord(value, keys);
if (
!latest ||
!(latest.time === null || isoTime(latest.time)) ||
![null, "conversation", "import", "write", "update"].includes(
latest.time_basis as null | string,
) ||
text(latest.excerpt, 200) === null ||
typeof latest.truncated !== "boolean" ||
text(latest.source_id, ID_MAX) === null
) {
return false;
}
if (latest.type === "raw") {
return (
mode === "lite" &&
nonemptyText(latest.raw_id, ID_MAX) &&
nonemptyText(latest.session_id, LABEL_MAX) &&
oneOf(latest.display_state, ["excerpt", "no_user_excerpt"] as const) &&
(latest.display_state !== "no_user_excerpt" ||
(latest.excerpt === "" && latest.truncated === false))
);
}
return (
latest.type === "memory" &&
mode !== "lite" &&
nonemptyText(latest.memory_id, ID_MAX) &&
oneOf(latest.scene, ["chat", "working", "coding"] as const) &&
oneOf(latest.kind, ["chat", "experience"] as const) &&
[null, "working", "coding"].includes(latest.variant as null | string) &&
((latest.kind === "chat" &&
latest.scene === "chat" &&
latest.variant === null) ||
(latest.kind === "experience" &&
latest.scene === latest.variant &&
oneOf(latest.scene, ["working", "coding"] as const)))
);
}
/** The bounded `/api/v2/raw?limit=1` envelope used only for its real total. */
export function validateRawTotal(value: unknown): number | null {
const page = exactRecord(value, [
"user_id",
"items",
"total",
"offset",
"limit",
"time_filtered",
]);
if (
!page ||
typeof page.user_id !== "string" ||
!USER_ID.test(page.user_id) ||
!Array.isArray(page.items) ||
page.items.length > 1 ||
count(page.total) === null ||
page.offset !== 0 ||
page.limit !== 1 ||
page.time_filtered !== false
) {
return null;
}
return page.total as number;
}
function identifiers(value: unknown): string[] | null {
return list(value ?? [], (item) => text(item, ID_MAX));
}
@@ -581,30 +430,6 @@ function record(value: unknown): Record<string, unknown> | null {
: null;
}
function exactRecord(
value: unknown,
keys: readonly string[],
): Record<string, unknown> | null {
const item = record(value);
return item &&
Object.keys(item).length === keys.length &&
keys.every((key) => Object.hasOwn(item, key))
? item
: null;
}
function isoTime(value: unknown): value is string {
return (
typeof value === "string" &&
ISO_TIME.test(value) &&
Number.isFinite(Date.parse(value))
);
}
function nonemptyText(value: unknown, max: number): value is string {
return typeof value === "string" && value.length > 0 && value.length <= max;
}
function text(value: unknown, max: number): string | null {
return typeof value === "string" && value.length <= max ? value : null;
}
@@ -250,20 +250,6 @@ describe("models activate with an embedding change", () => {
code: "embedding_switch_required",
});
});
it("does not treat a domain 404 as an old-Core route miss", async () => {
transport.post.mockResolvedValue({
status: 404,
data: { detail: "embedding_switch_not_found" },
});
const result = await capturedRun(["models", "activate", PROFILE, "--json"]);
expect(result.code).toBe(1);
expect(
transport.post.mock.calls.some(([url]) =>
String(url).endsWith("/activate"),
),
).toBe(false);
});
});
describe("models switch and spaces", () => {
+1 -6
View File
@@ -9,7 +9,6 @@
* makes it live. The CLI shows the plan, asks, starts the job and follows it.
*/
import { isBareNotFound } from "@one-memory/operations";
import type { CoreReader } from "./core-client.js";
import type {
CoreDeleteOperations,
@@ -388,11 +387,7 @@ async function requestPlan(
}
if (outcome.state === "rejected") {
// A Core before the registry has no plan route: activation as before.
if (
outcome.http_status === 404 &&
outcome.code === "http_404" &&
isBareNotFound(outcome.body)
) {
if (outcome.http_status === 404 && outcome.code === "http_404") {
return { state: "unsupported" };
}
const plan = parsePlan(record(outcome.body)?.plan);
+1 -11
View File
@@ -161,7 +161,6 @@ const CORE = new Map<string, { status: number; data: unknown }>([
blocked_stages: [
{ kind: "extract", reason: "model_missing", count: 2 },
],
pipeline: { mode: "pro", configured_mode: "pro" },
},
},
],
@@ -203,9 +202,7 @@ const READ_ONLY_PATHS = [
"/api/v1/live-sync",
"/api/v1/memory-processing",
"/api/v1/model-profiles",
"/api/v2/home-projection",
"/api/v2/home-summary",
"/api/v2/raw",
"/api/v2/memory-library",
"/api/v2/memories/",
"/api/v2/user-profile",
@@ -260,9 +257,6 @@ describe("CLI-20 status summary", () => {
expect(report.sections.model).toMatchObject({ state: "ok", profiles: 1 });
expect(report.sections.memories).toMatchObject({
state: "ok",
content_plane: "memory",
visible_count: 42,
effective_mode: "pro",
memory_count: 42,
});
expect(report.summary.state).toBe("ok");
@@ -909,10 +903,7 @@ describe("CLI-21 memory reads", () => {
expect(isReadOnlyPath("/api/v2/memories/ask")).toBe(false);
expect(isReadOnlyPath("/api/v2/memories/m1")).toBe(true);
expect(isReadOnlyPath("/api/v2/memory-library")).toBe(true);
expect(isReadOnlyPath("/api/v2/home-projection")).toBe(true);
expect(isReadOnlyPath("/api/v2/home-summary")).toBe(true);
expect(isReadOnlyPath("/api/v2/raw")).toBe(true);
expect(isReadOnlyPath("/api/v1/settings/memory-mode/plan")).toBe(true);
// The retired v1 data plane is no longer a CLI read.
expect(isReadOnlyPath("/api/v1/memories/m1")).toBe(false);
expect(isReadOnlyPath("/api/v1/home-summary")).toBe(false);
@@ -1094,8 +1085,7 @@ describe("malformed success responses", () => {
it("refuses a model profile whose string fields are not strings", async () => {
transport.get.mockImplementation(async (url: string) => {
if (url !== "/api/v1/model-profiles")
return CORE.get(url) ?? { status: 404, data: { detail: "not_found" } };
if (url !== "/api/v1/model-profiles") return CORE.get(url);
return {
status: 200,
data: [
-170
View File
@@ -109,39 +109,6 @@ describe("status on a paused model", () => {
});
expect(renderStatusText(report, "MemHub")).not.toContain("模型暂停");
});
it("reports a restarting write drain and its exit code", async () => {
const report = await compose({
...processing({ llm_backoff_until: null, llm_backoff_error: null }),
write_drain: "restarting",
write_drain_error: "operational_error",
});
expect(report.sections.processing).toMatchObject({
write_drain: "restarting",
write_drain_error: "operational_error",
});
const text = renderStatusText(report, "MemHub");
expect(text).toContain(
"写入队列:异常退出(operational_error),正在自动重启",
);
expect(text).not.toContain("write_drain");
});
it("says nothing about a running drain and nulls what an older Core omits", async () => {
const running = await compose({
...processing({ llm_backoff_until: null, llm_backoff_error: null }),
write_drain: "running",
write_drain_error: null,
});
expect(renderStatusText(running, "MemHub")).not.toContain("写入队列");
const older = await compose(
processing({ llm_backoff_until: null, llm_backoff_error: null }),
);
expect(older.sections.processing).toMatchObject({
write_drain: null,
write_drain_error: null,
});
});
});
describe("status and doctor on an unfinished upgrade migration", () => {
@@ -180,140 +147,3 @@ describe("status and doctor on an unfinished upgrade migration", () => {
expect(report.runtime.log_file).toBeUndefined();
});
});
describe("mode-aware status content projection", () => {
const runtime = {
state: "running" as const,
code: "runtime_verified",
host: "127.0.0.1",
port: 30001,
};
const processing = (mode: "lite" | "pro" | "ultra") => ({
pending: 0,
fresh_pending: 0,
backlog_pending: 0,
outside_window_pending: 0,
running: 0,
waiting_for_model: 0,
completed: 0,
failed: 0,
pipeline: { mode, configured_mode: mode },
});
const projection = {
projection_version: 1,
user_id: "local",
effective_mode: "lite",
capabilities: {
content_plane: "raw",
generation_required: false,
graph_enabled: false,
evolve_available: false,
},
visible_count: 7,
display_state: "excerpt",
latest: {
type: "raw",
raw_id: "raw-1",
session_id: "session-1",
display_state: "excerpt",
time: null,
time_basis: null,
excerpt: "hello",
truncated: false,
source_id: "cursor",
},
observed_at: "2026-11-04T00:00:00Z",
};
it("uses the projection without reading either legacy count endpoint", async () => {
const calls: string[] = [];
const reader: CoreReader = async <T>(path: string) => {
calls.push(path);
const data =
path === "/api/v2/home-projection"
? projection
: path === "/api/v1/memory-processing"
? processing("lite")
: null;
return { state: "ok", data } as ReadOutcome<T>;
};
const report = await composeStatusReport({
dataDir: "/tmp/isolated",
runtime,
background: null,
reader,
});
expect(report.sections.memories).toEqual({
state: "ok",
content_plane: "raw",
visible_count: 7,
effective_mode: "lite",
raw_count: 7,
});
expect(calls).not.toContain("/api/v2/raw");
expect(calls).not.toContain("/api/v2/home-summary");
expect(renderStatusText(report, "MemHub")).toContain(
"原始对话:原始对话 7",
);
});
it("falls back to the real raw total only for an old Lite Core", async () => {
const reader: CoreReader = async <T>(path: string) => {
if (path === "/api/v2/home-projection")
return { state: "not_found", code: "http_404" } as ReadOutcome<T>;
if (path === "/api/v1/memory-processing")
return {
state: "ok",
data: processing("lite"),
} as ReadOutcome<T>;
if (path === "/api/v2/raw")
return {
state: "ok",
data: {
user_id: "local",
items: [],
total: 9,
offset: 0,
limit: 1,
time_filtered: false,
},
} as ReadOutcome<T>;
return { state: "unavailable", code: "x" } as ReadOutcome<T>;
};
const report = await composeStatusReport({
dataDir: "/tmp/isolated",
runtime,
background: null,
reader,
});
expect(report.sections.memories).toMatchObject({
state: "ok",
content_plane: "raw",
visible_count: 9,
raw_count: 9,
});
expect(report.sections.memories).not.toHaveProperty("memory_count");
});
it("rejects extra projection fields instead of falling back or leaking them", async () => {
const reader: CoreReader = async <T>(path: string) =>
({
state: "ok",
data:
path === "/api/v2/home-projection"
? { ...projection, secret: "must-not-leak" }
: processing("lite"),
}) as ReadOutcome<T>;
const report = await composeStatusReport({
dataDir: "/tmp/isolated",
runtime,
background: null,
reader,
});
expect(report.sections.memories).toEqual({
state: "invalid",
code: "response_invalid",
});
expect(JSON.stringify(report)).not.toContain("must-not-leak");
});
});
+8 -123
View File
@@ -1,5 +1,4 @@
import type {
HomeProjectionV1,
LiveSyncHealth,
MemoryProcessingSummary,
ModelProfile,
@@ -13,12 +12,10 @@ import { sdkMigrationLogPath } from "@one-memory/runtime-control";
import type { CoreReader, ReadOutcome } from "./core-client.js";
import {
type SdkHomeSummary,
validateHomeProjection,
validateHomeSummary,
validateLiveSync,
validateModelProfiles,
validateProcessing,
validateRawTotal,
validateSources,
} from "./core-responses.js";
import { describeEmbeddingSwitchSummary } from "./embedding-switch-flow.js";
@@ -214,12 +211,12 @@ export async function composeStatusReport(options: {
for (const name of SECTION_NAMES) sections[name] = { state, code };
} else {
const reader = options.reader;
const [sources, sync, processing, model, projection] = await Promise.all([
const [sources, sync, processing, model, memories] = await Promise.all([
reader<unknown>("/api/v1/sources"),
reader<unknown>("/api/v1/live-sync"),
reader<unknown>("/api/v1/memory-processing"),
reader<unknown>("/api/v1/model-profiles"),
reader<unknown>("/api/v2/home-projection"),
reader<unknown>("/api/v2/home-summary"),
]);
sections.sources = section(sources, validateSources, describeSources);
sections.sync = section(sync, validateLiveSync, describeSync);
@@ -231,7 +228,11 @@ export async function composeStatusReport(options: {
pipeline =
processing.state === "ok" ? describePipeline(processing.data) : null;
sections.model = section(model, validateModelProfiles, describeModel);
sections.memories = await memorySection(reader, projection, pipeline);
sections.memories = section(
memories,
validateHomeSummary,
describeMemories,
);
}
// A bridge we could not inspect is unknown, not broken and not fine.
const bridge: StatusDocument["bridge"] =
@@ -311,9 +312,7 @@ export function renderStatusText(
];
for (const name of SECTION_NAMES) {
lines.push(
`${sectionLabel(name, report.sections[name])}:${sectionLine(
report.sections[name],
)}`,
`${SECTION_LABELS[name]}:${sectionLine(report.sections[name])}`,
);
}
if (report.pipeline) lines.push(`记忆模式:${pipelineLine(report.pipeline)}`);
@@ -323,8 +322,6 @@ export function renderStatusText(
if (embeddingSwitch !== null) lines.push(`向量模型:${embeddingSwitch}`);
const backoff = modelBackoffLine(report.sections.processing);
if (backoff !== null) lines.push(backoff);
const drain = writeDrainLine(report.sections.processing);
if (drain !== null) lines.push(drain);
if (report.summary.state === "partial") {
const notes = [
report.summary.degraded.length > 0
@@ -355,20 +352,6 @@ function modelBackoffLine(section: Section | undefined): string | null {
}`;
}
/** The durable write drain, when it is not simply running: it exited on an
* error and is coming back (`restarting`), or it is not running at all. */
function writeDrainLine(section: Section | undefined): string | null {
if (section?.state !== "ok") return null;
const state = section.write_drain;
if (state !== "restarting" && state !== "stopped") return null;
const error = section.write_drain_error;
const reason =
typeof error === "string" && error.length > 0 ? `(${error})` : "";
return state === "restarting"
? `写入队列:异常退出${reason},正在自动重启`
: `写入队列:已停止${reason}`;
}
export function pipelineLine(pipeline: PipelineSummary): string {
const parts: string[] = [pipeline.mode];
if (pipeline.forced_by_env)
@@ -426,19 +409,8 @@ const SECTION_LINE_OMITTED = new Set([
"embedding_switch",
"llm_backoff_until",
"llm_backoff_error",
"write_drain",
"write_drain_error",
"content_plane",
"visible_count",
"effective_mode",
]);
function sectionLabel(name: string, section: Section | undefined): string {
return name === "memories" && section?.content_plane === "raw"
? "原始对话"
: SECTION_LABELS[name];
}
function sectionLine(section: Section | undefined): string {
if (!section) return "未检查";
if (section.state === "not_checked") return "未检查";
@@ -490,10 +462,6 @@ function fieldLabel(key: string): string {
profiles: "配置",
active: "当前",
memory_count: "记忆",
raw_count: "原始对话",
visible_count: "可见",
content_plane: "内容层",
effective_mode: "模式",
};
return labels[key] ?? key;
}
@@ -567,7 +535,6 @@ function describeProcessing(
})),
suspended: typeof summary.suspended === "number" ? summary.suspended : 0,
...llmBackoffFields(summary),
...writeDrainFields(summary),
...embeddingSwitchField(summary),
};
}
@@ -588,24 +555,6 @@ function llmBackoffFields(summary: MemoryProcessingSummary): {
};
}
/** Durable write drain liveness. Null when Core does not report it (older
* Core, or recovery has not started the drain yet). */
function writeDrainFields(summary: MemoryProcessingSummary): {
write_drain: "running" | "restarting" | "stopped" | null;
write_drain_error: string | null;
} {
const state = summary.write_drain;
const error = summary.write_drain_error;
return {
write_drain:
state === "running" || state === "restarting" || state === "stopped"
? state
: null,
write_drain_error:
typeof error === "string" && /^[a-z_]{1,64}$/.test(error) ? error : null,
};
}
/** The running switch or merge, projected; absent when none runs. */
function embeddingSwitchField(
summary: MemoryProcessingSummary,
@@ -646,70 +595,6 @@ function describeMemories(summary: SdkHomeSummary): Record<string, unknown> {
};
}
async function memorySection(
reader: CoreReader,
projection: ReadOutcome<unknown>,
pipeline: PipelineSummary | null,
): Promise<Section> {
if (projection.state === "ok") {
const parsed = validateHomeProjection(projection.data);
return parsed === null
? { state: "invalid", code: "response_invalid" }
: { state: "ok", ...describeProjection(parsed) };
}
if (projection.state !== "not_found") {
return {
state: projection.state === "denied" ? "denied" : "unavailable",
code: projection.code,
};
}
if (pipeline === null) {
return { state: "unavailable", code: "effective_mode_unavailable" };
}
if (pipeline.mode === "lite") {
const raw = await reader<unknown>("/api/v2/raw", {
limit: 1,
offset: 0,
});
if (raw.state !== "ok") {
return {
state: raw.state === "denied" ? "denied" : "unavailable",
code: raw.code,
};
}
const total = validateRawTotal(raw.data);
return total === null
? { state: "invalid", code: "response_invalid" }
: {
state: "ok",
content_plane: "raw",
visible_count: total,
effective_mode: "lite",
raw_count: total,
};
}
const legacy = await reader<unknown>("/api/v2/home-summary");
return section(legacy, validateHomeSummary, (summary) => ({
content_plane: "memory",
visible_count: summary.memory_count,
effective_mode: pipeline.mode,
...describeMemories(summary),
}));
}
function describeProjection(
projection: HomeProjectionV1,
): Record<string, unknown> {
return {
content_plane: projection.capabilities.content_plane,
visible_count: projection.visible_count,
effective_mode: projection.effective_mode,
...(projection.capabilities.content_plane === "raw"
? { raw_count: projection.visible_count }
: { memory_count: projection.visible_count }),
};
}
function sum(
sources: SourceResponse[],
key: "imported_conversations" | "imported_messages",
+1 -1
View File
@@ -12,7 +12,7 @@ dependencies = [
"fastapi>=0.141.1",
"filelock>=3.32.4",
# Public SDK version validated against MemHub's permission and recall contracts.
"hy-memory==2.0.14",
"hy-memory==2.0.13",
# chromadb arrives through the HY SDK; locked here so an unrelated upgrade
# cannot move the SDK's store. Built-in embedding consumes numpy directly.
"chromadb==1.5.9",
File diff suppressed because it is too large. Load diff
@@ -20,15 +20,6 @@ SOURCE_READER_WORKERS = 4
SDK_WRITE_WORKERS = 12
# Recall's SDK reads: one search and the `get`s behind it (`_SDK_GET_CONCURRENCY`).
SDK_READ_WORKERS = 8
# The durable write queue's own bookkeeping: claim, settle, the per-job reads
# around an SDK write, and the drain's maintenance passes. Each is one short
# statement on the single writer or a reader; the drain runs at most
# `max_concurrent_sources` jobs, and they wait for the writer, not for CPU.
DURABLE_QUEUE_WORKERS = 4
# Lease renewal alone. Renewal is the drain's ownership fence: a lease that
# lapses while its SDK write is still running hands the job to recovery, so it
# must never wait for a thread behind any other work, its own queue's included.
LEASE_RENEWAL_WORKERS = 2
_pool: ThreadPoolExecutor | None = None
_snapshot_pool: ThreadPoolExecutor | None = None
@@ -37,8 +28,6 @@ _raw_read_pool: ThreadPoolExecutor | None = None
_memory_read_pool: ThreadPoolExecutor | None = None
_source_reader_pool: ThreadPoolExecutor | None = None
_sdk_write_pool: ThreadPoolExecutor | None = None
_durable_queue_pool: ThreadPoolExecutor | None = None
_lease_renewal_pool: ThreadPoolExecutor | None = None
_warmed_readers: weakref.WeakSet[ThreadPoolExecutor] = weakref.WeakSet()
_reader_warm_lock = threading.Lock()
_interactive_reads = 0
@@ -120,35 +109,6 @@ def sdk_write_pool() -> ThreadPoolExecutor:
return _sdk_write_pool
def durable_queue_pool() -> ThreadPoolExecutor:
"""The durable write queue's database work, apart from bulk work.
The queue used to hand its claims, settles and maintenance to the bulk
pool, whose three threads also carry import heartbeats and job listings.
Under an import peak all three sat in those for tens of seconds, and the
queue's next statement waited for a thread before it could even join the
writer's line, and leases lapsed while their SDK writes were still running.
"""
global _durable_queue_pool
if _durable_queue_pool is None:
_durable_queue_pool = ThreadPoolExecutor(
max_workers=DURABLE_QUEUE_WORKERS,
thread_name_prefix="one-memory-write-queue",
)
return _durable_queue_pool
def lease_renewal_pool() -> ThreadPoolExecutor:
"""Threads that only renew durable write leases (`DurableWriteDrain.renew`)."""
global _lease_renewal_pool
if _lease_renewal_pool is None:
_lease_renewal_pool = ThreadPoolExecutor(
max_workers=LEASE_RENEWAL_WORKERS,
thread_name_prefix="one-memory-lease-renewal",
)
return _lease_renewal_pool
def source_reader_pool() -> ThreadPoolExecutor:
"""Bounded process-wide pool for source iteration and parsing."""
global _source_reader_pool
@@ -168,21 +128,9 @@ def warm_bulk_pool() -> None:
Starting a thread is nothing on an idle machine and 2.5 seconds on a busy
one, which is precisely when bulk work is being handed over. So it is
done here instead, at startup, before there is anything to be late for.
The durable write queue's pools are warmed here too, for the same reason:
their first renewal must not pay for starting a thread.
"""
_start_every_worker(bulk_pool(), BULK_WORKERS, "bulk_pool_warmup_timeout")
_start_every_worker(
durable_queue_pool(), DURABLE_QUEUE_WORKERS, "durable_queue_pool_warmup_timeout"
)
_start_every_worker(
lease_renewal_pool(), LEASE_RENEWAL_WORKERS, "lease_renewal_pool_warmup_timeout"
)
def _start_every_worker(pool: ThreadPoolExecutor, workers: int, timeout_code: str) -> None:
started = tuple(threading.Event() for _ in range(workers))
pool = bulk_pool()
started = tuple(threading.Event() for _ in range(BULK_WORKERS))
release = threading.Event()
def occupy(marker: threading.Event) -> None:
@@ -198,7 +146,7 @@ def _start_every_worker(pool: ThreadPoolExecutor, workers: int, timeout_code: st
try:
for marker in started:
if not marker.wait(10):
raise RuntimeError(timeout_code)
raise RuntimeError("bulk_pool_warmup_timeout")
finally:
release.set()
for future in futures:
@@ -280,38 +228,6 @@ async def in_sdk_write[T](
)
async def in_the_write_queue[T](
work: Callable[..., T],
/,
*args: object,
**kwargs: object,
) -> T:
"""One durable write queue statement, in the queue's pool (`durable_queue_pool`).
It still goes through the one writer and its line, like every write; what
it no longer does is wait for a bulk thread first.
"""
return await asyncio.get_running_loop().run_in_executor(
durable_queue_pool(),
copy_context().run,
partial(work, *args, **kwargs),
)
async def in_lease_renewal[T](
work: Callable[..., T],
/,
*args: object,
**kwargs: object,
) -> T:
"""One lease renewal, in the threads kept for nothing else (`lease_renewal_pool`)."""
return await asyncio.get_running_loop().run_in_executor(
lease_renewal_pool(),
copy_context().run,
partial(work, *args, **kwargs),
)
def _retrieve[T](future: "asyncio.Future[T]") -> None:
"""Mark a shielded read's outcome as seen.
@@ -398,13 +314,7 @@ def _warm_reader_pool(reader: ThreadPoolExecutor) -> None:
def shutdown_bulk_pool() -> None:
global _pool, _snapshot_pool, _interactive_pool, _raw_read_pool, _memory_read_pool
global _source_reader_pool, _sdk_write_pool, _durable_queue_pool, _lease_renewal_pool
if _durable_queue_pool is not None:
_durable_queue_pool.shutdown(wait=False, cancel_futures=True)
_durable_queue_pool = None
if _lease_renewal_pool is not None:
_lease_renewal_pool.shutdown(wait=False, cancel_futures=True)
_lease_renewal_pool = None
global _source_reader_pool, _sdk_write_pool
if _sdk_write_pool is not None:
_sdk_write_pool.shutdown(wait=False, cancel_futures=True)
_sdk_write_pool = None
@@ -1,133 +0,0 @@
"""The hook side of live capture: hand the turn to the running Core, nothing more.
A host runs `one-memory capture <source>` as a short-lived process on every
hook, often several at once. That process used to open the main database
itself and write Evidence and the write queue from outside Core's single
writer. It now never opens the database: it reads the hook's JSON, posts it to
the running Core over loopback with the runtime token, and exits. Core applies
every gate and stages the turn in-process (`live_capture.capture_hook`).
Anything that goes wrong — Core not running, a stale receipt, a slow answer —
drops the turn silently. There is no local fallback write: file-backed sources
are read again by live sync from their persisted cursor once Core is back.
"""
import json
import os
from pathlib import Path
from typing import TextIO
CAPTURE_SOURCES = frozenset(
{
"workbuddy",
"claude-code",
"codex",
"cursor",
"pi",
"qwen-code",
"qwenwork",
"hermes",
"deepseek-harness",
"opencode",
"openclaw",
}
)
SDK_MIGRATION_CAPTURE_FENCE = ".sdk-migration-capture-fence.json"
#: Core's in-process intake for a hook's payload; see `live_capture`.
CAPTURE_ROUTE = "/api/v1/live-capture/{source_id}"
#: Whole request, connect to answer. Pi and OpenCode wait for the hook
#: synchronously (3 s, interpreter start included), so this stays well under.
HANDOFF_TIMEOUT_SECONDS = 1.5
#: A hook carries one turn, or one session's visible text at most.
MAX_PAYLOAD_BYTES = 8 * 1024 * 1024
def data_directory() -> Path:
from one_memory.config import load_settings
return load_settings().data_dir
def capture_stdio(source_id: str, stdin: TextIO, output: TextIO) -> int:
"""Host hooks must stay silent. A bad payload or an absent Core does not fail the turn."""
del output
if source_id not in CAPTURE_SOURCES:
return 2
try:
data_dir = data_directory()
if sdk_migration_capture_fenced(data_dir):
return 0
payload = json.loads(stdin.read() or "")
except Exception:
return 0
if not isinstance(payload, dict):
return 0
try:
hand_to_core(source_id, payload, data_dir)
except Exception:
return 0
return 0
def hand_to_core(
source_id: str,
payload: dict[str, object],
data_dir: Path,
*,
timeout: float = HANDOFF_TIMEOUT_SECONDS,
) -> bool:
"""Post the payload to the Core named by this home's receipt. True when Core took it.
Only the receipt, the runtime token and a loopback socket are touched; the
main database is never opened from here.
"""
from one_memory.runtime_client import core_endpoint
endpoint = core_endpoint(data_dir)
if endpoint is None:
return False
body = json.dumps(payload, ensure_ascii=False).encode("utf-8")
if len(body) > MAX_PAYLOAD_BYTES:
return False
from one_memory.http_client import create_client
with create_client(timeout=timeout) as client:
response = client.post(
f"http://{endpoint.host}:{endpoint.port}{CAPTURE_ROUTE.format(source_id=source_id)}",
content=body,
headers={
"Authorization": f"Bearer {endpoint.token}",
"Content-Type": "application/json",
},
)
return response.status_code == 202
def sdk_migration_capture_fenced(data_dir: Path) -> bool:
"""Ignore stale fences; a live migration or a failed one pauses hook writes.
A failed upgrade migration leaves the fence `frozen` (no pid): the home was
put back as it was before the upgrade and takes no writes until the next
migration attempt, which replaces the fence and lifts it on success.
"""
marker = data_dir / SDK_MIGRATION_CAPTURE_FENCE
try:
document = json.loads(marker.read_text(encoding="utf-8"))
if isinstance(document, dict) and document.get("frozen") is True:
return True
pid = int(document["pid"])
if pid < 1:
raise ValueError("invalid migration fence pid")
os.kill(pid, 0)
return True
except FileNotFoundError:
return False
except ProcessLookupError:
marker.unlink(missing_ok=True)
return False
except PermissionError:
return True
except (KeyError, TypeError, ValueError, json.JSONDecodeError, OSError):
marker.unlink(missing_ok=True)
return False
+1 -1
View File
@@ -119,7 +119,7 @@ def run(arguments: Sequence[str], *, output: TextIO | None = None) -> int:
if command == "sdk-migration-status":
return sdk_migration_status(options, stream)
if command == "capture" and len(options) == 1:
from one_memory.capture_handoff import capture_stdio
from one_memory.live_capture import capture_stdio
return capture_stdio(options[0], sys.stdin, stream)
if command not in {"status", "doctor"} or options not in ([], ["--json"]):
@@ -11,14 +11,14 @@ from one_memory.context_models import ContextSearchRequest
from one_memory.external_memory_access import ExternalMemoryAccess
from one_memory.memory_server.store_binding import store_for
#: The connected-source query, per engine.
#: The revoked-source query, per engine.
#:
#: Identical text today because it uses no placeholders and reads by position,
#: which both engines support. Kept as a named constant so that adding a
#: parameter later forces the `?`/`%s` difference to be considered here rather
#: than being introduced silently at the call site.
CONNECTED_SOURCES_SQL = (
"SELECT DISTINCT source_id FROM source_grants WHERE user_id='local' AND revoked_at IS NULL"
REVOKED_SOURCES_SQL = (
"SELECT source_id FROM source_grants WHERE user_id='local' AND revoked_at IS NOT NULL"
)
@@ -49,10 +49,16 @@ def resolve_context_authority(
scope for scope in access.scopes if scope == "all-apps" or scope.startswith("workspace:v1:")
)
memory_allowed = memory_requested and "om_recall" in access.tools and bool(memory_scopes)
raw_allowed = (
raw_requested and "om_conversations" in access.tools and "all-apps" in access.scopes
declared_sources = frozenset(
scope.removeprefix("source:") for scope in access.scopes if scope.startswith("source:")
)
if explicit is not None and not raw_allowed:
raw_allowed = (
raw_requested
and "om_conversations" in access.tools
and "all-apps" in access.scopes
and bool(declared_sources)
)
if explicit is not None and (not raw_allowed or not explicit <= declared_sources):
raise HTTPException(403, "external_context_scope_denied")
raw_sources: frozenset[str] = frozenset()
@@ -64,10 +70,10 @@ def resolve_context_authority(
# the same code runs on both engines without a branch per engine.
if hasattr(db, "set_progress_handler"):
db.set_progress_handler(lambda: int(time.monotonic() >= deadline), 1000)
connected = frozenset(str(row[0]) for row in db.execute(CONNECTED_SOURCES_SQL))
revoked = frozenset(str(row[0]) for row in db.execute(REVOKED_SOURCES_SQL))
if time.monotonic() >= deadline:
raise TimeoutError
raw_sources = connected
raw_sources = declared_sources - revoked
if explicit is not None:
if not explicit <= raw_sources:
raise HTTPException(403, "external_context_scope_denied")
@@ -66,8 +66,6 @@ class SdkContextSearchRequest(ContextSearchRequest, SdkSearchOptions):
limit: int = Field(default=10, ge=1, le=20, strict=True)
schema_limit: int = Field(default=3, ge=0, le=20, strict=True)
intention_limit: int = Field(default=1, ge=0, le=20, strict=True)
#: Product callers can request one item per turn instead of SDK raw batches.
raw_unit: Literal["raw", "turn"] = "raw"
#: Whose memories to search. The App never sends it and reads `local`.
user_id: UserId = DEFAULT_USER_ID
@@ -125,9 +123,6 @@ class SdkRawContextItem(BaseModel):
memory_at: datetime | None = None
score: float = Field(ge=0, le=1)
truncated: bool = False
turn_id: str | None = Field(default=None, min_length=1, max_length=96)
turn_index: int | None = Field(default=None, ge=0)
turn_count: int | None = Field(default=None, ge=1)
class SdkMemoryContextItem(BaseModel):
+23 -131
View File
@@ -1,7 +1,6 @@
"""Authenticated SDK context search route, without generation or reindex side effects."""
import asyncio
import logging
import time
from collections.abc import Sequence
from datetime import UTC, datetime
@@ -31,14 +30,11 @@ from one_memory.hy_sdk_types import (
parse_sdk_search,
)
from one_memory.imports.storage import EvidenceStore
from one_memory.raw_turns import raw_display_turns, turns_for_hit
from one_memory.sdk_search_options import SdkSearchFilters, SearchLane
from one_memory.semantic_deadline import SemanticMemoryHit
from one_memory.user_scope import owns
CALL_EVENT_RESPONSE_BUDGET_MS = 250
SDK_CONTEXT_CONTENT_LIMIT = 20_000
logger = logging.getLogger(__name__)
#: Pass-through filters the raw lane honours. Scene, layers, reader and the
#: schema/working floors describe memory lanes; the raw lane is always the
@@ -74,18 +70,6 @@ def raw_lane_overrides(value: SdkSearchFilters) -> dict[str, object]:
return found
def raw_lane_statuses(*, lite: bool) -> dict[str, list[str]]:
"""Which raw statuses the raw lane searches.
Lite extracts nothing, so its search is plain RAG over every raw,
including the shadow raws pro already processed before the switch to
lite. Pro and ultra keep the SDK default (active only): a shadow raw is
already represented by the memories extracted from it, and returning it
again would repeat that content in the caller's context budget.
"""
return {"statuses": ["active", "shadow"]} if lite else {}
def hit_namespace(hit: SemanticMemoryHit) -> dict[str, str]:
"""The `get` namespace of a hit: nothing for chat, kind and variant otherwise."""
if hit.kind in {"experience", "fact"} and hit.variant is not None:
@@ -93,13 +77,6 @@ def hit_namespace(hit: SemanticMemoryHit) -> dict[str, str]:
return {}
def bounded_context_content(content: str) -> tuple[str, bool]:
"""Fit SDK text into its wire contract before Pydantic validates it."""
if len(content) <= SDK_CONTEXT_CONTENT_LIMIT:
return content, False
return content[:SDK_CONTEXT_CONTENT_LIMIT], True
def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
"""SDK-native context: no Evidence/Governance ids or fabricated revisions."""
router = APIRouter(prefix="/api/v2/context", dependencies=dependencies)
@@ -143,9 +120,6 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
sources = raw_lane_sources(authority.raw_sources, value.include_sources)
if sources is None:
return ()
# Plain RAG: every vector hit over the floor is an answer, ranked
# by its score. Each hit yields at least one item, so the SDK
# needs to return no more hits than the caller's limit.
raw = await asyncio.to_thread(
client.search,
value.query,
@@ -161,65 +135,27 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
"scene": "chat",
"reader": HY_MEMORY_READER,
"layers": ["l1_raw"],
**raw_lane_statuses(lite=lite),
**raw_lane_overrides(value),
},
)
hits = tuple(
hit for hit in parse_sdk_raw_search(raw) if value.passes_floor("raw", hit.score)
)
# One item per hit first, so a hit is never crowded out by another
# hit's sibling turns; the siblings fill whatever room is left.
primary: list[SdkRawContextItem] = []
siblings: list[SdkRawContextItem] = []
for hit in hits:
memory_at = (
datetime.fromtimestamp(hit.memory_at, tz=UTC)
if hit.memory_at is not None
else None
return tuple(
SdkRawContextItem(
raw_id=hit.raw_id,
content=hit.content,
source=hit.source,
session_id=hit.session_id,
memory_at=(
datetime.fromtimestamp(hit.memory_at, tz=UTC)
if hit.memory_at is not None
else None
),
score=hit.score,
)
turns = raw_display_turns(hit) if value.raw_unit == "turn" else ()
if not turns:
content, truncated = bounded_context_content(hit.content)
primary.append(
SdkRawContextItem(
raw_id=hit.raw_id,
content=content,
source=hit.source,
session_id=hit.session_id,
memory_at=memory_at,
score=hit.score,
truncated=truncated,
)
)
continue
# The embedding is the batch's: the hit is shown by the turns
# that contain the query, or else by its closest turn, each
# with the hit's score.
shown: list[SdkRawContextItem] = []
for turn in turns_for_hit(turns, value.query):
content, truncated = bounded_context_content(turn.content)
shown.append(
SdkRawContextItem(
raw_id=hit.raw_id,
turn_id=turn.turn_id,
turn_index=turn.turn_index,
turn_count=turn.turn_count,
content=content,
source=hit.source,
session_id=hit.session_id,
memory_at=memory_at,
score=hit.score,
truncated=truncated,
)
)
primary.append(shown[0])
siblings.extend(shown[1:])
chosen = primary[: value.limit]
chosen.extend(siblings[: value.limit - len(chosen)])
# Stable: a hit's turns stay together, in hit-score order.
chosen.sort(key=lambda item: item.score, reverse=True)
return tuple(chosen)
for hit in hits
)
async def memory_lane() -> tuple[SdkMemoryContextItem, ...]:
# lite extracts nothing: its search answers raw turns, which the
@@ -265,15 +201,16 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
or not owns(memory.user_id, value.user_id)
):
continue
content, truncated = bounded_context_content(
hit.formatted_content
if hit.sdk_formatted and hit.formatting_complete and hit.formatted_content
else memory.content
)
items.append(
SdkMemoryContextItem(
memory_id=memory.memory_id,
content=content,
content=(
hit.formatted_content
if hit.sdk_formatted
and hit.formatting_complete
and hit.formatted_content
else memory.content
),
layer=memory.layer,
status=memory.status,
source=memory.source,
@@ -283,7 +220,6 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
else None
),
score=hit.score,
truncated=truncated,
)
)
return tuple(items)
@@ -333,45 +269,10 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
if isinstance(lane, EmbeddingUnavailableError):
# No lexical fallback: the query could not be embedded.
raise HTTPException(503, EMBEDDING_UNAVAILABLE)
requested_lanes = {
"raw": raw_items,
"memory": memory_items,
}
failed_lanes = {
name: lane
for name, lane in requested_lanes.items()
if name in value.kinds and isinstance(lane, Exception)
}
for name, error in failed_lanes.items():
logger.warning(
"sdk_context_lane_failed lane=%s error_class=%s",
name,
type(error).__name__,
)
all_requested_failed = len(failed_lanes) == len(value.kinds)
partial = bool(failed_lanes)
partial = isinstance(raw_items, Exception) or isinstance(memory_items, Exception)
raw_candidates = (
() if isinstance(raw_items, Exception) else cast(tuple[SdkContextItem, ...], raw_items)
)
if access is not None and raw_candidates:
try:
current = await asyncio.to_thread(
resolve_context_authority,
value,
access,
database,
deadline=started + value.timeout_ms / 1000,
)
except HTTPException:
raise
except Exception:
raise HTTPException(503, "context_authority_unavailable") from None
current_sources = current.raw_sources or frozenset()
raw_candidates = tuple(
item
for item in raw_candidates
if item.kind == "raw" and item.source in current_sources
)
memory_candidates = (
()
if isinstance(memory_items, Exception)
@@ -397,20 +298,11 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
if len(content) > remaining:
item = item.model_copy(update={"content": content[:remaining], "truncated": True})
truncated = True
truncated = truncated or item.truncated
selected.append(item)
used += len(item.content)
final = SdkContextSearchResponse(
request_id=str(uuid4()),
state=(
"unavailable"
if all_requested_failed
else "partial"
if partial
else "grounded"
if selected
else "empty"
),
state=("partial" if partial else "grounded" if selected else "empty"),
items=tuple(selected),
used_characters=used,
truncated=truncated,
@@ -242,75 +242,6 @@ class _InTurn:
self._line.remove(place)
class _Checkpointer:
"""The one thread that runs cadence checkpoints, for every main database.
A database asks for at most one checkpoint at a time: a request made
while one is already waiting is the same request. The checkpoint goes
through the database's own writer, in its line, like any other write.
A waiting request holds the database weakly: it must not keep alive a
database nobody else holds any more.
"""
def __init__(self) -> None:
self._state = threading.Condition()
self._due: OrderedDict[int, weakref.ref[MainDatabase]] = OrderedDict()
self._running = 0
self._thread: threading.Thread | None = None
def request(self, database: "MainDatabase") -> None:
with self._state:
self._due.setdefault(id(database), weakref.ref(database))
if self._thread is None or not self._thread.is_alive():
self._thread = threading.Thread(
target=self._run,
name="one-memory-checkpoint",
daemon=True,
)
self._thread.start()
self._state.notify_all()
def forget(self, database: "MainDatabase") -> None:
with self._state:
self._due.pop(id(database), None)
self._state.notify_all()
def wait_idle(self, timeout: float) -> bool:
"""Whether every requested checkpoint has run within `timeout` seconds."""
with self._state:
return self._state.wait_for(lambda: not self._due and not self._running, timeout)
def _run(self) -> None:
while True:
with self._state:
self._state.wait_for(lambda: bool(self._due))
_key, reference = self._due.popitem(last=False)
database = reference()
if database is None:
self._state.notify_all()
continue
self._running += 1
try:
database._cadence_checkpoint()
except Exception: # noqa: BLE001 - maintenance must not end the thread
logger.warning(
"main_database_checkpoint_failed file=%s", database.path.name, exc_info=True
)
finally:
del database
with self._state:
self._running -= 1
self._state.notify_all()
_checkpointer = _Checkpointer()
def wait_for_cadence_checkpoints(timeout: float = 5.0) -> bool:
"""Wait for the cadence checkpoints already requested; for tests and shutdown."""
return _checkpointer.wait_idle(timeout)
def _locked() -> sqlite3.OperationalError:
# Callers tell a busy database from a broken one by these, as they would
# for SQLite's own refusal.
@@ -683,23 +614,13 @@ class MainDatabase:
return None
def _maybe_checkpoint_after_write(self) -> None:
"""Count a write, and hand a due cadence checkpoint to the checkpointer.
"""Checkpoint on a write cadence without opening another connection.
SQLite's default autocheckpoint only runs when the writer itself
crosses the page threshold. Production hit a state where the WAL grew
to 13GB, so the main database needs its own bounded, observable
maintenance path: every `checkpoint_write_interval()` writes or
`MAIN_DATABASE_CHECKPOINT_SECONDS`, whichever comes first.
The checkpoint itself used to run right here, on the thread that had
just written, as one more turn in the writer's line. That thread was
an import heartbeat or a durable lease renewal as often as not, and
it then waited behind the whole line a second time before it could
return. The idle TRUNCATE (#278) cannot replace the cadence: under a
sustained import the database is never idle, and that is exactly when
the WAL grows. So the cadence stays, and one checkpointer thread runs
it (`_Checkpointer`); the writer stays the only connection that
writes, and no caller waits for maintenance it did not ask for.
maintenance path. This runs after the caller's checkout has been reset
and only re-enters the same serialized writer queue.
"""
now = time.monotonic()
with self._state:
@@ -711,12 +632,8 @@ class MainDatabase:
if due:
self._writes_since_checkpoint = 0
self._last_checkpoint_at = now
if due:
_checkpointer.request(self)
def _cadence_checkpoint(self) -> None:
"""One PASSIVE checkpoint through the writer, and the WAL alert after it."""
now = time.monotonic()
if not due:
return
try:
busy, log_frames, checkpointed = self._checkpoint_from_maintenance("PASSIVE")
except sqlite3.Error:
@@ -790,8 +707,6 @@ class MainDatabase:
def close(self) -> None:
"""Close every idle connection now; ones in use close when handed back."""
# A checkpoint still waiting would open the writer again after this.
_checkpointer.forget(self)
with self._state:
self._generation += 1
self._identity = None
@@ -14,7 +14,7 @@ from pathlib import Path
from typing import Literal, cast
HY_MEMORY_DISTRIBUTION = "hy-memory"
HY_MEMORY_VERSION = "2.0.14"
HY_MEMORY_VERSION = "2.0.13"
HY_MEMORY_NAMESPACE = "governed-v2"
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
@@ -503,7 +503,7 @@ def _ensure_coding_schema(path: Path) -> None:
CodingMemoryStore,
)
# Relies on the pinned SDK's internals (hy-memory==2.0.14 is pinned exactly).
# Relies on the pinned SDK's internals (hy-memory==2.0.13 is pinned exactly).
store = CodingMemoryStore.__new__(CodingMemoryStore)
store._db_path = str(path)
store._conn = None
@@ -376,9 +376,8 @@ def apply_sdk_language(config: MemoryConfig, language: str | None) -> MemoryConf
"""Set the SDK prompt language (`MemoryConfig.language`), when there is one.
None (no language set) sets nothing: the SDK detects zh/en as it always
did. Every display language, zh-Hans included, is set explicitly. Like a
feature, the language never touches `vector_store`, so the collection
identity never depends on it.
did. Every display language, zh-Hans included, is set explicitly. Like a feature, the language never touches `vector_store`, so
the collection identity never depends on it.
"""
if language is not None:
config.language = language
@@ -75,9 +75,6 @@ class HySdkMemoryRecord(BaseModel):
# Epoch seconds. `updated_at` falls back to `created_at`.
created_at: int | None = None
updated_at: int | None = None
# Actual scene=all ordering timestamp before the public fallback above.
# Internal-only: callers keep the existing SdkMemoryRecord wire shape.
sort_updated_at: int | None = Field(default=None, exclude=True, repr=False)
@classmethod
def from_sdk(cls, raw: object) -> HySdkMemoryRecord:
@@ -98,8 +95,6 @@ class HySdkMemoryRecord(BaseModel):
if created_at is None:
created_at = gmt_created
updated_at = _epoch(raw.get("updated_at"))
modified_at = _epoch(raw.get("gmt_modified"))
sort_updated_at = modified_at if modified_at is not None else updated_at
return cls(
memory_id=str(raw["memory_id"]),
content=str(raw["content"]),
@@ -121,7 +116,6 @@ class HySdkMemoryRecord(BaseModel):
gmt_created=gmt_created,
created_at=created_at,
updated_at=updated_at if updated_at is not None else created_at,
sort_updated_at=sort_updated_at,
)
except (KeyError, TypeError, ValueError) as error:
raise HySdkReadError("hy_memory_record_shape_unknown") from error
@@ -186,7 +180,6 @@ class HySdkMemoryRecord(BaseModel):
solution=solution,
created_at=created_at,
updated_at=updated_at if updated_at is not None else created_at,
sort_updated_at=updated_at,
)
except (KeyError, TypeError, ValueError) as error:
raise HySdkReadError("hy_memory_record_shape_unknown") from error
@@ -1,6 +1,6 @@
import hashlib
import os
from collections.abc import AsyncIterator, Callable, Iterator
from collections.abc import AsyncIterator, Iterator
from dataclasses import dataclass
from pathlib import Path
@@ -28,7 +28,6 @@ def discovered_off_loop(
suffixes: frozenset[str],
max_targets: int,
excluded_directory_names: frozenset[str] = frozenset(),
directory_predicate: Callable[[Path], bool] | None = None,
) -> AsyncIterator[SourceFileDiscoveryEvent]:
"""The same walk, for callers that have an event loop to keep free.
@@ -44,7 +43,6 @@ def discovered_off_loop(
suffixes=suffixes,
max_targets=max_targets,
excluded_directory_names=excluded_directory_names,
directory_predicate=directory_predicate,
),
scope_key=grant.source_id,
)
@@ -56,7 +54,6 @@ def discover_source_files(
suffixes: frozenset[str],
max_targets: int,
excluded_directory_names: frozenset[str] = frozenset(),
directory_predicate: Callable[[Path], bool] | None = None,
) -> Iterator[SourceFileDiscoveryEvent]:
"""Metadata only: one lstat per candidate, no open and no per-file grant lookup.
@@ -74,16 +71,10 @@ def discover_source_files(
grant.root,
followlinks=False,
):
current = Path(directory)
child_directories[:] = sorted(
child
for child in child_directories
if not (current / child).is_symlink()
and child not in excluded_directory_names
and (
directory_predicate is None
or directory_predicate((current / child).relative_to(grant.root))
)
if not (Path(directory) / child).is_symlink() and child not in excluded_directory_names
)
for filename in sorted(filenames):
if Path(filename).suffix.lower() not in suffixes:
+8 -188
View File
@@ -1,7 +1,6 @@
import logging
import re
import sqlite3
from collections.abc import Iterable
from contextlib import AbstractContextManager, nullcontext
from datetime import UTC, datetime
from enum import StrEnum
@@ -19,27 +18,13 @@ from one_memory.imports.adapters import ScanPhase, SourceCursor
#: the drift would only surface on a fresh database.
#:
#: This runs on SQLite today, so the SQLite rendering is used.
from one_memory.memory_server.migrations import (
import_job_lookup_indexes_ddl,
import_jobs_ddl,
)
from one_memory.memory_server.migrations import import_jobs_ddl
logger = logging.getLogger("one_memory.imports.jobs")
JOB_SCHEMA = import_jobs_ddl("sqlite")
#: Migration 63, applied by the store as well so a running home gets it.
JOB_LOOKUP_INDEXES = import_job_lookup_indexes_ddl("sqlite")
MAX_LIVE_BACKLOG_BYTES = 256 * 1024
#: Finished live syncs kept per (user, source). Live sync opens a job for every
#: change it imports, and nothing removed them: hundreds a day on an active
#: machine. The App shows the latest job per source and the active ones; twenty is
#: that plus enough recent history to read a failure back.
SYNC_JOBS_KEPT_PER_SOURCE = 20
#: Rows one pruning transaction may delete, so it holds the single writer for
#: a few milliseconds at most however large the table grew.
PRUNE_BATCH_ROWS = 200
class ImportStatus(StrEnum):
PENDING = "pending"
@@ -51,12 +36,6 @@ class ImportStatus(StrEnum):
CANCELLED = "cancelled"
#: States no job ever leaves (`ImportJobStore._TRANSITIONS`). Failed and
#: interrupted are not among them: both can be resumed, and both still hold the
#: source's one active slot.
FINISHED_STATUSES = (ImportStatus.COMPLETED, ImportStatus.CANCELLED)
class ImportJob(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
@@ -293,9 +272,6 @@ class ImportJobStore:
WHERE status IN ('pending', 'running', 'paused', 'interrupted', 'failed')
"""
)
for statement in JOB_LOOKUP_INDEXES.split(";"):
if statement.strip():
connection.execute(statement)
def create(
self,
@@ -358,8 +334,8 @@ class ImportJobStore:
read again.
"""
opened: list[ImportJob] = []
for job in self.find_jobs(operation="history", statuses=(ImportStatus.FAILED,)):
if job.stored != 0:
for job in self.list_jobs(status=ImportStatus.FAILED):
if job.operation != "history" or job.stored != 0:
continue
reasons = {issue.reason for issue in self.list_issues(job.job_id)}
if not reasons or not reasons <= self._UNPUBLISHED_SNAPSHOT_FAILURES:
@@ -823,172 +799,16 @@ class ImportJobStore:
return self.require(job_id)
def list_jobs(self, *, status: ImportStatus | None = None) -> list[ImportJob]:
"""Every job, oldest first. Hot paths ask `find_jobs` a narrower question."""
return self.find_jobs(statuses=(status,) if status is not None else None)
def find_jobs(
self,
*,
user_id: str | None = None,
source_id: str | None = None,
operation: Literal["history", "sync"] | None = None,
statuses: Iterable[ImportStatus] | None = None,
newest_first: bool = False,
limit: int | None = None,
) -> list[ImportJob]:
"""The jobs matching every given filter, oldest first unless asked otherwise.
The filters are SQL, served by `import_jobs_lane` / `import_jobs_status`
(migration 63). Reading the whole table and filtering here was what
three bulk threads and three request threads were all inside, for tens
of seconds, while the loop waited on them.
"""
clauses: list[str] = []
parameters: list[str | int] = []
if user_id is not None:
clauses.append("user_id = ?")
parameters.append(user_id)
if source_id is not None:
clauses.append("source_id = ?")
parameters.append(source_id)
if operation is not None:
clauses.append("operation = ?")
parameters.append(operation)
if statuses is not None:
wanted = tuple(dict.fromkeys(ImportStatus(status) for status in statuses))
if not wanted:
return []
clauses.append(f"status IN ({', '.join('?' for _ in wanted)})")
parameters.extend(wanted)
query = "SELECT * FROM import_jobs"
if clauses:
query += " WHERE " + " AND ".join(clauses)
query += (
" ORDER BY created_at DESC, job_id DESC"
if newest_first
else " ORDER BY created_at, job_id"
)
if limit is not None:
if limit < 1:
raise ValueError("limit must be positive")
query += " LIMIT ?"
parameters.append(limit)
parameters: tuple[str, ...] = ()
if status is not None:
query += " WHERE status = ?"
parameters = (status,)
query += " ORDER BY created_at, job_id"
with self._connect() as connection:
rows = connection.execute(query, parameters).fetchall()
return [self._to_job(row) for row in rows]
def latest_job(
self,
*,
user_id: str,
source_id: str,
operation: Literal["history", "sync"] | None = None,
statuses: Iterable[ImportStatus] | None = None,
) -> ImportJob | None:
jobs = self.find_jobs(
user_id=user_id,
source_id=source_id,
operation=operation,
statuses=statuses,
newest_first=True,
limit=1,
)
return jobs[0] if jobs else None
def prune_finished_sync_jobs(
self,
*,
keep: int = SYNC_JOBS_KEPT_PER_SOURCE,
max_rows: int = PRUNE_BATCH_ROWS,
) -> int:
"""Delete finished live-sync jobs beyond the newest `keep` of each source.
One bounded transaction: at most `max_rows` jobs, with their issues and
skip counts. Returns how many jobs went; fewer than `max_rows` means
nothing is left to prune, so calling again until then finishes the job.
Only `sync` jobs in a state nothing leaves (completed, cancelled) are
ever deleted. Failed and interrupted jobs hold the source's active slot
and can be resumed; history jobs are named by connection receipts, by
the onboarding gate and by the first-import review, and there are a
handful per source, so they are all kept. Cursors live in their own
tables keyed by file, not by job, so a pruned job takes no position
with it. The newest job of every source is always among those kept,
which is the one the App and live sync health name.
"""
if keep < 1:
raise ValueError("keep must be positive")
if max_rows < 1:
raise ValueError("max_rows must be positive")
finished = tuple(FINISHED_STATUSES)
placeholders = ", ".join("?" for _ in finished)
deleted = 0
with self._connect(writing=True) as connection:
lanes = connection.execute(
f"""
SELECT user_id, source_id, COUNT(*) AS finished
FROM import_jobs
WHERE operation = 'sync' AND status IN ({placeholders})
GROUP BY user_id, source_id
HAVING COUNT(*) > ?
""",
(*finished, keep),
).fetchall()
for lane in lanes:
if deleted >= max_rows:
break
# The oldest job still kept: everything strictly older goes.
boundary = connection.execute(
f"""
SELECT created_at, job_id FROM import_jobs
WHERE user_id = ? AND source_id = ? AND operation = 'sync'
AND status IN ({placeholders})
ORDER BY created_at DESC, job_id DESC
LIMIT 1 OFFSET ?
""",
(lane["user_id"], lane["source_id"], *finished, keep - 1),
).fetchone()
if boundary is None:
continue
victims = [
str(row["job_id"])
for row in connection.execute(
f"""
SELECT job_id FROM import_jobs
WHERE user_id = ? AND source_id = ? AND operation = 'sync'
AND status IN ({placeholders})
AND (created_at < ? OR (created_at = ? AND job_id < ?))
ORDER BY created_at, job_id
LIMIT ?
""",
(
lane["user_id"],
lane["source_id"],
*finished,
boundary["created_at"],
boundary["created_at"],
boundary["job_id"],
max_rows - deleted,
),
)
]
if not victims:
continue
marks = ", ".join("?" for _ in victims)
# Foreign keys cascade these too; deleting them by name keeps
# the outcome the same on a connection that has them off.
connection.execute(f"DELETE FROM import_issues WHERE job_id IN ({marks})", victims)
connection.execute(
f"DELETE FROM import_skip_reason_counts WHERE job_id IN ({marks})", victims
)
removed = connection.execute(
f"DELETE FROM import_jobs WHERE job_id IN ({marks}) "
f"AND operation = 'sync' AND status IN ({placeholders})",
(*victims, *finished),
)
deleted += removed.rowcount
return deleted
def get_source_cursors(
self,
*,
@@ -28,7 +28,7 @@ from one_memory.imports.adapters import (
)
from one_memory.imports.jobs import ImportJob, ImportJobStore, ImportStatus
from one_memory.imports.supervisor import ImportSupervisor
from one_memory.user_scope import DEFAULT_USER_ID, split_lane
from one_memory.user_scope import DEFAULT_USER_ID, lane_key, split_lane
#: How long an automatic retry will wait for a wedged worker to notice cancel.
ABANDON_TIMEOUT_SECONDS = 5.0
@@ -249,15 +249,13 @@ class PollingLiveSyncSupervisor:
),
)
return None
# This lane's unfinished jobs only: the newest unfinished sync, and
# whatever history shares its cursor. Asked in SQL; listing every
# job ever run and filtering here was the poll's biggest read.
source_jobs = await in_the_background(self._lane_jobs, source_id)
source_jobs = await in_the_background(self.job_store.list_jobs)
active_sync = next(
(
job
for job in reversed(source_jobs)
if job.operation == "sync"
if lane_key(job.user_id, job.source_id) == source_id
and job.operation == "sync"
and job.status
in {
ImportStatus.PENDING,
@@ -277,9 +275,7 @@ class PollingLiveSyncSupervisor:
active_sync.job_id
):
attempted_at = datetime.now(UTC)
await self.import_supervisor.enqueue_off_loop(
active_sync.job_id, job=active_sync
)
self.import_supervisor.enqueue(active_sync.job_id)
if not wait_for_completion:
self._observe_sync(source_id, attempted_at, active_sync.job_id)
return active_sync
@@ -299,20 +295,22 @@ class PollingLiveSyncSupervisor:
# by a process restart between yield and claim. Enqueue is
# the operation that actually creates the task.
attempted_at = datetime.now(UTC)
resumed = await self.import_supervisor.resume_off_loop(active_sync.job_id)
await self.import_supervisor.enqueue_off_loop(active_sync.job_id, job=resumed)
self.import_supervisor.resume(active_sync.job_id)
self.import_supervisor.enqueue(active_sync.job_id)
completed = await self.import_supervisor.wait(active_sync.job_id)
self._record_outcome(source_id, attempted_at, completed)
return completed
competing = [
job
for job in source_jobs
if job.operation != "sync"
if lane_key(job.user_id, job.source_id) == source_id
and job.operation != "sync"
and job.status
in {ImportStatus.PENDING, ImportStatus.RUNNING, ImportStatus.INTERRUPTED}
]
bounded_history_incomplete = any(
job.operation == "history"
lane_key(job.user_id, job.source_id) == source_id
and job.operation == "history"
and job.history_after is not None
and job.status is not ImportStatus.COMPLETED
for job in source_jobs
@@ -356,38 +354,6 @@ class PollingLiveSyncSupervisor:
# automatically after the realtime slice completes.
return await self._run_sync(source_id, wait_for_completion=wait_for_completion)
def _lane_jobs(self, source_id: str) -> list[ImportJob]:
"""The lane's jobs `_sync` decides on, oldest first.
Everything not completed, of either operation: an unfinished sync to
pick up, a history that competes for the cursor, and a bounded history
that has not completed yet (which includes a cancelled one). Finished
syncs, the rows that grow without end, are never read.
"""
user_id, bare_source_id = split_lane(source_id)
unfinished = tuple(
status for status in ImportStatus if status is not ImportStatus.COMPLETED
)
return [
*self.job_store.find_jobs(
user_id=user_id,
source_id=bare_source_id,
operation="history",
statuses=unfinished,
),
*self.job_store.find_jobs(
user_id=user_id,
source_id=bare_source_id,
operation="sync",
statuses=(
ImportStatus.PENDING,
ImportStatus.RUNNING,
ImportStatus.PAUSED,
ImportStatus.INTERRUPTED,
),
),
]
async def _has_work(self, source_id: str) -> bool:
"""Ask the source whether a scan would write anything, before taking a job out.
@@ -486,7 +452,7 @@ class PollingLiveSyncSupervisor:
),
)
self._active_jobs[source_id] = job.job_id
await self.import_supervisor.enqueue_off_loop(job.job_id, job=job)
self.import_supervisor.enqueue(job.job_id)
if not wait_for_completion:
self._observe_sync(source_id, attempted_at, job.job_id)
return job
@@ -591,11 +557,7 @@ class PollingLiveSyncSupervisor:
async def _poll_source(self, source_id: str) -> None:
while True:
# A watched source is normally triggered by its filesystem event.
# Its timer is only a missed-event backstop; recursively walking
# every multi-gigabyte host root once a minute starved unrelated
# Core reads even when no source had changed.
await asyncio.sleep(self._poll_interval(source_id))
await asyncio.sleep(self.interval_seconds)
try:
await self.trigger(source_id)
except asyncio.CancelledError:
@@ -612,11 +574,6 @@ class PollingLiveSyncSupervisor:
except Exception as again:
self._note_internal_error(source_id, again)
def _poll_interval(self, source_id: str) -> float:
if source_id in (self._watched_sources or ()):
return max(self.interval_seconds, 15 * 60)
return self.interval_seconds
def _note_worker_still_running(self, source_id: str) -> None:
current = self.health(source_id)
self._set_health(
@@ -23,7 +23,6 @@ from pathlib import Path
from watchfiles import awatch
from one_memory.background_work import in_the_background
from one_memory.diagnostic_log import diagnostic_event
from one_memory.imports.jobs import ImportStatus
from one_memory.imports.live_sync import PollingLiveSyncSupervisor
@@ -91,8 +90,7 @@ class FileWatchLiveSync:
# turn that caused setup. The watcher cannot observe a change
# from before it existed, so immediately run the same bounded
# trigger once instead of waiting for the 60-second backstop.
# The roots are this watcher's own sources: nothing to resolve.
self._queue_sources(tuple(self.roots), count=len(self.roots))
self._queue_changes(set(self.roots.values()))
async def shutdown(self) -> None:
task = self._task
@@ -141,54 +139,28 @@ class FileWatchLiveSync:
observed_at: datetime | None = None,
) -> tuple[str, ...]:
"""Queue affected sources and wait for their coalesced work to settle."""
workers = await self._queue_changes(changed, observed_at=observed_at)
workers = self._queue_changes(changed, observed_at=observed_at)
await asyncio.gather(*(asyncio.shield(worker) for worker in workers.values()))
return tuple(workers)
def sources_for(self, changed: set[Path]) -> tuple[str, ...]:
"""The sources owning `changed`, in path order. Resolves every path."""
return tuple(
dict.fromkeys(
source
for source in (self.source_for(path) for path in sorted(changed))
if source is not None
)
)
async def _queue_changes(
def _queue_changes(
self, changed: set[Path], *, observed_at: datetime | None = None
) -> dict[str, asyncio.Task[None]]:
"""Attribute a batch of changed paths to sources and queue them.
Attributing resolves each path, a `stat` per component, and a busy
host reports hundreds of paths at once. On the loop that was the main
thread inside `source_for` while nothing else was answered, so the
resolving happens off it and only the queueing here.
"""
if self._stop.is_set():
return {}
observed = observed_at or datetime.now(UTC)
sources = await in_the_background(self.sources_for, changed)
return self._queue_sources(sources, count=len(changed), observed_at=observed)
def _queue_sources(
self,
sources: tuple[str, ...],
*,
count: int,
observed_at: datetime | None = None,
) -> dict[str, asyncio.Task[None]]:
if self._stop.is_set():
return {}
observed = observed_at or datetime.now(UTC)
workers: dict[str, asyncio.Task[None]] = {}
for source_id in sources:
for source_id in dict.fromkeys(
source
for source in (self.source_for(path) for path in sorted(changed))
if source is not None
):
diagnostic_event(
level="debug",
component="import.live_watch",
event="live_sync.hook_received",
outcome="queued",
details={"source_id": source_id, "count": count},
details={"source_id": source_id, "count": len(changed)},
)
self._pending[source_id] = max(self._pending.get(source_id, observed), observed)
worker = self._workers.get(source_id)
@@ -267,7 +239,7 @@ class FileWatchLiveSync:
stop_event=self._stop,
ignore_permission_denied=True,
):
await self._queue_changes({Path(path) for _change, path in changes})
self._queue_changes({Path(path) for _change, path in changes})
except asyncio.CancelledError:
raise
except Exception:
@@ -226,9 +226,6 @@ class JsonlSourceAdapter:
self.header_gate = header_gate
self._header_cache: dict[str, tuple[str, dict[str, JsonValue] | None]] = {}
self._history_listing_cache: SourceListing | None = None
self._probe_listing_cache: (
tuple[dict[str, SourceCursor], int | None, SourceListing] | None
) = None
@property
def descriptor(self) -> SourceDescriptor:
@@ -284,11 +281,6 @@ class JsonlSourceAdapter:
predicate=self.file_predicate,
cursors=options.cursors,
)
self._probe_listing_cache = (
dict(options.cursors),
options.max_scan_targets,
listing,
)
return bool(listing.changed or listing.failures)
async def live_baseline(self) -> dict[str, SourceCursor]:
@@ -323,11 +315,9 @@ class JsonlSourceAdapter:
async for event in self._scan_frozen(options):
yield event
return
listing = (
await asyncio.to_thread(self._history_listing, options)
if options.reuse_listing
else self._take_probe_listing(options)
or await asyncio.to_thread(self._fresh_listing, options)
listing = await asyncio.to_thread(
self._history_listing if options.reuse_listing else self._fresh_listing,
options,
)
for failure in listing.failures:
yield ScanFailure(reason_code=discovery_failure_code(failure))
@@ -359,16 +349,6 @@ class JsonlSourceAdapter:
self._history_listing_cache = listing
return listing
def _take_probe_listing(self, options: ScanOptions) -> "SourceListing | None":
cached = self._probe_listing_cache
self._probe_listing_cache = None
if cached is None:
return None
cursors, max_scan_targets, listing = cached
if cursors != options.cursors or max_scan_targets != options.max_scan_targets:
return None
return listing
def _fresh_listing(self, options: ScanOptions) -> "SourceListing":
return list_changed_files(
self.grant,
@@ -9,7 +9,6 @@ from collections.abc import AsyncIterator, Iterator
from dataclasses import dataclass, replace
from datetime import UTC, datetime, timedelta, timezone
from functools import partial
from pathlib import Path
from pydantic import JsonValue
@@ -176,9 +175,6 @@ class CursorUnifiedSourceAdapter:
self.grant = grant
self.source_version = source_version
self._history_listing_cache: _CursorListing | None = None
self._probe_listing_cache: (
tuple[dict[str, SourceCursor], int | None, _CursorListing] | None
) = None
@property
def descriptor(self) -> SourceDescriptor:
@@ -211,15 +207,6 @@ class CursorUnifiedSourceAdapter:
state = await self.install_state()
return min(state.session_count, limit) if limit is not None else state.session_count
async def has_changes(self, options: ScanOptions) -> bool:
listing = await asyncio.to_thread(self._fresh_listing, options)
self._probe_listing_cache = (
dict(options.cursors),
options.max_scan_targets,
listing,
)
return bool(listing.transcripts or listing.stores or listing.legacy or listing.failures)
async def live_baseline(self) -> dict[str, SourceCursor]:
"""Capture Cursor's current tail so the realtime lane never backfills history."""
listing = await asyncio.to_thread(
@@ -235,11 +222,9 @@ class CursorUnifiedSourceAdapter:
async for event in self._scan_frozen(options):
yield event
return
listing = (
await asyncio.to_thread(self._history_listing, options)
if options.reuse_listing
else self._take_probe_listing(options)
or await asyncio.to_thread(self._fresh_listing, options)
listing = await asyncio.to_thread(
self._history_listing if options.reuse_listing else self._fresh_listing,
options,
)
for failure in listing.failures:
yield ScanFailure(reason_code=discovery_failure_code(failure))
@@ -344,16 +329,6 @@ class CursorUnifiedSourceAdapter:
self._history_listing_cache = listing
return listing
def _take_probe_listing(self, options: ScanOptions) -> "_CursorListing | None":
cached = self._probe_listing_cache
self._probe_listing_cache = None
if cached is None:
return None
cursors, max_scan_targets, listing = cached
if cursors != options.cursors or max_scan_targets != options.max_scan_targets:
return None
return listing
def _fresh_listing(self, options: ScanOptions) -> "_CursorListing":
return _list_cursor_files(
self.grant,
@@ -401,7 +376,6 @@ class CursorUnifiedSourceAdapter:
self.grant,
suffixes=frozenset({".db"}),
max_targets=options.max_scan_targets or 100_000,
directory_predicate=_cursor_data_directory,
):
if isinstance(discovered, SourceFileDiscoveryFailure):
yield ScanFailure(reason_code=discovery_failure_code(discovered))
@@ -503,10 +477,7 @@ def _list_cursor_files(
legacy: list[AuthorizedSourceFile] = []
failures: list[SourceFileDiscoveryFailure] = []
for item in discover_source_files(
grant,
suffixes=frozenset({".jsonl", ".db", ".vscdb"}),
max_targets=max_targets,
directory_predicate=_cursor_data_directory,
grant, suffixes=frozenset({".jsonl", ".db", ".vscdb"}), max_targets=max_targets
):
if isinstance(item, SourceFileDiscoveryFailure):
failures.append(item)
@@ -615,25 +586,6 @@ def _is_cursor_data_file(source_file: AuthorizedSourceFile) -> bool:
)
def _cursor_data_directory(relative_path: Path) -> bool:
"""Enter only Cursor directories that can contain conversation authority.
The granted root also contains extensions, skills, caches, logs and agent
artifacts. Recursively walking all of them every live-sync poll made the
reader monopolize Core even though none can pass the file predicates.
"""
parts = relative_path.parts
if not parts:
return False
if parts[0] in {"chats", "acp-sessions"}:
return True
if parts[0] == "projects":
return len(parts) <= 2 or parts[2] == "agent-transcripts"
if parts[0] == "User":
return len(parts) == 1 or parts[1] in {"globalStorage", "workspaceStorage"}
return False
def _is_transcript_file(source_file: AuthorizedSourceFile) -> bool:
parts = source_file.relative_path.parts
return source_file.relative_path.suffix == ".jsonl" and "agent-transcripts" in parts
@@ -682,7 +634,6 @@ def _select_transcript_files(
grant,
suffixes=frozenset({".jsonl"}),
max_targets=100_000,
directory_predicate=_cursor_data_directory,
):
if not isinstance(item, AuthorizedSourceFile) or not _is_transcript_file(item):
continue
@@ -703,7 +654,6 @@ def _subagent_conversation_ids(grant: AuthorizedSourceRoot) -> set[str]:
grant,
suffixes=frozenset({".jsonl"}),
max_targets=100_000,
directory_predicate=_cursor_data_directory,
)
if isinstance(item, AuthorizedSourceFile)
and _is_transcript_file(item)
@@ -10,10 +10,7 @@ from pydantic import JsonValue
from one_memory.input_sanitization import sanitize_model_input
_REMINDER_TAGS = r"additional_data|system[-_]reminder|working_memory_reminder"
_CONTEXT_TAGS = (
r"user_info|rules|agent_transcripts|attached_files|image_local_path|"
r"system[-_]notification|pipeline_log"
)
_CONTEXT_TAGS = r"user_info|rules|agent_transcripts|system[-_]notification|pipeline_log"
_TAG_ATTRIBUTES = r"""(?:\s+[\w:-]+\s*=\s*(?:"[^"<>]*"|'[^'<>]*'))*\s*"""
_HOST_PREFIX = re.compile(rf"<(?P<tag>{_REMINDER_TAGS}|{_CONTEXT_TAGS}){_TAG_ATTRIBUTES}>")
_REMINDER_START = re.compile(rf"<(?P<tag>{_REMINDER_TAGS}){_TAG_ATTRIBUTES}>")
@@ -1,7 +1,6 @@
import asyncio
from collections.abc import Awaitable, Callable, Mapping
from contextlib import suppress
from functools import partial
from time import monotonic
from typing import Literal
@@ -83,10 +82,6 @@ class ImportSupervisor:
)
self._tasks: dict[str, asyncio.Task[None]] = {}
self._drains: dict[str, asyncio.Task[None]] = {}
#: Pause and cancel settle a stopped job with a read and a transition.
#: Made on the loop, two of them could not interleave; made off it,
#: they are serialized here so they still land in the order they woke.
self._settling = asyncio.Lock()
def register_adapter(self, adapter: SourceAdapter, *, user_id: str = "local") -> None:
"""Adapters are keyed by lane: the source id for `local`, `source@user`
@@ -121,44 +116,11 @@ class ImportSupervisor:
raise UnknownSourceError(lane)
if self.source_enabled is not None and not self.source_enabled(lane):
return job
self._start(job)
return job
async def enqueue_off_loop(self, job_id: str, *, job: ImportJob | None = None) -> ImportJob:
"""`enqueue`, with its job read and its grant check made off the loop.
Both are database reads, and the grant check resolves the source root
on disk. Live sync enqueues from its poll, so on the loop they were
the main thread waiting for a reader behind an import peak. `job` is
the row the caller has just read, when it has one.
"""
if job_id in self._drains:
raise InvalidJobTransitionError("import_cleanup_in_progress")
def admit() -> tuple[ImportJob, bool]:
current = job if job is not None and job.job_id == job_id else None
if current is None:
current = self.job_store.require(job_id)
lane = self.lane_of(current)
return current, self.source_enabled is None or self.source_enabled(lane)
current, enabled = await in_the_background(admit)
lane = self.lane_of(current)
if lane not in self.adapters:
raise UnknownSourceError(lane)
if job_id in self._drains:
raise InvalidJobTransitionError("import_cleanup_in_progress")
if enabled:
self._start(current)
return current
def _start(self, job: ImportJob) -> None:
lane = self.lane_of(job)
existing = self._tasks.get(job.job_id)
existing = self._tasks.get(job_id)
if existing is None or existing.done():
self._tasks[job.job_id] = asyncio.create_task(
self._tasks[job_id] = asyncio.create_task(
self._execute(
job.job_id,
job_id,
source_id=lane,
operation=job.operation,
)
@@ -168,24 +130,10 @@ class ImportSupervisor:
component="import.supervisor",
event="import.queued",
outcome="queued",
job_id=job.job_id,
job_id=job_id,
details={"source_id": job.source_id, "mode": job.operation},
)
async def resume_interrupted_off_loop(self) -> None:
"""`resume_interrupted` for the running Core: every read and write off the loop."""
for status in (ImportStatus.INTERRUPTED, ImportStatus.PENDING):
jobs = await in_the_background(partial(self.job_store.find_jobs, statuses=(status,)))
for job in jobs:
if self.lane_of(job) not in self.adapters:
continue
if job.status is not ImportStatus.PENDING:
if self.source_enabled is not None and not await in_the_background(
self.source_enabled, self.lane_of(job)
):
continue
job = await in_the_background(self.job_store.queue_for_resume, job.job_id)
await self.enqueue_off_loop(job.job_id, job=job)
return job
def resume_interrupted(self) -> None:
# A process restart must not place recoverable work behind every newly
@@ -218,64 +166,40 @@ class ImportSupervisor:
self.enqueue(job_id)
return queued
async def resume_off_loop(self, job_id: str) -> ImportJob:
"""`resume`, with its reads and its write made off the event loop."""
if job_id in self._drains:
raise InvalidJobTransitionError("import_cleanup_in_progress")
queued = await in_the_background(self._queue_resumable, job_id)
if queued.status is not ImportStatus.PENDING:
return queued
return await self.enqueue_off_loop(job_id, job=queued)
def _queue_resumable(self, job_id: str) -> ImportJob:
job = self.job_store.require(job_id)
if job.status not in {
ImportStatus.PENDING,
ImportStatus.PAUSED,
ImportStatus.INTERRUPTED,
ImportStatus.FAILED,
}:
return job
return self.job_store.queue_for_resume(job_id)
async def wait(self, job_id: str) -> ImportJob:
task = self._tasks.get(job_id)
if task is not None:
await asyncio.shield(task)
return await in_the_background(self.job_store.require, job_id)
return self.job_store.require(job_id)
async def pause(self, job_id: str) -> ImportJob:
job = await in_the_background(self.job_store.require, job_id)
job = self.job_store.require(job_id)
if job.status in {ImportStatus.PAUSED, ImportStatus.INTERRUPTED}:
return job
await self._cancel_task(job_id)
async with self._settling:
return await in_the_background(self._settle_stopped, job_id, ImportStatus.PAUSED)
async def cancel(self, job_id: str) -> ImportJob:
job = await in_the_background(self.job_store.require, job_id)
if job.status is ImportStatus.CANCELLED:
return job
await self._cancel_task(job_id)
async with self._settling:
return await in_the_background(self._settle_stopped, job_id, ImportStatus.CANCELLED)
def _settle_stopped(self, job_id: str, target: ImportStatus) -> ImportJob:
"""A stopped runner's job moves to `target`, unless it completed first."""
current = self.job_store.require(job_id)
if current.status is ImportStatus.COMPLETED:
return current
return self.job_store.transition(job_id, target)
return self.job_store.transition(job_id, ImportStatus.PAUSED)
async def cancel(self, job_id: str) -> ImportJob:
job = self.job_store.require(job_id)
if job.status is ImportStatus.CANCELLED:
return job
await self._cancel_task(job_id)
current = self.job_store.require(job_id)
if current.status is ImportStatus.COMPLETED:
return current
return self.job_store.transition(job_id, ImportStatus.CANCELLED)
async def shutdown(self) -> None:
mine = frozenset(self._tasks)
def interrupt_mine() -> None:
for job in self.job_store.find_jobs(statuses=(ImportStatus.RUNNING,)):
if job.job_id in mine:
self.job_store.interrupt_running(job.job_id)
await in_the_background(interrupt_mine)
running = [
job
for job in self.job_store.list_jobs(status=ImportStatus.RUNNING)
if job.job_id in self._tasks
]
for job in running:
self.job_store.interrupt_running(job.job_id)
await asyncio.gather(
*(self._cancel_task(job_id) for job_id in tuple(self._tasks)),
)
@@ -431,7 +355,9 @@ class ImportSupervisor:
try:
await asyncio.wait_for(asyncio.shield(task), timeout)
except TimeoutError:
await in_the_background(self._interrupt_if_running, job_id)
current = self.job_store.require(job_id)
if current.status is ImportStatus.RUNNING:
self.job_store.interrupt_running(job_id)
return False
except asyncio.CancelledError:
# The worker noticed cancel and exited. A cancel of this
@@ -439,16 +365,10 @@ class ImportSupervisor:
caller = asyncio.current_task()
if caller is not None and caller.cancelling():
raise
# A read and a write on the single writer: off the loop. Live sync
# calls this from its poll, and on the loop it was the main thread
# waiting its turn for the writer behind an import peak.
await in_the_background(self._interrupt_if_running, job_id)
return True
def _interrupt_if_running(self, job_id: str) -> None:
current = self.job_store.require(job_id)
if current.status is ImportStatus.RUNNING:
self.job_store.interrupt_running(job_id)
return True
async def _cancel_task(self, job_id: str) -> None:
drain = self._drains.get(job_id)
@@ -15,7 +15,9 @@ class HyLifecycle:
object.__setattr__(self, "hidden", MappingProxyType(dict(self.hidden)))
def __eq__(self, other: object) -> bool:
return isinstance(other, HyLifecycle) and dict(self.hidden) == dict(other.hidden)
return isinstance(other, HyLifecycle) and dict(self.hidden) == dict(
other.hidden
)
def __hash__(self) -> int:
return hash(tuple(sorted(self.hidden.items())))
@@ -25,5 +27,9 @@ class HyLifecycle:
def replacing(self, memory_id: str) -> tuple[str, ...]:
return tuple(
sorted(old for old, replacements in self.hidden.items() if memory_id in replacements)
sorted(
old
for old, replacements in self.hidden.items()
if memory_id in replacements
)
)
+137 -155
View File
@@ -1,35 +1,15 @@
"""Core's intake for host hooks: visible text only, staged under Core's writer.
"""Write visible hook text into Evidence. Tool output stays out."""
The hook process (`capture_handoff`) never opens the database; it posts the
hook's JSON to `POST /api/v1/live-capture/{source}` on the running Core, which
calls `capture_hook` here. Tool output stays out.
A source that live sync already reads from its own files is left to that
path: the hook and the file name the same message differently (the hook has
no workspace and often no message id), so staging both sent one turn to the
SDK twice. Only a source no file poller follows is staged from the hook.
"""
import asyncio
import json
import os
import time
from collections.abc import Callable
from datetime import UTC, datetime
from enum import StrEnum
from pathlib import Path
from typing import cast
from typing import TextIO, cast
from fastapi import APIRouter, HTTPException, Request
from fastapi.params import Depends
from pydantic import JsonValue
from one_memory.capture_handoff import (
CAPTURE_ROUTE,
CAPTURE_SOURCES,
MAX_PAYLOAD_BYTES,
SDK_MIGRATION_CAPTURE_FENCE,
sdk_migration_capture_fenced,
)
from one_memory.database_access import main_database
from one_memory.derived_echo import PREFACE, strip_injected_memory
from one_memory.imports.access import opaque_workspace_id, safe_project_label
@@ -39,7 +19,6 @@ from one_memory.imports.models import (
EvidenceRecord,
OriginMetadata,
WrittenBy,
materialize_write_input,
)
from one_memory.imports.sources.claude_code import parse_claude_code_record
from one_memory.imports.sources.codex import HARNESS_PREFIXES, parse_codex_record
@@ -59,21 +38,23 @@ from one_memory.imports.sources.parsing import (
)
from one_memory.imports.sources.pi import _is_automated_session
from one_memory.imports.sources.workbuddy import _is_scheduled_automation, parse_workbuddy_record
from one_memory.memory_server.durable_queue import DurableWriteQueue
from one_memory.sdk_data_plane_migration import MemoryAuthority
__all__ = [
"CAPTURE_SOURCES",
"SDK_MIGRATION_CAPTURE_FENCE",
"CaptureOutcome",
"CaptureRejection",
"capture_hook",
"capture_rejection",
"live_capture_router",
"record_capture_rejection",
"sdk_migration_capture_fenced",
]
from one_memory.imports.storage import EvidenceStore
CAPTURE_SOURCES = frozenset(
{
"workbuddy",
"claude-code",
"codex",
"cursor",
"pi",
"qwen-code",
"qwenwork",
"hermes",
"deepseek-harness",
"opencode",
"openclaw",
}
)
_UNKNOWN_USER = frozenset({"hermes", "opencode", "openclaw"})
_USER_FIELDS = {
"UserPromptSubmit": "prompt",
@@ -94,130 +75,112 @@ _FILE_FALLBACK = {
}
_FALLBACK_EVENTS = frozenset({"Stop", "stop", "SubagentStop", "StopFailure"})
_TEXT_BLOCKS = frozenset({"text"})
SDK_MIGRATION_CAPTURE_FENCE = ".sdk-migration-capture-fence.json"
class CaptureOutcome(StrEnum):
"""What Core did with one hook payload. Never the payload itself."""
def database_path() -> Path:
from one_memory.config import load_settings
STAGED = "staged"
#: Live sync reads this source's own files; that path owns the turn.
FILE_POLLED = "file_polled"
REJECTED = "rejected"
IGNORED = "ignored"
return load_settings().data_dir / "one-memory.sqlite3"
def capture_hook(
source_id: str,
payload: object,
database: Path,
*,
queue: DurableWriteQueue,
file_polled: Callable[[str], bool] = lambda _source_id: False,
) -> CaptureOutcome:
"""Gate one hook payload and stage its visible lines, inside Core.
def capture_stdio(source_id: str, stdin: TextIO, output: TextIO) -> int:
"""Host hooks must stay silent. A bad payload does not fail the turn."""
Runs on SDK authority through Core's single writer: the lines become
write-job input exactly as live file sync stages them
(`materialize_write_input` -> `stage_sdk_input` -> `finalize_sdk_file`,
live priority). No Evidence table is involved.
del output
if source_id not in CAPTURE_SOURCES:
return 2
database = database_path()
if sdk_migration_capture_fenced(database.parent):
return 0
try:
payload = json.loads(stdin.read() or "")
except json.JSONDecodeError:
return 0
try:
capture_hook(source_id, payload, database)
except Exception:
return 0
return 0
def sdk_migration_capture_fenced(data_dir: Path) -> bool:
"""Ignore stale fences; a live migration or a failed one pauses hook writes.
A failed upgrade migration leaves the fence `frozen` (no pid): the home was
put back as it was before the upgrade and takes no writes until the next
migration attempt, which replaces the fence and lifts it on success.
"""
marker = data_dir / SDK_MIGRATION_CAPTURE_FENCE
try:
document = json.loads(marker.read_text(encoding="utf-8"))
if isinstance(document, dict) and document.get("frozen") is True:
return True
pid = int(document["pid"])
if pid < 1:
raise ValueError("invalid migration fence pid")
os.kill(pid, 0)
return True
except FileNotFoundError:
return False
except ProcessLookupError:
marker.unlink(missing_ok=True)
return False
except PermissionError:
return True
except (KeyError, TypeError, ValueError, json.JSONDecodeError, OSError):
marker.unlink(missing_ok=True)
return False
def capture_hook(source_id: str, payload: object, database: Path) -> bool:
if source_id not in CAPTURE_SOURCES or not isinstance(payload, dict):
return CaptureOutcome.IGNORED
return False
# Cursor also runs ~/.claude/settings.json hooks; its own hook already files the turn.
if source_id != "cursor" and "cursor_version" in payload:
return CaptureOutcome.IGNORED
return False
if not database.is_file():
return CaptureOutcome.IGNORED
return False
rejection = capture_rejection(database, source_id)
if rejection is not None:
# Silent, and the reason is counted rather than printed: a hook that
# spoke would put this into the host's turn as text.
record_capture_rejection(database, source_id, rejection)
return CaptureOutcome.REJECTED
return False
if source_id == "pi" and _is_automated_session(first_string(payload.get("cwd"))):
return CaptureOutcome.IGNORED
if file_polled(source_id):
# The same turn reaches the SDK once, from the file: the two lanes give
# one message different identities, so staging both wrote it twice.
return CaptureOutcome.FILE_POLLED
inputs = _hook_inputs(source_id, payload) or _capture_transcript(source_id, payload, database)
if not inputs:
return CaptureOutcome.IGNORED
observed = datetime.now(UTC)
conversations: dict[str, list[EvidenceRecord]] = {}
for value in inputs:
conversations.setdefault(value.conversation_id, []).append(
materialize_write_input(value, observed_at=observed)
)
for conversation_id, records in conversations.items():
# The turn rule decides what is written (`turn_batches`): a turn is
# complete when the next user message arrives, batches are 5 turns or
# 8000 characters, and Core writes the rest after 10 quiet minutes.
queue.assemble_live_records(tuple(records), file_key=f"hook:{conversation_id}")
return CaptureOutcome.STAGED
return False
lines = _messages(source_id, payload)
if lines:
conversation_id = _conversation(payload)
store = EvidenceStore(database)
wrote = False
for role, written_by, verbatim, message_id in lines:
target = conversation_id
if target is None:
continue
result = store.put(
EvidenceInput(
source_id=source_id,
conversation_id=target,
message_id=message_id,
source_ordinal=None if message_id else "hook-visible",
role=role,
written_by=written_by,
created_at=datetime.now(UTC),
verbatim=verbatim,
origin_meta={EvidenceOrigin.HOOK: OriginMetadata()},
)
)
_submit_live_memory(
store, result.record, changed=result.was_created or result.was_revised
)
wrote = True
return wrote
return _capture_transcript(source_id, payload, database)
def _hook_inputs(source_id: str, payload: dict[str, object]) -> list[EvidenceInput]:
conversation_id = _conversation(payload)
if conversation_id is None:
return []
observed = datetime.now(UTC)
return [
EvidenceInput(
source_id=source_id,
conversation_id=conversation_id,
message_id=message_id,
source_ordinal=None if message_id else "hook-visible",
role=role,
written_by=written_by,
created_at=observed,
verbatim=verbatim,
origin_meta={EvidenceOrigin.HOOK: OriginMetadata()},
)
for role, written_by, verbatim, message_id in _messages(source_id, payload)
]
def live_capture_router(
dependencies: list[Depends],
*,
file_polled: Callable[[Request, str], bool],
) -> APIRouter:
"""`POST /api/v1/live-capture/{source_id}`: a hook's payload, handed over by the hook.
202 whatever the gates decide, so a hook has nothing to print. The answer
names the outcome only, never any of the payload.
"""
router = APIRouter(dependencies=dependencies)
@router.post(CAPTURE_ROUTE, status_code=202)
async def live_capture(source_id: str, request: Request) -> dict[str, str]:
if source_id not in CAPTURE_SOURCES:
raise HTTPException(status_code=404, detail="capture_source_not_found")
body = await request.body()
if len(body) > MAX_PAYLOAD_BYTES:
raise HTTPException(status_code=413, detail="capture_payload_too_large")
try:
payload = json.loads(body)
except (json.JSONDecodeError, UnicodeError) as error:
raise HTTPException(status_code=422, detail="capture_payload_invalid") from error
state = request.app.state
queue = getattr(state, "memory_write_queue", None)
if getattr(state, "memory_authority", None) is not MemoryAuthority.SDK or queue is None:
# Only an SDK-authority home is served; nothing here may bring the
# retired Evidence tables back for any other.
return {"outcome": CaptureOutcome.IGNORED.value}
outcome = await asyncio.to_thread(
capture_hook,
source_id,
payload,
queue.database_path,
queue=queue,
file_polled=lambda source: file_polled(request, source),
)
return {"outcome": outcome.value}
return router
def _sync_open(database: Path, source_id: str) -> bool:
return capture_rejection(database, source_id) is None
class CaptureRejection(StrEnum):
@@ -292,6 +255,22 @@ def record_capture_rejection(
return
def _submit_live_memory(store: EvidenceStore, record: EvidenceRecord, *, changed: bool) -> None:
"""Add the record to its conversation's live assembling row.
The turn rule decides what is written (`turn_batches`): a turn is
complete when the next user message arrives, batches are 5 turns or
8000 characters, and Core writes the rest after 10 quiet minutes.
"""
if not changed or not _sync_open(store.database_path, record.source_id):
return
from one_memory.memory_server.durable_queue import DurableWriteQueue
DurableWriteQueue(store.database_path, evidence=store).assemble_live_records(
(record,), file_key=f"hook:{record.conversation_id}"
)
def _conversation(payload: dict[str, object]) -> str | None:
return first_string(
cast(JsonValue | None, payload.get("session_id")),
@@ -491,35 +470,34 @@ def _opencode_messages(
return lines
def _capture_transcript(
source_id: str, payload: dict[str, object], database: Path
) -> list[EvidenceInput]:
def _capture_transcript(source_id: str, payload: dict[str, object], database: Path) -> bool:
"""Stop without in-memory text reads the granted transcript, then the same parser."""
event = payload.get("hook_event_name") or payload.get("hookEventName")
parser = _FILE_FALLBACK.get(source_id)
transcript = payload.get("transcript_path")
if event not in _FALLBACK_EVENTS or parser is None or not isinstance(transcript, str):
return []
return False
root = _grant_root(database, source_id)
if root is None:
return []
return False
try:
granted = root.resolve()
target = Path(transcript).resolve()
except OSError:
return []
return False
if target != granted and granted not in target.parents:
return []
return False
if not target.is_file():
return []
return False
_wait_stable(target)
try:
raw = target.read_text(encoding="utf-8")
except OSError:
return []
return False
relative = target.relative_to(granted).as_posix()
inputs: list[EvidenceInput] = []
store = EvidenceStore(database)
wrote = False
for index, line in enumerate(raw.splitlines()):
if not line.strip():
continue
@@ -536,7 +514,7 @@ def _capture_transcript(
if not isinstance(parsed, ParsedMessage):
continue
for row_index, row in enumerate(parsed.rows()):
inputs.append(
result = store.put(
EvidenceInput(
source_id=source_id,
workspace=opaque_workspace_id(source_id, row.workspace),
@@ -555,7 +533,11 @@ def _capture_transcript(
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
)
return inputs
_submit_live_memory(
store, result.record, changed=result.was_created or result.was_revised
)
wrote = True
return wrote
def _grant_root(database: Path, source_id: str) -> Path | None:
@@ -28,11 +28,10 @@ from pathlib import Path
from typing import Any, Literal
from uuid import uuid4
from one_memory.background_work import in_lease_renewal, in_the_write_queue
from one_memory.background_work import in_the_background
from one_memory.conversation_processing import TURN_IDLE_FLUSH_SECONDS, turn_batches
from one_memory.database_access import main_database
from one_memory.diagnostic_log import diagnostic_event
from one_memory.exception_reporting import report_exception
from one_memory.imports.models import EvidenceOrigin, EvidenceRecord
from one_memory.imports.storage import EvidenceStore
from one_memory.memory_server.contracts import (
@@ -452,14 +451,6 @@ class _LeaseRenewalFatalError(RuntimeError):
"""A non-transient renewal failure that must stop the worker."""
def _drain_exit_code(error: BaseException) -> str:
"""A stable, lowercase code for why the drain exited; never message text."""
if isinstance(error, _LeaseRenewalFatalError):
return "lease_renewal_failed"
name = re.sub(r"(?<!^)(?=[A-Z])", "_", type(error).__name__).lower()
return name if re.fullmatch(r"[a-z_]{1,64}", name) else "drain_error"
@dataclass(frozen=True)
class UnknownWriteJob:
"""An `unknown` row that still needs an external inquiry.
@@ -2549,7 +2540,7 @@ async def execute_job(
evidence: EvidenceStore,
owner: str | None = None,
source_writes_permitted: Callable[[str], bool] | None = None,
admit_hy_invocation: Callable[[str, bool, Callable[[], Awaitable[bool]]], Awaitable[bool]]
admit_hy_invocation: Callable[[str, Callable[[], Awaitable[bool]]], Awaitable[bool]]
| None = None,
model_failures: Callable[[], AbstractContextManager[ModelAttempt | None]] | None = None,
) -> None:
@@ -2571,14 +2562,14 @@ async def execute_job(
# Every store call here is synchronous SQLite work on the single writer or
# a reader; made from the loop it would block health checks and the API for
# as long as the write lock is held.
request = await in_the_write_queue(queue.payload_for, job_id)
request = await in_the_background(queue.payload_for, job_id)
if request is None:
# A claimed row whose payload is missing or unreadable (reaped while
# still claimable, corrupted, or a foreign row) must not be left
# `running`: nothing was sent, so it is a terminal failure rather than
# an unknown, and the lease must be released so it is not held until it
# lapses.
await in_the_write_queue(
await in_the_background(
queue.settle,
job_id,
state=FAILED,
@@ -2588,9 +2579,9 @@ async def execute_job(
)
return
try:
records = await in_the_write_queue(evidence_for_request, evidence, request)
records = await in_the_background(evidence_for_request, evidence, request)
except LookupError as error:
await in_the_write_queue(
await in_the_background(
queue.settle,
job_id,
state=FAILED,
@@ -2602,7 +2593,7 @@ async def execute_job(
hy_invoked = False
async def mark_hy_invocation(requires_generation: bool) -> None:
async def mark_hy_invocation() -> None:
"""Persist the durable boundary immediately before the real HY call."""
nonlocal hy_invoked
if hy_invoked:
@@ -2627,10 +2618,10 @@ async def execute_job(
source_id = lane_key(next(iter(users), DEFAULT_USER_ID), source_id)
async def mark() -> bool:
return await in_the_write_queue(queue.mark_invoked, job_id, owner=owner)
return await in_the_background(queue.mark_invoked, job_id, owner=owner)
if admit_hy_invocation is not None:
admitted = await admit_hy_invocation(source_id, requires_generation, mark)
admitted = await admit_hy_invocation(source_id, mark)
else:
if source_writes_permitted is not None and not source_writes_permitted(source_id):
raise RuntimeError("model_content_source_disabled")
@@ -2642,7 +2633,7 @@ async def execute_job(
raise RuntimeError("durable_write_lease_lost_before_invocation")
hy_invoked = True
attempt = await in_the_write_queue(queue.sdk_attempt, job_id)
attempt = await in_the_background(queue.sdk_attempt, job_id)
window = model_failures() if model_failures is not None else nullcontext(None)
provider: ModelAttempt | None = None
try:
@@ -2684,7 +2675,7 @@ async def execute_job(
}
and not hy_invoked
):
await in_the_write_queue(queue.release_unstarted, job_id, owner=owner)
await in_the_background(queue.release_unstarted, job_id, owner=owner)
return
provider_failure = provider.failure if provider is not None else None
if hy_invoked and provider_failure is not None:
@@ -2692,7 +2683,7 @@ async def execute_job(
# gone, a provider that is down): the job is fine and waits for
# the provider, in its place, without spending a retry. The
# breaker paces when it is tried again.
await in_the_write_queue(
await in_the_background(
queue.requeue_for_model, job_id, error_code=provider_failure, owner=owner
)
logger.info(
@@ -2721,7 +2712,7 @@ async def execute_job(
elif not hy_invoked and code in MODEL_NOT_READY_CODES:
state = WAITING_FOR_MODEL
if state is not None:
await in_the_write_queue(
await in_the_background(
queue.settle,
job_id,
state=state,
@@ -2733,7 +2724,7 @@ async def execute_job(
# Anything else is this job's own trouble: content the model or
# the SDK refused, a reply that did not parse, a check that failed
# after the write. Retried a few times, then `failed`.
state = await in_the_write_queue(
state = await in_the_background(
queue.retry_or_fail, job_id, error_code=code, owner=owner
)
# Never `logger.exception` here: it prints the exception's message,
@@ -2760,7 +2751,7 @@ async def execute_job(
error_chain_text(error),
)
return
await in_the_write_queue(
await in_the_background(
queue.settle,
job_id,
state=COMPLETED,
@@ -2769,13 +2760,13 @@ async def execute_job(
sdk_memory_ids=memory_ids,
)
release = getattr(evidence, "release_sdk_staging", None)
durable_receipt = await in_the_write_queue(queue.sdk_receipt, job_id)
durable_receipt = await in_the_background(queue.sdk_receipt, job_id)
if (
callable(release)
and durable_receipt is not None
and (durable_receipt.raw_ids or durable_receipt.memory_ids)
):
await in_the_write_queue(
await in_the_background(
release,
tuple(record.evidence_id for record in records),
)
@@ -2900,16 +2891,6 @@ class DurableWriteDrain:
#: a minute against a provider that refused every one (#245).
self.model_claim_allowance = model_claim_allowance
self._last_unknown_reconcile: datetime | None = None
#: What `run_supervised` says about the drain: None before it was
#: first started, then `running`, `restarting` (it exited on an error
#: and waits out its backoff) or `stopped`. Shown by memory-processing.
self.liveness: Literal["running", "restarting", "stopped"] | None = None
#: The stable code of the last unexpected exit, kept after a restart.
self.last_error_code: str | None = None
self.restarts = 0
self._stop_requested = False
#: Wakes a restart backoff, so `stop()` does not wait it out.
self._stopping = asyncio.Event()
def claim(
self,
@@ -3237,7 +3218,7 @@ class DurableWriteDrain:
# Claim is a blocking SQLite write on the single writer connection.
# Run it off the loop, like every other DB call in this chain; a
# busy writer would otherwise stall the whole event loop.
ids = await in_the_write_queue(
ids = await in_the_background(
self.claim,
limit=limit,
budget_ms=self.claim_budget_ms,
@@ -3250,10 +3231,10 @@ class DurableWriteDrain:
await asyncio.gather(*(self._run_claimed_job(job_id) for job_id in ids))
async def _run_claimed_job(self, job_id: str) -> None:
if not await in_the_write_queue(self.still_owns, job_id):
if not await in_the_background(self.still_owns, job_id):
logger.warning("durable_write_drain_lease_lost_before_execution job_id=%s", job_id)
return
deadline = await in_the_write_queue(self.lease_deadline, job_id)
deadline = await in_the_background(self.lease_deadline, job_id)
if deadline is None:
return
renewal = asyncio.create_task(self._renew_loop(job_id, deadline))
@@ -3280,9 +3261,11 @@ class DurableWriteDrain:
"durable_write_drain_job_ceiling_exceeded job_id=%s",
job_id,
)
await self._settle_worker_error(
await in_the_background(
self.queue.settle_worker_error,
job_id,
error_code="durable_work_ceiling_exceeded",
owner=self.owner,
)
except Exception as error:
logger.error(
@@ -3290,15 +3273,12 @@ class DurableWriteDrain:
job_id,
type(error).__name__,
)
report_exception(
"memory.write_queue",
"memory.write_queue.executor_failed",
error,
)
self._record_database_error(error)
await self._settle_worker_error(
await in_the_background(
self.queue.settle_worker_error,
job_id,
error_code="durable_executor_error",
owner=self.owner,
)
finally:
execution.cancel()
@@ -3307,39 +3287,9 @@ class DurableWriteDrain:
with contextlib.suppress(asyncio.CancelledError, Exception):
await task
async def _settle_worker_error(self, job_id: str, *, error_code: str) -> None:
"""Best-effort settlement must not kill the long-lived drain.
The row still has a durable lease when this fallback runs. If the
single SQLite writer is busy, waiting and retrying here only extends
the same contention that made the executor fail. Leave the row where
it is instead: lease recovery later moves a never-invoked job back to
pending and a possibly-invoked job to unknown, preserving the
no-blind-retry boundary.
Only transient lock/busy refusals are deferred. A malformed query or
other programming error still escapes to the outer per-job fence.
"""
try:
await in_the_write_queue(
self.queue.settle_worker_error,
job_id,
error_code=error_code,
owner=self.owner,
)
except sqlite3.OperationalError as error:
if "locked" not in str(error).lower() and "busy" not in str(error).lower():
raise
self._sample_database_failure("settle_worker_error", error)
logger.warning(
"durable_write_worker_settlement_deferred job_id=%s error_type=%s",
job_id,
type(error).__name__,
)
async def _execute_with_ceiling(self, job_id: str) -> None:
ceiling = (
await in_the_write_queue(self.work_ceiling)
await in_the_background(self.work_ceiling)
if callable(self.work_ceiling)
else self.work_ceiling
)
@@ -3374,33 +3324,7 @@ class DurableWriteDrain:
finished = {task for task in active if task.done()}
active.difference_update(finished)
for task in finished:
try:
await task
except asyncio.CancelledError:
raise
except _LeaseRenewalFatalError:
# Renewal is the ownership fence. If it is broken,
# continuing could let another worker reclaim a job
# while this process is still executing it.
raise
except Exception as error:
# A job task is a fault-containment boundary. Its row
# retains the durable lease and is recovered according
# to hy_invoked; killing this only consumer would strand
# every unrelated write behind it.
self._record_database_error(error)
logger.error(
"durable_write_drain_job_task_failed task_name=%s error_type=%s",
task.get_name(),
type(error).__name__,
)
# With its stack: a type name alone cannot tell one
# stray failure from a bug that fails every job.
report_exception(
"memory.write_queue",
"memory.write_queue.job_task_failed",
error,
)
await task
if self.budget is not None and work_deadline is not None:
now = _time.monotonic()
# Always admit the first unit of a cycle even when the
@@ -3416,7 +3340,7 @@ class DurableWriteDrain:
try:
available = self.max_concurrent_sources - len(active)
ids = (
await in_the_write_queue(
await in_the_background(
self.claim,
limit=available,
budget_ms=self.claim_budget_ms,
@@ -3425,12 +3349,7 @@ class DurableWriteDrain:
else ()
)
for job_id in ids:
active.add(
asyncio.create_task(
self._run_claimed_job(job_id),
name=f"durable-write:{job_id}",
)
)
active.add(asyncio.create_task(self._run_claimed_job(job_id)))
executed = len(ids)
self._database_failure_streaks.pop("claim", None)
except asyncio.CancelledError:
@@ -3528,79 +3447,7 @@ class DurableWriteDrain:
await asyncio.gather(*active, return_exceptions=True)
def stop(self) -> None:
self._stop_requested = True
self.running = False
self._stopping.set()
async def run_supervised(
self,
*,
idle_pause: float = 0.05,
restart_backoff: float = 1.0,
max_restart_backoff: float = 60.0,
) -> None:
"""`run_forever`, started again whenever it exits on an error.
The drain is the queue's only consumer, so a drain that died stranded
every write behind it until the next start, and nothing said so: an
unexpected SQLite error on the claim path (disk I/O, a vanished file)
or an exception from a claim gate ended it for good. Each exit is now
reported with its stack, and the drain comes back after a pause that
doubles up to `max_restart_backoff`, back to `restart_backoff` once it
has run for that long. `stop()` and cancellation end it as before.
Ownership across a restart: `run_forever` only returns once every job
task it started has finished (its `finally`), and a task whose lease
renewal broke (`_LeaseRenewalFatalError`) waits for its execution to
return first, so nothing of the old drain is still running a job when
the new one claims. The new one also takes a new lease owner, so a
statement of the old one still in a pool thread can neither renew nor
settle a row under the new drain's name, and the old drain's leases
are only ever recovered by expiry (`reclaim_expired`), which decides
by `hy_invoked` exactly as for a drain that died with its process.
"""
backoff = restart_backoff
while not self._stop_requested:
started = _time.monotonic()
self.liveness = "running"
try:
await self.run_forever(idle_pause=idle_pause)
except asyncio.CancelledError:
self.liveness = "stopped"
raise
except Exception as error:
self.last_error_code = _drain_exit_code(error)
self.restarts += 1
report_exception(
"memory.write_queue",
"memory.write_queue.drain_exited",
error,
context={"restarts": self.restarts},
)
logger.error(
"durable_write_drain_exited error_code=%s restarts=%d chain=%s",
self.last_error_code,
self.restarts,
error_chain_text(error),
)
else:
if self._stop_requested:
break
if self._stop_requested:
break
self.owner = f"durable-drain:{uuid4()}"
if _time.monotonic() - started >= max_restart_backoff:
backoff = restart_backoff
self.liveness = "restarting"
try:
await asyncio.wait_for(self._stopping.wait(), timeout=backoff)
except TimeoutError:
pass
except asyncio.CancelledError:
self.liveness = "stopped"
raise
backoff = min(backoff * 2, max_restart_backoff)
self.liveness = "stopped"
async def _run_idle_maintenance(
self,
@@ -3618,7 +3465,7 @@ class DurableWriteDrain:
persisting provider or user content.
"""
try:
await in_the_write_queue(operation)
await in_the_background(operation)
self._database_failure_streaks.pop(f"maintenance:{operation_name}", None)
except asyncio.CancelledError:
raise
@@ -3646,7 +3493,7 @@ class DurableWriteDrain:
renewed_at = datetime.now(UTC)
remaining = (deadline - renewed_at).total_seconds()
if remaining <= 0 or not await asyncio.wait_for(
in_lease_renewal(self.renew, job_id), timeout=remaining
in_the_background(self.renew, job_id), timeout=remaining
):
return False
deadline = renewed_at + self.lease
@@ -3190,110 +3190,6 @@ async def _061_write_job_retries(cursor: Any) -> None:
await cursor.execute(statement)
ONBOARDING_PROGRESS_MIGRATION_VERSION = 62
def onboarding_progress_ddl(dialect: str = "mysql") -> str:
"""Append resumable onboarding choices without rewriting migration 12."""
if dialect == "sqlite":
return """
ALTER TABLE onboarding_state
ADD COLUMN mode_intent TEXT NULL;
ALTER TABLE onboarding_state
ADD COLUMN setup_step TEXT NULL;
ALTER TABLE onboarding_state
ADD COLUMN model_profile_id TEXT NULL;
"""
return """
ALTER TABLE onboarding_state
ADD COLUMN mode_intent VARCHAR(16) NULL AFTER updated_at;
ALTER TABLE onboarding_state
ADD COLUMN setup_step VARCHAR(16) NULL AFTER mode_intent;
ALTER TABLE onboarding_state
ADD COLUMN model_profile_id VARCHAR(36) NULL AFTER setup_step;
"""
async def _062_onboarding_progress(cursor: Any) -> None:
if _dialect(cursor) == "sqlite":
existing = await cursor.columns("onboarding_state")
statements = _split_statements(onboarding_progress_ddl("sqlite"))
for column, statement in zip(
("mode_intent", "setup_step", "model_profile_id"),
statements,
strict=True,
):
if column not in existing:
await cursor.execute(statement)
return
for statement in _split_statements(onboarding_progress_ddl("mysql")):
try:
await cursor.execute(statement)
except Exception as error: # noqa: BLE001 - MySQL driver error type varies
if _is_duplicate_column(error):
continue
raise
IMPORT_JOB_LOOKUP_INDEXES_MIGRATION_VERSION = 63
def import_job_lookup_indexes_ddl(dialect: str = "mysql") -> str:
"""Index the per-source and per-status import job lookups.
Migration 7 gave SQLite `import_jobs` no index beyond its primary key (the
one-active-job index is partial), so every "this source's jobs" or "the
jobs in this state" question read and sorted the whole table. Live sync
adds a row per change it imports and nothing removed them, so on a machine
a few weeks old that table was thousands of rows read several times a poll.
`import_jobs_lane` serves the lane lookups (one user, one source, one
operation, a status, newest first) and the pruning of finished syncs;
`import_jobs_status` serves restart recovery and the status filters.
MySQL's migration 7 already has a status index and no `operation` column,
so it only gains the ordered source index.
"""
if dialect == "sqlite":
return """
CREATE INDEX IF NOT EXISTS import_jobs_lane
ON import_jobs(user_id, source_id, operation, status, created_at);
CREATE INDEX IF NOT EXISTS import_jobs_status
ON import_jobs(status, created_at);
"""
return """
CREATE INDEX idx_import_jobs_source_created
ON import_jobs(user_id, source_id, created_at);
"""
def _is_duplicate_index(error: Exception) -> bool:
"""MySQL's "index name already there" (1061): the index is in place."""
args: tuple[object, ...] = getattr(error, "args", ())
return bool(args) and args[0] == 1061
async def _063_import_job_lookup_indexes(cursor: Any) -> None:
dialect = _dialect(cursor)
if dialect == "sqlite":
# `operation` was added by the job store, not by migration 7, so a
# database built from the catalog alone may not have it yet. Same
# declaration as the store's own addition.
if "operation" not in await cursor.columns("import_jobs"):
await cursor.execute(
"ALTER TABLE import_jobs ADD COLUMN operation TEXT NOT NULL DEFAULT 'history'"
)
for statement in _split_statements(import_job_lookup_indexes_ddl(dialect)):
await cursor.execute(statement)
return
for statement in _split_statements(import_job_lookup_indexes_ddl(dialect)):
try:
await cursor.execute(statement)
except Exception as error: # noqa: BLE001 - MySQL driver error type varies
if _is_duplicate_index(error):
continue
raise
def _split_statements(sql: str) -> list[str]:
"""Split a multi-statement script, dropping blanks, PRAGMA and comments.
@@ -3594,15 +3490,4 @@ MIGRATIONS: tuple[Migration, ...] = (
("memory_note_requests",),
),
Migration(61, "write_job_retries", _061_write_job_retries),
Migration(
ONBOARDING_PROGRESS_MIGRATION_VERSION,
"onboarding_progress",
_062_onboarding_progress,
("onboarding_state",),
),
Migration(
IMPORT_JOB_LOOKUP_INDEXES_MIGRATION_VERSION,
"import_job_lookup_indexes",
_063_import_job_lookup_indexes,
),
)
@@ -247,17 +247,15 @@ class ModelBackoff:
and self._clock() < self._until
)
def allowance(self, *, kind: str | None = None) -> int | None:
"""How many matching provider jobs may be claimed now.
def allowance(self) -> int | None:
"""How many model jobs may be claimed now: None for no limit.
0 while paused; once the pause is over, one probe at a time until it
reports or `PROBE_HOLD_SECONDS` pass. ``kind`` keeps a failed chat
model from pausing raw-only work that needs embeddings, while an
embedding outage still paces that work.
reports or `PROBE_HOLD_SECONDS` pass.
"""
now = self._clock()
with self._lock:
if self._failures == 0 or (kind is not None and kind not in self._failing):
if self._failures == 0:
return None
if now < self._until or now < self._probe_until:
return 0
+3 -16
View File
@@ -26,7 +26,6 @@ if TYPE_CHECKING:
_operation: ContextVar[tuple["ModelProbeOperations", str] | None] = ContextVar(
"model_probe_operation", default=None
)
_dispatch_started: ContextVar[bool] = ContextVar("model_probe_dispatch_started", default=False)
class ProbeCode(StrEnum):
@@ -114,26 +113,16 @@ class ModelProber:
if existing is not None:
return existing
token = _operation.set((operations, request.operation_id))
dispatch_token = _dispatch_started.set(False)
try:
await self._probe(profile_id)
except ProbeOperationConflict:
await asyncio.to_thread(
operations.fail, request.operation_id, "model_probe_profile_changed"
)
except Exception as exc:
if _dispatch_started.get():
# Once a request may have reached the provider, preserve the
# reservation so a retry cannot duplicate a possibly billed call.
raise ProbeOperationUnavailable("model_probe_operation_unavailable") from None
logging.getLogger(__name__).warning(
"model probe failed before dispatch: %s", type(exc).__name__
)
await asyncio.to_thread(
operations.fail, request.operation_id, "model_probe_execution_failed"
)
except Exception:
# Unknown dispatch/storage outcomes stay reserved, including across restart.
raise ProbeOperationUnavailable("model_probe_operation_unavailable") from None
finally:
_dispatch_started.reset(dispatch_token)
_operation.reset(token)
return await asyncio.to_thread(operations.lookup, request.operation_id)
@@ -328,8 +317,6 @@ class ModelProber:
listing_optional: bool = False,
) -> ProbeStage:
await self._check_operation()
if _operation.get() is not None:
_dispatch_started.set(True)
started = monotonic()
try:
response = await client.request(
+32 -171
View File
@@ -4,10 +4,9 @@ from collections.abc import Iterator
from contextlib import contextmanager
from datetime import UTC, datetime
from pathlib import Path
from typing import Any, Literal
from uuid import UUID
from typing import Any
from pydantic import BaseModel, ConfigDict, Field, field_validator
from pydantic import BaseModel, ConfigDict
from one_memory.database import (
take_the_write_lock_first,
@@ -15,25 +14,11 @@ from one_memory.database import (
from one_memory.database_access import main_database
#: Schema owned by `memory_server.migrations` (migration 12). No local copy.
from one_memory.memory_server.migrations import (
onboarding_progress_ddl,
onboarding_state_ddl,
)
from one_memory.memory_server.sql_compat import execute, is_mysql, row_to_dict
from one_memory.memory_server.migrations import onboarding_state_ddl
from one_memory.memory_server.sql_compat import execute, is_mysql
from one_memory.memory_server.store_binding import store_for
SCHEMA = onboarding_state_ddl("sqlite")
ONBOARDING_COLUMNS = (
"singleton",
"status",
"selected_source_ids_json",
"started_at",
"completed_at",
"updated_at",
"mode_intent",
"setup_step",
"model_profile_id",
)
#: The start/complete upserts, per engine.
#:
@@ -66,13 +51,11 @@ START_MYSQL = """
updated_at = VALUES(updated_at)
"""
COMPLETE_UPDATE_SQLITE = (
"UPDATE onboarding_state SET status = 'completed', completed_at = ?, updated_at = ?, "
"mode_intent = NULL, setup_step = NULL, model_profile_id = NULL "
"UPDATE onboarding_state SET status = 'completed', completed_at = ?, updated_at = ? "
"WHERE singleton = 1"
)
COMPLETE_UPDATE_MYSQL = (
"UPDATE onboarding_state SET status = 'completed', completed_at = %s, updated_at = %s, "
"mode_intent = NULL, setup_step = NULL, model_profile_id = NULL "
"UPDATE onboarding_state SET status = 'completed', completed_at = %s, updated_at = %s "
"WHERE singleton = 1"
)
COMPLETE_INSERT_SQLITE = (
@@ -88,27 +71,13 @@ COMPLETE_INSERT_MYSQL = (
RESET_UPDATE_SQLITE = (
"UPDATE onboarding_state SET status = 'not_started', "
"selected_source_ids_json = '[]', started_at = NULL, completed_at = NULL, "
"mode_intent = NULL, setup_step = NULL, model_profile_id = NULL, "
"updated_at = ? WHERE singleton = 1"
)
RESET_UPDATE_MYSQL = (
"UPDATE onboarding_state SET status = 'not_started', "
"selected_source_ids_json = '[]', started_at = NULL, completed_at = NULL, "
"mode_intent = NULL, setup_step = NULL, model_profile_id = NULL, "
"updated_at = %s WHERE singleton = 1"
)
ENSURE_PROGRESS_SQLITE = (
"INSERT INTO onboarding_state ("
"singleton,status,selected_source_ids_json,started_at,completed_at,updated_at"
") VALUES (1,'not_started','[]',NULL,NULL,?) "
"ON CONFLICT(singleton) DO NOTHING"
)
ENSURE_PROGRESS_MYSQL = (
"INSERT INTO onboarding_state ("
"singleton,status,selected_source_ids_json,started_at,completed_at,updated_at"
") VALUES (1,'not_started','[]',NULL,NULL,%s) "
"ON DUPLICATE KEY UPDATE singleton=VALUES(singleton)"
)
class OnboardingState(BaseModel):
@@ -118,35 +87,6 @@ class OnboardingState(BaseModel):
selected_source_ids: tuple[str, ...]
started_at: datetime | None
completed_at: datetime | None
mode_intent: Literal["lite", "pro", "ultra"] | None
setup_step: Literal["review", "mode", "model"] | None
model_profile_id: str | None
class OnboardingProgressUpdate(BaseModel):
"""Only non-sensitive, resumable choices accepted from the onboarding UI."""
model_config = ConfigDict(frozen=True, extra="forbid")
mode_intent: Literal["lite", "pro", "ultra"] | None = None
setup_step: Literal["review", "mode", "model"] | None = None
model_profile_id: str | None = Field(default=None, min_length=36, max_length=36)
@field_validator("model_profile_id")
@classmethod
def validate_model_profile_id(cls, value: str | None) -> str | None:
if value is None:
return None
try:
if str(UUID(value)) != value:
raise ValueError
except (ValueError, TypeError, AttributeError):
raise ValueError("invalid_model_profile_id") from None
return value
class OnboardingCompletedError(RuntimeError):
pass
class OnboardingStore:
@@ -163,31 +103,12 @@ class OnboardingStore:
"""
with self._connect(initialize=True) as connection:
execute(connection, is_mysql(self._store), SCHEMA)
if not is_mysql(self._store):
existing = {
str(row[1]) for row in connection.execute("PRAGMA table_info(onboarding_state)")
}
statements = tuple(
statement.strip()
for statement in onboarding_progress_ddl("sqlite").split(";")
if statement.strip()
)
for column, statement in zip(
("mode_intent", "setup_step", "model_profile_id"),
statements,
strict=True,
):
if column not in existing:
connection.execute(statement)
def get(self) -> OnboardingState:
mysql = is_mysql(self._store)
with self._connect() as connection:
row = execute(
connection,
mysql,
"SELECT " + ", ".join(ONBOARDING_COLUMNS) + " "
"FROM onboarding_state WHERE singleton = 1",
connection, mysql, "SELECT * FROM onboarding_state WHERE singleton = 1"
).fetchone()
if row is None:
return OnboardingState(
@@ -195,11 +116,8 @@ class OnboardingStore:
selected_source_ids=(),
started_at=None,
completed_at=None,
mode_intent=None,
setup_step=None,
model_profile_id=None,
)
return self._to_state(row, mysql=mysql)
return self._to_state(row)
def start(self, source_ids: tuple[str, ...]) -> OnboardingState:
now = datetime.now(UTC).isoformat()
@@ -232,43 +150,6 @@ class OnboardingStore:
)
return self.get()
def update_progress(self, update: OnboardingProgressUpdate) -> OnboardingState:
fields = tuple(
field
for field in ("mode_intent", "setup_step", "model_profile_id")
if field in update.model_fields_set
)
if not fields:
state = self.get()
if state.status == "completed":
raise OnboardingCompletedError("onboarding_completed")
return state
mysql = is_mysql(self._store)
placeholder = "%s" if mysql else "?"
assignments = [f"{field} = {placeholder}" for field in fields]
assignments.append(f"updated_at = {placeholder}")
values = [getattr(update, field) for field in fields]
now = datetime.now(UTC).isoformat()
values.append(now)
with self._connect(writing=True) as connection:
execute(
connection,
mysql,
ENSURE_PROGRESS_MYSQL if mysql else ENSURE_PROGRESS_SQLITE,
(now,),
)
updated = execute(
connection,
mysql,
"UPDATE onboarding_state SET "
+ ", ".join(assignments)
+ " WHERE singleton = 1 AND status != 'completed'",
tuple(values),
)
if updated.rowcount != 1:
raise OnboardingCompletedError("onboarding_completed")
return self.get()
def reset(self) -> OnboardingState:
"""Return onboarding to its first-run state, touching nothing else.
@@ -326,53 +207,33 @@ class OnboardingStore:
yield connection
@staticmethod
def _to_state(row: Any, *, mysql: bool) -> OnboardingState:
"""Map SQLite rows and MySQL tuples through the shared database adapter."""
if mysql:
if not isinstance(row, tuple):
raise ValueError("onboarding_row_shape_invalid")
elif not isinstance(row, sqlite3.Row):
raise ValueError("onboarding_row_shape_invalid")
if len(row) != len(ONBOARDING_COLUMNS):
raise ValueError("onboarding_row_shape_invalid")
if mysql:
values = dict(zip(ONBOARDING_COLUMNS, tuple(row), strict=True))
else:
values = row_to_dict(row, ONBOARDING_COLUMNS, mysql=False)
if set(values) != set(ONBOARDING_COLUMNS):
raise ValueError("onboarding_row_shape_invalid")
_as_datetime(values["updated_at"], mysql=mysql, nullable=False)
raw_ids = json.loads(values["selected_source_ids_json"])
def _to_state(row: Any) -> OnboardingState:
"""Read the row by position, so it works for `sqlite3.Row` and tuples.
`SELECT *` makes the column order part of the contract here, which is
why the query is `SELECT *` and the indices are named: relying on
`row["..."]` would only work on SQLite, and relying on position alone
without saying which column is which would be unreadable.
Timestamps are a further engine difference: SQLite hands back the stored
string, MySQL hands back a `datetime`. Accepting both is not padding —
parsing a `datetime` raises, and treating a string as one would be
silently wrong.
"""
# singleton, status, selected_source_ids_json, started_at, completed_at, updated_at
raw_ids = json.loads(row[2])
return OnboardingState(
status=values["status"],
status=row[1],
selected_source_ids=tuple(value for value in raw_ids if isinstance(value, str)),
started_at=_as_datetime(values["started_at"], mysql=mysql),
completed_at=_as_datetime(values["completed_at"], mysql=mysql),
mode_intent=values["mode_intent"],
setup_step=values["setup_step"],
model_profile_id=values["model_profile_id"],
started_at=_as_datetime(row[3]),
completed_at=_as_datetime(row[4]),
)
def _as_datetime(
value: object,
*,
mysql: bool,
nullable: bool = True,
) -> datetime | None:
"""Validate the engine's timestamp shape and normalize it to UTC."""
if value is None:
if not nullable:
raise ValueError("onboarding_datetime_invalid")
def _as_datetime(value: object) -> datetime | None:
"""SQLite stores these as text; MySQL returns `datetime` objects."""
if value is None or value == "":
return None
if mysql:
if not isinstance(value, datetime):
raise ValueError("onboarding_datetime_invalid")
parsed = value
else:
if not isinstance(value, str) or value == "":
raise ValueError("onboarding_datetime_invalid")
parsed = datetime.fromisoformat(value)
if parsed.tzinfo is None:
return parsed.replace(tzinfo=UTC)
return parsed.astimezone(UTC)
if isinstance(value, datetime):
return value
return datetime.fromisoformat(str(value))
@@ -1,123 +0,0 @@
"""Turn-level read projection for SDK raw batches.
HY keeps several complete conversation turns in one L1 raw node so model
writes stay batched. Product readers must not expose that storage batch as
one conversation item: this module gives every contained turn a stable
identity while retaining the parent raw id for traceability.
"""
from __future__ import annotations
import hashlib
import re
from dataclasses import dataclass
from one_memory.hy_sdk_types import HySdkRawRecord, raw_turn_messages
@dataclass(frozen=True)
class RawDisplayTurn:
turn_id: str
turn_index: int
turn_count: int
content: str
messages: tuple[tuple[str, str], ...]
def raw_display_turns(record: HySdkRawRecord) -> tuple[RawDisplayTurn, ...]:
"""Split one stored raw batch into user-opened turns.
A user message opens a new turn; assistant/tool/system messages stay with
that user message until the next user message. Legacy content with no user
marker remains one turn so it never disappears from the product.
"""
messages = raw_turn_messages(record.content)
grouped: list[list[tuple[str, str]]] = []
for message in messages:
role, _ = message
has_user = bool(grouped) and any(item[0] == "user" for item in grouped[-1])
if not grouped or (role == "user" and has_user):
grouped.append([message])
else:
grouped[-1].append(message)
if not grouped:
grouped = [[("", record.content)]]
digest = hashlib.sha256(record.raw_id.encode("utf-8")).hexdigest()
count = len(grouped)
return tuple(
RawDisplayTurn(
turn_id=f"raw-turn:{digest}:{index}",
turn_index=index,
turn_count=count,
content=_turn_content(turn),
messages=tuple(turn),
)
for index, turn in enumerate(grouped)
)
def _turn_content(messages: list[tuple[str, str]]) -> str:
return "\n".join(f"[{role}]: {content}" if role else content for role, content in messages)
def raw_prefix_added(
cached: tuple[tuple[str, str | None], ...],
current: tuple[tuple[str, str | None], ...],
cached_total: int,
current_total: int,
) -> int | None:
"""How many raws were only put on top of a cached newest-first listing.
`cached` and `current` are the (raw id, status) of the first page of each
listing. The answer is a count when the current page is the cached one
with new raws before it and nothing else changed; None when anything else
happened (a delete, a status change, a raw dated into the middle), which
needs a full re-listing.
"""
added = current_total - cached_total
if added <= 0 or added > len(current):
return None
kept = current[added:]
if not kept and cached_total:
return None
if kept != cached[: len(kept)]:
return None
return added
_WORD = re.compile(r"\w+")
def _query_terms(query: str) -> frozenset[str]:
"""Words of the query, plus character bigrams of unspaced (CJK) words."""
terms: set[str] = set()
for word in _WORD.findall(query.casefold()):
terms.add(word)
if not word.isascii() and len(word) > 1:
terms.update(word[index : index + 2] for index in range(len(word) - 1))
return frozenset(terms)
def turns_for_hit(turns: tuple[RawDisplayTurn, ...], query: str) -> tuple[RawDisplayTurn, ...]:
"""The turns one semantic raw hit shows, never none while it has turns.
The SDK embeds a whole raw batch, so a vector hit cannot say which of its
turns matched. Every turn containing the query text is shown; when none
does, the turn sharing the most query terms (the earliest on a tie) stands
for the hit. A semantic hit is never dropped for lacking the literal query.
"""
if not turns:
return ()
needle = query.casefold().strip()
if needle:
lexical = tuple(turn for turn in turns if needle in turn.content.casefold())
if lexical:
return lexical
terms = _query_terms(query)
def overlap(turn: RawDisplayTurn) -> tuple[int, int]:
content = turn.content.casefold()
return (sum(1 for term in terms if term in content), -turn.turn_index)
return (max(turns, key=overlap),)
@@ -11,7 +11,6 @@ instead: plain retrieval over what was imported, the same lane
from __future__ import annotations
import asyncio
import re
import time
from collections.abc import Awaitable, Callable
from datetime import UTC, datetime
@@ -47,50 +46,6 @@ def recall_total_timeout_seconds(timeout_ms: int) -> float:
return min(timeout_ms / 1_000 + RECALL_TOTAL_GRACE_SECONDS, RECALL_TOTAL_CEILING_SECONDS)
def _bounded_raw_excerpt(content: str, query: str, limit: int) -> str:
"""Fit one raw hit without discarding the highest-ranked candidate.
Prefer the literal query, then its longest matching word. A semantic hit
with no lexical anchor keeps its beginning: that is also the part the
embedding service preserves when an input exceeds its own token limit.
"""
if len(content) <= limit:
return content
if limit <= 2:
return content[:limit]
anchor = 0
needle = query.strip()
if needle:
exact = re.search(re.escape(needle), content, re.IGNORECASE)
if exact is not None:
anchor = exact.start()
else:
terms = sorted(re.findall(r"\w+", needle), key=len, reverse=True)
match = next(
(
found
for term in terms
if (found := re.search(re.escape(term), content, re.IGNORECASE)) is not None
),
None,
)
if match is not None:
anchor = match.start()
payload = limit
while True:
start = max(0, min(anchor - payload // 3, len(content) - payload))
end = min(len(content), start + payload)
adjusted = limit - int(start > 0) - int(end < len(content))
if adjusted == payload:
break
payload = adjusted
prefix = "…" if start > 0 else ""
suffix = "…" if end < len(content) else ""
return f"{prefix}{content[start:end]}{suffix}"
async def recall_with_fallback(
request: RecallRequest,
*,
@@ -168,9 +123,6 @@ async def _raw_recall(
)
except TimeoutError:
return receipt(truncated_by=TruncationBudget.TIMEOUT)
current_sources = await raw_sources()
if sources is not None:
sources &= current_sources or frozenset()
hits: list[RecallHit] = []
used = 0
truncated_by: TruncationBudget | None = None
@@ -185,23 +137,13 @@ async def _raw_recall(
truncated_by=truncated_by,
unexamined=len(found) - index,
)
remaining = request.character_budget - used
if remaining <= 0:
truncated_by = TruncationBudget.CHARACTER
return receipt(
tuple(hits),
used=used,
truncated_by=truncated_by,
unexamined=len(found) - index,
)
content = hit.content
if len(content) > remaining:
content = _bounded_raw_excerpt(content, request.query, remaining)
if used + len(hit.content) > request.character_budget:
truncated_by = TruncationBudget.CHARACTER
continue
hits.append(
RecallHit(
memory_id=hit.raw_id,
content=content,
content=hit.content,
kind="raw",
scopes=(),
evidence_ids=(),
@@ -216,5 +158,5 @@ async def _raw_recall(
session_id=hit.session_id,
)
)
used += len(content)
used += len(hit.content)
return receipt(tuple(hits), used=used, truncated_by=truncated_by)
@@ -113,37 +113,6 @@ def inspect_runtime(
)
@dataclass(frozen=True)
class CoreEndpoint:
host: Literal["127.0.0.1"]
port: int
token: str
def core_endpoint(
data_dir: Path,
*,
alive: Callable[[int], bool] = process_is_alive,
) -> CoreEndpoint | None:
"""Where this home's Core listens and the token it accepts, from files alone.
The receipt and token checks `inspect_runtime` makes, without its identity
round trip: a caller that only has to hand something over quickly lets the
authenticated request itself be the proof. None when no live Core is named.
"""
try:
raw = (data_dir / "runtime.json").read_text(encoding="utf-8")
receipt = _receipt(json.loads(raw))
except (OSError, UnicodeError, json.JSONDecodeError):
return None
if receipt is None or not alive(receipt.pid):
return None
token = _runtime_token(data_dir)
if token is None:
return None
return CoreEndpoint(host=receipt.host, port=receipt.port, token=token)
def _receipt(value: object) -> RuntimeReceipt | None:
if not isinstance(value, dict):
return None
@@ -25,7 +25,7 @@ from one_memory.hy_runtime_config import HY_MEMORY_READER
from one_memory.hy_sdk_runtime import SdkClientHandle, sdk_relevance_floor
from one_memory.hy_sdk_types import HySdkError, parse_sdk_search
from one_memory.memory_queue import ExtractionWindowPreset, QueueDepths
from one_memory.memory_server.durable_queue import WAITING_FOR_MODEL, DurableWriteQueue
from one_memory.memory_server.durable_queue import DurableWriteQueue
from one_memory.sdk_write_processor import SdkWriteCoordinator
from one_memory.semantic_deadline import SemanticMemoryHit
@@ -309,10 +309,7 @@ class SdkMemoryRuntime:
return []
async def resume_allowed(self) -> None:
await asyncio.to_thread(
self.write_queue.readmit_parked,
states=(WAITING_FOR_MODEL,),
)
await asyncio.to_thread(self.write_queue.readmit_parked)
def accepted_backfill_status(self) -> None:
return None
@@ -288,3 +288,4 @@ def verify_readback(
productivity_readable=sum(1 for memory_id in legacy.productivity if readable(memory_id)),
chat_missing_before_migration=chat_missing_before_migration,
)
@@ -94,11 +94,6 @@ class ActiveProfileReadHandle(SdkClientHandle):
handle = await self.current_handle()
return self._guarded(handle, await handle.client())
async def client_for_mode(self, mode: str) -> Any:
"""Build/read with the caller's one mode snapshot, without rereading it."""
handle = await self.current_handle(mode=mode)
return self._guarded(handle, await handle.client())
def _guarded(self, handle: SdkClientHandle, client: Any) -> Any:
"""A remote embedding's searches report `embedding_unavailable` (no fallback)."""
profile = getattr(handle, "profile", None)
@@ -116,15 +111,15 @@ class ActiveProfileReadHandle(SdkClientHandle):
),
)
async def current_handle(self, *, mode: str | None = None) -> SdkClientHandle:
async def current_handle(self) -> SdkClientHandle:
"""The handle of the live space, held for the asking task."""
space, identity = await in_sdk_read(self._resolve)
selected_mode, features = mode or self.memory_mode(), self.current_features()
mode, features = self.memory_mode(), self.current_features()
language = self.current_language()
key = (
space.space_id if space is not None else "",
*identity,
selected_mode,
mode,
features,
language,
)
@@ -133,9 +128,7 @@ class ActiveProfileReadHandle(SdkClientHandle):
if handle is None:
# Only on a change of space or profile: resolving a key can
# reach the system keychain, so it is not done on every read.
handle = await in_sdk_read(
self._build, space, identity, selected_mode, features, language
)
handle = await in_sdk_read(self._build, space, identity, mode, features, language)
self._handles[key] = handle
self._retired.update(value for name, value in self._handles.items() if name != key)
self._handles = {key: handle}
@@ -23,12 +23,11 @@ from one_memory.hy_sdk_runtime import (
SdkClientHandle,
builtin_retrieval_snapshot,
language_kwargs,
retrieval_view,
)
from one_memory.imports.models import EvidenceContentKind, EvidenceOrigin, EvidenceRecord
from one_memory.input_sanitization import sanitize_model_input
from one_memory.memory_features import FeatureOverrides
from one_memory.model_profiles import ModelProfileSnapshot, ModelProfileStore, ProviderKind
from one_memory.model_profiles import ModelProfileSnapshot, ModelProfileStore
from one_memory.secrets import SecretStore, SecretStoreUnavailableError
PRODUCT_AGENT_ID = "all-apps"
@@ -113,7 +112,7 @@ class SdkWriteProcessor:
records: tuple[EvidenceRecord, ...],
*,
request_id: str,
before_hy_invocation: Callable[[bool], Awaitable[None]],
before_hy_invocation: Callable[[], Awaitable[None]],
) -> SdkWriteReceipt:
if not records:
raise RuntimeError("hy_memory_task_without_inputs")
@@ -140,7 +139,7 @@ class SdkWriteProcessor:
if extracted and not self._lite:
contents = [sanitize_model_input(record.verbatim) for record in records]
await before_hy_invocation(True)
await before_hy_invocation()
result = await _write(
request_id,
client.add_extracted,
@@ -168,7 +167,7 @@ class SdkWriteProcessor:
]
if not messages:
raise RuntimeError("hy_memory_task_without_inputs")
await before_hy_invocation(not self._lite)
await before_hy_invocation()
result = await _write(
request_id,
client.add,
@@ -284,8 +283,10 @@ class SdkWriteCoordinator:
records: tuple[EvidenceRecord, ...],
*,
request_id: str,
before_hy_invocation: Callable[[bool], Awaitable[None]],
before_hy_invocation: Callable[[], Awaitable[None]],
) -> SdkWriteReceipt:
if not self.sedimentation_enabled():
raise RuntimeError("model_sedimentation_disabled")
if not self.writes_open():
# An embedding switch paused writes: the drain puts the job back.
raise RuntimeError("embedding_switch_in_progress")
@@ -296,8 +297,9 @@ class SdkWriteCoordinator:
self._writing[user_id] = self._writing.get(user_id, 0) + 1
try:
async with self._active_handle() as handle:
lite = getattr(handle, "mode", None) == "lite"
return await SdkWriteProcessor(handle, lite=lite).process_sdk_job(
return await SdkWriteProcessor(
handle, lite=getattr(handle, "mode", None) == "lite"
).process_sdk_job(
records,
request_id=request_id,
before_hy_invocation=before_hy_invocation,
@@ -464,27 +466,24 @@ class SdkWriteCoordinator:
return (builtin.profile_id, builtin.revision), builtin, None, space
identity = (space.profile_id, space.profile_revision)
profile = self.profiles.snapshot(*identity)
view = retrieval_view(profile)
secret: str | None = None
if view.embedding_provider_kind is ProviderKind.OPENAI_COMPATIBLE:
try:
secret = self.profiles.resolve_snapshot_secret(*identity, self.secrets)
except SecretStoreUnavailableError:
raise RuntimeError("secure_storage_unavailable") from None
if secret is None:
raise RuntimeError("model_secret_unavailable")
return identity, view, secret, space
try:
secret = self.profiles.resolve_snapshot_secret(*identity, self.secrets)
except SecretStoreUnavailableError:
raise RuntimeError("secure_storage_unavailable") from None
if profile.has_api_key and secret is None:
raise RuntimeError("model_secret_unavailable")
return identity, profile, secret, space
def _read_active_credentials(
self,
mode: str = "pro",
) -> tuple[tuple[str, int], ModelProfileSnapshot, str | None, EmbeddingSpace | None]:
if mode == "lite":
# A lite write is an embedding-only view of the live space. Never
# resolve or inject the generation model endpoint or its secret.
return self._lite_credentials()
identity = self.profiles.active_identity()
if identity is None:
if mode == "lite":
# lite stores raw turns with one embedding and calls no model:
# the live space's own, so they land where reads look.
return self._lite_credentials()
raise RuntimeError("model_profile_not_active")
profile = self.profiles.snapshot(*identity)
space = self._live_space() if self.spaces is not None else None
@@ -1,283 +0,0 @@
"""Stable UI projections over the HY SDK's mode-specific storage shapes."""
from __future__ import annotations
import asyncio
import re
from datetime import UTC, datetime
from typing import Any, Literal, cast
from pydantic import BaseModel, ConfigDict, Field
from one_memory.hy_sdk_types import (
HySdkMemoryPage,
HySdkMemoryRecord,
HySdkRawPage,
HySdkRawRecord,
raw_turn_messages,
)
from one_memory.imports.sources.parsing import unwrap_host_envelope_with_positions
MemoryMode = Literal["lite", "pro", "ultra"]
TimeBasis = Literal["conversation", "import", "write", "update"]
ProjectionDisplayState = Literal["empty", "excerpt", "no_user_excerpt", "no_displayable_memory"]
RAW_STATUSES = ("active", "shadow")
HOME_PROJECTION_CANDIDATES = 10
HOME_EXCERPT_CODEPOINTS = 200
_SPACE = re.compile(r"\s+")
_CONTROL_ONLY = re.compile(
r"^(?:"
r"<<?\s*(?:image|attachment|system[_-]?reminder|no[_ -]?content)\s*>>?"
r"|\[\s*(?:image|attachment|system[_ -]?reminder|no[_ -]?content)\s*\]"
r")$",
re.IGNORECASE,
)
class HomeProjectionCapabilities(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
content_plane: Literal["raw", "memory"]
generation_required: bool
graph_enabled: bool
evolve_available: bool
class _HomeLatest(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
time: datetime | None
time_basis: TimeBasis | None
excerpt: str
truncated: bool
source_id: str
class HomeRawLatest(_HomeLatest):
type: Literal["raw"] = "raw"
raw_id: str
session_id: str
display_state: Literal["excerpt", "no_user_excerpt"]
class HomeMemoryLatest(_HomeLatest):
type: Literal["memory"] = "memory"
memory_id: str
scene: Literal["chat", "working", "coding"]
# scene=all lists chat rows and productivity experiences; facts are not
# part of that page or its total.
kind: Literal["chat", "experience"]
variant: Literal["working", "coding"] | None
class HomeProjectionV1(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
projection_version: Literal[1] = 1
user_id: str
effective_mode: MemoryMode
capabilities: HomeProjectionCapabilities
visible_count: int = Field(ge=0)
display_state: ProjectionDisplayState
latest: HomeRawLatest | HomeMemoryLatest | None
observed_at: datetime
def raw_list_filters(statuses: list[str] | tuple[str, ...] | None = None) -> dict[str, list[str]]:
"""The one kept-raw definition shared by `/raw` and the home projection."""
return {
"layers": ["l1_raw"],
"statuses": list(statuses or RAW_STATUSES),
}
def home_capabilities(mode: MemoryMode) -> HomeProjectionCapabilities:
return HomeProjectionCapabilities(
content_plane="raw" if mode == "lite" else "memory",
generation_required=mode != "lite",
graph_enabled=mode == "ultra",
evolve_available=mode == "ultra",
)
def raw_user_excerpt(content: str, *, limit: int = HOME_EXCERPT_CODEPOINTS) -> tuple[str, bool]:
"""Return only displayable user text from one SDK raw turn.
Role selection happens before wrapper removal, so assistant/system content
can never become a homepage excerpt after sanitization.
"""
candidates: list[str] = []
for role, text in raw_turn_messages(content):
if role != "user":
continue
envelope = unwrap_host_envelope_with_positions(text)
if envelope.status == "ambiguous":
# An uncertain host envelope may contain system context. Showing
# nothing is safer than guessing which bytes the person authored.
continue
cleaned = envelope.text or ""
cleaned = _SPACE.sub(" ", cleaned).strip()
if not cleaned or _CONTROL_ONLY.fullmatch(cleaned):
continue
candidates.append(cleaned)
joined = " ".join(candidates)
return joined[:limit], len(joined) > limit
def _epoch(value: int | None) -> datetime | None:
return datetime.fromtimestamp(value, tz=UTC) if value is not None else None
def _raw_time(record: HySdkRawRecord) -> tuple[datetime | None, TimeBasis | None]:
if record.memory_at is not None:
return _epoch(record.memory_at), "conversation"
if record.gmt_created is not None:
return _epoch(record.gmt_created), "import"
return None, None
def _memory_time(record: HySdkMemoryRecord) -> tuple[datetime | None, TimeBasis | None]:
# Exactly the scene=all merge key. `memory_at` is event time and does not
# decide which row list_memories returns first.
if record.sort_updated_at is not None:
return _epoch(record.sort_updated_at), "update"
written = record.gmt_created if record.scene == "chat" else record.created_at
if written is not None:
return _epoch(written), "write"
return None, None
def _bounded(text: str) -> tuple[str, bool]:
flattened = _SPACE.sub(" ", text).strip()
return flattened[:HOME_EXCERPT_CODEPOINTS], len(flattened) > HOME_EXCERPT_CODEPOINTS
def project_raw_latest(records: tuple[HySdkRawRecord, ...]) -> HomeRawLatest | None:
if not records:
return None
for record in records:
excerpt, truncated = raw_user_excerpt(record.content)
if excerpt:
at, basis = _raw_time(record)
return HomeRawLatest(
raw_id=record.raw_id,
session_id=record.session_id,
display_state="excerpt",
time=at,
time_basis=basis,
excerpt=excerpt,
truncated=truncated,
source_id=record.source,
)
newest = records[0]
at, basis = _raw_time(newest)
return HomeRawLatest(
raw_id=newest.raw_id,
session_id=newest.session_id,
display_state="no_user_excerpt",
time=at,
time_basis=basis,
excerpt="",
truncated=False,
source_id=newest.source,
)
def project_memory_latest(
records: tuple[HySdkMemoryRecord, ...],
) -> HomeMemoryLatest | None:
record = next(
(
item
for item in records
if item.kind in {"chat", "experience"}
and (
(item.kind == "chat" and item.scene == "chat" and item.variant is None)
or (
item.kind == "experience"
and item.scene in {"working", "coding"}
and item.variant == item.scene
)
)
),
None,
)
if record is None:
return None
excerpt, truncated = _bounded(record.content)
at, basis = _memory_time(record)
return HomeMemoryLatest(
memory_id=record.memory_id,
scene=record.scene,
kind=cast(Literal["chat", "experience"], record.kind),
variant=record.variant,
time=at,
time_basis=basis,
excerpt=excerpt,
truncated=truncated,
source_id=record.source or "",
)
async def read_home_projection(
client: Any,
*,
user_id: str,
mode: MemoryMode,
) -> HomeProjectionV1:
"""Read one authoritative projection; any SDK/shape failure escapes."""
client_mode = getattr(client, "mode", mode)
if client_mode != mode:
raise RuntimeError("home_projection_mode_mismatch")
if mode == "lite":
raw = await asyncio.to_thread(
client.list_memories,
user_id=user_id,
**raw_list_filters(),
limit=HOME_PROJECTION_CANDIDATES,
offset=0,
order="desc",
)
raw_page = HySdkRawPage.from_memory_list(raw, limit=HOME_PROJECTION_CANDIDATES, offset=0)
visible_count = raw_page.total
raw_latest = project_raw_latest(raw_page.raw)
latest: HomeRawLatest | HomeMemoryLatest | None = raw_latest
display_state: ProjectionDisplayState = (
"empty"
if visible_count == 0
else raw_latest.display_state
if raw_latest is not None
else "no_user_excerpt"
)
else:
raw = await asyncio.to_thread(
client.list_memories,
user_id=user_id,
scene="all",
statuses=["active"],
limit=HOME_PROJECTION_CANDIDATES,
offset=0,
order="desc",
)
memory_page, _skipped = HySdkMemoryPage.from_sdk_skipping_invalid(
raw, limit=HOME_PROJECTION_CANDIDATES, offset=0
)
visible_count = memory_page.total
latest = project_memory_latest(memory_page.memories)
display_state = (
"empty"
if visible_count == 0
else "excerpt"
if latest is not None
else "no_displayable_memory"
)
return HomeProjectionV1(
user_id=user_id,
effective_mode=mode,
capabilities=home_capabilities(mode),
visible_count=visible_count,
display_state=display_state,
latest=latest,
observed_at=datetime.now(UTC),
)
+2 -5
View File
@@ -111,7 +111,6 @@
| `test_health.py` | `contract` | 覆盖 a backfill that fails says so rather than disappearing、a failed catch up leaves a traceback behind、configured model without content consent does not start backfill 等 15 个场景。 |
| `test_hermes_gateway_injection.py` | `contract` | Hermes gateway text in `user` rows: notices are skipped, relayed messages unwrapped. |
| `test_history_listing_rotation.py` | `contract` | 覆盖 history listing alternates started and untouched files、history listing drains whichever cohort is longer 场景。 |
| `test_home_projection.py` | `contract` | 覆盖 memory projection covers pro and ultra、memory projection preserves productivity namespace、project raw latest uses import time without conversation time 等 7 个场景。 |
| `test_host_capabilities.py` | `unit` | A plugin does not become realtime on every host or every unknown version. |
| `test_hourly_c1_peak_replay_contract.py` | `contract` | Guard the package peak-replay probe's error accounting. |
| `test_hourly_c1_recall_stage_probe_contract.py` | `contract` | Guard the packaged recall stage probe's command and error semantics. |
@@ -123,7 +122,6 @@
| `test_import_api.py` | `integration` | 覆盖 resume during cleanup returns conflict without internal error、authorize detect import and revoke api flow、protected api rejects missing token and foreign origin 等 5 个场景。 |
| `test_import_control_races_review.py` | `contract` | Read-only review repro: real runner/store, synthetic delayed adapter cleanup. |
| `test_import_evidence_atomicity.py` | `contract` | 覆盖 counter failure rolls back evidence versions and cursor、same identity cursor rewind is blocked and logged、replaced identity may reset checkpoint 等 20 个场景。 |
| `test_import_job_retention.py` | `integration` | `import_jobs` stays bounded, and the questions asked of it are SQL, not scans. |
| `test_import_oversized_verbatim.py` | `contract` | 覆盖 normal verbatim is unchanged、oversized verbatim is truncated instead of rejected 场景。 |
| `test_import_runner.py` | `contract` | 覆盖 job transitions are explicit、runner keeps per record errors and continues、sync slices reuse the same source listing 等 22 个场景。 |
| `test_import_skip_batching.py` | `contract` | 覆盖 batch limit and tail use fewer commits、batched skips keep their order against the next write、flushes before every nonmatching event 等 14 个场景。 |
@@ -138,7 +136,7 @@
| `test_internal_preset.py` | `contract` | The optional model preset a configured build carries. |
| `test_jsonl_snapshot_tail.py` | `contract` | Bounded EOF handling on synthetic files only, never original host history. |
| `test_library_search_window.py` | `unit` | The library search window store and the unified record/keyword parsing. |
| `test_live_capture.py` | `integration` | Hooks hand visible text to Core, which stages it once, without Evidence. |
| `test_live_capture.py` | `integration` | Visible hook text lands in Evidence before the transcript file is flushed. |
| `test_live_sync.py` | `contract` | 覆盖 live sync preempts a reader batch before it emits work、live sync cuts in front of a running history import、live sync does not resume a history import the user paused 等 24 个场景。 |
| `test_live_sync_file_incremental.py` | `contract` | 覆盖 unchanged file metadata avoids a job and content read、changed file content is read once by the import job、workbuddy first live poll baselines history then imports only append 等 6 个场景。 |
| `test_live_sync_job_coalescing.py` | `contract` | A poll that finds nothing must not leave an import job behind. |
@@ -214,7 +212,6 @@
| `test_onboarding_profile.py` | `integration` | R-09:首次安装引导把画像录下来,不问用户一句话。 |
| `test_oversized_file_guard.py` | `contract` | One pass over a source file is bounded by a byte budget, not skipped. |
| `test_parent_watchdog.py` | `integration` | A Core that outlives its launcher makes the next launch impossible. |
| `test_pinned_sdk_raw_search.py` | `integration` | MemHub's raw search against the hy-memory wheel it pins, not the repo source. |
| `test_policy_api_wiring.py` | `integration` | 覆盖 global model permission off does not backfill、backfill puts history on write jobs without scheduling memory tasks、source pause skips history until the source is reenabled 场景。 |
| `test_policy_production_entry.py` | `contract` | 覆盖 production serve passes only supported app options、maintenance serve does not own the runtime 场景。 |
| `test_precipitation_concurrency_setting.py` | `contract` | Rule 2/9: precipitation concurrency is configurable, not hardcoded to 1. |
@@ -323,4 +320,4 @@
| `test_workspace_recall.py` | `contract` | Workspace memories were unreachable from every host; these pin the way in. |
| `test_write_queue_lease_renewal.py` | `contract` | Rule 8: a lease must outlive the work it covers, and be renewed while it runs. |
共 318 个测试文件。
共 315 个测试文件。
@@ -14,8 +14,8 @@ from pathlib import Path
import pytest
PRIMARY_MARKERS = ("unit", "contract", "integration", "performance", "live")
EXPECTED_TEST_FILE_COUNT = 318
EXPECTED_TEST_FILES_SHA256 = "df8752566a0bf9e540f68b4948f3f5dbc409bd6836707b1330206e2349d70940"
EXPECTED_TEST_FILE_COUNT = 315
EXPECTED_TEST_FILES_SHA256 = "681258b2a13ca852e273707071b286afe6b0b2fe537f90a8a2c64d83ec662772"
# Pure transformations, validation models, parity checks and state machines.
# This list is intentionally small: uncertain files default to contract.
@@ -70,7 +70,6 @@ INTEGRATION_FILES = frozenset(
"test_file_import_operations.py",
"test_graph_width_endpoint.py",
"test_import_api.py",
"test_import_job_retention.py",
"test_live_capture.py",
"test_live_watch_grants.py",
"test_log_privacy.py",
@@ -94,7 +93,6 @@ INTEGRATION_FILES = frozenset(
"test_model_profile_context_reset.py",
"test_onboarding_profile.py",
"test_parent_watchdog.py",
"test_pinned_sdk_raw_search.py",
"test_policy_api_wiring.py",
"test_recall_scope_control.py",
"test_retrieval_lifespan.py",
+2 -1
View File
@@ -39,7 +39,8 @@ def kuzu_graph_with_embedding_width(db_path: Path, width: int) -> None:
database = kuzu.Database(str(db_path))
connection = kuzu.Connection(database)
connection.execute(
f"CREATE NODE TABLE Memory(node_id STRING, embedding FLOAT[{width}], PRIMARY KEY(node_id))"
f"CREATE NODE TABLE Memory("
f"node_id STRING, embedding FLOAT[{width}], PRIMARY KEY(node_id))"
)
del connection
del database
@@ -29,6 +29,7 @@ from one_memory.imports.sources.registry import (
build_source_catalog,
)
from one_memory.imports.supervisor import ImportSupervisor
from one_memory.live_capture import capture_hook
FREEZE_THRESHOLD_SECONDS = 1.0
@@ -227,17 +228,31 @@ async def _run(
hook_text = "realtime hook while history import is running"
hook_started = time.monotonic()
# A hook hands its turn to Core over loopback; it no longer writes the
# database itself. What is measured is that hand-off under import load.
handed = await client.post(
"/api/v1/live-capture/workbuddy",
json={
result["hook_accepted"] = await asyncio.to_thread(
capture_hook,
"workbuddy",
{
"hook_event_name": "UserPromptSubmit",
"session_id": "pressure-live-session",
"prompt": hook_text,
},
app.state.evidence_store.database_path,
)
result["hook_accepted"] = handed.status_code == 202
hook_deadline = hook_started + 10
hook_stored = False
while time.monotonic() < hook_deadline:
with sqlite3.connect(app.state.evidence_store.database_path) as connection:
hook_stored = (
connection.execute(
"SELECT 1 FROM evidence WHERE source_id='workbuddy' AND verbatim=?",
(hook_text,),
).fetchone()
is not None
)
if hook_stored:
break
await asyncio.sleep(0.01)
result["hook_stored"] = hook_stored
result["hook_visible_ms"] = round((time.monotonic() - hook_started) * 1000, 1)
during, during_unavailable, sampled_terminal = await _measure_recall_until(
@@ -104,7 +104,9 @@ SDK_PRIVATE_ATTRIBUTES = frozenset(
}
)
SDK_PRIVATE_ACCESS_EXCEPTIONS = {
f"{PACKAGE_LABEL}/hy_graph_projection.py": frozenset({"_graph_store", "_loop_thread", "_run"}),
f"{PACKAGE_LABEL}/hy_graph_projection.py": frozenset(
{"_graph_store", "_loop_thread", "_run"}
),
# Startup migration runs while Runtime writers are stopped. It is the one
# non-runtime path allowed to manipulate SDK storage directly so that
# migration-only mechanics never become public HyMemoryClient methods.
@@ -264,7 +266,9 @@ def test_no_sdk_private_implementation_access() -> None:
)
}
if used:
unexpected = used - SDK_PRIVATE_ACCESS_EXCEPTIONS.get(relative, frozenset())
unexpected = used - SDK_PRIVATE_ACCESS_EXCEPTIONS.get(
relative, frozenset()
)
if unexpected:
offenders.append(f"{relative}: {sorted(unexpected)}")
assert not offenders, (
@@ -287,7 +291,9 @@ def test_sdk_modules_import_only_the_public_package_root() -> None:
):
offenders.append(relative)
break
if isinstance(node, ast.ImportFrom) and (node.module or "").startswith("hy_memory."):
if isinstance(node, ast.ImportFrom) and (
node.module or ""
).startswith("hy_memory."):
offenders.append(relative)
break
if (
@@ -131,24 +131,3 @@ def test_run_as_the_entry_it_ends_itself_while_still_importing(tmp_path) -> None
_reap([process])
assert process.returncode == -signal.SIGALRM
assert time.monotonic() - started < 10
def test_with_no_core_a_real_capture_exits_at_once_and_writes_nothing(tmp_path) -> None:
# The process a host actually starts: no Core is running for this home,
# so the turn is dropped. Nothing may fall back to writing the database.
data_dir = tmp_path / "data"
env = {
key: value for key, value in os.environ.items() if key != "ONE_MEMORY_CAPTURE_DEADLINE_S"
}
env.update(HOME=str(tmp_path), ONE_MEMORY_DATA_DIR=str(data_dir))
payload = b'{"hook_event_name": "UserPromptSubmit", "session_id": "s", "prompt": "hi"}'
result = subprocess.run(
[sys.executable, "-m", "one_memory.cli", "capture", "workbuddy"],
input=payload,
capture_output=True,
env=env,
timeout=60,
check=False,
)
assert (result.returncode, result.stdout, result.stderr) == (0, b"", b"")
assert not (data_dir / "one-memory.sqlite3").exists()
+13 -20
View File
@@ -16,8 +16,6 @@ def database(tmp_path):
path = tmp_path / "context.sqlite3"
with closing(sqlite3.connect(path)) as db, db:
db.execute("CREATE TABLE source_grants(user_id TEXT, source_id TEXT, revoked_at TEXT)")
db.execute("INSERT INTO source_grants VALUES('local','cursor',NULL)")
db.execute("INSERT INTO source_grants VALUES('local','codex',NULL)")
db.execute("INSERT INTO source_grants VALUES('local','revoked','2026-09-19')")
db.execute("INSERT INTO source_grants VALUES('other','cursor','2026-09-19')")
return path
@@ -45,14 +43,16 @@ def test_memory_only_never_reads_raw_authority(database, monkeypatch):
assert not result.raw_allowed and result.raw_sources == frozenset()
def test_all_apps_raw_reads_every_connected_source_and_removes_revoked_sources(database):
result = resolve(database, access(["om_conversations"], ["all-apps", "source:cursor"]))
assert result.raw_allowed and result.raw_sources == frozenset({"cursor", "codex"})
def test_raw_permission_does_not_grant_memory_and_removes_revoked_sources(database):
result = resolve(
database, access(["om_conversations"], ["all-apps", "source:cursor", "source:revoked"])
)
assert result.raw_allowed and result.raw_sources == frozenset({"cursor"})
assert not result.memory_allowed
@pytest.mark.parametrize("scopes", [["source:cursor"], []])
def test_raw_requires_all_apps(database, scopes):
@pytest.mark.parametrize("scopes", [["source:cursor"], ["all-apps"], []])
def test_raw_requires_all_apps_and_source(database, scopes):
with pytest.raises(HTTPException) as error:
resolve(database, access(["om_conversations"], scopes))
assert error.value.status_code == 403
@@ -80,7 +80,9 @@ def test_explicit_raw_filter_must_be_currently_authorized(database, source):
with pytest.raises(HTTPException) as error:
resolve(
database,
access(["om_recall", "om_conversations"], ["all-apps"]),
access(
["om_recall", "om_conversations"], ["all-apps", "source:cursor", "source:revoked"]
),
raw_source_ids=(source,),
)
assert error.value.status_code == 403
@@ -102,19 +104,10 @@ def test_filters_do_not_change_memory_scope(database):
assert result.raw_sources == frozenset({"cursor"})
def test_explicit_raw_filter_accepts_any_connected_source_under_all_apps(database):
def test_implicit_revoked_raw_path_can_be_reported_beside_allowed_memory(database):
result = resolve(
database,
access(["om_conversations"], ["all-apps", "source:cursor"]),
raw_source_ids=("codex",),
database, access(["om_recall", "om_conversations"], ["all-apps", "source:revoked"])
)
assert result.raw_allowed and result.raw_sources == frozenset({"codex"})
def test_no_connected_raw_source_can_be_reported_beside_allowed_memory(database):
with closing(sqlite3.connect(database)) as db, db:
db.execute("UPDATE source_grants SET revoked_at='2026-09-20' WHERE user_id='local'")
result = resolve(database, access(["om_recall", "om_conversations"], ["all-apps"]))
assert result.memory_allowed and not result.raw_allowed
@@ -131,7 +124,7 @@ def test_expired_deadline_does_not_open_database(database, monkeypatch):
with pytest.raises(TimeoutError):
resolve_context_authority(
ContextSearchRequest(query="x"),
access(["om_conversations"], ["all-apps"]),
access(["om_conversations"], ["all-apps", "source:cursor"]),
database,
deadline=time.monotonic() - 1,
)
@@ -53,8 +53,8 @@ def test_external_marker_is_not_an_installation_grant(tmp_path):
"/api/v2/memories/ask", json={"question": "history"}, headers=headers()
)
assert denied_tool.status_code == 403
# om_conversations reaches Core only as the v2 raw lane. all-apps
# has nothing to open because this fresh home has no connected source.
# om_conversations reaches Core only as the v2 raw lane, and only
# with a source grant: all-apps alone opens no raw source.
no_raw = await client.post(
"/api/v2/context/search",
json={"query": "history", "kinds": ["raw"]},
+1 -12
View File
@@ -150,18 +150,7 @@ def test_lifespan_content_backfill_failure_is_visible(tmp_path, monkeypatch) ->
async def scenario() -> None:
async with app.router.lifespan_context(app):
# The rebuild is released when post-ready recovery finishes, and
# recovery now resumes imports off the loop, so it can still be
# running at the first request.
for _ in range(500):
if app.state.recovery in {"done", "failed"}:
break
await asyncio.sleep(0.01)
for _ in range(500):
health = await _get_json(app, "/api/v1/health")
if health["content_candidate_backfill"] != "running":
break
await asyncio.sleep(0.01)
health = await _get_json(app, "/api/v1/health")
assert health["content_candidate_backfill"] == "failed"
assert health["content_candidate_backfill_failure"] == "database_locked"
@@ -1,397 +0,0 @@
from __future__ import annotations
import asyncio
import pytest
from fastapi.testclient import TestClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.hy_sdk_types import HySdkReadError
from one_memory.ui_projection import (
HOME_PROJECTION_CANDIDATES,
project_raw_latest,
raw_user_excerpt,
read_home_projection,
)
def raw_row(
raw_id: str,
content: str,
*,
status: str = "active",
memory_at: int | None = 1_700_000_000,
created: int | None = 1_699_000_000,
) -> dict[str, object]:
return {
"memory_id": raw_id,
"content": content,
"layer": "l1_raw",
"status": status,
"user_id": "local",
"agent_id": "all-apps",
"session_id": f"session-{raw_id}",
"source": "cursor",
"memory_at": memory_at,
"gmt_created": created,
"custom": {"private": "not projected"},
}
def chat_row(memory_id: str = "m-1") -> dict[str, object]:
return {
"memory_id": memory_id,
"content": "The user ships on Friday",
"layer": "l2_fact",
"status": "active",
"is_latest": True,
"user_id": "local",
"agent_id": "all-apps",
"session_id": "s-1",
"source": "cursor",
"memory_at": 1_700_000_000,
"gmt_created": 1_699_000_000,
"updated_at": 1_800_000_000,
"scene": "chat",
"kind": "chat",
}
class ProjectionSdk:
def __init__(self, rows: list[dict[str, object]], *, total: int | None = None) -> None:
self.rows = rows
self.total = len(rows) if total is None else total
self.calls: list[dict[str, object]] = []
def list_memories(self, **kwargs):
self.calls.append(kwargs)
if kwargs.get("layers") == ["l1_raw"]:
return {"vdb": {"memories": self.rows, "total": self.total}}
return {
"scene": "all",
"memories": self.rows,
"total": self.total,
"totals": {"chat": len(self.rows), "working": 0, "coding": 0},
"errors": {},
}
def memory_stats(self, **kwargs):
raise AssertionError("home projection must use one scene=all snapshot")
def test_raw_excerpt_keeps_only_user_query_and_truncates_by_codepoint() -> None:
text = (
"[user]: <system_reminder>ignore this</system_reminder>\n"
"<user_info>private host data</user_info>\n"
"<user_query>短句 https://example.com/a /tmp/x " + "🙂" * 220 + "</user_query>\n"
"[assistant]: secret answer"
)
excerpt, truncated = raw_user_excerpt(text)
assert excerpt.startswith("短句 https://example.com/a /tmp/x ")
assert "ignore" not in excerpt and "private" not in excerpt and "secret" not in excerpt
assert len(excerpt) == 200
assert truncated is True
def test_raw_excerpt_fails_closed_on_nested_host_wrappers() -> None:
assert raw_user_excerpt(
"[user]: <system_reminder><user_query>keep me</user_query></system_reminder>"
) == ("", False)
@pytest.mark.parametrize(
"wrapper",
[
"system_reminder",
"system-reminder",
"user_info",
"attached_files",
"image_local_path",
"agent_transcripts",
"rules",
"additional_data",
"working_memory_reminder",
],
)
def test_raw_excerpt_uses_shared_envelope_parser_for_host_context(wrapper) -> None:
excerpt, truncated = raw_user_excerpt(
f"[user]: <{wrapper}>private</{wrapper}><user_query>keep me</user_query>"
)
assert (excerpt, truncated) == ("keep me", False)
@pytest.mark.parametrize(
"text",
[
"[user]: <system_reminder>unclosed <user_query>do not guess</user_query>",
"[user]: <user_query>unclosed",
"[user]: <rules><rules>nested</rules><user_query>ambiguous</user_query></rules>",
],
)
def test_raw_excerpt_fails_closed_on_ambiguous_envelopes(text) -> None:
assert raw_user_excerpt(text) == ("", False)
def test_raw_projection_uses_shadow_rows_and_k10_without_leaking_payload() -> None:
sdk = ProjectionSdk(
[
raw_row("r-1", "[assistant]: no user here", status="shadow"),
raw_row("r-2", "[user]: deploy Friday", status="active"),
],
total=42,
)
result = asyncio.run(read_home_projection(sdk, user_id="local", mode="lite"))
assert result.visible_count == 42
assert result.latest is not None and result.latest.type == "raw"
assert result.latest.raw_id == "r-2"
assert result.latest.excerpt == "deploy Friday"
assert result.latest.time_basis == "conversation"
assert result.display_state == "excerpt"
assert sdk.calls == [
{
"user_id": "local",
"layers": ["l1_raw"],
"statuses": ["active", "shadow"],
"limit": HOME_PROJECTION_CANDIDATES,
"offset": 0,
"order": "desc",
}
]
assert set(result.latest.model_dump()) == {
"type",
"raw_id",
"session_id",
"display_state",
"time",
"time_basis",
"excerpt",
"truncated",
"source_id",
}
serialized = result.model_dump(mode="json")
assert serialized["projection_version"] == 1
assert serialized["latest"]["time"].endswith("Z")
assert "content" not in serialized["latest"]
def test_raw_projection_reports_no_user_excerpt_when_k_has_none() -> None:
sdk = ProjectionSdk([raw_row("r-1", "[assistant]: answer")])
result = asyncio.run(read_home_projection(sdk, user_id="local", mode="lite"))
assert result.latest is not None and result.latest.type == "raw"
assert result.latest.raw_id == "r-1"
assert result.latest.display_state == "no_user_excerpt"
assert result.latest.excerpt == ""
@pytest.mark.parametrize(
("mode", "graph", "evolve"),
[("pro", False, False), ("ultra", True, True)],
)
def test_memory_projection_covers_pro_and_ultra(mode, graph, evolve) -> None:
sdk = ProjectionSdk([chat_row()], total=9)
result = asyncio.run(read_home_projection(sdk, user_id="local", mode=mode))
assert result.visible_count == 9
assert result.display_state == "excerpt"
assert result.capabilities.content_plane == "memory"
assert result.capabilities.graph_enabled is graph
assert result.capabilities.evolve_available is evolve
assert result.latest is not None and result.latest.type == "memory"
assert result.latest.memory_id == "m-1"
assert result.latest.time_basis == "update"
assert result.latest.time.isoformat() == "2027-01-15T08:00:00+00:00"
def test_chat_projection_prefers_gmt_modified_when_the_sdk_supplies_it() -> None:
row = {**chat_row(), "gmt_modified": 1_810_000_000}
result = asyncio.run(read_home_projection(ProjectionSdk([row]), user_id="local", mode="pro"))
assert result.latest is not None
assert result.latest.time_basis == "update"
assert result.latest.time.timestamp() == 1_810_000_000
def test_memory_projection_uses_productivity_updated_at() -> None:
row = {
"kind": "experience",
"experience_id": "e-1",
"task": "Release",
"solution": "Run tests",
"scene": "working",
"status": "active",
"user_id": "local",
"agent_id": "all-apps",
"source": "manual",
"memory_at": 1_600_000_000,
"created_at": 1_700_000_000,
"updated_at": 1_710_000_000,
}
result = asyncio.run(read_home_projection(ProjectionSdk([row]), user_id="local", mode="pro"))
assert result.latest is not None and result.latest.type == "memory"
assert (result.latest.scene, result.latest.kind, result.latest.variant) == (
"working",
"experience",
"working",
)
assert result.latest.time_basis == "update"
assert result.latest.time.isoformat() == "2024-03-09T16:00:00+00:00"
@pytest.mark.parametrize(
"row",
[
{**chat_row(), "updated_at": None},
{
"kind": "experience",
"experience_id": "e-1",
"task": "Release",
"solution": "Run tests",
"scene": "coding",
"status": "active",
"user_id": "local",
"agent_id": "all-apps",
"source": "manual",
"created_at": 1_700_000_000,
"updated_at": None,
},
],
)
def test_memory_projection_sort_time_falls_back_to_scene_creation(row) -> None:
result = asyncio.run(read_home_projection(ProjectionSdk([row]), user_id="local", mode="pro"))
assert result.latest is not None
assert result.latest.time_basis == "write"
expected = row.get("gmt_created") or row.get("created_at")
assert result.latest.time.timestamp() == expected
def test_fact_is_not_a_home_scene_all_item() -> None:
row = {
"kind": "fact",
"fact_id": "f-1",
"statement": "Uses pytest",
"scene": "coding",
"status": "active",
"user_id": "local",
"agent_id": "all-apps",
"source": "manual",
"created_at": 1_700_000_000,
}
result = asyncio.run(read_home_projection(ProjectionSdk([row]), user_id="local", mode="pro"))
assert result.visible_count == 1
assert result.latest is None
assert result.display_state == "no_displayable_memory"
def test_invalid_latest_uses_next_parseable_candidate_from_same_snapshot() -> None:
sdk = ProjectionSdk(
[
{"memory_id": "broken", "content": "", "scene": "chat", "kind": "chat"},
chat_row("m-2"),
],
total=2,
)
result = asyncio.run(read_home_projection(sdk, user_id="local", mode="pro"))
assert result.visible_count == 2
assert result.latest is not None and result.latest.memory_id == "m-2"
assert len(sdk.calls) == 1
def test_scene_all_rejects_boolean_total() -> None:
sdk = ProjectionSdk([chat_row()])
sdk.total = True # type: ignore[assignment]
with pytest.raises(HySdkReadError):
asyncio.run(read_home_projection(sdk, user_id="local", mode="pro"))
def test_projection_read_failure_is_not_converted_to_an_empty_projection() -> None:
class Broken:
def list_memories(self, **kwargs):
raise OSError("down")
with pytest.raises(OSError, match="down"):
asyncio.run(read_home_projection(Broken(), user_id="local", mode="lite"))
def test_projection_refuses_a_client_built_for_another_mode() -> None:
sdk = ProjectionSdk([chat_row()])
sdk.mode = "lite"
with pytest.raises(RuntimeError, match="home_projection_mode_mismatch"):
asyncio.run(read_home_projection(sdk, user_id="local", mode="pro"))
def test_home_projection_route_maps_whole_read_failure_to_503(tmp_path, monkeypatch) -> None:
class Broken:
def list_memories(self, **kwargs):
raise OSError("down")
class Handle:
async def client(self):
return Broken()
app = create_app(
Settings(
_env_file=None,
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("test"),
imports_enabled=False,
automatic_memory_processing=False,
live_sync_enabled=False,
),
source_catalog={},
)
monkeypatch.setattr("one_memory.api._sdk_client_handle", lambda _app: Handle())
with TestClient(app, headers={"Authorization": "Bearer test"}) as client:
response = client.get("/api/v2/home-projection")
assert response.status_code == 503
assert response.json() == {"detail": "home_projection_read_failed"}
def test_home_projection_reads_mode_once_and_matches_the_client(tmp_path, monkeypatch) -> None:
sdk = ProjectionSdk([chat_row()])
sdk.mode = "pro"
class Handle:
async def client(self):
return sdk
app = create_app(
Settings(
_env_file=None,
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("test"),
imports_enabled=False,
automatic_memory_processing=False,
live_sync_enabled=False,
),
source_catalog={},
)
reads = 0
def mode(_app):
nonlocal reads
reads += 1
return "pro"
with TestClient(app, headers={"Authorization": "Bearer test"}) as client:
monkeypatch.setattr("one_memory.api._memory_mode", mode)
monkeypatch.setattr("one_memory.api._sdk_client_handle", lambda _app: Handle())
response = client.get("/api/v2/home-projection")
assert response.status_code == 200, response.text
assert response.json()["effective_mode"] == "pro"
assert reads == 1
def test_project_raw_latest_uses_import_time_without_conversation_time() -> None:
from one_memory.hy_sdk_types import HySdkRawRecord
record = HySdkRawRecord.from_sdk(
{
**raw_row("r-1", "[user]: hello", memory_at=None),
"raw_id": "r-1",
}
)
latest = project_raw_latest((record,))
assert latest is not None
assert latest.time_basis == "import"
+2 -2
View File
@@ -34,10 +34,10 @@ def test_resume_during_cleanup_returns_conflict_without_internal_error(tmp_path,
job = supervisor.job_store.create(user_id="local", source_id="fixture")
supervisor.job_store.transition(job.job_id, ImportStatus.PAUSED)
async def draining(_job_id):
def draining(_job_id):
raise InvalidJobTransitionError("import_cleanup_in_progress")
monkeypatch.setattr(supervisor, "resume_off_loop", draining)
monkeypatch.setattr(supervisor, "resume", draining)
response = await client.post(f"/api/v1/import-jobs/{job.job_id}/resume")
assert response.status_code == 409
assert response.json() == {"detail": "invalid_job_transition"}
@@ -1,383 +0,0 @@
"""`import_jobs` stays bounded, and the questions asked of it are SQL, not scans.
Live sync opened a job per change it imported and nothing ever removed one, so
the table only grew; every caller then read all of it and filtered in Python.
These tests hold the three halves of the fix: what pruning keeps, that the
filtered queries answer exactly what the old filters did, and that the indexes
behind them arrive by migration.
"""
from __future__ import annotations
import asyncio
import random
import sqlite3
from datetime import UTC, datetime, timedelta
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import _prune_finished_import_jobs, create_app
from one_memory.config import Settings
from one_memory.imports.jobs import (
FINISHED_STATUSES,
SYNC_JOBS_KEPT_PER_SOURCE,
ImportJob,
ImportJobStore,
ImportStatus,
)
from one_memory.memory_server.migration_runner import SqliteMigrationStore, migrate
from one_memory.memory_server.migrations import (
IMPORT_JOB_LOOKUP_INDEXES_MIGRATION_VERSION,
MIGRATIONS,
)
_BASE = datetime(2026, 1, 1, tzinfo=UTC)
def _store(tmp_path) -> ImportJobStore:
return ImportJobStore(tmp_path / "one-memory.sqlite3")
def _job(
store: ImportJobStore,
*,
user_id: str = "local",
source_id: str = "fixture",
operation: str = "sync",
status: ImportStatus = ImportStatus.COMPLETED,
minute: int,
issue: bool = False,
) -> str:
"""Insert one row as its terminal shape, at a chosen creation time."""
created = (_BASE + timedelta(minutes=minute)).isoformat()
job_id = f"{source_id}-{user_id}-{operation}-{minute:05d}"
with store._connect(writing=True) as connection:
connection.execute(
"INSERT INTO import_jobs (job_id,user_id,source_id,status,created_at,updated_at,"
"operation) VALUES (?,?,?,?,?,?,?)",
(job_id, user_id, source_id, status, created, created, operation),
)
if issue:
connection.execute(
"INSERT INTO import_issues (job_id,kind,conversation_id,reason,created_at) "
"VALUES (?,'error',NULL,'synthetic_reason',?)",
(job_id, created),
)
connection.execute(
"INSERT INTO import_skip_reason_counts (job_id,reason,count) VALUES (?,'r',1)",
(job_id,),
)
return job_id
def _ids(store: ImportJobStore, **filters) -> list[str]:
return [job.job_id for job in store.find_jobs(**filters)]
def _count(store: ImportJobStore, table: str, job_id: str) -> int:
with store._connect() as connection:
return int(
connection.execute(
f"SELECT COUNT(*) FROM {table} WHERE job_id=?", (job_id,)
).fetchone()[0]
)
def test_pruning_keeps_the_newest_finished_syncs_of_every_source(tmp_path) -> None:
store = _store(tmp_path)
keep = 5
for source in ("alpha", "beta"):
for minute in range(12):
status = ImportStatus.CANCELLED if minute % 4 == 0 else ImportStatus.COMPLETED
_job(store, source_id=source, minute=minute, status=status, issue=True)
# A source with fewer than `keep` keeps all of them.
for minute in range(3):
_job(store, source_id="gamma", minute=minute)
# Never pruned: the active sync, history of any state, another user's lane.
running = _job(store, source_id="alpha", minute=1000, status=ImportStatus.RUNNING)
failed_history = _job(
store, source_id="alpha", operation="history", minute=2, status=ImportStatus.FAILED
)
histories = [
_job(store, source_id="alpha", operation="history", minute=100 + minute)
for minute in range(10)
]
other_user = [_job(store, user_id="other", source_id="alpha", minute=m) for m in range(8)]
removed = store.prune_finished_sync_jobs(keep=keep)
assert removed == (12 - keep) * 2 + (8 - keep)
for source in ("alpha", "beta"):
finished = _ids(
store, user_id="local", source_id=source, operation="sync", statuses=FINISHED_STATUSES
)
assert finished == [f"{source}-local-sync-{m:05d}" for m in range(12 - keep, 12)]
# Their issues and skip counts went with them; the kept ones still have theirs.
assert _count(store, "import_issues", f"{source}-local-sync-00000") == 0
assert _count(store, "import_skip_reason_counts", f"{source}-local-sync-00000") == 0
assert _count(store, "import_issues", f"{source}-local-sync-00011") == 1
assert len(_ids(store, source_id="gamma")) == 3
assert store.require(running).status is ImportStatus.RUNNING
assert store.require(failed_history).status is ImportStatus.FAILED
assert all(store.get(job_id) is not None for job_id in histories)
assert _ids(store, user_id="other") == other_user[-keep:]
# The newest job of every lane, which the App and live sync health show,
# is always kept.
assert store.latest_job(user_id="local", source_id="beta").job_id == "beta-local-sync-00011"
before = _ids(store)
assert store.prune_finished_sync_jobs(keep=keep) == 0
assert _ids(store) == before
def test_pruning_is_bounded_per_call_and_finishes_when_repeated(tmp_path) -> None:
store = _store(tmp_path)
for minute in range(60):
_job(store, minute=minute)
passes = []
while True:
removed = store.prune_finished_sync_jobs(keep=10, max_rows=15)
passes.append(removed)
if removed < 15:
break
assert passes == [15, 15, 15, 5]
assert len(_ids(store)) == 10
def test_the_default_keep_count_is_twenty(tmp_path) -> None:
store = _store(tmp_path)
for minute in range(SYNC_JOBS_KEPT_PER_SOURCE + 7):
_job(store, minute=minute)
assert store.prune_finished_sync_jobs() == 7
assert len(_ids(store)) == SYNC_JOBS_KEPT_PER_SOURCE == 20
def test_startup_pruning_runs_to_completion_off_the_loop(tmp_path) -> None:
store = _store(tmp_path)
for minute in range(500):
_job(store, minute=minute)
async def scenario() -> None:
task = asyncio.create_task(_prune_finished_import_jobs(store, interval_seconds=3600))
for _ in range(500):
await asyncio.sleep(0.01)
if len(await asyncio.to_thread(_ids, store)) == SYNC_JOBS_KEPT_PER_SOURCE:
break
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
asyncio.run(scenario())
assert len(_ids(store)) == SYNC_JOBS_KEPT_PER_SOURCE
def test_filtered_queries_answer_what_the_python_filters_did(tmp_path) -> None:
store = _store(tmp_path)
rng = random.Random(7)
lanes = [(user, source) for user in ("local", "other") for source in ("a", "b", "c")]
for minute in range(240):
user, source = rng.choice(lanes)
_job(
store,
user_id=user,
source_id=source,
operation=rng.choice(("sync", "history")),
# Only terminal shapes: the active-slot index allows one active job per lane.
status=rng.choice(FINISHED_STATUSES),
minute=minute,
)
everything: list[ImportJob] = store.list_jobs()
assert len(everything) == 240
status_sets = [
None,
(ImportStatus.COMPLETED,),
(ImportStatus.CANCELLED, ImportStatus.COMPLETED),
(ImportStatus.RUNNING,),
]
for user_id in (None, "local", "other"):
for source_id in (None, "a", "c"):
for operation in (None, "sync", "history"):
for statuses in status_sets:
expected = [
job
for job in everything
if (user_id is None or job.user_id == user_id)
and (source_id is None or job.source_id == source_id)
and (operation is None or job.operation == operation)
and (statuses is None or job.status in statuses)
]
found = store.find_jobs(
user_id=user_id,
source_id=source_id,
operation=operation,
statuses=statuses,
)
assert found == expected
newest = store.find_jobs(
user_id=user_id,
source_id=source_id,
operation=operation,
statuses=statuses,
newest_first=True,
limit=1,
)
assert newest == (
[max(expected, key=lambda job: (job.created_at, job.job_id))]
if expected
else []
)
for status in ImportStatus:
assert store.list_jobs(status=status) == [job for job in everything if job.status is status]
assert store.find_jobs(statuses=()) == []
def test_the_lookup_indexes_arrive_by_migration_and_serve_the_lane_query(tmp_path) -> None:
path = tmp_path / "one-memory.sqlite3"
older = tuple(
item for item in MIGRATIONS if item.version < IMPORT_JOB_LOOKUP_INDEXES_MIGRATION_VERSION
)
asyncio.run(migrate(SqliteMigrationStore(path), older))
with sqlite3.connect(path) as database:
before = {
row[0] for row in database.execute("SELECT name FROM sqlite_master WHERE type='index'")
}
assert "import_jobs_lane" not in before
result = asyncio.run(migrate(SqliteMigrationStore(path), MIGRATIONS))
assert result.applied == (IMPORT_JOB_LOOKUP_INDEXES_MIGRATION_VERSION,)
with sqlite3.connect(path) as database:
indexes = {
row[0] for row in database.execute("SELECT name FROM sqlite_master WHERE type='index'")
}
plan = " ".join(
str(row[-1])
for row in database.execute(
"EXPLAIN QUERY PLAN SELECT * FROM import_jobs "
"WHERE user_id=? AND source_id=? AND operation=? AND status IN (?,?) "
"ORDER BY created_at DESC LIMIT 1",
("local", "fixture", "sync", "pending", "running"),
)
)
assert {"import_jobs_lane", "import_jobs_status"} <= indexes
assert "import_jobs_lane" in plan
# Re-running it is a no-op, and a store opened on the database agrees.
asyncio.run(migrate(SqliteMigrationStore(path), MIGRATIONS))
ImportJobStore(path)
def test_a_running_home_gets_the_indexes_from_the_store(tmp_path) -> None:
path = tmp_path / "one-memory.sqlite3"
ImportJobStore(path)
with sqlite3.connect(path) as database:
database.execute("DROP INDEX import_jobs_lane")
database.execute("DROP INDEX import_jobs_status")
ImportJobStore(path)
with sqlite3.connect(path) as database:
indexes = {
row[0] for row in database.execute("SELECT name FROM sqlite_master WHERE type='index'")
}
assert {"import_jobs_lane", "import_jobs_status"} <= indexes
def test_the_mysql_rendering_tolerates_an_index_already_there() -> None:
class Cursor:
dialect = "mysql"
def __init__(self) -> None:
self.attempted: list[str] = []
async def execute(self, statement: str) -> None:
self.attempted.append(statement)
raise RuntimeError(1061, "duplicate key name")
cursor = Cursor()
migration = next(
item for item in MIGRATIONS if item.version == IMPORT_JOB_LOOKUP_INDEXES_MIGRATION_VERSION
)
asyncio.run(migration.apply(cursor))
assert cursor.attempted and "import_jobs" in cursor.attempted[0]
def test_the_import_jobs_route_keeps_its_shape_and_order(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path / "home",
api_token=SecretStr("synthetic"),
automatic_memory_processing=False,
live_sync_enabled=False,
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer synthetic"},
) as client,
):
store: ImportJobStore = app.state.import_job_store
for minute in (3, 1, 2):
_job(store, source_id="fixture", minute=minute)
_job(store, source_id="fixture", operation="history", minute=0)
_job(store, user_id="other", source_id="hermes", minute=4)
response = await client.get("/api/v1/import-jobs")
other = await client.get("/api/v1/import-jobs", params={"user_id": "other"})
assert response.status_code == 200
body = response.json()
expected = [job for job in store.list_jobs() if job.user_id == "local"]
assert [item["job_id"] for item in body] == [job.job_id for job in expected]
assert [item["created_at"] for item in body] == sorted(
item["created_at"] for item in body
)
assert set(body[0]) == set(ImportJob.model_fields)
assert [item["user_id"] for item in other.json()] == ["other"]
asyncio.run(scenario())
def test_memory_processing_reports_the_write_drain(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path / "home",
api_token=SecretStr("synthetic"),
automatic_memory_processing=False,
live_sync_enabled=False,
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer synthetic"},
) as client,
):
for _ in range(500):
if app.state.recovery in {"done", "failed"}:
break
await asyncio.sleep(0.01)
body = (await client.get("/api/v1/memory-processing")).json()
assert body["write_drain"] == "running"
assert body["write_drain_error"] is None
drain = app.state.memory_write_drain
drain.liveness = "restarting"
drain.last_error_code = "operational_error"
body = (await client.get("/api/v1/memory-processing")).json()
assert body["write_drain"] == "restarting"
assert body["write_drain_error"] == "operational_error"
drain.liveness = "running"
asyncio.run(scenario())
@@ -415,15 +415,15 @@ def test_shutdown_preserves_a_job_that_finished_after_the_running_snapshot(
task = asyncio.create_task(waiting())
supervisor._tasks[job.job_id] = task
await asyncio.sleep(0)
original = jobs.find_jobs
original = jobs.list_jobs
def finish_after_snapshot(*args, **kwargs):
found = original(*args, **kwargs)
if tuple(kwargs.get("statuses") or ()) == (ImportStatus.RUNNING,):
if kwargs.get("status") is ImportStatus.RUNNING:
jobs.finish_owned(job.job_id, runner_token=token, target=terminal)
return found
monkeypatch.setattr(jobs, "find_jobs", finish_after_snapshot)
monkeypatch.setattr(jobs, "list_jobs", finish_after_snapshot)
try:
await supervisor.shutdown()
assert jobs.require(job.job_id).status is terminal
@@ -433,45 +433,3 @@ def test_shutdown_preserves_a_job_that_finished_after_the_running_snapshot(
await asyncio.gather(task, return_exceptions=True)
asyncio.run(scenario())
def test_abandon_makes_its_database_calls_off_the_event_loop(tmp_path, monkeypatch) -> None:
"""Live sync abandons from its poll; a writer wait there froze the whole Core."""
import threading
async def scenario() -> None:
evidence, jobs = stores(tmp_path)
supervisor = ImportSupervisor(adapters={}, evidence_store=evidence, job_store=jobs)
job = jobs.create(user_id="local", source_id="fixture", operation="sync")
job, _token = jobs.claim(job.job_id)
loop_thread = threading.current_thread()
calls: list[tuple[str, threading.Thread]] = []
for name in ("require", "interrupt_running"):
original = getattr(jobs, name)
def recorded(*args, _name=name, _original=original, **kwargs):
calls.append((_name, threading.current_thread()))
return _original(*args, **kwargs)
monkeypatch.setattr(jobs, name, recorded)
async def wedged() -> None:
# Ignores the first cancel, like a runner stuck in a thread.
try:
await asyncio.Event().wait()
except asyncio.CancelledError:
await asyncio.sleep(0.2)
task = asyncio.create_task(wedged())
supervisor._tasks[job.job_id] = task
await asyncio.sleep(0)
try:
assert await supervisor.abandon(job.job_id, timeout=0.01) is False
finally:
await asyncio.gather(task, return_exceptions=True)
assert {name for name, _thread in calls} == {"require", "interrupt_running"}
assert all(thread is not loop_thread for _name, thread in calls)
assert jobs.require(job.job_id).status is ImportStatus.INTERRUPTED
asyncio.run(scenario())
File diff suppressed because it is too large. Load diff
-32
View File
@@ -293,9 +293,6 @@ def test_health_states_that_watching_is_layered_on_the_timer(tmp_path: Path) ->
assert live.health(PI_DESCRIPTOR.source_id).mode == "watch_and_polling"
assert live.health("unknown-source").mode == "watch_and_polling"
live.set_watched_sources((PI_DESCRIPTOR.source_id,))
assert live._poll_interval(PI_DESCRIPTOR.source_id) == max(live.interval_seconds, 15 * 60)
assert live._poll_interval("unknown-source") == live.interval_seconds
await imports.shutdown()
asyncio.run(scenario())
@@ -644,32 +641,3 @@ def _job(*, created_at: datetime) -> ImportJob:
created_at=created_at,
updated_at=datetime.now(UTC),
)
def test_attributing_changed_paths_happens_off_the_event_loop(tmp_path: Path, monkeypatch) -> None:
"""Resolving a host's batch of changed paths blocked the loop for seconds."""
import threading
async def scenario() -> None:
root, (_evidence, live, imports) = _source(tmp_path)
watch = FileWatchLiveSync(
roots={PI_DESCRIPTOR.source_id: root},
supervisor=live,
debounce_seconds=0.1,
min_interval_seconds=0.1,
)
loop_thread = threading.current_thread()
resolved_on: list[threading.Thread] = []
original = watch.source_for
def recorded(path: Path) -> str | None:
resolved_on.append(threading.current_thread())
return original(path)
monkeypatch.setattr(watch, "source_for", recorded)
assert await watch.handle_changes({tmp_path / "elsewhere/session.jsonl"}) == ()
assert resolved_on and loop_thread not in resolved_on
await imports.shutdown()
asyncio.run(scenario())
@@ -318,31 +318,9 @@ def test_main_database_checkpoints_on_a_write_cadence(tmp_path, monkeypatch):
with database.write() as db, db:
db.execute("UPDATE counter SET value = 2 WHERE id = 1")
assert _access.wait_for_cadence_checkpoints()
assert checkpoints and checkpoints[0][0] == "PASSIVE"
def test_the_cadence_checkpoint_does_not_run_on_the_writing_thread(tmp_path, monkeypatch):
"""The thread that wrote returns at once; the checkpointer takes the next turn."""
path = _database(tmp_path)
database = main_database(path)
ran_on = []
original = database._checkpoint_from_maintenance
def observed(mode):
ran_on.append(threading.current_thread().name)
return original(mode)
monkeypatch.setattr(database, "_checkpoint_from_maintenance", observed)
monkeypatch.setattr(_access, "MAIN_DATABASE_CHECKPOINT_WRITE_INTERVAL", 1)
with database.write() as db, db:
db.execute("UPDATE counter SET value = 1 WHERE id = 1")
assert _access.wait_for_cadence_checkpoints()
assert ran_on == ["one-memory-checkpoint"]
def test_main_database_warns_when_wal_stays_above_the_high_water_mark(
tmp_path, monkeypatch, caplog
):
@@ -354,7 +332,6 @@ def test_main_database_warns_when_wal_stays_above_the_high_water_mark(
with database.write() as db, db:
db.execute("UPDATE counter SET value = 1 WHERE id = 1")
assert _access.wait_for_cadence_checkpoints()
assert any("main_database_wal_high" in record.message for record in caplog.records)
@@ -372,7 +349,6 @@ def test_a_wal_whose_frames_are_all_checkpointed_is_not_reported_high(
with database.write() as db, db:
db.execute("UPDATE counter SET value = 1 WHERE id = 1")
assert _access.wait_for_cadence_checkpoints()
assert database._wal_bytes() > 0
assert not any("main_database_wal_high" in record.message for record in caplog.records)
@@ -524,11 +524,6 @@ def test_features_api_get_put_reset_and_status(tmp_path, monkeypatch) -> None:
"episode_recall": True,
}
monkeypatch.setattr(
app.state.model_profile_store,
"active_identity",
lambda: ("active-profile", 1),
)
await client.put("/api/v1/settings/memory-mode", json={"mode": "ultra"})
await app.state.memory_graph_backfill.wait()
ultra = _by_name((await client.get("/api/v1/settings/memory-features")).json())
@@ -177,37 +177,6 @@ def test_the_read_handle_rebuilds_on_a_mode_or_profile_change(monkeypatch, tmp_p
asyncio.run(scenario())
def test_read_handle_can_bind_one_caller_mode_snapshot(monkeypatch, tmp_path) -> None:
from one_memory import sdk_read_handle
callback_reads = 0
def current_mode() -> str:
nonlocal callback_reads
callback_reads += 1
return "pro"
monkeypatch.setattr(
sdk_read_handle,
"SdkClientHandle",
lambda profile, **kwargs: _FakeHandle(kwargs.get("mode") or "pro", profile.profile_id),
)
reads = sdk_read_handle.ActiveProfileReadHandle(
profiles=_Profiles(identity=None), # type: ignore[arg-type]
secrets=None, # type: ignore[arg-type]
data_dir=tmp_path,
memory_mode=current_mode,
)
async def scenario():
client = await reads.client_for_mode("lite")
assert client.mode == "lite"
await reads.close()
asyncio.run(scenario())
assert callback_reads == 0
class _FakeHandle:
def __init__(self, mode: str, profile_id: str = "p") -> None:
self.mode = mode
@@ -338,7 +307,7 @@ class _SdkHandle:
return None
async def _noop(_requires_generation: bool) -> None:
async def _noop() -> None:
return None
@@ -361,14 +330,14 @@ def test_a_note_in_lite_is_stored_as_raw_and_never_calls_add_extracted() -> None
]
def test_a_lite_write_ignores_the_model_sedimentation_pause(monkeypatch) -> None:
def test_a_lite_write_runs_through_the_coordinator_without_a_model(monkeypatch) -> None:
sdk = _WritingSdk()
coordinator = sdk_write_processor.SdkWriteCoordinator(
profiles=_Profiles(identity=None), # type: ignore[arg-type]
secrets=None, # type: ignore[arg-type]
data_dir=None,
transport_factory=lambda _profile: None,
sedimentation_enabled=lambda: False,
sedimentation_enabled=lambda: True,
memory_mode=lambda: "lite",
)
monkeypatch.setattr(
@@ -385,37 +354,6 @@ def test_a_lite_write_ignores_the_model_sedimentation_pause(monkeypatch) -> None
assert [name for name, _ in sdk.calls] == ["add"]
def test_a_lite_write_ignores_the_model_sedimentation_switch(monkeypatch) -> None:
sdk = _WritingSdk()
coordinator = sdk_write_processor.SdkWriteCoordinator(
profiles=_Profiles(identity=None), # type: ignore[arg-type]
secrets=None, # type: ignore[arg-type]
data_dir=None,
transport_factory=lambda _profile: None,
sedimentation_enabled=lambda: False,
memory_mode=lambda: "lite",
)
monkeypatch.setattr(
sdk_write_processor, "SdkClientHandle", lambda profile, **kwargs: _SdkHandle(sdk, "lite")
)
admitted = 0
async def admit(_requires_generation: bool) -> None:
nonlocal admitted
admitted += 1
receipt = asyncio.run(
coordinator.process_sdk_job(
(_record("still stored as raw", extracted=False),),
request_id="r-off-lite",
before_hy_invocation=admit,
)
)
assert receipt.raw_ids == ("raw-1",)
assert admitted == 1
assert [name for name, _ in sdk.calls] == ["add"]
def test_no_raw_is_ever_deleted_in_any_mode() -> None:
"""Owner decision: MemHub keeps every raw; there is no retention window."""
from one_memory import sdk_memory_runtime
@@ -874,35 +812,22 @@ def test_mode_api_get_put_and_its_effects(tmp_path, monkeypatch) -> None:
assert current["explicitly_configured"] is False
assert current["forced_by_env"] is False
assert app.state.sdk_write_coordinator.memory_mode() == "pro"
assert (
await client.put("/api/v1/sedimentation", json={"enabled": False})
).status_code == 200
from one_memory.api import _sedimentation_claim_enabled
app.state.history_import_gate_released = True
assert not _sedimentation_claim_enabled(app, "one-memory-manual")
app.state.config_store.set_sedimentation_enabled(False)
lite = await client.put("/api/v1/settings/memory-mode", json={"mode": "lite"})
assert lite.status_code == 200, lite.text
body = lite.json()
assert body["mode"] == "lite" and body["previous_mode"] == "pro"
assert body["explicitly_configured"] is True
assert body["effects"] == ["higher_mode_memories_hidden"]
resume = getattr(app.state, "sedimentation_resume", None)
assert resume is not None
await asyncio.wait_for(asyncio.shield(resume), 5)
assert app.state.sdk_write_coordinator.memory_mode() == "lite"
assert _sdk_client_handle(app).memory_mode() == "lite"
# lite claims queued work without a generation model.
from one_memory.api import _sedimentation_claim_enabled
assert app.state.model_profile_store.active_identity() is None
app.state.history_import_gate_released = True
assert _sedimentation_claim_enabled(app, "one-memory-manual")
monkeypatch.setattr(
app.state.model_profile_store,
"active_identity",
lambda: ("active-profile", 1),
)
ultra = await client.put("/api/v1/settings/memory-mode", json={"mode": "ultra"})
assert ultra.status_code == 200, ultra.text
assert "existing_memories_added_to_graph" in ultra.json()["effects"]
@@ -926,114 +851,6 @@ def test_mode_api_get_put_and_its_effects(tmp_path, monkeypatch) -> None:
run_app(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_mode_plan_and_put_share_validation_and_effects(tmp_path, monkeypatch) -> None:
async def scenario(app, _sdk, client):
plan = await client.post(
"/api/v1/settings/memory-mode/plan",
json={"mode": "lite"},
)
assert plan.status_code == 200
body = plan.json()
assert body == {
"current": "pro",
"target": "lite",
"effective_target": "lite",
"capabilities": {
"content_plane": "raw",
"generation_required": False,
"graph_enabled": False,
"evolve_available": False,
},
"effects": ["higher_mode_memories_hidden"],
"warnings": [],
"blockers": [],
}
changed = await client.put("/api/v1/settings/memory-mode", json={"mode": "lite"})
assert changed.status_code == 200
assert changed.json()["effects"] == body["effects"]
missing_model = (
await client.post(
"/api/v1/settings/memory-mode/plan",
json={"mode": "pro"},
)
).json()
assert missing_model["blockers"] == ["active_model_required"]
assert missing_model["effects"] == []
refused = await client.put("/api/v1/settings/memory-mode", json={"mode": "pro"})
assert refused.status_code == 409
assert refused.json()["detail"] == missing_model["blockers"][0]
monkeypatch.setenv("ONE_MEMORY_GRAPH_STORE", "kuzu")
forced = (
await client.post(
"/api/v1/settings/memory-mode/plan",
json={"mode": "lite"},
)
).json()
assert forced["effective_target"] == "ultra"
assert forced["warnings"] == ["forced_by_env", "effective_no_change"]
run_app(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_mode_plan_reports_all_blockers_without_planned_effects(tmp_path, monkeypatch) -> None:
async def scenario(app, _sdk, client):
changed = await client.put("/api/v1/settings/memory-mode", json={"mode": "lite"})
assert changed.status_code == 200
switch = app.state.embedding_switch
switch._scan_hold = True
hold = asyncio.Event()
task = asyncio.create_task(hold.wait())
app.state.memory_graph_backfill._task = task
monkeypatch.setattr("one_memory.api.importlib.util.find_spec", lambda _name: None)
try:
plan = (
await client.post(
"/api/v1/settings/memory-mode/plan",
json={"mode": "ultra"},
)
).json()
assert plan["blockers"] == [
"embedding_maintenance_in_progress",
"active_model_required",
"memory_lifecycle_busy",
"graph_store_unavailable",
]
assert plan["effects"] == []
finally:
switch._scan_hold = False
task.cancel()
await asyncio.gather(task, return_exceptions=True)
app.state.memory_graph_backfill._task = None
run_app(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_lite_startup_readmits_jobs_while_model_sedimentation_is_off(tmp_path, monkeypatch) -> None:
store = ConfigStore(tmp_path / "data")
store.set_memory_mode("lite")
store.set_sedimentation_enabled(False)
resumed = asyncio.Event()
original = SdkMemoryRuntime.resume_allowed
async def resume_allowed(runtime):
await original(runtime)
resumed.set()
monkeypatch.setattr(SdkMemoryRuntime, "resume_allowed", resume_allowed)
async def scenario(app, _sdk, _client):
await asyncio.wait_for(resumed.wait(), 5)
assert app.state.config_store.sedimentation_enabled() is False
assert app.state.config_store.memory_mode() == "lite"
run_app(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_embedding_maintenance_blocks_mode_put_and_evolve_but_not_mode_get(
tmp_path, monkeypatch
@@ -1100,13 +917,7 @@ def test_evolve_api_is_ultra_only_async_and_scoped_per_user(tmp_path, monkeypatc
assert refused.status_code == 409
assert refused.json()["detail"] == "memory_mode_not_ultra"
monkeypatch.setattr(
app.state.model_profile_store,
"active_identity",
lambda: ("active-profile", 1),
)
changed = await client.put("/api/v1/settings/memory-mode", json={"mode": "ultra"})
assert changed.status_code == 200, changed.text
await client.put("/api/v1/settings/memory-mode", json={"mode": "ultra"})
await app.state.memory_graph_backfill.wait()
started = await client.post(
"/api/v2/evolve", json={"user_id": "wife", "agent_id": "all-apps", "max_batches": 2}
@@ -1223,87 +1034,3 @@ def test_a_job_written_in_lite_stays_done_and_a_pending_job_takes_the_new_mode(
await coordinator.close()
run_app(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_paused_model_still_writes_a_lite_job_through_the_durable_boundary(
tmp_path, monkeypatch
) -> None:
from one_memory.api import _admit_pipeline_invocation
from one_memory.memory_server.durable_queue import DurableWriteDrain, execute_job
seen: list[str] = []
class Sdk:
mode = "lite"
def add(self, messages, **kwargs):
seen.append("add")
return {"success": True, "raw_ids": ["raw-lite"]}
def add_extracted(self, contents, **kwargs):
seen.append("add_extracted")
raise AssertionError("lite must never call add_extracted")
monkeypatch.setattr(
sdk_write_processor,
"SdkClientHandle",
lambda profile, **kwargs: _SdkHandle(Sdk(), "lite"),
)
coordinator = sdk_write_processor.SdkWriteCoordinator(
profiles=_Profiles(identity=None), # type: ignore[arg-type]
secrets=None, # type: ignore[arg-type]
data_dir=None,
transport_factory=lambda _profile: None,
sedimentation_enabled=lambda: False,
memory_mode=lambda: "lite",
)
async def never(_job_id):
return None
async def scenario(app, _sdk, client):
app.state.memory_write_drain.stop()
if app.state.memory_write_drain_task is not None:
await app.state.memory_write_drain_task
app.state.memory_write_drain_task = None
await client.put("/api/v1/sedimentation", json={"enabled": False})
await client.put("/api/v1/settings/memory-mode", json={"mode": "lite"})
app.state.history_import_gate_released = True
written = await client.post(
"/api/v2/memory-writes",
json={
"request_id": "paused-lite-turn",
"session_id": "paused-lite-session",
"source_id": "one-memory-manual",
"messages": [
{"role": "user", "content": "保留原始会话"},
{"role": "assistant", "content": "已保留"},
],
},
)
assert written.status_code == 202
queue = app.state.memory_write_queue
drain = DurableWriteDrain(queue, execute=never, lease=timedelta(seconds=30))
[job_id] = drain.claim(limit=1)
await execute_job(
job_id,
queue=queue,
supervisor=coordinator,
evidence=None, # type: ignore[arg-type]
admit_hy_invocation=lambda source_id, requires_generation, mark: (
_admit_pipeline_invocation(
app,
source_id,
requires_generation,
mark,
)
),
)
current = queue.get("paused-lite-turn")
assert current is not None and current.state == "completed"
assert current.sdk_raw_ids == ("raw-lite",)
assert seen == ["add"]
await coordinator.close()
run_app(tmp_path, monkeypatch, scenario)
@@ -832,7 +832,7 @@ class _Recorder:
async def process_sdk_job(self, records, **_kwargs):
self.seen.append(tuple(record.evidence_id for record in records))
if self.invoke_hy:
await _kwargs["before_hy_invocation"](True)
await _kwargs["before_hy_invocation"]()
if self.error is not None:
raise self.error
return type("Receipt", (), {"raw_ids": (), "memory_ids": ()})()
@@ -844,10 +844,10 @@ class _TwoInvocationRecorder:
self.invocations = 0
async def process_sdk_job(self, _records, **kwargs):
await kwargs["before_hy_invocation"](True)
await kwargs["before_hy_invocation"]()
self.invocations += 1
self.pause_after_first()
await kwargs["before_hy_invocation"](True)
await kwargs["before_hy_invocation"]()
self.invocations += 1
return type("Receipt", (), {"raw_ids": (), "memory_ids": ()})()
@@ -916,7 +916,7 @@ def test_execute_job_persists_sdk_ids_returned_after_the_single_add(tmp_path) ->
@classmethod
async def process_sdk_job(cls, _records, **kwargs):
cls.calls += 1
await kwargs["before_hy_invocation"](True)
await kwargs["before_hy_invocation"]()
return type(
"Result",
(),
@@ -1045,7 +1045,7 @@ def test_sdk_write_processor_calls_add_once_with_public_scope_and_receipt() -> N
)
async def run() -> SdkWriteReceipt:
async def before(_requires_generation: bool) -> None:
async def before() -> None:
client.events.append("before")
return await processor.process_sdk_job(
@@ -1105,7 +1105,7 @@ def test_sdk_write_processor_uses_add_extracted_for_memory_content_and_additions
processor = SdkWriteProcessor(_SdkPublicHandle(client)) # type: ignore[arg-type]
async def run() -> SdkWriteReceipt:
async def before(_requires_generation: bool) -> None:
async def before() -> None:
client.events.append("before")
return await processor.process_sdk_job(
@@ -1260,7 +1260,7 @@ def test_a_profile_change_does_not_close_the_client_of_a_write_in_flight(monkeyp
async def process(self, records, *, request_id, before_hy_invocation):
client = await self._client_handle.client()
await before_hy_invocation(getattr(client, "mode", None) != "lite")
await before_hy_invocation()
seen.append((client.profile_id, client.closed))
if client.profile_id == "profile-a":
await writing.wait()
@@ -1270,7 +1270,7 @@ def test_a_profile_change_does_not_close_the_client_of_a_write_in_flight(monkeyp
monkeypatch.setattr(sdk_write_processor.SdkWriteProcessor, "process_sdk_job", process)
async def noop(_requires_generation: bool):
async def noop():
return None
async def scenario():
@@ -1728,7 +1728,7 @@ def test_a_failed_job_logs_where_it_failed_but_never_the_message(tmp_path, caplo
class _Failing:
async def process_sdk_job(self, _records, **kwargs):
await kwargs["before_hy_invocation"](True)
await kwargs["before_hy_invocation"]()
try:
closed_store(type("Client", (), {"private_store_name": None})())
except AttributeError as error:
@@ -4138,71 +4138,6 @@ def test_a_failing_renewal_does_not_kill_the_drain(tmp_path) -> None:
assert queue.get("idem-1").state == COMPLETED
def test_a_locked_fallback_settlement_does_not_kill_the_drain(tmp_path, monkeypatch) -> None:
"""The incident shape: executor and its fallback both lose the writer.
The first row keeps its lease for normal reclaim; an unrelated row must
still run instead of being stranded behind a dead drain task.
"""
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
first = queue.submit(_request(key="first", source="cursor"))
second = queue.submit(_request(key="second", source="codex"))
executed: list[str] = []
async def execute(job_id: str) -> None:
executed.append(job_id)
if job_id == first.job_id:
raise RuntimeError("executor failed")
queue.settle(job_id, state=COMPLETED)
drain.stop()
drain = DurableWriteDrain(queue, execute=execute, max_concurrent_sources=1)
def locked_settlement(*_args, **_kwargs):
raise sqlite3.OperationalError("database is locked")
monkeypatch.setattr(queue, "settle_worker_error", locked_settlement)
asyncio.run(asyncio.wait_for(drain.run_forever(idle_pause=0.005), 1))
assert executed == [first.job_id, second.job_id]
assert queue.get("first").state == RUNNING
assert queue.get("second").state == COMPLETED
def test_an_unexpected_job_task_failure_is_contained_by_the_drain(
tmp_path, monkeypatch, caplog
) -> None:
"""A programming error stays visible without taking down the consumer."""
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
first = queue.submit(_request(key="first", source="cursor"))
second = queue.submit(_request(key="second", source="codex"))
executed: list[str] = []
async def run_job(job_id: str) -> None:
executed.append(job_id)
if job_id == first.job_id:
raise RuntimeError("unexpected job task failure")
drain.stop()
drain = DurableWriteDrain(
queue,
execute=_never_execute,
max_concurrent_sources=1,
)
monkeypatch.setattr(drain, "_run_claimed_job", run_job)
with caplog.at_level(logging.ERROR, logger="one_memory.memory_server.durable_queue"):
asyncio.run(asyncio.wait_for(drain.run_forever(idle_pause=0.005), 1))
assert executed == [first.job_id, second.job_id]
assert any(
"durable_write_drain_job_task_failed" in message.getMessage()
and "error_type=RuntimeError" in message.getMessage()
for message in caplog.records
)
@pytest.mark.parametrize("terminal", [COMPLETED, FAILED, DEAD])
def test_a_terminal_row_is_sticky_against_a_late_settle(tmp_path, terminal: str) -> None:
"""A late settle must not overwrite an outcome that already happened.
@@ -4823,7 +4758,7 @@ def test_sedimentation_pause_releases_claim_without_spending_an_attempt(tmp_path
drain = DurableWriteDrain(queue, execute=_never_execute, owner="pause-owner")
assert drain.claim(limit=1) == (record.job_id,)
async def refuse_before_hy(_source_id, _requires_generation, _mark_invoked):
async def refuse_before_hy(_source_id, _mark_invoked):
raise RuntimeError("model_sedimentation_disabled")
asyncio.run(
@@ -5045,27 +4980,6 @@ def test_permanent_renewal_error_stops_the_worker(tmp_path, monkeypatch) -> None
asyncio.run(drain.run_one())
def test_permanent_renewal_error_stops_the_long_lived_drain(tmp_path, monkeypatch) -> None:
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
queue.submit(_request())
drain = DurableWriteDrain(
queue,
execute=lambda _job_id: asyncio.sleep(0.03),
lease=timedelta(seconds=1),
renew_every=timedelta(milliseconds=10),
)
monkeypatch.setattr(
drain,
"renew",
lambda *_args, **_kwargs: (_ for _ in ()).throw(
sqlite3.OperationalError("database schema is corrupt")
),
)
with pytest.raises(RuntimeError):
asyncio.run(asyncio.wait_for(drain.run_forever(idle_pause=0.005), 1))
def test_lost_renewal_waits_for_execution_before_releasing_source(tmp_path, monkeypatch) -> None:
path = tmp_path / "one-memory.sqlite3"
queue = DurableWriteQueue(path)
@@ -5528,190 +5442,3 @@ def test_a_disabled_source_suspends_its_jobs_and_enabling_it_resumes_them(tmp_pa
assert runtime.status_counts() == {PENDING: 3, COMPLETED: 1}
assert runtime.queue_depths().fresh == 3
assert len(drain.claim(limit=10)) == 1
def test_a_claim_path_error_restarts_the_drain_and_later_jobs_still_run(
tmp_path, monkeypatch
) -> None:
"""An unexpected SQLite error on the claim path used to end the drain for good."""
from one_memory.memory_server import durable_queue as _queue_module
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
queue.submit(_request(key="later", source="cursor"))
reported: list[tuple[str, BaseException]] = []
monkeypatch.setattr(
_queue_module,
"report_exception",
lambda component, event, error, **_kwargs: reported.append((event, error)),
)
async def execute(job_id: str) -> None:
queue.settle(job_id, state=COMPLETED)
drain.stop()
drain = DurableWriteDrain(queue, execute=execute, max_concurrent_sources=1)
real_claim = drain.claim
calls = {"claim": 0}
def failing_once(**kwargs):
calls["claim"] += 1
if calls["claim"] == 1:
raise sqlite3.OperationalError("disk I/O error")
return real_claim(**kwargs)
monkeypatch.setattr(drain, "claim", failing_once)
first_owner = drain.owner
asyncio.run(
asyncio.wait_for(
drain.run_supervised(idle_pause=0.005, restart_backoff=0.01, max_restart_backoff=0.05),
2,
)
)
assert queue.get("later").state == COMPLETED
assert drain.restarts == 1
assert drain.last_error_code == "operational_error"
assert drain.liveness == "stopped"
# A restarted drain leases under a new name: nothing the old one left in a
# pool thread can renew or settle as it.
assert drain.owner != first_owner
assert [event for event, _error in reported] == ["memory.write_queue.drain_exited"]
assert reported[0][1].__traceback__ is not None
def test_a_claim_gate_exception_restarts_the_drain(tmp_path, monkeypatch) -> None:
from one_memory.memory_server import durable_queue as _queue_module
monkeypatch.setattr(_queue_module, "report_exception", lambda *_args, **_kwargs: None)
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
queue.submit(_request(key="gated", source="cursor"))
asked = {"count": 0}
def gate(_source_key: str) -> bool:
asked["count"] += 1
if asked["count"] == 1:
raise KeyError("gate bug")
return True
async def execute(job_id: str) -> None:
queue.settle(job_id, state=COMPLETED)
drain.stop()
drain = DurableWriteDrain(queue, execute=execute, source_claim_enabled=gate)
asyncio.run(asyncio.wait_for(drain.run_supervised(idle_pause=0.005, restart_backoff=0.01), 2))
assert queue.get("gated").state == COMPLETED
assert drain.last_error_code == "key_error"
def test_stop_ends_a_restart_backoff_at_once(tmp_path, monkeypatch) -> None:
from one_memory.memory_server import durable_queue as _queue_module
monkeypatch.setattr(_queue_module, "report_exception", lambda *_args, **_kwargs: None)
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
drain = DurableWriteDrain(queue, execute=_never_execute)
def broken(**_kwargs):
raise sqlite3.DatabaseError("database disk image is malformed")
monkeypatch.setattr(drain, "claim", broken)
async def scenario() -> None:
task = asyncio.create_task(drain.run_supervised(restart_backoff=30))
for _ in range(200):
if drain.liveness == "restarting":
break
await asyncio.sleep(0.005)
assert drain.liveness == "restarting"
drain.stop()
await asyncio.wait_for(task, 1)
asyncio.run(scenario())
assert drain.liveness == "stopped"
assert drain.last_error_code == "database_error"
def test_a_cancelled_supervised_drain_stays_cancelled(tmp_path) -> None:
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
drain = DurableWriteDrain(queue, execute=_never_execute)
async def scenario() -> None:
task = asyncio.create_task(drain.run_supervised(idle_pause=0.005))
await asyncio.sleep(0.05)
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
asyncio.run(scenario())
assert drain.liveness == "stopped"
assert drain.restarts == 0
def test_a_contained_job_task_failure_is_reported_with_its_stack(tmp_path, monkeypatch) -> None:
from one_memory.memory_server import durable_queue as _queue_module
reported: list[tuple[str, BaseException]] = []
monkeypatch.setattr(
_queue_module,
"report_exception",
lambda component, event, error, **_kwargs: reported.append((event, error)),
)
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
first = queue.submit(_request(key="first", source="cursor"))
queue.submit(_request(key="second", source="codex"))
async def run_job(job_id: str) -> None:
if job_id == first.job_id:
raise RuntimeError("systematic job task bug")
drain.stop()
drain = DurableWriteDrain(queue, execute=_never_execute, max_concurrent_sources=1)
monkeypatch.setattr(drain, "_run_claimed_job", run_job)
asyncio.run(asyncio.wait_for(drain.run_forever(idle_pause=0.005), 1))
assert [event for event, _error in reported] == ["memory.write_queue.job_task_failed"]
assert reported[0][1].__traceback__ is not None
def test_a_lease_is_renewed_while_every_bulk_thread_is_blocked(tmp_path) -> None:
"""The incident: bulk threads stuck in job listings starved lease renewal."""
from one_memory.background_work import BULK_WORKERS, bulk_pool, shutdown_bulk_pool
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
queue.submit(_request(key="slow", source="cursor"))
release = threading.Event()
blocked = [bulk_pool().submit(release.wait, 10) for _ in range(BULK_WORKERS * 2)]
renewed: list[bool] = []
async def execute(job_id: str) -> None:
# Longer than the lease: only renewals keep it.
await asyncio.sleep(0.8)
queue.settle(job_id, state=COMPLETED, owner=drain.owner)
drain = DurableWriteDrain(
queue,
execute=execute,
lease=timedelta(milliseconds=400),
renew_every=timedelta(milliseconds=50),
)
real_renew = drain.renew
def counted(job_id: str, **kwargs) -> bool:
result = real_renew(job_id, **kwargs)
renewed.append(result)
return result
drain.renew = counted # type: ignore[method-assign]
try:
assert asyncio.run(asyncio.wait_for(drain.run_one(), 3)) == 1
finally:
release.set()
for future in blocked:
future.result(timeout=5)
shutdown_bulk_pool()
assert renewed and all(renewed)
assert queue.get("slow").state == COMPLETED
@@ -17,7 +17,6 @@ from one_memory.memory_server.migration_runner import (
InMemoryMigrationStore,
MigrationResult,
MigrationSchemaMismatchError,
SqliteMigrationStore,
ledger_sql,
migrate,
)
@@ -139,76 +138,6 @@ def test_sdk_write_receipts_are_owned_by_migration_56() -> None:
assert "CREATE TABLE IF NOT EXISTS memory_task_sdk_receipts" in ddl
def test_onboarding_progress_migrates_old_sqlite_columns_in_append_order(tmp_path) -> None:
path = tmp_path / "old-onboarding.sqlite3"
with sqlite3.connect(path) as database:
database.execute(
"""
CREATE TABLE onboarding_state (
singleton INTEGER PRIMARY KEY CHECK (singleton = 1),
status TEXT NOT NULL,
selected_source_ids_json TEXT NOT NULL,
started_at TEXT,
completed_at TEXT,
updated_at TEXT NOT NULL
)
"""
)
database.execute(
"INSERT INTO onboarding_state VALUES "
"(1,'importing','[\"cursor\"]',NULL,NULL,'2026-11-04T00:00:00+00:00')"
)
migration = next(item for item in MIGRATIONS if item.version == 62)
result = asyncio.run(migrate(SqliteMigrationStore(path), (migration,)))
assert result.applied == (62,)
with sqlite3.connect(path) as database:
columns = [str(row[1]) for row in database.execute("PRAGMA table_info(onboarding_state)")]
row = database.execute(
"SELECT mode_intent, setup_step, model_profile_id FROM onboarding_state"
).fetchone()
assert columns == [
"singleton",
"status",
"selected_source_ids_json",
"started_at",
"completed_at",
"updated_at",
"mode_intent",
"setup_step",
"model_profile_id",
]
assert row == (None, None, None)
def test_onboarding_progress_mysql_retries_each_duplicate_column() -> None:
class DuplicateColumnError(Exception):
pass
class Cursor:
dialect = "mysql"
def __init__(self) -> None:
self.attempted: list[str] = []
self.applied: list[str] = []
async def execute(self, statement: str) -> None:
self.attempted.append(statement)
if "ADD COLUMN mode_intent" in statement or "ADD COLUMN setup_step" in statement:
raise DuplicateColumnError(1060, "duplicate column")
self.applied.append(statement)
cursor = Cursor()
migration = next(item for item in MIGRATIONS if item.version == 62)
asyncio.run(migration.apply(cursor))
assert len(cursor.attempted) == 3
assert len(cursor.applied) == 1
assert "model_profile_id" in cursor.applied[0]
def test_sdk_matter_references_are_owned_by_migration_57() -> None:
from one_memory.memory_server.migrations import MIGRATIONS, sdk_matter_references_ddl
@@ -259,7 +259,7 @@ def test_old_branch_job_executes_its_frozen_revision_after_harness_rewind(tmp_pa
):
assert request_id == old_job.request_id
captured.extend(records)
await before_hy_invocation(True)
await before_hy_invocation()
return SimpleNamespace(raw_ids=(), memory_ids=())
asyncio.run(
+1 -12
View File
@@ -184,17 +184,6 @@ def test_a_working_embedding_endpoint_does_not_clear_a_broken_chat_model() -> No
assert backoff.paused()
def test_claim_allowance_can_pace_embedding_without_pausing_lite_for_chat() -> None:
backoff = ModelBackoff(clock=_Clock())
backoff.record_failure("model_not_found", "p:1", "chat")
assert backoff.allowance(kind="embedding") is None
assert backoff.allowance(kind="chat") == 0
backoff.reset()
backoff.record_failure("embedding_service_failed", "p:1", "embedding")
assert backoff.allowance(kind="embedding") == 0
def test_an_unreachable_provider_is_paused_like_a_refusing_one() -> None:
clock = _Clock()
backoff = ModelBackoff(clock=clock)
@@ -321,7 +310,7 @@ class _FakeSdk:
self.fail_with: str | None = None
async def process_sdk_job(self, records, *, request_id, before_hy_invocation):
await before_hy_invocation(True)
await before_hy_invocation()
self.request_ids.append(request_id)
if request_id in self.outcomes:
if self.outcomes[request_id] == "failed":
@@ -91,51 +91,6 @@ def test_reserved_before_transport_and_atomic_result_reopens_without_resending(e
asyncio.run(scenario())
def test_pre_dispatch_failure_marks_operation_failed_and_unfences(env, monkeypatch):
store, secrets, profile, ops, calls, respond = env
operation = str(uuid4())
def fail_require(*args, **kwargs):
raise RuntimeError("profile load failed")
prober = ModelProber(store=store, secrets=secrets, transport=httpx.MockTransport(respond))
with monkeypatch.context() as patch:
patch.setattr(store, "require", fail_require)
receipt = asyncio.run(
prober.probe(
profile.profile_id,
operation_id=operation,
expected_profile_revision=profile.revision,
)
)
assert receipt.state == "failed"
assert receipt.code == "model_probe_execution_failed"
assert receipt.result is None
assert calls == []
assert ops.lookup(operation) == receipt
assert (
asyncio.run(
prober.probe(
profile.profile_id,
operation_id=operation,
expected_profile_revision=profile.revision,
)
)
== receipt
)
completed = asyncio.run(
prober.probe(
profile.profile_id,
operation_id=str(uuid4()),
expected_profile_revision=profile.revision,
)
)
assert completed.state == "completed"
assert len(calls) == 3
def test_same_id_and_same_profile_concurrency_refused_before_second_call(env):
store, secrets, profile, ops, calls, respond = env
@@ -35,12 +35,6 @@ def test_profile_uses_one_model_and_keeps_key_out_of_database(tmp_path) -> None:
)
assert profile.has_api_key is True
assert profile.embedding_provider_kind is ProviderKind.LOCAL_ONNX
assert profile.embedding_model == "bge-small-zh-v1.5"
assert profile.embedding_dimensions == 512
assert profile.timeout_seconds == 240
assert profile.max_concurrency == 4
assert profile.context_window is None
assert "api_key" not in profile.model_dump()
assert API_KEY.encode() not in database.read_bytes()
assert secrets.resolve(store.secret_ref(profile.profile_id) or "") == API_KEY
@@ -1,17 +1,13 @@
"""Onboarding endpoints manage UI state without touching SDK memory."""
import asyncio
import sqlite3
from datetime import UTC, datetime
from pathlib import Path
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.onboarding import OnboardingStore
def _app(tmp_path: Path):
@@ -67,29 +63,13 @@ def test_reset_returns_to_first_run_without_touching_sdk_data(tmp_path: Path) ->
sdk_marker = tmp_path / "data" / "hy-memory" / "governed-v2" / "memory.marker"
sdk_marker.parent.mkdir(parents=True, exist_ok=True)
sdk_marker.write_text("keep-sdk-memory", encoding="utf-8")
progress = await client.patch(
"/api/v1/onboarding/progress",
json={
"mode_intent": "pro",
"setup_step": "model",
"model_profile_id": "9be6d465-4cf2-4c72-93f7-15f3c012bc80",
},
)
assert progress.status_code == 200
completed = await client.post("/api/v1/onboarding/complete")
assert completed.status_code == 200
assert completed.json()["mode_intent"] is None
assert completed.json()["setup_step"] is None
assert completed.json()["model_profile_id"] is None
assert (await client.post("/api/v1/onboarding/complete")).status_code == 200
reset = await client.post("/api/v1/onboarding/reset")
assert reset.status_code == 200
assert reset.json()["status"] == "not_started"
assert reset.json()["completed_at"] is None
assert reset.json()["mode_intent"] is None
assert reset.json()["setup_step"] is None
assert reset.json()["model_profile_id"] is None
assert (await client.get("/api/v1/onboarding")).json()["status"] == "not_started"
assert sdk_marker.read_text(encoding="utf-8") == "keep-sdk-memory"
@@ -112,7 +92,6 @@ def test_sdk_unavailability_cannot_block_onboarding_completion(tmp_path: Path) -
async def scenario() -> None:
app = _app(tmp_path)
async with app.router.lifespan_context(app), _client(app) as client:
async def unavailable() -> dict[str, object]:
raise RuntimeError("hy_sdk_unavailable")
@@ -123,204 +102,3 @@ def test_sdk_unavailability_cannot_block_onboarding_completion(tmp_path: Path) -
assert finished.json()["status"] == "completed"
asyncio.run(scenario())
def test_progress_patch_and_put_round_trip_only_resumable_choices(tmp_path: Path) -> None:
async def scenario() -> None:
app = _app(tmp_path)
async with app.router.lifespan_context(app), _client(app) as client:
patched = await client.patch(
"/api/v1/onboarding/progress",
json={"mode_intent": "pro", "setup_step": "mode"},
)
assert patched.status_code == 200
assert patched.json()["mode_intent"] == "pro"
assert patched.json()["setup_step"] == "mode"
assert patched.json()["model_profile_id"] is None
profile_id = "9be6d465-4cf2-4c72-93f7-15f3c012bc80"
replaced = await client.put(
"/api/v1/onboarding/progress",
json={"setup_step": "model", "model_profile_id": profile_id},
)
assert replaced.status_code == 200
assert replaced.json()["mode_intent"] == "pro"
assert replaced.json()["setup_step"] == "model"
assert replaced.json()["model_profile_id"] == profile_id
assert (await client.get("/api/v1/onboarding")).json() == replaced.json()
ultra = await client.put(
"/api/v1/onboarding/progress",
json={"mode_intent": "ultra", "setup_step": "model"},
)
assert ultra.status_code == 200
assert ultra.json()["mode_intent"] == "ultra"
asyncio.run(scenario())
def test_progress_rejects_unknown_or_invalid_fields(tmp_path: Path) -> None:
async def scenario() -> None:
app = _app(tmp_path)
async with app.router.lifespan_context(app), _client(app) as client:
secret = await client.patch(
"/api/v1/onboarding/progress",
json={"mode_intent": "lite", "api_key": "must-not-be-stored"},
)
invalid_id = await client.put(
"/api/v1/onboarding/progress",
json={"model_profile_id": "not-a-profile-id"},
)
invalid_mode = await client.patch(
"/api/v1/onboarding/progress",
json={"mode_intent": "future"},
)
assert secret.status_code == 422
assert "must-not-be-stored" not in secret.text
assert invalid_id.status_code == 422
assert invalid_mode.status_code == 422
state = (await client.get("/api/v1/onboarding")).json()
assert state["mode_intent"] is None
assert state["model_profile_id"] is None
asyncio.run(scenario())
def test_progress_row_adapter_accepts_mysql_tuple_shape() -> None:
started_at = datetime(2026, 11, 4, 1, 0, tzinfo=UTC)
updated_at = datetime(2026, 11, 4, 1, 1)
state = OnboardingStore._to_state(
(
1,
"importing",
'["cursor"]',
started_at,
None,
updated_at,
"lite",
"review",
None,
),
mysql=True,
)
assert state.selected_source_ids == ("cursor",)
assert state.started_at == started_at
assert state.mode_intent == "lite"
assert state.setup_step == "review"
def test_progress_row_adapter_accepts_real_sqlite_row_and_text_datetime() -> None:
with sqlite3.connect(":memory:") as connection:
connection.row_factory = sqlite3.Row
row = connection.execute(
"SELECT 1 AS singleton, 'importing' AS status, '[\"cursor\"]' "
"AS selected_source_ids_json, '2026-11-04T01:00:00+00:00' AS started_at, "
"NULL AS completed_at, '2026-11-04T01:01:00+00:00' AS updated_at, "
"'pro' AS mode_intent, 'model' AS setup_step, "
"'9be6d465-4cf2-4c72-93f7-15f3c012bc80' AS model_profile_id"
).fetchone()
assert row is not None
state = OnboardingStore._to_state(row, mysql=False)
assert state.started_at == datetime(2026, 11, 4, 1, 0, tzinfo=UTC)
assert state.mode_intent == "pro"
assert state.setup_step == "model"
def test_progress_row_adapter_rejects_wrong_length() -> None:
with pytest.raises(ValueError, match="onboarding_row_shape_invalid"):
OnboardingStore._to_state((1, "importing"), mysql=True)
def test_progress_row_adapter_rejects_wrong_row_types() -> None:
values = (
1,
"importing",
"[]",
None,
None,
datetime(2026, 11, 4, 1, 1),
None,
None,
None,
)
with pytest.raises(ValueError, match="onboarding_row_shape_invalid"):
OnboardingStore._to_state(list(values), mysql=True)
with pytest.raises(ValueError, match="onboarding_row_shape_invalid"):
OnboardingStore._to_state(values, mysql=False)
def test_progress_row_adapter_rejects_wrong_engine_datetime_shape() -> None:
with pytest.raises(ValueError, match="onboarding_datetime_invalid"):
OnboardingStore._to_state(
(
1,
"importing",
"[]",
"2026-11-04T01:00:00+00:00",
None,
datetime(2026, 11, 4, 1, 1),
None,
None,
None,
),
mysql=True,
)
def test_runtime_sqlite_upgrade_does_not_record_progress_migration(tmp_path: Path) -> None:
database = tmp_path / "old.sqlite3"
with sqlite3.connect(database) as connection:
connection.executescript(
"""
CREATE TABLE schema_migrations (
version INTEGER PRIMARY KEY,
name TEXT NOT NULL,
applied_at TEXT NOT NULL
);
INSERT INTO schema_migrations VALUES (61, 'write_job_retries', '2026-11-03');
CREATE TABLE onboarding_state (
singleton INTEGER PRIMARY KEY CHECK (singleton = 1),
status TEXT NOT NULL,
selected_source_ids_json TEXT NOT NULL,
started_at TEXT,
completed_at TEXT,
updated_at TEXT NOT NULL
);
"""
)
OnboardingStore(database)
with sqlite3.connect(database) as connection:
recorded = connection.execute(
"SELECT name FROM schema_migrations WHERE version=62"
).fetchone()
columns = {str(row[1]) for row in connection.execute("PRAGMA table_info(onboarding_state)")}
assert recorded is None
assert {"mode_intent", "setup_step", "model_profile_id"} <= columns
def test_progress_rejects_updates_after_completion(tmp_path: Path) -> None:
async def scenario() -> None:
app = _app(tmp_path)
async with app.router.lifespan_context(app), _client(app) as client:
assert (await client.post("/api/v1/onboarding/complete")).status_code == 200
populated = await client.patch(
"/api/v1/onboarding/progress",
json={"mode_intent": "lite"},
)
empty = await client.put("/api/v1/onboarding/progress", json={})
assert populated.status_code == 409
assert populated.json()["detail"] == "onboarding_completed"
assert empty.status_code == 409
state = (await client.get("/api/v1/onboarding")).json()
assert state["status"] == "completed"
assert state["mode_intent"] is None
asyncio.run(scenario())
@@ -1,237 +0,0 @@
"""MemHub's raw search against the hy-memory wheel it pins, not the repo source.
The suite's pythonpath puts the repository root first, so `import hy_memory`
in a test reads the repo's `hy_memory/`, which can be ahead of the pinned
release. Search kwargs the release does not accept then pass every test and
fail every raw search in a packaged build (a `statuses` argument did exactly
that against 2.0.13). These tests run in a child interpreter started with
`-I`, so `hy_memory` resolves to the wheel installed in the Core venv, and
drive the real routes over a real lite SDK client with a deterministic local
embedder (no model, no network, no ONNX assets).
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
import textwrap
import pytest
from tests.test_sdk_api_surface import SDK_SEARCH_PARAMETERS
pytestmark = pytest.mark.integration
_CHILD = textwrap.dedent(
r'''
import asyncio
import hashlib
import inspect
import json
import math
import re
import sys
from importlib.metadata import version
from pathlib import Path
import hy_memory
from hy_memory import HyMemoryClient
from one_memory.hy_runtime_config import HY_MEMORY_VERSION
home = Path(sys.argv[1])
wheel = Path(hy_memory.__file__).resolve()
assert "site-packages" in wheel.parts, f"not the installed wheel: {wheel}"
assert version("hy-memory") == HY_MEMORY_VERSION, version("hy-memory")
parameters = sorted(
name
for name, parameter in inspect.signature(HyMemoryClient.search).parameters.items()
if name not in {"self", "query"} and parameter.kind is not inspect.Parameter.VAR_KEYWORD
)
var_keyword = any(
parameter.kind is inspect.Parameter.VAR_KEYWORD
for parameter in inspect.signature(HyMemoryClient.search).parameters.values()
)
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
import one_memory.api as api
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.embedding_assets import BUILTIN_EMBEDDING_DIMENSIONS
from one_memory.hy_sdk_runtime import (
LocalOnnxEmbeddingService,
SdkClientHandle,
builtin_retrieval_snapshot,
)
from one_memory.secrets import InMemorySecretStore
WORD = re.compile(r"\w+")
class HashedEmbedder:
"""Bag of words and CJK bigrams, hashed: texts sharing words are near."""
dimensions = BUILTIN_EMBEDDING_DIMENSIONS
def encode(self, texts):
vectors = []
for text in texts:
vector = [0.0] * self.dimensions
for word in WORD.findall(text.casefold()):
grams = [word]
if not word.isascii():
grams += [word[i : i + 2] for i in range(len(word) - 1)]
for gram in grams:
digest = hashlib.sha256(gram.encode()).digest()
vector[int.from_bytes(digest[:4], "big") % self.dimensions] += 1.0
norm = math.sqrt(sum(value * value for value in vector)) or 1.0
vectors.append([value / norm for value in vector])
return vectors
LocalOnnxEmbeddingService._embedder = HashedEmbedder()
async def lite_has_no_model(_app):
return False
api._recall_model_ready = lite_has_no_model
async def main():
settings = Settings(
data_dir=home / "data",
source_home=home,
api_token=SecretStr("pinned-sdk"),
imports_enabled=False,
automatic_memory_processing=False,
live_sync_enabled=False,
)
app = create_app(settings, secret_store=InMemorySecretStore())
app.state.memory_mode = lambda: "lite"
handle = SdkClientHandle(
builtin_retrieval_snapshot(),
api_key=None,
data_dir=home / "data",
mode="lite",
)
result = {"parameters": parameters, "var_keyword": var_keyword}
async with app.router.lifespan_context(app):
app.state.sdk_client_handle = handle
client = await handle.client()
for session, messages in (
(
"s-hike",
[
("user", "我叫小林"),
("assistant", "你好小林"),
("user", "I love hiking mountain trails at dawn"),
("assistant", "Noted, early hikes."),
],
),
(
"s-tea",
[
("user", "my favourite drink is green tea"),
("assistant", "Green tea, noted."),
],
),
):
written = client.add(
[{"role": role, "content": content} for role, content in messages],
user_id="local",
source="hermes",
session_id=session,
)
assert written.get("success", True), written
listed = client.list_memories(user_id="local", layers=["l1_raw"], limit=10)
rows = listed.get("vdb", listed)["memories"]
tea = next(row for row in rows if "green tea" in row["content"])
# What pro leaves behind: the raw it already processed is shadow.
client._loop_thread.run(
client._vector_store.update_payload(tea["memory_id"], {"status": "shadow"})
)
async with AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer pinned-sdk"},
) as http:
for name, body in (
("hike_turn", {"query": "hiking trails mountain", "raw_unit": "turn"}),
("tea_turn", {"query": "favourite green tea drink", "raw_unit": "turn"}),
("tea_raw", {"query": "favourite green tea drink"}),
):
response = await http.post(
"/api/v2/context/search", json={"kinds": ["raw"], "limit": 5, **body}
)
result[name] = {"status": response.status_code, "body": response.json()}
response = await http.post(
"/api/v1/recall", json={"query": "favourite green tea drink"}
)
result["recall"] = {"status": response.status_code, "body": response.json()}
await handle.close()
print("RESULT " + json.dumps(result, ensure_ascii=False))
asyncio.run(main())
'''
)
@pytest.fixture(scope="module")
def pinned(tmp_path_factory) -> dict:
root = tmp_path_factory.mktemp("pinned-sdk")
script = root / "scripts" / "child.py"
script.parent.mkdir()
script.write_text(_CHILD, encoding="utf-8")
home = root / "home"
home.mkdir()
environment = {
**os.environ,
"ANONYMIZED_TELEMETRY": "False",
"ONE_MEMORY_DATA_DIR": str(home / "data"),
}
completed = subprocess.run(
[sys.executable, "-I", str(script), str(home)],
cwd=home,
env=environment,
capture_output=True,
text=True,
timeout=240,
check=False,
)
assert completed.returncode == 0, completed.stderr[-4000:]
[line] = [line for line in completed.stdout.splitlines() if line.startswith("RESULT ")]
return json.loads(line.removeprefix("RESULT "))
def test_the_fake_sdk_accepts_exactly_the_pinned_search_parameters(pinned) -> None:
# The route tests' stand-in rejects what the pinned SDK rejects.
assert pinned["var_keyword"] is False
assert sorted(SDK_SEARCH_PARAMETERS) == pinned["parameters"]
def test_lite_turn_search_returns_semantic_hits_without_the_literal_query(pinned) -> None:
hike = pinned["hike_turn"]
assert hike["status"] == 200, hike
items = hike["body"]["items"]
assert items, hike["body"]
top = items[0]
# No turn contains "hiking trails mountain"; the hit still comes back,
# shown by its closest turn and carrying the hit's score.
assert "hiking mountain trails" in top["content"]
assert "小林" not in top["content"]
assert top["turn_index"] == 1 and top["turn_count"] == 2
assert [item["score"] for item in items] == sorted(
(item["score"] for item in items), reverse=True
)
def test_lite_search_and_recall_include_shadow_raws(pinned) -> None:
for name in ("tea_turn", "tea_raw"):
result = pinned[name]
assert result["status"] == 200, result
assert any("green tea" in item["content"] for item in result["body"]["items"]), result
recall = pinned["recall"]
assert recall["status"] == 200, recall
assert recall["body"]["mode"] == "raw"
assert any("green tea" in hit["content"] for hit in recall["body"]["hits"]), recall
@@ -189,11 +189,7 @@ def test_first_import_gate_waits_for_every_selected_history_job() -> None:
app = SimpleNamespace(
state=SimpleNamespace(
onboarding_store=SimpleNamespace(get=lambda: state),
import_job_store=SimpleNamespace(
find_jobs=lambda *, user_id, operation: [
job for job in jobs if job.user_id == user_id and job.operation == operation
]
),
import_job_store=SimpleNamespace(list_jobs=lambda: jobs),
)
)
@@ -77,36 +77,6 @@ def test_workbuddy_record_parsing_never_blocks_the_event_loop(tmp_path) -> None:
asyncio.run(scenario())
def test_jsonl_live_probe_listing_is_reused_by_the_import(tmp_path, monkeypatch) -> None:
root = tmp_path / ".workbuddy" / "projects"
write_jsonl(
root / "project" / "session.jsonl",
[{"role": "user", "sessionId": "session", "content": "new conversation"}],
)
adapter = create_workbuddy_adapter(grant("workbuddy", root))
from one_memory.imports.sources import common
calls = 0
original = common.list_changed_files
def counted(*args, **kwargs):
nonlocal calls
calls += 1
return original(*args, **kwargs)
monkeypatch.setattr(common, "list_changed_files", counted)
async def probe_then_scan():
options = ScanOptions()
assert await adapter.has_changes(options)
return [event async for event in adapter.scan(options)]
records = evidence(asyncio.run(probe_then_scan()))
assert [item.evidence.verbatim for item in records] == ["new conversation"]
assert calls == 1
def test_cursor_store_user_info_envelope_is_not_attributed_to_the_person(tmp_path) -> None:
"""A Cursor store blob can be the host's session context, not the user's words.
@@ -1382,68 +1352,6 @@ def test_cursor_transcripts_with_the_same_name_in_two_projects_are_both_kept(
assert len({item.evidence.source_ordinal for item in records}) == 2
def test_cursor_discovery_does_not_enter_non_conversation_trees(tmp_path) -> None:
root = tmp_path / ".cursor"
write_jsonl(
root / "projects" / "project" / "agent-transcripts" / "kept" / "kept.jsonl",
[
{
"role": "user",
"message": {"content": [{"type": "text", "text": "kept conversation"}]},
}
],
)
write_jsonl(
root / "extensions" / "large-cache" / "agent-transcripts" / "ignored" / "ignored.jsonl",
[
{
"role": "user",
"message": {"content": [{"type": "text", "text": "must not be scanned"}]},
}
],
)
records = evidence(asyncio.run(scan(CursorUnifiedSourceAdapter(grant=grant("cursor", root)))))
assert [item.evidence.verbatim for item in records] == ["kept conversation"]
def test_cursor_live_probe_listing_is_reused_by_the_import(tmp_path, monkeypatch) -> None:
root = tmp_path / ".cursor"
write_jsonl(
root / "projects" / "project" / "agent-transcripts" / "new" / "new.jsonl",
[
{
"role": "user",
"message": {"content": [{"type": "text", "text": "new conversation"}]},
}
],
)
adapter = CursorUnifiedSourceAdapter(grant=grant("cursor", root))
from one_memory.imports.adapters import ScanOptions
from one_memory.imports.sources import cursor_unified
calls = 0
original = cursor_unified._list_cursor_files
def counted(*args, **kwargs):
nonlocal calls
calls += 1
return original(*args, **kwargs)
monkeypatch.setattr(cursor_unified, "_list_cursor_files", counted)
async def probe_then_scan():
options = ScanOptions()
assert await adapter.has_changes(options)
return [event async for event in adapter.scan(options)]
records = evidence(asyncio.run(probe_then_scan()))
assert [item.evidence.verbatim for item in records] == ["new conversation"]
assert calls == 1
def test_cursor_bounded_scans_reuse_the_authorized_file_listing(tmp_path, monkeypatch) -> None:
root = tmp_path / ".cursor"
transcript = root / "projects" / "project" / "agent-transcripts" / "session" / "session.jsonl"
@@ -2,9 +2,8 @@
Without a usable generation model no memory can be distilled, and with no
memory in the library a memory search can only come back empty. In both cases
recall searches the SDK raw layer and says so with `mode: "raw"`. External
callers need the conversations tool and all-apps scope; Core resolves that to
every connected, unrevoked source.
recall searches the SDK raw layer and says so with `mode: "raw"`. Original
conversations stay under their own grant: recall's grant does not widen it.
"""
import asyncio
@@ -19,7 +18,6 @@ from one_memory import api
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.hy_sdk_types import HySdkRawHit
from one_memory.imports.sources.registry import RootCandidateKind
from one_memory.recall import RecallHit, RecallReceipt, RecallRequest, RecallState
from one_memory.recall_fallback import recall_with_fallback
from one_memory.secrets import InMemorySecretStore
@@ -158,26 +156,6 @@ def test_raw_fallback_respects_the_callers_raw_sources():
assert [hit.memory_id for hit in receipt.hits] == ["r1"]
def test_raw_fallback_rechecks_sources_after_search_before_releasing_content():
_, fakes = _fakes(
ready=False,
memories=[],
has_memories=False,
raw=[_raw("r1", "private", source="cursor")],
sources=frozenset({"cursor"}),
)
checks = iter((frozenset({"cursor"}), frozenset()))
async def sources():
return next(checks)
fakes["raw_sources"] = sources
receipt = run(RecallRequest(query="private"), fakes)
assert receipt.state == RecallState.EMPTY
assert receipt.hits == ()
def test_a_caller_with_no_raw_grant_gets_nothing_rather_than_a_wider_read():
calls, fakes = _fakes(
ready=False,
@@ -209,43 +187,6 @@ def test_raw_fallback_keeps_the_count_and_character_budgets():
assert by_chars.used_characters == 300
def test_oversized_raw_hit_returns_a_query_anchored_excerpt_instead_of_empty():
_, fakes = _fakes(
ready=False,
memories=[],
has_memories=False,
raw=[_raw("large", ("x" * 5_000) + "WebGL 黄色节点" + ("y" * 5_000))],
)
receipt = run(
RecallRequest(query="WebGL 黄色节点", limit=3, character_budget=300),
fakes,
)
assert receipt.state == RecallState.GROUNDED
assert len(receipt.hits) == 1
assert len(receipt.hits[0].content) == receipt.used_characters == 300
assert "WebGL 黄色节点" in receipt.hits[0].content
assert receipt.truncated_by.value == "character_budget"
@pytest.mark.parametrize("prefix_length", [0, 100, 5_000, 9_980])
def test_raw_excerpt_never_exceeds_its_character_budget(prefix_length):
needle = "WebGL"
content = ("x" * prefix_length) + needle + ("y" * (10_000 - prefix_length))
_, fakes = _fakes(
ready=False,
memories=[],
has_memories=False,
raw=[_raw("large", content)],
)
receipt = run(RecallRequest(query=needle, character_budget=300), fakes)
assert len(receipt.hits[0].content) == receipt.used_characters == 300
assert needle in receipt.hits[0].content
def test_the_probe_never_falls_back():
calls, fakes = _fakes(ready=False, memories=[], has_memories=False, raw=[_raw("r1", "x")])
@@ -296,14 +237,6 @@ def test_recall_route_falls_back_and_keeps_the_raw_grant(tmp_path, monkeypatch):
headers={"Authorization": "Bearer recall-test"},
) as client,
):
cursor_root = tmp_path / "cursor"
cursor_root.mkdir()
app.state.source_grant_store.grant(
user_id="local",
source_id="cursor",
root=cursor_root,
candidate_kind=RootCandidateKind.DEFAULT,
)
local = await client.post("/api/v1/recall", json={"query": "数据库"})
recall_only = await client.post(
"/api/v1/recall",
@@ -314,18 +247,9 @@ def test_recall_route_falls_back_and_keeps_the_raw_grant(tmp_path, monkeypatch):
"X-One-Memory-Allowed-Scopes": json.dumps(["all-apps"]),
},
)
cross_app = await client.post(
"/api/v1/recall",
json={"query": "数据库"},
headers={
**EXTERNAL,
"X-One-Memory-Allowed-Tools": json.dumps(["om_recall", "om_conversations"]),
"X-One-Memory-Allowed-Scopes": json.dumps(["all-apps", "source:workbuddy"]),
},
)
return local.json(), recall_only.json(), cross_app.json()
return local.json(), recall_only.json()
local, recall_only, cross_app = asyncio.run(scenario())
local, recall_only = asyncio.run(scenario())
assert local["mode"] == "raw" and local["fallback_reason"] == "model_unavailable"
assert local["hits"][0]["content_format"] == "raw"
@@ -334,9 +258,7 @@ def test_recall_route_falls_back_and_keeps_the_raw_grant(tmp_path, monkeypatch):
assert searched[0] is None
# A host granted recall alone may not read original conversations.
assert recall_only["mode"] == "raw" and recall_only["hits"] == []
# all-apps covers every connected source, not only the host's source scope.
assert cross_app["hits"][0]["source"] == "cursor"
assert searched == [None, frozenset({"cursor"})]
assert len(searched) == 1
def test_the_total_recall_budget_stays_under_the_brokers_wait():
+3 -363
View File
@@ -42,10 +42,7 @@ RAW_ROWS = {
"local": [
{
"memory_id": "raw-1",
"content": (
"[user]: 我叫费天翔\n[assistant]: 好的,记住了。"
"\n[user]: 我喜欢绿茶\n[assistant]: 也记住了。"
),
"content": "[user]: 我叫费天翔\n[assistant]: 好的,记住了。",
"status": "shadow",
"session_id": "s-1",
"source": "hermes",
@@ -91,40 +88,6 @@ def _chat(user_id: str, memory_id: str, content: str, layer: str, source: str) -
}
#: The keyword parameters of the pinned `HyMemoryClient.search`, which takes
#: no **kwargs. `test_pinned_sdk_raw_search` checks this set against the
#: installed wheel, so the stand-in below rejects what the release rejects.
SDK_SEARCH_PARAMETERS = frozenset(
{
"queries",
"scene",
"search_mode",
"user_ids",
"agent_ids",
"session_ids",
"include_sources",
"exclude_sources",
"limit",
"min_score",
"working_fact_min_score",
"working_experience_min_score",
"schema_min_score",
"schema_limit",
"intention_limit",
"created_after",
"reader",
"workspace_id",
"branch",
"request_id",
"layers",
"statuses",
"embed_override",
"query_embedding",
"rerank",
}
)
class RecordingSdk:
"""The public client surface, recording every call's keyword arguments."""
@@ -143,10 +106,6 @@ class RecordingSdk:
# The SDK answers rows below a caller's floor in some lanes (a schema
# floor, a fused rank score), so Core must filter what it returns.
def search(self, query, **kwargs):
unknown = set(kwargs) - SDK_SEARCH_PARAMETERS
if unknown:
# The pinned `HyMemoryClient.search` takes no **kwargs.
raise TypeError(f"search() got unexpected keyword arguments {sorted(unknown)}")
self._record("search", {"query": query, **kwargs})
[user_id] = kwargs["user_ids"]
if kwargs.get("layers") == ["l1_raw"]:
@@ -214,15 +173,7 @@ class RecordingSdk:
)
and (not kwargs.get("session_ids") or row["session_id"] in kwargs["session_ids"])
]
total = len(rows)
offset = kwargs.get("offset", 0)
limit = kwargs.get("limit", 100)
return {
"vdb": {
"memories": rows[offset : offset + limit],
"total": total,
}
}
return {"vdb": {"memories": rows, "total": len(rows)}}
rows = [_chat(user_id, *row[:2], *row[3:]) for row in CHAT_ROWS.get(user_id, [])]
if layers:
rows = [row for row in rows if row["layer"] in layers]
@@ -320,10 +271,6 @@ async def _model_ready(_app) -> bool:
return True
async def _model_not_ready(_app) -> bool:
return False
def run_with_sdk(tmp_path, monkeypatch, scenario) -> None:
async def main() -> None:
app = application(tmp_path)
@@ -400,170 +347,6 @@ def test_every_context_search_option_reaches_the_sdk_and_absent_ones_keep_the_ol
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_raw_context_search_can_return_turns_instead_of_storage_batches(
tmp_path, monkeypatch
) -> None:
async def scenario(app, sdk, client):
response = await client.post(
"/api/v2/context/search",
json={
"query": "绿茶",
"kinds": ["raw"],
"raw_unit": "turn",
"limit": 20,
},
)
assert response.status_code == 200, response.text
raw_one = [item for item in response.json()["items"] if item["raw_id"] == "raw-1"]
assert len(raw_one) == 1
assert raw_one[0]["turn_index"] == 1
assert "我喜欢绿茶" in raw_one[0]["content"]
assert "我叫费天翔" not in raw_one[0]["content"]
# pro/ultra: a shadow raw is already in its extracted memories.
assert "statuses" not in sdk.last("search")
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_lite_raw_search_is_plain_rag_over_active_and_shadow_raws(tmp_path, monkeypatch) -> None:
async def scenario(app, sdk, client):
app.state.memory_mode = lambda: "lite"
response = await client.post(
"/api/v2/context/search",
json={"query": "喜欢喝绿茶的人", "kinds": ["raw"], "raw_unit": "turn", "limit": 5},
)
assert response.status_code == 200, response.text
sent = sdk.last("search")
assert sent["statuses"] == ["active", "shadow"] and sent["limit"] == 5
items = response.json()["items"]
# No turn contains the query, yet every vector hit is returned, in
# score order: raw-1 by its closest turn, raw-2 by its only one.
assert [(item["raw_id"], item["turn_index"]) for item in items] == [
("raw-1", 1),
("raw-2", 0),
]
assert [item["score"] for item in items] == [0.97, 0.3]
assert "我喜欢绿茶" in items[0]["content"]
floored = await client.post(
"/api/v2/context/search",
json={
"query": "喜欢喝绿茶的人",
"kinds": ["raw"],
"raw_unit": "turn",
"min_score": 0.5,
},
)
assert [item["raw_id"] for item in floored.json()["items"]] == ["raw-1"]
# Lite has no model, so recall answers from the raw layer.
monkeypatch.setattr("one_memory.api._recall_model_ready", _model_not_ready)
recall = await client.post("/api/v1/recall", json={"query": "喜欢喝绿茶的人"})
assert recall.status_code == 200, recall.text
assert recall.json()["mode"] == "raw"
assert sdk.last("search")["statuses"] == ["active", "shadow"]
assert [hit["memory_id"] for hit in recall.json()["hits"]][:1] == ["raw-1"]
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_long_semantic_raw_hit_cannot_erase_an_earlier_match(tmp_path, monkeypatch) -> None:
async def scenario(app, sdk, client):
app.state.memory_mode = lambda: "lite"
def search(query, **kwargs):
sdk._record("search", {"query": query, **kwargs})
memories = [
{
"memory_id": "raw-target",
"content": "[user]: lite 模式的 3d 图需要调整",
"layer": "l1_raw",
"status": "active",
"user_id": "local",
"agent_id": "all-apps",
"session_id": "target-session",
"source": "cursor",
"memory_at": 1_790_100_000,
"score": 0.97,
},
{
"memory_id": "raw-long",
"content": f"[user]: {'x' * 20_001}",
"layer": "l1_raw",
"status": "active",
"user_id": "local",
"agent_id": "all-apps",
"session_id": "long-session",
"source": "cursor",
"memory_at": 1_790_000_000,
"score": 0.9,
},
]
return {
"request_id": "long-raw",
"memories": memories[1:] if query == "oversized" else memories,
}
sdk.search = search
response = await client.post(
"/api/v2/context/search",
json={
"query": "3d",
"kinds": ["raw"],
"raw_unit": "turn",
"limit": 20,
"character_budget": 20_000,
},
)
assert response.status_code == 200, response.text
body = response.json()
assert body["state"] == "grounded"
assert body["items"][0]["raw_id"] == "raw-target"
assert "3d" in body["items"][0]["content"]
assert body["items"][1]["raw_id"] == "raw-long"
assert body["items"][1]["truncated"] is True
assert body["truncated"] is True
single = await client.post(
"/api/v2/context/search",
json={
"query": "oversized",
"kinds": ["raw"],
"raw_unit": "turn",
"character_budget": 20_000,
},
)
assert single.json()["items"][0]["truncated"] is True
assert single.json()["truncated"] is True
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_raw_only_lane_failure_is_unavailable_not_an_empty_partial(tmp_path, monkeypatch) -> None:
async def scenario(app, sdk, client):
app.state.memory_mode = lambda: "lite"
def search(_query, **_kwargs):
raise ValueError("private conversation text must not reach logs")
sdk.search = search
response = await client.post(
"/api/v2/context/search",
json={"query": "3d", "kinds": ["raw"], "raw_unit": "turn"},
)
assert response.status_code == 200, response.text
assert response.json()["state"] == "unavailable"
assert response.json()["items"] == []
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_every_recall_and_ask_option_reaches_the_sdk(tmp_path, monkeypatch) -> None:
async def scenario(app, sdk, client):
@@ -791,12 +574,7 @@ def test_raw_lists_active_and_shadow_turns_with_roles_and_hashes(tmp_path, monke
assert sent["layers"] == ["l1_raw"] and sent["statuses"] == ["active", "shadow"]
first = next(item for item in body["items"] if item["raw_id"] == "raw-1")
assert first["status"] == "shadow"
assert [message["role"] for message in first["messages"]] == [
"user",
"assistant",
"user",
"assistant",
]
assert [message["role"] for message in first["messages"]] == ["user", "assistant"]
assert first["messages"][0]["content"] == "我叫费天翔"
assert first["role"] is None
expected = hashlib.sha256(first["content"].encode()).hexdigest()
@@ -806,144 +584,6 @@ def test_raw_lists_active_and_shadow_turns_with_roles_and_hashes(tmp_path, monke
second = next(item for item in body["items"] if item["raw_id"] == "raw-2")
assert second["role"] == "user"
turns = (await client.get("/api/v2/raw", params={"unit": "turn"})).json()
assert turns["unit"] == "turn" and turns["total"] == 3
first_turns = [item for item in turns["items"] if item["raw_id"] == "raw-1"]
assert [item["turn_index"] for item in first_turns] == [1, 0]
assert len({item["turn_id"] for item in first_turns}) == 2
assert [[message["content"] for message in item["messages"]] for item in first_turns] == [
["我喜欢绿茶", "也记住了。"],
["我叫费天翔", "好的,记住了。"],
]
cursor = None
first_cursor = None
cursor_turns = []
cursor_calls = len(sdk.all("list_memories"))
while True:
params = {"unit": "turn", "paging": "cursor", "limit": 1}
if cursor is not None:
params["cursor"] = cursor
response = await client.get("/api/v2/raw", params=params)
assert response.status_code == 200, response.text
cursor_page = response.json()
assert cursor_page["total_exact"] is False
assert len(cursor_page["items"]) == 1
cursor_turns.append(cursor_page["items"][0]["messages"][0]["content"])
cursor = cursor_page["next_cursor"]
if first_cursor is None:
first_cursor = cursor
if cursor is None:
assert cursor_page["has_more"] is False
break
assert cursor_page["has_more"] is True
assert cursor_turns == ["我喜欢绿茶", "我叫费天翔", "你好"]
assert len(sdk.all("list_memories")) == cursor_calls + 3
graph_calls = len(sdk.all("list_memories"))
graph = await client.get("/api/v2/raw/graph", params={"limit": 2})
assert graph.status_code == 200, graph.text
assert len(sdk.all("list_memories")) == graph_calls + 1
graph_body = graph.json()
assert graph_body["total"] == 3
assert graph_body["has_more"] is True
assert [item["label"] for item in graph_body["items"]] == [
"[user]: 我喜欢绿茶 [assistant]: 也记住了。",
"[user]: 我叫费天翔 [assistant]: 好的,记住了。",
]
assert (await client.get("/api/v2/raw/graph", params={"limit": 301})).status_code == 422
assert (
await client.get(
"/api/v2/raw/graph",
params={"limit": 2, "offset": 2},
)
).status_code == 422
inserted = {
"memory_id": "raw-new",
"content": "[user]: 新消息",
"status": "active",
"session_id": "s-new",
"source": "cursor",
"memory_at": 1_790_200_000,
"gmt_created": 1_790_200_100,
}
RAW_ROWS["local"].insert(0, inserted)
try:
updated_graph = (await client.get("/api/v2/raw/graph")).json()
assert updated_graph["total"] == 4
assert updated_graph["items"][0]["label"] == "[user]: 新消息"
continued = await client.get(
"/api/v2/raw",
params={
"unit": "turn",
"paging": "cursor",
"limit": 1,
"cursor": first_cursor,
},
)
assert continued.status_code == 200, continued.text
assert continued.json()["items"][0]["messages"][0]["content"] == "我叫费天翔"
finally:
RAW_ROWS["local"].remove(inserted)
assert (
await client.get(
"/api/v2/raw",
params={
"unit": "turn",
"paging": "cursor",
"cursor": first_cursor,
"source": "hermes",
},
)
).status_code == 422
assert (
await client.get(
"/api/v2/raw",
params={
"unit": "turn",
"paging": "cursor",
"cursor": "turn:²:0:0:2::0000000000000000",
},
)
).status_code == 422
calls_before_refresh = len(sdk.all("list_memories"))
await client.get("/api/v2/raw", params={"unit": "turn"})
assert len(sdk.all("list_memories")) == calls_before_refresh + 1
# A live session puts a new raw on top: one listing call, not a rescan.
newest = {
**RAW_ROWS["local"][1],
"memory_id": "raw-new",
"content": "[user]: 刚刚说的",
"status": "active",
}
original = RAW_ROWS["local"]
monkeypatch.setitem(RAW_ROWS, "local", [newest, *original])
calls_before_append = len(sdk.all("list_memories"))
appended = (await client.get("/api/v2/raw", params={"unit": "turn"})).json()
assert len(sdk.all("list_memories")) == calls_before_append + 1
assert appended["total"] == 4
assert [item["raw_id"] for item in appended["items"]][:1] == ["raw-new"]
assert [item["turn_id"] for item in appended["items"][1:]] == [
item["turn_id"] for item in turns["items"]
]
# Anything else (here a raw going away) lists the raw layer again.
monkeypatch.setitem(RAW_ROWS, "local", [newest, original[1]])
calls_before_delete = len(sdk.all("list_memories"))
shrunk = (await client.get("/api/v2/raw", params={"unit": "turn"})).json()
assert len(sdk.all("list_memories")) > calls_before_delete + 1
assert shrunk["total"] == 2
monkeypatch.setitem(RAW_ROWS, "local", original)
calls_before_restore = len(sdk.all("list_memories"))
restored = (await client.get("/api/v2/raw", params={"unit": "turn"})).json()
assert len(sdk.all("list_memories")) > calls_before_restore + 1
assert restored["total"] == 3
ascending = (
await client.get("/api/v2/raw", params={"unit": "turn", "order": "asc"})
).json()
ascending_first = [item for item in ascending["items"] if item["raw_id"] == "raw-1"]
assert [item["turn_index"] for item in ascending_first] == [0, 1]
await client.get(
"/api/v2/raw",
params=[("source", "hermes,codex"), ("session_id", "s-1"), ("order", "asc")],
@@ -1613,7 +1613,7 @@ def test_a_completed_sdk_write_does_not_touch_retired_evidence(store) -> None:
class Supervisor:
async def process_sdk_job(self, records, *, request_id, before_hy_invocation):
del records, request_id
await before_hy_invocation(True)
await before_hy_invocation()
return SimpleNamespace(raw_ids=("raw-1",), memory_ids=("m-1",))
asyncio.run(
@@ -91,7 +91,7 @@ def test_bundle_contains_management_sdk_config_and_verified_manifest(tmp_path):
assert manifest["bundle_id"] == "migration-1"
assert manifest["sdk_namespace"] == "governed-v2"
assert manifest["runtime_build"] == "build-13"
assert manifest["snapshot_sdk_version"] == "2.0.14"
assert manifest["snapshot_sdk_version"] == "2.0.13"
paths = {item["path"]: item for item in manifest["files"]}
assert {
"management/one-memory.sqlite3",
@@ -293,22 +293,6 @@ def _retry_succeeds(home: Path, monkeypatch) -> None:
assert len(list((home / "backups").glob("*-pre-cutover.sdk-bundle"))) == 1
def test_evidence_an_older_capture_hook_recreated_is_dropped_on_the_next_start(home):
# Builds through 1.0.6 let the hook process open `EvidenceStore`, which
# rebuilt the retired tables on an SDK home. Nothing reads them there, and
# the next startup migration's leftovers cleanup removes them again.
assert _run(home).state == "cutover"
database = home / "one-memory.sqlite3"
EvidenceStore(database)
assert "evidence" in legacy_tables_present(database)
result = _run(home)
assert result.state == "complete"
assert {"evidence", "evidence_search"} <= set(result.dropped_tables)
assert legacy_tables_present(database) == ()
def test_a_failure_after_the_cutover_commit_is_not_restored(home, monkeypatch):
import one_memory.sdk_migration_runner as runner
@@ -1,12 +1,13 @@
"""The sedimentation API persists decisions without rewriting parked work."""
"""The sedimentation API persists decisions and resumes the SDK queue."""
import asyncio
import threading
from fastapi.testclient import TestClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.memory_server.contracts import BatchIdentity, WriteRequest
from one_memory.memory_server.durable_queue import WAITING_FOR_PERMISSION
def _app(tmp_path):
@@ -24,62 +25,51 @@ def _app(tmp_path):
)
def test_sedimentation_route_persists_without_restart(tmp_path) -> None:
def test_sedimentation_route_persists_and_resumes_without_restart(tmp_path) -> None:
with TestClient(_app(tmp_path), headers={"Authorization": "Bearer test"}) as client:
assert client.get("/api/v1/sedimentation").json()["enabled"] is True
assert (
client.put("/api/v1/sedimentation", json={"enabled": False}).json()["enabled"] is False
)
assert client.put(
"/api/v1/sedimentation", json={"enabled": False}
).json()["enabled"] is False
assert client.get("/api/v1/sedimentation").json()["enabled"] is False
assert client.put("/api/v1/sedimentation", json={"enabled": True}).json()["enabled"] is True
assert client.put(
"/api/v1/sedimentation", json={"enabled": True}
).json()["enabled"] is True
def test_switch_does_not_run_the_broad_parked_queue_resume(tmp_path, monkeypatch) -> None:
def test_switch_answers_before_sdk_queue_resume_finishes(tmp_path, monkeypatch) -> None:
app = _app(tmp_path)
called = False
released = threading.Event()
finished = threading.Event()
with TestClient(app, headers={"Authorization": "Bearer test"}) as client:
async def slow_resume() -> None:
await asyncio.get_running_loop().run_in_executor(None, released.wait, 10.0)
finished.set()
async def forbidden_resume() -> None:
nonlocal called
called = True
monkeypatch.setattr(app.state.sdk_memory_runtime, "resume_allowed", forbidden_resume)
assert client.put("/api/v1/sedimentation", json={"enabled": False}).status_code == 200
monkeypatch.setattr(app.state.sdk_memory_runtime, "resume_allowed", slow_resume)
assert client.put(
"/api/v1/sedimentation", json={"enabled": False}
).status_code == 200
resumed = client.put("/api/v1/sedimentation", json={"enabled": True})
assert resumed.status_code == 200
assert resumed.json()["enabled"] is True
assert called is False
assert finished.is_set() is False
released.set()
def test_switch_does_not_rewrite_permission_parked_jobs(tmp_path) -> None:
def test_a_failing_sdk_queue_resume_does_not_fail_the_switch(tmp_path, monkeypatch) -> None:
app = _app(tmp_path)
with TestClient(app, headers={"Authorization": "Bearer test"}) as client:
queue = app.state.memory_write_queue
queue.evidence = None
request = WriteRequest(
internal_session_id="session-1",
source_id="cursor",
batch_identity=BatchIdentity(
internal_session_id="session-1",
first_round=1,
last_round=1,
),
evidence_ids=["e-1"],
idempotency_key="permission-parked",
)
record = queue.submit(request)
queue.settle(
record.job_id,
state=WAITING_FOR_PERMISSION,
error_code="model_content_permission_required",
hy_invoked=False,
)
async def broken_resume() -> None:
raise RuntimeError("sdk_queue_unavailable")
assert client.put("/api/v1/sedimentation", json={"enabled": False}).status_code == 200
assert client.put("/api/v1/sedimentation", json={"enabled": True}).status_code == 200
monkeypatch.setattr(app.state.sdk_memory_runtime, "resume_allowed", broken_resume)
assert client.put(
"/api/v1/sedimentation", json={"enabled": False}
).status_code == 200
resumed = client.put("/api/v1/sedimentation", json={"enabled": True})
parked = queue.get("permission-parked")
assert parked.state == WAITING_FOR_PERMISSION
assert parked.detail == "model_content_permission_required"
assert resumed.status_code == 200
assert resumed.json()["enabled"] is True
@@ -23,8 +23,9 @@ def test_workbuddy_recall_remains_available_during_100k_import(tmp_path, monkeyp
assert result["import_status"] == "completed"
assert result["stored"] == 100_000
assert result["evidence_rows"] == 100_000
assert result["evidence_rows"] == 100_001
assert result["hook_accepted"] is True
assert result["hook_stored"] is True
assert result["hook_visible_ms"] < 10_000
assert result["recall_sampling_terminal"] is True
assert result["during_unavailable"] == 0
@@ -16,7 +16,7 @@ from one_memory.imports.connections import SourceConnectionStore
from one_memory.imports.grants import SourceGrantStore
from one_memory.imports.jobs import ImportJobStore
from one_memory.imports.sources.registry import build_source_catalog
from one_memory.onboarding import OnboardingProgressUpdate, OnboardingStore
from one_memory.onboarding import OnboardingStore
@pytest.mark.parametrize("entry", ["connect", "authorize"])
@@ -197,82 +197,6 @@ def test_connection_retry_reuses_completed_job_and_checks_root_snapshot(tmp_path
asyncio.run(scenario())
def test_connection_returns_committed_receipt_before_live_activation_finishes(
tmp_path, monkeypatch
):
monkeypatch.setattr("one_memory.model_presets._preset_data", lambda: None)
async def scenario():
home = tmp_path / "home"
root = home / ".claude/projects"
root.mkdir(parents=True)
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=home,
api_token=SecretStr("test"),
automatic_memory_processing=False,
live_sync_enabled=False,
),
source_catalog=build_source_catalog(home=home, environment={}),
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer test"},
) as client,
):
found = await client.post("/api/v1/sources/claude-code/location")
started = asyncio.Event()
release = asyncio.Event()
original = app.state.live_sync_supervisor.initialize_source
async def blocked_initialize(source_id, *, baseline_missing_files):
started.set()
await release.wait()
return await original(
source_id,
baseline_missing_files=baseline_missing_files,
)
monkeypatch.setattr(
app.state.live_sync_supervisor,
"initialize_source",
blocked_initialize,
)
try:
response = await asyncio.wait_for(
client.post(
"/api/v1/sources/connect",
json={
"source_ids": ["claude-code"],
"request_id": "fast-receipt",
"source_roots": {
"claude-code": found.json()["root_id"],
},
"onboarding": True,
"full_rescan": True,
"enable_sync": True,
},
),
timeout=1,
)
assert response.status_code == 200
assert response.json()["request_id"] == "fast-receipt"
assert app.state.config_store.sync_enabled("claude-code")
await asyncio.wait_for(started.wait(), timeout=1)
assert not release.is_set()
finally:
release.set()
tasks = tuple(app.state.source_activation_tasks.values())
if tasks:
await asyncio.gather(*tasks)
asyncio.run(scenario())
@pytest.mark.parametrize(
("turn_count", "age_days", "limited"),
[
@@ -447,14 +371,7 @@ def test_onboarding_persists_one_history_window_on_every_accepted_source(tmp_pat
database = tmp_path / "data.sqlite"
grants = SourceGrantStore(database, home=home)
jobs = ImportJobStore(database)
onboarding = OnboardingStore(database)
onboarding.update_progress(
OnboardingProgressUpdate(
mode_intent="pro",
setup_step="model",
model_profile_id="9be6d465-4cf2-4c72-93f7-15f3c012bc80",
)
)
OnboardingStore(database)
store = SourceConnectionStore(grants, jobs, home=home)
catalog = build_source_catalog(home=home, environment={})
cutoff = datetime(2026, 10, 13, 5, 0, tzinfo=UTC)
@@ -474,7 +391,3 @@ def test_onboarding_persists_one_history_window_on_every_accepted_source(tmp_pat
assert receipt.accepted_source_ids == ("claude-code", "codex")
assert {job.history_after for job in receipt.jobs} == {cutoff}
assert store.get("bounded-first-import") == receipt
state = onboarding.get()
assert state.mode_intent == "pro"
assert state.setup_step == "model"
assert state.model_profile_id == "9be6d465-4cf2-4c72-93f7-15f3c012bc80"
@@ -10,7 +10,6 @@ from pydantic import SecretStr
from one_memory import api
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.sources.registry import RootCandidateKind
from one_memory.sdk_data_plane_migration import MemoryAuthority
from one_memory.sdk_memory_qa import SdkAskMemory, SdkMemoryAnswer
from one_memory.secrets import InMemorySecretStore
@@ -97,11 +96,8 @@ class Sdk:
class Handle:
def __init__(self, sdk=None):
self.sdk = sdk or Sdk()
async def client(self):
return self.sdk
return Sdk()
async def close(self):
return None
@@ -199,50 +195,3 @@ def test_v2_host_calls_are_recorded_and_renderer_calls_are_not(tmp_path, monkeyp
assert listed.json()["total"] == 3
asyncio.run(scenario())
@pytest.mark.fresh_sdk
def test_v2_raw_rechecks_a_source_revoked_during_search(tmp_path):
app = application(tmp_path)
class RevokingRawSdk:
def search(self, query, **kwargs):
app.state.source_grant_store.revoke(user_id="local", source_id="cursor")
return {
"memories": {
"schema": [],
"proactive": [],
"normal": [
{
"memory_id": "raw-1",
"content": "must not escape",
"user_id": "local",
"agent_id": "all-apps",
"session_id": "session-1",
"source": "cursor",
"score": 0.9,
}
],
}
}
async def scenario():
async with app.router.lifespan_context(app), client_for(app, EXTERNAL) as host:
root = tmp_path / "cursor"
root.mkdir()
app.state.source_grant_store.grant(
user_id="local",
source_id="cursor",
root=root,
candidate_kind=RootCandidateKind.DEFAULT,
)
app.state.sdk_client_handle = Handle(RevokingRawSdk())
return await host.post(
"/api/v2/context/search",
json={"query": "private", "kinds": ["raw"]},
)
response = asyncio.run(scenario())
assert response.status_code == 403
assert "must not escape" not in response.text
+4 -4
View File
@@ -991,7 +991,7 @@ wheels = [
[[package]]
name = "hy-memory"
version = "2.0.14"
version = "2.0.13"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "chromadb" },
@@ -1008,9 +1008,9 @@ dependencies = [
{ name = "scikit-learn" },
{ name = "tiktoken" },
]
sdist = { url = "https://files.pythonhosted.org/packages/4a/68/932ac0f684a971741643d8dab0a209c6a09d5a132ca607bbb9355e0b8cf2/hy_memory-2.0.14.tar.gz", hash = "sha256:94cf354f51ab8b506c0ac4909a95a6b13fd542f81c894322210beb20996f16fd", size = 1253180, upload-time = "2026-11-04T09:52:17.127Z" }
sdist = { url = "https://files.pythonhosted.org/packages/88/46/7d5a2d76be135664302d91e15f0d5018cf8e228956e4c80b86583b131cb3/hy_memory-2.0.13.tar.gz", hash = "sha256:f95850d492de74e91891502716da96e8b5297c3fc045dc70ec3474ed240da958", size = 1252802, upload-time = "2026-11-02T09:30:40.331Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/76/7f/9ecbaa2589dae9bd83657adc6388b0c7b0be415c5584fef045db309911eb/hy_memory-2.0.14-py3-none-any.whl", hash = "sha256:6df76d8dc84aaa352149e85055331d22ef904e4acd4540a2328bf058a3f0f2a6", size = 1322364, upload-time = "2026-11-04T09:52:14.096Z" },
{ url = "https://files.pythonhosted.org/packages/99/63/2b4d21d555a5dd50ab838fad944c7fdd6cf90d2dc814e2a8e79eca3d4e03/hy_memory-2.0.13-py3-none-any.whl", hash = "sha256:128cf4729e6ea10b9da3c7fdf21005191294621a65378cd9583827904eea064a", size = 1322154, upload-time = "2026-11-02T09:30:37.746Z" },
]
[[package]]
@@ -1837,7 +1837,7 @@ requires-dist = [
{ name = "fastapi", specifier = ">=0.141.1" },
{ name = "filelock", specifier = ">=3.32.4" },
{ name = "httpx", specifier = ">=0.28.1" },
{ name = "hy-memory", specifier = "==2.0.14" },
{ name = "hy-memory", specifier = "==2.0.13" },
{ name = "keyring", specifier = ">=25.7.0" },
{ name = "numpy", specifier = "==2.5.2" },
{ name = "onnxruntime", specifier = "==1.23.2" },
@@ -366,7 +366,7 @@ it("refuses a ui trace from an untrusted window", async () => {
).rejects.toThrow(/untrusted_window/);
});
it("marks a problem and exports the complete local log bundle", async () => {
it("marks a problem and exports a redacted local bundle", async () => {
const value = await fixture();
const mark = await value.handlers.get(
"one-memory:mark-diagnostic-incident",
@@ -375,14 +375,6 @@ it("marks a problem and exports the complete local log bundle", async () => {
incident_id: expect.any(String),
session_id: expect.any(String),
});
await writeFile(
path.join(value.dataDir, "logs", "core.log"),
"2026-10-14 10:31:00 WARNING database wait\n",
);
await writeFile(
path.join(value.dataDir, "logs", "service-error.log"),
"worker failed\n",
);
const exported = await value.handlers.get("one-memory:export-diagnostics")?.(
{},
);
@@ -406,16 +398,6 @@ it("marks a problem and exports the complete local log bundle", async () => {
),
);
expect(summary).toMatchObject({ build: "3800" });
const entries = execFileSync(
"/usr/bin/unzip",
["-Z1", path.join(value.dataDir, zip)],
{ encoding: "utf8" },
)
.trim()
.split("\n");
expect(entries).toContain("logs/core.log");
expect(entries).toContain("logs/service-error.log");
expect(entries).toContain("logs/diagnostic-desktop.jsonl");
value.dispose();
});
+1 -3
View File
@@ -196,7 +196,6 @@ export function resolveDiagnosticBuildIdentity(options: {
export async function exportDesktopDiagnostics(
options: DiagnosticExportOptions,
exportOptions: { includeRawLogs?: boolean } = {},
) {
const build = options.buildNumber ?? options.productVersion;
const target = path.join(
@@ -210,7 +209,6 @@ export async function exportDesktopDiagnostics(
dataDir: options.dataDir,
target,
productVersion: build,
includeRawLogs: exportOptions.includeRawLogs === true,
...(options.commit === undefined ? {} : { commit: options.commit }),
...(options.coreSha256 === undefined
? {}
@@ -646,7 +644,7 @@ export function installDiagnosticsIpc(
options.ipc.handle(EXPORT, async (event, ...args) => {
if (args.length) throw new Error("diagnostic_arguments_rejected");
options.requireTrustedWindow(event);
return exportDesktopDiagnostics(options, { includeRawLogs: true });
return exportDesktopDiagnostics(options);
});
return () => {
+1 -34
View File
@@ -9,11 +9,6 @@ import {
import userEvent from "@testing-library/user-event";
import { afterEach, beforeEach, expect, it, vi } from "vitest";
import { App } from "./App";
import {
commitConfirmedMemoryMode,
issueConfirmedModeTicket,
resetConfirmedMemoryModeForTests,
} from "./confirmed-memory-mode";
import { LANGUAGE_STORAGE_KEY, LanguageProvider } from "./gui-language";
it("localizes shared navigation and footer in Spanish", async () => {
@@ -41,7 +36,6 @@ it("localizes shared navigation and footer in Spanish", async () => {
});
const memoryProps = vi.hoisted(() => vi.fn());
const progressProps = vi.hoisted(() => vi.fn());
vi.mock("./api", () => ({
getOnboarding: vi.fn().mockResolvedValue({ status: "completed" }),
@@ -68,21 +62,11 @@ vi.mock("./MemorySpace", () => ({
vi.mock("./ModelSettings", () => ({
ModelSettings: () => <p>模型设置内容</p>,
}));
vi.mock("./R6Settings", () => ({
R6Settings: () => <p>设置视图内容</p>,
}));
vi.mock("./ImportProgressStrip", () => ({
ImportProgressStrip: (props: unknown) => {
progressProps(props);
return null;
},
}));
vi.mock("./ImportProgressStrip", () => ({ ImportProgressStrip: () => null }));
vi.mock("./HealthPanel", () => ({ HealthPanel: () => null }));
vi.mock("./OnboardingFlow", () => ({ OnboardingFlow: () => null }));
beforeEach(() => {
resetConfirmedMemoryModeForTests();
progressProps.mockClear();
window.history.replaceState({}, "", "/");
localStorage.clear();
});
@@ -121,23 +105,6 @@ it("uses explicit memory deep links and restores the applications default URL",
).toBe("page");
});
it("passes confirmed mode capabilities to the production progress strip", async () => {
commitConfirmedMemoryMode("lite", issueConfirmedModeTicket("settings"));
render(<App />);
const nav = await screen.findByRole("navigation", { name: "主要导航" });
await userEvent.click(within(nav).getByRole("button", { name: "设置" }));
await waitFor(() =>
expect(progressProps).toHaveBeenCalledWith(
expect.objectContaining({
modeCapabilities: expect.objectContaining({
content_plane: "raw",
generation_required: false,
}),
}),
),
);
});
it("keeps an independent scroll position for each primary page", async () => {
render(<App />);
const nav = await screen.findByRole("navigation", { name: "主要导航" });
+2 -5
View File
@@ -153,9 +153,7 @@ it("enters D47 settings without a tour or mandatory second scope-confirmation fl
).toBeNull();
noModelWrites();
expect(screen.queryByRole("button", { name: "管理已有授权" })).toBeNull();
expect(screen.getByRole("button", { name: "界面语言" })).toBeTruthy();
expect(screen.getByRole("button", { name: "记忆模式" })).toBeTruthy();
expect(screen.getByRole("button", { name: "模型配置" })).toBeTruthy();
expect(screen.getByRole("button", { name: "记忆模式与模型" })).toBeTruthy();
expect(screen.getByRole("button", { name: "连接其他应用" })).toBeTruthy();
expect(screen.getByRole("button", { name: "帮助与诊断" })).toBeTruthy();
noModelWrites();
@@ -218,14 +216,13 @@ it("keeps an open model draft while switching settings groups", async () => {
window.history.replaceState({}, "", "/?view=settings");
render(<App />);
await waitFor(() => expect(modelProps).toHaveBeenCalled());
await userEvent.click(screen.getByRole("button", { name: "模型配置" }));
await userEvent.click(screen.getByRole("button", { name: "配置模型" }));
await userEvent.type(
screen.getByLabelText("保留的模型草稿"),
"unsaved draft",
);
await userEvent.click(screen.getByRole("button", { name: "连接其他应用" }));
await userEvent.click(screen.getByRole("button", { name: "模型配置" }));
await userEvent.click(screen.getByRole("button", { name: "记忆模式与模型" }));
expect(window.location.search).toBe("?view=settings");
expect(screen.getByLabelText("保留的模型草稿")).toHaveProperty(
"value",
+4 -12
View File
@@ -23,7 +23,6 @@ import {
import { getOnboarding } from "./api";
import { useApplicationsLanguage } from "./applications-language";
import { CallHistory } from "./CallHistory";
import { useConfirmedMemoryMode } from "./confirmed-memory-mode";
import { DebugLogPanel } from "./DebugLogPanel";
import { DesktopFrame } from "./DesktopFrame";
import { EmbeddingSwitchGlobalStatus } from "./EmbeddingSwitchExperience";
@@ -36,7 +35,6 @@ import { OnboardingFlow } from "./OnboardingFlow";
import { R6Button, R6LocalStorage } from "./R6Controls";
import { R6Settings } from "./R6Settings";
import { prefetchDataRuntime } from "./runtime-state";
import { EmbeddingSwitchProvider } from "./use-embedding-switch";
import "./styles.css";
import "./core-visual.css";
import "./r6-shell.css";
@@ -50,19 +48,16 @@ const rendererStartedAt = Date.now();
export function App() {
return (
<EmbeddingSwitchProvider>
<DesktopFrame>
<AppContent />
<DebugLogPanel />
</DesktopFrame>
</EmbeddingSwitchProvider>
<DesktopFrame>
<AppContent />
<DebugLogPanel />
</DesktopFrame>
);
}
function AppContent() {
const { t } = useGuiLanguage();
const { a } = useApplicationsLanguage();
const confirmedMode = useConfirmedMemoryMode();
const [onboarding, setOnboarding] = useState<OnboardingState | null>(null);
const [startupRead, setStartupRead] = useState<"loading" | "ready" | "error">(
"loading",
@@ -332,9 +327,6 @@ function AppContent() {
sourceSnapshot={imports.sourceSnapshot}
onRefresh={imports.refresh}
arrival={memoryArrival}
modeCapabilities={
confirmedMode.capabilities ?? memoryArrival?.capabilities
}
onConfigureModel={() => navigate("settings")}
onJobAction={imports.changeJob}
showSourceLink={displayedView !== "settings"}
+23 -369
View File
@@ -15,11 +15,6 @@ import {
CALL_HISTORY_ENTRY_VISIBLE,
} from "./ApplicationsHome";
import * as api from "./api";
import {
commitConfirmedMemoryMode,
issueConfirmedModeTicket,
resetConfirmedMemoryModeForTests,
} from "./confirmed-memory-mode";
import { saveConnectionPreferences } from "./connection-preferences";
import { resetConnectionReconcile } from "./connection-reconcile";
import { LanguageProvider } from "./gui-language";
@@ -72,48 +67,23 @@ it("localizes the whole home live without repeating reads or changing imported t
expect(summary).toHaveBeenCalledTimes(calls);
});
const {
summary,
projection,
processing,
sedimentation,
history,
liveSync,
evolve,
} = vi.hoisted(() => ({
summary: vi.fn(),
projection: vi.fn(),
processing: vi.fn(),
sedimentation: vi.fn(),
history: vi.fn(),
liveSync: vi.fn(),
evolve: vi.fn(),
}));
const { summary, processing, sedimentation, history, liveSync } = vi.hoisted(
() => ({
summary: vi.fn(),
processing: vi.fn(),
sedimentation: vi.fn(),
history: vi.fn(),
liveSync: vi.fn(),
}),
);
vi.mock("./call-history-client", async (original) => ({
...(await original<typeof import("./call-history-client")>()),
getCallHistory: history,
}));
vi.mock("./api", () => ({
getHomeProjection: projection,
getHomeSummary: summary,
homeProjectionUnsupported: vi.fn().mockReturnValue(true),
getMemoryMode: vi.fn().mockResolvedValue({
mode: "pro",
configured_mode: "pro",
explicitly_configured: true,
forced_by_env: false,
modes: ["lite", "pro", "ultra"],
graph_build: {
state: "idle",
started_at: null,
finished_at: null,
users: {},
},
}),
getSdkRaw: vi.fn(),
getLiveSyncHealth: liveSync,
getMemoryProcessing: processing,
getEvolveOverview: evolve,
getSedimentation: sedimentation,
getSources: vi.fn().mockResolvedValue([]),
}));
@@ -136,31 +106,9 @@ const imports = {
sourceSnapshot: { stale: false },
} as unknown as Parameters<typeof ApplicationsHome>[0]["imports"];
beforeEach(() => {
resetConfirmedMemoryModeForTests();
vi.mocked(api.getMemoryMode).mockResolvedValue({
mode: "pro",
configured_mode: "pro",
explicitly_configured: true,
forced_by_env: false,
modes: ["lite", "pro", "ultra"],
graph_build: {
state: "idle",
started_at: null,
finished_at: null,
users: {},
},
});
vi.mocked(api.getSdkRaw).mockReset();
vi.mocked(api.homeProjectionUnsupported).mockReturnValue(true);
history
.mockReset()
.mockResolvedValue({ available: true, recording_error: false, items: [] });
evolve.mockReset().mockResolvedValue({
mode: "ultra",
schedule: { enabled: true, time: "03:00" },
users: {},
running: [],
});
imports.refresh = vi.fn();
summary.mockReset().mockResolvedValue({
memory_count: 601,
@@ -174,7 +122,6 @@ beforeEach(() => {
},
observed_at: "2026-09-19T00:00:00Z",
});
projection.mockReset().mockRejectedValue(new Error("legacy core"));
processing.mockReset().mockResolvedValue({ running: 0, pending: 0 });
sedimentation.mockReset().mockResolvedValue({
enabled: true,
@@ -202,218 +149,6 @@ beforeEach(() => {
]),
} as unknown as NonNullable<typeof window.oneMemory>;
});
it("hides an old projection immediately when confirmed mode changes", async () => {
vi.mocked(api.homeProjectionUnsupported).mockReturnValue(false);
commitConfirmedMemoryMode("pro", issueConfirmedModeTicket("settings"));
projection
.mockResolvedValueOnce({
projection_version: 1,
user_id: "local",
effective_mode: "pro",
capabilities: {
content_plane: "memory",
generation_required: true,
graph_enabled: false,
evolve_available: false,
},
visible_count: 1,
display_state: "excerpt",
latest: {
type: "memory",
memory_id: "old",
scene: "chat",
kind: "chat",
variant: null,
source_id: "app-0",
excerpt: "旧模式投影",
truncated: false,
time: null,
time_basis: "update",
},
observed_at: "2026-11-04T00:00:00Z",
})
.mockImplementationOnce(() => new Promise(() => {}));
render(
<ApplicationsHome
imports={imports}
onManage={() => {}}
onMemories={() => {}}
/>,
);
expect(await screen.findByText("旧模式投影")).toBeTruthy();
act(() =>
commitConfirmedMemoryMode("lite", issueConfirmedModeTicket("settings")),
);
expect(screen.queryByText("旧模式投影")).toBeNull();
expect(screen.getAllByText("正在核对本机原始对话…")).toHaveLength(2);
});
it("renders the lite projection as raw data with mode metadata and no navigation tail", async () => {
vi.mocked(api.homeProjectionUnsupported).mockReturnValue(false);
projection.mockResolvedValue({
projection_version: 1,
user_id: "local",
effective_mode: "lite",
capabilities: {
content_plane: "raw",
generation_required: false,
graph_enabled: false,
evolve_available: false,
},
visible_count: 7,
display_state: "excerpt",
latest: {
type: "raw",
raw_id: "raw-1",
session_id: "session-1",
source_id: "app-0",
excerpt: "用户原始对话摘要",
truncated: false,
display_state: "excerpt",
time: "2026-09-19T00:00:00Z",
time_basis: "conversation",
},
observed_at: "2026-09-19T00:00:01Z",
});
render(
<ApplicationsHome
imports={imports}
onManage={() => {}}
onMemories={vi.fn()}
onCalls={() => {}}
/>,
);
expect(await screen.findByText("原始对话")).toBeTruthy();
expect(screen.getByText("7")).toBeTruthy();
expect(screen.getByText("用户原始对话摘要")).toBeTruthy();
expect(screen.getByText(/原始对话 · App 0 · 对话时间/)).toBeTruthy();
expect(screen.getByText("lite")).toBeTruthy();
expect(screen.queryByRole("button", { name: "查看记忆" })).toBeNull();
expect(screen.queryByRole("button", { name: "查看记录" })).toBeNull();
});
it.each([
["conversation", "对话时间"],
["import", "导入时间"],
["write", "写入时间"],
["update", "更新时间"],
[null, null],
] as const)(
"renders projection time basis %s truthfully",
async (basis, label) => {
vi.mocked(api.homeProjectionUnsupported).mockReturnValue(false);
projection.mockResolvedValue({
projection_version: 1,
user_id: "local",
effective_mode: "lite",
capabilities: {
content_plane: "raw",
generation_required: false,
graph_enabled: false,
evolve_available: false,
},
visible_count: 1,
display_state: "excerpt",
latest: {
type: "raw",
raw_id: "raw-basis",
session_id: "session-basis",
source_id: "app-0",
excerpt: "basis excerpt",
truncated: false,
display_state: "excerpt",
time: "2026-09-19T00:00:00Z",
time_basis: basis,
},
observed_at: "2026-09-19T00:00:01Z",
});
render(mountHome());
const activity = (await screen.findByText("basis excerpt")).closest(
"section",
);
const meta = activity?.querySelector("small")?.textContent ?? "";
if (label) expect(meta).toContain(label);
else expect(meta).not.toMatch(/对话时间|导入时间|写入时间|更新时间/);
},
);
it("does not publish a stopped legacy raw fallback", async () => {
vi.mocked(api.getMemoryMode).mockResolvedValue({
mode: "lite",
configured_mode: "lite",
explicitly_configured: true,
forced_by_env: false,
modes: ["lite", "pro", "ultra"],
graph_build: {
state: "idle",
started_at: null,
finished_at: null,
users: {},
},
});
let resolveRaw!: (value: {
user_id: string;
items: [];
total: number;
limit: number;
offset: number;
}) => void;
vi.mocked(api.getSdkRaw).mockReturnValue(
new Promise((resolve) => {
resolveRaw = resolve;
}),
);
const view = render(mountHome(true));
await waitFor(() => expect(api.getSdkRaw).toHaveBeenCalledWith({ limit: 1 }));
view.rerender(mountHome(false));
await act(async () =>
resolveRaw({
user_id: "local",
items: [],
total: 73,
limit: 1,
offset: 0,
}),
);
expect(screen.queryByText("73")).toBeNull();
});
it("uses only total from the legacy raw fallback", async () => {
vi.mocked(api.getMemoryMode).mockResolvedValue({
mode: "lite",
configured_mode: "lite",
explicitly_configured: true,
forced_by_env: false,
modes: ["lite", "pro", "ultra"],
graph_build: {
state: "idle",
started_at: null,
finished_at: null,
users: {},
},
});
vi.mocked(api.getSdkRaw).mockResolvedValue({
user_id: "local",
items: [
{
raw_id: "raw-private",
content: "正文不应进入首页",
user_id: "local",
agent_id: "agent",
session_id: "session",
source: "app-0",
memory_at: null,
gmt_created: null,
custom: {},
},
],
total: 9,
limit: 1,
offset: 0,
});
render(mountHome());
expect(await screen.findByText("9")).toBeTruthy();
expect(screen.queryByText("正文不应进入首页")).toBeNull();
});
it.each([
[true, "Organizando", "Organizando nuevas conversaciones"],
[false, "En pausa", "Se dejó de generar memorias nuevas"],
@@ -462,7 +197,7 @@ it("localizes startup recovery and empty states without leaking errors", async (
"El servicio se está iniciando; se actualizará automáticamente",
),
).toBeTruthy();
expect(screen.getByText("—")).toBeTruthy();
expect(screen.getByText("0")).toBeTruthy();
expect(screen.getByText("Aún no hay aplicaciones para mostrar")).toBeTruthy();
expect(
screen.queryByRole("button", { name: "Reintentar lectura" }),
@@ -500,7 +235,7 @@ it("shows latest recorded call separately without exposing recent-update detail
);
await screen.findByText("实际治理记忆");
fireEvent.click(screen.getByRole("tab", { name: "最近调用" }));
await screen.findByText("Codex 找到 2 条记忆 · 1 条原始会话");
await screen.findByText("Codex 查到 3 条相关内容");
expect(document.querySelector(".r6-home-call-direction")).toBeTruthy();
expect(screen.queryByText("实际治理记忆")).toBeNull();
expect(screen.getByText(/Codex · 今天 .+ · 查询成功/)).toBeTruthy();
@@ -557,7 +292,7 @@ it("skips direct diagnostics and shows the latest attributed application call",
);
fireEvent.click(await screen.findByRole("tab", { name: "最近调用" }));
expect(await screen.findByText("WorkBuddy 找到 5 条记忆")).toBeTruthy();
expect(await screen.findByText("WorkBuddy 查到 5 条相关内容")).toBeTruthy();
expect(screen.queryByText(/直接调用/)).toBeNull();
expect(screen.getByText(/WorkBuddy · 今天 .+ · 查询成功/)).toBeTruthy();
const direction = document.querySelector(".r6-home-call-direction");
@@ -595,7 +330,7 @@ it("shows only the host for memory hits whose references have no source", async
/>,
);
fireEvent.click(await screen.findByRole("tab", { name: "最近调用" }));
await screen.findByText("WorkBuddy 找到 2 条记忆");
await screen.findByText("WorkBuddy 查到 2 条相关内容");
const direction = document.querySelector(".r6-home-call-direction");
expect(direction?.querySelectorAll("img")).toHaveLength(1);
expect(direction?.querySelector("img")?.getAttribute("src")).toContain(
@@ -633,7 +368,7 @@ it("keeps source to host direction for raw hits with a source", async () => {
/>,
);
fireEvent.click(await screen.findByRole("tab", { name: "最近调用" }));
await screen.findByText("WorkBuddy 找到 1 条原始会话");
await screen.findByText("WorkBuddy 查到 1 条相关内容");
const direction = document.querySelector(".r6-home-call-direction");
expect(direction?.querySelectorAll("img")).toHaveLength(1);
expect(direction?.querySelector("svg")).toBeTruthy();
@@ -681,9 +416,7 @@ it("shows unlinked apps and the latest returned call on matching rows", async ()
onCalls={() => {}}
/>,
);
expect(
await screen.findByText(/Cursor 找到 2 条记忆 · 1 条原始会话 ·/),
).toBeTruthy();
expect(await screen.findByText(/Cursor 查到 3 条相关内容 ·/)).toBeTruthy();
expect(screen.getByText("尚未关联")).toBeTruthy();
});
it("keeps the latest call visible when recording is incomplete", async () => {
@@ -711,7 +444,7 @@ it("keeps the latest call visible when recording is incomplete", async () => {
/>,
);
fireEvent.click(await screen.findByRole("tab", { name: "最近调用" }));
await screen.findByText("Cursor 找到 1 条记忆");
await screen.findByText("Cursor 查到 1 条相关内容");
expect(screen.getByText(/有几次调用没被记下来/)).toBeTruthy();
expect(screen.queryByText("暂时无法读取调用记录")).toBeNull();
expect(screen.queryByText("暂无已记录调用")).toBeNull();
@@ -763,7 +496,7 @@ it("marks a retained application activity as last-read when call history cannot
/>,
);
await screen.findByText(/App 0 找到 2 条记忆 · 1 条原始会话 ·/);
await screen.findByText(/App 0 查到 3 条相关内容 ·/);
view.rerender(
<ApplicationsHome
active={false}
@@ -787,10 +520,8 @@ it("marks a retained application activity as last-read when call history cannot
await waitFor(() => expect(history).toHaveBeenCalledTimes(2));
expect(await screen.findByText("实际治理记忆")).toBeTruthy();
expect(
screen.getByText(/上次读取:App 0 找到 2 条记忆 · 1 条原始会话/),
).toBeTruthy();
expect(screen.queryByText(/^App 0 找到/)).toBeNull();
expect(screen.getByText(/上次读取:App 0 查到 3 条相关内容/)).toBeTruthy();
expect(screen.queryByText(/^App 0 查到/)).toBeNull();
});
it("does not say an unavailable call returned memories", async () => {
history.mockResolvedValue({
@@ -955,39 +686,6 @@ it.each([
},
);
it("keeps the home snapshot visible during Evolve and refreshes when it ends", async () => {
evolve.mockResolvedValue({
mode: "ultra",
schedule: { enabled: true, time: "03:00" },
users: {},
running: [
{
run_id: "run-1",
user_id: "local",
state: "running",
},
],
});
render(mountHome());
expect(await screen.findByText("601")).toBeTruthy();
expect(await screen.findByText("正在整合记忆…")).toBeTruthy();
const readsBeforeCompletion = projection.mock.calls.length;
act(() => {
queryClient.setQueryData(["evolve", "local"], {
mode: "ultra",
schedule: { enabled: true, time: "03:00" },
users: {},
running: [],
});
});
await waitFor(() =>
expect(projection.mock.calls.length).toBeGreaterThan(readsBeforeCompletion),
);
});
afterEach(() => {
cleanup();
resetClient();
@@ -1072,7 +770,7 @@ it("keeps authoritative home and processing data when only the source snapshot b
expect(screen.getAllByText("整理中")).toHaveLength(2);
expect(screen.queryByText("状态待核对")).toBeNull();
});
it("keeps an unknown dash while the first home read recovers", async () => {
it("keeps a neutral zero while the first home summary read recovers", async () => {
summary.mockRejectedValue(new Error("private offline detail"));
render(
<ApplicationsHome
@@ -1085,7 +783,7 @@ it("keeps an unknown dash while the first home read recovers", async () => {
await waitFor(() =>
expect(screen.getByText("后台正在启动,稍后自动更新")).toBeTruthy(),
);
expect(screen.getByText("—")).toBeTruthy();
expect(screen.getByText("0")).toBeTruthy();
expect(screen.getByText("正在准备记忆概览")).toBeTruthy();
expect(screen.queryByText(/暂时?无法读取|读取失败/)).toBeNull();
expect(document.body.textContent).not.toContain("private offline detail");
@@ -1246,50 +944,6 @@ it("shows a paused switch immediately even when task rows still say running", as
expect(screen.queryByText("新的对话正在整理")).toBeNull();
});
it("shows lite raw ingestion as running even when model sedimentation is paused", async () => {
vi.mocked(api.homeProjectionUnsupported).mockReturnValue(false);
projection.mockResolvedValue({
projection_version: 1,
user_id: "local",
effective_mode: "lite",
capabilities: {
content_plane: "raw",
generation_required: false,
graph_enabled: false,
evolve_available: false,
},
visible_count: 0,
display_state: "empty",
latest: null,
observed_at: "2026-11-04T00:00:00Z",
});
processing.mockResolvedValue({
running: 0,
pending: 10,
pipeline: { mode: "lite" },
});
sedimentation.mockResolvedValue({
enabled: false,
pending_retained: true,
imports_unaffected: true,
});
render(
<ApplicationsHome
imports={imports}
onManage={() => {}}
onMemories={() => {}}
onSettings={() => {}}
/>,
);
expect(await screen.findAllByText("正在保存")).toHaveLength(2);
expect(screen.getByText("暂无更新")).toBeTruthy();
expect(screen.getByText("已核对,暂无可显示的原始对话更新")).toBeTruthy();
expect(screen.getByText("原始对话持续保存,不需要生成模型")).toBeTruthy();
expect(screen.queryByText("已暂停")).toBeNull();
});
it("keeps the import-in-progress empty state stable without a progress link", async () => {
summary.mockResolvedValue({
memory_count: 0,
@@ -1658,7 +1312,7 @@ it("keeps the conversation-sync checking copy before the first home read", () =>
expect(screen.queryByText(/个应用已开启/)).toBeNull();
});
it("shows an unknown dash while the first projection read is pending", async () => {
it("shows a provisional zero while the first summary read is pending", async () => {
summary.mockImplementation(() => new Promise(() => {}));
history.mockResolvedValue({
available: true,
@@ -1675,7 +1329,7 @@ it("shows an unknown dash while the first projection read is pending", async ()
/>,
);
await screen.findByText("正在准备记忆概览");
expect(screen.getByText("—")).toBeTruthy();
expect(screen.getByText("0")).toBeTruthy();
expect(screen.getByText("正在核对本机记忆…")).toBeTruthy();
expect(screen.queryByText("还没有已生成的记忆")).toBeNull();
});
+113 -265
View File
@@ -1,5 +1,4 @@
import type {
HomeProjectionV1,
HostHomeSummary,
LiveSyncHealth,
MemoryProcessingSummary,
@@ -9,15 +8,7 @@ import { type GuiLocale, isMessageKey } from "@one-memory/i18n";
import { ArrowLeft, ArrowRight } from "lucide-react";
import { useEffect, useId, useLayoutEffect, useRef, useState } from "react";
import { flushSync } from "react-dom";
import {
getHomeProjection,
getHomeSummary,
getLiveSyncHealth,
getMemoryMode,
getMemoryProcessing,
getSdkRaw,
homeProjectionUnsupported,
} from "./api";
import { getHomeSummary, getLiveSyncHealth, getMemoryProcessing } from "./api";
import { AppMark } from "./app-mark";
import {
type CallEvent,
@@ -26,13 +17,6 @@ import {
callRowDetail,
getCallHistory,
} from "./call-history-client";
import {
commitConfirmedMemoryMode,
confirmedMemoryModeSnapshot,
isConfirmedModeTicketCurrent,
issueConfirmedModeTicket,
useConfirmedMemoryMode,
} from "./confirmed-memory-mode";
import { hostMatchesSource, sameConnectionHost } from "./connection-hosts";
import { readConnectionPreferences } from "./connection-preferences";
import {
@@ -41,13 +25,8 @@ import {
retryAutoReconcile,
useAutoConnect,
} from "./connection-reconcile";
import { localEvolveRunning, useEvolveState } from "./evolve-state";
import { useGuiLanguage } from "./gui-language";
import { HomeCardBoundary } from "./HomeCardBoundary";
import {
buildHomeViewModel,
type HomeProjectionReadState,
} from "./home-view-model";
import {
hostBridgeAvailable,
hostConnectionApproved,
@@ -56,7 +35,6 @@ import {
sourceConnectionIncomplete,
useRecallHosts,
} from "./host-state";
import { useOnboardingTranslation } from "./onboarding-language";
import { subscribePoll } from "./poll-scheduler";
import { R6Button } from "./R6Controls";
import { prefetchDataRuntime } from "./runtime-state";
@@ -107,17 +85,6 @@ function stamped(value: string, locale: GuiLocale, t: Translate, suffix = "") {
return `${parts.today ? t("apps.today") : ""}${parts.clock}${suffix}`;
}
function projectionTimeBasisLabel(
basis: string | null,
t: Translate,
): string | null {
if (basis === "conversation") return t("apps.conversationTime");
if (basis === "import") return t("apps.importTime");
if (basis === "write") return t("apps.writeTime");
if (basis === "update") return t("apps.updateTime");
return null;
}
type ActivityChoice = "call" | "update";
function sourceMatchesHost(sourceId: string, hostId: string | null) {
@@ -200,6 +167,7 @@ function SourceMark({ source }: { source: SourceResponse }) {
export function ApplicationsHome({
imports,
onManage,
onMemories,
onCalls,
active = true,
}: {
@@ -211,10 +179,6 @@ export function ApplicationsHome({
active?: boolean;
}) {
const { locale, t, tn } = useGuiLanguage();
const { text } = useOnboardingTranslation();
const evolveState = useEvolveState(active);
const evolveRunning = localEvolveRunning(evolveState.data);
const evolveUncertain = evolveState.isError && evolveRunning;
const hostQuery = useRecallHosts({ poll: active });
const hostBridge = hostBridgeAvailable();
const hosts = hostQuery.data ?? [];
@@ -228,11 +192,6 @@ export function ApplicationsHome({
const hostStale = hostQuery.isError && hostQuery.data !== undefined;
const auto = useAutoConnect();
const sedimentation = useSedimentation();
const confirmedMode = useConfirmedMemoryMode();
const [projection, setProjection] = useState<HomeProjectionV1 | null>(null);
const [projectionState, setProjectionState] =
useState<HomeProjectionReadState>("loading");
const [legacyCore, setLegacyCore] = useState(false);
const [summary, setSummary] = useState<HostHomeSummary | null>(null);
const [processing, setProcessing] = useState<MemoryProcessingSummary | null>(
null,
@@ -319,13 +278,6 @@ export function ApplicationsHome({
const [expanded, setExpanded] = useState(retained.expanded);
const [phase, setPhase] = useState("idle");
const [revision, setRevision] = useState(0);
const previousEvolveRunning = useRef(evolveRunning);
useEffect(() => {
const previous = previousEvolveRunning.current;
previousEvolveRunning.current = evolveRunning;
if (previous && !evolveRunning && active) setRevision((value) => value + 1);
}, [active, evolveRunning]);
const confirmedModeSeen = useRef(confirmedMode);
const expandButton = useRef<HTMLButtonElement>(null);
const backButton = useRef<HTMLButtonElement>(null);
const root = useRef<HTMLElement>(null);
@@ -586,18 +538,6 @@ export function ApplicationsHome({
.map((s) => `${s.source_id}:${s.authorized}:${s.grant_revision}`)
.join("|");
const requestKey = `${sourceVersion}:${revision}`;
useEffect(() => {
const previous = confirmedModeSeen.current;
if (previous.revision === confirmedMode.revision) return;
confirmedModeSeen.current = confirmedMode;
// The first confirmed mode accompanies the first self-consistent
// projection. There is no old-mode projection to hide yet.
if (previous.mode === null) return;
setProjection(null);
setSummary(null);
setProjectionState("loading");
if (active) setRevision((value) => value + 1);
}, [active, confirmedMode]);
// requestKey retriggers this read when the source snapshot or a manual retry changes.
// biome-ignore lint/correctness/useExhaustiveDependencies: requestKey is the refresh token
useEffect(() => {
@@ -607,9 +547,8 @@ export function ApplicationsHome({
async function refresh() {
if (busy || document.hidden) return;
busy = true;
const homeTicket = issueConfirmedModeTicket("home");
const [home, state, sync] = await Promise.allSettled([
getHomeProjection(),
getHomeSummary(),
getMemoryProcessing(),
getLiveSyncHealth(),
refreshSedimentation(),
@@ -617,99 +556,17 @@ export function ApplicationsHome({
busy = false;
if (stopped) return;
if (home.status === "fulfilled") {
if (
!commitConfirmedMemoryMode(
home.value.effective_mode,
homeTicket,
home.value.capabilities,
)
)
return;
setLegacyCore(false);
setSummaryFailures(0);
setProjection(home.value);
setProjectionState("ready");
} else {
if (homeProjectionUnsupported(home.reason)) {
setLegacyCore(true);
try {
const modeTicket = issueConfirmedModeTicket("home");
const modeResult = await getMemoryMode();
if (stopped) return;
const modeAccepted = commitConfirmedMemoryMode(
modeResult.mode,
modeTicket,
);
const effectiveMode = modeAccepted
? modeResult.mode
: confirmedMemoryModeSnapshot().mode;
if (effectiveMode === "lite") {
const raw = await getSdkRaw({ limit: 1 });
if (stopped || !isConfirmedModeTicketCurrent(modeTicket)) return;
setProjection({
projection_version: 1,
user_id: raw.user_id ?? "local",
effective_mode: "lite",
capabilities: {
content_plane: "raw",
generation_required: false,
graph_enabled: false,
evolve_available: false,
},
visible_count: raw.total,
display_state: raw.total > 0 ? "no_user_excerpt" : "empty",
latest: null,
observed_at: new Date().toISOString(),
});
setSummaryFailures(0);
setProjectionState("ready");
return;
}
const legacy = await getHomeSummary();
if (stopped || !isConfirmedModeTicketCurrent(modeTicket)) return;
if (!("stale" in legacy) || !legacy.stale) {
setSummary(legacy);
setSummaryFailures(0);
setProjectionState("ready");
} else {
setSummaryFailures((count) => {
const next = count + 1;
setProjectionState((current) =>
next < 2
? current
: current === "ready" || current === "stale"
? "stale"
: "unavailable",
);
return next;
});
}
} catch {
setSummaryFailures((count) => {
const next = count + 1;
setProjectionState((current) =>
next < 2
? current
: current === "ready" || current === "stale"
? "stale"
: "unavailable",
);
return next;
});
}
// stale=true 表示这轮读取超时,数字不代表现状:保留上一次的概览,
// 等 10 秒后的下一轮轮询补上。重复 stale 仍要进入可恢复状态,
// 不能永远停在“正在核对”。
if (!("stale" in home.value) || !home.value.stale) {
setSummaryFailures(0);
setSummary(home.value);
} else {
setSummaryFailures((count) => {
const next = count + 1;
setProjectionState((current) =>
next < 2
? current
: current === "ready" || current === "stale"
? "stale"
: "unavailable",
);
return next;
});
setSummaryFailures((count) => count + 1);
}
} else {
setSummaryFailures((count) => count + 1);
}
if (state.status === "fulfilled") setProcessing(state.value);
if (sync.status === "fulfilled") setSyncHealth(sync.value);
@@ -725,31 +582,10 @@ export function ApplicationsHome({
stop();
};
}, [active, requestKey]);
const modeChangedPending =
confirmedModeSeen.current.mode !== null &&
confirmedModeSeen.current.revision !== confirmedMode.revision;
const effectiveProjection = modeChangedPending ? null : projection;
const visible = modeChangedPending ? null : summary;
const historyImporting = imports.jobs.some(
(job) => job.status === "running" || job.status === "pending",
);
const viewModel = buildHomeViewModel({
projection: effectiveProjection,
projectionState: modeChangedPending ? "loading" : projectionState,
confirmedMode: confirmedMode.mode,
legacySummary: legacyCore ? visible : null,
processing,
sedimentation: sedimentation ?? null,
recentCall,
importsRunning: historyImporting,
hostState,
syncEnabledCount: imports.sources.filter(
(source) => source.authorized && source.sync_enabled,
).length,
});
const latest = viewModel.recent;
const summaryUnavailable = viewModel.state === "unavailable";
const summaryStale = viewModel.state === "stale";
const visible = summary;
const latest = visible?.latest;
const summaryUnavailable = visible === null && summaryFailures >= 2;
const summaryStale = visible !== null && summaryFailures >= 2;
const hasCall =
recentCall !== null &&
(callState === "ready" || callState === "incomplete");
@@ -763,6 +599,9 @@ export function ApplicationsHome({
activityView !== null && availableActivities.includes(activityView)
? activityView
: null;
const historyImporting = imports.jobs.some(
(job) => job.status === "running" || job.status === "pending",
);
const recentCallSourceId = (recentCall?.references ?? []).find(
(reference) => reference.source_id,
)?.source_id;
@@ -777,46 +616,72 @@ export function ApplicationsHome({
: null;
const attention = callAttention(callEvents);
const attentionHost = attention ? callHostName(attention.host, locale) : null;
const baseActivity =
viewModel.activity.kind === "update"
? ("memory" as const)
: viewModel.activity.kind;
const activity = summaryUnavailable
? "unavailable"
: pinned === "call"
? "call"
: pinned === "update"
? updateKind
: updateKind
: visible === null
? "loading"
: pinned === "call"
? "call"
: pinned === "update"
? updateKind
: hasCall
? "call"
: callState === "unavailable"
? "unavailable"
: callState === "loading" && recentCall !== null
? "call"
: baseActivity;
: updateKind
? updateKind
: hasCall
? "call"
: callState === "unavailable"
? "unavailable"
: historyImporting
? "importing"
: callState === "loading"
? recentCall !== null
? "call"
: "loading"
: processing?.running
? "first-running"
: processing?.pending || processing?.waiting_for_model
? "first-queued"
: "empty";
const apps = imports.sources.filter(
(s) => s.authorized || s.detected === true || s.location_state === "found",
);
const stateLabel = t(viewModel.processingLabel as never);
const stateDetail = t(viewModel.processingDetail as never);
const latestUpdatedAt = latest?.time ?? null;
const stateLabel =
sedimentation?.enabled === true
? t("apps.running")
: sedimentation?.enabled === false
? t("apps.paused")
: t("apps.checkingStatus");
const stateDetail =
sedimentation?.enabled === false
? t("apps.stoppedNew")
: sedimentation?.enabled === true
? t("apps.organizingNew")
: t("apps.checkingProcessing");
const latestUpdatedAt =
latest === null || latest === undefined
? null
: "revision" in latest
? latest.updated_at
: latest.gmt_created === null
? null
: new Date(latest.gmt_created * 1000).toISOString();
const latestClock = latestUpdatedAt
? clockParts(latestUpdatedAt, locale)
: null;
const latestSourceIds = latest?.sourceId ? [latest.sourceId] : [];
const latestSourceIds =
latest === null || latest === undefined
? []
: "revision" in latest
? latest.source_ids
: latest.source
? [latest.source]
: [];
const latestSourceNames = latestSourceIds.map(
(id) =>
imports.sources.find((source) => source.source_id === id)?.display_name ??
id,
);
const time = latestClock?.clock ?? (latest ? t("apps.unknownTime") : null);
const checked = effectiveProjection
? clockParts(effectiveProjection.observed_at, locale)
: visible
? clockParts(visible.observed_at, locale)
: null;
const checked = visible ? clockParts(visible.observed_at, locale) : null;
// A failed read leaves the last health in place on purpose: clearing it would
// drop the instance it names, and that instance is what keeps memory writes
// and backfill receipts attached to the run that produced them. So the card
@@ -919,7 +784,7 @@ export function ApplicationsHome({
? tn("apps.enabledCount", authorizedSyncSources.length)
: syncApplyFailed
? t("apps.enableFailed")
: wantsSync || viewModel.state === "loading"
: wantsSync || !visible
? t("apps.checkingState")
: t("apps.notEnabled");
function toggleAll(next: boolean) {
@@ -965,38 +830,31 @@ export function ApplicationsHome({
<header>
<h2>{t("apps.overview")}</h2>
</header>
{evolveRunning ? (
<p className="r6-evolve-status" role="status">
{text(evolveUncertain ? "正在核对整合状态…" : "正在整合记忆…")}
</p>
) : null}
<div className="r6-home-metrics">
<div role="status" aria-live="polite" aria-atomic="true">
<p>{t(viewModel.countLabel)}</p>
<p>{t("apps.memories")}</p>
<p className="r6-home-figure">
<strong>
{viewModel.count === null
? "—"
: new Intl.NumberFormat(locale).format(viewModel.count)}
{new Intl.NumberFormat(locale).format(
visible?.memory_count ?? 0,
)}
</strong>
{viewModel.count !== null && t("apps.unit") ? (
<i>{t("apps.unit")}</i>
) : null}
{t("apps.unit") ? <i>{t("apps.unit")}</i> : null}
</p>
<small>
{viewModel.state !== "loading"
{visible
? summaryStale
? t("apps.showingPrevious")
: t(viewModel.countDetail)
: t("apps.scope")
: summaryUnavailable
? t("apps.countWillUpdate")
: summaryFailures
? t("apps.startingRetry")
: t(viewModel.countDetail)}
: t("apps.readingCount")}
</small>
</div>
<div>
<p>{t(viewModel.processingTitle)}</p>
<p>{t("apps.processing")}</p>
<strong className="r6-home-word">{stateLabel}</strong>
<small>{stateDetail}</small>
</div>
@@ -1004,14 +862,16 @@ export function ApplicationsHome({
<p>{t("apps.recentUpdate")}</p>
<strong className={time ? "r6-home-time" : "r6-home-word"}>
{time ??
(viewModel.recentMetric.valueKey
? t(viewModel.recentMetric.valueKey)
: "")}
(visible
? t("apps.notGenerated")
: t("apps.checkingRecent"))}
</strong>
<small>
{time
? `${latestClock?.today ? `${t("apps.today")}· ` : ""}${t(viewModel.recentMetric.detailKey)}`
: t(viewModel.recentMetric.detailKey)}
? `${latestClock?.today ? `${t("apps.today")}· ` : ""}${t("apps.saved")}`
: visible
? t("apps.separate")
: t("apps.recentWillUpdate")}
</small>
</div>
</div>
@@ -1045,7 +905,7 @@ export function ApplicationsHome({
? t("apps.recentCall")
: activity === "memory"
? t("apps.recentUpdate")
: t(viewModel.activity.headingKey)}
: t("apps.firstBatch")}
</p>
)}
<div>
@@ -1075,13 +935,7 @@ export function ApplicationsHome({
})()
) : null}
<section>
<h3
className={
activity === "memory" && latest?.kind === "raw"
? "r6-home-raw-excerpt"
: undefined
}
>
<h3>
{activity === "unavailable"
? callState === "unavailable"
? t("apps.callsUnavailable")
@@ -1093,12 +947,16 @@ export function ApplicationsHome({
locale,
)
: activity === "memory" && latest
? latest.emptyExcerpt
? t("apps.rawNoExcerpt")
: latest.excerpt
: viewModel.activity.titleKey
? t(viewModel.activity.titleKey)
: ""}
? `${latest.content}${"truncated" in latest && latest.truncated ? "…" : ""}`
: activity === "importing"
? t("apps.importing")
: activity === "first-running"
? t("apps.firstRunning")
: activity === "first-queued"
? t("apps.firstQueued")
: activity === "loading"
? t("apps.readingActivity")
: t("apps.emptyMemories")}
</h3>
<small>
{activity === "unavailable"
@@ -1108,24 +966,14 @@ export function ApplicationsHome({
: activity === "call" && recentCall
? `${recentCallHostName} · ${stamped(recentCall.occurred_at, locale, t)} · ${callRowDetail(recentCall, locale)}${callState === "incomplete" ? t("apps.incompleteCalls") : ""}`
: activity === "memory" && latest
? (latest.kind === "raw"
? [
t("apps.rawConversation"),
latestSourceNames.join(", "),
projectionTimeBasisLabel(latest.timeBasis, t),
time,
]
: [
time,
latestSourceNames.join(", "),
projectionTimeBasisLabel(latest.timeBasis, t),
]
)
? [time, latestSourceNames.join(", ")]
.filter(Boolean)
.join(" · ")
: viewModel.activity.detailKey
? t(viewModel.activity.detailKey)
: ""}
: activity === "importing"
? t("apps.savingConversations")
: activity === "loading"
? t("apps.recentWillUpdate")
: t("apps.waitingMemories")}
</small>
</section>
{(activity === "unavailable" ||
@@ -1139,12 +987,16 @@ export function ApplicationsHome({
setRevision((n) => n + 1);
setCallRevision((n) => n + 1);
}
: onCalls
: activity === "call"
? onCalls
: () => onMemories(latest?.memory_id)
}
>
{activity === "unavailable"
? t("apps.reread")
: t("apps.viewCalls")}
: activity === "call"
? t("apps.viewCalls")
: t("apps.viewMemory")}
{activity === "unavailable" ? null : (
<ArrowRight size={14} aria-hidden="true" />
)}
@@ -1197,12 +1049,8 @@ export function ApplicationsHome({
<dd>{syncState}</dd>
</div>
<div>
<dt>{t(viewModel.organizingTitle)}</dt>
<dd>{t(viewModel.organizingLabel as never)}</dd>
</div>
<div>
<dt>{t("apps.currentMode")}</dt>
<dd>{viewModel.currentModeLabel}</dd>
<dt>{t("apps.organizing")}</dt>
<dd>{stateLabel}</dd>
</div>
<div>
<dt>{t("apps.access")}</dt>
+10 -10
View File
@@ -159,9 +159,7 @@ it("traduce el historial en vivo sin cambiar nombres ni contenido importado", as
expect(
screen.getByRole("heading", { name: "Historial de uso" }),
).toBeTruthy();
expect(first.textContent).toBe(
"Cursor encontró 1 memoria · 2 conversaciones originales",
);
expect(first.textContent).toBe("Cursor encontró 3 contenidos relacionados");
fireEvent.click(first);
expect(
screen.getByRole("heading", { name: "Esta consulta de memorias" }),
@@ -366,7 +364,7 @@ it("shows exact identity, timestamp, result counts and metadata-only references"
).toBeTruthy();
expect(
within(detail).getByText((value) =>
value.includes("Cursor 找到 1 条记忆 · 2 条原始会话"),
value.includes("Cursor 查到 3 条相关内容"),
),
).toBeTruthy();
expect(within(detail).getAllByText("source-1").length).toBeGreaterThan(0);
@@ -467,9 +465,13 @@ it("keeps detail bound to event ID across refresh reordering, then removes missi
});
it.each([
["grounded", "查询成功", "直接调用(诊断) 调用已完成"],
["grounded", "查询成功", "直接调用(诊断) 查到 0 条相关内容"],
["empty", "查询成功,没有找到相关记忆", "直接调用(诊断) 没有查到相关记忆"],
["partial", "查询成功,部分内容还在整理中", "直接调用(诊断) 调用已完成"],
[
"partial",
"查询成功,部分内容还在整理中",
"直接调用(诊断) 查到 0 条相关内容",
],
[
"timed_out",
"查询超时,没有提供记忆",
@@ -544,13 +546,11 @@ it.each(["om_recall", "om_conversations", "om_ask", "om_matters"] as const)(
mount();
expect(
await screen.findByRole("button", {
name: "Cursor 找到 1 条记忆 · 2 条原始会话",
name: "Cursor 查到 3 条相关内容",
}),
).toBeTruthy();
fireEvent.click(
screen.getByRole("button", {
name: "Cursor 找到 1 条记忆 · 2 条原始会话",
}),
screen.getByRole("button", { name: "Cursor 查到 3 条相关内容" }),
);
expect(
within(screen.getByRole("article")).getByText("查记忆"),
@@ -27,7 +27,6 @@ import {
EmbeddingSwitchConfirmDialog,
EmbeddingSwitchGlobalStatus,
} from "./EmbeddingSwitchExperience";
import { LANGUAGE_STORAGE_KEY, LanguageProvider } from "./gui-language";
const NOW = "2026-11-03T09:00:00Z";
const space = (id: string | null, model: string) => ({
@@ -102,59 +101,9 @@ const job: EmbeddingSwitchJob = {
afterEach(() => {
cleanup();
current = null;
localStorage.removeItem(LANGUAGE_STORAGE_KEY);
vi.clearAllMocks();
});
it("switches Chinese, Spanish, and English without leaking Chinese", async () => {
localStorage.setItem(LANGUAGE_STORAGE_KEY, "zh-Hans");
const view = render(
<LanguageProvider>
<EmbeddingSwitchConfirmDialog
plan={{ ...plan, users: 2 }}
busy={false}
errorCode={null}
onCancel={vi.fn()}
onStart={vi.fn()}
/>
</LanguageProvider>,
);
await waitFor(() =>
expect(
screen.getByRole("heading", { name: "更新记忆搜索索引?" }),
).toBeTruthy(),
);
localStorage.setItem(LANGUAGE_STORAGE_KEY, "es");
window.dispatchEvent(
new StorageEvent("storage", { key: LANGUAGE_STORAGE_KEY }),
);
await waitFor(() =>
expect(
screen.getByRole("heading", {
name: "¿Actualizar el índice de búsqueda de memorias?",
}),
).toBeTruthy(),
);
expect(view.container.textContent).not.toMatch(/[一-鿿]/);
expect(view.container.textContent).toContain("8421");
localStorage.setItem(LANGUAGE_STORAGE_KEY, "en");
window.dispatchEvent(
new StorageEvent("storage", { key: LANGUAGE_STORAGE_KEY }),
);
await waitFor(() =>
expect(
screen.getByRole("heading", {
name: "Update the memory search index?",
}),
).toBeTruthy(),
);
expect(view.container.textContent).not.toMatch(/[一-鿿]/);
expect(view.container.textContent).toContain("8,421");
});
it("explains the index update in user terms before exposing technical details", async () => {
const onStart = vi.fn();
render(
+104 -143
View File
@@ -3,68 +3,50 @@ import type {
EmbeddingSwitchJob,
EmbeddingSwitchPlan,
} from "@one-memory/contracts";
import type { GuiLocale, MessageKey } from "@one-memory/i18n";
import { ChevronDown, CircleAlert, X } from "lucide-react";
import { useEffect, useRef, useState } from "react";
import { deleteRetiredEmbeddingSpace } from "./api";
import { useGuiLanguage } from "./gui-language";
import { R6Button } from "./R6Controls";
import { useEmbeddingSwitch } from "./use-embedding-switch";
import "./embedding-switch.css";
type Translator = (
key: MessageKey,
params?: Record<string, string | number>,
) => string;
function count(locale: GuiLocale, value: number | null | undefined) {
return new Intl.NumberFormat(locale).format(value ?? 0);
function count(value: number | null | undefined) {
return new Intl.NumberFormat().format(value ?? 0);
}
function duration(locale: GuiLocale, t: Translator, seconds: number | null) {
if (seconds === null) return t("embeddingSwitch.duration.unknown");
const value = Math.max(1, Math.round(seconds < 60 ? seconds : seconds / 60));
return t(
seconds < 60
? "embeddingSwitch.duration.seconds"
: "embeddingSwitch.duration.minutes",
{ count: count(locale, value) },
);
function duration(seconds: number | null) {
if (seconds === null) return "暂时无法估算";
if (seconds < 60) return `约 ${Math.max(1, Math.round(seconds))} 秒`;
return `约 ${Math.max(1, Math.round(seconds / 60))} 分钟`;
}
function size(locale: GuiLocale, bytes: number) {
const megabytes = bytes >= 1024 * 1024;
const value = Math.ceil(bytes / 1024 / (megabytes ? 1024 : 1));
return `${count(locale, value)} ${megabytes ? "MB" : "KB"}`;
function size(bytes: number) {
if (bytes < 1024 * 1024) return `${Math.ceil(bytes / 1024)} KB`;
return `${Math.ceil(bytes / 1024 / 1024)} MB`;
}
const blockerKeys: Record<string, MessageKey> = {
embedding_probe_required: "embeddingSwitch.blocker.embedding_probe_required",
const blockerCopy: Record<string, string> = {
embedding_probe_required: "请先测试新的搜索模型连接。",
embedding_dimensions_unknown:
"embeddingSwitch.blocker.embedding_dimensions_unknown",
"还无法确认新搜索模型的索引规格,请重新测试连接。",
embedding_switch_requires_migration:
"embeddingSwitch.blocker.embedding_switch_requires_migration",
sdk_reembed_unavailable: "embeddingSwitch.blocker.sdk_reembed_unavailable",
insufficient_disk_space: "embeddingSwitch.blocker.insufficient_disk_space",
embedding_switch_running: "embeddingSwitch.blocker.embedding_switch_running",
memory_lifecycle_busy: "embeddingSwitch.blocker.memory_lifecycle_busy",
"旧版本数据仍在升级,请等待升级完成后再更换。",
sdk_reembed_unavailable: "当前安装版本还不能安全更新记忆索引。",
insufficient_disk_space: "本机剩余空间不足,无法安全建立新索引。",
embedding_switch_running: "另一次索引更新正在进行。",
memory_lifecycle_busy: "记忆正在整合,请等待完成后再更换搜索模型。",
};
const failureKeys: Record<string, MessageKey> = {
writes_did_not_drain: "embeddingSwitch.failure.writes_did_not_drain",
embedding_service_failed: "embeddingSwitch.failure.embedding_service_failed",
embedding_dimensions_mismatch:
"embeddingSwitch.failure.embedding_dimensions_mismatch",
embedding_verification_failed:
"embeddingSwitch.failure.embedding_verification_failed",
target_profile_changed: "embeddingSwitch.failure.target_profile_changed",
embedding_switch_cancelled:
"embeddingSwitch.failure.embedding_switch_cancelled",
model_secret_unavailable: "embeddingSwitch.failure.model_secret_unavailable",
secure_storage_unavailable:
"embeddingSwitch.failure.secure_storage_unavailable",
embedding_switch_clients_busy:
"embeddingSwitch.failure.embedding_switch_clients_busy",
const failureCopy: Record<string, string> = {
writes_did_not_drain: "等待正在进行的记忆整理结束超时。",
embedding_service_failed: "搜索模型服务连续请求失败。",
embedding_dimensions_mismatch: "搜索服务返回的索引规格与测试结果不一致。",
embedding_verification_failed: "新的记忆索引没有通过完整性校验。",
target_profile_changed: "更新期间搜索模型配置发生了变化。",
embedding_switch_cancelled: "索引更新已取消。",
model_secret_unavailable: "无法读取搜索服务的 API Key。",
secure_storage_unavailable: "系统安全存储暂时不可用。",
embedding_switch_clients_busy: "记忆服务一直被占用,暂时无法完成切换。",
};
export function EmbeddingSwitchConfirmDialog({
@@ -80,10 +62,9 @@ export function EmbeddingSwitchConfirmDialog({
onCancel(): void;
onStart(): void;
}) {
const { locale, t } = useGuiLanguage();
const panel = useRef<HTMLElement>(null);
const blockers = plan.blockers.map((code) =>
t(blockerKeys[code] ?? "embeddingSwitch.blocker.default"),
const blockers = plan.blockers.map(
(code) => blockerCopy[code] ?? "当前状态暂时不允许更换搜索模型。",
);
useEffect(() => {
@@ -132,88 +113,78 @@ export function EmbeddingSwitchConfirmDialog({
>
<header>
<div>
<h2 id="embedding-switch-title">
{t("embeddingSwitch.confirm.title")}
</h2>
<p>{t("embeddingSwitch.confirm.description")}</p>
<h2 id="embedding-switch-title">更新记忆搜索索引?</h2>
<p>
不同搜索模型建立的索引不能混用。为了继续找到已有记忆,需要重新建立索引。
</p>
</div>
</header>
<dl className="embedding-switch-impact">
<div>
<dt>{t("embeddingSwitch.impact.memories")}</dt>
<dt>已有记忆</dt>
<dd>
{t("embeddingSwitch.impact.memoryCount", {
count: count(locale, plan.total_vectors),
})}
{count(plan.total_vectors)} 条
{plan.users && plan.users > 1
? t("embeddingSwitch.impact.userCount", {
count: count(locale, plan.users),
})
? ` · ${count(plan.users)} 位用户`
: ""}
</dd>
</div>
<div>
<dt>{t("embeddingSwitch.impact.duration")}</dt>
<dd>{duration(locale, t, plan.estimate_s)}</dd>
<dt>预计耗时</dt>
<dd>{duration(plan.estimate_s)}</dd>
</div>
<div>
<dt>{t("embeddingSwitch.impact.requests")}</dt>
<dt>远程请求</dt>
<dd>
{plan.target_local
? t("embeddingSwitch.requests.none")
? "不会产生"
: plan.requests === null
? t("embeddingSwitch.requests.billed")
: t("embeddingSwitch.requests.approx", {
count: count(locale, plan.requests),
})}
? "按服务商计费"
: `约 ${count(plan.requests)} 次`}
</dd>
</div>
</dl>
<div className="embedding-switch-safety">
<strong>{t("embeddingSwitch.safety.searchTitle")}</strong>
<p>{t("embeddingSwitch.safety.searchBody")}</p>
<strong>{t("embeddingSwitch.safety.dataTitle")}</strong>
<p>{t("embeddingSwitch.safety.dataBody")}</p>
<strong>更新期间仍可搜索记忆</strong>
<p>
MemHub
会继续使用原来的搜索模型。新对话仍会保存,更新完成后再整理成记忆。
</p>
<strong>失败不会影响已有数据</strong>
<p>
只有全部更新并校验通过后才会切换;失败或取消时仍使用原来的模型。
</p>
</div>
<details className="embedding-switch-details">
<summary>
<ChevronDown size={14} aria-hidden="true" />
{t("embeddingSwitch.details.title")}
查看技术详情
</summary>
<dl>
<div>
<dt>{t("embeddingSwitch.details.currentModel")}</dt>
<dt>当前搜索模型</dt>
<dd>{plan.source.model ?? "旧版本索引"}</dd>
</div>
<div>
<dt>新的搜索模型</dt>
<dd>{plan.target.model ?? "待确认"}</dd>
</div>
<div>
<dt>索引维度</dt>
<dd>
{plan.source.model ?? t("embeddingSwitch.details.legacyIndex")}
{plan.source.dims ?? "未知"} → {plan.target.dims ?? "未知"}
</dd>
</div>
<div>
<dt>{t("embeddingSwitch.details.newModel")}</dt>
<dd>
{plan.target.model ?? t("embeddingSwitch.details.pending")}
</dd>
<dt>额外磁盘空间</dt>
<dd>{size(plan.disk_bytes_needed)}</dd>
</div>
<div>
<dt>{t("embeddingSwitch.details.dimensions")}</dt>
<dd>
{plan.source.dims ?? t("embeddingSwitch.details.unknown")} →{" "}
{plan.target.dims ?? t("embeddingSwitch.details.unknown")}
</dd>
</div>
<div>
<dt>{t("embeddingSwitch.details.disk")}</dt>
<dd>{size(locale, plan.disk_bytes_needed)}</dd>
</div>
<div>
<dt>{t("embeddingSwitch.details.retention")}</dt>
<dd>
{t("embeddingSwitch.details.days", {
count: count(locale, plan.retention_days),
})}
</dd>
<dt>旧索引保留</dt>
<dd>{plan.retention_days} 天</dd>
</div>
</dl>
</details>
@@ -229,15 +200,15 @@ export function EmbeddingSwitchConfirmDialog({
<CircleAlert size={16} aria-hidden="true" />
<span>
{errorCode === "embedding_switch_plan_stale"
? t("embeddingSwitch.error.planStale")
: t("embeddingSwitch.error.startFailed")}
? "记忆状态发生了变化,请重新检查更换影响。"
: "索引更新暂时无法开始,请稍后重试。"}
</span>
</div>
) : null}
<footer>
<R6Button disabled={busy} onClick={onCancel}>
{t("embeddingSwitch.action.notNow")}
暂不更换
</R6Button>
<R6Button
variant="primary"
@@ -245,7 +216,7 @@ export function EmbeddingSwitchConfirmDialog({
disabled={blockers.length > 0}
onClick={onStart}
>
{t("embeddingSwitch.action.start")}
开始更新索引
</R6Button>
</footer>
</section>
@@ -253,14 +224,14 @@ export function EmbeddingSwitchConfirmDialog({
);
}
function jobTitle(t: Translator, job: EmbeddingSwitchJob) {
if (job.kind === "merge") return t("embeddingSwitch.job.merge");
if (job.state === "preparing") return t("embeddingSwitch.job.preparing");
if (job.state === "reembedding") return t("embeddingSwitch.job.reembedding");
if (job.state === "verifying") return t("embeddingSwitch.job.verifying");
if (job.state === "committing") return t("embeddingSwitch.job.committing");
if (job.state === "failed") return t("embeddingSwitch.job.failed");
return t("embeddingSwitch.job.default");
function jobTitle(job: EmbeddingSwitchJob) {
if (job.kind === "merge") return "正在整理旧版本留下的搜索索引";
if (job.state === "preparing") return "正在准备更新记忆索引";
if (job.state === "reembedding") return "正在更新记忆索引";
if (job.state === "verifying") return "正在校验新的记忆索引";
if (job.state === "committing") return "正在启用新的搜索模型";
if (job.state === "failed") return "更新没有完成,正在恢复原设置";
return "记忆索引更新";
}
function canCancel(job: EmbeddingSwitchJob) {
@@ -276,36 +247,30 @@ function ActiveSwitchBanner({
onCancel(): void;
cancelling: boolean;
}) {
const { locale, t } = useGuiLanguage();
const job = current.job;
if (!job && current.phase !== "scanning") return null;
const percent = job?.progress.percent ?? 0;
return (
<aside className="embedding-switch-strip" aria-live="polite">
<div className="embedding-switch-strip-copy">
<strong>
{job ? jobTitle(t, job) : t("embeddingSwitch.job.scanning")}
</strong>
<strong>{job ? jobTitle(job) : "正在检查本机的记忆搜索索引"}</strong>
<span>
{job?.state === "failed"
? t("embeddingSwitch.job.failedSafety")
? "原来的搜索模型仍在使用,记忆没有丢失。"
: job
? `${new Intl.NumberFormat(locale, {
style: "percent",
maximumFractionDigits: 0,
}).format(percent / 100)}${
? `${Math.round(percent)}%${
job.progress.eta_s === null
? ""
: ` · ${duration(locale, t, job.progress.eta_s)}`
: ` · ${duration(job.progress.eta_s)}`
}`
: t("embeddingSwitch.job.scanPaused")}
: "完成前暂不整理新记忆,搜索仍可使用。"}
</span>
</div>
{job ? (
<div
className="embedding-switch-progress"
role="progressbar"
aria-label={t("embeddingSwitch.progress.aria")}
aria-label="记忆索引更新进度"
aria-valuemin={0}
aria-valuemax={100}
aria-valuenow={Math.round(percent)}
@@ -315,7 +280,7 @@ function ActiveSwitchBanner({
) : null}
{job && canCancel(job) ? (
<R6Button variant="quiet" pending={cancelling} onClick={onCancel}>
{t("embeddingSwitch.action.cancel")}
取消更新
</R6Button>
) : null}
</aside>
@@ -329,7 +294,6 @@ export function EmbeddingSwitchGlobalStatus({
onConfigureSearch(): void;
onDiagnostics(): void;
}) {
const { t } = useGuiLanguage();
const controller = useEmbeddingSwitch();
const [recent, setRecent] = useState<EmbeddingSwitchJob | null>(null);
const [cleaning, setCleaning] = useState(false);
@@ -368,11 +332,11 @@ export function EmbeddingSwitchGlobalStatus({
{controller.refreshFailed && controller.supported === true ? (
<aside className="embedding-switch-outcome is-error" role="status">
<div>
<strong>{t("embeddingSwitch.refresh.title")}</strong>
<span>{t("embeddingSwitch.refresh.body")}</span>
<strong>索引状态暂时无法读取</strong>
<span>当前显示可能不是最新状态,可以继续使用已有记忆。</span>
</div>
<R6Button variant="quiet" onClick={() => void controller.refresh()}>
{t("embeddingSwitch.action.refresh")}
重新读取
</R6Button>
</aside>
) : null}
@@ -380,15 +344,15 @@ export function EmbeddingSwitchGlobalStatus({
["failed", "unavailable"].includes(controller.current.scan_state) ? (
<aside className="embedding-switch-outcome is-error" role="alert">
<div>
<strong>{t("embeddingSwitch.scan.title")}</strong>
<strong>旧版本搜索索引检查未完成</strong>
<span>
{controller.current.writes_paused
? t("embeddingSwitch.scan.paused")
: t("embeddingSwitch.scan.recovered")}
? "新的记忆整理暂时暂停,已有记忆仍可搜索。"
: "记忆整理已经恢复;可以继续使用,诊断中保留了失败原因。"}
</span>
</div>
<R6Button variant="quiet" onClick={onDiagnostics}>
{t("embeddingSwitch.action.diagnostics")}
查看诊断
</R6Button>
</aside>
) : null}
@@ -399,8 +363,8 @@ export function EmbeddingSwitchGlobalStatus({
aria-live="polite"
>
<div>
<strong>{t("embeddingSwitch.success.title")}</strong>
<span>{t("embeddingSwitch.success.body")}</span>
<strong>搜索模型已更换</strong>
<span>暂停期间保存的新对话正在整理成记忆。</span>
</div>
<div className="embedding-switch-outcome-actions">
{recent.source.space_id ? (
@@ -423,12 +387,12 @@ export function EmbeddingSwitchGlobalStatus({
});
}}
>
{t("embeddingSwitch.action.cleanup")}
清理旧索引
</R6Button>
) : null}
<R6Button
variant="quiet"
aria-label={t("embeddingSwitch.action.closeSuccess")}
aria-label="关闭完成提示"
onClick={() => setRecent(null)}
>
<X size={16} aria-hidden="true" />
@@ -438,7 +402,7 @@ export function EmbeddingSwitchGlobalStatus({
) : null}
{cleanupError ? (
<p className="embedding-switch-cleanup-error" role="alert">
{t("embeddingSwitch.cleanup.failed")}
旧索引暂时无法清理,MemHub 会在保留期结束后自动重试。
</p>
) : null}
{recent?.state === "rolled_back" ? (
@@ -448,26 +412,23 @@ export function EmbeddingSwitchGlobalStatus({
aria-live="assertive"
>
<div>
<strong>{t("embeddingSwitch.rollback.title")}</strong>
<strong>索引更新没有完成</strong>
<span>
{failureKeys[recent.error_code ?? ""]
? t("embeddingSwitch.rollback.reason", {
reason: t(failureKeys[recent.error_code ?? ""]),
safety: t("embeddingSwitch.rollback.safety"),
})
: t("embeddingSwitch.rollback.safety")}
{failureCopy[recent.error_code ?? ""]
? `${failureCopy[recent.error_code ?? ""]} 仍在使用原来的搜索模型,记忆没有丢失。`
: "仍在使用原来的搜索模型,记忆没有丢失。"}
</span>
</div>
<div className="embedding-switch-outcome-actions">
<R6Button variant="quiet" onClick={onDiagnostics}>
{t("embeddingSwitch.action.diagnostics")}
查看诊断
</R6Button>
<R6Button variant="quiet" onClick={onConfigureSearch}>
{t("embeddingSwitch.action.checkConnection")}
检查连接
</R6Button>
<R6Button
variant="quiet"
aria-label={t("embeddingSwitch.action.closeFailure")}
aria-label="关闭失败提示"
onClick={() => setRecent(null)}
>
<X size={16} aria-hidden="true" />
@@ -239,7 +239,7 @@ it("emits controlled connection drafts only and passes continuation without comp
screen.getByRole<HTMLInputElement>("checkbox", { name: "一键接入应用" })
.checked,
).toBe(false);
fireEvent.click(screen.getByRole("button", { name: "选择记忆模式" }));
fireEvent.click(screen.getByRole("button", { name: "配置模型,沉淀记忆" }));
expect(next).toHaveBeenCalledWith("settings");
});
@@ -371,14 +371,16 @@ it("uses user-facing connection confirmation copy without implementation narrati
/>,
);
expect(screen.getByRole("complementary", { name: "连接应用" })).toBeTruthy();
expect(screen.getByRole("checkbox", { name: "一键接入应用" })).toBeTruthy();
expect(
screen.queryByText(
/点击底部按钮后|选择草稿|按各应用回执|本页任务控制尚未接通/,
screen.getByText(
"点击底部按钮后,后台分别接入各应用;某个应用失败不影响其他应用。",
),
).toBeTruthy();
expect(
screen.queryByText(/选择草稿|按各应用回执|本页任务控制尚未接通/),
).toBeNull();
expect(screen.queryByText(/下一步选择记忆模式/)).toBeNull();
expect(screen.queryByText(/之后仍可在设置中更改/)).toBeNull();
expect(screen.getByText(/不会自动开放应用读取权限/)).toBeTruthy();
expect(screen.getByText(/配置模型后,才会开始生成记忆/)).toBeTruthy();
});
it.each(["paused", "failed", "cancelled", "interrupted"] as const)(
@@ -396,7 +398,11 @@ it.each(["paused", "failed", "cancelled", "interrupted"] as const)(
expect(
screen.getByRole("heading", { name: "对话正在汇入,记忆从这里开始" }),
).toBeTruthy();
expect(screen.queryByText(/已保存的对话还在/)).toBeNull();
expect(
screen.getByText(/已保存的对话还在/, {
selector: ".first-import-r6-footer-note p",
}),
).toBeTruthy();
},
);
@@ -433,7 +439,9 @@ it.each(["stale", "revoked", "missing"])(
onContinue={vi.fn()}
/>,
);
expect(screen.queryByText("导入在后台继续,无需等待。")).toBeNull();
if (kind === "missing")
expect(screen.getByText("导入在后台继续,无需等待。")).toBeTruthy();
else expect(screen.queryByText("导入在后台继续,无需等待。")).toBeNull();
expect(
screen.getByRole("heading", { name: "对话正在汇入,记忆从这里开始" }),
).toBeTruthy();
@@ -147,7 +147,7 @@ it("explains that one large harness limits every first-import source to three da
it("continues to step3 while sample GET is pending without calling completion APIs", () => {
api.receipt.mockReturnValue(new Promise(() => {}));
const { props } = setup();
fireEvent.click(screen.getByRole("button", { name: "选择记忆模式" }));
fireEvent.click(screen.getByRole("button", { name: "配置模型,沉淀记忆" }));
expect(props.onContinue).toHaveBeenCalledWith("settings");
expect(screen.queryByText(/^已生成.*记忆/)).toBeNull();
});
@@ -199,8 +199,8 @@ it("keeps the saved count when authority or freshness changes, without a convers
expect(view.container.querySelector('[data-running="true"]')).toBeTruthy();
expect(screen.getByText("已保存 7 条消息")).toBeTruthy();
expect(
screen.queryByText("正在重新核对来源;已保存数量仍按任务回执显示。"),
).toBeNull();
screen.getAllByText("正在重新核对来源;已保存数量仍按任务回执显示。"),
).toHaveLength(1);
view.update({
sourceSnapshot: { confirmedAt: "2026-09-14T00:00:02Z", stale: false },
jobs: [{ ...job, stored: 20 }],
Loaded 100 of 246 files, more files were not shown because too many files have changed in this diff. Show more