Compare commits
23
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a218a92d4e | ||
|
|
6845adf083 | ||
|
|
850c728c9d | ||
|
|
ce9801b449 | ||
|
|
576d9bf408 | ||
|
|
34f406e96d | ||
|
|
a653bd4065 | ||
|
|
07b426ba7f | ||
|
|
4b83b79b2b | ||
|
|
675cc30e5a | ||
|
|
3511153f9f | ||
|
|
23facf75d5 | ||
|
|
71b8f9be24 | ||
|
|
d257188d09 | ||
|
|
bde0220373 | ||
|
|
a88d227cc0 | ||
|
|
0d0cefb25e | ||
|
|
be8cf028ff | ||
|
|
d0a4fd60f2 | ||
|
|
aae877b880 | ||
|
|
ef5ff06590 | ||
|
|
7eb38aa6ad | ||
|
|
2624228f6f |
No files matched your search
@@ -4,6 +4,77 @@ Hy Memory and MemHub are versioned and released independently.
|
||||
|
||||
## Unreleased
|
||||
|
||||
- MemHub multi-user hardening (follow-up to #250):
|
||||
- Read responses name the user they were read for: `user_id` (the one
|
||||
asked for, `local` by default) is now on `GET /api/v2/memory-library`,
|
||||
`/memory-library/layers`, `/user-profile`, `/home-summary`,
|
||||
`/memories/graph`, `/raw`, and `POST /api/v2/context/search`,
|
||||
`/api/v1/recall`, `/api/v2/memories/ask`. `ask/stream` carries it on its
|
||||
first (`recall`) event. The Host Bridge context validator accepts the new
|
||||
field and still accepts a Core without it.
|
||||
- `DELETE /api/v2/memories/{id}` answers 404 `memory_not_found` for another
|
||||
user's memory and for an id that does not exist (the same response, so
|
||||
existence is still not revealed), and also when the SDK finds nothing to
|
||||
delete; 204 only when the caller's own memory was deleted. The App treats
|
||||
that 404 as already gone (a double-click delete no longer shows an
|
||||
error); `memhub memories delete` prints "未找到记忆 <id>(可能已删除)",
|
||||
exits 0, and its JSON is `state: "committed"` with `already_gone: true`.
|
||||
- Hermes never reads a `profiles/` directory at any depth (scan, detect and
|
||||
live sync), so a root granted above `~/.hermes` cannot pick up another
|
||||
user's profile. Connecting refuses a root that is the `profiles`
|
||||
container itself (422 `hermes_root_is_profiles_container`) and a root
|
||||
equal to, containing or inside another user's active Hermes grant (409
|
||||
`hermes_root_overlaps_other_user`, symlinks resolved); `local` on
|
||||
`~/.hermes` with another user on `~/.hermes/profiles/<name>` stays
|
||||
allowed. The CLI explains both codes.
|
||||
- `memhub sources connect --user-id` with an invalid id now says the user
|
||||
id need not equal the Hermes profile name (e.g. `--user-id wife-work
|
||||
--root ~/.hermes/profiles/wife:work`).
|
||||
|
||||
- MemHub's DeepSeek Harness app page can grant recall again (#230). The
|
||||
"一键插入应用" switch used to be greyed out with `unsupported_host`, and a
|
||||
profile installed from the CLI stayed `install-only` with no way to approve
|
||||
it. The one switch now covers every detected profile
|
||||
(`deepseek-harness:<profile>`): on installs each profile's MCP patch and
|
||||
grants recall, off removes the patch and revokes the grant. The page lists
|
||||
each profile with its state, and names any profile that failed while
|
||||
keeping the others. While the switch is on, a profile created later is
|
||||
installed and granted automatically, without restarting MemHub; the Runtime
|
||||
also no longer fails every host read when it meets a profile created after
|
||||
it started. The harness docs no longer describe an unmounted sources page.
|
||||
|
||||
- MemHub's `deepseek-harness` source imports session files whose names carry the
|
||||
format generation: `session.v3.jsonl.zstd` and `session.v4.jsonl.zstd` are now
|
||||
discovered alongside the original `session.jsonl[.zstd]`. Discovery accepts
|
||||
any `session[.vN].jsonl[.zstd]` name and the header's `version` decides
|
||||
whether the session is parsed (v0/v3/v4) or refused as `unsupported_schema`.
|
||||
A refused file is reported and keeps no cursor, so the release that learns
|
||||
its format reads it from the top.
|
||||
v4 tool results, which are written as plain `text` blocks rather than
|
||||
`tool-result` blocks, are read through the same clipping and credential
|
||||
scrubbing path. Previously every versioned session was invisible: the import
|
||||
job completed with `discovered 0` while the files sat on disk.
|
||||
|
||||
## Hy Memory 2.0.12 — 2026-11-02
|
||||
|
||||
- Intention dedup is now configurable per client
|
||||
(`MemoryConfig.recall.intention_dedup_enabled`, default `true`;
|
||||
`recall.intention_dedup_threshold`, default `MEMORY_DEDUP_THRESHOLD` or
|
||||
`0.95`). Env defaults: `MEMORY_INTENTION_DEDUP_ENABLED`,
|
||||
`MEMORY_INTENTION_DEDUP_THRESHOLD`. When on, behaviour is unchanged
|
||||
(duplicates found at read time are still deleted); when off, no intention
|
||||
dedup runs, at read time or within a write batch, and nothing is hidden,
|
||||
dropped or deleted. The threshold applies to read-time dedup only. Refs #241.
|
||||
|
||||
- MemHub pins `hy-memory==2.0.12` and exposes that intention dedup switch in
|
||||
its memory-features setting (`GET`/`PUT /api/v1/settings/memory-features`,
|
||||
`memhub settings features`): the feature `intention_dedup` (default on) and
|
||||
the numeric `intention_dedup_threshold` (default 0.95, accepted 0.8–1.0,
|
||||
anything else is 422 or a CLI usage error). Unset, both keep the SDK
|
||||
default, so nothing changes for the App; a change rebuilds the SDK clients
|
||||
like the other feature switches. The API answer gains `parameters`, and
|
||||
`pipeline.features` can now carry the threshold as a number. Refs #241.
|
||||
|
||||
## MemHub 1.0.6 — 2026-11-02
|
||||
|
||||
Build 37, on `hy-memory==2.0.11`. Linux x64/arm64 Runtime and macOS arm64
|
||||
|
||||
@@ -182,9 +182,29 @@ SDK 不设置 Rerank Search 并发上限。开启后模型在 `HyMemoryClient`
|
||||
| `MEMORY_RECALL_STRENGTH_TAU` | `180` | 衰减时间常数,天 |
|
||||
| `MEMORY_RECALL_ACCESS_TRACKING` | `true` | Strength 开启时是否回写访问信息 |
|
||||
| `MEMORY_ENTITY_STORE_ENABLED` | `false` | 是否维护独立 entity collection;当前 Reader 不直接读取 |
|
||||
| `MEMORY_INTENTION_DEDUP_ENABLED` | `true` | 是否对 L7 Intention 判重(读侧和写入侧批内);见下文 |
|
||||
| `MEMORY_INTENTION_DEDUP_THRESHOLD` | `MEMORY_DEDUP_THRESHOLD`,再缺省为 `0.95` | Intention 读侧判重的 cosine 阈值 |
|
||||
|
||||
旧数据可调用当前公开方法 `client.build_entity_store(user_id)` 补建 Entity Store。
|
||||
|
||||
Intention 判重:开启时(默认,与以往行为一致),Search 召回的 L7 Intention 中,
|
||||
同 `user_id` + 同 `agent_id` + `memory_at` 同一天 + `valid_until` 相同、且 cosine ≥ 阈值的
|
||||
条目视为复本,只返回最早写入的一条,其余在后台从向量库**删除**。判重不区分内容中的项目、
|
||||
对象等语义差异,内容高度相似但对象不同的意图可能被误删;写入时同一批抽取结果也会做一次
|
||||
批内判重。关闭后这两处都不判重,不隐藏、不丢弃、也不删除意图。两项对应
|
||||
`MemoryConfig.recall.intention_dedup_enabled` /
|
||||
`recall.intention_dedup_threshold`,按 client 生效,环境变量只提供默认值:
|
||||
|
||||
```python
|
||||
client = HyMemoryClient.from_config({
|
||||
"recall": {"intention_dedup_enabled": False},
|
||||
})
|
||||
```
|
||||
|
||||
`MEMORY_DEDUP_THRESHOLD` 同时被写入侧批内意图判重和 dedup 内核使用;
|
||||
`MEMORY_INTENTION_DEDUP_THRESHOLD` 只影响读侧 Intention 判重,写入侧批内判重的阈值
|
||||
仍取 `MEMORY_DEDUP_THRESHOLD`。
|
||||
|
||||
## 日志、Trace 与历史
|
||||
|
||||
| 变量 | 默认值 | 说明 |
|
||||
|
||||
@@ -432,6 +432,14 @@ class RecallConfig:
|
||||
retrieval_clauses_require_keyword: bool = None # env: MEMORY_RETRIEVAL_CLAUSES_REQUIRE_KEYWORD
|
||||
retrieval_clauses_min_keyword_score: float = None # normalized BM25 floor
|
||||
supplemental_propositions_enabled: bool = None # env: MEMORY_SUPPLEMENTAL_PROPOSITIONS_ENABLED
|
||||
# ── Intention(L7)判重 ──
|
||||
# 开启(默认):召回时同 user+agent+同天 memory_at+同 valid_until 且
|
||||
# cosine≥阈值 的意图视为复本,只返回最早一条,其余异步从 VDB 删除(历史行为)。
|
||||
# 关闭:读侧与写入侧批内都不判重——不隐藏、不丢弃、不删除任何意图。
|
||||
intention_dedup_enabled: bool = None # env: MEMORY_INTENTION_DEDUP_ENABLED
|
||||
# 读侧意图判重 cosine 阈值。env: MEMORY_INTENTION_DEDUP_THRESHOLD,
|
||||
# 未设置时回退到共享的 MEMORY_DEDUP_THRESHOLD(兼容旧部署),默认 0.95。
|
||||
intention_dedup_threshold: float = None
|
||||
|
||||
# 最小分数阈值
|
||||
min_score_threshold: float = 0.0
|
||||
@@ -750,6 +758,15 @@ class RecallConfig:
|
||||
self.supplemental_propositions_enabled = _get_env_bool(
|
||||
"MEMORY_SUPPLEMENTAL_PROPOSITIONS_ENABLED", False
|
||||
)
|
||||
if self.intention_dedup_enabled is None:
|
||||
self.intention_dedup_enabled = _get_env_bool(
|
||||
"MEMORY_INTENTION_DEDUP_ENABLED", True
|
||||
)
|
||||
if self.intention_dedup_threshold is None:
|
||||
self.intention_dedup_threshold = _get_env_float(
|
||||
"MEMORY_INTENTION_DEDUP_THRESHOLD",
|
||||
_get_env_float("MEMORY_DEDUP_THRESHOLD", 0.95),
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -1384,6 +1401,8 @@ class MemoryConfig:
|
||||
"supplemental_propositions_enabled": (
|
||||
self.recall.supplemental_propositions_enabled
|
||||
),
|
||||
"intention_dedup_enabled": self.recall.intention_dedup_enabled,
|
||||
"intention_dedup_threshold": self.recall.intention_dedup_threshold,
|
||||
},
|
||||
"pipeline": {
|
||||
"default_version": self.pipeline.default_version,
|
||||
|
||||
@@ -15,6 +15,9 @@ HY Memory - Intention (L7) 召回 + 惰性过期转换 + 复本判定
|
||||
→ 视为同一个意图的复本(extractor 复读机 / 多 session 重复表达)。
|
||||
保留组里 `gmt_created` 最早的(老的先来先得,稳定 id);
|
||||
- **异步删库**:其余复本 fire-and-forget 从 VDB 删除,不影响本次调用 RT。
|
||||
- **开关**:判重受 `MemoryConfig.recall.intention_dedup_enabled`(默认开)与
|
||||
`recall.intention_dedup_threshold` 控制,由 reader 按 client 配置传入。
|
||||
关闭时完全跳过判重:不隐藏、不删除任何意图(写入侧批内判重同样跳过)。
|
||||
|
||||
Q: 为什么要看 memory_at 和 valid_until?
|
||||
A: 意图是 point-in-time 信号。同一句"我周五要开会",如果一条 memory_at 在
|
||||
@@ -38,8 +41,14 @@ from ...models.memory import MemoryLayer, MemoryStatus
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _get_dedup_threshold() -> float:
|
||||
"""去重 cosine 阈值:沿用 dedup 内核的 MEMORY_DEDUP_THRESHOLD,默认 0.95。"""
|
||||
def _get_dedup_threshold(configured: Optional[float] = None) -> float:
|
||||
"""去重 cosine 阈值:优先用 client 配置;否则(或配置非法)沿用 dedup 内核的
|
||||
MEMORY_DEDUP_THRESHOLD,默认 0.95。"""
|
||||
try:
|
||||
if configured is not None:
|
||||
return float(configured)
|
||||
except (TypeError, ValueError):
|
||||
logger.warning(f"[intention] invalid dedup threshold: {configured!r}")
|
||||
try:
|
||||
return float(os.environ.get("MEMORY_DEDUP_THRESHOLD", 0.95))
|
||||
except (TypeError, ValueError):
|
||||
@@ -244,6 +253,8 @@ async def recall_intentions(
|
||||
stats_out: Optional[Dict[str, Any]] = None,
|
||||
include_sources: Optional[List[str]] = None,
|
||||
exclude_sources: Optional[List[str]] = None,
|
||||
dedup_enabled: bool = True,
|
||||
dedup_threshold: Optional[float] = None,
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
召回未过期的 L7_INTENTION 节点(VDB),惰性过期转 L2_FACT,判重并异步删复本。
|
||||
@@ -260,6 +271,11 @@ async def recall_intentions(
|
||||
now: 当前时间(默认 datetime.now(),测试可注入)
|
||||
stats_out: 可选 dict,本函数把分段耗时与数量**原地回填**进去,供调用方落
|
||||
READ_INTENTION pipeline log。传 None 则完全不统计(零开销)。
|
||||
dedup_enabled: 是否执行读侧判重(及复本异步删库)。False 时跳过判重,
|
||||
所有未过期意图原样返回,不删除任何节点。默认 True(历史行为)。
|
||||
dedup_threshold: 判重 cosine 阈值。None 时回退到 env
|
||||
MEMORY_DEDUP_THRESHOLD(默认 0.95),兼容直接调用本函数的旧代码;
|
||||
reader 总是传入 `recall.intention_dedup_threshold`。
|
||||
|
||||
Returns:
|
||||
[{"node_id", "score", "node", "layer": "l7_intention", "source": "vdb_intention"}, ...]
|
||||
@@ -272,7 +288,7 @@ async def recall_intentions(
|
||||
埋点字段(回填进 stats_out):
|
||||
耗时(ms): vdb_search_ms / expire_ms / backfill_embed_ms / dedup_ms / total_ms
|
||||
数量: fetched / expired / survivors / dedup_dropped / returned
|
||||
配置: fetch_limit / out_limit / dedup_threshold
|
||||
配置: fetch_limit / out_limit / dedup_enabled / dedup_threshold
|
||||
说明: expire_ms、backfill_embed_ms 只有真的发生了对应操作才非 0,
|
||||
借此可判断慢在 VDB 检索、逐条过期转换、补拉向量还是判重计算。
|
||||
"""
|
||||
@@ -358,7 +374,7 @@ async def recall_intentions(
|
||||
_t = time.perf_counter()
|
||||
_backfilled = 0
|
||||
_missing_cnt = 0
|
||||
if survivors:
|
||||
if survivors and dedup_enabled: # 补拉只为判重;关闭时无需这次 VDB 读
|
||||
missing_ids = [
|
||||
h.get("node_id") for h in survivors
|
||||
if h.get("node_id") and not getattr(h.get("node"), "embedding", None)
|
||||
@@ -391,18 +407,23 @@ async def recall_intentions(
|
||||
)
|
||||
|
||||
# 2. 判重:同 user + 同天 memory_at + valid_until 同日 + cosine≥threshold
|
||||
# dedup_enabled=False → 完全跳过:不隐藏、不删除。
|
||||
_t = time.perf_counter()
|
||||
threshold = _get_dedup_threshold()
|
||||
try:
|
||||
kept, delete_ids = _partition_duplicates(survivors, threshold)
|
||||
except Exception as e:
|
||||
# 任何 dedup 异常 → 降级为不去重(返回全部 survivors 截断 limit)
|
||||
logger.warning(f"[intention] dedup partition failed, degrade to raw: {e}")
|
||||
threshold = _get_dedup_threshold(dedup_threshold)
|
||||
if dedup_enabled:
|
||||
try:
|
||||
kept, delete_ids = _partition_duplicates(survivors, threshold)
|
||||
except Exception as e:
|
||||
# 任何 dedup 异常 → 降级为不去重(返回全部 survivors 截断 limit)
|
||||
logger.warning(f"[intention] dedup partition failed, degrade to raw: {e}")
|
||||
kept, delete_ids = survivors, []
|
||||
else:
|
||||
kept, delete_ids = survivors, []
|
||||
# 纯 CPU(cosine 两两比较);survivors 多时可能显著
|
||||
_stat(
|
||||
dedup_ms=round((time.perf_counter() - _t) * 1000, 2),
|
||||
dedup_dropped=len(delete_ids),
|
||||
dedup_enabled=bool(dedup_enabled),
|
||||
dedup_threshold=threshold,
|
||||
)
|
||||
|
||||
|
||||
@@ -1856,6 +1856,12 @@ class BaseReadPipeline(ReadPipeline):
|
||||
stats_out=_intent_stats,
|
||||
include_sources=request.include_sources,
|
||||
exclude_sources=request.exclude_sources,
|
||||
dedup_enabled=bool(getattr(
|
||||
self.config.recall, "intention_dedup_enabled", True
|
||||
)),
|
||||
dedup_threshold=getattr(
|
||||
self.config.recall, "intention_dedup_threshold", None
|
||||
),
|
||||
)
|
||||
await trace_log.log_intention(
|
||||
stats=_intent_stats,
|
||||
|
||||
@@ -635,6 +635,7 @@ class MemoryWriter(WritePipeline):
|
||||
不查 VDB、不做"新写入 vs 库里已有"的对比,避免给 VDB 增加读压力。
|
||||
库内存量污染由 recall 侧异步删除逐步清理。best-effort:任何异常都
|
||||
回退到"不判重、全写入",绝不影响 upsert 主流程。
|
||||
`recall.intention_dedup_enabled=False` 时跳过批内判重,全部写入。
|
||||
"""
|
||||
if not intentions:
|
||||
return []
|
||||
@@ -660,7 +661,11 @@ class MemoryWriter(WritePipeline):
|
||||
# ============ 批内 dedup(不查 VDB,只对本批 intentions 互斥判重) ============
|
||||
keep_mask: List[bool] = [True] * len(intentions)
|
||||
try:
|
||||
dropped = self._intention_batch_dedup(intentions, embeddings)
|
||||
dropped = (
|
||||
self._intention_batch_dedup(intentions, embeddings)
|
||||
if getattr(self.config.recall, "intention_dedup_enabled", True)
|
||||
else []
|
||||
)
|
||||
for i in dropped:
|
||||
if 0 <= i < len(keep_mask):
|
||||
keep_mask[i] = False
|
||||
|
||||
@@ -137,6 +137,15 @@ memhub profile
|
||||
`user-profile` / `memhub profile` 调用 SDK `get_user_profile()`,返回组装后的
|
||||
用户长期画像,不通过普通 Search 拼装。
|
||||
|
||||
读接口都回显 `user_id`(实际读的用户,未指定时为 `local`):
|
||||
`GET /api/v2/memory-library`、`/memory-library/layers`、`/user-profile`、
|
||||
`/user-profile/versions`、`/home-summary`、`/memories/graph`、`/raw`,
|
||||
`POST /api/v2/context/search`、`/api/v1/recall`、`/api/v2/memories/ask`;
|
||||
`/memories/ask/stream` 写在第一个 `recall` 事件上。
|
||||
`DELETE /api/v2/memories/{id}` 只在删掉调用者自己的记忆时回 204;另一个用户的
|
||||
记忆和不存在的 ID 一律回 404 `memory_not_found`(不暴露是否存在),调用方按
|
||||
「已不存在」处理。
|
||||
|
||||
返回值:
|
||||
|
||||
```python
|
||||
|
||||
+2
-2
@@ -18,7 +18,7 @@ If you run into a problem, please open an Issue with the version, platform,repro
|
||||
|
||||
## 发行与平台合同
|
||||
|
||||
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.11`。MemHub 源码版本为 `1.0.6`
|
||||
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.12`。MemHub 源码版本为 `1.0.6`
|
||||
(build 37),GitHub 仓库为
|
||||
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
|
||||
`memhub-cli`。当前代码架构见
|
||||
@@ -118,7 +118,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
|
||||
|
||||
## Hy Memory
|
||||
|
||||
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.11`。SDK 配置、权限边界及 search
|
||||
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.12`。SDK 配置、权限边界及 search
|
||||
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
|
||||
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
|
||||
|
||||
|
||||
@@ -2352,7 +2352,7 @@
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-hy-memory",
|
||||
"name": "hy-memory",
|
||||
"versionInfo": "2.0.11",
|
||||
"versionInfo": "2.0.12",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
"filesAnalyzed": false,
|
||||
"licenseConcluded": "Apache-2.0",
|
||||
|
||||
@@ -9,7 +9,7 @@ Per-package license texts are not reproduced here yet; the license
|
||||
column is filled in as part of release review, and the SBOM records
|
||||
`NOASSERTION` until then rather than guessing.
|
||||
|
||||
## hy-memory 2.0.11
|
||||
## hy-memory 2.0.12
|
||||
|
||||
License: Apache-2.0
|
||||
|
||||
|
||||
+29
-13
@@ -1938,7 +1938,7 @@ async function runCommand(
|
||||
return reject(
|
||||
descriptor,
|
||||
asJson,
|
||||
"--user-id 和 --root 只能用于接入单个 hermes 来源(Hermes profile),用户 ID 只能是 1–64 位字母、数字、点、下划线或连字符。",
|
||||
"--user-id 和 --root 只能用于接入单个 hermes 来源(Hermes profile),用户 ID 只能是 1–64 位字母、数字、点、下划线或连字符。用户 ID 不必等于 Hermes profile 名,例如 --user-id wife-work --root ~/.hermes/profiles/wife:work。",
|
||||
);
|
||||
}
|
||||
if (!parsed) return usage();
|
||||
@@ -2328,29 +2328,41 @@ async function runCommand(
|
||||
...userIdParameters(userId),
|
||||
}),
|
||||
);
|
||||
// A delete answered 404 `memory_not_found` is already gone (deleted
|
||||
// before, or never this user's): what was asked for holds, so it exits 0
|
||||
// as committed and says so, instead of failing a retry or a double run.
|
||||
const alreadyGone =
|
||||
parsed.kind === "delete" &&
|
||||
outcome.state === "rejected" &&
|
||||
outcome.code === "memory_not_found";
|
||||
const state = alreadyGone ? "committed" : outcome.state;
|
||||
const document = {
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
command: `memories ${parsed.kind}`,
|
||||
state: outcome.state,
|
||||
state,
|
||||
operation: parsed.kind,
|
||||
memory_id: parsed.memoryId,
|
||||
...(outcome.state === "committed"
|
||||
? parsed.kind === "update"
|
||||
? { result: outcome.data }
|
||||
: {}
|
||||
: { code: outcome.code }),
|
||||
...(alreadyGone
|
||||
? { already_gone: true }
|
||||
: outcome.state === "committed"
|
||||
? parsed.kind === "update"
|
||||
? { result: outcome.data }
|
||||
: {}
|
||||
: { code: outcome.code }),
|
||||
};
|
||||
emit(
|
||||
asJson
|
||||
? `${JSON.stringify(document)}\n`
|
||||
: outcome.state === "committed"
|
||||
? `已${parsed.kind === "update" ? "更新" : "删除"}记忆 ${parsed.memoryId}\n`
|
||||
: outcome.code === "memory_id_ambiguous"
|
||||
? "这个 ID 同时存在于多个场景;请加 --kind(以及 --variant)指明是哪一条。\n"
|
||||
: `操作失败:${outcome.code}\n`,
|
||||
: alreadyGone
|
||||
? `未找到记忆 ${parsed.memoryId}(可能已删除),无需再删。\n`
|
||||
: outcome.state === "committed"
|
||||
? `已${parsed.kind === "update" ? "更新" : "删除"}记忆 ${parsed.memoryId}\n`
|
||||
: outcome.code === "memory_id_ambiguous"
|
||||
? "这个 ID 同时存在于多个场景;请加 --kind(以及 --variant)指明是哪一条。\n"
|
||||
: `操作失败:${outcome.code}\n`,
|
||||
);
|
||||
return outcome.state === "committed" ? EXIT_OK : EXIT_FAILED;
|
||||
return state === "committed" ? EXIT_OK : EXIT_FAILED;
|
||||
}
|
||||
if (
|
||||
named === "settings" &&
|
||||
@@ -4371,6 +4383,10 @@ function renderWriteReceiptText(receipt: MemoryWriteReceipt): string {
|
||||
...(receipt.memory_ids && receipt.memory_ids.length > 0
|
||||
? [`memory_ids: ${receipt.memory_ids.join(",")}`]
|
||||
: []),
|
||||
...(receipt.error_code ? [`error_code: ${receipt.error_code}`] : []),
|
||||
...(receipt.next_retry_at
|
||||
? [`next_retry_at: ${receipt.next_retry_at}`]
|
||||
: []),
|
||||
"",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
@@ -258,7 +258,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
{
|
||||
name: "sources",
|
||||
summary:
|
||||
"列出来源,直接接入来源,或按 request id 查接入回执。connect 不加 --import 只授权(处理保持冻结:不导入历史、不扫描新会话);加 --import 同时导入历史并开启持续同步,返回 request id。目录来自环境变量覆盖的来源必须加 --confirm-overrides。Hermes 的每个 profile 是一个独立用户:用 --user-id 和 --root 接入,授权并导入历史(会自动排除 ~/.hermes/profiles/**,各用户互不读取)。",
|
||||
"列出来源,直接接入来源,或按 request id 查接入回执。connect 不加 --import 只授权(处理保持冻结:不导入历史、不扫描新会话);加 --import 同时导入历史并开启持续同步,返回 request id。目录来自环境变量覆盖的来源必须加 --confirm-overrides。Hermes 的每个 profile 是一个独立用户:用 --user-id 和 --root 接入,授权并导入历史(任何层级的 profiles/ 目录都会被排除,各用户互不读取)。--root 不能是 profiles 目录本身,也不能与另一个用户的 Hermes 目录相同或互相包含(~/.hermes 与 ~/.hermes/profiles/<name> 除外)。",
|
||||
usage:
|
||||
"memhub sources list [--json]\n memhub sources connect <id> [<id> ...] [--import] [--confirm-overrides] [--request-id <uuid>] [--json]\n memhub sources connect hermes --user-id <id> --root <path> [--import] [--request-id <uuid>] [--json]\n memhub sources connect --all-detected [--import] [--confirm-overrides] [--json]\n memhub sources receipt <request-id> [--json]",
|
||||
options: [
|
||||
@@ -270,7 +270,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
{
|
||||
flag: "--user-id <id>",
|
||||
summary:
|
||||
"只用于 hermes:这条接入属于哪个用户(默认 local)。和 --root 一起接入一个 Hermes profile,例如 --user-id wife --root ~/.hermes/profiles/wife;其他来源只能是 local。",
|
||||
"只用于 hermes:这条接入属于哪个用户(默认 local)。和 --root 一起接入一个 Hermes profile,例如 --user-id wife --root ~/.hermes/profiles/wife;用户 ID 不必等于 profile 名,例如 --user-id wife-work --root ~/.hermes/profiles/wife:work。其他来源只能是 local。",
|
||||
},
|
||||
{
|
||||
flag: "--root <path>",
|
||||
@@ -397,7 +397,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
{
|
||||
name: "memories",
|
||||
summary:
|
||||
"列出、搜索、查看、添加、修改或物理删除记忆(对话、工作、编码三个场景)。add 和 App 的「添加记忆」是同一个入口;delete 不可逆,需 --yes。",
|
||||
"列出、搜索、查看、添加、修改或物理删除记忆(对话、工作、编码三个场景)。add 和 App 的「添加记忆」是同一个入口;delete 不可逆,需 --yes;要删的记忆已不存在(或不属于该用户)时提示未找到,按已删除处理并退出 0。",
|
||||
usage:
|
||||
"memhub memories list [--scene <all|chat|working|coding>] [--source <id>] [--query <text>] [--status <state>] [--offset <n>] [--limit <n>] [--user-id <id>] [--json]\n memhub memories show <memory-id> [--kind <kind>] [--variant <variant>] [--user-id <id>] [--json]\n memhub memories add <text> [--request-id <uuid>] [--user-id <id>] [--json]\n memhub memories add --content-file <path> [--request-id <uuid>] [--user-id <id>] [--json]\n memhub memories add-status <request-id> [--user-id <id>] [--json]\n memhub memories update <memory-id> --content-file <path> [--kind <kind>] [--variant <variant>] [--task <text>] [--user-id <id>] [--json]\n memhub memories delete <memory-id> [--kind <kind>] [--variant <variant>] [--yes] [--user-id <id>] [--json]",
|
||||
options: [
|
||||
@@ -461,11 +461,12 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
{
|
||||
name: "settings",
|
||||
summary:
|
||||
"查看或切换记忆模式(lite / pro / ultra)。lite 不调用模型,只在原始对话里检索;pro 由模型提取记忆(System 1);ultra 在 pro 之上每晚自动进化(System 2,使用知识图谱)。任何模式都保留全部原始对话。切换不迁移数据:已处理的不会重做,排队中的和新的按新模式处理;需 --yes,不加时只显示计划。环境变量 ONE_MEMORY_GRAPH_STORE 会强制 ultra。settings features 查看或开关 SDK 的可选检索与提取功能(全局设置,默认全部保持 SDK 默认值):graph_rerank 图谱重排(仅 ultra)、same_series_timeline 同系列时间线(需 graph_rerank)、episode_recall 片段召回(pro / ultra,需先开 session_episodes 写入片段)、session_episodes 片段写入、episode_summary 片段摘要(每个片段一次模型调用)、intention 意图提取(默认开)、few_shot 提取示例;set 需 --yes,不加时只显示计划;reset 恢复默认值。settings language 查看或设置界面语言(App 切换语言时会自动设置):简体中文(zh-Hans)和 system 不指定 SDK 语言,SDK 照旧自动识别中文或英文;其余语言(en、zh-Hant、ja、es、de、pt、fr、ko、ru、id)会让提取等 SDK 提示使用该语言,立即生效。",
|
||||
"查看或切换记忆模式(lite / pro / ultra)。lite 不调用模型,只在原始对话里检索;pro 由模型提取记忆(System 1);ultra 在 pro 之上每晚自动进化(System 2,使用知识图谱)。任何模式都保留全部原始对话。切换不迁移数据:已处理的不会重做,排队中的和新的按新模式处理;需 --yes,不加时只显示计划。环境变量 ONE_MEMORY_GRAPH_STORE 会强制 ultra。settings features 查看或开关 SDK 的可选检索与提取功能(全局设置,默认全部保持 SDK 默认值):graph_rerank 图谱重排(仅 ultra)、same_series_timeline 同系列时间线(需 graph_rerank)、episode_recall 片段召回(pro / ultra,需先开 session_episodes 写入片段)、session_episodes 片段写入、episode_summary 片段摘要(每个片段一次模型调用)、intention 意图提取(默认开)、intention_dedup 意图判重(默认开;关闭则检索与写入都不判重、不删除意图)、few_shot 提取示例,以及数值 intention_dedup_threshold 意图判重阈值(默认 0.95,可设 0.8–1.0);set 需 --yes,不加时只显示计划;reset 恢复默认值。settings language 查看或设置界面语言(App 切换语言时会自动设置):简体中文(zh-Hans)和 system 不指定 SDK 语言,SDK 照旧自动识别中文或英文;其余语言(en、zh-Hant、ja、es、de、pt、fr、ko、ru、id)会让提取等 SDK 提示使用该语言,立即生效。",
|
||||
usage:
|
||||
"memhub settings mode [lite|pro|ultra] [--yes] [--json]\n" +
|
||||
" memhub settings features [--json]\n" +
|
||||
" memhub settings features set <name>=<true|false> [...] [--yes] [--json]\n" +
|
||||
" memhub settings features set intention_dedup_threshold=<0.8-1.0> [--yes] [--json]\n" +
|
||||
" memhub settings features reset [<name> ...] [--json]\n" +
|
||||
" memhub settings language [<code>|system] [--json]",
|
||||
options: [
|
||||
@@ -476,7 +477,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
{
|
||||
flag: "features set <name>=<true|false>",
|
||||
summary:
|
||||
"开关一个或多个 SDK 功能;功能在当前模式下不生效时会注明原因(例如 graph_rerank 需要 ultra)。",
|
||||
"开关一个或多个 SDK 功能;功能在当前模式下不生效时会注明原因(例如 graph_rerank 需要 ultra)。intention_dedup_threshold 取 0.8–1.0 的数值。",
|
||||
},
|
||||
{
|
||||
flag: "features reset [<name> ...]",
|
||||
|
||||
@@ -268,6 +268,11 @@ export interface MemoryWriteReceipt {
|
||||
detail?: string;
|
||||
raw_ids?: string[];
|
||||
memory_ids?: string[];
|
||||
/** Stable code of the last failure while the write still waits; null when
|
||||
* none. Optional so older Runtimes stay readable. */
|
||||
error_code?: string | null;
|
||||
/** When Core will try this write again (ISO); null when not scheduled. */
|
||||
next_retry_at?: string | null;
|
||||
}
|
||||
|
||||
export interface OpenAIMessage {
|
||||
|
||||
@@ -348,12 +348,25 @@ export function parseApiKey(bytes: Buffer | null): string | undefined {
|
||||
return value;
|
||||
}
|
||||
|
||||
/** What to do about a refusal whose code alone does not say. */
|
||||
const FLOW_CODE_HINTS: Record<string, string> = {
|
||||
hermes_root_is_profiles_container:
|
||||
"--root 指向的是 profiles 目录本身;请指向其中一个 profile,例如 ~/.hermes/profiles/wife。",
|
||||
hermes_root_overlaps_other_user:
|
||||
"--root 与另一个用户已接入的 Hermes 目录相同或互相包含,会读到对方的记录;请换一个目录,或先撤销对方的接入。~/.hermes 与 ~/.hermes/profiles/<name> 可以分属两个用户。",
|
||||
};
|
||||
|
||||
export function renderFlowText(document: FlowDocument): string {
|
||||
const lines = [
|
||||
`操作状态:${document.state}`,
|
||||
`操作 ID:${document.operation_id ?? "无"}`,
|
||||
"code" in document ? `代码:${document.code}` : "已返回有界回执。",
|
||||
];
|
||||
const hint =
|
||||
"code" in document && typeof document.code === "string"
|
||||
? FLOW_CODE_HINTS[document.code]
|
||||
: undefined;
|
||||
if (hint) lines.push(hint);
|
||||
// A partial success or an all-rejected batch must name each source that was
|
||||
// not accepted; a bare "partial" leaves the user with nothing to act on.
|
||||
if (
|
||||
|
||||
@@ -233,6 +233,31 @@ describe("memories add-status", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("passes the retry code and time of an addition still waiting", async () => {
|
||||
transport.get.mockResolvedValue({
|
||||
status: 200,
|
||||
data: {
|
||||
...addReceipt("pending"),
|
||||
error_code: "model_auth_failed",
|
||||
next_retry_at: "2026-11-02T10:00:00+00:00",
|
||||
},
|
||||
});
|
||||
const json = await capturedRun([
|
||||
"memories",
|
||||
"add-status",
|
||||
REQUEST,
|
||||
"--json",
|
||||
]);
|
||||
expect(JSON.parse(json.stdout).receipt).toMatchObject({
|
||||
state: "pending",
|
||||
error_code: "model_auth_failed",
|
||||
next_retry_at: "2026-11-02T10:00:00+00:00",
|
||||
});
|
||||
const text = await capturedRun(["memories", "add-status", REQUEST]);
|
||||
expect(text.stdout).toContain("error_code: model_auth_failed");
|
||||
expect(text.stdout).toContain("next_retry_at: 2026-11-02T10:00:00+00:00");
|
||||
});
|
||||
|
||||
it("reports an unknown request id as not_found, exit 1", async () => {
|
||||
transport.get.mockResolvedValue({
|
||||
status: 404,
|
||||
@@ -366,6 +391,33 @@ describe("memories update / delete pass the namespace", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("reports a delete answered 404 as already gone and exits 0", async () => {
|
||||
transport.delete.mockResolvedValue({
|
||||
status: 404,
|
||||
data: { detail: "memory_not_found" },
|
||||
});
|
||||
const json = await capturedRun([
|
||||
"memories",
|
||||
"delete",
|
||||
"gone-1",
|
||||
"--yes",
|
||||
"--json",
|
||||
]);
|
||||
expect(json.code).toBe(0);
|
||||
expect(JSON.parse(json.stdout)).toEqual({
|
||||
schema_version: 2,
|
||||
product: expect.any(String),
|
||||
command: "memories delete",
|
||||
state: "committed",
|
||||
operation: "delete",
|
||||
memory_id: "gone-1",
|
||||
already_gone: true,
|
||||
});
|
||||
const text = await capturedRun(["memories", "delete", "gone-1", "--yes"]);
|
||||
expect(text.code).toBe(0);
|
||||
expect(text.stdout).toContain("未找到记忆 gone-1");
|
||||
});
|
||||
|
||||
it("explains an ambiguous id instead of a bare code", async () => {
|
||||
transport.delete.mockResolvedValue({
|
||||
status: 409,
|
||||
|
||||
@@ -36,15 +36,17 @@ const DEFAULTS: Record<string, boolean> = {
|
||||
session_episodes: false,
|
||||
episode_summary: false,
|
||||
intention: true,
|
||||
intention_dedup: true,
|
||||
few_shot: false,
|
||||
};
|
||||
const THRESHOLD_DEFAULT = 0.95;
|
||||
|
||||
function featuresState(
|
||||
configured: Record<string, boolean> = {},
|
||||
configured: Record<string, boolean | number> = {},
|
||||
mode = "pro",
|
||||
extra: Record<string, unknown> = {},
|
||||
) {
|
||||
const nonDefault: Record<string, boolean> = {};
|
||||
const nonDefault: Record<string, boolean | number> = {};
|
||||
const features = Object.entries(DEFAULTS).map(([name, fallback]) => {
|
||||
const enabled = configured[name] ?? fallback;
|
||||
if (enabled !== fallback) nonDefault[name] = enabled;
|
||||
@@ -70,7 +72,34 @@ function featuresState(
|
||||
description: name,
|
||||
};
|
||||
});
|
||||
return { mode, features, non_default: nonDefault, ...extra };
|
||||
const threshold =
|
||||
(configured.intention_dedup_threshold as number | undefined) ??
|
||||
THRESHOLD_DEFAULT;
|
||||
if (threshold !== THRESHOLD_DEFAULT)
|
||||
nonDefault.intention_dedup_threshold = threshold;
|
||||
const dedupOn =
|
||||
(configured.intention_dedup as boolean | undefined) ??
|
||||
DEFAULTS.intention_dedup;
|
||||
const parameters = [
|
||||
{
|
||||
name: "intention_dedup_threshold",
|
||||
value: threshold,
|
||||
configured:
|
||||
"intention_dedup_threshold" in configured
|
||||
? configured.intention_dedup_threshold
|
||||
: null,
|
||||
default: THRESHOLD_DEFAULT,
|
||||
minimum: 0.8,
|
||||
maximum: 1,
|
||||
active: dedupOn,
|
||||
reason: dedupOn ? null : "requires_intention_dedup",
|
||||
requires_feature: "intention_dedup",
|
||||
sdk_field: "recall.intention_dedup_threshold",
|
||||
env: "MEMORY_INTENTION_DEDUP_THRESHOLD",
|
||||
description: "threshold",
|
||||
},
|
||||
];
|
||||
return { mode, features, parameters, non_default: nonDefault, ...extra };
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
@@ -110,11 +139,16 @@ describe("memhub settings features", () => {
|
||||
expect(result.code).toBe(0);
|
||||
const json = JSON.parse(result.stdout);
|
||||
expect(json.state).toBe("ok");
|
||||
expect(json.features.features).toHaveLength(7);
|
||||
expect(json.features.features).toHaveLength(8);
|
||||
expect(json.features.parameters[0].name).toBe("intention_dedup_threshold");
|
||||
expect(transport.put).not.toHaveBeenCalled();
|
||||
const text = (await capturedRun(["settings", "features"])).stdout;
|
||||
expect(text).toContain("graph_rerank:关(默认)");
|
||||
expect(text).toContain("intention:开(默认)");
|
||||
expect(text).toContain("intention_dedup:开(默认)");
|
||||
expect(text).toContain(
|
||||
"intention_dedup_threshold:0.95(默认,可设 0.8–1)",
|
||||
);
|
||||
expect(text).toContain("先开启 session_episodes");
|
||||
});
|
||||
|
||||
@@ -160,6 +194,130 @@ describe("memhub settings features", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("turns intention dedup off and sets its threshold, round trip", async () => {
|
||||
transport.put.mockResolvedValueOnce({
|
||||
status: 200,
|
||||
data: featuresState({ intention_dedup: false }, "pro", {
|
||||
changed: ["intention_dedup"],
|
||||
}),
|
||||
});
|
||||
const off = await capturedRun([
|
||||
"settings",
|
||||
"features",
|
||||
"set",
|
||||
"intention_dedup=false",
|
||||
"--yes",
|
||||
"--json",
|
||||
]);
|
||||
expect(off.code).toBe(0);
|
||||
expect(transport.put).toHaveBeenLastCalledWith(
|
||||
"/api/v1/settings/memory-features",
|
||||
{ intention_dedup: false },
|
||||
undefined,
|
||||
);
|
||||
const offJson = JSON.parse(off.stdout);
|
||||
expect(offJson.state).toBe("committed");
|
||||
expect(offJson.changed).toEqual(["intention_dedup"]);
|
||||
expect(offJson.features.parameters[0].reason).toBe(
|
||||
"requires_intention_dedup",
|
||||
);
|
||||
|
||||
transport.get.mockResolvedValue({
|
||||
status: 200,
|
||||
data: featuresState({ intention_dedup: false }),
|
||||
});
|
||||
transport.put.mockResolvedValueOnce({
|
||||
status: 200,
|
||||
data: featuresState(
|
||||
{ intention_dedup: true, intention_dedup_threshold: 0.9 },
|
||||
"pro",
|
||||
{ changed: ["intention_dedup", "intention_dedup_threshold"] },
|
||||
),
|
||||
});
|
||||
const on = await capturedRun([
|
||||
"settings",
|
||||
"features",
|
||||
"set",
|
||||
"intention_dedup=true",
|
||||
"intention_dedup_threshold=0.9",
|
||||
"--yes",
|
||||
]);
|
||||
expect(on.code).toBe(0);
|
||||
expect(transport.put).toHaveBeenLastCalledWith(
|
||||
"/api/v1/settings/memory-features",
|
||||
{ intention_dedup: true, intention_dedup_threshold: 0.9 },
|
||||
undefined,
|
||||
);
|
||||
expect(on.stdout).toContain(
|
||||
"已更新:intention_dedup、intention_dedup_threshold",
|
||||
);
|
||||
expect(on.stdout).toContain("intention_dedup_threshold:0.9(已设置");
|
||||
|
||||
transport.get.mockResolvedValue({
|
||||
status: 200,
|
||||
data: featuresState({ intention_dedup_threshold: 0.9 }),
|
||||
});
|
||||
transport.put.mockResolvedValueOnce({
|
||||
status: 200,
|
||||
data: featuresState({}, "pro", {
|
||||
changed: ["intention_dedup_threshold"],
|
||||
}),
|
||||
});
|
||||
const reset = await capturedRun([
|
||||
"settings",
|
||||
"features",
|
||||
"reset",
|
||||
"intention_dedup_threshold",
|
||||
]);
|
||||
expect(reset.code).toBe(0);
|
||||
expect(transport.put).toHaveBeenLastCalledWith(
|
||||
"/api/v1/settings/memory-features",
|
||||
{ intention_dedup_threshold: null },
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
it("previews a threshold without --yes and refuses one out of range", async () => {
|
||||
const preview = await capturedRun([
|
||||
"settings",
|
||||
"features",
|
||||
"set",
|
||||
"intention_dedup_threshold=0.85",
|
||||
]);
|
||||
expect(preview.code).toBe(0);
|
||||
expect(preview.stdout).toContain("- intention_dedup_threshold = 0.85");
|
||||
for (const value of ["0.5", "0.79", "1.01", "2", "-0.9", "abc", "1e0"]) {
|
||||
const result = await capturedRun([
|
||||
"settings",
|
||||
"features",
|
||||
"set",
|
||||
`intention_dedup_threshold=${value}`,
|
||||
"--yes",
|
||||
]);
|
||||
expect(result.code, value).toBe(2);
|
||||
}
|
||||
expect(transport.put).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not send a threshold to a Runtime that does not list it", async () => {
|
||||
const { parameters: _unused, ...older } = featuresState();
|
||||
transport.get.mockResolvedValue({ status: 200, data: older });
|
||||
const result = await capturedRun([
|
||||
"settings",
|
||||
"features",
|
||||
"set",
|
||||
"intention_dedup_threshold=0.9",
|
||||
"--yes",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
state: "rejected",
|
||||
code: "memory_feature_unsupported",
|
||||
});
|
||||
expect(transport.put).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does nothing when the values are already set", async () => {
|
||||
transport.get.mockResolvedValue({
|
||||
status: 200,
|
||||
@@ -210,6 +368,8 @@ describe("memhub settings features", () => {
|
||||
["settings", "features", "set", "graph_rerank=yes"],
|
||||
["settings", "features", "set", "graph_rerank"],
|
||||
["settings", "features", "set", "few_shot=true", "few_shot=false"],
|
||||
["settings", "features", "set", "intention_dedup=0.9"],
|
||||
["settings", "features", "set", "intention_dedup_threshold=true"],
|
||||
["settings", "features", "reset", "nope"],
|
||||
["settings", "features", "toggle"],
|
||||
]) {
|
||||
@@ -269,7 +429,11 @@ describe("status pipeline features", () => {
|
||||
last_evolve_at: null,
|
||||
graph_build: { state: "idle", users: {} },
|
||||
evolve: { mode: "pro", schedule: null, running: [], users: {} },
|
||||
features: { episode_recall: true, intention: false },
|
||||
features: {
|
||||
episode_recall: true,
|
||||
intention: false,
|
||||
intention_dedup_threshold: 0.9,
|
||||
},
|
||||
},
|
||||
};
|
||||
const report = await composeStatusReport({
|
||||
@@ -284,9 +448,12 @@ describe("status pipeline features", () => {
|
||||
expect(report.pipeline?.features).toEqual({
|
||||
episode_recall: true,
|
||||
intention: false,
|
||||
intention_dedup_threshold: 0.9,
|
||||
});
|
||||
const text = renderStatusText(report, "MemHub");
|
||||
expect(text).toContain("SDK 功能:episode_recall=开、intention=关");
|
||||
expect(text).toContain(
|
||||
"SDK 功能:episode_recall=开、intention=关、intention_dedup_threshold=0.9",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -2,7 +2,8 @@
|
||||
* `memhub settings features [set <name>=<true|false> ... | reset [<name> ...]]`.
|
||||
*
|
||||
* The SDK's optional retrieval and extraction features (graph rerank, episode
|
||||
* recall, the episode write side, intention, few-shot). One global setting in
|
||||
* recall, the episode write side, intention and its dedup, few-shot), plus the
|
||||
* intention dedup threshold, the one numeric setting. One global setting in
|
||||
* the Runtime; an unset feature keeps the SDK default, so an install nobody
|
||||
* touched behaves as before. `set` prints its plan first and needs `--yes`,
|
||||
* like `settings mode`, because some features spend model calls; `reset`
|
||||
@@ -20,10 +21,19 @@ export const MEMORY_FEATURES = [
|
||||
"session_episodes",
|
||||
"episode_summary",
|
||||
"intention",
|
||||
"intention_dedup",
|
||||
"few_shot",
|
||||
] as const;
|
||||
export type MemoryFeature = (typeof MEMORY_FEATURES)[number];
|
||||
|
||||
/** Numeric settings, with the range the Runtime accepts (inclusive). */
|
||||
export const MEMORY_PARAMETERS = {
|
||||
intention_dedup_threshold: { minimum: 0.8, maximum: 1 },
|
||||
} as const;
|
||||
export type MemoryParameter = keyof typeof MEMORY_PARAMETERS;
|
||||
export type MemorySetting = MemoryFeature | MemoryParameter;
|
||||
type SettingValues = Partial<Record<MemorySetting, boolean | number>>;
|
||||
|
||||
const FEATURE_LABEL: Record<MemoryFeature, string> = {
|
||||
graph_rerank: "图谱重排(沿知识图谱的边传播支持度;仅 ultra)",
|
||||
same_series_timeline:
|
||||
@@ -33,6 +43,8 @@ const FEATURE_LABEL: Record<MemoryFeature, string> = {
|
||||
session_episodes: "片段写入(同一会话提取的记忆归入同一片段)",
|
||||
episode_summary: "片段摘要(片段结束时总结一次;每个片段一次模型调用)",
|
||||
intention: "意图提取(随事实一起提取用户的计划与意图)",
|
||||
intention_dedup:
|
||||
"意图判重(检索时把同一天、相似度达到阈值的意图视为重复,只返回最早一条并删除其余;关闭则不判重、不删除)",
|
||||
few_shot: "提取示例(在提取提示里附上示例;不加调用,输入更长)",
|
||||
};
|
||||
|
||||
@@ -41,21 +53,42 @@ const REASON_TEXT: Record<string, string> = {
|
||||
requires_pro: "lite 模式下不生效(需要 pro 或 ultra)",
|
||||
requires_ultra: "当前模式下不生效(需要 ultra)",
|
||||
requires_graph_rerank: "需要先开启并生效 graph_rerank",
|
||||
requires_intention_dedup: "intention_dedup 已关闭,阈值不起作用",
|
||||
};
|
||||
|
||||
const PARAMETER_LABEL: Record<MemoryParameter, string> = {
|
||||
intention_dedup_threshold: "意图判重阈值(余弦相似度,越高保留越多)",
|
||||
};
|
||||
|
||||
export type FeaturesCommand =
|
||||
| { kind: "show" }
|
||||
| {
|
||||
kind: "set";
|
||||
values: Partial<Record<MemoryFeature, boolean>>;
|
||||
values: SettingValues;
|
||||
confirmed: boolean;
|
||||
}
|
||||
| { kind: "reset"; names: MemoryFeature[] };
|
||||
| { kind: "reset"; names: MemorySetting[] };
|
||||
|
||||
function isFeature(value: string): value is MemoryFeature {
|
||||
return (MEMORY_FEATURES as readonly string[]).includes(value);
|
||||
}
|
||||
|
||||
function isParameter(value: string): value is MemoryParameter {
|
||||
return Object.hasOwn(MEMORY_PARAMETERS, value);
|
||||
}
|
||||
|
||||
function isSetting(value: string): value is MemorySetting {
|
||||
return isFeature(value) || isParameter(value);
|
||||
}
|
||||
|
||||
/** A parameter's value: a plain decimal in its range; null = usage. */
|
||||
function parameterValue(name: MemoryParameter, raw: string): number | null {
|
||||
if (!/^\d+(\.\d+)?$/.test(raw)) return null;
|
||||
const value = Number(raw);
|
||||
const { minimum, maximum } = MEMORY_PARAMETERS[name];
|
||||
return value >= minimum && value <= maximum ? value : null;
|
||||
}
|
||||
|
||||
/** Arguments after `settings features`; null = usage. */
|
||||
export function parseFeaturesCommand(
|
||||
values: readonly string[],
|
||||
@@ -68,18 +101,28 @@ export function parseFeaturesCommand(
|
||||
const positional = tail.filter((value) => value !== "--yes");
|
||||
if (verb === "set") {
|
||||
if (positional.length === 0) return null;
|
||||
const parsed: Partial<Record<MemoryFeature, boolean>> = {};
|
||||
const parsed: SettingValues = {};
|
||||
for (const item of positional) {
|
||||
const match = /^([a-z_]+)=(true|false)$/.exec(item);
|
||||
if (!match || !isFeature(match[1]) || match[1] in parsed) return null;
|
||||
parsed[match[1]] = match[2] === "true";
|
||||
const match = /^([a-z_]+)=(.+)$/.exec(item);
|
||||
if (!match || match[1] in parsed) return null;
|
||||
const [, name, raw] = match;
|
||||
if (isFeature(name)) {
|
||||
if (raw !== "true" && raw !== "false") return null;
|
||||
parsed[name] = raw === "true";
|
||||
} else if (isParameter(name)) {
|
||||
const value = parameterValue(name, raw);
|
||||
if (value === null) return null;
|
||||
parsed[name] = value;
|
||||
} else {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
return { kind: "set", values: parsed, confirmed: yes === 1 };
|
||||
}
|
||||
if (verb === "reset") {
|
||||
if (new Set(positional).size !== positional.length) return null;
|
||||
if (!positional.every(isFeature)) return null;
|
||||
return { kind: "reset", names: positional as MemoryFeature[] };
|
||||
if (!positional.every(isSetting)) return null;
|
||||
return { kind: "reset", names: positional as MemorySetting[] };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@@ -96,11 +139,26 @@ export interface FeatureState {
|
||||
description: string;
|
||||
}
|
||||
|
||||
export interface ParameterState {
|
||||
name: MemoryParameter;
|
||||
value: number;
|
||||
configured: number | null;
|
||||
default: number;
|
||||
minimum: number;
|
||||
maximum: number;
|
||||
active: boolean;
|
||||
reason: string | null;
|
||||
requires_feature: string;
|
||||
description: string;
|
||||
}
|
||||
|
||||
export interface FeaturesState {
|
||||
mode: string;
|
||||
features: FeatureState[];
|
||||
non_default: Partial<Record<MemoryFeature, boolean>>;
|
||||
changed?: MemoryFeature[];
|
||||
/** Absent from a Runtime older than the intention dedup threshold. */
|
||||
parameters?: ParameterState[];
|
||||
non_default: SettingValues;
|
||||
changed?: MemorySetting[];
|
||||
}
|
||||
|
||||
function featuresState(value: unknown): FeaturesState | null {
|
||||
@@ -125,6 +183,20 @@ function featuresState(value: unknown): FeaturesState | null {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
if (item.parameters !== undefined) {
|
||||
if (!Array.isArray(item.parameters)) return null;
|
||||
for (const parameter of item.parameters) {
|
||||
if (
|
||||
!parameter ||
|
||||
typeof parameter !== "object" ||
|
||||
typeof (parameter as ParameterState).name !== "string" ||
|
||||
typeof (parameter as ParameterState).value !== "number" ||
|
||||
typeof (parameter as ParameterState).active !== "boolean"
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
return item as unknown as FeaturesState;
|
||||
}
|
||||
|
||||
@@ -137,12 +209,12 @@ export type FeaturesDocument = {
|
||||
| {
|
||||
state: "preview";
|
||||
features: FeaturesState;
|
||||
target: Partial<Record<MemoryFeature, boolean>>;
|
||||
target: SettingValues;
|
||||
applied: false;
|
||||
next_step: string;
|
||||
}
|
||||
| { state: "unchanged"; features: FeaturesState }
|
||||
| { state: "committed"; features: FeaturesState; changed: MemoryFeature[] }
|
||||
| { state: "committed"; features: FeaturesState; changed: MemorySetting[] }
|
||||
| { state: "rejected" | "unknown" | "unavailable"; code: string }
|
||||
);
|
||||
|
||||
@@ -170,13 +242,23 @@ export async function runFeaturesCommand(
|
||||
if (command.kind === "show") {
|
||||
return { ...BASE, state: "ok", features: current };
|
||||
}
|
||||
// What a person set, per name this Runtime lists; null = the default.
|
||||
const configured = new Map<string, boolean | number | null>([
|
||||
...current.features.map(
|
||||
(feature) => [feature.name, feature.configured] as const,
|
||||
),
|
||||
...(current.parameters ?? []).map(
|
||||
(parameter) => [parameter.name, parameter.configured] as const,
|
||||
),
|
||||
]);
|
||||
let body: Record<string, unknown>;
|
||||
if (command.kind === "set") {
|
||||
const configured = new Map(
|
||||
current.features.map((feature) => [feature.name, feature.configured]),
|
||||
);
|
||||
if (Object.keys(command.values).some((name) => !configured.has(name))) {
|
||||
// An older Runtime does not know the name; it would answer 422.
|
||||
return { ...BASE, state: "rejected", code: "memory_feature_unsupported" };
|
||||
}
|
||||
const differs = Object.entries(command.values).some(
|
||||
([name, value]) => configured.get(name as MemoryFeature) !== value,
|
||||
([name, value]) => configured.get(name) !== value,
|
||||
);
|
||||
if (!differs) return { ...BASE, state: "unchanged", features: current };
|
||||
if (!command.confirmed) {
|
||||
@@ -197,16 +279,10 @@ export async function runFeaturesCommand(
|
||||
const names =
|
||||
command.names.length > 0
|
||||
? command.names
|
||||
: current.features
|
||||
.filter((feature) => feature.configured !== null)
|
||||
.map((feature) => feature.name);
|
||||
if (
|
||||
names.every(
|
||||
(name) =>
|
||||
current.features.find((feature) => feature.name === name)
|
||||
?.configured === null,
|
||||
)
|
||||
) {
|
||||
: [...configured]
|
||||
.filter(([, value]) => value !== null)
|
||||
.map(([name]) => name as MemorySetting);
|
||||
if (names.every((name) => (configured.get(name) ?? null) === null)) {
|
||||
return { ...BASE, state: "unchanged", features: current };
|
||||
}
|
||||
body = Object.fromEntries(names.map((name) => [name, null]));
|
||||
@@ -233,6 +309,16 @@ export function featuresExitCode(document: FeaturesDocument): number {
|
||||
: 1;
|
||||
}
|
||||
|
||||
function parameterLine(parameter: ParameterState): string {
|
||||
const label = PARAMETER_LABEL[parameter.name] ?? parameter.name;
|
||||
const origin = parameter.configured === null ? "默认" : "已设置";
|
||||
const activity =
|
||||
!parameter.active && parameter.reason
|
||||
? `,${REASON_TEXT[parameter.reason] ?? parameter.reason}`
|
||||
: "";
|
||||
return `${parameter.name}:${parameter.value}(${origin}${activity},可设 ${parameter.minimum}–${parameter.maximum})— ${label}`;
|
||||
}
|
||||
|
||||
function featureLine(feature: FeatureState): string {
|
||||
const label = FEATURE_LABEL[feature.name] ?? feature.name;
|
||||
const value = feature.enabled ? "开" : "关";
|
||||
@@ -251,6 +337,8 @@ export function renderFeaturesText(document: FeaturesDocument): string {
|
||||
const state = document.features;
|
||||
const lines = [`当前模式:${state.mode}`];
|
||||
for (const feature of state.features) lines.push(featureLine(feature));
|
||||
for (const parameter of state.parameters ?? [])
|
||||
lines.push(parameterLine(parameter));
|
||||
lines.push(
|
||||
"片段召回(episode_recall)只对带片段的记忆有用:先开启 session_episodes,之后提取的记忆才会带片段。",
|
||||
);
|
||||
@@ -258,7 +346,9 @@ export function renderFeaturesText(document: FeaturesDocument): string {
|
||||
if (document.state === "preview") {
|
||||
lines.push("将设置:");
|
||||
for (const [name, value] of Object.entries(document.target)) {
|
||||
lines.push(`- ${name} = ${value ? "开" : "关"}`);
|
||||
lines.push(
|
||||
`- ${name} = ${typeof value === "number" ? value : value ? "开" : "关"}`,
|
||||
);
|
||||
}
|
||||
lines.push(`未执行任何改动。确认后运行:${document.next_step}`);
|
||||
}
|
||||
|
||||
@@ -30,8 +30,7 @@ vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
|
||||
}));
|
||||
|
||||
function languageState(uiLanguage: string | null, extra = {}) {
|
||||
const sdk =
|
||||
uiLanguage === null || uiLanguage === "zh-Hans" ? "auto" : uiLanguage;
|
||||
const sdk = uiLanguage === null ? "auto" : uiLanguage;
|
||||
return {
|
||||
ui_language: uiLanguage,
|
||||
sdk_language: sdk,
|
||||
@@ -95,7 +94,7 @@ describe("memhub settings language", () => {
|
||||
});
|
||||
expect(transport.put).not.toHaveBeenCalled();
|
||||
const text = (await capturedRun(["settings", "language"])).stdout;
|
||||
expect(text).toContain("界面语言:跟随系统(未选择)");
|
||||
expect(text).toContain("界面语言:未设置(SDK 自动识别)");
|
||||
expect(text).toContain("SDK 提示语言:auto");
|
||||
});
|
||||
|
||||
|
||||
@@ -2,10 +2,11 @@
|
||||
* `memhub settings language [<code>|system]`.
|
||||
*
|
||||
* The display language the App reports to the Runtime (`ui_language`), and
|
||||
* the SDK prompt language it implies. zh-Hans and "system" (no explicit
|
||||
* choice) pass no language, so the SDK keeps detecting Chinese or English as
|
||||
* before; every other code, en included, makes extraction and the other SDK
|
||||
* prompts use that language. A change applies at once: a write in flight
|
||||
* the SDK prompt language it implies. Every code, zh-Hans and en included,
|
||||
* makes extraction and the other SDK prompts use that language; "system"
|
||||
* (no language set) passes none, so the SDK keeps detecting Chinese or
|
||||
* English as before. The App sends the language it shows, so it is only
|
||||
* unset on an install without the App. A change applies at once: a write in flight
|
||||
* finishes in the old language, the next ones use the new one. The App sends
|
||||
* its own choice again whenever it changes there.
|
||||
*/
|
||||
@@ -162,14 +163,14 @@ export function renderLanguageText(document: LanguageDocument): string {
|
||||
const lines = [
|
||||
`界面语言:${
|
||||
state.ui_language === null
|
||||
? "跟随系统(未选择)"
|
||||
? "未设置(SDK 自动识别)"
|
||||
: `${state.ui_language}(${NAMES[state.ui_language]})`
|
||||
}`,
|
||||
`SDK 提示语言:${sdkText(state.sdk_language)}`,
|
||||
];
|
||||
if (document.state === "ok") {
|
||||
lines.push(
|
||||
`可选:${UI_LANGUAGES.join("、")}、system。简体中文和 system 不指定语言,其余语言(包括 en)会让 SDK 使用该语言。`,
|
||||
`可选:${UI_LANGUAGES.join("、")}、system。任一语言(包括 zh-Hans、en)都会让 SDK 使用该语言;system 不指定语言,SDK 自动识别中文或英文。`,
|
||||
);
|
||||
}
|
||||
if (document.state === "unchanged") lines.push("没有需要改动的设置。");
|
||||
|
||||
@@ -248,6 +248,10 @@ function addReceipt(receipt: MemoryAddReceipt): Record<string, unknown> {
|
||||
task_id: receipt.task_id,
|
||||
memory_ids: receipt.memory_ids,
|
||||
error: receipt.error,
|
||||
error_code:
|
||||
typeof receipt.error_code === "string" ? receipt.error_code : null,
|
||||
next_retry_at:
|
||||
typeof receipt.next_retry_at === "string" ? receipt.next_retry_at : null,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -347,6 +351,12 @@ export function renderMemoryAddText(document: MemoryAddDocument): string {
|
||||
...((document.receipt.memory_ids as string[]).length > 0
|
||||
? [`memory_ids: ${(document.receipt.memory_ids as string[]).join(",")}`]
|
||||
: []),
|
||||
...(document.receipt.error_code
|
||||
? [`error_code: ${document.receipt.error_code}`]
|
||||
: []),
|
||||
...(document.receipt.next_retry_at
|
||||
? [`next_retry_at: ${document.receipt.next_retry_at}`]
|
||||
: []),
|
||||
document.command === "memories add"
|
||||
? `查询进度:memhub memories add-status ${document.request_id};重试请带上同一个 --request-id,不会重复添加。`
|
||||
: "",
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import type { CoreReader, ReadOutcome } from "./core-client.js";
|
||||
import { composeDoctorReport, renderDoctorText } from "./doctor-report.js";
|
||||
import {
|
||||
composeStatusReport,
|
||||
@@ -41,6 +42,75 @@ describe("status default output", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("status on a paused model", () => {
|
||||
const processing = (backoff: {
|
||||
llm_backoff_until: string | null;
|
||||
llm_backoff_error: string | null;
|
||||
}) => ({
|
||||
pending: 4,
|
||||
fresh_pending: 4,
|
||||
backlog_pending: 0,
|
||||
outside_window_pending: 0,
|
||||
running: 0,
|
||||
waiting_for_model: 0,
|
||||
completed: 1,
|
||||
failed: 0,
|
||||
blocked_stages: [],
|
||||
suspended: 2,
|
||||
...backoff,
|
||||
});
|
||||
const reader =
|
||||
(summary: unknown): CoreReader =>
|
||||
async <T>(path: string) =>
|
||||
(path === "/api/v1/memory-processing"
|
||||
? { state: "ok", data: summary }
|
||||
: { state: "unavailable", code: "x" }) as ReadOutcome<T>;
|
||||
const compose = (summary: unknown) =>
|
||||
composeStatusReport({
|
||||
dataDir: "/tmp/isolated/.one-memory",
|
||||
runtime: {
|
||||
state: "running",
|
||||
code: "runtime_verified",
|
||||
host: "127.0.0.1",
|
||||
port: 30001,
|
||||
},
|
||||
background: null,
|
||||
reader: reader(summary),
|
||||
});
|
||||
|
||||
it("names the code and the retry time while the breaker pauses", async () => {
|
||||
const report = await compose(
|
||||
processing({
|
||||
llm_backoff_until: "2026-11-02T10:00:00+00:00",
|
||||
llm_backoff_error: "model_auth_failed",
|
||||
}),
|
||||
);
|
||||
expect(report.sections.processing).toMatchObject({
|
||||
state: "ok",
|
||||
pending: 4,
|
||||
suspended: 2,
|
||||
llm_backoff_until: "2026-11-02T10:00:00+00:00",
|
||||
llm_backoff_error: "model_auth_failed",
|
||||
});
|
||||
const text = renderStatusText(report, "MemHub");
|
||||
expect(text).toContain(
|
||||
"模型暂停:model_auth_failed,将于 2026-11-02T10:00:00+00:00 自动重试",
|
||||
);
|
||||
expect(text).not.toContain("llm_backoff");
|
||||
});
|
||||
|
||||
it("prints no pause line and null fields when nothing is paused", async () => {
|
||||
const report = await compose(
|
||||
processing({ llm_backoff_until: null, llm_backoff_error: null }),
|
||||
);
|
||||
expect(report.sections.processing).toMatchObject({
|
||||
llm_backoff_until: null,
|
||||
llm_backoff_error: null,
|
||||
});
|
||||
expect(renderStatusText(report, "MemHub")).not.toContain("模型暂停");
|
||||
});
|
||||
});
|
||||
|
||||
describe("status and doctor on an unfinished upgrade migration", () => {
|
||||
it("names the migration log and the issues page", async () => {
|
||||
const report = await composeStatusReport({
|
||||
|
||||
@@ -85,7 +85,7 @@ export interface PipelineSummary {
|
||||
>;
|
||||
graph_build: string | null;
|
||||
/** SDK features (`memory_features`) whose value differs from the default. */
|
||||
features: Record<string, boolean>;
|
||||
features: Record<string, boolean | number>;
|
||||
/** The App's display language; null = never chosen or "follow the system". */
|
||||
ui_language: string | null;
|
||||
/** The SDK prompt language: `auto` (zh/en detection) or the code passed. */
|
||||
@@ -134,12 +134,13 @@ export function describePipeline(value: unknown): PipelineSummary | null {
|
||||
};
|
||||
}
|
||||
}
|
||||
const features: Record<string, boolean> = {};
|
||||
const features: Record<string, boolean | number> = {};
|
||||
if (item.features && typeof item.features === "object") {
|
||||
for (const [name, value] of Object.entries(
|
||||
item.features as Record<string, unknown>,
|
||||
)) {
|
||||
if (typeof value === "boolean") features[name] = value;
|
||||
if (typeof value === "boolean" || typeof value === "number")
|
||||
features[name] = value;
|
||||
}
|
||||
}
|
||||
const graph =
|
||||
@@ -319,6 +320,8 @@ export function renderStatusText(
|
||||
report.sections.processing?.embedding_switch,
|
||||
);
|
||||
if (embeddingSwitch !== null) lines.push(`向量模型:${embeddingSwitch}`);
|
||||
const backoff = modelBackoffLine(report.sections.processing);
|
||||
if (backoff !== null) lines.push(backoff);
|
||||
if (report.summary.state === "partial") {
|
||||
const notes = [
|
||||
report.summary.degraded.length > 0
|
||||
@@ -334,6 +337,21 @@ export function renderStatusText(
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
/** "Model paused" while Core's circuit breaker holds model work back; null
|
||||
* otherwise. Nothing to do here: the queue resumes by itself once the model
|
||||
* answers again (or after the model is fixed). */
|
||||
function modelBackoffLine(section: Section | undefined): string | null {
|
||||
if (section?.state !== "ok") return null;
|
||||
const error = section.llm_backoff_error;
|
||||
if (typeof error !== "string" || error.length === 0) return null;
|
||||
const until = section.llm_backoff_until;
|
||||
return `模型暂停:${error},${
|
||||
typeof until === "string" && until.length > 0
|
||||
? `将于 ${until} 自动重试`
|
||||
: "稍后自动重试"
|
||||
}`;
|
||||
}
|
||||
|
||||
export function pipelineLine(pipeline: PipelineSummary): string {
|
||||
const parts: string[] = [pipeline.mode];
|
||||
if (pipeline.forced_by_env)
|
||||
@@ -357,7 +375,10 @@ export function pipelineLine(pipeline: PipelineSummary): string {
|
||||
if (changed.length > 0) {
|
||||
parts.push(
|
||||
`;SDK 功能:${changed
|
||||
.map(([name, value]) => `${name}=${value ? "开" : "关"}`)
|
||||
.map(
|
||||
([name, value]) =>
|
||||
`${name}=${typeof value === "number" ? value : value ? "开" : "关"}`,
|
||||
)
|
||||
.join("、")}`,
|
||||
);
|
||||
}
|
||||
@@ -380,15 +401,22 @@ function runtimeLine(report: StatusDocument): string {
|
||||
})`;
|
||||
}
|
||||
|
||||
/** Fields that have their own line (or none) instead of a place in the
|
||||
* section's one-line summary. */
|
||||
const SECTION_LINE_OMITTED = new Set([
|
||||
"state",
|
||||
"code",
|
||||
"embedding_switch",
|
||||
"llm_backoff_until",
|
||||
"llm_backoff_error",
|
||||
]);
|
||||
|
||||
function sectionLine(section: Section | undefined): string {
|
||||
if (!section) return "未检查";
|
||||
if (section.state === "not_checked") return "未检查";
|
||||
if (section.state !== "ok") return "暂时不可用,请稍后重试";
|
||||
return Object.entries(section)
|
||||
.filter(
|
||||
([key]) =>
|
||||
key !== "state" && key !== "code" && key !== "embedding_switch",
|
||||
)
|
||||
.filter(([key]) => !SECTION_LINE_OMITTED.has(key))
|
||||
.map(([key, value]) => `${fieldLabel(key)} ${formatValue(value)}`)
|
||||
.join(" ");
|
||||
}
|
||||
@@ -424,6 +452,7 @@ function fieldLabel(key: string): string {
|
||||
backlog_pending: "历史待处理",
|
||||
outside_window_pending: "稍后处理",
|
||||
waiting_for_model: "等待模型",
|
||||
suspended: "来源已暂停",
|
||||
completed: "已完成",
|
||||
failed: "失败",
|
||||
blocked: "卡住",
|
||||
@@ -504,10 +533,28 @@ function describeProcessing(
|
||||
reason: block.reason,
|
||||
count: block.count,
|
||||
})),
|
||||
suspended: typeof summary.suspended === "number" ? summary.suspended : 0,
|
||||
...llmBackoffFields(summary),
|
||||
...embeddingSwitchField(summary),
|
||||
};
|
||||
}
|
||||
|
||||
/** The model circuit breaker: set only while Core pauses model work because
|
||||
* the provider keeps failing. Both null when nothing is paused. */
|
||||
function llmBackoffFields(summary: MemoryProcessingSummary): {
|
||||
llm_backoff_until: string | null;
|
||||
llm_backoff_error: string | null;
|
||||
} {
|
||||
const until = summary.llm_backoff_until;
|
||||
const error = summary.llm_backoff_error;
|
||||
return {
|
||||
llm_backoff_until:
|
||||
typeof until === "string" && until.length <= 64 ? until : null,
|
||||
llm_backoff_error:
|
||||
typeof error === "string" && /^[a-z_]{1,64}$/.test(error) ? error : null,
|
||||
};
|
||||
}
|
||||
|
||||
/** The running switch or merge, projected; absent when none runs. */
|
||||
function embeddingSwitchField(
|
||||
summary: MemoryProcessingSummary,
|
||||
|
||||
@@ -208,6 +208,27 @@ describe("--user-id on the memory commands", () => {
|
||||
expect(direct.get.mock.calls[0][1].params).toEqual({ user_id: "wife" });
|
||||
});
|
||||
|
||||
it("prints the retry code and time of a write still waiting", async () => {
|
||||
direct.get.mockResolvedValue({
|
||||
data: {
|
||||
request_id: REQUEST,
|
||||
state: "accepted",
|
||||
created_at: "T1",
|
||||
updated_at: "T2",
|
||||
error_code: "model_timeout",
|
||||
next_retry_at: "2026-11-02T10:00:00+00:00",
|
||||
},
|
||||
});
|
||||
const text = await capturedRun(["write", "status", REQUEST]);
|
||||
expect(text.stdout).toContain("error_code: model_timeout");
|
||||
expect(text.stdout).toContain("next_retry_at: 2026-11-02T10:00:00+00:00");
|
||||
const json = await capturedRun(["write", "status", REQUEST, "--json"]);
|
||||
expect(JSON.parse(json.stdout).write).toMatchObject({
|
||||
error_code: "model_timeout",
|
||||
next_retry_at: "2026-11-02T10:00:00+00:00",
|
||||
});
|
||||
});
|
||||
|
||||
it("passes the user in the body of memory writes and searches", async () => {
|
||||
await capturedRun(["memories", "add", "Prefers tea", "--user-id", "wife"]);
|
||||
expect(transport.post.mock.calls[0][0]).toBe("/api/v2/memories");
|
||||
@@ -373,6 +394,48 @@ describe("sources connect for a Hermes profile", () => {
|
||||
}
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("says the user id need not equal the Hermes profile name", async () => {
|
||||
const result = await capturedRun([
|
||||
"sources",
|
||||
"connect",
|
||||
"hermes",
|
||||
"--user-id",
|
||||
"wife:work",
|
||||
"--root",
|
||||
"~/.hermes/profiles/wife:work",
|
||||
]);
|
||||
expect(result.code).toBe(2);
|
||||
expect(result.stderr).toContain("用户 ID 不必等于 Hermes profile 名");
|
||||
expect(result.stderr).toContain(
|
||||
"--user-id wife-work --root ~/.hermes/profiles/wife:work",
|
||||
);
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("explains a refused overlapping or profiles-container root", async () => {
|
||||
for (const code of [
|
||||
"hermes_root_is_profiles_container",
|
||||
"hermes_root_overlaps_other_user",
|
||||
]) {
|
||||
transport.post.mockResolvedValueOnce({
|
||||
status: code.endsWith("container") ? 422 : 409,
|
||||
data: { detail: code },
|
||||
});
|
||||
const result = await capturedRun([
|
||||
"sources",
|
||||
"connect",
|
||||
"hermes",
|
||||
"--user-id",
|
||||
"kid",
|
||||
"--root",
|
||||
"/profiles",
|
||||
]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(result.stdout).toContain(code);
|
||||
expect(result.stdout).toContain("--root");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
async function capturedRun(arguments_: string[]) {
|
||||
|
||||
@@ -12,7 +12,7 @@ dependencies = [
|
||||
"fastapi>=0.141.1",
|
||||
"filelock>=3.32.4",
|
||||
# Public SDK version validated against MemHub's permission and recall contracts.
|
||||
"hy-memory==2.0.11",
|
||||
"hy-memory==2.0.12",
|
||||
# chromadb arrives through the HY SDK; locked here so an unrelated upgrade
|
||||
# cannot move the SDK's store. Built-in embedding consumes numpy directly.
|
||||
"chromadb==1.5.9",
|
||||
|
||||
@@ -46,7 +46,12 @@ from pydantic import (
|
||||
|
||||
from one_memory import __version__, memory_visibility
|
||||
from one_memory.auth import local_api_guard, renderer_origins
|
||||
from one_memory.background_work import shutdown_bulk_pool, warm_bulk_pool
|
||||
from one_memory.background_work import (
|
||||
in_sdk_read,
|
||||
in_the_background,
|
||||
shutdown_bulk_pool,
|
||||
warm_bulk_pool,
|
||||
)
|
||||
from one_memory.config import Settings, load_settings
|
||||
from one_memory.context_routes import sdk_context_router
|
||||
from one_memory.conversation_processing import (
|
||||
@@ -55,6 +60,7 @@ from one_memory.conversation_processing import (
|
||||
)
|
||||
from one_memory.database_access import (
|
||||
MAIN_DATABASE_REPORT_SECONDS,
|
||||
checkpoint_idle_main_databases,
|
||||
close_main_databases,
|
||||
log_main_database_stats,
|
||||
main_database,
|
||||
@@ -130,6 +136,7 @@ from one_memory.hy_sdk_types import (
|
||||
from one_memory.imports.access import (
|
||||
AuthorizedSourceRoot,
|
||||
SourceAccessError,
|
||||
resolved_source_root,
|
||||
root_identity_matches,
|
||||
source_root_identity,
|
||||
)
|
||||
@@ -169,6 +176,11 @@ from one_memory.imports.locations import SourceLocation, inspect_source_location
|
||||
from one_memory.imports.models import EvidenceRecord
|
||||
from one_memory.imports.runner import live_sync_arrival
|
||||
from one_memory.imports.sources.common import SourceInstallState
|
||||
from one_memory.imports.sources.hermes import (
|
||||
HERMES_ROOT_IS_PROFILES_CONTAINER,
|
||||
hermes_root_conflict,
|
||||
hermes_root_is_profiles_container,
|
||||
)
|
||||
from one_memory.imports.sources.registry import (
|
||||
RootCandidateKind,
|
||||
SourceRegistration,
|
||||
@@ -198,12 +210,17 @@ from one_memory.memory_evolution import (
|
||||
)
|
||||
from one_memory.memory_features import (
|
||||
FEATURES,
|
||||
PARAMETERS,
|
||||
SETTING_NAMES,
|
||||
FeatureCost,
|
||||
FeatureOverrides,
|
||||
FeatureReason,
|
||||
FeatureSide,
|
||||
MemoryFeatureName,
|
||||
MemoryFeatures,
|
||||
MemoryParameterName,
|
||||
MemorySettingName,
|
||||
SettingValue,
|
||||
effective_features,
|
||||
feature_activity,
|
||||
non_default_features,
|
||||
@@ -222,10 +239,10 @@ from one_memory.memory_server.config_state import (
|
||||
MemoryMode,
|
||||
)
|
||||
from one_memory.memory_tasks import (
|
||||
NON_RETRYABLE_POST_WRITE_CODES,
|
||||
MemoryTaskStatus,
|
||||
MemoryTaskStore,
|
||||
)
|
||||
from one_memory.model_backoff import BackedOffModelTransport, ModelBackoff
|
||||
from one_memory.model_presets import seed_preset_profile
|
||||
from one_memory.model_probe import ModelProber, ModelProbeResult
|
||||
from one_memory.model_profiles import (
|
||||
@@ -279,7 +296,7 @@ from one_memory.sdk_search_options import (
|
||||
SearchLane,
|
||||
split_csv,
|
||||
)
|
||||
from one_memory.sdk_write_processor import SdkWriteCoordinator
|
||||
from one_memory.sdk_write_processor import SdkWriteCoordinator, sdk_write_in_flight
|
||||
from one_memory.secrets import (
|
||||
KeyringSecretStore,
|
||||
SecretStore,
|
||||
@@ -526,11 +543,25 @@ class SdkMemoryUpdateRequest(BaseModel):
|
||||
class SdkUserProfileResponse(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
#: The user whose profile this is: the `user_id` asked for, else `local`.
|
||||
user_id: str
|
||||
request_id: str
|
||||
profile: dict[str, object] | None
|
||||
elapsed_ms: float = Field(ge=0)
|
||||
|
||||
|
||||
class SdkMemoryLibraryPage(HySdkMemoryPage):
|
||||
"""`GET /api/v2/memory-library`: one SDK page and the user it was read for."""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
user_id: str
|
||||
|
||||
@classmethod
|
||||
def of(cls, page: HySdkMemoryPage, user_id: str) -> "SdkMemoryLibraryPage":
|
||||
return cls(**dict(page), user_id=user_id)
|
||||
|
||||
|
||||
class SdkProfileVersionsResponse(HySdkProfileVersions):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
@@ -585,6 +616,8 @@ class SdkRawItem(HySdkRawRecord):
|
||||
class SdkRawPage(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
#: The user whose raw turns these are.
|
||||
user_id: str
|
||||
items: tuple[SdkRawItem, ...] = ()
|
||||
total: int = Field(ge=0)
|
||||
offset: int = Field(ge=0)
|
||||
@@ -598,6 +631,8 @@ class SdkRawPage(BaseModel):
|
||||
class SdkLayerCounts(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
#: The user whose memories were counted.
|
||||
user_id: str
|
||||
#: Every SDK layer, zero when empty.
|
||||
layers: dict[str, int]
|
||||
total: int = Field(ge=0)
|
||||
@@ -683,6 +718,17 @@ class MemoryProcessingSummary(BaseModel):
|
||||
#: The running embedding switch or merge, `{job_id, kind, state, percent,
|
||||
#: eta_s}`; None when none runs. Model writes wait while it is set.
|
||||
embedding_switch: dict[str, object] | None = None
|
||||
#: Jobs held back because their source is off or its grant was revoked.
|
||||
#: Not in `pending` or the queue numbers: they run again, unchanged, once
|
||||
#: the source is turned back on. Nothing is cancelled or deleted.
|
||||
suspended: int = 0
|
||||
#: Until when model work is paused because the provider kept failing
|
||||
#: (bad key, wrong URL, model missing, provider down); None when it is
|
||||
#: not. Ends early on success or when the model configuration changes.
|
||||
llm_backoff_until: datetime | None = None
|
||||
#: The last provider failure behind the pause, as a stable code
|
||||
#: (`model_auth_failed`, `model_not_found`, `model_unreachable`, ...).
|
||||
llm_backoff_error: str | None = None
|
||||
|
||||
|
||||
class MemoryModeState(BaseModel):
|
||||
@@ -763,10 +809,11 @@ class PipelineStatus(BaseModel):
|
||||
last_evolve_at: datetime | None = None
|
||||
graph_build: GraphBuildState
|
||||
#: The SDK features (`memory_features`) whose value differs from the SDK
|
||||
#: default; empty while every feature is at its default.
|
||||
features: dict[MemoryFeatureName, bool] = Field(default_factory=dict)
|
||||
#: default; empty while every feature is at its default. A parameter
|
||||
#: (`intention_dedup_threshold`) appears as its number.
|
||||
features: dict[MemorySettingName, SettingValue] = Field(default_factory=dict)
|
||||
#: The display language the App last reported (`ui_language`); null =
|
||||
#: never chosen or "follow the system".
|
||||
#: none set (the App sends the language it shows, even following the system).
|
||||
ui_language: UiLanguage | None = None
|
||||
#: The SDK prompt language: `auto` (the SDK detects zh/en) or a code.
|
||||
sdk_language: str = SDK_LANGUAGE_AUTO
|
||||
@@ -777,9 +824,9 @@ class UiLanguageState(BaseModel):
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
#: What the App reported; null = never chosen or "follow the system".
|
||||
#: What the App reported; null = none set (CLI/API-only, or `system`).
|
||||
ui_language: UiLanguage | None
|
||||
#: What SDK clients are built with: `auto` (zh-Hans or no choice: the SDK
|
||||
#: What SDK clients are built with: `auto` (no language set: the SDK
|
||||
#: detects zh/en, as before) or the code passed as `MemoryConfig.language`.
|
||||
sdk_language: str
|
||||
languages: tuple[UiLanguage, ...] = UI_LANGUAGES
|
||||
@@ -824,6 +871,30 @@ class MemoryFeatureState(BaseModel):
|
||||
description: str
|
||||
|
||||
|
||||
class MemoryParameterState(BaseModel):
|
||||
"""One numeric setting in `GET /api/v1/settings/memory-features`."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
name: MemoryParameterName
|
||||
#: What clients are built with: `configured`, else `default`.
|
||||
value: float
|
||||
#: What a person set; null = not set, the SDK default applies.
|
||||
configured: float | None
|
||||
#: The SDK default (its code default, or its own env var when set).
|
||||
default: float
|
||||
#: The range a `PUT` accepts, inclusive.
|
||||
minimum: float
|
||||
maximum: float
|
||||
#: Whether it acts: exactly when `requires_feature` acts.
|
||||
active: bool
|
||||
reason: FeatureReason | None
|
||||
requires_feature: MemoryFeatureName
|
||||
sdk_field: str
|
||||
env: str
|
||||
description: str
|
||||
|
||||
|
||||
class MemoryFeaturesState(BaseModel):
|
||||
"""`GET /api/v1/settings/memory-features`."""
|
||||
|
||||
@@ -831,15 +902,16 @@ class MemoryFeaturesState(BaseModel):
|
||||
|
||||
mode: MemoryMode
|
||||
features: tuple[MemoryFeatureState, ...]
|
||||
parameters: tuple[MemoryParameterState, ...] = ()
|
||||
#: Same as `pipeline.features`: the values that differ from the defaults.
|
||||
non_default: dict[MemoryFeatureName, bool]
|
||||
non_default: dict[MemorySettingName, SettingValue]
|
||||
|
||||
|
||||
class MemoryFeaturesChange(MemoryFeaturesState):
|
||||
"""`PUT /api/v1/settings/memory-features`: the new state and what changed."""
|
||||
|
||||
#: Features whose value changed; empty = nothing rebuilt.
|
||||
changed: tuple[MemoryFeatureName, ...] = ()
|
||||
#: Features and parameters whose value changed; empty = nothing rebuilt.
|
||||
changed: tuple[MemorySettingName, ...] = ()
|
||||
|
||||
|
||||
def memory_mode_effects(
|
||||
@@ -873,15 +945,6 @@ def memory_mode_effects(
|
||||
MemoryProcessingSummary.model_rebuild()
|
||||
|
||||
|
||||
class RetryUnknownMemoryTasksRequest(BaseModel):
|
||||
confirm: Literal[True]
|
||||
|
||||
|
||||
class RetryUnknownMemoryTasksReceipt(BaseModel):
|
||||
reconciled: int = 0
|
||||
requeued: int = 0
|
||||
|
||||
|
||||
class GraphWidthStatus(BaseModel):
|
||||
"""What the app needs to offer the choice before extraction fails on it."""
|
||||
|
||||
@@ -967,6 +1030,8 @@ class AskMemoryRequest(SdkSearchFilters):
|
||||
|
||||
class SdkHomeSummary(BaseModel):
|
||||
schema_version: Literal[2] = 2
|
||||
#: The user the card was read for.
|
||||
user_id: str
|
||||
memory_count: int
|
||||
latest: HySdkMemoryRecord | None
|
||||
observed_at: datetime
|
||||
@@ -993,6 +1058,8 @@ class MemoryGraphEdge(BaseModel):
|
||||
|
||||
|
||||
class MemoryGraphResponse(BaseModel):
|
||||
#: The user whose memories were mapped.
|
||||
user_id: str
|
||||
total_memories: int
|
||||
projected_memories: int
|
||||
nodes: tuple[MemoryGraphNode, ...]
|
||||
@@ -1145,7 +1212,6 @@ def create_app(
|
||||
)
|
||||
from one_memory.memory_server.data_safety import MIN_FREE_BYTES
|
||||
from one_memory.memory_server.durable_queue import (
|
||||
COMPLETED,
|
||||
DurableWriteDrain,
|
||||
DurableWriteQueue,
|
||||
execute_job,
|
||||
@@ -1304,6 +1370,9 @@ def create_app(
|
||||
)
|
||||
app.state.model_request_budget = model_request_budget
|
||||
app.state.model_request_audit_store = ModelRequestAuditStore(database_path)
|
||||
#: One pause for every model request while the provider keeps
|
||||
#: failing (`model_backoff`); reset when the configuration changes.
|
||||
app.state.model_backoff = ModelBackoff()
|
||||
# Before anything asks for an active profile: a first launch that
|
||||
# carries a preset should show it already filled in, so the user
|
||||
# meets one empty field instead of a form they cannot answer.
|
||||
@@ -1322,17 +1391,23 @@ def create_app(
|
||||
profiles=app.state.model_profile_store,
|
||||
secrets=app.state.secret_store,
|
||||
data_dir=runtime_settings.data_dir,
|
||||
# Outermost, so a request the pause refuses never went out and
|
||||
# leaves no audit row behind.
|
||||
transport_factory=lambda profile: (
|
||||
lambda: AuditedModelTransport(
|
||||
app.state.model_request_audit_store,
|
||||
profile_id=profile.profile_id,
|
||||
profile_revision=profile.revision,
|
||||
provider_kind=str(profile.provider_kind),
|
||||
model=profile.model,
|
||||
inner=(
|
||||
BudgetedModelTransport(model_request_budget)
|
||||
if model_request_budget is not None
|
||||
else None
|
||||
lambda: BackedOffModelTransport(
|
||||
app.state.model_backoff,
|
||||
profile_id=f"{profile.profile_id}:{profile.revision}",
|
||||
inner=AuditedModelTransport(
|
||||
app.state.model_request_audit_store,
|
||||
profile_id=profile.profile_id,
|
||||
profile_revision=profile.revision,
|
||||
provider_kind=str(profile.provider_kind),
|
||||
model=profile.model,
|
||||
inner=(
|
||||
BudgetedModelTransport(model_request_budget)
|
||||
if model_request_budget is not None
|
||||
else None
|
||||
),
|
||||
),
|
||||
)
|
||||
),
|
||||
@@ -1351,6 +1426,9 @@ def create_app(
|
||||
data_dir=runtime_settings.data_dir,
|
||||
memory_mode=lambda: _memory_mode(app),
|
||||
writes_open=lambda: _embedding_writes_open(app),
|
||||
# The claim gate's own source question, so what is counted as
|
||||
# waiting is what the drain would claim (#245).
|
||||
source_active=lambda source_key: _source_writes_on(app, source_key),
|
||||
)
|
||||
app.state.evolve_state_store = EvolveStateStore(runtime_settings.data_dir)
|
||||
app.state.evolve_service = EvolveService(
|
||||
@@ -1375,24 +1453,15 @@ def create_app(
|
||||
drain_budget = production_drain_budget()
|
||||
|
||||
async def reconcile_unknown_writes() -> None:
|
||||
unknown = await asyncio.to_thread(
|
||||
memory_write_queue.unknown_jobs,
|
||||
limit=128,
|
||||
exclude_error_codes=NON_RETRYABLE_POST_WRITE_CODES,
|
||||
# A write whose worker never learned how it ended (a restart,
|
||||
# a lapsed lease, the work ceiling) goes back into the queue
|
||||
# under a new SDK request id; the re-run first asks the SDK
|
||||
# whether the old one landed (`execute_job`). Nothing stays
|
||||
# `unknown` waiting for a person.
|
||||
await in_the_background(
|
||||
memory_write_queue.recover_unknown,
|
||||
in_flight=sdk_write_in_flight,
|
||||
)
|
||||
for job in unknown:
|
||||
receipt = await app.state.sdk_write_coordinator.inquire_write_outcome(
|
||||
job.request_id
|
||||
)
|
||||
if receipt is None:
|
||||
continue
|
||||
await asyncio.to_thread(
|
||||
memory_write_queue.settle,
|
||||
job.job_id,
|
||||
state=COMPLETED,
|
||||
sdk_raw_ids=receipt.raw_ids,
|
||||
sdk_memory_ids=receipt.memory_ids,
|
||||
)
|
||||
|
||||
async def admit_hy_invocation(
|
||||
source_id: str,
|
||||
@@ -1422,6 +1491,7 @@ def create_app(
|
||||
evidence=app.state.evidence_store,
|
||||
source_writes_permitted=lambda source_id: _source_writes_on(app, source_id),
|
||||
admit_hy_invocation=admit_hy_invocation,
|
||||
model_failures=app.state.model_backoff.attempt,
|
||||
),
|
||||
execute_owned=lambda job_id, owner: execute_job(
|
||||
job_id,
|
||||
@@ -1431,6 +1501,7 @@ def create_app(
|
||||
owner=owner,
|
||||
source_writes_permitted=lambda source_id: _source_writes_on(app, source_id),
|
||||
admit_hy_invocation=admit_hy_invocation,
|
||||
model_failures=app.state.model_backoff.attempt,
|
||||
),
|
||||
drain_handle_budget=drain_budget.handle_budget,
|
||||
drain_cleanup_reserve=drain_budget.cleanup_reserve,
|
||||
@@ -1458,6 +1529,7 @@ def create_app(
|
||||
# therefore skipped on every drain tick — the note stayed
|
||||
# "saved, waiting" with no task behind it, forever.
|
||||
source_claim_enabled=lambda source_id: _sedimentation_claim_enabled(app, source_id),
|
||||
model_claim_allowance=app.state.model_backoff.allowance,
|
||||
)
|
||||
app.state.memory_write_drain = memory_write_drain
|
||||
|
||||
@@ -1495,6 +1567,7 @@ def create_app(
|
||||
app.state.memory_task_store.rebind_unstarted_profile, profile
|
||||
)
|
||||
await asyncio.to_thread(memory_write_queue.readmit_model_profile_waiters)
|
||||
_reset_model_backoff(app)
|
||||
_resume_sedimentation(app)
|
||||
_start_backfill(app)
|
||||
|
||||
@@ -1970,6 +2043,10 @@ def create_app(
|
||||
# Last: what has just been shut down was using it.
|
||||
shutdown_bulk_pool()
|
||||
log_main_database_stats("shutdown")
|
||||
# Leave an empty WAL behind: everything that wrote is stopped
|
||||
# by now. Busy (another process reading) is fine; the next
|
||||
# start's idle pass truncates it.
|
||||
await asyncio.to_thread(checkpoint_idle_main_databases)
|
||||
close_main_databases()
|
||||
diagnostic_event(
|
||||
level="info",
|
||||
@@ -2667,6 +2744,7 @@ def create_app(
|
||||
)
|
||||
if not root_identity_matches(registration.candidate.path, request.expected_root_id):
|
||||
raise HTTPException(status_code=409, detail="source_identity_changed")
|
||||
await _refuse_overlapping_root(source_id, registration.candidate.path, DEFAULT_USER_ID)
|
||||
root_changed = prior is not None and prior.root_id != request.expected_root_id
|
||||
if root_changed:
|
||||
source_jobs = [job for job in _job_store(app).list_jobs() if job.source_id == source_id]
|
||||
@@ -3067,6 +3145,20 @@ def create_app(
|
||||
registrations = [_require_registration(app, source_id) for source_id in source_ids]
|
||||
if request.request_id is None:
|
||||
raise HTTPException(status_code=409, detail="source_confirmation_required")
|
||||
for registration in registrations:
|
||||
if registration.descriptor.source_id in MULTI_USER_SOURCES:
|
||||
try:
|
||||
resolved = await asyncio.to_thread(
|
||||
app.state.source_candidate_store.resolve, registration
|
||||
)
|
||||
except SourceCandidateError:
|
||||
# The connect below reports the missing candidate itself.
|
||||
continue
|
||||
await _refuse_overlapping_root(
|
||||
registration.descriptor.source_id,
|
||||
resolved.candidate.path,
|
||||
DEFAULT_USER_ID,
|
||||
)
|
||||
history_after = await _onboarding_history_after(
|
||||
request,
|
||||
registrations,
|
||||
@@ -3166,6 +3258,42 @@ def create_app(
|
||||
raise
|
||||
return ConnectSourcesResponse(**result.model_dump())
|
||||
|
||||
async def _refuse_overlapping_root(source_id: str, root: Path, user_id: str) -> None:
|
||||
"""Refuse a Hermes root that would read another user's files.
|
||||
|
||||
422 `hermes_root_is_profiles_container` for `.../profiles` itself;
|
||||
409 `hermes_root_overlaps_other_user` for a root equal to, holding or
|
||||
inside another user's active grant, unless one is the other's
|
||||
`profiles/<name>` (the scan never enters `profiles/`).
|
||||
"""
|
||||
if source_id not in MULTI_USER_SOURCES:
|
||||
return
|
||||
if hermes_root_is_profiles_container(root):
|
||||
raise HTTPException(status_code=422, detail=HERMES_ROOT_IS_PROFILES_CONTAINER)
|
||||
|
||||
def conflict_with_others() -> str | None:
|
||||
try:
|
||||
mine = resolved_source_root(root)
|
||||
except SourceAccessError:
|
||||
# Unreadable or missing: the grant that follows refuses it
|
||||
# with its own code.
|
||||
return None
|
||||
others = [
|
||||
Path(path)
|
||||
for receipt, path in _grant_store(app).grants()
|
||||
if receipt.source_id == source_id
|
||||
and receipt.user_id != user_id
|
||||
and receipt.revoked_at is None
|
||||
]
|
||||
return hermes_root_conflict(mine, others)
|
||||
|
||||
conflict = await asyncio.to_thread(conflict_with_others)
|
||||
if conflict is not None:
|
||||
raise HTTPException(
|
||||
status_code=422 if conflict == HERMES_ROOT_IS_PROFILES_CONTAINER else 409,
|
||||
detail=conflict,
|
||||
)
|
||||
|
||||
async def _connect_other_root(request: ConnectSourcesRequest) -> ConnectSourcesResponse:
|
||||
"""Grant one Hermes root to a user: `~/.hermes/profiles/<name>` for its own user.
|
||||
|
||||
@@ -3195,6 +3323,7 @@ def create_app(
|
||||
root = Path(request.root)
|
||||
if not root.is_absolute():
|
||||
raise HTTPException(status_code=422, detail="source_root_must_be_absolute")
|
||||
await _refuse_overlapping_root(source_id, root, request.user_id)
|
||||
# Named outright by the caller, like an environment override:
|
||||
# no saved candidate may replace it, and naming it confirms it.
|
||||
registration = replace(
|
||||
@@ -3428,7 +3557,7 @@ def create_app(
|
||||
)
|
||||
def create_model_profile(request: ModelProfileInput) -> ModelProfile:
|
||||
try:
|
||||
return _model_profile_store(app).create(request, _secret_store(app))
|
||||
created = _model_profile_store(app).create(request, _secret_store(app))
|
||||
except EmbeddingBaseUrlRequiredError:
|
||||
raise HTTPException(422, "embedding_base_url_required") from None
|
||||
except ModelCreationOperationConflictError:
|
||||
@@ -3440,6 +3569,8 @@ def create_app(
|
||||
status_code=503,
|
||||
detail=_secret_store_detail(error),
|
||||
) from None
|
||||
_reset_model_backoff(app)
|
||||
return created
|
||||
|
||||
@app.get("/api/v1/model-profile-creations/{operation_id}", dependencies=protected)
|
||||
def model_creation_operation(operation_id: str) -> ModelCreationOperationReceipt:
|
||||
@@ -3474,7 +3605,7 @@ def create_app(
|
||||
) -> ModelProfile:
|
||||
_refuse_embedding_edit_during_switch(app, profile_id, request)
|
||||
try:
|
||||
return _model_profile_store(app).update(
|
||||
updated = _model_profile_store(app).update(
|
||||
profile_id,
|
||||
request,
|
||||
_secret_store(app),
|
||||
@@ -3490,6 +3621,10 @@ def create_app(
|
||||
status_code=503,
|
||||
detail=_secret_store_detail(error),
|
||||
) from None
|
||||
# A saved key, URL or model is what the paused provider was waiting
|
||||
# for: try it now, not after the rest of the pause.
|
||||
_reset_model_backoff(app)
|
||||
return updated
|
||||
|
||||
@app.post(
|
||||
"/api/v1/model-profiles/{profile_id}/activate",
|
||||
@@ -3552,6 +3687,7 @@ def create_app(
|
||||
# D75: work queued for the previous model continues with this one.
|
||||
# Only tasks that never reached the model are re-bound.
|
||||
await asyncio.to_thread(app.state.memory_task_store.rebind_unstarted_profile, profile)
|
||||
_reset_model_backoff(app)
|
||||
# A missing profile is an event-driven wait, not a provider outage.
|
||||
# Release those durable rows only now that activation has changed the
|
||||
# condition; the ordinary drain poll will claim them without a restart.
|
||||
@@ -3730,6 +3866,7 @@ def create_app(
|
||||
raise _sdk_http_error(error, "hy_memory_list_failed") from None
|
||||
total = stats.get("total") if isinstance(stats, dict) else None
|
||||
return SdkHomeSummary(
|
||||
user_id=user_id,
|
||||
memory_count=total if isinstance(total, int) and total >= 0 else page.total,
|
||||
latest=page.memories[0] if page.memories else None,
|
||||
observed_at=datetime.now(UTC),
|
||||
@@ -3737,7 +3874,7 @@ def create_app(
|
||||
|
||||
@app.get(
|
||||
"/api/v2/memory-library",
|
||||
response_model=HySdkMemoryPage,
|
||||
response_model=SdkMemoryLibraryPage,
|
||||
dependencies=strict,
|
||||
)
|
||||
async def sdk_memory_library(
|
||||
@@ -3754,7 +3891,7 @@ def create_app(
|
||||
exclude_sources: list[str] | None = Query(default=None), # noqa: B008
|
||||
session_ids: list[str] | None = Query(default=None), # noqa: B008
|
||||
agent_ids: list[str] | None = Query(default=None), # noqa: B008
|
||||
) -> HySdkMemoryPage:
|
||||
) -> SdkMemoryLibraryPage:
|
||||
"""One page of memories across chat, working and coding.
|
||||
|
||||
Without `query`: `list_memories(scene="all")`, newest update first;
|
||||
@@ -3800,7 +3937,7 @@ def create_app(
|
||||
status_code=422,
|
||||
detail="sdk_search_supports_active_status_only",
|
||||
)
|
||||
return await _library_search_page(
|
||||
searched = await _library_search_page(
|
||||
app,
|
||||
client,
|
||||
query=search_query,
|
||||
@@ -3813,6 +3950,7 @@ def create_app(
|
||||
user_id=user_id,
|
||||
filters=filters,
|
||||
)
|
||||
return SdkMemoryLibraryPage.of(searched, user_id)
|
||||
raw = await asyncio.to_thread(
|
||||
client.list_memories,
|
||||
user_id=user_id,
|
||||
@@ -3830,7 +3968,7 @@ def create_app(
|
||||
raw, limit=limit, offset=offset
|
||||
)
|
||||
_log_skipped_sdk_memories(skipped)
|
||||
return page
|
||||
return SdkMemoryLibraryPage.of(page, user_id)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as error:
|
||||
@@ -3876,6 +4014,7 @@ def create_app(
|
||||
except Exception as error:
|
||||
raise _sdk_http_error(error, "hy_memory_list_failed") from None
|
||||
return SdkLayerCounts(
|
||||
user_id=user_id,
|
||||
layers=counts,
|
||||
total=sum(counts.values()),
|
||||
statuses=tuple(chosen) if chosen else None,
|
||||
@@ -4036,6 +4175,7 @@ def create_app(
|
||||
)
|
||||
)
|
||||
return MemoryGraphResponse(
|
||||
user_id=user_id,
|
||||
total_memories=total if isinstance(total, int) else len(memories),
|
||||
projected_memories=len(memories),
|
||||
nodes=tuple(nodes.values()),
|
||||
@@ -4057,7 +4197,7 @@ def create_app(
|
||||
profile = HySdkUserProfile.from_sdk(value)
|
||||
except Exception as error:
|
||||
raise _sdk_http_error(error, "hy_memory_profile_read_failed") from None
|
||||
return SdkUserProfileResponse.model_validate(profile.model_dump())
|
||||
return SdkUserProfileResponse(user_id=user_id, **profile.model_dump())
|
||||
|
||||
@app.get(
|
||||
"/api/v2/user-profile/versions",
|
||||
@@ -4181,15 +4321,16 @@ def create_app(
|
||||
try:
|
||||
async with app.state.sdk_write_coordinator.use_client() as client:
|
||||
# `delete` takes no user. Another user's memory is answered
|
||||
# exactly like an id that does not exist: 204, nothing
|
||||
# touched, so the status cannot tell that it exists.
|
||||
# exactly like an id that does not exist: 404
|
||||
# `memory_not_found`, nothing touched, so the status cannot
|
||||
# tell that it exists. Callers read 404 as "already gone".
|
||||
current = await asyncio.to_thread(
|
||||
client.get, memory_id, **_namespace_kwargs(kind, variant)
|
||||
)
|
||||
if current is not None and not owns(
|
||||
if current is None or not owns(
|
||||
HySdkMemoryRecord.from_sdk(current).user_id, user_id
|
||||
):
|
||||
return Response(status_code=204)
|
||||
raise HTTPException(status_code=404, detail="memory_not_found")
|
||||
result = await asyncio.to_thread(
|
||||
client.delete, memory_id, **_namespace_kwargs(kind, variant)
|
||||
)
|
||||
@@ -4198,7 +4339,11 @@ def create_app(
|
||||
error_code = result.get("error_code")
|
||||
if error_code or result.get("partial") is True:
|
||||
raise HySdkError(str(error_code or "hy_memory_delete_incomplete"))
|
||||
if result.get("success") is not True and result.get("deleted_count") != 0:
|
||||
if result.get("success") is not True:
|
||||
# Read a moment ago, found by nobody now (a second delete won
|
||||
# the race): the same answer as an id that never existed.
|
||||
if result.get("deleted_count") == 0:
|
||||
raise HTTPException(status_code=404, detail="memory_not_found")
|
||||
raise HySdkError("hy_memory_delete_rejected")
|
||||
except HTTPException:
|
||||
raise
|
||||
@@ -4271,6 +4416,7 @@ def create_app(
|
||||
and (before is None or at < before)
|
||||
)
|
||||
return SdkRawPage(
|
||||
user_id=user_id,
|
||||
items=tuple(SdkRawItem.of(record) for record in records),
|
||||
total=page.total,
|
||||
offset=offset,
|
||||
@@ -4303,6 +4449,7 @@ def create_app(
|
||||
)
|
||||
except MemoryAnswerError as error:
|
||||
raise HTTPException(status_code=503, detail=str(error)) from None
|
||||
answer = answer.model_copy(update={"user_id": request.user_id})
|
||||
if request.question != "__om_probe__" and access is not None:
|
||||
from one_memory.call_event_recorder import ask_call_event, try_record
|
||||
|
||||
@@ -4344,6 +4491,9 @@ def create_app(
|
||||
):
|
||||
if event.get("type") == "final":
|
||||
final = event
|
||||
elif event.get("type") == "recall":
|
||||
# The first event names whose memories answer.
|
||||
event = {**event, "user_id": request.user_id}
|
||||
yield _sse(_serialize_ask_event(event))
|
||||
except MemoryAnswerError:
|
||||
yield _sse({"type": "error", "code": "memory_answer_failed"})
|
||||
@@ -4379,7 +4529,10 @@ def create_app(
|
||||
http_request: Request,
|
||||
) -> RecallReceipt:
|
||||
from one_memory.call_event_recorder import record_legacy_result
|
||||
from one_memory.recall_fallback import recall_with_fallback
|
||||
from one_memory.recall_fallback import (
|
||||
recall_total_timeout_seconds,
|
||||
recall_with_fallback,
|
||||
)
|
||||
|
||||
started = time.monotonic()
|
||||
await _require_external_recall_scope(
|
||||
@@ -4395,21 +4548,36 @@ def create_app(
|
||||
)
|
||||
raw_options = {"filters": options} if options is not None else {}
|
||||
access = external_access(http_request.headers)
|
||||
receipt = await recall_with_fallback(
|
||||
request,
|
||||
model_ready=lambda: _recall_model_ready(app),
|
||||
recall_memories=target.recall_async,
|
||||
has_memories=lambda: _library_has_memories(app, request.user_id),
|
||||
raw_sources=lambda: _recall_raw_sources(app, request, access, started),
|
||||
search_raw=lambda query, limit, sources: _search_sdk_raw(
|
||||
app,
|
||||
query,
|
||||
limit=limit,
|
||||
sources=sources,
|
||||
user_id=request.user_id,
|
||||
**raw_options,
|
||||
),
|
||||
)
|
||||
try:
|
||||
# The whole answer, not only the search: the checks before and
|
||||
# after it used to wait for a thread with no limit at all, so a
|
||||
# pool held by stuck writes left recall hanging while /health
|
||||
# still answered (#231). Late is a clear 504, not silence.
|
||||
receipt = await asyncio.wait_for(
|
||||
recall_with_fallback(
|
||||
request,
|
||||
model_ready=lambda: _recall_model_ready(app),
|
||||
recall_memories=target.recall_async,
|
||||
has_memories=lambda: _library_has_memories(app, request.user_id),
|
||||
raw_sources=lambda: _recall_raw_sources(app, request, access, started),
|
||||
search_raw=lambda query, limit, sources: _search_sdk_raw(
|
||||
app,
|
||||
query,
|
||||
limit=limit,
|
||||
sources=sources,
|
||||
user_id=request.user_id,
|
||||
**raw_options,
|
||||
),
|
||||
),
|
||||
timeout=recall_total_timeout_seconds(request.timeout_ms),
|
||||
)
|
||||
except TimeoutError:
|
||||
logger.warning(
|
||||
"recall_timed_out timeout_ms=%d elapsed_ms=%.0f",
|
||||
request.timeout_ms,
|
||||
(time.monotonic() - started) * 1000,
|
||||
)
|
||||
raise HTTPException(status_code=504, detail="recall_timeout") from None
|
||||
event_store = getattr(app.state, "call_event_store", None)
|
||||
if event_store is not None and request.query != "__om_probe__":
|
||||
try:
|
||||
@@ -4422,7 +4590,7 @@ def create_app(
|
||||
)
|
||||
except Exception:
|
||||
app.state.call_event_recording_error = True
|
||||
return receipt
|
||||
return receipt.model_copy(update={"user_id": request.user_id})
|
||||
|
||||
@app.get(
|
||||
"/api/v1/memory-processing",
|
||||
@@ -4447,6 +4615,8 @@ def create_app(
|
||||
# running" is how a stalled queue looks busy, so the strand is given
|
||||
# its own number.
|
||||
running_rows = counts.get(MemoryTaskStatus.RUNNING, 0)
|
||||
backoff = cast(ModelBackoff | None, getattr(app.state, "model_backoff", None))
|
||||
backoff_state = backoff.state() if backoff is not None else None
|
||||
return MemoryProcessingSummary(
|
||||
pending=(
|
||||
sdk_counts.get("pending", 0)
|
||||
@@ -4487,6 +4657,13 @@ def create_app(
|
||||
metrics=ProcessingMetrics.model_validate(store.processing_metrics()),
|
||||
pipeline=_pipeline_status(app),
|
||||
embedding_switch=_embedding_switch_summary(app),
|
||||
suspended=sdk_counts.get("suspended", 0),
|
||||
llm_backoff_until=backoff_state.until if backoff_state is not None else None,
|
||||
llm_backoff_error=(
|
||||
backoff_state.last_error
|
||||
if backoff_state is not None and backoff_state.until is not None
|
||||
else None
|
||||
),
|
||||
)
|
||||
|
||||
@app.get(
|
||||
@@ -4565,7 +4742,8 @@ def create_app(
|
||||
"""Turn SDK features on or off; one global setting, not per user.
|
||||
|
||||
A key sets that feature, null resets it to the SDK default, an absent
|
||||
key is left as it is; an unknown key or a non-boolean is 422. A
|
||||
key is left as it is; an unknown key, a non-boolean feature or an
|
||||
`intention_dedup_threshold` outside its range is 422. A
|
||||
change rebuilds the SDK clients like a mode change: a write in flight
|
||||
finishes with the old features, the next job and search use the new
|
||||
ones. Nothing stored is migrated and no collection is renamed. See
|
||||
@@ -4581,7 +4759,7 @@ def create_app(
|
||||
current = _memory_features(app)
|
||||
before = effective_features(previous, defaults)
|
||||
after = effective_features(current, defaults)
|
||||
changed = tuple(spec.name for spec in FEATURES if before[spec.name] != after[spec.name])
|
||||
changed = tuple(name for name in SETTING_NAMES if before[name] != after[name])
|
||||
if current != previous:
|
||||
# Even a reset to the default value rebuilds: the client then
|
||||
# carries no override at all, byte for byte the default config.
|
||||
@@ -4616,9 +4794,9 @@ def create_app(
|
||||
async def put_ui_language(request: UiLanguageUpdate) -> UiLanguageChange:
|
||||
"""Record the App's display language; one global setting, not per user.
|
||||
|
||||
zh-Hans and null pass no language to the SDK, which keeps detecting
|
||||
zh/en as before; every other code (en included) is passed as
|
||||
`MemoryConfig.language`. A change of that SDK language rebuilds the
|
||||
Every code (zh-Hans and en included) is passed as
|
||||
`MemoryConfig.language`; only null passes no language, so the SDK
|
||||
keeps detecting zh/en as before. A change of that SDK language rebuilds the
|
||||
SDK clients like a feature change: a write in flight finishes in the
|
||||
old language. Nothing stored is migrated and no collection is renamed.
|
||||
"""
|
||||
@@ -4727,29 +4905,6 @@ def create_app(
|
||||
raise HTTPException(status_code=404, detail="evolve_run_not_found")
|
||||
return receipt
|
||||
|
||||
@app.post(
|
||||
"/api/v1/memory-processing/retry-unknown",
|
||||
response_model=RetryUnknownMemoryTasksReceipt,
|
||||
dependencies=protected,
|
||||
)
|
||||
async def retry_unknown_memory_tasks(
|
||||
request: RetryUnknownMemoryTasksRequest,
|
||||
) -> RetryUnknownMemoryTasksReceipt:
|
||||
"""Retry unknown writes only behind the user's explicit confirmation."""
|
||||
del request
|
||||
reconciled = await _sdk_memory_runtime(app).reconcile_unknown_outputs()
|
||||
# The startup queue knows the authority. A fresh legacy-mode queue
|
||||
# would recreate the job-evidence table the SDK cutover dropped.
|
||||
queue = app.state.memory_write_queue
|
||||
unknown = queue.unknown_jobs(limit=2048)
|
||||
if app.state.memory_authority is MemoryAuthority.LEGACY:
|
||||
for job in unknown:
|
||||
queue.resolve_unknown_from_linked_output(job.job_id)
|
||||
requeued = sum(queue.retry_unknown_confirmed(job.job_id) for job in unknown)
|
||||
if app.state.memory_authority is MemoryAuthority.LEGACY:
|
||||
requeued += app.state.memory_task_store.retry_unknown_confirmed()
|
||||
return RetryUnknownMemoryTasksReceipt(reconciled=reconciled, requeued=requeued)
|
||||
|
||||
@app.get(
|
||||
"/api/v1/graph-width",
|
||||
response_model=GraphWidthStatus,
|
||||
@@ -4916,7 +5071,7 @@ async def _require_external_recall_scope(
|
||||
# 299 rows from a covering index, about 52ms warm — enough, repeated and
|
||||
# serialised on the loop, to answer nobody for seconds.
|
||||
scope_started = time.monotonic()
|
||||
known = await asyncio.to_thread(_evidence_store(app).known_workspaces, user_id="local")
|
||||
known = await in_sdk_read(_evidence_store(app).known_workspaces, user_id="local")
|
||||
# Measured because nothing else records it. The ~52ms figure is from a copy
|
||||
# of one database at one point in time; this says what it costs here, on a
|
||||
# database that has since grown, often enough to show whether the read is
|
||||
@@ -5852,7 +6007,7 @@ async def _recall_model_ready(app: FastAPI) -> bool:
|
||||
if _memory_mode(app) == "lite":
|
||||
return False
|
||||
try:
|
||||
profile = await asyncio.to_thread(_model_profile_store(app).active, _secret_store(app))
|
||||
profile = await in_sdk_read(_model_profile_store(app).active, _secret_store(app))
|
||||
except Exception:
|
||||
return False
|
||||
return profile is not None and bool(profile.capabilities.model_available)
|
||||
@@ -5891,7 +6046,7 @@ async def _library_has_memories(app: FastAPI, user_id: str = DEFAULT_USER_ID) ->
|
||||
"""Any active memory at all. Asked only after an empty memory search."""
|
||||
try:
|
||||
client = await _sdk_client_handle(app).client()
|
||||
raw = await asyncio.to_thread(
|
||||
raw = await in_sdk_read(
|
||||
client.list_memories,
|
||||
user_id=user_id,
|
||||
statuses=["active"],
|
||||
@@ -5931,7 +6086,7 @@ async def _recall_raw_sources(
|
||||
timeout_ms=request.timeout_ms,
|
||||
)
|
||||
try:
|
||||
authority = await asyncio.to_thread(
|
||||
authority = await in_sdk_read(
|
||||
resolve_context_authority,
|
||||
query,
|
||||
access,
|
||||
@@ -5971,7 +6126,7 @@ async def _search_sdk_raw(
|
||||
if not include:
|
||||
return ()
|
||||
client = await _sdk_client_handle(app).client()
|
||||
raw = await asyncio.to_thread(
|
||||
raw = await in_sdk_read(
|
||||
client.search,
|
||||
query,
|
||||
**{
|
||||
@@ -6066,7 +6221,7 @@ def _ui_language_state(app: FastAPI) -> UiLanguageState:
|
||||
|
||||
|
||||
def _memory_features_state(
|
||||
app: FastAPI, defaults: dict[str, bool] | None = None
|
||||
app: FastAPI, defaults: dict[str, SettingValue] | None = None
|
||||
) -> MemoryFeaturesState:
|
||||
defaults = sdk_feature_defaults() if defaults is None else defaults
|
||||
overrides = _memory_features(app)
|
||||
@@ -6079,9 +6234,9 @@ def _memory_features_state(
|
||||
features.append(
|
||||
MemoryFeatureState(
|
||||
name=spec.name,
|
||||
enabled=effective[spec.name],
|
||||
configured=configured.get(spec.name),
|
||||
default=defaults[spec.name],
|
||||
enabled=bool(effective[spec.name]),
|
||||
configured=cast(bool | None, configured.get(spec.name)),
|
||||
default=bool(defaults[spec.name]),
|
||||
active=reason is None,
|
||||
reason=reason,
|
||||
side=spec.side,
|
||||
@@ -6093,10 +6248,32 @@ def _memory_features_state(
|
||||
description=spec.description,
|
||||
)
|
||||
)
|
||||
parameters = []
|
||||
for parameter in PARAMETERS:
|
||||
reason = feature_activity(parameter.name, effective, mode)
|
||||
parameters.append(
|
||||
MemoryParameterState(
|
||||
name=parameter.name,
|
||||
value=float(effective[parameter.name]),
|
||||
configured=cast(float | None, configured.get(parameter.name)),
|
||||
default=float(defaults[parameter.name]),
|
||||
minimum=parameter.minimum,
|
||||
maximum=parameter.maximum,
|
||||
active=reason is None,
|
||||
reason=reason,
|
||||
requires_feature=parameter.requires_feature,
|
||||
sdk_field=parameter.sdk_field,
|
||||
env=parameter.env,
|
||||
description=parameter.description,
|
||||
)
|
||||
)
|
||||
return MemoryFeaturesState(
|
||||
mode=mode,
|
||||
features=tuple(features),
|
||||
non_default=cast(dict[MemoryFeatureName, bool], non_default_features(overrides, defaults)),
|
||||
parameters=tuple(parameters),
|
||||
non_default=cast(
|
||||
dict[MemorySettingName, SettingValue], non_default_features(overrides, defaults)
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@@ -6117,7 +6294,7 @@ def _pipeline_status(app: FastAPI) -> PipelineStatus | None:
|
||||
last_evolve_at=max(finished, default=None),
|
||||
graph_build=state.graph_build,
|
||||
features=cast(
|
||||
dict[MemoryFeatureName, bool],
|
||||
dict[MemorySettingName, SettingValue],
|
||||
non_default_features(_memory_features(app), sdk_feature_defaults()),
|
||||
),
|
||||
**_ui_language_state(app).model_dump(exclude={"languages"}),
|
||||
@@ -6160,7 +6337,7 @@ async def _get_sdk_memory(
|
||||
variant: str | None = None,
|
||||
) -> HySdkMemoryRecord | None:
|
||||
client = await handle.client()
|
||||
raw = await asyncio.to_thread(client.get, memory_id, **_namespace_kwargs(kind, variant))
|
||||
raw = await in_sdk_read(client.get, memory_id, **_namespace_kwargs(kind, variant))
|
||||
return None if raw is None else HySdkMemoryRecord.from_sdk(raw)
|
||||
|
||||
|
||||
@@ -6322,7 +6499,7 @@ async def _search_sdk_memories(
|
||||
) -> tuple[object, ...]:
|
||||
client = await handle.client()
|
||||
overrides = options.search_overrides() if options is not None else {}
|
||||
raw = await asyncio.to_thread(
|
||||
raw = await in_sdk_read(
|
||||
client.search,
|
||||
query,
|
||||
**{
|
||||
@@ -6398,6 +6575,13 @@ def _user_evolving(app: FastAPI, user_id: str) -> bool:
|
||||
return bool(callable(evolving) and evolving(user_id))
|
||||
|
||||
|
||||
def _reset_model_backoff(app: FastAPI) -> None:
|
||||
"""The user changed the model configuration: end any provider pause now."""
|
||||
backoff = getattr(app.state, "model_backoff", None)
|
||||
if isinstance(backoff, ModelBackoff):
|
||||
backoff.reset()
|
||||
|
||||
|
||||
def _embedding_switch(app: FastAPI) -> EmbeddingSwitchService:
|
||||
return cast(EmbeddingSwitchService, app.state.embedding_switch)
|
||||
|
||||
|
||||
@@ -15,6 +15,11 @@ BULK_WORKERS = 3
|
||||
SNAPSHOT_WORKERS = 8
|
||||
INTERACTIVE_READ_WORKERS = 4
|
||||
SOURCE_READER_WORKERS = 4
|
||||
# One per source the drain runs side by side (`max_concurrent_sources`), and
|
||||
# two over for the outcome inquiries that follow them.
|
||||
SDK_WRITE_WORKERS = 12
|
||||
# Recall's SDK reads: one search and the `get`s behind it (`_SDK_GET_CONCURRENCY`).
|
||||
SDK_READ_WORKERS = 8
|
||||
|
||||
_pool: ThreadPoolExecutor | None = None
|
||||
_snapshot_pool: ThreadPoolExecutor | None = None
|
||||
@@ -22,6 +27,7 @@ _interactive_pool: ThreadPoolExecutor | None = None
|
||||
_raw_read_pool: ThreadPoolExecutor | None = None
|
||||
_memory_read_pool: ThreadPoolExecutor | None = None
|
||||
_source_reader_pool: ThreadPoolExecutor | None = None
|
||||
_sdk_write_pool: ThreadPoolExecutor | None = None
|
||||
_warmed_readers: weakref.WeakSet[ThreadPoolExecutor] = weakref.WeakSet()
|
||||
_reader_warm_lock = threading.Lock()
|
||||
_interactive_reads = 0
|
||||
@@ -71,15 +77,38 @@ def raw_read_pool() -> ThreadPoolExecutor:
|
||||
|
||||
|
||||
def memory_read_pool() -> ThreadPoolExecutor:
|
||||
"""Recall's own threads: SDK searches, gets, and what decides between them.
|
||||
|
||||
Recall used to hand these to the default pool, which SDK writes share. A
|
||||
write that times out keeps its thread until the SDK returns (#231), so a
|
||||
few stuck writes left recall waiting for a thread while `/health`, which
|
||||
needs none, still answered.
|
||||
"""
|
||||
global _memory_read_pool
|
||||
if _memory_read_pool is None:
|
||||
_memory_read_pool = ThreadPoolExecutor(
|
||||
max_workers=INTERACTIVE_READ_WORKERS,
|
||||
max_workers=SDK_READ_WORKERS,
|
||||
thread_name_prefix="one-memory-memory-read",
|
||||
)
|
||||
return _memory_read_pool
|
||||
|
||||
|
||||
def sdk_write_pool() -> ThreadPoolExecutor:
|
||||
"""SDK writes, bounded and apart from everything else.
|
||||
|
||||
Each thread here only waits for the SDK's own loop thread to answer
|
||||
(`_LoopThread.run`), so any thread outside that loop may make the call; a
|
||||
stuck write therefore costs a thread of this pool and nobody else's.
|
||||
"""
|
||||
global _sdk_write_pool
|
||||
if _sdk_write_pool is None:
|
||||
_sdk_write_pool = ThreadPoolExecutor(
|
||||
max_workers=SDK_WRITE_WORKERS,
|
||||
thread_name_prefix="one-memory-sdk-write",
|
||||
)
|
||||
return _sdk_write_pool
|
||||
|
||||
|
||||
def source_reader_pool() -> ThreadPoolExecutor:
|
||||
"""Bounded process-wide pool for source iteration and parsing."""
|
||||
global _source_reader_pool
|
||||
@@ -175,6 +204,30 @@ async def in_interactive_read[T](
|
||||
return await in_read_pool(interactive_read_pool(), work, *args, **kwargs)
|
||||
|
||||
|
||||
async def in_sdk_read[T](
|
||||
work: Callable[..., T],
|
||||
/,
|
||||
*args: object,
|
||||
**kwargs: object,
|
||||
) -> T:
|
||||
"""A recall-path read, in recall's own pool (`memory_read_pool`)."""
|
||||
return await in_read_pool(memory_read_pool(), work, *args, **kwargs)
|
||||
|
||||
|
||||
async def in_sdk_write[T](
|
||||
work: Callable[..., T],
|
||||
/,
|
||||
*args: object,
|
||||
**kwargs: object,
|
||||
) -> T:
|
||||
"""One SDK write call, in the write pool (`sdk_write_pool`)."""
|
||||
return await asyncio.get_running_loop().run_in_executor(
|
||||
sdk_write_pool(),
|
||||
copy_context().run,
|
||||
partial(work, *args, **kwargs),
|
||||
)
|
||||
|
||||
|
||||
def _retrieve[T](future: "asyncio.Future[T]") -> None:
|
||||
"""Mark a shielded read's outcome as seen.
|
||||
|
||||
@@ -261,7 +314,10 @@ def _warm_reader_pool(reader: ThreadPoolExecutor) -> None:
|
||||
|
||||
def shutdown_bulk_pool() -> None:
|
||||
global _pool, _snapshot_pool, _interactive_pool, _raw_read_pool, _memory_read_pool
|
||||
global _source_reader_pool
|
||||
global _source_reader_pool, _sdk_write_pool
|
||||
if _sdk_write_pool is not None:
|
||||
_sdk_write_pool.shutdown(wait=False, cancel_futures=True)
|
||||
_sdk_write_pool = None
|
||||
if _pool is not None:
|
||||
_pool.shutdown(wait=False, cancel_futures=True)
|
||||
_pool = None
|
||||
|
||||
@@ -155,6 +155,8 @@ class SdkContextSearchResponse(BaseModel):
|
||||
used_characters: int = Field(ge=0, le=20000)
|
||||
truncated: bool
|
||||
total_ms: float = Field(ge=0)
|
||||
#: Whose memories were searched: the request's `user_id`, else `local`.
|
||||
user_id: str = Field(min_length=1, max_length=64)
|
||||
|
||||
|
||||
class ContextLaneResult(BaseModel):
|
||||
|
||||
@@ -261,6 +261,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
used_characters=0,
|
||||
truncated=True,
|
||||
total_ms=(time.monotonic() - started) * 1000,
|
||||
user_id=value.user_id,
|
||||
)
|
||||
await record(final, timed_out=True)
|
||||
return final
|
||||
@@ -306,6 +307,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
used_characters=used,
|
||||
truncated=truncated,
|
||||
total_ms=(time.monotonic() - started) * 1000,
|
||||
user_id=value.user_id,
|
||||
)
|
||||
await record(final, timed_out=False)
|
||||
return final
|
||||
|
||||
@@ -111,10 +111,26 @@ MAIN_DATABASE_CHECKPOINT_WRITE_INTERVAL = 64
|
||||
MAIN_DATABASE_CHECKPOINT_SECONDS = 30.0
|
||||
|
||||
# Advisory only: PASSIVE cannot force a frame past a long reader. Crossing this
|
||||
# threshold is a signal that the WAL is not converging and that read snapshots
|
||||
# or write volume need investigation.
|
||||
# threshold with frames still waiting to be copied back is a signal that the
|
||||
# WAL is not converging and that read snapshots or write volume need
|
||||
# investigation. A large file whose frames are all checkpointed is not: the
|
||||
# idle TRUNCATE below gives that space back.
|
||||
MAIN_DATABASE_WAL_WARN_BYTES = 512 * 1024 * 1024
|
||||
|
||||
# What a WAL file is cut back to once a checkpoint has copied every frame and
|
||||
# the writer starts it over. Without a limit SQLite reuses the file in place
|
||||
# and never shrinks it, so one import peak left a 639 MB WAL behind for good
|
||||
# (#245). 64 MiB is about sixteen times what the default autocheckpoint (1000
|
||||
# pages) lets build up between checkpoints, so ordinary write bursts reuse the
|
||||
# file without growing and shrinking it each time, and it is still a tenth of
|
||||
# what that user's disk was left holding.
|
||||
MAIN_DATABASE_JOURNAL_SIZE_LIMIT_BYTES = 64 * 1024 * 1024
|
||||
|
||||
# How long an idle TRUNCATE may wait for the writer or for a reader to leave.
|
||||
# It runs only when nothing holds the database; anyone arriving in the
|
||||
# meantime wins, and the checkpoint is tried again on the next pass.
|
||||
MAIN_DATABASE_IDLE_CHECKPOINT_WAIT_MS = 50
|
||||
|
||||
|
||||
def checkpoint_write_interval() -> int:
|
||||
raw = os.environ.get(
|
||||
@@ -276,6 +292,8 @@ class MainDatabase:
|
||||
self._writes_since_checkpoint = 0
|
||||
self._last_checkpoint_at = time.monotonic()
|
||||
self._wal_warned_at = 0.0
|
||||
#: Learned from the first checkpoint; turns WAL frames into bytes.
|
||||
self._page_size = 4096
|
||||
self._reader_started_at: dict[int, float] = {}
|
||||
|
||||
@contextmanager
|
||||
@@ -542,16 +560,59 @@ class MainDatabase:
|
||||
def _checkpoint_from_maintenance(self, mode: str) -> tuple[int, int, int]:
|
||||
return self._checkpoint(mode, maintenance=True)
|
||||
|
||||
def _checkpoint(self, mode: str, *, maintenance: bool) -> tuple[int, int, int]:
|
||||
def _checkpoint(
|
||||
self,
|
||||
mode: str,
|
||||
*,
|
||||
maintenance: bool,
|
||||
busy_timeout_ms: int = WRITE_LOCK_WAIT_MS,
|
||||
) -> tuple[int, int, int]:
|
||||
normalized = mode.strip().upper()
|
||||
if normalized not in {"PASSIVE", "FULL", "RESTART", "TRUNCATE"}:
|
||||
raise ValueError("unsupported WAL checkpoint mode")
|
||||
with self.write(autocommit=True, maintenance=maintenance) as connection:
|
||||
with self.write(
|
||||
autocommit=True, maintenance=maintenance, busy_timeout_ms=busy_timeout_ms
|
||||
) as connection:
|
||||
row = connection.execute(f"PRAGMA wal_checkpoint({normalized})").fetchone()
|
||||
page_size = connection.execute("PRAGMA page_size").fetchone()
|
||||
if row is None or len(row) != 3:
|
||||
raise sqlite3.DatabaseError("SQLite returned an invalid checkpoint result")
|
||||
if page_size is not None:
|
||||
self._page_size = int(page_size[0])
|
||||
return int(row[0]), int(row[1]), int(row[2])
|
||||
|
||||
def idle(self) -> bool:
|
||||
"""Nobody holds or waits for a connection to this database right now."""
|
||||
with self._state:
|
||||
return (
|
||||
self._writer_owner is None
|
||||
and not self._reader_holders
|
||||
and not len(self._write_line)
|
||||
and not len(self._read_line)
|
||||
)
|
||||
|
||||
def checkpoint_if_idle(self) -> tuple[int, int, int] | None:
|
||||
"""Give the WAL file back to the disk while nothing is using the database.
|
||||
|
||||
PASSIVE copies frames back but leaves the file its size; only TRUNCATE
|
||||
empties it, and TRUNCATE has to wait for every reader to finish, so it
|
||||
is attempted only when this process holds no checkout at all, and with
|
||||
a wait of a few milliseconds. A busy result, or someone arriving in
|
||||
between, is not a failure: the next idle pass tries again. Returns the
|
||||
checkpoint result, or None when it was not attempted or not admitted.
|
||||
"""
|
||||
if not self.idle() or self._wal_bytes() == 0:
|
||||
return None
|
||||
try:
|
||||
return self._checkpoint(
|
||||
"TRUNCATE",
|
||||
maintenance=True,
|
||||
busy_timeout_ms=MAIN_DATABASE_IDLE_CHECKPOINT_WAIT_MS,
|
||||
)
|
||||
except sqlite3.Error:
|
||||
logger.debug("main_database_idle_checkpoint_skipped file=%s", self.path.name)
|
||||
return None
|
||||
|
||||
def _maybe_checkpoint_after_write(self) -> None:
|
||||
"""Checkpoint on a write cadence without opening another connection.
|
||||
|
||||
@@ -579,17 +640,23 @@ class MainDatabase:
|
||||
logger.warning("main_database_checkpoint_failed file=%s", self.path.name, exc_info=True)
|
||||
return
|
||||
wal_bytes = self._wal_bytes()
|
||||
if wal_bytes >= MAIN_DATABASE_WAL_WARN_BYTES:
|
||||
# Frames the checkpoint could not copy back yet: what a reader is
|
||||
# holding up. The file size alone also counts space already
|
||||
# checkpointed and only waiting to be reused or truncated, which is
|
||||
# not the WAL failing to converge.
|
||||
pending_bytes = max(0, log_frames - checkpointed) * self._page_size
|
||||
if pending_bytes >= MAIN_DATABASE_WAL_WARN_BYTES:
|
||||
with self._state:
|
||||
should_warn = now - self._wal_warned_at >= MAIN_DATABASE_REPORT_SECONDS
|
||||
if should_warn:
|
||||
self._wal_warned_at = now
|
||||
if should_warn:
|
||||
logger.warning(
|
||||
"main_database_wal_high file=%s wal_bytes=%d busy=%d "
|
||||
"main_database_wal_high file=%s wal_bytes=%d pending_bytes=%d busy=%d "
|
||||
"log_frames=%d checkpointed=%d readers_in_use=%d",
|
||||
self.path.name,
|
||||
wal_bytes,
|
||||
pending_bytes,
|
||||
busy,
|
||||
log_frames,
|
||||
checkpointed,
|
||||
@@ -784,6 +851,10 @@ class MainDatabase:
|
||||
try:
|
||||
if wal:
|
||||
enable_write_ahead_log(connection, wait_ms=busy_timeout_ms)
|
||||
# Only the writer restarts the WAL, so only the writer needs the cap.
|
||||
connection.execute(
|
||||
f"PRAGMA journal_size_limit = {int(MAIN_DATABASE_JOURNAL_SIZE_LIMIT_BYTES)}"
|
||||
)
|
||||
_seal(connection)
|
||||
if busy_timeout_ms != WRITE_LOCK_WAIT_MS:
|
||||
# Opened at the caller's wait; every checkout starts at the default.
|
||||
@@ -1027,6 +1098,18 @@ def main_database_transaction(
|
||||
yield connection
|
||||
|
||||
|
||||
def checkpoint_idle_main_databases() -> int:
|
||||
"""TRUNCATE the WAL of every database nobody is using; how many were emptied."""
|
||||
with _databases_lock:
|
||||
databases = list(_alive.values())
|
||||
emptied = 0
|
||||
for database in databases:
|
||||
result = database.checkpoint_if_idle()
|
||||
if result is not None and result[0] == 0:
|
||||
emptied += 1
|
||||
return emptied
|
||||
|
||||
|
||||
def close_main_databases() -> None:
|
||||
"""Close every connection this process holds; used when the Core stops."""
|
||||
with _databases_lock:
|
||||
@@ -1050,6 +1133,9 @@ async def report_main_database(seconds: float) -> None:
|
||||
while True:
|
||||
await asyncio.sleep(seconds)
|
||||
log_main_database_stats("periodic")
|
||||
# The idle signal is this process's own checkouts: none held, none
|
||||
# waiting. Off the loop, since a checkpoint is disk work however short.
|
||||
await asyncio.to_thread(checkpoint_idle_main_databases)
|
||||
|
||||
|
||||
def main_database_stats() -> dict[str, float | int]:
|
||||
|
||||
@@ -14,7 +14,7 @@ from pathlib import Path
|
||||
from typing import Literal, cast
|
||||
|
||||
HY_MEMORY_DISTRIBUTION = "hy-memory"
|
||||
HY_MEMORY_VERSION = "2.0.11"
|
||||
HY_MEMORY_VERSION = "2.0.12"
|
||||
HY_MEMORY_NAMESPACE = "governed-v2"
|
||||
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
|
||||
|
||||
|
||||
@@ -503,7 +503,7 @@ def _ensure_coding_schema(path: Path) -> None:
|
||||
CodingMemoryStore,
|
||||
)
|
||||
|
||||
# Relies on the pinned SDK's internals (hy-memory==2.0.11 is pinned exactly).
|
||||
# Relies on the pinned SDK's internals (hy-memory==2.0.12 is pinned exactly).
|
||||
store = CodingMemoryStore.__new__(CodingMemoryStore)
|
||||
store._db_path = str(path)
|
||||
store._conn = None
|
||||
|
||||
@@ -47,8 +47,10 @@ from one_memory.local_embedding import LocalEmbedder
|
||||
from one_memory.log_file import confine_hy_sdk_logging
|
||||
from one_memory.memory_features import (
|
||||
FEATURES,
|
||||
FEATURES_BY_NAME,
|
||||
PARAMETERS,
|
||||
SETTINGS_BY_NAME,
|
||||
FeatureOverrides,
|
||||
SettingValue,
|
||||
feature_overrides,
|
||||
)
|
||||
from one_memory.model_profiles import (
|
||||
@@ -321,21 +323,27 @@ def sdk_relevance_floor() -> float:
|
||||
return min(1.0, max(0.0, value))
|
||||
|
||||
|
||||
def sdk_feature_defaults() -> dict[str, bool]:
|
||||
"""Each `memory_features` feature's SDK default, as a fresh config has it.
|
||||
def sdk_feature_defaults() -> dict[str, SettingValue]:
|
||||
"""Each `memory_features` setting's SDK default, as a fresh config has it.
|
||||
|
||||
That is the SDK's code default unless the process environment sets the
|
||||
SDK's own variable (`FeatureSpec.env`), exactly what an unset feature
|
||||
leaves on a client.
|
||||
"""
|
||||
config = MemoryConfig()
|
||||
return {
|
||||
defaults: dict[str, SettingValue] = {
|
||||
spec.name: bool(getattr(getattr(config, spec.section), spec.field)) for spec in FEATURES
|
||||
}
|
||||
for parameter in PARAMETERS:
|
||||
defaults[parameter.name] = float(
|
||||
getattr(getattr(config, parameter.section), parameter.field)
|
||||
)
|
||||
return defaults
|
||||
|
||||
|
||||
def apply_sdk_features(
|
||||
config: MemoryConfig, features: Mapping[str, bool] | Iterable[tuple[str, bool]]
|
||||
config: MemoryConfig,
|
||||
features: Mapping[str, SettingValue] | Iterable[tuple[str, SettingValue]],
|
||||
) -> MemoryConfig:
|
||||
"""Set each feature a person chose on its `MemoryConfig` field.
|
||||
|
||||
@@ -346,7 +354,7 @@ def apply_sdk_features(
|
||||
"""
|
||||
values = dict(features.items() if isinstance(features, Mapping) else features)
|
||||
for name, value in feature_overrides(values):
|
||||
spec = FEATURES_BY_NAME[name]
|
||||
spec = SETTINGS_BY_NAME[name]
|
||||
setattr(getattr(config, spec.section), spec.field, value)
|
||||
return config
|
||||
|
||||
@@ -367,8 +375,8 @@ def language_kwargs(language: str | None) -> LanguageKwargs:
|
||||
def apply_sdk_language(config: MemoryConfig, language: str | None) -> MemoryConfig:
|
||||
"""Set the SDK prompt language (`MemoryConfig.language`), when there is one.
|
||||
|
||||
None (zh-Hans, or no choice) sets nothing: the SDK detects zh/en as it
|
||||
always did. Like a feature, the language never touches `vector_store`, so
|
||||
None (no language set) sets nothing: the SDK detects zh/en as it always
|
||||
did. Every display language, zh-Hans included, is set explicitly. Like a feature, the language never touches `vector_store`, so
|
||||
the collection identity never depends on it.
|
||||
"""
|
||||
if language is not None:
|
||||
@@ -383,7 +391,7 @@ def build_sdk_config(
|
||||
*,
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
features: Mapping[str, bool] | Iterable[tuple[str, bool]] = (),
|
||||
features: Mapping[str, SettingValue] | Iterable[tuple[str, SettingValue]] = (),
|
||||
language: str | None = None,
|
||||
) -> MemoryConfig:
|
||||
"""Map a model snapshot and its embedding space to an SDK configuration.
|
||||
|
||||
@@ -436,6 +436,11 @@ def root_identity_matches(root: Path, expected: str) -> bool:
|
||||
return _root_identity_matches_details(resolved, details, expected)
|
||||
|
||||
|
||||
def resolved_source_root(root: Path) -> Path:
|
||||
"""The real directory a root names (symlinks followed), as a grant records it."""
|
||||
return _root_details(root)[0]
|
||||
|
||||
|
||||
def _check_root_identity(root: Path, expected: str | None) -> None:
|
||||
if expected is not None and not root_identity_matches(root, expected):
|
||||
raise SourceAccessError(SourceAccessErrorCode.SOURCE_IDENTITY_CHANGED)
|
||||
|
||||
@@ -27,7 +27,7 @@ def discovered_off_loop(
|
||||
*,
|
||||
suffixes: frozenset[str],
|
||||
max_targets: int,
|
||||
excluded_top_directories: frozenset[str] = frozenset(),
|
||||
excluded_directory_names: frozenset[str] = frozenset(),
|
||||
) -> AsyncIterator[SourceFileDiscoveryEvent]:
|
||||
"""The same walk, for callers that have an event loop to keep free.
|
||||
|
||||
@@ -42,7 +42,7 @@ def discovered_off_loop(
|
||||
grant,
|
||||
suffixes=suffixes,
|
||||
max_targets=max_targets,
|
||||
excluded_top_directories=excluded_top_directories,
|
||||
excluded_directory_names=excluded_directory_names,
|
||||
),
|
||||
scope_key=grant.source_id,
|
||||
)
|
||||
@@ -53,15 +53,17 @@ def discover_source_files(
|
||||
*,
|
||||
suffixes: frozenset[str],
|
||||
max_targets: int,
|
||||
excluded_top_directories: frozenset[str] = frozenset(),
|
||||
excluded_directory_names: frozenset[str] = frozenset(),
|
||||
) -> Iterator[SourceFileDiscoveryEvent]:
|
||||
"""Metadata only: one lstat per candidate, no open and no per-file grant lookup.
|
||||
|
||||
The grant and root identity are checked once before the walk. Each listed
|
||||
file keeps the grant's check, which runs again when that file is opened;
|
||||
the open itself refuses symlinks and any inode other than the one listed.
|
||||
`excluded_top_directories` are directories directly under the root that
|
||||
are never entered, such as another user's Hermes profiles.
|
||||
`excluded_directory_names` are directory names never entered, at any
|
||||
depth under the root, such as the `profiles` container that holds other
|
||||
users' Hermes profiles. Any depth, because the granted root may sit above
|
||||
the Hermes home (`~/.hermes` inside it is then `.hermes/profiles`).
|
||||
"""
|
||||
grant.check_root()
|
||||
inspected = 0
|
||||
@@ -69,12 +71,10 @@ def discover_source_files(
|
||||
grant.root,
|
||||
followlinks=False,
|
||||
):
|
||||
at_root = Path(directory) == grant.root
|
||||
child_directories[:] = sorted(
|
||||
child
|
||||
for child in child_directories
|
||||
if not (Path(directory) / child).is_symlink()
|
||||
and not (at_root and child in excluded_top_directories)
|
||||
if not (Path(directory) / child).is_symlink() and child not in excluded_directory_names
|
||||
)
|
||||
for filename in sorted(filenames):
|
||||
if Path(filename).suffix.lower() not in suffixes:
|
||||
|
||||
@@ -209,6 +209,7 @@ class JsonlSourceAdapter:
|
||||
file_predicate: Callable[[AuthorizedSourceFile], bool] | None = None,
|
||||
suffixes: frozenset[str] = frozenset({".jsonl"}),
|
||||
header_selector: Callable[[dict[str, JsonValue]], bool] | None = None,
|
||||
header_gate: Callable[[dict[str, JsonValue] | None], bool] | None = None,
|
||||
) -> None:
|
||||
if descriptor.source_id != grant.source_id:
|
||||
raise ValueError("Descriptor and authorized root source IDs must match")
|
||||
@@ -219,6 +220,10 @@ class JsonlSourceAdapter:
|
||||
self.file_predicate = file_predicate or (lambda _: True)
|
||||
self.suffixes = suffixes
|
||||
self.header_selector = header_selector
|
||||
# False refuses the whole file before any record is read. It reports
|
||||
# and leaves no cursor, so a release that learns the format reads the
|
||||
# file from the top instead of finding it already behind a checkpoint.
|
||||
self.header_gate = header_gate
|
||||
self._header_cache: dict[str, tuple[str, dict[str, JsonValue] | None]] = {}
|
||||
self._history_listing_cache: SourceListing | None = None
|
||||
|
||||
@@ -371,6 +376,9 @@ class JsonlSourceAdapter:
|
||||
source_file, checkpoint, header = await asyncio.to_thread(
|
||||
self._prepare, listed, options
|
||||
)
|
||||
if self.header_gate is not None and not self.header_gate(header):
|
||||
yield ScanFailure(reason_code=ScanIssueCode.UNSUPPORTED_SCHEMA)
|
||||
continue
|
||||
_log.debug(
|
||||
"import_file_read source=%s file=%d/%d file_bytes=%d planned_bytes=%d",
|
||||
self.descriptor.source_id,
|
||||
@@ -574,6 +582,9 @@ class JsonlSourceAdapter:
|
||||
# resumed scan starts past the header, and the records after it need
|
||||
# it just as much as the first batch did.
|
||||
header = await _read_header(reader, source_file, self.header_selector)
|
||||
if self.header_gate is not None and not self.header_gate(header):
|
||||
yield ScanFailure(reason_code=ScanIssueCode.UNSUPPORTED_SCHEMA)
|
||||
continue
|
||||
last: SourceCursor = _start_cursor(source_file, options, checkpoint)
|
||||
async for raw in iterate_in_worker(
|
||||
partial(reader, source_file, checkpoint=checkpoint),
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import re
|
||||
from pathlib import PurePosixPath
|
||||
|
||||
from pydantic import JsonValue
|
||||
@@ -37,6 +38,19 @@ DEEPSEEK_HARNESS_DESCRIPTOR = SourceDescriptor(
|
||||
),
|
||||
)
|
||||
|
||||
#: One session is one file, and the name carries the format generation it was
|
||||
#: written in: `session.jsonl[.zstd]` for the first format, then
|
||||
#: `session.v3.jsonl.zstd` and `session.v4.jsonl.zstd`. The version segment is
|
||||
#: part of the published on-disk contract, so discovery must accept it — the
|
||||
#: earlier exact-name match imported none of the versioned files.
|
||||
SESSION_FILE = re.compile(r"^session(?:\.v[0-9]+)?\.jsonl(?:\.zstd)?$")
|
||||
|
||||
#: Session generations this parser has been verified against: the first format
|
||||
#: (header `version: 0`), v3 and v4. Discovery is intentionally wider than
|
||||
#: parsing: a future generation is found by pattern and then skipped from its
|
||||
#: header, so it is reported as unsupported instead of silently vanishing.
|
||||
SUPPORTED_SESSION_VERSIONS = frozenset({0, 3, 4})
|
||||
|
||||
|
||||
def create_deepseek_harness_adapter(
|
||||
grant: AuthorizedSourceRoot,
|
||||
@@ -49,25 +63,43 @@ def create_deepseek_harness_adapter(
|
||||
grant=grant,
|
||||
parser=parse_deepseek_harness_record,
|
||||
source_version=source_version,
|
||||
file_predicate=lambda source_file: (
|
||||
source_file.relative_path.name in {"session.jsonl", "session.jsonl.zstd"}
|
||||
),
|
||||
file_predicate=lambda source_file: bool(SESSION_FILE.match(source_file.relative_path.name)),
|
||||
suffixes=frozenset({".jsonl", ".zstd"}),
|
||||
header_selector=_is_session_header,
|
||||
header_gate=_supported_session,
|
||||
)
|
||||
|
||||
|
||||
def _is_session_header(record: dict[str, JsonValue]) -> bool:
|
||||
"""Harness writes the session's `cwd` once, in the first line only.
|
||||
|
||||
Verified against a session this machine produced: the header is
|
||||
`{"type":"session","version":0,"id":...,"cwd":"/private/tmp/dsh-verify"}`
|
||||
and no later record repeats it. Read per-record, `cwd` is therefore always
|
||||
absent and every turn imports with no workspace.
|
||||
Verified against sessions this machine produced: the header is
|
||||
`{"type":"session","version":0|4,"id":...,"cwd":...}` and no later record
|
||||
repeats it. Read per-record, `cwd` is therefore always absent and every turn
|
||||
imports with no workspace.
|
||||
"""
|
||||
return record.get("type") == "session"
|
||||
|
||||
|
||||
def _supported_session(header: dict[str, JsonValue] | None) -> bool:
|
||||
"""Whether the session's declared format generation is one we can read.
|
||||
|
||||
A missing header, or a header without `version`, is the first format — the
|
||||
earliest files this adapter already read. Any declared version outside
|
||||
`SUPPORTED_SESSION_VERSIONS` is refused rather than interpreted with today's
|
||||
field rules: a future format may change field semantics, and importing it
|
||||
wrongly is harder to undo than not importing it yet.
|
||||
"""
|
||||
if header is None:
|
||||
return True
|
||||
version = header.get("version")
|
||||
if version is None:
|
||||
return True
|
||||
if not isinstance(version, int) or isinstance(version, bool):
|
||||
return False
|
||||
return version in SUPPORTED_SESSION_VERSIONS
|
||||
|
||||
|
||||
def parse_deepseek_harness_record(
|
||||
record: dict[str, JsonValue],
|
||||
context: SourceRecordContext,
|
||||
@@ -80,6 +112,11 @@ def parse_deepseek_harness_record(
|
||||
)
|
||||
event_type = record.get("type")
|
||||
data = as_record(record.get("data"))
|
||||
if not _supported_session(context.header):
|
||||
return ParsedSkip(
|
||||
reason_code=ScanIssueCode.UNSUPPORTED_SCHEMA,
|
||||
conversation_id=conversation_id,
|
||||
)
|
||||
if event_type == "session":
|
||||
return ParsedSkip(reason_code=ScanIssueCode.UNKNOWN_RECORD)
|
||||
if data is None:
|
||||
@@ -148,11 +185,18 @@ def _tool_message(
|
||||
"""`tool/call` carries name and arguments; `tool/result` a user-role message.
|
||||
|
||||
The same call also appears as a `tool-call` block in `assistant/message`;
|
||||
only the event is read, so it is stored once.
|
||||
only the event is read, so it is stored once. The first format and v3 wrap
|
||||
the result in `tool-result` blocks; v4 writes plain `text` blocks, so both
|
||||
shapes are read through the same clipping and credential-scrubbing path.
|
||||
"""
|
||||
message = as_record(data.get("message"))
|
||||
if record.get("type") == "tool/result" and message is not None:
|
||||
text = tool_blocks_evidence(message.get("content"), block_types=frozenset({"tool-result"}))
|
||||
if text is None:
|
||||
text = tool_blocks_evidence(
|
||||
[{"type": "tool-result", "content": message.get("content")}],
|
||||
block_types=frozenset({"tool-result"}),
|
||||
)
|
||||
message_id = first_string(message.get("id"), data.get("callId"))
|
||||
else:
|
||||
text = tool_evidence_text(data)
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
import json
|
||||
import re
|
||||
import sqlite3
|
||||
from collections.abc import AsyncIterator, Iterator
|
||||
from collections.abc import AsyncIterator, Iterable, Iterator
|
||||
from functools import partial
|
||||
from pathlib import PurePosixPath
|
||||
from pathlib import Path, PurePosixPath
|
||||
|
||||
from pydantic import JsonValue
|
||||
|
||||
@@ -88,7 +88,7 @@ class HermesSourceAdapter:
|
||||
self.grant,
|
||||
suffixes=frozenset({".jsonl", ".db"}),
|
||||
max_targets=10_000,
|
||||
excluded_top_directories=frozenset({HERMES_PROFILES_DIRECTORY}),
|
||||
excluded_directory_names=frozenset({HERMES_PROFILES_DIRECTORY}),
|
||||
):
|
||||
if isinstance(item, AuthorizedSourceFile) and _is_hermes_target(
|
||||
item.relative_path.as_posix()
|
||||
@@ -102,7 +102,7 @@ class HermesSourceAdapter:
|
||||
self.grant,
|
||||
suffixes=frozenset({".jsonl", ".db"}),
|
||||
max_targets=options.max_scan_targets or 100_000,
|
||||
excluded_top_directories=frozenset({HERMES_PROFILES_DIRECTORY}),
|
||||
excluded_directory_names=frozenset({HERMES_PROFILES_DIRECTORY}),
|
||||
):
|
||||
yield ScanProgressEvent(
|
||||
phase=ScanPhase.DISCOVER,
|
||||
@@ -253,16 +253,62 @@ def parse_hermes_rollout_record(
|
||||
#: own `state.db` and sessions. A profile is another person's history: it is
|
||||
#: read only through its own grant (its own MemHub user), never as part of the
|
||||
#: root above it. Hermes' own export skips this directory for the same reason.
|
||||
#:
|
||||
#: Skipped at any depth, not only directly under the root: a root granted
|
||||
#: above the Hermes home holds `.hermes/profiles/`. Hermes keeps nothing of
|
||||
#: its own under a directory of that name (its history is `state.db` and
|
||||
#: `sessions/`), so nothing real is lost.
|
||||
HERMES_PROFILES_DIRECTORY = "profiles"
|
||||
|
||||
|
||||
def _is_hermes_target(relative_path: str) -> bool:
|
||||
path = PurePosixPath(relative_path)
|
||||
if path.parts and path.parts[0] == HERMES_PROFILES_DIRECTORY:
|
||||
if HERMES_PROFILES_DIRECTORY in path.parts[:-1]:
|
||||
return False
|
||||
return path.name == "state.db" or (path.suffix == ".jsonl" and "sessions" in path.parts)
|
||||
|
||||
|
||||
def hermes_root_is_profiles_container(root: Path) -> bool:
|
||||
"""`profiles` itself holds everyone's profiles; it is never one person's history."""
|
||||
return root.name == HERMES_PROFILES_DIRECTORY
|
||||
|
||||
|
||||
def hermes_root_conflict(mine: Path, other_users_roots: Iterable[Path]) -> str | None:
|
||||
"""Why the resolved root `mine` may not be granted beside other users' Hermes roots.
|
||||
|
||||
Paths are compared resolved (symlinks followed; the other users' roots are
|
||||
resolved here). Two roots that are equal or nest would read the same files
|
||||
for two users, except when one is exactly the other's `profiles/<name>`:
|
||||
the scan above never enters `profiles/`, so the two stay apart (`local` on
|
||||
`~/.hermes`, a user on `~/.hermes/profiles/<name>`). None when it may.
|
||||
"""
|
||||
if hermes_root_is_profiles_container(mine):
|
||||
return HERMES_ROOT_IS_PROFILES_CONTAINER
|
||||
for other in other_users_roots:
|
||||
theirs = _real(other)
|
||||
if mine == theirs:
|
||||
return HERMES_ROOT_OVERLAPS_OTHER_USER
|
||||
for parent, child in ((mine, theirs), (theirs, mine)):
|
||||
if parent in child.parents:
|
||||
relative = child.relative_to(parent).parts
|
||||
if len(relative) != 2 or relative[0] != HERMES_PROFILES_DIRECTORY:
|
||||
return HERMES_ROOT_OVERLAPS_OTHER_USER
|
||||
return None
|
||||
|
||||
|
||||
#: 422: the root is the `profiles` container, not a profile.
|
||||
HERMES_ROOT_IS_PROFILES_CONTAINER = "hermes_root_is_profiles_container"
|
||||
#: 409: the root is, holds or sits inside another user's Hermes root.
|
||||
HERMES_ROOT_OVERLAPS_OTHER_USER = "hermes_root_overlaps_other_user"
|
||||
|
||||
|
||||
def _real(path: Path) -> Path:
|
||||
try:
|
||||
return path.expanduser().resolve(strict=False)
|
||||
except (OSError, RuntimeError):
|
||||
return path.expanduser().absolute()
|
||||
|
||||
|
||||
def _read_rollout(
|
||||
source_file: AuthorizedSourceFile,
|
||||
options: ScanOptions,
|
||||
|
||||
@@ -34,6 +34,7 @@ from one_memory.memory_governance import MemoryGovernanceStore, MemoryStatus
|
||||
from one_memory.memory_server.durable_queue import DurableWriteQueue
|
||||
from one_memory.memory_server.unified_entry import UnifiedWriteIntake
|
||||
from one_memory.memory_tasks import MemoryTaskStatus, MemoryTaskStore
|
||||
from one_memory.model_backoff import next_retry_at
|
||||
from one_memory.secrets import SecretStoreUnavailableError
|
||||
from one_memory.user_scope import DEFAULT_USER_ID, USER_ID_PATTERN, UserId
|
||||
|
||||
@@ -77,6 +78,12 @@ class MemoryAddReceipt(BaseModel):
|
||||
task_id: str | None = None
|
||||
memory_ids: list[str] = Field(default_factory=list)
|
||||
error: str | None = None
|
||||
#: Why a note still being worked on last failed (`model_auth_failed`, ...),
|
||||
#: or why a failed one failed. A provider failure keeps it `pending`.
|
||||
error_code: str | None = None
|
||||
#: When a pending note is next tried: its own retry wait, or the model
|
||||
#: provider's pause while the provider is failing.
|
||||
next_retry_at: str | None = None
|
||||
|
||||
|
||||
class LegacyMemoryAdditionReader:
|
||||
@@ -258,6 +265,8 @@ class MemoryAdditionStore:
|
||||
durable_state = durable.state if durable is not None else "failed"
|
||||
mapped: MemoryAddState = {
|
||||
"pending": "pending",
|
||||
# Re-run automatically within seconds (`recover_unknown`).
|
||||
"unknown": "pending",
|
||||
"running": "running",
|
||||
"waiting_for_model": "waiting_for_model",
|
||||
"waiting_for_permission": "waiting_for_permission",
|
||||
@@ -272,6 +281,12 @@ class MemoryAdditionStore:
|
||||
state=mapped,
|
||||
memory_ids=(list(durable.sdk_memory_ids) if durable is not None else []),
|
||||
error=durable.detail if durable is not None else "write_missing",
|
||||
error_code=(
|
||||
durable.error_code
|
||||
if durable is not None and durable.state != "completed"
|
||||
else None
|
||||
),
|
||||
next_retry_at=durable.retry_at if durable is not None else None,
|
||||
)
|
||||
if self.processor is None:
|
||||
raise RuntimeError("legacy_memory_add_processor_unavailable")
|
||||
@@ -402,10 +417,15 @@ def memory_add_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
user_id: str = Query(default=DEFAULT_USER_ID, pattern=USER_ID_PATTERN),
|
||||
) -> MemoryAddReceipt:
|
||||
try:
|
||||
return store(request).receipt_for_request(request_id, user_id=user_id)
|
||||
receipt = store(request).receipt_for_request(request_id, user_id=user_id)
|
||||
except KeyError as error:
|
||||
raise HTTPException(404, "memory_add_not_found") from error
|
||||
except ValueError as error:
|
||||
raise HTTPException(409, str(error)) from error
|
||||
return receipt.model_copy(
|
||||
update={
|
||||
"next_retry_at": next_retry_at(request.app, receipt.state, receipt.next_retry_at)
|
||||
}
|
||||
)
|
||||
|
||||
return router
|
||||
@@ -21,6 +21,11 @@ exercises (query rewrite needs `queries=[...]`, session narratives need
|
||||
cross-encoder download, access write-back); and everything MemHub already
|
||||
controls elsewhere (pipeline log, trace, memory operations, telemetry, mode).
|
||||
|
||||
Beside the on/off features there is one numeric parameter,
|
||||
`intention_dedup_threshold` (`PARAMETERS`): the cosine similarity at which the
|
||||
read-time intention dedup treats two intentions as copies. It is set, stored,
|
||||
reset and keyed exactly like a feature.
|
||||
|
||||
No SDK import here: only `hy_sdk_runtime` and `hy_sdk_migration` import it.
|
||||
`config_state` imports this module, so it imports nothing from there.
|
||||
"""
|
||||
@@ -31,7 +36,7 @@ from collections.abc import Mapping
|
||||
from dataclasses import dataclass
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, StrictBool
|
||||
from pydantic import BaseModel, ConfigDict, Field, StrictBool
|
||||
|
||||
MemoryFeatureName = Literal[
|
||||
"graph_rerank",
|
||||
@@ -40,8 +45,13 @@ MemoryFeatureName = Literal[
|
||||
"session_episodes",
|
||||
"episode_summary",
|
||||
"intention",
|
||||
"intention_dedup",
|
||||
"few_shot",
|
||||
]
|
||||
#: Numeric settings beside the on/off features.
|
||||
MemoryParameterName = Literal["intention_dedup_threshold"]
|
||||
#: Every key of the setting: a feature or a parameter.
|
||||
MemorySettingName = MemoryFeatureName | MemoryParameterName
|
||||
#: Read features change search; write features change what extraction stores.
|
||||
FeatureSide = Literal["read", "write"]
|
||||
#: none: no extra call. extra_query: one more store query per search.
|
||||
@@ -49,9 +59,25 @@ FeatureSide = Literal["read", "write"]
|
||||
#: a longer extraction prompt.
|
||||
FeatureCost = Literal["none", "extra_query", "llm_call", "prompt_tokens"]
|
||||
#: Why an enabled feature does nothing right now (`active: false`).
|
||||
FeatureReason = Literal["disabled", "requires_pro", "requires_ultra", "requires_graph_rerank"]
|
||||
FeatureReason = Literal[
|
||||
"disabled",
|
||||
"requires_pro",
|
||||
"requires_ultra",
|
||||
"requires_graph_rerank",
|
||||
"requires_intention_dedup",
|
||||
]
|
||||
#: A feature is on or off; a parameter is a number.
|
||||
type SettingValue = bool | float
|
||||
#: A client's features as a hashable key: sorted (name, value) pairs, set ones only.
|
||||
type FeatureOverrides = tuple[tuple[str, bool], ...]
|
||||
type FeatureOverrides = tuple[tuple[str, SettingValue], ...]
|
||||
|
||||
#: The range a person may set `intention_dedup_threshold` to. With dedup on,
|
||||
#: the SDK deletes every intention at or above it from the store at read time,
|
||||
#: which cannot be undone; below 0.8 the embedding models MemHub uses score
|
||||
#: different plans on one topic as that close, so a lower value would delete
|
||||
#: intentions that are not copies. 1.0 keeps only exact copies' dedup.
|
||||
INTENTION_DEDUP_THRESHOLD_MIN = 0.8
|
||||
INTENTION_DEDUP_THRESHOLD_MAX = 1.0
|
||||
|
||||
_MODE_RANK: dict[str, int] = {"lite": 0, "pro": 1, "ultra": 2}
|
||||
|
||||
@@ -169,6 +195,23 @@ FEATURES: tuple[FeatureSpec, ...] = (
|
||||
"No extra call."
|
||||
),
|
||||
),
|
||||
FeatureSpec(
|
||||
name="intention_dedup",
|
||||
section="recall",
|
||||
field="intention_dedup_enabled",
|
||||
env="MEMORY_INTENTION_DEDUP_ENABLED",
|
||||
side="read",
|
||||
requires_mode="pro",
|
||||
requires_feature=None,
|
||||
cost="none",
|
||||
description=(
|
||||
"Treat same-day intentions (L7) with the same expiry and a cosine "
|
||||
"similarity at or above intention_dedup_threshold as copies: a "
|
||||
"search returns the earliest and deletes the others from the store. "
|
||||
"On by default; off hides, drops and deletes no intention, at read "
|
||||
"time or in a write batch. No extra call."
|
||||
),
|
||||
),
|
||||
FeatureSpec(
|
||||
name="few_shot",
|
||||
section="llm",
|
||||
@@ -189,12 +232,57 @@ FEATURE_NAMES: tuple[MemoryFeatureName, ...] = tuple(spec.name for spec in FEATU
|
||||
FEATURES_BY_NAME: dict[str, FeatureSpec] = {spec.name: spec for spec in FEATURES}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ParameterSpec:
|
||||
"""A numeric setting; it acts only while `requires_feature` acts."""
|
||||
|
||||
name: MemoryParameterName
|
||||
section: Literal["recall"]
|
||||
field: str
|
||||
env: str
|
||||
minimum: float
|
||||
maximum: float
|
||||
requires_feature: MemoryFeatureName
|
||||
description: str
|
||||
|
||||
@property
|
||||
def sdk_field(self) -> str:
|
||||
return f"{self.section}.{self.field}"
|
||||
|
||||
|
||||
PARAMETERS: tuple[ParameterSpec, ...] = (
|
||||
ParameterSpec(
|
||||
name="intention_dedup_threshold",
|
||||
section="recall",
|
||||
field="intention_dedup_threshold",
|
||||
env="MEMORY_INTENTION_DEDUP_THRESHOLD",
|
||||
minimum=INTENTION_DEDUP_THRESHOLD_MIN,
|
||||
maximum=INTENTION_DEDUP_THRESHOLD_MAX,
|
||||
requires_feature="intention_dedup",
|
||||
description=(
|
||||
"Cosine similarity at or above which intention_dedup treats two "
|
||||
"intentions as copies. Higher keeps more. The SDK default is 0.95 "
|
||||
"(MEMORY_DEDUP_THRESHOLD also sets it when its own env var is unset)."
|
||||
),
|
||||
),
|
||||
)
|
||||
PARAMETER_NAMES: tuple[MemoryParameterName, ...] = tuple(spec.name for spec in PARAMETERS)
|
||||
PARAMETERS_BY_NAME: dict[str, ParameterSpec] = {spec.name: spec for spec in PARAMETERS}
|
||||
#: Every key of the setting, features first, each in its catalog order.
|
||||
SETTING_NAMES: tuple[MemorySettingName, ...] = (*FEATURE_NAMES, *PARAMETER_NAMES)
|
||||
SETTINGS_BY_NAME: dict[str, FeatureSpec | ParameterSpec] = {
|
||||
**FEATURES_BY_NAME,
|
||||
**PARAMETERS_BY_NAME,
|
||||
}
|
||||
|
||||
|
||||
class MemoryFeatures(BaseModel):
|
||||
"""`config.json` `memory_features`, and the `PUT` body.
|
||||
|
||||
Every key is optional; absent (or null) means the SDK default. In a `PUT`
|
||||
an absent key is left as it is and null resets it. Unknown keys and
|
||||
non-boolean values are refused (422).
|
||||
an absent key is left as it is and null resets it. Unknown keys,
|
||||
non-boolean feature values and a threshold that is not a number in its
|
||||
range are refused (422).
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid", strict=True)
|
||||
@@ -205,7 +293,14 @@ class MemoryFeatures(BaseModel):
|
||||
session_episodes: StrictBool | None = None
|
||||
episode_summary: StrictBool | None = None
|
||||
intention: StrictBool | None = None
|
||||
intention_dedup: StrictBool | None = None
|
||||
few_shot: StrictBool | None = None
|
||||
intention_dedup_threshold: float | None = Field(
|
||||
default=None,
|
||||
ge=INTENTION_DEDUP_THRESHOLD_MIN,
|
||||
le=INTENTION_DEDUP_THRESHOLD_MAX,
|
||||
allow_inf_nan=False,
|
||||
)
|
||||
|
||||
def overrides(self) -> FeatureOverrides:
|
||||
"""The features a person set, as the client key."""
|
||||
@@ -221,24 +316,47 @@ class MemoryFeatures(BaseModel):
|
||||
return MemoryFeatures(**{key: value for key, value in values.items() if value is not None})
|
||||
|
||||
|
||||
def feature_overrides(values: Mapping[str, bool]) -> FeatureOverrides:
|
||||
unknown = set(values) - set(FEATURES_BY_NAME)
|
||||
def feature_overrides(values: Mapping[str, SettingValue]) -> FeatureOverrides:
|
||||
unknown = set(values) - set(SETTINGS_BY_NAME)
|
||||
if unknown:
|
||||
raise ValueError("unknown_memory_feature")
|
||||
return tuple(sorted((name, bool(value)) for name, value in values.items()))
|
||||
return tuple(sorted((name, _setting_value(name, value)) for name, value in values.items()))
|
||||
|
||||
|
||||
def _setting_value(name: str, value: SettingValue) -> SettingValue:
|
||||
parameter = PARAMETERS_BY_NAME.get(name)
|
||||
if parameter is None:
|
||||
return bool(value)
|
||||
if isinstance(value, bool) or not parameter.minimum <= float(value) <= parameter.maximum:
|
||||
raise ValueError("invalid_memory_feature_value")
|
||||
return float(value)
|
||||
|
||||
|
||||
def effective_features(
|
||||
overrides: FeatureOverrides, defaults: Mapping[str, bool]
|
||||
) -> dict[str, bool]:
|
||||
"""Every feature's value: what a person set, else the SDK default."""
|
||||
values: dict[str, bool] = {spec.name: bool(defaults.get(spec.name, False)) for spec in FEATURES}
|
||||
overrides: FeatureOverrides, defaults: Mapping[str, SettingValue]
|
||||
) -> dict[str, SettingValue]:
|
||||
"""Every setting's value: what a person set, else the SDK default."""
|
||||
values: dict[str, SettingValue] = {
|
||||
spec.name: bool(defaults.get(spec.name, False)) for spec in FEATURES
|
||||
}
|
||||
for parameter in PARAMETERS:
|
||||
if parameter.name in defaults:
|
||||
values[parameter.name] = float(defaults[parameter.name])
|
||||
values.update(dict(overrides))
|
||||
return values
|
||||
|
||||
|
||||
def feature_activity(name: str, effective: Mapping[str, bool], mode: str) -> FeatureReason | None:
|
||||
"""None when the feature acts in this mode; else why it does not."""
|
||||
def feature_activity(
|
||||
name: str, effective: Mapping[str, SettingValue], mode: str
|
||||
) -> FeatureReason | None:
|
||||
"""None when the feature acts in this mode; else why it does not.
|
||||
|
||||
A parameter acts exactly when the feature it tunes acts.
|
||||
"""
|
||||
parameter = PARAMETERS_BY_NAME.get(name)
|
||||
if parameter is not None:
|
||||
reason = feature_activity(parameter.requires_feature, effective, mode)
|
||||
return "requires_intention_dedup" if reason == "disabled" else reason
|
||||
spec = FEATURES_BY_NAME[name]
|
||||
if not effective.get(name, False):
|
||||
return "disabled"
|
||||
@@ -252,11 +370,12 @@ def feature_activity(name: str, effective: Mapping[str, bool], mode: str) -> Fea
|
||||
|
||||
|
||||
def non_default_features(
|
||||
overrides: FeatureOverrides, defaults: Mapping[str, bool]
|
||||
) -> dict[str, bool]:
|
||||
"""The features whose value differs from the SDK default, for status."""
|
||||
overrides: FeatureOverrides, defaults: Mapping[str, SettingValue]
|
||||
) -> dict[str, SettingValue]:
|
||||
"""The settings whose value differs from the SDK default, for status."""
|
||||
baseline = effective_features((), defaults)
|
||||
return {
|
||||
name: value
|
||||
for name, value in effective_features(overrides, defaults).items()
|
||||
if value != bool(defaults.get(name, False))
|
||||
if value != baseline.get(name)
|
||||
}
|
||||
@@ -74,9 +74,9 @@ class ConfigDocument(BaseModel):
|
||||
#: SDK retrieval and extraction features a person set; an unset one keeps
|
||||
#: the SDK default. See `one_memory.memory_features`.
|
||||
memory_features: MemoryFeatures = Field(default_factory=MemoryFeatures)
|
||||
#: The display language a person chose in the App; None = never chosen or
|
||||
#: "follow the system". See `one_memory.ui_language` for what it does to
|
||||
#: the SDK.
|
||||
#: The display language the App shows (its choice, or the system language
|
||||
#: it follows); None = none set. See `one_memory.ui_language` for what it
|
||||
#: does to the SDK.
|
||||
ui_language: UiLanguage | None = None
|
||||
|
||||
@field_validator("sources")
|
||||
|
||||
@@ -21,6 +21,7 @@ import shutil
|
||||
import sqlite3
|
||||
import time as _time
|
||||
from collections.abc import Awaitable, Callable
|
||||
from contextlib import AbstractContextManager, nullcontext
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
@@ -52,6 +53,7 @@ from one_memory.memory_tasks import (
|
||||
from one_memory.memory_tasks import (
|
||||
POLICY_STOPPED_CODES as _TASK_POLICY_STOPPED_CODES,
|
||||
)
|
||||
from one_memory.model_backoff import ModelAttempt
|
||||
from one_memory.sdk_migration_log import error_chain_text
|
||||
from one_memory.service_levels import ServiceLevel, service_level_context
|
||||
from one_memory.user_scope import DEFAULT_USER_ID, lane_key, other_user_lane_prefix
|
||||
@@ -107,6 +109,43 @@ PARKED_STATES = (WAITING_FOR_PERMISSION, WAITING_FOR_MODEL)
|
||||
#: re-admitted on a timer. The gap keeps a still-down model from spinning.
|
||||
DEFAULT_MODEL_READMIT_AFTER = timedelta(minutes=1)
|
||||
|
||||
#: How long a job the execute gate handed back (`release_unstarted`) is left
|
||||
#: alone before it can be claimed again.
|
||||
RELEASE_HOLD = timedelta(seconds=5)
|
||||
|
||||
#: Automatic retries of a job whose attempt failed for a reason other than the
|
||||
#: model provider: content the model or the SDK refused, a parse error, a
|
||||
#: crash or an interruption mid-write. After the last one the job is `failed`,
|
||||
#: shown with its code, and the queue moves on. A provider failure (a broken
|
||||
#: key, a model that is gone, a provider that is down) never counts: the job
|
||||
#: waits in `pending` for as long as the provider is broken, paced by
|
||||
#: `model_backoff`.
|
||||
JOB_RETRY_LIMIT = 3
|
||||
JOB_RETRY_FIRST_BACKOFF = timedelta(seconds=30)
|
||||
JOB_RETRY_BACKOFF_CAP = timedelta(minutes=3)
|
||||
|
||||
#: The code a job interrupted mid-write carries when it has no better one.
|
||||
INTERRUPTED_ERROR_CODE = "durable_write_interrupted"
|
||||
|
||||
|
||||
def job_retry_backoff(retry: int) -> timedelta:
|
||||
"""The wait before automatic retry number `retry` (1-based)."""
|
||||
wait: timedelta = JOB_RETRY_FIRST_BACKOFF * (2 ** min(max(retry - 1, 0), 16))
|
||||
return min(wait, JOB_RETRY_BACKOFF_CAP)
|
||||
|
||||
|
||||
def sdk_request_id(idempotency_key: str, attempt: int) -> str:
|
||||
"""The SDK request id of a job's `attempt`-th write.
|
||||
|
||||
The SDK keeps one outcome per request id and replays it: a write that
|
||||
failed under an id fails under it forever, without calling the model.
|
||||
Every re-run therefore gets an id of its own. Attempt 0 is the job's own
|
||||
key, so every job written before this existed keeps the id its outcome
|
||||
was recorded under.
|
||||
"""
|
||||
return idempotency_key if attempt <= 0 else f"{idempotency_key}#r{attempt}"
|
||||
|
||||
|
||||
#: Production drain-budget defaults. These are the single source consumed by
|
||||
#: the composition root; callers must not invent a second formula or copy a
|
||||
#: test probe's numbers into the product path.
|
||||
@@ -337,6 +376,10 @@ LEGACY_MEMORY_WRITE_JOB_COLUMNS = (
|
||||
("time_quality", "TEXT NOT NULL DEFAULT 'approximate'"),
|
||||
("sdk_raw_ids_json", "TEXT NOT NULL DEFAULT '[]'"),
|
||||
("sdk_memory_ids_json", "TEXT NOT NULL DEFAULT '[]'"),
|
||||
#: Which SDK request id the next write uses (`sdk_request_id`).
|
||||
("sdk_attempt", "INTEGER NOT NULL DEFAULT 0"),
|
||||
#: Automatic retries spent (`JOB_RETRY_LIMIT`); provider failures spend none.
|
||||
("retries", "INTEGER NOT NULL DEFAULT 0"),
|
||||
)
|
||||
|
||||
_FIFO_BLOCKING_STATES = (PENDING, RUNNING, WAITING_FOR_PERMISSION, WAITING_FOR_MODEL)
|
||||
@@ -381,6 +424,10 @@ class DurableWriteRecord:
|
||||
detail: str = ""
|
||||
sdk_raw_ids: tuple[str, ...] = ()
|
||||
sdk_memory_ids: tuple[str, ...] = ()
|
||||
#: The stable code of the last failure, if the row carries one.
|
||||
error_code: str | None = None
|
||||
#: When a waiting row may next be claimed, if a retry wait holds it.
|
||||
retry_at: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -746,10 +793,10 @@ class DurableWriteQueue:
|
||||
"""Recover expired leases on the only engine D56 permits.
|
||||
|
||||
A lease whose owner died before HY was invoked is safe to retry. Once
|
||||
`hy_invoked` is recorded, the outcome may already exist in HY and its
|
||||
requests have no idempotency key, so that row becomes `unknown`
|
||||
instead of `pending`. Unknown is intentionally sticky: a later reclaim
|
||||
pass cannot turn it back into retryable work.
|
||||
`hy_invoked` is recorded the write may already have landed, so that
|
||||
row becomes `unknown` instead of `pending`. `unknown` is only a stop
|
||||
on the way: `recover_unknown` re-runs it under a new SDK request id,
|
||||
after asking the SDK whether the old one succeeded.
|
||||
"""
|
||||
now = datetime.now(UTC).isoformat()
|
||||
changed = 0
|
||||
@@ -1509,13 +1556,15 @@ class DurableWriteQueue:
|
||||
with main_database(self.database_path).read(row_factory=sqlite3.Row) as connection:
|
||||
_require_queue_schema(connection)
|
||||
row = connection.execute(
|
||||
"SELECT job_id, state, created_at, updated_at, last_error, "
|
||||
"SELECT job_id, state, created_at, updated_at, last_error, lease_expires_at, "
|
||||
"sdk_raw_ids_json,sdk_memory_ids_json "
|
||||
"FROM memory_write_jobs WHERE idempotency_key=?",
|
||||
(request_id,),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
last_error = str(row["last_error"] or "")
|
||||
retry_at = row["lease_expires_at"]
|
||||
return DurableWriteRecord(
|
||||
job_id=str(row["job_id"]),
|
||||
request_id=request_id,
|
||||
@@ -1526,6 +1575,14 @@ class DurableWriteQueue:
|
||||
detail=str(row["last_error"] or ""),
|
||||
sdk_raw_ids=tuple(json.loads(str(row["sdk_raw_ids_json"]))),
|
||||
sdk_memory_ids=tuple(json.loads(str(row["sdk_memory_ids_json"]))),
|
||||
error_code=last_error if _ERROR_CODE.fullmatch(last_error) else None,
|
||||
retry_at=(
|
||||
str(retry_at)
|
||||
if str(row["state"]) == PENDING
|
||||
and retry_at is not None
|
||||
and str(retry_at) > datetime.now(UTC).isoformat()
|
||||
else None
|
||||
),
|
||||
)
|
||||
|
||||
def payload_for(self, job_id: str) -> WriteRequest | None:
|
||||
@@ -1726,9 +1783,23 @@ class DurableWriteQueue:
|
||||
)
|
||||
return updated.rowcount == 1
|
||||
|
||||
def release_unstarted(self, job_id: str, *, owner: str | None = None) -> bool:
|
||||
"""Return a claimed job to pending when its source was paused pre-HY."""
|
||||
now = datetime.now(UTC).isoformat()
|
||||
def release_unstarted(
|
||||
self,
|
||||
job_id: str,
|
||||
*,
|
||||
owner: str | None = None,
|
||||
hold: timedelta = RELEASE_HOLD,
|
||||
) -> bool:
|
||||
"""Return a claimed job to pending when its source was paused pre-HY.
|
||||
|
||||
The row comes back with its lease still set `hold` into the future.
|
||||
Claim only takes a pending row whose lease has expired, so a job the
|
||||
execute gate just refused is not claimed again on the very next tick:
|
||||
when the claim gate and the execute gate disagree for good (a
|
||||
pre-upgrade payload whose Evidence belongs to another user than its
|
||||
lane), the job is retried every few seconds instead of in a tight loop.
|
||||
"""
|
||||
now = datetime.now(UTC)
|
||||
where = "job_id=? AND state=? AND hy_invoked=0"
|
||||
params: tuple[Any, ...] = (job_id, RUNNING)
|
||||
if owner is not None:
|
||||
@@ -1738,11 +1809,175 @@ class DurableWriteQueue:
|
||||
updated = connection.execute(
|
||||
"UPDATE memory_write_jobs SET state=?,lease_owner=NULL,"
|
||||
"attempts=CASE WHEN attempts>0 THEN attempts-1 ELSE 0 END,"
|
||||
"lease_expires_at=NULL,updated_at=? WHERE " + where,
|
||||
(PENDING, now, *params),
|
||||
"lease_expires_at=?,updated_at=? WHERE " + where,
|
||||
(PENDING, (now + hold).isoformat(), now.isoformat(), *params),
|
||||
)
|
||||
return updated.rowcount == 1
|
||||
|
||||
def sdk_attempt(self, job_id: str) -> int:
|
||||
"""Which SDK request id this job's next write uses (`sdk_request_id`)."""
|
||||
with main_database(self.database_path).read() as connection:
|
||||
row = connection.execute(
|
||||
"SELECT sdk_attempt FROM memory_write_jobs WHERE job_id=?", (job_id,)
|
||||
).fetchone()
|
||||
return int(row[0]) if row is not None else 0
|
||||
|
||||
def requeue_for_model(
|
||||
self,
|
||||
job_id: str,
|
||||
*,
|
||||
error_code: str,
|
||||
owner: str | None = None,
|
||||
) -> bool:
|
||||
"""Put a job the model provider failed back in line, as it was.
|
||||
|
||||
The provider is broken, not the job: it keeps its place in the queue
|
||||
(`event_at`, `queue_seq`), spends no attempt and no retry, and is
|
||||
claimed again once `model_backoff` lets a probe through. Whatever the
|
||||
SDK may have kept of the failed write stays under that write's request
|
||||
id; the next one gets an id of its own (`sdk_attempt`), so the SDK
|
||||
does not replay the failure instead of calling the model.
|
||||
"""
|
||||
now = datetime.now(UTC).isoformat()
|
||||
where = "job_id=? AND state=?"
|
||||
params: tuple[Any, ...] = (job_id, RUNNING)
|
||||
if owner is not None:
|
||||
where += " AND lease_owner=?"
|
||||
params += (owner,)
|
||||
with main_database(self.database_path).write() as connection, connection:
|
||||
updated = connection.execute(
|
||||
"UPDATE memory_write_jobs SET state=?,lease_owner=NULL,lease_expires_at=NULL,"
|
||||
"attempts=CASE WHEN attempts>0 THEN attempts-1 ELSE 0 END,"
|
||||
"sdk_attempt=sdk_attempt+hy_invoked,hy_invoked=0,"
|
||||
"last_error=?,updated_at=? WHERE " + where,
|
||||
(PENDING, error_code, now, *params),
|
||||
)
|
||||
return updated.rowcount == 1
|
||||
|
||||
def retry_or_fail(
|
||||
self,
|
||||
job_id: str,
|
||||
*,
|
||||
error_code: str,
|
||||
owner: str | None = None,
|
||||
now: datetime | None = None,
|
||||
) -> str:
|
||||
"""Retry a job that failed for a reason of its own, or give up on it.
|
||||
|
||||
Up to `JOB_RETRY_LIMIT` retries, each after a longer wait
|
||||
(`job_retry_backoff`), held by the lease so the claim skips the job
|
||||
until then. After the last one the job is `failed`: shown with its
|
||||
code, out of the queue's way. A write that reached the SDK is re-run
|
||||
under a new request id. Returns the state written, or "" when the row
|
||||
is no longer this owner's.
|
||||
"""
|
||||
now = now or datetime.now(UTC)
|
||||
with (
|
||||
main_database(self.database_path).write(row_factory=sqlite3.Row) as connection,
|
||||
connection,
|
||||
):
|
||||
row = connection.execute(
|
||||
"SELECT state,lease_owner,retries FROM memory_write_jobs WHERE job_id=?",
|
||||
(job_id,),
|
||||
).fetchone()
|
||||
if (
|
||||
row is None
|
||||
or str(row["state"]) != RUNNING
|
||||
or (owner is not None and row["lease_owner"] != owner)
|
||||
):
|
||||
return ""
|
||||
retry = int(row["retries"]) + 1
|
||||
if retry > JOB_RETRY_LIMIT:
|
||||
connection.execute(
|
||||
"UPDATE memory_write_jobs SET state=?,lease_owner=NULL,lease_expires_at=NULL,"
|
||||
"last_error=?,updated_at=? WHERE job_id=? AND state=?",
|
||||
(FAILED, error_code, now.isoformat(), job_id, RUNNING),
|
||||
)
|
||||
return FAILED
|
||||
connection.execute(
|
||||
"UPDATE memory_write_jobs SET state=?,lease_owner=NULL,lease_expires_at=?,"
|
||||
"attempts=CASE WHEN attempts>0 THEN attempts-1 ELSE 0 END,"
|
||||
"sdk_attempt=sdk_attempt+hy_invoked,hy_invoked=0,retries=?,"
|
||||
"last_error=?,updated_at=? WHERE job_id=? AND state=?",
|
||||
(
|
||||
PENDING,
|
||||
(now + job_retry_backoff(retry)).isoformat(),
|
||||
retry,
|
||||
error_code,
|
||||
now.isoformat(),
|
||||
job_id,
|
||||
RUNNING,
|
||||
),
|
||||
)
|
||||
return PENDING
|
||||
|
||||
def recover_unknown(
|
||||
self,
|
||||
*,
|
||||
in_flight: Callable[[str], bool] = lambda _request_id: False,
|
||||
limit: int = 128,
|
||||
now: datetime | None = None,
|
||||
) -> int:
|
||||
"""Move every `unknown` job back into the queue, or out of it.
|
||||
|
||||
A job is `unknown` when its write may have reached the SDK and its
|
||||
worker never learned how it ended: the process stopped, the lease
|
||||
lapsed, or the work ceiling gave up on it. It is re-run, counted as a
|
||||
retry so a crash loop ends in `failed`. The re-run gets a new SDK
|
||||
request id and, before it starts, asks the SDK how the previous id
|
||||
ended (`execute_job`): a write that did succeed is completed from its
|
||||
receipt instead of being made twice.
|
||||
|
||||
Jobs from before this existed (upgrade) are handled the same way,
|
||||
once: from then on they follow the normal rules. A write whose SDK
|
||||
call is still running in this process (`in_flight`) is left until it
|
||||
ends. A legacy row whose write is known to have landed against an
|
||||
authority Core could not verify (`NON_RETRYABLE_POST_WRITE_CODES`) is
|
||||
never re-run: it is `failed`, which says so.
|
||||
"""
|
||||
now = now or datetime.now(UTC)
|
||||
changed = 0
|
||||
with (
|
||||
main_database(self.database_path).write(row_factory=sqlite3.Row) as connection,
|
||||
connection,
|
||||
):
|
||||
rows = connection.execute(
|
||||
"SELECT job_id,idempotency_key,sdk_attempt,retries,last_error "
|
||||
"FROM memory_write_jobs WHERE state=? ORDER BY queue_seq LIMIT ?",
|
||||
(UNKNOWN, limit),
|
||||
).fetchall()
|
||||
for row in rows:
|
||||
if in_flight(sdk_request_id(str(row["idempotency_key"]), int(row["sdk_attempt"]))):
|
||||
continue
|
||||
last_error = str(row["last_error"] or "")
|
||||
code = last_error if _ERROR_CODE.fullmatch(last_error) else INTERRUPTED_ERROR_CODE
|
||||
retry = int(row["retries"]) + 1
|
||||
if last_error in NON_RETRYABLE_POST_WRITE_CODES or retry > JOB_RETRY_LIMIT:
|
||||
changed += connection.execute(
|
||||
"UPDATE memory_write_jobs SET state=?,lease_owner=NULL,"
|
||||
"lease_expires_at=NULL,last_error=?,updated_at=? "
|
||||
"WHERE job_id=? AND state=?",
|
||||
(FAILED, code, now.isoformat(), str(row["job_id"]), UNKNOWN),
|
||||
).rowcount
|
||||
continue
|
||||
changed += connection.execute(
|
||||
"UPDATE memory_write_jobs SET state=?,lease_owner=NULL,lease_expires_at=?,"
|
||||
"attempts=0,sdk_attempt=sdk_attempt+1,hy_invoked=0,retries=?,"
|
||||
"last_error=?,updated_at=? WHERE job_id=? AND state=?",
|
||||
(
|
||||
PENDING,
|
||||
(now + job_retry_backoff(retry)).isoformat(),
|
||||
retry,
|
||||
code,
|
||||
now.isoformat(),
|
||||
str(row["job_id"]),
|
||||
UNKNOWN,
|
||||
),
|
||||
).rowcount
|
||||
if changed:
|
||||
logger.info("durable_write_unknown_recovered count=%d", changed)
|
||||
return changed
|
||||
|
||||
def settle_worker_error(self, job_id: str, *, error_code: str, owner: str | None = None) -> str:
|
||||
"""Fallback settlement when the injected executor raises past its guard.
|
||||
|
||||
@@ -2008,60 +2243,6 @@ class DurableWriteQueue:
|
||||
)
|
||||
return str(target)
|
||||
|
||||
def retry_unknown_confirmed(self, job_id: str) -> int:
|
||||
"""Requeue one unknown write only after an explicit user decision.
|
||||
|
||||
Automatic workers may never move ``unknown`` back to pending because
|
||||
HY may already contain the result. This separate command boundary is
|
||||
intentionally named for the confirmation it requires. The durable
|
||||
job and any task rows created from the same Evidence are reset in one
|
||||
short transaction, so the unified queue remains the only execution
|
||||
entry and a half-reset attempt cannot become permanently stranded.
|
||||
"""
|
||||
now = datetime.now(UTC).isoformat()
|
||||
with (
|
||||
main_database(self.database_path).write(row_factory=sqlite3.Row) as connection,
|
||||
connection,
|
||||
):
|
||||
row = connection.execute(
|
||||
"SELECT state,last_error FROM memory_write_jobs WHERE job_id=?", (job_id,)
|
||||
).fetchone()
|
||||
if (
|
||||
row is None
|
||||
or str(row["state"]) != UNKNOWN
|
||||
or str(row["last_error"]) in NON_RETRYABLE_POST_WRITE_CODES
|
||||
):
|
||||
return 0
|
||||
# After the SDK cutover the legacy task tables are gone and the
|
||||
# job is the only row to reset.
|
||||
if not self.sdk_authority and _legacy_task_links_present(connection):
|
||||
connection.execute(
|
||||
"""
|
||||
UPDATE memory_tasks
|
||||
SET status='pending', runner_token=NULL, attempts=0,
|
||||
last_error_code='model_content_manual_retry_requested', updated_at=?
|
||||
WHERE status='unknown' AND task_id IN (
|
||||
SELECT inputs.task_id
|
||||
FROM memory_task_inputs AS inputs, json_each(inputs.refs_json) AS ref
|
||||
JOIN memory_write_job_evidence AS linked
|
||||
ON linked.evidence_id=json_extract(ref.value, '$.evidence_id')
|
||||
WHERE linked.job_id=?
|
||||
)
|
||||
""",
|
||||
(now, job_id),
|
||||
)
|
||||
changed = connection.execute(
|
||||
"""
|
||||
UPDATE memory_write_jobs
|
||||
SET state='pending', attempts=0, lease_owner=NULL,
|
||||
lease_expires_at=NULL, hy_invoked=0,
|
||||
last_error='model_content_manual_retry_requested', updated_at=?
|
||||
WHERE job_id=? AND state='unknown'
|
||||
""",
|
||||
(now, job_id),
|
||||
).rowcount
|
||||
return changed
|
||||
|
||||
def readmit_cancelled_permission_confirmed(self, job_id: str, request: WriteRequest) -> int:
|
||||
"""Requeue one unsent permission cancellation after explicit consent.
|
||||
|
||||
@@ -2114,115 +2295,6 @@ class DurableWriteQueue:
|
||||
).rowcount
|
||||
return changed
|
||||
|
||||
def retry_timed_out_unknown_after_reconcile(
|
||||
self,
|
||||
*,
|
||||
older_than: datetime,
|
||||
limit: int = 32,
|
||||
) -> int:
|
||||
"""Requeue bounded timeout unknowns only after the caller queried HY.
|
||||
|
||||
This is not the general unknown retry path. Only a timed-out write with
|
||||
attempts remaining is eligible, and attempts are preserved so repeated
|
||||
timeouts stop at the durable ceiling instead of looping forever.
|
||||
"""
|
||||
if type(limit) is not int or limit < 1:
|
||||
raise ValueError("limit must be positive")
|
||||
now = datetime.now(UTC).isoformat()
|
||||
with (
|
||||
main_database(self.database_path).write(row_factory=sqlite3.Row) as connection,
|
||||
connection,
|
||||
):
|
||||
rows = connection.execute(
|
||||
"SELECT job_id FROM memory_write_jobs "
|
||||
"WHERE state=? AND last_error='hy_memory_timeout' "
|
||||
"AND attempts < max_attempts AND updated_at <= ? "
|
||||
"ORDER BY updated_at, job_id LIMIT ?",
|
||||
(UNKNOWN, older_than.isoformat(), limit),
|
||||
).fetchall()
|
||||
job_ids = tuple(str(row["job_id"]) for row in rows)
|
||||
if not job_ids:
|
||||
return 0
|
||||
placeholders = ",".join("?" for _ in job_ids)
|
||||
connection.execute(
|
||||
f"UPDATE memory_write_jobs SET state=?, lease_owner=NULL, "
|
||||
f"lease_expires_at=NULL, hy_invoked=0, "
|
||||
f"last_error='model_content_reconciled_absent_retry', updated_at=? "
|
||||
f"WHERE state=? AND job_id IN ({placeholders})",
|
||||
(PENDING, now, UNKNOWN, *job_ids),
|
||||
)
|
||||
if _legacy_task_links_present(connection):
|
||||
connection.execute(
|
||||
f"UPDATE memory_tasks SET status=?, runner_token=NULL, "
|
||||
f"last_error_code='model_content_reconciled_absent_retry', updated_at=? "
|
||||
f"WHERE status=? AND task_id IN ("
|
||||
f"SELECT inputs.task_id FROM memory_task_inputs AS inputs "
|
||||
f"JOIN memory_write_job_evidence AS linked ON EXISTS ("
|
||||
f"SELECT 1 FROM json_each(inputs.refs_json) AS ref "
|
||||
f"WHERE json_extract(ref.value, '$.evidence_id')=linked.evidence_id) "
|
||||
f"WHERE linked.job_id IN ({placeholders}))",
|
||||
(MemoryTaskStatus.PENDING, now, MemoryTaskStatus.UNKNOWN, *job_ids),
|
||||
)
|
||||
return len(job_ids)
|
||||
|
||||
def retry_first_unknown_after_reconcile(
|
||||
self,
|
||||
*,
|
||||
older_than: datetime,
|
||||
limit: int = 32,
|
||||
) -> int:
|
||||
"""Retry one unresolved unknown outcome once, after reconciliation.
|
||||
|
||||
The caller first asks HY and checks linked governed output. Rows still
|
||||
unknown may have lost the response after the SDK call, so one bounded
|
||||
retry is useful. `attempts=1` is the guard: the retry claim increments
|
||||
it to two, and a second unknown is left for explicit user recovery.
|
||||
"""
|
||||
if type(limit) is not int or limit < 1:
|
||||
raise ValueError("limit must be positive")
|
||||
now = datetime.now(UTC).isoformat()
|
||||
with (
|
||||
main_database(self.database_path).write(row_factory=sqlite3.Row) as connection,
|
||||
connection,
|
||||
):
|
||||
rows = connection.execute(
|
||||
"SELECT job_id FROM memory_write_jobs "
|
||||
"WHERE state=? AND attempts=1 AND attempts < max_attempts "
|
||||
f"AND COALESCE(last_error, '') NOT IN "
|
||||
f"({','.join('?' for _ in NON_RETRYABLE_POST_WRITE_CODES)}) "
|
||||
"AND updated_at <= ? ORDER BY updated_at, job_id LIMIT ?",
|
||||
(
|
||||
UNKNOWN,
|
||||
*NON_RETRYABLE_POST_WRITE_CODES,
|
||||
older_than.isoformat(),
|
||||
limit,
|
||||
),
|
||||
).fetchall()
|
||||
job_ids = tuple(str(row["job_id"]) for row in rows)
|
||||
if not job_ids:
|
||||
return 0
|
||||
placeholders = ",".join("?" for _ in job_ids)
|
||||
connection.execute(
|
||||
f"UPDATE memory_write_jobs SET state=?, lease_owner=NULL, "
|
||||
f"lease_expires_at=NULL, hy_invoked=0, "
|
||||
f"last_error='model_content_unknown_auto_retry', updated_at=? "
|
||||
f"WHERE state=? AND attempts=1 AND job_id IN ({placeholders})",
|
||||
(PENDING, now, UNKNOWN, *job_ids),
|
||||
)
|
||||
if _legacy_task_links_present(connection):
|
||||
connection.execute(
|
||||
f"UPDATE memory_tasks SET status=?, runner_token=NULL, "
|
||||
f"last_error_code='model_content_unknown_auto_retry', updated_at=? "
|
||||
f"WHERE status=? AND task_id IN ("
|
||||
f"SELECT inputs.task_id FROM memory_task_inputs AS inputs "
|
||||
f"JOIN memory_write_job_evidence AS linked ON EXISTS ("
|
||||
f"SELECT 1 FROM json_each(inputs.refs_json) AS ref "
|
||||
f"WHERE json_extract(ref.value, '$.evidence_id')=linked.evidence_id) "
|
||||
f"WHERE linked.job_id IN ({placeholders}))",
|
||||
(MemoryTaskStatus.PENDING, now, MemoryTaskStatus.UNKNOWN, *job_ids),
|
||||
)
|
||||
return len(job_ids)
|
||||
|
||||
def resolve_unknown_from_linked_output(self, job_id: str) -> bool:
|
||||
"""Close an unknown durable row when local governance output proves success."""
|
||||
with main_database(self.database_path).read(row_factory=sqlite3.Row) as connection:
|
||||
@@ -2361,7 +2433,9 @@ class DurableWriteRegistry:
|
||||
# distinguishing detail. Parked and cancelled states keep their own
|
||||
# names — collapsing an authorization stop into `failed` is exactly the
|
||||
# pollution the task layer already rejects.
|
||||
if record.state in (PENDING, RUNNING):
|
||||
if record.state in (PENDING, RUNNING, UNKNOWN):
|
||||
# `unknown` no longer waits for anyone: the drain re-runs it
|
||||
# within seconds (`recover_unknown`), so to a caller it is queued.
|
||||
state = "accepted"
|
||||
elif record.state == DEAD:
|
||||
state = "failed"
|
||||
@@ -2385,6 +2459,8 @@ class DurableWriteRegistry:
|
||||
detail=detail,
|
||||
raw_ids=record.sdk_raw_ids,
|
||||
memory_ids=record.sdk_memory_ids,
|
||||
error_code=record.error_code if record.state != COMPLETED else None,
|
||||
next_retry_at=record.retry_at,
|
||||
)
|
||||
|
||||
def submit(
|
||||
@@ -2466,12 +2542,22 @@ async def execute_job(
|
||||
source_writes_permitted: Callable[[str], bool] | None = None,
|
||||
admit_hy_invocation: Callable[[str, Callable[[], Awaitable[bool]]], Awaitable[bool]]
|
||||
| None = None,
|
||||
model_failures: Callable[[], AbstractContextManager[ModelAttempt | None]] | None = None,
|
||||
) -> None:
|
||||
"""Run one durable job through the SDK-authority processor.
|
||||
|
||||
``process_sdk_job`` owns the one public SDK write. ``hy_invoked`` is
|
||||
recorded immediately before that call, so a crash mid-write is later
|
||||
reclaimed to ``unknown`` instead of being retried blindly.
|
||||
reclaimed to ``unknown`` and re-run (`recover_unknown`).
|
||||
|
||||
Each write uses the SDK request id of the job's current attempt
|
||||
(`sdk_request_id`). A re-run first asks the SDK how the previous id
|
||||
ended, and completes from its receipt if it succeeded.
|
||||
|
||||
``model_failures`` (`ModelBackoff.attempt`) says whether the provider
|
||||
failed while this write was in flight: such a job goes back to `pending`
|
||||
as it was and waits for the provider. Any other failure is retried
|
||||
`JOB_RETRY_LIMIT` times and then `failed`.
|
||||
"""
|
||||
# Every store call here is synchronous SQLite work on the single writer or
|
||||
# a reader; made from the loop it would block health checks and the API for
|
||||
@@ -2547,16 +2633,31 @@ async def execute_job(
|
||||
raise RuntimeError("durable_write_lease_lost_before_invocation")
|
||||
hy_invoked = True
|
||||
|
||||
attempt = await in_the_background(queue.sdk_attempt, job_id)
|
||||
window = model_failures() if model_failures is not None else nullcontext(None)
|
||||
provider: ModelAttempt | None = None
|
||||
try:
|
||||
level = ServiceLevel.REALTIME if request.priority == "live" else ServiceLevel.BACKGROUND
|
||||
with service_level_context(level):
|
||||
sdk_result = await supervisor.process_sdk_job(
|
||||
records,
|
||||
request_id=request.idempotency_key,
|
||||
before_hy_invocation=mark_hy_invocation,
|
||||
with service_level_context(level), window as provider:
|
||||
previous = (
|
||||
await _previous_write_receipt(
|
||||
supervisor, sdk_request_id(request.idempotency_key, attempt - 1)
|
||||
)
|
||||
if attempt > 0
|
||||
else None
|
||||
)
|
||||
raw_ids = sdk_result.raw_ids
|
||||
memory_ids = sdk_result.memory_ids
|
||||
if previous is not None:
|
||||
# The write before this one did land: its worker only never
|
||||
# learned so. Complete from its receipt rather than write twice.
|
||||
raw_ids, memory_ids = previous.raw_ids, previous.memory_ids
|
||||
else:
|
||||
sdk_result = await supervisor.process_sdk_job(
|
||||
records,
|
||||
request_id=sdk_request_id(request.idempotency_key, attempt),
|
||||
before_hy_invocation=mark_hy_invocation,
|
||||
)
|
||||
raw_ids = sdk_result.raw_ids
|
||||
memory_ids = sdk_result.memory_ids
|
||||
except Exception as error: # noqa: BLE001 - classified below
|
||||
# Never persist the raw message: it may be a provider response or the
|
||||
# user's own text, and `last_error` is shown over HTTP as `detail`.
|
||||
@@ -2576,51 +2677,56 @@ async def execute_job(
|
||||
):
|
||||
await in_the_background(queue.release_unstarted, job_id, owner=owner)
|
||||
return
|
||||
# The write coordinator classifies these before any SDK client is leased, so HY
|
||||
# was provably not called. They must not read as `failed`:
|
||||
provider_failure = provider.failure if provider is not None else None
|
||||
if hy_invoked and provider_failure is not None:
|
||||
# The provider failed this write (a broken key, a model that is
|
||||
# gone, a provider that is down): the job is fine and waits for
|
||||
# the provider, in its place, without spending a retry. The
|
||||
# breaker paces when it is tried again.
|
||||
await in_the_background(
|
||||
queue.requeue_for_model, job_id, error_code=provider_failure, owner=owner
|
||||
)
|
||||
logger.info(
|
||||
"durable_write_job_waiting_for_provider job_id=%s error_code=%s sdk_error=%s",
|
||||
job_id,
|
||||
provider_failure,
|
||||
code,
|
||||
)
|
||||
return
|
||||
# Refusals made before the SDK was called. They prove nothing was
|
||||
# sent, and are states, not failures:
|
||||
# - authorization stop -> cancelled (the user turned it off);
|
||||
# - no permission yet -> waiting_for_permission (parked, re-admitted
|
||||
# when a snapshot admits it);
|
||||
# - transient model/provider trouble -> waiting_for_model (parked).
|
||||
# Everything else may have crossed into HY, and HY has no idempotency
|
||||
# key, so it is `unknown` rather than a blind retry.
|
||||
# - no permission yet -> waiting_for_permission (re-admitted when a
|
||||
# snapshot admits it);
|
||||
# - no usable model yet -> waiting_for_model (re-admitted when one is).
|
||||
state: str | None = None
|
||||
if code in PRE_SEND_REFUSAL_CODES:
|
||||
# The frozen input window is checked before the payload reaches
|
||||
# HY. This one refusal therefore has a proven pre-send boundary.
|
||||
state, invoked_marker = FAILED, False
|
||||
elif hy_invoked:
|
||||
# Error names do not prove execution phase: permission and
|
||||
# evidence rechecks also run after HY returns. Once the durable
|
||||
# marker is set, every failure is possibly-sent and must stay out
|
||||
# of the automatic retry path.
|
||||
state, invoked_marker = UNKNOWN, True
|
||||
elif code in POLICY_STOPPED_CODES:
|
||||
state, invoked_marker = CANCELLED, False
|
||||
elif code in PERMISSION_REQUIRED_CODES or code.startswith("model_content_policy"):
|
||||
state, invoked_marker = WAITING_FOR_PERMISSION, False
|
||||
elif code in MODEL_NOT_READY_CODES:
|
||||
state, invoked_marker = WAITING_FOR_MODEL, False
|
||||
elif code == "model_content_transport_request_failed":
|
||||
# The generation or embedding HTTP exchange can fail after HY has
|
||||
# already persisted a node. Treating that as a pre-send failure
|
||||
# permits a blind duplicate; the confirmed unknown flow first
|
||||
# reconciles by request identity and only then requeues absence.
|
||||
state, invoked_marker = UNKNOWN, True
|
||||
elif code.startswith("model_content_") and code != "model_content_outcome_unknown":
|
||||
state, invoked_marker = FAILED, False
|
||||
# Deterministic: the same input exceeds the same window again.
|
||||
state = FAILED
|
||||
elif not hy_invoked and code in POLICY_STOPPED_CODES:
|
||||
state = CANCELLED
|
||||
elif not hy_invoked and (
|
||||
code in PERMISSION_REQUIRED_CODES or code.startswith("model_content_policy")
|
||||
):
|
||||
state = WAITING_FOR_PERMISSION
|
||||
elif not hy_invoked and code in MODEL_NOT_READY_CODES:
|
||||
state = WAITING_FOR_MODEL
|
||||
if state is not None:
|
||||
await in_the_background(
|
||||
queue.settle,
|
||||
job_id,
|
||||
state=state,
|
||||
error_code=code,
|
||||
hy_invoked=hy_invoked,
|
||||
owner=owner,
|
||||
)
|
||||
else:
|
||||
# Force the marker even if this row was not claimed first (a direct
|
||||
# call): the outcome is only `unknown` because HY may have been
|
||||
# reached, and reclaim must treat it as possibly-sent.
|
||||
state, invoked_marker = UNKNOWN, True
|
||||
await in_the_background(
|
||||
queue.settle,
|
||||
job_id,
|
||||
state=state,
|
||||
error_code=code,
|
||||
hy_invoked=invoked_marker,
|
||||
owner=owner,
|
||||
)
|
||||
# Anything else is this job's own trouble: content the model or
|
||||
# the SDK refused, a reply that did not parse, a check that failed
|
||||
# after the write. Retried a few times, then `failed`.
|
||||
state = await in_the_background(
|
||||
queue.retry_or_fail, job_id, error_code=code, owner=owner
|
||||
)
|
||||
# Never `logger.exception` here: it prints the exception's message,
|
||||
# which may be a provider response or the user's own text. Log the
|
||||
# already-sanitised code instead, matching `hy-failure-classification`.
|
||||
@@ -2638,9 +2744,10 @@ async def execute_job(
|
||||
# chain, never its message: `error_code=AttributeError` alone
|
||||
# named no location.
|
||||
logger.error(
|
||||
"durable_write_job_failed job_id=%s error_code=%s chain=%s",
|
||||
"durable_write_job_failed job_id=%s error_code=%s state=%s chain=%s",
|
||||
job_id,
|
||||
code,
|
||||
state or "stale",
|
||||
error_chain_text(error),
|
||||
)
|
||||
return
|
||||
@@ -2665,6 +2772,14 @@ async def execute_job(
|
||||
)
|
||||
|
||||
|
||||
async def _previous_write_receipt(supervisor: Any, request_id: str) -> Any | None:
|
||||
"""The receipt of an earlier write of this job, if the SDK says it succeeded."""
|
||||
inquire = getattr(supervisor, "inquire_write_outcome", None)
|
||||
if inquire is None:
|
||||
return None
|
||||
return await inquire(request_id)
|
||||
|
||||
|
||||
class DurableWriteDrain:
|
||||
"""Run SQLite durable jobs with lease, periodic renewal and no blind retry.
|
||||
|
||||
@@ -2694,6 +2809,7 @@ class DurableWriteDrain:
|
||||
reconcile_unknown: Callable[[], Awaitable[None]] | None = None,
|
||||
source_writes_permitted: Callable[[str], bool] | None = None,
|
||||
source_claim_enabled: Callable[[str], bool] | None = None,
|
||||
model_claim_allowance: Callable[[], int | None] | None = None,
|
||||
) -> None:
|
||||
self.queue = queue
|
||||
self.execute = execute
|
||||
@@ -2769,6 +2885,11 @@ class DurableWriteDrain:
|
||||
self.reconcile_unknown = reconcile_unknown
|
||||
self.source_writes_permitted = source_writes_permitted
|
||||
self.source_claim_enabled = source_claim_enabled or source_writes_permitted
|
||||
#: How many jobs may be claimed now while the model provider is being
|
||||
#: given a rest (`model_backoff`): None for no limit, 0 to claim none.
|
||||
#: Without it a backlog failing in 50ms each was hundreds of requests
|
||||
#: a minute against a provider that refused every one (#245).
|
||||
self.model_claim_allowance = model_claim_allowance
|
||||
self._last_unknown_reconcile: datetime | None = None
|
||||
|
||||
def claim(
|
||||
@@ -2812,6 +2933,14 @@ class DurableWriteDrain:
|
||||
return ()
|
||||
except sqlite3.Error:
|
||||
pass
|
||||
if self.model_claim_allowance is not None:
|
||||
# Asked only once there is work to claim: while the provider is
|
||||
# paused nothing is, and after the pause one job probes it.
|
||||
allowance = self.model_claim_allowance()
|
||||
if allowance is not None:
|
||||
if allowance < 1:
|
||||
return ()
|
||||
limit = min(limit, allowance)
|
||||
enabled_sources: set[str] | None = None
|
||||
if self.source_writes_permitted is not None:
|
||||
with main_database(self.queue.database_path).read() as connection:
|
||||
|
||||
@@ -3163,6 +3163,33 @@ async def _060_memory_note_requests(cursor: Any) -> None:
|
||||
await cursor.execute(statement)
|
||||
|
||||
|
||||
def write_job_retries_ddl(dialect: str = "mysql") -> str:
|
||||
"""Per-job retry state: which SDK request id is next, and retries spent.
|
||||
|
||||
The SDK replays a request id's first outcome forever, so every re-run of
|
||||
a job needs an id of its own (`sdk_attempt`). `retries` counts automatic
|
||||
retries after failures of the job itself; provider failures spend none.
|
||||
"""
|
||||
if dialect == "sqlite":
|
||||
return """
|
||||
ALTER TABLE memory_write_jobs
|
||||
ADD COLUMN sdk_attempt INTEGER NOT NULL DEFAULT 0;
|
||||
ALTER TABLE memory_write_jobs
|
||||
ADD COLUMN retries INTEGER NOT NULL DEFAULT 0;
|
||||
"""
|
||||
return """
|
||||
ALTER TABLE memory_write_jobs
|
||||
ADD COLUMN sdk_attempt INT NOT NULL DEFAULT 0;
|
||||
ALTER TABLE memory_write_jobs
|
||||
ADD COLUMN retries INT NOT NULL DEFAULT 0;
|
||||
"""
|
||||
|
||||
|
||||
async def _061_write_job_retries(cursor: Any) -> None:
|
||||
for statement in _split_statements(write_job_retries_ddl(_dialect(cursor))):
|
||||
await cursor.execute(statement)
|
||||
|
||||
|
||||
def _split_statements(sql: str) -> list[str]:
|
||||
"""Split a multi-statement script, dropping blanks, PRAGMA and comments.
|
||||
|
||||
@@ -3462,4 +3489,5 @@ MIGRATIONS: tuple[Migration, ...] = (
|
||||
_060_memory_note_requests,
|
||||
("memory_note_requests",),
|
||||
),
|
||||
Migration(61, "write_job_retries", _061_write_job_retries),
|
||||
)
|
||||
@@ -57,6 +57,11 @@ class WriteRecord:
|
||||
detail: str = ""
|
||||
raw_ids: tuple[str, ...] = ()
|
||||
memory_ids: tuple[str, ...] = ()
|
||||
#: The stable code of the last failure of a job still being worked on
|
||||
#: (or of a failed one).
|
||||
error_code: str | None = None
|
||||
#: When a waiting job is next tried, if a retry wait holds it.
|
||||
next_retry_at: str | None = None
|
||||
|
||||
|
||||
def _now() -> str:
|
||||
|
||||
@@ -44,6 +44,7 @@ from one_memory.memory_server.durable_queue import (
|
||||
)
|
||||
from one_memory.memory_server.registry import WriteRegistry, WriteState
|
||||
from one_memory.memory_server.submit import WriteConflictError
|
||||
from one_memory.model_backoff import next_retry_at
|
||||
from one_memory.user_scope import DEFAULT_USER_ID, USER_ID_PATTERN, UserId
|
||||
|
||||
#: Durable keys of another user's writes start with this. A `local` request
|
||||
@@ -72,6 +73,12 @@ class MemoryWriteReceipt(BaseModel):
|
||||
detail: str = ""
|
||||
raw_ids: tuple[str, ...] = ()
|
||||
memory_ids: tuple[str, ...] = ()
|
||||
#: Why a job still being worked on last failed (`model_auth_failed`, ...),
|
||||
#: or why a failed one failed. A provider failure keeps `state` queued.
|
||||
error_code: str | None = None
|
||||
#: When it is next tried: the job's own retry wait, or the model
|
||||
#: provider's pause (`model_backoff`) while the provider is failing.
|
||||
next_retry_at: str | None = None
|
||||
|
||||
|
||||
class OpenAIMessage(BaseModel):
|
||||
@@ -235,6 +242,8 @@ def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
|
||||
detail=record.detail,
|
||||
raw_ids=record.raw_ids,
|
||||
memory_ids=record.memory_ids,
|
||||
error_code=record.error_code,
|
||||
next_retry_at=next_retry_at(request.app, record.state, record.next_retry_at),
|
||||
)
|
||||
|
||||
return instance
|
||||
@@ -0,0 +1,328 @@
|
||||
"""One pause for every model request while the provider keeps failing (#245).
|
||||
|
||||
A broken model configuration (a revoked key, a wrong base URL, a model that
|
||||
does not exist, a provider that is down) used to fail each queued job on its
|
||||
own: the drain claimed the next job the moment the last one failed, so a
|
||||
backlog of hundreds of jobs turned into requests as fast as the provider could
|
||||
refuse them. The pause is global, not per job: a failure says something about
|
||||
the provider, not about the conversation that happened to hit it.
|
||||
|
||||
After the first failure no further model work is claimed for 5 seconds; each
|
||||
failed probe after that doubles the pause, up to 3 minutes. A pause admits one
|
||||
probe job at a time. The first success, or any change the user makes to the
|
||||
model configuration, ends it at once. Jobs not yet sent stay `pending`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
from collections.abc import Callable, Iterator
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
BACKOFF_FIRST_SECONDS = 5.0
|
||||
BACKOFF_CAP_SECONDS = 180.0
|
||||
#: How long one probe may hold the pause open before another is admitted. A
|
||||
#: probe job that never reaches the provider (nothing to remember, a refusal
|
||||
#: before the SDK) reports nothing, and the queue must not wait on it forever.
|
||||
PROBE_HOLD_SECONDS = 30.0
|
||||
#: Requests already in flight when the pause began are refused without going
|
||||
#: out once this many failures in a row confirm it is not a one-off.
|
||||
REFUSE_IN_FLIGHT_AFTER = 2
|
||||
|
||||
#: Statuses that say the configuration or the provider is wrong, not the one
|
||||
#: request: authentication, a missing model or route, throttling, a timeout
|
||||
#: and the provider's own failures. A 400/413/422 is about the request's
|
||||
#: content and is left to that job.
|
||||
_PROVIDER_STATUSES = {401: "model_auth_failed", 403: "model_auth_failed", 404: "model_not_found"}
|
||||
|
||||
#: Every code a provider failure is recorded under: a job that failed with one
|
||||
#: of these waits for the provider instead of spending a retry.
|
||||
MODEL_FAILURE_CODES = frozenset(
|
||||
{
|
||||
"model_auth_failed",
|
||||
"model_not_found",
|
||||
"model_timeout",
|
||||
"model_rate_limited",
|
||||
"model_server_error",
|
||||
"model_unreachable",
|
||||
"model_transport_failed",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def backoff_seconds(level: int) -> float:
|
||||
"""The pause after the `level`-th failed attempt in a row (1-based)."""
|
||||
if level < 1:
|
||||
return 0.0
|
||||
return min(BACKOFF_FIRST_SECONDS * float(2 ** min(level - 1, 16)), BACKOFF_CAP_SECONDS)
|
||||
|
||||
|
||||
def failure_code(status: int) -> str | None:
|
||||
"""The stable code for a provider HTTP status, or None if it is no provider failure."""
|
||||
if status in _PROVIDER_STATUSES:
|
||||
return _PROVIDER_STATUSES[status]
|
||||
if status in {408, 504}:
|
||||
return "model_timeout"
|
||||
if status == 429:
|
||||
return "model_rate_limited"
|
||||
if status >= 500:
|
||||
return "model_server_error"
|
||||
return None
|
||||
|
||||
|
||||
def transport_failure_code(error: BaseException) -> str:
|
||||
if isinstance(error, httpx.TimeoutException):
|
||||
return "model_timeout"
|
||||
if isinstance(error, httpx.ConnectError | httpx.NetworkError):
|
||||
return "model_unreachable"
|
||||
return "model_transport_failed"
|
||||
|
||||
|
||||
def request_kind(request: httpx.Request) -> str:
|
||||
"""`embedding` for the embeddings route, `chat` for everything else."""
|
||||
return "embedding" if request.url.path.rstrip("/").endswith("/embeddings") else "chat"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ModelBackoffState:
|
||||
#: When the pause ends; None when no pause is in force.
|
||||
until: datetime | None
|
||||
#: Failed attempts in a row.
|
||||
failures: int
|
||||
#: The last failure's stable code (`model_auth_failed`, ...).
|
||||
last_error: str | None
|
||||
profile_id: str | None
|
||||
|
||||
|
||||
class ModelBackoffRefusal(httpx.TransportError):
|
||||
"""A request refused locally because the provider is being given a rest."""
|
||||
|
||||
|
||||
@dataclass(eq=False)
|
||||
class ModelAttempt:
|
||||
"""One job's write, as the provider saw it.
|
||||
|
||||
`failure` is the code of the first provider failure (or local refusal)
|
||||
that happened while this write was in flight, None if there was none.
|
||||
"""
|
||||
|
||||
failure: str | None = None
|
||||
|
||||
|
||||
class ModelBackoff:
|
||||
"""Process-wide circuit breaker for the active model profile.
|
||||
|
||||
Thread-safe: the transport records from the SDK's own loop thread, the
|
||||
drain asks from its claim thread, and status reads from the event loop.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
clock: Callable[[], float] = time.monotonic,
|
||||
wall_clock: Callable[[], datetime] = lambda: datetime.now(UTC),
|
||||
) -> None:
|
||||
self._clock = clock
|
||||
self._wall_clock = wall_clock
|
||||
self._lock = threading.Lock()
|
||||
#: Writes in flight (`attempt`); never cleared by a reset.
|
||||
self._attempts: set[ModelAttempt] = set()
|
||||
self._reset_locked()
|
||||
|
||||
def _reset_locked(self) -> None:
|
||||
#: Which kinds of request are failing (`chat`, `embedding`): a working
|
||||
#: embedding endpoint says nothing about a chat model that is gone.
|
||||
self._failing: set[str] = set()
|
||||
self._level = 0
|
||||
self._failures = 0
|
||||
self._until = 0.0
|
||||
self._probe_until = 0.0
|
||||
self._last_error: str | None = None
|
||||
self._profile_id: str | None = None
|
||||
|
||||
def reset(self, reason: str = "model_configuration_changed") -> None:
|
||||
"""Forget every failure: the user changed what is being called."""
|
||||
with self._lock:
|
||||
was_paused = self._failures > 0
|
||||
self._reset_locked()
|
||||
if was_paused:
|
||||
logger.info("model_backoff_reset reason=%s", reason)
|
||||
|
||||
def record_success(self, profile_id: str | None = None, kind: str = "model") -> None:
|
||||
with self._lock:
|
||||
if self._failures == 0:
|
||||
return
|
||||
if profile_id is not None and self._profile_id not in {None, profile_id}:
|
||||
return
|
||||
self._failing.discard(kind)
|
||||
if self._failing:
|
||||
return
|
||||
self._reset_locked()
|
||||
logger.info("model_backoff_cleared profile_id=%s", profile_id)
|
||||
|
||||
def record_failure(self, code: str, profile_id: str | None = None, kind: str = "model") -> None:
|
||||
now = self._clock()
|
||||
with self._lock:
|
||||
if profile_id is not None and self._profile_id not in {None, profile_id}:
|
||||
# Another profile's history says nothing about this one.
|
||||
self._reset_locked()
|
||||
self._profile_id = profile_id
|
||||
self._failing.add(kind)
|
||||
self._mark_attempts_locked(code)
|
||||
self._failures += 1
|
||||
self._last_error = code
|
||||
self._probe_until = 0.0
|
||||
# Requests that were already out when the pause began fail too;
|
||||
# they are the same outage, not a reason to wait longer.
|
||||
if now >= self._until:
|
||||
self._level += 1
|
||||
self._until = now + backoff_seconds(self._level)
|
||||
level, pause = self._level, self._until - now
|
||||
logger.warning(
|
||||
"model_backoff_failure error_code=%s level=%d pause_s=%.0f profile_id=%s",
|
||||
code,
|
||||
level,
|
||||
pause,
|
||||
profile_id,
|
||||
)
|
||||
|
||||
@contextmanager
|
||||
def attempt(self) -> Iterator[ModelAttempt]:
|
||||
"""Hold one job's write open, and learn whether the provider failed it.
|
||||
|
||||
Every provider failure and every local refusal while the write is in
|
||||
flight is charged to it. A request cannot be traced to the job that
|
||||
made it: the SDK sends embeddings from one shared batching worker, so
|
||||
neither the calling thread nor a context variable identifies the job.
|
||||
Charging every write then in flight can name a write the provider did
|
||||
not fail (another job's request failed beside it); that write waits
|
||||
for the provider and is retried with the rest. It can never miss the
|
||||
write the provider did fail, which would spend that job's retries on
|
||||
an outage.
|
||||
"""
|
||||
record = ModelAttempt()
|
||||
with self._lock:
|
||||
self._attempts.add(record)
|
||||
try:
|
||||
yield record
|
||||
finally:
|
||||
with self._lock:
|
||||
self._attempts.discard(record)
|
||||
|
||||
def record_refusal(self) -> str:
|
||||
"""A request refused locally; charged to the writes in flight like a failure."""
|
||||
with self._lock:
|
||||
code = self._last_error or "model_transport_failed"
|
||||
self._mark_attempts_locked(code)
|
||||
return code
|
||||
|
||||
def _mark_attempts_locked(self, code: str) -> None:
|
||||
for record in self._attempts:
|
||||
if record.failure is None:
|
||||
record.failure = code
|
||||
|
||||
def paused(self) -> bool:
|
||||
with self._lock:
|
||||
return self._failures > 0 and self._clock() < self._until
|
||||
|
||||
def refuses_requests(self, kind: str = "model") -> bool:
|
||||
"""Whether a request of this kind should not go out at all right now.
|
||||
|
||||
Only the failing kind is refused: a chat model that is gone must not
|
||||
take recall's embedding calls down with it.
|
||||
"""
|
||||
with self._lock:
|
||||
return (
|
||||
kind in self._failing
|
||||
and self._failures >= REFUSE_IN_FLIGHT_AFTER
|
||||
and self._clock() < self._until
|
||||
)
|
||||
|
||||
def allowance(self) -> int | None:
|
||||
"""How many model jobs may be claimed now: None for no limit.
|
||||
|
||||
0 while paused; once the pause is over, one probe at a time until it
|
||||
reports or `PROBE_HOLD_SECONDS` pass.
|
||||
"""
|
||||
now = self._clock()
|
||||
with self._lock:
|
||||
if self._failures == 0:
|
||||
return None
|
||||
if now < self._until or now < self._probe_until:
|
||||
return 0
|
||||
self._probe_until = now + PROBE_HOLD_SECONDS
|
||||
return 1
|
||||
|
||||
def state(self) -> ModelBackoffState:
|
||||
now = self._clock()
|
||||
with self._lock:
|
||||
remaining = self._until - now if self._failures else 0.0
|
||||
return ModelBackoffState(
|
||||
until=(
|
||||
self._wall_clock() + timedelta(seconds=remaining) if remaining > 0 else None
|
||||
),
|
||||
failures=self._failures,
|
||||
last_error=self._last_error,
|
||||
profile_id=self._profile_id,
|
||||
)
|
||||
|
||||
|
||||
def next_retry_at(app: Any, state: str, own: str | None) -> str | None:
|
||||
"""When a queued job is tried next: its own retry wait, or the provider's pause.
|
||||
|
||||
While the breaker pauses, nothing is claimed at all, so the pause holds
|
||||
every queued job, whatever it last failed with.
|
||||
"""
|
||||
if state not in {"accepted", "pending"}:
|
||||
return None
|
||||
backoff = getattr(getattr(app, "state", None), "model_backoff", None)
|
||||
until = backoff.state().until if isinstance(backoff, ModelBackoff) else None
|
||||
paused = until.isoformat() if until is not None else None
|
||||
candidates = [value for value in (own, paused) if value]
|
||||
return max(candidates) if candidates else None
|
||||
|
||||
|
||||
class BackedOffModelTransport(httpx.AsyncBaseTransport):
|
||||
"""Report each provider exchange to the breaker; refuse while it confirms a pause."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
backoff: ModelBackoff,
|
||||
*,
|
||||
profile_id: str,
|
||||
inner: httpx.AsyncBaseTransport | None = None,
|
||||
) -> None:
|
||||
self.backoff = backoff
|
||||
self.profile_id = profile_id
|
||||
self.inner = inner or httpx.AsyncHTTPTransport(retries=0, trust_env=False)
|
||||
|
||||
async def handle_async_request(self, request: httpx.Request) -> httpx.Response:
|
||||
kind = request_kind(request)
|
||||
if self.backoff.refuses_requests(kind):
|
||||
self.backoff.record_refusal()
|
||||
raise ModelBackoffRefusal("model_backoff_active", request=request)
|
||||
try:
|
||||
response = await self.inner.handle_async_request(request)
|
||||
except ModelBackoffRefusal:
|
||||
raise
|
||||
except httpx.TransportError as error:
|
||||
self.backoff.record_failure(transport_failure_code(error), self.profile_id, kind)
|
||||
raise
|
||||
code = failure_code(response.status_code)
|
||||
if code is not None:
|
||||
self.backoff.record_failure(code, self.profile_id, kind)
|
||||
elif response.status_code < 400:
|
||||
self.backoff.record_success(self.profile_id, kind)
|
||||
return response
|
||||
|
||||
async def aclose(self) -> None:
|
||||
await self.inner.aclose()
|
||||
@@ -135,6 +135,8 @@ class RecallReceipt(BaseModel):
|
||||
#: exists yet (`fallback_reason`). Raw hits are excerpts, not memories.
|
||||
mode: Literal["memory", "raw"] = "memory"
|
||||
fallback_reason: Literal["model_unavailable", "no_memories"] | None = None
|
||||
#: Whose memories were searched: the request's `user_id`, else `local`.
|
||||
user_id: str = DEFAULT_USER_ID
|
||||
|
||||
|
||||
class RecallTarget(Protocol):
|
||||
|
||||
@@ -32,6 +32,19 @@ FallbackReason = Literal["model_unavailable", "no_memories"]
|
||||
#: caller). An empty set means the caller may read no raw conversation.
|
||||
RawSources = frozenset[str] | None
|
||||
|
||||
#: What the whole recall request may take beyond the caller's own
|
||||
#: `timeout_ms`: the search keeps to that budget, but the checks around it
|
||||
#: (is a model configured, is the library empty, which raw sources) and the
|
||||
#: raw fallback need a little room of their own. Well under the broker's 15 s
|
||||
#: wait on Core, so a slow Core answers 504 before the broker gives up.
|
||||
RECALL_TOTAL_GRACE_SECONDS = 3.0
|
||||
RECALL_TOTAL_CEILING_SECONDS = 12.0
|
||||
|
||||
|
||||
def recall_total_timeout_seconds(timeout_ms: int) -> float:
|
||||
"""The most one recall request may take from start to answer."""
|
||||
return min(timeout_ms / 1_000 + RECALL_TOTAL_GRACE_SECONDS, RECALL_TOTAL_CEILING_SECONDS)
|
||||
|
||||
|
||||
async def recall_with_fallback(
|
||||
request: RecallRequest,
|
||||
|
||||
@@ -33,7 +33,7 @@ from one_memory.model_profiles import (
|
||||
)
|
||||
from one_memory.sdk_search_options import SdkSearchFilters, SearchLane
|
||||
from one_memory.secrets import SecretStore, SecretStoreUnavailableError
|
||||
from one_memory.user_scope import owns
|
||||
from one_memory.user_scope import DEFAULT_USER_ID, owns
|
||||
|
||||
_NO_GROUNDING = "现有记忆不足以回答这个问题。"
|
||||
_GROUNDING_LIMIT = 6
|
||||
@@ -62,6 +62,8 @@ class SdkMemoryAnswer(BaseModel):
|
||||
answer: str
|
||||
memories: tuple[SdkAskMemory, ...]
|
||||
mode: Literal["generated", "extractive", "empty"]
|
||||
#: Whose memories answered: the request's `user_id`, else `local`.
|
||||
user_id: str = DEFAULT_USER_ID
|
||||
|
||||
|
||||
class _PromptMemory:
|
||||
|
||||
@@ -25,7 +25,7 @@ from one_memory.hy_runtime_config import HY_MEMORY_READER
|
||||
from one_memory.hy_sdk_runtime import SdkClientHandle, sdk_relevance_floor
|
||||
from one_memory.hy_sdk_types import HySdkError, parse_sdk_search
|
||||
from one_memory.memory_queue import ExtractionWindowPreset, QueueDepths
|
||||
from one_memory.memory_server.durable_queue import COMPLETED, DurableWriteQueue
|
||||
from one_memory.memory_server.durable_queue import DurableWriteQueue
|
||||
from one_memory.sdk_write_processor import SdkWriteCoordinator
|
||||
from one_memory.semantic_deadline import SemanticMemoryHit
|
||||
|
||||
@@ -34,6 +34,10 @@ _STABLE_CODE = re.compile(r"[a-z][a-z0-9_]{0,95}")
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: Jobs not yet run that a suspended source holds back.
|
||||
_WAITING_STATES = frozenset({"pending", "waiting_for_model", "waiting_for_permission"})
|
||||
_WAITING_SQL = ",".join(f"'{state}'" for state in sorted(_WAITING_STATES))
|
||||
|
||||
|
||||
class SdkMemoryRuntime:
|
||||
"""Product services that remain after legacy task/governance writes retire."""
|
||||
@@ -48,12 +52,19 @@ class SdkMemoryRuntime:
|
||||
data_dir: Path,
|
||||
memory_mode: Callable[[], str] = lambda: "pro",
|
||||
writes_open: Callable[[], bool] = lambda: True,
|
||||
source_active: Callable[[str], bool] = lambda _source_key: True,
|
||||
) -> None:
|
||||
self.coordinator = coordinator
|
||||
#: The SDK mode, for the graph store's path (`hy_graph_admin`).
|
||||
self.memory_mode = memory_mode
|
||||
#: False while an embedding switch pauses model writes.
|
||||
self.writes_open = writes_open
|
||||
#: Whether a queue lane's source may still become memories: the same
|
||||
#: question the drain's claim asks of it. A lane that is off (the
|
||||
#: user turned the source off, or its grant was revoked) has its
|
||||
#: waiting jobs suspended: kept, never claimed, and not counted as
|
||||
#: pending until the source is on again.
|
||||
self.source_active = source_active
|
||||
self.read_handle = read_handle
|
||||
self.write_queue = write_queue
|
||||
self.extraction_settings = extraction_settings
|
||||
@@ -241,22 +252,11 @@ class SdkMemoryRuntime:
|
||||
now = datetime.now(UTC)
|
||||
fresh_since = (now - timedelta(days=7)).isoformat()
|
||||
with main_database(self.write_queue.database_path).read() as database:
|
||||
fresh = int(
|
||||
database.execute(
|
||||
"SELECT COUNT(*) FROM memory_write_jobs "
|
||||
"WHERE state IN ('pending','waiting_for_model','waiting_for_permission') "
|
||||
"AND event_at>=?",
|
||||
(fresh_since,),
|
||||
).fetchone()[0]
|
||||
)
|
||||
backlog = int(
|
||||
database.execute(
|
||||
"SELECT COUNT(*) FROM memory_write_jobs "
|
||||
"WHERE state IN ('pending','waiting_for_model','waiting_for_permission') "
|
||||
"AND event_at<?",
|
||||
(fresh_since,),
|
||||
).fetchone()[0]
|
||||
)
|
||||
rows = database.execute(
|
||||
"SELECT source_key,event_at>=? AS fresh,COUNT(*) FROM memory_write_jobs "
|
||||
f"WHERE state IN ({_WAITING_SQL}) GROUP BY source_key,fresh",
|
||||
(fresh_since,),
|
||||
).fetchall()
|
||||
recent = tuple(
|
||||
datetime.fromisoformat(str(row[0]))
|
||||
for row in database.execute(
|
||||
@@ -264,6 +264,9 @@ class SdkMemoryRuntime:
|
||||
"ORDER BY updated_at DESC LIMIT 8"
|
||||
)
|
||||
)
|
||||
active = self._active_lanes({str(row[0]) for row in rows})
|
||||
fresh = sum(int(row[2]) for row in rows if str(row[0]) in active and row[1])
|
||||
backlog = sum(int(row[2]) for row in rows if str(row[0]) in active and not row[1])
|
||||
return QueueDepths(fresh=fresh, backlog=backlog, recent_completions=recent)
|
||||
|
||||
def active_depths(self) -> tuple[int, int]:
|
||||
@@ -276,13 +279,27 @@ class SdkMemoryRuntime:
|
||||
return running, running
|
||||
|
||||
def status_counts(self) -> dict[str, int]:
|
||||
"""Jobs per state, with a suspended source's waiting jobs as `suspended`.
|
||||
|
||||
Those jobs keep their state in the table: suspension is the source's
|
||||
switch, not the row's, so turning the source back on is all it takes
|
||||
for the drain to claim them again, and nothing is moved or deleted.
|
||||
"""
|
||||
with main_database(self.write_queue.database_path).read() as database:
|
||||
return {
|
||||
str(row[0]): int(row[1])
|
||||
for row in database.execute(
|
||||
"SELECT state,COUNT(*) FROM memory_write_jobs GROUP BY state"
|
||||
)
|
||||
}
|
||||
rows = database.execute(
|
||||
"SELECT source_key,state,COUNT(*) FROM memory_write_jobs GROUP BY source_key,state"
|
||||
).fetchall()
|
||||
active = self._active_lanes({str(row[0]) for row in rows if str(row[1]) in _WAITING_STATES})
|
||||
counts: dict[str, int] = {}
|
||||
for source_key, state, count in rows:
|
||||
name = str(state)
|
||||
if name in _WAITING_STATES and str(source_key) not in active:
|
||||
name = "suspended"
|
||||
counts[name] = counts.get(name, 0) + int(count)
|
||||
return counts
|
||||
|
||||
def _active_lanes(self, lanes: set[str]) -> set[str]:
|
||||
return {lane for lane in lanes if self.source_active(lane)}
|
||||
|
||||
def manual_batch_progress(self) -> tuple[int, int]:
|
||||
return 0, 0
|
||||
@@ -291,31 +308,6 @@ class SdkMemoryRuntime:
|
||||
del task_ids
|
||||
return []
|
||||
|
||||
async def reconcile_unknown_outputs(
|
||||
self,
|
||||
*,
|
||||
limit: int = 128,
|
||||
exclude_error_codes: tuple[str, ...] = (),
|
||||
) -> int:
|
||||
reconciled = 0
|
||||
for job in await asyncio.to_thread(
|
||||
self.write_queue.unknown_jobs,
|
||||
limit=limit,
|
||||
exclude_error_codes=exclude_error_codes,
|
||||
):
|
||||
receipt = await self.coordinator.inquire_write_outcome(job.request_id)
|
||||
if receipt is None:
|
||||
continue
|
||||
await asyncio.to_thread(
|
||||
self.write_queue.settle,
|
||||
job.job_id,
|
||||
state=COMPLETED,
|
||||
sdk_raw_ids=receipt.raw_ids,
|
||||
sdk_memory_ids=receipt.memory_ids,
|
||||
)
|
||||
reconciled += 1
|
||||
return reconciled
|
||||
|
||||
async def resume_allowed(self) -> None:
|
||||
await asyncio.to_thread(self.write_queue.readmit_parked)
|
||||
|
||||
|
||||
@@ -29,6 +29,7 @@ from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from one_memory.background_work import in_sdk_read
|
||||
from one_memory.embedding_health import EmbeddingGuardedClient, EmbeddingHealth
|
||||
from one_memory.embedding_spaces import (
|
||||
EmbeddingSpace,
|
||||
@@ -112,7 +113,7 @@ class ActiveProfileReadHandle(SdkClientHandle):
|
||||
|
||||
async def current_handle(self) -> SdkClientHandle:
|
||||
"""The handle of the live space, held for the asking task."""
|
||||
space, identity = await asyncio.to_thread(self._resolve)
|
||||
space, identity = await in_sdk_read(self._resolve)
|
||||
mode, features = self.memory_mode(), self.current_features()
|
||||
language = self.current_language()
|
||||
key = (
|
||||
@@ -127,9 +128,7 @@ class ActiveProfileReadHandle(SdkClientHandle):
|
||||
if handle is None:
|
||||
# Only on a change of space or profile: resolving a key can
|
||||
# reach the system keychain, so it is not done on every read.
|
||||
handle = await asyncio.to_thread(
|
||||
self._build, space, identity, mode, features, language
|
||||
)
|
||||
handle = await in_sdk_read(self._build, space, identity, mode, features, language)
|
||||
self._handles[key] = handle
|
||||
self._retired.update(value for name, value in self._handles.items() if name != key)
|
||||
self._handles = {key: handle}
|
||||
|
||||
@@ -3,12 +3,14 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import threading
|
||||
from collections.abc import AsyncIterator, Awaitable, Callable, Mapping, Sequence
|
||||
from contextlib import asynccontextmanager
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from one_memory.background_work import in_sdk_write
|
||||
from one_memory.embedding_spaces import (
|
||||
EmbeddingSpace,
|
||||
EmbeddingSpaceStore,
|
||||
@@ -28,6 +30,61 @@ from one_memory.model_profiles import ModelProfileSnapshot, ModelProfileStore
|
||||
from one_memory.secrets import SecretStore, SecretStoreUnavailableError
|
||||
|
||||
PRODUCT_AGENT_ID = "all-apps"
|
||||
|
||||
#: SDK request ids whose write call is running on a thread right now. A write
|
||||
#: the drain gave up on (its work ceiling, a lost lease) keeps running on its
|
||||
#: thread; it must not be re-run beside itself (`recover_unknown`).
|
||||
_IN_FLIGHT: set[str] = set()
|
||||
_IN_FLIGHT_LOCK = threading.Lock()
|
||||
|
||||
|
||||
def sdk_write_in_flight(request_id: str) -> bool:
|
||||
with _IN_FLIGHT_LOCK:
|
||||
return request_id in _IN_FLIGHT
|
||||
|
||||
|
||||
class _Tracked:
|
||||
"""One SDK write call, held in `_IN_FLIGHT` from submission until it ends.
|
||||
|
||||
If the wait for it is cancelled before a thread picked it up, it is
|
||||
abandoned: it will not start, and it leaves `_IN_FLIGHT` at once.
|
||||
"""
|
||||
|
||||
def __init__(self, request_id: str, work: Callable[..., Any]) -> None:
|
||||
self.request_id = request_id
|
||||
self.work = work
|
||||
self.started = False
|
||||
self.abandoned = False
|
||||
with _IN_FLIGHT_LOCK:
|
||||
_IN_FLIGHT.add(request_id)
|
||||
|
||||
def __call__(self, *args: Any, **kwargs: Any) -> Any:
|
||||
with _IN_FLIGHT_LOCK:
|
||||
if self.abandoned:
|
||||
raise RuntimeError("hy_memory_write_abandoned")
|
||||
self.started = True
|
||||
try:
|
||||
return self.work(*args, **kwargs)
|
||||
finally:
|
||||
with _IN_FLIGHT_LOCK:
|
||||
_IN_FLIGHT.discard(self.request_id)
|
||||
|
||||
def abandon(self) -> None:
|
||||
with _IN_FLIGHT_LOCK:
|
||||
if not self.started:
|
||||
self.abandoned = True
|
||||
_IN_FLIGHT.discard(self.request_id)
|
||||
|
||||
|
||||
async def _write(tracked_id: str, work: Callable[..., Any], /, *args: Any, **kwargs: Any) -> Any:
|
||||
call = _Tracked(tracked_id, work)
|
||||
try:
|
||||
return await in_sdk_write(call, *args, **kwargs)
|
||||
except asyncio.CancelledError:
|
||||
call.abandon()
|
||||
raise
|
||||
|
||||
|
||||
_MESSAGE_ROLES = frozenset({"user", "assistant", "tool"})
|
||||
_CONVERSATION_ROLES = frozenset({"user", "assistant"})
|
||||
|
||||
@@ -82,7 +139,8 @@ class SdkWriteProcessor:
|
||||
if extracted and not self._lite:
|
||||
contents = [sanitize_model_input(record.verbatim) for record in records]
|
||||
await before_hy_invocation()
|
||||
result = await asyncio.to_thread(
|
||||
result = await _write(
|
||||
request_id,
|
||||
client.add_extracted,
|
||||
contents,
|
||||
user_id=scope.user_id,
|
||||
@@ -109,7 +167,8 @@ class SdkWriteProcessor:
|
||||
if not messages:
|
||||
raise RuntimeError("hy_memory_task_without_inputs")
|
||||
await before_hy_invocation()
|
||||
result = await asyncio.to_thread(
|
||||
result = await _write(
|
||||
request_id,
|
||||
client.add,
|
||||
messages,
|
||||
user_id=scope.user_id,
|
||||
@@ -130,7 +189,7 @@ class SdkWriteProcessor:
|
||||
if not request_id:
|
||||
raise ValueError("request_id is required")
|
||||
client = await self._client_handle.client()
|
||||
outcome = await asyncio.to_thread(client.get_write_outcome, request_id)
|
||||
outcome = await in_sdk_write(client.get_write_outcome, request_id)
|
||||
if outcome is None:
|
||||
return None
|
||||
if not isinstance(outcome, Mapping):
|
||||
|
||||
@@ -3,14 +3,19 @@
|
||||
The App owns the choice (its language selector) and tells Core whenever it
|
||||
changes, so Core can build every SDK client with it. It is one global value in
|
||||
`config.json` (`ui_language`), beside `memory_mode` and `memory_features`;
|
||||
absent (never chosen, or "follow the system") means no explicit choice.
|
||||
absent means no language was set (a CLI/API-only install, or
|
||||
`settings language system`). An App that follows the system sends the
|
||||
language it resolved and shows, not null.
|
||||
|
||||
The owner's rule for the SDK:
|
||||
|
||||
- zh-Hans, or no explicit choice: pass no language. The SDK keeps detecting
|
||||
Chinese or English from the text, exactly as before this setting existed.
|
||||
- Any other code, including en: pass it as `MemoryConfig.language`, so
|
||||
extraction and the other prompts use that language.
|
||||
- Any supported code, zh-Hans and en included: pass it as
|
||||
`MemoryConfig.language`, so extraction and the other prompts use that
|
||||
language. Every display language is passed explicitly (#243).
|
||||
- No language at all (never set, as on a CLI/API-only install): pass no
|
||||
language. The SDK keeps detecting Chinese or English from the text, exactly
|
||||
as before this setting existed. The App does not leave it unset: it sends
|
||||
the language it shows, including when it follows the system.
|
||||
|
||||
The language never names a collection; a change rebuilds the SDK handles the
|
||||
way a mode or feature change does (#218 use count).
|
||||
@@ -51,8 +56,6 @@ UI_LANGUAGES: tuple[UiLanguage, ...] = (
|
||||
)
|
||||
#: Codes earlier App builds stored, and the code each one is now.
|
||||
LEGACY_UI_LANGUAGES: dict[str, UiLanguage] = {"zh-CN": "zh-Hans"}
|
||||
#: Display languages that leave the SDK's own zh/en detection in charge.
|
||||
AUTO_DETECTED: frozenset[str] = frozenset({"zh-Hans"})
|
||||
#: What status shows when no language is passed to the SDK.
|
||||
SDK_LANGUAGE_AUTO = "auto"
|
||||
|
||||
@@ -66,8 +69,9 @@ def normalize_ui_language(value: object) -> UiLanguage | None:
|
||||
|
||||
|
||||
def sdk_language(ui_language: str | None) -> str | None:
|
||||
"""The `MemoryConfig.language` for a display language; None = SDK auto-detect."""
|
||||
language = normalize_ui_language(ui_language)
|
||||
if language is None or language in AUTO_DETECTED:
|
||||
return None
|
||||
return language
|
||||
"""The `MemoryConfig.language` for a display language; None = SDK auto-detect.
|
||||
|
||||
Every supported code is passed as is; only an unset (or unknown) value
|
||||
leaves the SDK to detect zh/en.
|
||||
"""
|
||||
return normalize_ui_language(ui_language)
|
||||
@@ -25,6 +25,7 @@
|
||||
| `test_c1_cross_source_session_isolation.py` | `contract` | Same session name in two sources must stay two conversations. |
|
||||
| `test_c1_damage_per_source.py` | `contract` | Damage and unknown schema, per source: a bad record must not sink the import. |
|
||||
| `test_c1_deepseek_profiles.py` | `contract` | DeepSeek Harness has two profiles; only the headless session form is importable. |
|
||||
| `test_c1_deepseek_versioned_sessions.py` | `contract` | DeepSeek Harness versions its session file names; every generation must import. |
|
||||
| `test_c1_import_event_matrix.py` | `contract` | C1 owned matrix: existing history, new file, append, truncate/replace, restart, revoke. |
|
||||
| `test_c1_import_failure_injection.py` | `contract` | C1 failure injection: atomic receipt, disk-full, read-only source, rotation. |
|
||||
| `test_c1_import_instrumentation.py` | `contract` | Instrumented import accounting: bytes read, opens, hashes, jobs, duplicates. |
|
||||
@@ -189,6 +190,7 @@
|
||||
| `test_memory_write_registry.py` | `contract` | A3-01: acceptance and outcome recording for the unified write path. |
|
||||
| `test_memory_write_routes.py` | `integration` | A3-01: the HTTP write entry, tested against a real FastAPI app. |
|
||||
| `test_migrate_off_sqlite.py` | `contract` | A1-06: the migration is only done when reconciliation agrees. |
|
||||
| `test_model_backoff.py` | `contract` | A broken model configuration must not turn a backlog into a request storm (#245). |
|
||||
| `test_model_creation_api.py` | `integration` | Normal protected model creation routes with temporary data and in-memory secrets. |
|
||||
| `test_model_creation_operations.py` | `contract` | 覆盖 lookup without table is read only and legacy create does not initialize、operation commits with profile and version without autoactivation、any replay conflicts before secret put 等 21 个场景。 |
|
||||
| `test_model_creation_validation_boundary.py` | `integration` | Real ASGI validation must precede creation reservation and secret writes. |
|
||||
@@ -318,4 +320,4 @@
|
||||
| `test_workspace_recall.py` | `contract` | Workspace memories were unreachable from every host; these pin the way in. |
|
||||
| `test_write_queue_lease_renewal.py` | `contract` | Rule 8: a lease must outlive the work it covers, and be renewed while it runs. |
|
||||
|
||||
共 313 个测试文件。
|
||||
共 315 个测试文件。
|
||||
@@ -14,8 +14,8 @@ from pathlib import Path
|
||||
import pytest
|
||||
|
||||
PRIMARY_MARKERS = ("unit", "contract", "integration", "performance", "live")
|
||||
EXPECTED_TEST_FILE_COUNT = 313
|
||||
EXPECTED_TEST_FILES_SHA256 = "dcada5d5421767196a79e415dd47c2b6c6d67681972dbe81481b31867d318928"
|
||||
EXPECTED_TEST_FILE_COUNT = 315
|
||||
EXPECTED_TEST_FILES_SHA256 = "681258b2a13ca852e273707071b286afe6b0b2fe537f90a8a2c64d83ec662772"
|
||||
|
||||
# Pure transformations, validation models, parity checks and state machines.
|
||||
# This list is intentionally small: uncertain files default to contract.
|
||||
|
||||
@@ -324,3 +324,48 @@ def test_saturated_raw_reads_do_not_delay_memory_reads() -> None:
|
||||
finally:
|
||||
release_raw.set()
|
||||
shutdown_bulk_pool()
|
||||
|
||||
|
||||
def test_recall_reads_answer_while_every_sdk_write_thread_is_stuck() -> None:
|
||||
"""A write that hangs in the SDK keeps its thread, not recall's (#231)."""
|
||||
from one_memory.background_work import SDK_WRITE_WORKERS, in_sdk_read, in_sdk_write
|
||||
|
||||
stuck = threading.Barrier(SDK_WRITE_WORKERS + 1)
|
||||
release = threading.Event()
|
||||
|
||||
def hung_write() -> None:
|
||||
stuck.wait(timeout=2)
|
||||
assert release.wait(5)
|
||||
|
||||
async def scenario() -> tuple[float, int]:
|
||||
writes = [asyncio.create_task(in_sdk_write(hung_write)) for _ in range(SDK_WRITE_WORKERS)]
|
||||
# One more than the pool holds: it waits for a write thread, as the
|
||||
# drain's next write would.
|
||||
queued = asyncio.create_task(in_sdk_write(lambda: None))
|
||||
await asyncio.to_thread(stuck.wait, 2)
|
||||
started = time.monotonic()
|
||||
answer = await asyncio.wait_for(in_sdk_read(lambda: 7), 1)
|
||||
elapsed = time.monotonic() - started
|
||||
assert not queued.done()
|
||||
release.set()
|
||||
await asyncio.gather(*writes, queued)
|
||||
return elapsed, answer
|
||||
|
||||
try:
|
||||
elapsed, answer = asyncio.run(scenario())
|
||||
assert answer == 7
|
||||
assert elapsed < 0.2
|
||||
finally:
|
||||
release.set()
|
||||
shutdown_bulk_pool()
|
||||
|
||||
|
||||
def test_sdk_writes_run_on_their_own_bounded_threads() -> None:
|
||||
from one_memory.background_work import SDK_WRITE_WORKERS, in_sdk_write, sdk_write_pool
|
||||
|
||||
try:
|
||||
name = asyncio.run(in_sdk_write(lambda: threading.current_thread().name))
|
||||
assert name.startswith("one-memory-sdk-write")
|
||||
assert sdk_write_pool()._max_workers == SDK_WRITE_WORKERS
|
||||
finally:
|
||||
shutdown_bulk_pool()
|
||||
@@ -0,0 +1,246 @@
|
||||
"""DeepSeek Harness versions its session file names; every generation must import.
|
||||
|
||||
The first format writes `<session>/session.jsonl[.zstd]`; v3 and v4 write
|
||||
`session.v3.jsonl.zstd` / `session.v4.jsonl.zstd`. Discovery used to match only
|
||||
the two unversioned names, so every v3/v4 session was invisible to the import
|
||||
job — `discovered 0` while the files sat on disk. These tests pin the file-name
|
||||
contract, the header gate that keeps an unknown future version out of the
|
||||
database, and the upgrade path where a library imported before the fix picks up
|
||||
the versioned files on its next scan.
|
||||
"""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import zstandard
|
||||
|
||||
from one_memory.imports.adapters import ScanFailure, ScanIssueCode
|
||||
from one_memory.imports.sources import deepseek_harness
|
||||
from one_memory.imports.sources.deepseek_harness import (
|
||||
DEEPSEEK_HARNESS_DESCRIPTOR,
|
||||
create_deepseek_harness_adapter,
|
||||
)
|
||||
from tests.test_secondary_source_adapters import (
|
||||
_cursors,
|
||||
_evidence,
|
||||
_evidence_text,
|
||||
_grant,
|
||||
_scan,
|
||||
_skip_codes,
|
||||
)
|
||||
|
||||
|
||||
def _write_session(directory: Path, name: str, records: list[dict[str, object]]) -> None:
|
||||
directory.mkdir(parents=True, exist_ok=True)
|
||||
payload = "".join(f"{json.dumps(record)}\n" for record in records).encode()
|
||||
(directory / name).write_bytes(zstandard.ZstdCompressor().compress(payload))
|
||||
|
||||
|
||||
def _conversation(version: int | None, *, directory: str = "run-a") -> list[dict[str, object]]:
|
||||
header: dict[str, object] = {
|
||||
"type": "session",
|
||||
"version": version,
|
||||
"id": directory,
|
||||
"cwd": "/private/versioned-project",
|
||||
}
|
||||
return [
|
||||
header,
|
||||
{
|
||||
"type": "user/message",
|
||||
"seq": 1,
|
||||
"data": {
|
||||
"id": "human",
|
||||
"role": "user",
|
||||
"source": {"kind": "user"},
|
||||
"content": [{"type": "text", "text": "visible human"}],
|
||||
},
|
||||
},
|
||||
{
|
||||
"type": "assistant/message",
|
||||
"seq": 2,
|
||||
"data": {
|
||||
"message": {
|
||||
"id": "assistant",
|
||||
"role": "assistant",
|
||||
"content": [{"type": "text", "text": "visible assistant"}],
|
||||
}
|
||||
},
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def _session_adapter(tmp_path: Path):
|
||||
root = tmp_path / "dsh"
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
return root, create_deepseek_harness_adapter(
|
||||
_grant(DEEPSEEK_HARNESS_DESCRIPTOR.source_id, root, tmp_path)
|
||||
)
|
||||
|
||||
|
||||
def test_versioned_v4_session_file_is_discovered_and_imported(tmp_path: Path) -> None:
|
||||
root, adapter = _session_adapter(tmp_path)
|
||||
session = root / "run-v4"
|
||||
records = _conversation(4, directory="run-v4")
|
||||
records.append(
|
||||
{
|
||||
"type": "tool/result",
|
||||
"seq": 3,
|
||||
"time": 1793341068763,
|
||||
"data": {
|
||||
"callId": "call-1",
|
||||
"message": {"role": "tool", "content": [{"type": "text", "text": "tool output"}]},
|
||||
},
|
||||
}
|
||||
)
|
||||
_write_session(session, "session.v4.jsonl.zstd", records)
|
||||
# A rotated backup next to the live file must not be imported as a second copy.
|
||||
(session / "session.v4.jsonl.zstd.orig-20261027").write_bytes(
|
||||
zstandard.ZstdCompressor().compress(b'{"type":"session","version":4}\n')
|
||||
)
|
||||
|
||||
events = _scan(adapter)
|
||||
|
||||
assert _evidence_text(events) == ["visible human", "visible assistant", "tool output"]
|
||||
evidence = _evidence(events)
|
||||
assert {event.evidence.project for event in evidence} == {"versioned-project"}
|
||||
assert {event.evidence.conversation_id for event in evidence} == {"run-v4"}
|
||||
|
||||
|
||||
def test_versioned_v3_session_file_is_discovered_and_imported(tmp_path: Path) -> None:
|
||||
root, adapter = _session_adapter(tmp_path)
|
||||
_write_session(root / "run-v3", "session.v3.jsonl.zstd", _conversation(3))
|
||||
|
||||
events = _scan(adapter)
|
||||
|
||||
assert _evidence_text(events) == ["visible human", "visible assistant"]
|
||||
|
||||
|
||||
def _tool_result(content: list[dict[str, object]]) -> dict[str, object]:
|
||||
return {
|
||||
"type": "tool/result",
|
||||
"seq": 3,
|
||||
"data": {"callId": "call-1", "message": {"role": "tool", "content": content}},
|
||||
}
|
||||
|
||||
|
||||
def test_tool_output_imports_in_either_block_shape(tmp_path: Path) -> None:
|
||||
"""`tool/result` output written as `text` blocks used to be dropped.
|
||||
|
||||
Only `tool-result` blocks were read, so a session that writes plain `text`
|
||||
blocks lost every tool output. Both shapes must arrive, in any generation.
|
||||
"""
|
||||
root, adapter = _session_adapter(tmp_path)
|
||||
v3 = _conversation(3, directory="run-v3-text")
|
||||
v3.append(_tool_result([{"type": "text", "text": "v3 text output"}]))
|
||||
_write_session(root / "run-v3-text", "session.v3.jsonl.zstd", v3)
|
||||
v0 = _conversation(0, directory="run-v0-block")
|
||||
v0.append(_tool_result([{"type": "tool-result", "content": "v0 block output"}]))
|
||||
_write_session(root / "run-v0-block", "session.jsonl.zstd", v0)
|
||||
|
||||
texts = _evidence_text(_scan(adapter))
|
||||
|
||||
assert "v3 text output" in texts
|
||||
assert "v0 block output" in texts
|
||||
|
||||
|
||||
def test_text_tool_output_is_scrubbed_like_block_output(tmp_path: Path) -> None:
|
||||
root, adapter = _session_adapter(tmp_path)
|
||||
secret = "sk-" + "a" * 40
|
||||
records = _conversation(4, directory="run-secret")
|
||||
records.append(_tool_result([{"type": "text", "text": f"token {secret} " + "x" * 5000}]))
|
||||
_write_session(root / "run-secret", "session.v4.jsonl.zstd", records)
|
||||
|
||||
tool = _evidence_text(_scan(adapter))[-1]
|
||||
|
||||
assert secret not in tool
|
||||
assert len(tool) < 2000
|
||||
|
||||
|
||||
def test_session_file_names_outside_the_contract_are_not_imported(tmp_path: Path) -> None:
|
||||
root, adapter = _session_adapter(tmp_path)
|
||||
for name in (
|
||||
"session.v4.jsonl.zstd.orig-20261027",
|
||||
"session.vx.jsonl.zstd",
|
||||
"other.v4.jsonl.zstd",
|
||||
"session.v4.json",
|
||||
):
|
||||
_write_session(root / name.replace(".", "-"), name, _conversation(4))
|
||||
|
||||
assert _evidence(_scan(adapter)) == []
|
||||
|
||||
|
||||
def test_unknown_future_session_version_imports_nothing(tmp_path: Path) -> None:
|
||||
root, adapter = _session_adapter(tmp_path)
|
||||
records = _conversation(5)
|
||||
records.append(
|
||||
{
|
||||
"type": "user/message",
|
||||
"seq": 9,
|
||||
"data": {
|
||||
"id": "future",
|
||||
"role": "user",
|
||||
"source": {"kind": "user"},
|
||||
"content": [{"type": "text", "text": "future generation payload"}],
|
||||
},
|
||||
}
|
||||
)
|
||||
_write_session(root / "run-v5", "session.v5.jsonl.zstd", records)
|
||||
|
||||
events = _scan(adapter)
|
||||
|
||||
assert _evidence(events) == []
|
||||
assert ScanIssueCode.UNSUPPORTED_SCHEMA in _skip_codes(events)
|
||||
|
||||
|
||||
def test_unknown_session_version_is_reported_and_read_once_supported(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""Not importing a future format yet must stay "yet".
|
||||
|
||||
A cursor left behind by the refused scan would put the whole session behind
|
||||
the checkpoint, so the release that learns the format would read nothing.
|
||||
"""
|
||||
root, adapter = _session_adapter(tmp_path)
|
||||
_write_session(root / "run-v5", "session.v5.jsonl.zstd", _conversation(5))
|
||||
|
||||
refused = _scan(adapter)
|
||||
|
||||
assert _cursors(refused) == {}
|
||||
assert [event.reason_code for event in refused if isinstance(event, ScanFailure)] == [
|
||||
ScanIssueCode.UNSUPPORTED_SCHEMA
|
||||
]
|
||||
|
||||
monkeypatch.setattr(deepseek_harness, "SUPPORTED_SESSION_VERSIONS", frozenset({0, 3, 4, 5}))
|
||||
_, upgraded = _session_adapter(tmp_path)
|
||||
|
||||
assert _evidence_text(_scan(upgraded, cursors=_cursors(refused))) == [
|
||||
"visible human",
|
||||
"visible assistant",
|
||||
]
|
||||
|
||||
|
||||
def test_upgrade_scan_rediscovers_versioned_files_already_on_disk(tmp_path: Path) -> None:
|
||||
"""A library imported before this fix must pick up the files on its next scan.
|
||||
|
||||
The first scan sees only the unversioned session that the old predicate
|
||||
matched; after the upgrade, the versioned file next to it is discovered by
|
||||
the next scan while the already imported session stays deduplicated by the
|
||||
cursor it left behind.
|
||||
"""
|
||||
root, adapter = _session_adapter(tmp_path)
|
||||
old = root / "run-old"
|
||||
_write_session(old, "session.jsonl.zstd", _conversation(0, directory="run-old"))
|
||||
first_scan = _scan(adapter)
|
||||
assert _evidence_text(first_scan) == ["visible human", "visible assistant"]
|
||||
assert _evidence(first_scan)[0].evidence.conversation_id == "run-old"
|
||||
|
||||
versioned = root / "run-v4"
|
||||
_write_session(versioned, "session.v4.jsonl.zstd", _conversation(4, directory="run-v4"))
|
||||
cursors = _cursors(first_scan)
|
||||
assert cursors # the old session left a cursor behind
|
||||
|
||||
upgrade_scan = _scan(adapter, cursors=cursors)
|
||||
|
||||
assert _evidence_text(upgrade_scan) == ["visible human", "visible assistant"]
|
||||
assert {event.evidence.conversation_id for event in _evidence(upgrade_scan)} == {"run-v4"}
|
||||
@@ -193,7 +193,7 @@ def _qwenwork() -> Harness:
|
||||
|
||||
def _deepseek() -> Harness:
|
||||
def write(path: Path, turns: list[Turn]) -> None:
|
||||
# deepseek only accepts a file literally named session.jsonl, and nests
|
||||
# deepseek only accepts files named session[.vN].jsonl[.zstd], and nests
|
||||
# the turn under "data".
|
||||
lines: list[dict] = [{"type": "session", "version": 0, "id": "s1", "cwd": "/w"}]
|
||||
for index, (role, text, _session) in enumerate(turns):
|
||||
@@ -225,7 +225,7 @@ def _deepseek() -> Harness:
|
||||
|
||||
from one_memory.imports.sources.deepseek_harness import create_deepseek_harness_adapter
|
||||
|
||||
# deepseek accepts only a file literally named session.jsonl.
|
||||
# deepseek accepts only session[.vN].jsonl[.zstd]; the first format is used here.
|
||||
return Harness(
|
||||
"deepseek-harness",
|
||||
write,
|
||||
|
||||
@@ -336,6 +336,79 @@ def test_main_database_warns_when_wal_stays_above_the_high_water_mark(
|
||||
assert any("main_database_wal_high" in record.message for record in caplog.records)
|
||||
|
||||
|
||||
def test_a_wal_whose_frames_are_all_checkpointed_is_not_reported_high(
|
||||
tmp_path, monkeypatch, caplog
|
||||
):
|
||||
"""#245: the alert judges frames still waiting, not the file's size."""
|
||||
path = _database(tmp_path)
|
||||
database = main_database(path)
|
||||
monkeypatch.setattr(_access, "MAIN_DATABASE_CHECKPOINT_WRITE_INTERVAL", 1)
|
||||
# One page: any frame left uncopied would cross it, a fully copied WAL not.
|
||||
monkeypatch.setattr(_access, "MAIN_DATABASE_WAL_WARN_BYTES", 1)
|
||||
caplog.set_level("WARNING", logger="one_memory.database_access")
|
||||
|
||||
with database.write() as db, db:
|
||||
db.execute("UPDATE counter SET value = 1 WHERE id = 1")
|
||||
|
||||
assert database._wal_bytes() > 0
|
||||
assert not any("main_database_wal_high" in record.message for record in caplog.records)
|
||||
|
||||
|
||||
def test_the_writer_caps_the_wal_it_leaves_behind(tmp_path):
|
||||
path = _database(tmp_path)
|
||||
with main_database(path).write() as db:
|
||||
limit = db.execute("PRAGMA journal_size_limit").fetchone()[0]
|
||||
assert limit == _access.MAIN_DATABASE_JOURNAL_SIZE_LIMIT_BYTES
|
||||
|
||||
|
||||
def test_an_idle_checkpoint_shrinks_the_wal_after_a_large_burst(tmp_path, monkeypatch):
|
||||
"""#245: a WAL that grew once used to stay that size for good."""
|
||||
path = _database(tmp_path)
|
||||
database = main_database(path)
|
||||
# No cadence checkpoint during the burst: the WAL has to grow.
|
||||
monkeypatch.setattr(_access, "MAIN_DATABASE_CHECKPOINT_WRITE_INTERVAL", 10_000_000)
|
||||
monkeypatch.setattr(_access, "MAIN_DATABASE_CHECKPOINT_SECONDS", 10_000_000.0)
|
||||
database._last_checkpoint_at = time.monotonic()
|
||||
with database.write() as db, db:
|
||||
db.execute("PRAGMA wal_autocheckpoint = 0")
|
||||
db.execute("CREATE TABLE blobs (body BLOB)")
|
||||
db.executemany(
|
||||
"INSERT INTO blobs VALUES (randomblob(?))", [(64 * 1024,) for _ in range(1_500)]
|
||||
)
|
||||
with database.write() as db:
|
||||
db.execute("PRAGMA wal_autocheckpoint = 1000")
|
||||
grown = database._wal_bytes()
|
||||
assert grown > _access.MAIN_DATABASE_JOURNAL_SIZE_LIMIT_BYTES
|
||||
|
||||
with database.read() as reader:
|
||||
reader.execute("SELECT COUNT(*) FROM blobs").fetchone()
|
||||
# Someone is reading: no truncate is even attempted.
|
||||
assert database.checkpoint_if_idle() is None
|
||||
assert _access.checkpoint_idle_main_databases() == 1
|
||||
|
||||
assert database._wal_bytes() <= _access.MAIN_DATABASE_JOURNAL_SIZE_LIMIT_BYTES
|
||||
with database.read() as reader:
|
||||
assert reader.execute("SELECT COUNT(*) FROM blobs").fetchone() == (1_500,)
|
||||
|
||||
|
||||
def test_a_busy_idle_checkpoint_is_skipped_not_raised(tmp_path):
|
||||
path = _database(tmp_path)
|
||||
database = main_database(path)
|
||||
with database.write() as db, db:
|
||||
db.execute("UPDATE counter SET value = 1 WHERE id = 1")
|
||||
# Another process's reader holds a snapshot: TRUNCATE cannot finish.
|
||||
outsider = sqlite3.connect(path)
|
||||
try:
|
||||
outsider.execute("BEGIN")
|
||||
outsider.execute("SELECT value FROM counter").fetchone()
|
||||
result = database.checkpoint_if_idle()
|
||||
assert result is None or result[0] == 1
|
||||
finally:
|
||||
outsider.close()
|
||||
assert database.checkpoint_if_idle() == (0, 0, 0)
|
||||
assert database._wal_bytes() == 0
|
||||
|
||||
|
||||
def test_main_database_maintenance_checkpoint_does_not_break_an_active_reader(
|
||||
tmp_path, monkeypatch
|
||||
):
|
||||
|
||||
@@ -293,6 +293,7 @@ def test_sdk_user_profile_http_returns_native_assembled_profile(tmp_path):
|
||||
response = await client.get("/api/v2/user-profile")
|
||||
assert response.status_code == 200, response.text
|
||||
assert response.json() == {
|
||||
"user_id": "local",
|
||||
"request_id": "profile-request",
|
||||
"profile": {
|
||||
"memory_id": "l0-head",
|
||||
@@ -660,6 +661,8 @@ def test_memory_add_routes_do_not_collide_with_single_memory_routes(tmp_path):
|
||||
"task_id",
|
||||
"memory_ids",
|
||||
"error",
|
||||
"error_code",
|
||||
"next_retry_at",
|
||||
}
|
||||
status = await client.get(f"/api/v2/memories/additions/{request_id}")
|
||||
assert status.json()["evidence_id"] == created.json()["evidence_id"]
|
||||
|
||||
@@ -28,6 +28,8 @@ from one_memory.memory_features import (
|
||||
FEATURE_NAMES,
|
||||
FEATURES,
|
||||
FEATURES_BY_NAME,
|
||||
PARAMETERS,
|
||||
SETTING_NAMES,
|
||||
MemoryFeatures,
|
||||
effective_features,
|
||||
feature_activity,
|
||||
@@ -46,15 +48,20 @@ TODAY = {
|
||||
"session_episodes": False,
|
||||
"episode_summary": False,
|
||||
"intention": True,
|
||||
"intention_dedup": True,
|
||||
"few_shot": False,
|
||||
}
|
||||
#: The same for the numeric settings.
|
||||
TODAY_PARAMETERS = {"intention_dedup_threshold": 0.95}
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean_sdk_env(monkeypatch):
|
||||
"""The SDK's own env vars feed its defaults; tests start without them."""
|
||||
for spec in FEATURES:
|
||||
for spec in (*FEATURES, *PARAMETERS):
|
||||
monkeypatch.delenv(spec.env, raising=False)
|
||||
# The threshold falls back to the SDK's shared dedup variable.
|
||||
monkeypatch.delenv("MEMORY_DEDUP_THRESHOLD", raising=False)
|
||||
monkeypatch.delenv("ONE_MEMORY_GRAPH_STORE", raising=False)
|
||||
|
||||
|
||||
@@ -95,17 +102,25 @@ def _field(config, name: str) -> bool:
|
||||
|
||||
|
||||
def test_the_document_names_exactly_the_catalog() -> None:
|
||||
assert tuple(MemoryFeatures.model_fields) == FEATURE_NAMES
|
||||
assert tuple(MemoryFeatures.model_fields) == SETTING_NAMES
|
||||
assert set(TODAY) == set(FEATURE_NAMES)
|
||||
# Each feature is a MemoryConfig field the SDK has; no env var is written.
|
||||
defaults = hy_sdk_runtime.sdk_feature_defaults()
|
||||
assert set(defaults) == set(FEATURE_NAMES)
|
||||
assert set(defaults) == set(SETTING_NAMES)
|
||||
|
||||
|
||||
def test_the_defaults_are_todays_behaviour() -> None:
|
||||
assert hy_sdk_runtime.sdk_feature_defaults() == TODAY
|
||||
everything = {**TODAY, **TODAY_PARAMETERS}
|
||||
assert hy_sdk_runtime.sdk_feature_defaults() == everything
|
||||
assert MemoryFeatures().overrides() == ()
|
||||
assert effective_features((), TODAY) == TODAY
|
||||
assert effective_features((), everything) == everything
|
||||
|
||||
|
||||
@pytest.mark.parametrize("profile", PROFILES, ids=("builtin", "remote"))
|
||||
def test_intention_dedup_defaults_to_on_at_095_in_the_built_config(tmp_path, profile) -> None:
|
||||
config = _config(tmp_path, profile)
|
||||
assert config.recall.intention_dedup_enabled is True
|
||||
assert config.recall.intention_dedup_threshold == 0.95
|
||||
|
||||
|
||||
def test_the_sdk_env_var_still_moves_the_default_of_an_unset_feature(monkeypatch) -> None:
|
||||
@@ -126,6 +141,32 @@ def test_unknown_or_non_boolean_values_are_refused() -> None:
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bad", (0.79, 1.01, -1, 2, True, "0.9", float("nan"), float("inf")))
|
||||
def test_an_intention_dedup_threshold_outside_its_range_is_refused(bad) -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
MemoryFeatures.model_validate({"intention_dedup_threshold": bad})
|
||||
|
||||
|
||||
def test_the_intention_dedup_threshold_range_is_inclusive() -> None:
|
||||
for value in (0.8, 0.9, 1, 1.0):
|
||||
document = MemoryFeatures.model_validate({"intention_dedup_threshold": value})
|
||||
assert document.overrides() == (("intention_dedup_threshold", float(value)),)
|
||||
with pytest.raises(ValueError, match="invalid_memory_feature_value"):
|
||||
hy_sdk_runtime.build_sdk_config(
|
||||
PROFILES[0],
|
||||
api_key=None,
|
||||
data_dir=Path("/tmp/unused"),
|
||||
features={"intention_dedup_threshold": 0.5},
|
||||
)
|
||||
|
||||
|
||||
def test_the_sdk_env_vars_still_move_the_threshold_default(monkeypatch) -> None:
|
||||
monkeypatch.setenv("MEMORY_DEDUP_THRESHOLD", "0.9")
|
||||
assert hy_sdk_runtime.sdk_feature_defaults()["intention_dedup_threshold"] == 0.9
|
||||
monkeypatch.setenv("MEMORY_INTENTION_DEDUP_THRESHOLD", "0.97")
|
||||
assert hy_sdk_runtime.sdk_feature_defaults()["intention_dedup_threshold"] == 0.97
|
||||
|
||||
|
||||
def test_merge_sets_resets_and_leaves_absent_keys_alone() -> None:
|
||||
saved = MemoryFeatures(graph_rerank=True, few_shot=True)
|
||||
update = MemoryFeatures.model_validate({"few_shot": None, "episode_recall": True})
|
||||
@@ -165,6 +206,20 @@ def test_each_feature_reaches_the_built_memory_config(tmp_path, profile) -> None
|
||||
assert _field(config, other.name) is TODAY[other.name], (spec.name, other.name)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("profile", PROFILES, ids=("builtin", "remote"))
|
||||
def test_intention_dedup_on_off_and_threshold_reach_the_built_memory_config(
|
||||
tmp_path, profile
|
||||
) -> None:
|
||||
off = _config(tmp_path, profile, {"intention_dedup": False})
|
||||
assert off.recall.intention_dedup_enabled is False
|
||||
assert off.recall.intention_dedup_threshold == 0.95
|
||||
on = _config(tmp_path, profile, {"intention_dedup": True, "intention_dedup_threshold": 0.9})
|
||||
assert on.recall.intention_dedup_enabled is True
|
||||
assert on.recall.intention_dedup_threshold == 0.9
|
||||
# Nothing else moves.
|
||||
assert _field(on, "intention") is True and _field(on, "graph_rerank") is False
|
||||
|
||||
|
||||
@pytest.mark.parametrize("profile", PROFILES, ids=("builtin", "remote"))
|
||||
def test_nothing_set_is_the_config_of_before_byte_for_byte(tmp_path, profile) -> None:
|
||||
before = repr(dataclasses.asdict(_config(tmp_path, profile)))
|
||||
@@ -172,6 +227,8 @@ def test_nothing_set_is_the_config_of_before_byte_for_byte(tmp_path, profile) ->
|
||||
assert repr(dataclasses.asdict(_config(tmp_path, profile, {}))) == before
|
||||
# Setting every feature to its default value changes nothing either.
|
||||
assert repr(dataclasses.asdict(_config(tmp_path, profile, TODAY))) == before
|
||||
everything = {**TODAY, **TODAY_PARAMETERS}
|
||||
assert repr(dataclasses.asdict(_config(tmp_path, profile, everything))) == before
|
||||
|
||||
|
||||
@pytest.mark.parametrize("profile", PROFILES, ids=("builtin", "remote"))
|
||||
@@ -190,6 +247,7 @@ def test_features_never_change_the_collection_identity(tmp_path, profile) -> Non
|
||||
{name: True for name in FEATURE_NAMES},
|
||||
{name: False for name in FEATURE_NAMES},
|
||||
{"graph_rerank": True, "episode_recall": True},
|
||||
{"intention_dedup": False, "intention_dedup_threshold": 0.8},
|
||||
):
|
||||
assert identity(_config(tmp_path, profile, values)) == baseline
|
||||
|
||||
@@ -243,6 +301,13 @@ def test_a_feature_needing_a_higher_mode_is_reported_inactive() -> None:
|
||||
series = effective_features((("same_series_timeline", True), ("graph_rerank", True)), TODAY)
|
||||
assert feature_activity("same_series_timeline", series, "pro") == "requires_ultra"
|
||||
assert feature_activity("same_series_timeline", series, "ultra") is None
|
||||
defaults = {**TODAY, **TODAY_PARAMETERS}
|
||||
assert feature_activity("intention_dedup", defaults, "pro") is None
|
||||
assert feature_activity("intention_dedup_threshold", defaults, "pro") is None
|
||||
assert feature_activity("intention_dedup_threshold", defaults, "lite") == "requires_pro"
|
||||
off = effective_features((("intention_dedup", False),), defaults)
|
||||
assert feature_activity("intention_dedup", off, "pro") == "disabled"
|
||||
assert feature_activity("intention_dedup_threshold", off, "pro") == "requires_intention_dedup"
|
||||
|
||||
|
||||
# --- rebuilding handles -------------------------------------------------------
|
||||
@@ -497,3 +562,99 @@ def test_features_api_get_put_reset_and_status(tmp_path, monkeypatch) -> None:
|
||||
assert (await client.get("/api/v1/settings/memory-features")).json()["non_default"] == {}
|
||||
|
||||
run_app(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_intention_dedup_api_on_off_threshold_and_422(tmp_path, monkeypatch) -> None:
|
||||
built: list[tuple[bool, float]] = []
|
||||
original = hy_sdk_runtime.build_sdk_config
|
||||
|
||||
def recording(*args, **kwargs):
|
||||
config = original(*args, **kwargs)
|
||||
built.append(
|
||||
(config.recall.intention_dedup_enabled, config.recall.intention_dedup_threshold)
|
||||
)
|
||||
return config
|
||||
|
||||
monkeypatch.setattr(hy_sdk_runtime, "build_sdk_config", recording)
|
||||
|
||||
def built_now(app) -> tuple[bool, float]:
|
||||
"""What a client built with the read handle's current key carries."""
|
||||
recording(
|
||||
hy_sdk_runtime.builtin_retrieval_snapshot(),
|
||||
api_key=None,
|
||||
data_dir=tmp_path,
|
||||
features=app.state.sdk_client_handle.current_features(),
|
||||
)
|
||||
return built[-1]
|
||||
|
||||
async def scenario(app, client):
|
||||
path = "/api/v1/settings/memory-features"
|
||||
current = (await client.get(path)).json()
|
||||
dedup = _by_name(current)["intention_dedup"]
|
||||
assert dedup["enabled"] is True and dedup["default"] is True
|
||||
assert dedup["configured"] is None and dedup["active"] is True
|
||||
assert dedup["sdk_field"] == "recall.intention_dedup_enabled"
|
||||
assert dedup["env"] == "MEMORY_INTENTION_DEDUP_ENABLED"
|
||||
(threshold,) = current["parameters"]
|
||||
assert threshold == {
|
||||
"name": "intention_dedup_threshold",
|
||||
"value": 0.95,
|
||||
"configured": None,
|
||||
"default": 0.95,
|
||||
"minimum": 0.8,
|
||||
"maximum": 1.0,
|
||||
"active": True,
|
||||
"reason": None,
|
||||
"requires_feature": "intention_dedup",
|
||||
"sdk_field": "recall.intention_dedup_threshold",
|
||||
"env": "MEMORY_INTENTION_DEDUP_THRESHOLD",
|
||||
"description": threshold["description"],
|
||||
}
|
||||
assert built_now(app) == (True, 0.95)
|
||||
|
||||
off = await client.put(path, json={"intention_dedup": False})
|
||||
assert off.status_code == 200, off.text
|
||||
body = off.json()
|
||||
assert body["changed"] == ["intention_dedup"]
|
||||
assert body["non_default"] == {"intention_dedup": False}
|
||||
assert body["parameters"][0]["active"] is False
|
||||
assert body["parameters"][0]["reason"] == "requires_intention_dedup"
|
||||
assert app.state.sdk_write_coordinator.memory_features() == (("intention_dedup", False),)
|
||||
assert built_now(app) == (False, 0.95)
|
||||
|
||||
on = (
|
||||
await client.put(path, json={"intention_dedup": True, "intention_dedup_threshold": 0.9})
|
||||
).json()
|
||||
assert on["changed"] == ["intention_dedup", "intention_dedup_threshold"]
|
||||
assert on["non_default"] == {"intention_dedup_threshold": 0.9}
|
||||
assert on["parameters"][0]["value"] == 0.9 and on["parameters"][0]["active"] is True
|
||||
expected = (("intention_dedup", True), ("intention_dedup_threshold", 0.9))
|
||||
assert app.state.sdk_client_handle.current_features() == expected
|
||||
assert built_now(app) == (True, 0.9)
|
||||
processing = (await client.get("/api/v1/memory-processing")).json()
|
||||
assert processing["pipeline"]["features"] == {"intention_dedup_threshold": 0.9}
|
||||
saved = json.loads((tmp_path / "data" / "config.json").read_text())
|
||||
assert saved["memory_features"] == {
|
||||
"intention_dedup": True,
|
||||
"intention_dedup_threshold": 0.9,
|
||||
}
|
||||
|
||||
for bad in (0.5, 0.79, 1.5, -0.1, "0.9", True):
|
||||
response = await client.put(path, json={"intention_dedup_threshold": bad})
|
||||
assert response.status_code == 422, bad
|
||||
assert (await client.put(path, json={"intention_dedup": "false"})).status_code == 422
|
||||
# A refused body changes nothing.
|
||||
assert app.state.sdk_client_handle.current_features() == expected
|
||||
|
||||
reset = (
|
||||
await client.put(
|
||||
path, json={"intention_dedup": None, "intention_dedup_threshold": None}
|
||||
)
|
||||
).json()
|
||||
assert reset["changed"] == ["intention_dedup_threshold"]
|
||||
assert reset["non_default"] == {}
|
||||
assert app.state.sdk_client_handle.current_features() == ()
|
||||
assert built_now(app) == (True, 0.95)
|
||||
|
||||
run_app(tmp_path, monkeypatch, scenario)
|
||||
@@ -1433,25 +1433,6 @@ def test_a_claimed_row_with_no_readable_payload_is_settled_not_left_running(tmp_
|
||||
assert _rows(path)[0]["lease_owner"] is None
|
||||
|
||||
|
||||
def test_an_unknown_outcome_is_recorded_as_unknown_not_retried(tmp_path) -> None:
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
record = queue.submit(_request())
|
||||
supervisor = _Recorder(error=RuntimeError("model_content_outcome_unknown"))
|
||||
_claim_through_drain(queue, record.job_id)
|
||||
|
||||
asyncio.run(
|
||||
execute_job(
|
||||
record.job_id,
|
||||
queue=queue,
|
||||
supervisor=supervisor,
|
||||
evidence=_EvidenceStore({"e1"}),
|
||||
)
|
||||
)
|
||||
|
||||
assert _state(path, record.job_id) == "unknown"
|
||||
|
||||
|
||||
def test_unpermitted_evidence_parks_and_is_readmitted_after_confirmation(tmp_path) -> None:
|
||||
"""A note claimed before its confirmation must not become terminal failed."""
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
@@ -1478,34 +1459,6 @@ def test_unpermitted_evidence_parks_and_is_readmitted_after_confirmation(tmp_pat
|
||||
assert replay.state == PENDING
|
||||
|
||||
|
||||
def test_a_permission_error_after_hy_invocation_stays_unknown(tmp_path) -> None:
|
||||
"""An error code cannot erase the durable possibly-sent boundary."""
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
record = queue.submit(_request())
|
||||
_claim_through_drain(queue, record.job_id)
|
||||
|
||||
asyncio.run(
|
||||
execute_job(
|
||||
record.job_id,
|
||||
queue=queue,
|
||||
supervisor=_Recorder(
|
||||
error=RuntimeError("model_content_evidence_changed"),
|
||||
invoke_hy=True,
|
||||
),
|
||||
evidence=_EvidenceStore({"e1"}),
|
||||
)
|
||||
)
|
||||
|
||||
assert queue.get(record.request_id).state == UNKNOWN
|
||||
with sqlite3.connect(path) as connection:
|
||||
assert connection.execute(
|
||||
"SELECT hy_invoked,last_error FROM memory_write_jobs WHERE job_id=?",
|
||||
(record.job_id,),
|
||||
).fetchone() == (1, "model_content_evidence_changed")
|
||||
assert queue.reclaim_expired() == 0
|
||||
|
||||
|
||||
def test_replayed_live_evidence_readmits_a_pre_send_identity_failure(tmp_path) -> None:
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
@@ -1623,90 +1576,6 @@ def test_startup_never_repairs_a_possibly_sent_identity_failure(tmp_path) -> Non
|
||||
assert queue.get(job.request_id).state == UNKNOWN
|
||||
|
||||
|
||||
def test_reconciled_absent_timeout_is_requeued_once_without_resetting_attempts(
|
||||
tmp_path,
|
||||
) -> None:
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
record = queue.submit(_request())
|
||||
with sqlite3.connect(path) as connection, connection:
|
||||
connection.execute(
|
||||
"UPDATE memory_write_jobs SET state='unknown', attempts=1, "
|
||||
"hy_invoked=1, last_error='hy_memory_timeout', updated_at=? WHERE job_id=?",
|
||||
("2026-01-01T00:00:00+00:00", record.job_id),
|
||||
)
|
||||
|
||||
assert (
|
||||
queue.retry_timed_out_unknown_after_reconcile(
|
||||
older_than=datetime(2026, 1, 1, 0, 5, tzinfo=UTC)
|
||||
)
|
||||
== 1
|
||||
)
|
||||
with sqlite3.connect(path) as connection:
|
||||
state, attempts, invoked, error = connection.execute(
|
||||
"SELECT state, attempts, hy_invoked, last_error FROM memory_write_jobs WHERE job_id=?",
|
||||
(record.job_id,),
|
||||
).fetchone()
|
||||
assert (state, attempts, invoked, error) == (
|
||||
"pending",
|
||||
1,
|
||||
0,
|
||||
"model_content_reconciled_absent_retry",
|
||||
)
|
||||
|
||||
|
||||
def test_unresolved_unknown_is_automatically_retried_only_once(tmp_path) -> None:
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
record = queue.submit(_request())
|
||||
with sqlite3.connect(path) as connection, connection:
|
||||
connection.execute(
|
||||
"UPDATE memory_write_jobs SET state='unknown',attempts=1,hy_invoked=1,"
|
||||
"last_error='model_content_outcome_unknown',updated_at=? WHERE job_id=?",
|
||||
("2026-01-01T00:00:00+00:00", record.job_id),
|
||||
)
|
||||
|
||||
cutoff = datetime(2026, 1, 1, 0, 5, tzinfo=UTC)
|
||||
assert queue.retry_first_unknown_after_reconcile(older_than=cutoff) == 1
|
||||
first_retry = queue.get(record.request_id)
|
||||
assert first_retry.state == PENDING
|
||||
with sqlite3.connect(path) as connection:
|
||||
assert (
|
||||
connection.execute(
|
||||
"SELECT attempts FROM memory_write_jobs WHERE job_id=?",
|
||||
(record.job_id,),
|
||||
).fetchone()[0]
|
||||
== 1
|
||||
)
|
||||
|
||||
drain = DurableWriteDrain(queue, execute=_never_execute, owner="retry-owner")
|
||||
assert drain.claim(limit=1) == (record.job_id,)
|
||||
assert queue.mark_invoked(record.job_id, owner="retry-owner")
|
||||
queue.settle(
|
||||
record.job_id,
|
||||
state=UNKNOWN,
|
||||
owner="retry-owner",
|
||||
error_code="model_content_outcome_unknown",
|
||||
hy_invoked=True,
|
||||
)
|
||||
|
||||
with sqlite3.connect(path) as connection:
|
||||
assert (
|
||||
connection.execute(
|
||||
"SELECT attempts FROM memory_write_jobs WHERE job_id=?",
|
||||
(record.job_id,),
|
||||
).fetchone()[0]
|
||||
== 2
|
||||
)
|
||||
assert (
|
||||
queue.retry_first_unknown_after_reconcile(
|
||||
older_than=datetime.now(UTC) + timedelta(minutes=1)
|
||||
)
|
||||
== 0
|
||||
)
|
||||
assert queue.get(record.request_id).state == UNKNOWN
|
||||
|
||||
|
||||
def test_drain_runs_unknown_reconciliation_during_busy_safe_maintenance(tmp_path) -> None:
|
||||
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
|
||||
calls = 0
|
||||
@@ -1759,26 +1628,6 @@ def test_a_reclaimed_uninvoked_job_with_attempts_left_is_retryable(tmp_path) ->
|
||||
assert _rows(path)[0]["attempts"] == 0
|
||||
|
||||
|
||||
def test_post_invocation_transport_failure_stays_unknown_not_pending(tmp_path) -> None:
|
||||
"""Once the bridge may have been opened, no error is safe to replay."""
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
record = queue.submit(_request())
|
||||
_claim_through_drain(queue, record.job_id)
|
||||
|
||||
asyncio.run(
|
||||
execute_job(
|
||||
record.job_id,
|
||||
queue=queue,
|
||||
supervisor=_Recorder(error=RuntimeError("hy_memory_timeout")),
|
||||
evidence=_EvidenceStore({"e1"}),
|
||||
)
|
||||
)
|
||||
|
||||
assert _state(path, record.job_id) == UNKNOWN
|
||||
assert _rows(path)[0]["hy_invoked"] == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"code,expected",
|
||||
[
|
||||
@@ -1900,7 +1749,9 @@ def test_a_failed_job_logs_where_it_failed_but_never_the_message(tmp_path, caplo
|
||||
for message in caplog.records
|
||||
if "durable_write_job_failed" in message.getMessage()
|
||||
]
|
||||
assert f"job_id={record.job_id} error_code=RuntimeError chain=RuntimeError@" in line
|
||||
assert (
|
||||
f"job_id={record.job_id} error_code=RuntimeError state=pending chain=RuntimeError@" in line
|
||||
)
|
||||
assert ";AttributeError@" in line
|
||||
assert "test_memory_server_durable_queue.py:" in line and ":closed_store" in line
|
||||
assert "secret diary" not in line
|
||||
@@ -2620,57 +2471,6 @@ def test_a_wedged_job_hits_the_work_ceiling_and_settles_unknown(tmp_path) -> Non
|
||||
assert (state, owner, expiry) == ("unknown", None, None)
|
||||
|
||||
|
||||
def test_user_confirmed_unknown_retry_requeues_job_and_linked_tasks(tmp_path) -> None:
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
record = queue.submit(_request(evidence=("e1",)))
|
||||
with sqlite3.connect(path) as connection, connection:
|
||||
connection.execute(
|
||||
"UPDATE memory_write_jobs SET state='unknown', attempts=2, hy_invoked=1, "
|
||||
"last_error='hy_memory_timeout' WHERE job_id=?",
|
||||
(record.job_id,),
|
||||
)
|
||||
connection.executescript(
|
||||
"""
|
||||
CREATE TABLE memory_tasks (
|
||||
task_id TEXT PRIMARY KEY,
|
||||
status TEXT NOT NULL,
|
||||
runner_token TEXT,
|
||||
attempts INTEGER NOT NULL,
|
||||
last_error_code TEXT,
|
||||
updated_at TEXT NOT NULL
|
||||
);
|
||||
CREATE TABLE memory_task_inputs (
|
||||
task_id TEXT PRIMARY KEY,
|
||||
refs_json TEXT NOT NULL
|
||||
);
|
||||
"""
|
||||
)
|
||||
connection.execute(
|
||||
"INSERT INTO memory_tasks VALUES (?,?,?,?,?,?)",
|
||||
("task-1", "unknown", None, 2, "model_content_outcome_unknown", "old"),
|
||||
)
|
||||
connection.execute(
|
||||
"INSERT INTO memory_task_inputs VALUES (?,?)",
|
||||
("task-1", '[{"evidence_id":"e1"}]'),
|
||||
)
|
||||
|
||||
assert queue.retry_unknown_confirmed(record.job_id) == 1
|
||||
retried = queue.get(record.request_id)
|
||||
assert retried is not None
|
||||
assert retried.state == PENDING
|
||||
with sqlite3.connect(path) as connection:
|
||||
assert connection.execute(
|
||||
"SELECT attempts, hy_invoked FROM memory_write_jobs WHERE job_id=?",
|
||||
(record.job_id,),
|
||||
).fetchone() == (0, 0)
|
||||
assert connection.execute(
|
||||
"SELECT status, attempts, last_error_code FROM memory_tasks WHERE task_id='task-1'"
|
||||
).fetchone() == ("pending", 0, "model_content_manual_retry_requested")
|
||||
|
||||
assert queue.retry_unknown_confirmed(record.job_id) == 0
|
||||
|
||||
|
||||
def test_restart_redrains_a_job_claimed_by_a_worker_that_died(tmp_path) -> None:
|
||||
"""End-to-end restart recovery: a claimed-but-not-run job runs once.
|
||||
|
||||
@@ -2922,30 +2722,6 @@ def test_forever_drain_admits_a_new_source_while_an_earlier_source_is_running(tm
|
||||
assert queue.get("workbuddy-1").state == COMPLETED
|
||||
|
||||
|
||||
def test_sqlite_drain_preserves_unknown_after_execute_job(tmp_path) -> None:
|
||||
"""A drain failure after invocation must remain unknown, not look retryable."""
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
queue.submit(_request(evidence=("e1",)))
|
||||
|
||||
async def execute(job_id: str) -> None:
|
||||
await execute_job(
|
||||
job_id,
|
||||
queue=queue,
|
||||
supervisor=_Recorder(error=RuntimeError("model_content_outcome_unknown")),
|
||||
evidence=_EvidenceStore({"e1"}),
|
||||
)
|
||||
|
||||
async def scenario() -> None:
|
||||
drain = DurableWriteDrain(queue, execute=execute)
|
||||
assert await drain.run_one() == 1
|
||||
|
||||
asyncio.run(scenario())
|
||||
assert queue.get("idem-1").state == "unknown"
|
||||
assert queue.reclaim_expired() == 0
|
||||
assert queue.get("idem-1").state == "unknown"
|
||||
|
||||
|
||||
def test_sqlite_drain_keeps_running_after_an_executor_raises(tmp_path) -> None:
|
||||
"""One executor exception must not kill the drain or strand other rows.
|
||||
|
||||
@@ -4222,27 +3998,6 @@ def test_set_state_rejects_an_unvalidated_state(tmp_path) -> None:
|
||||
assert registry.get(record.request_id).state == "accepted"
|
||||
|
||||
|
||||
def test_set_state_cannot_resolve_unknown_without_an_inquiry(tmp_path) -> None:
|
||||
"""The HTTP-facing setter must not bypass the inquiry-only exit.
|
||||
|
||||
`DurableWriteQueue.settle` deliberately treats `unknown` as sticky; a
|
||||
direct `set_state("completed"/"failed")` would otherwise turn a
|
||||
possibly-delivered write into a terminal claim without asking HY. The
|
||||
durable registry is the shared caller surface, so it must preserve that
|
||||
boundary even though today only tests call it.
|
||||
"""
|
||||
from one_memory.memory_server.durable_queue import DurableWriteRegistry
|
||||
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
registry = DurableWriteRegistry(path)
|
||||
durable = registry.queue.submit(_request())
|
||||
registry.queue.settle(durable.job_id, state=UNKNOWN, hy_invoked=True)
|
||||
|
||||
assert registry.set_state(durable.request_id, "completed") is None
|
||||
assert registry.set_state(durable.request_id, "failed") is None
|
||||
assert registry.get(durable.request_id).state == UNKNOWN
|
||||
|
||||
|
||||
@pytest.mark.parametrize("state", (WAITING_FOR_PERMISSION, WAITING_FOR_MODEL))
|
||||
def test_set_state_accepts_the_route_parked_states(tmp_path, state: str) -> None:
|
||||
"""The durable setter must accept every non-terminal route state.
|
||||
@@ -5435,83 +5190,6 @@ def test_readmit_paths_never_requeue_a_possibly_sent_row(tmp_path) -> None:
|
||||
assert DurableWriteDrain(queue, execute=_never_execute).claim(limit=5) == ()
|
||||
|
||||
|
||||
def test_a_refused_source_context_is_nonblocking_unknown(
|
||||
tmp_path,
|
||||
) -> None:
|
||||
"""An ambiguous call-boundary failure is retained but never blocks FIFO."""
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
record = queue.submit(_request(key="refused-scope", source="opencode"))
|
||||
claimer = DurableWriteDrain(queue, execute=_never_execute, owner="refused-owner")
|
||||
assert claimer.claim(limit=1) == (record.job_id,)
|
||||
|
||||
asyncio.run(
|
||||
execute_job(
|
||||
record.job_id,
|
||||
queue=queue,
|
||||
supervisor=_Recorder(
|
||||
error=RuntimeError("hy_memory_write_verification_context_scope"),
|
||||
invoke_hy=True,
|
||||
),
|
||||
evidence=_EvidenceStore({"e1"}),
|
||||
owner="refused-owner",
|
||||
)
|
||||
)
|
||||
|
||||
with sqlite3.connect(path.as_uri() + "?mode=ro", uri=True) as db:
|
||||
state, invoked, last_error = db.execute(
|
||||
"SELECT state, hy_invoked, last_error FROM memory_write_jobs WHERE job_id=?",
|
||||
(record.job_id,),
|
||||
).fetchone()
|
||||
|
||||
assert state == UNKNOWN
|
||||
assert invoked == 1
|
||||
assert last_error == "hy_memory_write_verification_context_scope"
|
||||
assert DurableWriteDrain(queue, execute=_never_execute).claim(limit=1) == ()
|
||||
|
||||
|
||||
def test_a_failure_after_the_call_still_parks_as_unknown(tmp_path) -> None:
|
||||
"""The fix must not widen: a post-call refusal is still possibly-sent."""
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
record = queue.submit(_request(key="after-call", source="opencode"))
|
||||
claimer = DurableWriteDrain(queue, execute=_never_execute, owner="after-call-owner")
|
||||
assert claimer.claim(limit=1) == (record.job_id,)
|
||||
recorder = _Recorder(
|
||||
error=RuntimeError("hy_memory_write_verification_authority_invalid"),
|
||||
invoke_hy=True,
|
||||
)
|
||||
|
||||
asyncio.run(
|
||||
execute_job(
|
||||
record.job_id,
|
||||
queue=queue,
|
||||
supervisor=recorder,
|
||||
evidence=_EvidenceStore({"e1"}),
|
||||
owner="after-call-owner",
|
||||
)
|
||||
)
|
||||
|
||||
with sqlite3.connect(path.as_uri() + "?mode=ro", uri=True) as db:
|
||||
state, invoked = db.execute(
|
||||
"SELECT state, hy_invoked FROM memory_write_jobs WHERE job_id=?",
|
||||
(record.job_id,),
|
||||
).fetchone()
|
||||
|
||||
assert state == UNKNOWN
|
||||
assert invoked == 1
|
||||
assert len(recorder.seen) == 1
|
||||
assert (
|
||||
queue.retry_first_unknown_after_reconcile(
|
||||
older_than=datetime.now(UTC) + timedelta(minutes=1)
|
||||
)
|
||||
== 0
|
||||
)
|
||||
assert queue.retry_unknown_confirmed(record.job_id) == 0
|
||||
assert DurableWriteDrain(queue, execute=_never_execute).claim(limit=1) == ()
|
||||
assert len(recorder.seen) == 1
|
||||
|
||||
|
||||
def test_event_time_not_enqueue_order_drives_the_window(
|
||||
tmp_path,
|
||||
) -> None:
|
||||
@@ -5674,3 +5352,93 @@ def test_hand_added_memories_still_run_one_at_a_time_in_order(tmp_path) -> None:
|
||||
assert drain.claim(limit=5) == ()
|
||||
assert queue.get("note-2").state == PENDING
|
||||
assert second.job_id
|
||||
|
||||
|
||||
def test_a_job_the_execute_gate_releases_is_not_reclaimed_while_it_still_refuses(
|
||||
tmp_path,
|
||||
) -> None:
|
||||
"""The claim gate and the execute gate can disagree for good (#245).
|
||||
|
||||
The claim asks about the lane's user, the execute gate about the user of
|
||||
the job's Evidence. An id-only pre-upgrade payload sits in the `local`
|
||||
lane even when its Evidence belongs to someone else: while that user
|
||||
evolves, the claim admits the job and the execute gate hands it back.
|
||||
Released with no hold, the very next tick claimed it again, in a loop.
|
||||
"""
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
record = queue.submit(_request(key="evolving-user", source="cursor"))
|
||||
drain = DurableWriteDrain(queue, execute=_never_execute, owner="gate-owner")
|
||||
assert drain.claim(limit=1) == (record.job_id,)
|
||||
|
||||
asyncio.run(
|
||||
execute_job(
|
||||
record.job_id,
|
||||
queue=queue,
|
||||
supervisor=_Recorder(error=RuntimeError("memory_evolve_in_progress")),
|
||||
evidence=_EvidenceStore({"e1"}),
|
||||
owner="gate-owner",
|
||||
)
|
||||
)
|
||||
|
||||
assert queue.get(record.request_id).state == PENDING
|
||||
now = datetime.now(UTC)
|
||||
assert drain.claim(limit=1, now=now) == ()
|
||||
assert drain.claim(limit=1, now=now + timedelta(seconds=4)) == ()
|
||||
assert drain.claim(limit=1, now=now + timedelta(seconds=6)) == (record.job_id,)
|
||||
with sqlite3.connect(path) as connection:
|
||||
assert connection.execute(
|
||||
"SELECT attempts,hy_invoked FROM memory_write_jobs WHERE job_id=?",
|
||||
(record.job_id,),
|
||||
).fetchone() == (1, 0)
|
||||
|
||||
|
||||
def _suspension_runtime(queue: DurableWriteQueue, enabled: set[str]):
|
||||
from one_memory.sdk_memory_runtime import SdkMemoryRuntime
|
||||
|
||||
return SdkMemoryRuntime(
|
||||
coordinator=SimpleNamespace(),
|
||||
read_handle=SimpleNamespace(), # type: ignore[arg-type]
|
||||
write_queue=queue,
|
||||
extraction_settings=SimpleNamespace(), # type: ignore[arg-type]
|
||||
data_dir=queue.database_path.parent,
|
||||
source_active=lambda source_key: source_key in enabled,
|
||||
)
|
||||
|
||||
|
||||
def test_a_disabled_source_suspends_its_jobs_and_enabling_it_resumes_them(tmp_path) -> None:
|
||||
"""A source the user turned off keeps its jobs, out of the pending count (#245)."""
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
recent = datetime.now(UTC)
|
||||
for index in range(3):
|
||||
queue.submit(_request(key=f"cursor-{index}", source="cursor", event_at=recent))
|
||||
codex = queue.submit(_request(key="codex-0", source="codex", event_at=recent))
|
||||
enabled = {"cursor", "codex"}
|
||||
runtime = _suspension_runtime(queue, enabled)
|
||||
drain = DurableWriteDrain(
|
||||
queue,
|
||||
execute=_never_execute,
|
||||
owner="suspend-owner",
|
||||
source_writes_permitted=lambda source_key: source_key in enabled,
|
||||
)
|
||||
|
||||
assert runtime.status_counts() == {PENDING: 4}
|
||||
assert runtime.queue_depths().fresh == 4
|
||||
|
||||
enabled.discard("cursor")
|
||||
|
||||
assert runtime.status_counts() == {PENDING: 1, "suspended": 3}
|
||||
assert runtime.queue_depths().fresh == 1
|
||||
assert drain.claim(limit=10) == (codex.job_id,)
|
||||
queue.settle(codex.job_id, state=COMPLETED, hy_invoked=True, owner="suspend-owner")
|
||||
assert drain.claim(limit=10) == ()
|
||||
assert runtime.status_counts() == {COMPLETED: 1, "suspended": 3}
|
||||
# Nothing was cancelled, moved or deleted: the rows are as they were.
|
||||
assert [row["state"] for row in _rows(path) if row["source_key"] == "cursor"] == [PENDING] * 3
|
||||
|
||||
enabled.add("cursor")
|
||||
|
||||
assert runtime.status_counts() == {PENDING: 3, COMPLETED: 1}
|
||||
assert runtime.queue_depths().fresh == 3
|
||||
assert len(drain.claim(limit=10)) == 1
|
||||
@@ -1181,7 +1181,7 @@ def test_authority_invalid_task_is_never_reinvoked(tmp_path) -> None:
|
||||
assert retained.last_error_code == "hy_memory_write_verification_authority_invalid"
|
||||
|
||||
|
||||
def test_linked_unknown_is_retried_only_by_the_durable_job(tmp_path) -> None:
|
||||
def test_linked_unknown_is_rerun_only_by_the_durable_job(tmp_path) -> None:
|
||||
from one_memory.memory_server.contracts import BatchIdentity, WriteRequest
|
||||
from one_memory.memory_server.durable_queue import DurableWriteQueue
|
||||
|
||||
@@ -1241,8 +1241,8 @@ def test_linked_unknown_is_retried_only_by_the_durable_job(tmp_path) -> None:
|
||||
cutoff = datetime(2026, 1, 1, 0, 5, tzinfo=UTC)
|
||||
assert store.retry_first_unknown_after_reconcile(older_than=cutoff) == 0
|
||||
assert store.require(task.task_id).status is MemoryTaskStatus.UNKNOWN
|
||||
assert queue.retry_first_unknown_after_reconcile(older_than=cutoff) == 1
|
||||
assert store.require(task.task_id).status is MemoryTaskStatus.PENDING
|
||||
# The durable job is the only execution entry: it re-runs itself.
|
||||
assert queue.recover_unknown() == 1
|
||||
assert queue.get(job.request_id).state == "pending"
|
||||
assert (
|
||||
store.active_window(
|
||||
|
||||
@@ -0,0 +1,607 @@
|
||||
"""A broken model configuration must not turn a backlog into a request storm (#245)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import sqlite3
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from types import SimpleNamespace
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
from pydantic import SecretStr
|
||||
|
||||
from one_memory.api import create_app
|
||||
from one_memory.config import Settings
|
||||
from one_memory.memory_server.contracts import BatchIdentity, WriteRequest
|
||||
from one_memory.memory_server.durable_queue import (
|
||||
FAILED,
|
||||
PENDING,
|
||||
DurableWriteDrain,
|
||||
DurableWriteQueue,
|
||||
execute_job,
|
||||
)
|
||||
from one_memory.model_backoff import (
|
||||
BACKOFF_CAP_SECONDS,
|
||||
BackedOffModelTransport,
|
||||
ModelBackoff,
|
||||
ModelBackoffRefusal,
|
||||
backoff_seconds,
|
||||
)
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
CHAT = "https://models.example.com/v1/chat/completions"
|
||||
EMBEDDINGS = "https://models.example.com/v1/embeddings"
|
||||
|
||||
|
||||
class _Clock:
|
||||
def __init__(self) -> None:
|
||||
self.now = 0.0
|
||||
|
||||
def __call__(self) -> float:
|
||||
return self.now
|
||||
|
||||
|
||||
def _request(index: int) -> WriteRequest:
|
||||
session = f"sess-{index}"
|
||||
return WriteRequest(
|
||||
internal_session_id=session,
|
||||
batch_identity=BatchIdentity(internal_session_id=session, first_round=1, last_round=5),
|
||||
evidence_ids=[f"e-{index}"],
|
||||
idempotency_key=f"idem-{index}",
|
||||
# Twenty sources, so the drain could claim ten jobs at once.
|
||||
source_id=f"source-{index % 20}",
|
||||
)
|
||||
|
||||
|
||||
async def _never_execute(job_id: str) -> None:
|
||||
raise AssertionError(f"the simulation settles {job_id} itself")
|
||||
|
||||
|
||||
def _send(transport: BackedOffModelTransport, url: str = CHAT) -> int | None:
|
||||
try:
|
||||
response = asyncio.run(transport.handle_async_request(httpx.Request("POST", url)))
|
||||
except ModelBackoffRefusal:
|
||||
return None
|
||||
return response.status_code
|
||||
|
||||
|
||||
def test_the_pause_doubles_from_five_seconds_and_stops_at_three_minutes() -> None:
|
||||
assert [backoff_seconds(level) for level in range(1, 9)] == [
|
||||
5.0,
|
||||
10.0,
|
||||
20.0,
|
||||
40.0,
|
||||
80.0,
|
||||
160.0,
|
||||
180.0,
|
||||
180.0,
|
||||
]
|
||||
assert backoff_seconds(1000) == BACKOFF_CAP_SECONDS == 180.0
|
||||
|
||||
|
||||
def test_an_always_failing_provider_sees_a_bounded_number_of_calls(tmp_path) -> None:
|
||||
"""200 pending jobs against a provider that refuses every one.
|
||||
|
||||
Before the pause the drain claimed the next job the moment the last one
|
||||
failed: hundreds of requests a minute. Now the calls follow the schedule.
|
||||
"""
|
||||
clock = _Clock()
|
||||
backoff = ModelBackoff(clock=clock)
|
||||
calls: list[float] = []
|
||||
|
||||
def provider(_request: httpx.Request) -> httpx.Response:
|
||||
calls.append(clock.now)
|
||||
return httpx.Response(401)
|
||||
|
||||
transport = BackedOffModelTransport(
|
||||
backoff, profile_id="p:1", inner=httpx.MockTransport(provider)
|
||||
)
|
||||
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
|
||||
for index in range(200):
|
||||
queue.submit(_request(index))
|
||||
drain = DurableWriteDrain(
|
||||
queue,
|
||||
execute=_never_execute,
|
||||
owner="sim",
|
||||
model_claim_allowance=backoff.allowance,
|
||||
)
|
||||
|
||||
claimed = 0
|
||||
for tick in range(6001):
|
||||
# Ten minutes in tenths of a second, without float drift.
|
||||
clock.now = tick / 10
|
||||
for job_id in drain.claim(limit=10):
|
||||
claimed += 1
|
||||
_send(transport)
|
||||
queue.settle(job_id, state=FAILED, owner="sim", error_code="model_auth_failed")
|
||||
|
||||
# The first claim was taken before anything had failed: two of its
|
||||
# requests reached the provider and the rest were refused locally. After
|
||||
# that one probe per pause: 5, 10, 20, 40, 80, 160, then 180 seconds.
|
||||
assert calls[:2] == [0.0, 0.0]
|
||||
probes = calls[2:]
|
||||
assert probes == pytest.approx([5.0, 15.0, 35.0, 75.0, 155.0, 315.0, 495.0], abs=0.11)
|
||||
gaps = [later - earlier for earlier, later in zip(probes, probes[1:], strict=False)]
|
||||
assert max(gaps) <= BACKOFF_CAP_SECONDS + 0.11
|
||||
assert len(calls) == 9
|
||||
assert claimed == 17
|
||||
with sqlite3.connect(tmp_path / "one-memory.sqlite3") as database:
|
||||
(pending,) = database.execute(
|
||||
"SELECT COUNT(*) FROM memory_write_jobs WHERE state=?", (PENDING,)
|
||||
).fetchone()
|
||||
assert pending == 200 - claimed
|
||||
|
||||
|
||||
def test_a_success_ends_the_pause_at_once() -> None:
|
||||
clock = _Clock()
|
||||
backoff = ModelBackoff(clock=clock)
|
||||
replies = [httpx.Response(503), httpx.Response(503), httpx.Response(200)]
|
||||
transport = BackedOffModelTransport(
|
||||
backoff, profile_id="p:1", inner=httpx.MockTransport(lambda _r: replies.pop(0))
|
||||
)
|
||||
|
||||
assert _send(transport) == 503
|
||||
clock.now = 5.0
|
||||
assert backoff.allowance() == 1
|
||||
assert _send(transport) == 503
|
||||
assert backoff.allowance() == 0
|
||||
assert backoff.state().last_error == "model_server_error"
|
||||
|
||||
clock.now = 15.0
|
||||
assert backoff.allowance() == 1
|
||||
assert _send(transport) == 200
|
||||
assert backoff.allowance() is None
|
||||
assert backoff.state().until is None
|
||||
assert backoff.state().failures == 0
|
||||
|
||||
|
||||
def test_a_working_embedding_endpoint_does_not_clear_a_broken_chat_model() -> None:
|
||||
clock = _Clock()
|
||||
backoff = ModelBackoff(clock=clock)
|
||||
|
||||
def provider(request: httpx.Request) -> httpx.Response:
|
||||
return httpx.Response(200 if request.url.path.endswith("/embeddings") else 404)
|
||||
|
||||
transport = BackedOffModelTransport(
|
||||
backoff, profile_id="p:1", inner=httpx.MockTransport(provider)
|
||||
)
|
||||
assert _send(transport) == 404
|
||||
assert _send(transport) == 404
|
||||
# Recall's embeddings still go out while the chat model is refused.
|
||||
assert _send(transport, EMBEDDINGS) == 200
|
||||
assert _send(transport) is None
|
||||
assert backoff.state().last_error == "model_not_found"
|
||||
assert backoff.paused()
|
||||
|
||||
|
||||
def test_an_unreachable_provider_is_paused_like_a_refusing_one() -> None:
|
||||
clock = _Clock()
|
||||
backoff = ModelBackoff(clock=clock)
|
||||
|
||||
def provider(request: httpx.Request) -> httpx.Response:
|
||||
raise httpx.ConnectError("name resolution failed", request=request)
|
||||
|
||||
transport = BackedOffModelTransport(
|
||||
backoff, profile_id="p:1", inner=httpx.MockTransport(provider)
|
||||
)
|
||||
with pytest.raises(httpx.ConnectError):
|
||||
_send(transport)
|
||||
assert backoff.state().last_error == "model_unreachable"
|
||||
assert backoff.allowance() == 0
|
||||
|
||||
|
||||
def test_a_request_problem_is_left_to_its_job() -> None:
|
||||
backoff = ModelBackoff(clock=_Clock())
|
||||
transport = BackedOffModelTransport(
|
||||
backoff, profile_id="p:1", inner=httpx.MockTransport(lambda _r: httpx.Response(400))
|
||||
)
|
||||
assert _send(transport) == 400
|
||||
assert backoff.allowance() is None
|
||||
|
||||
|
||||
def test_a_configuration_change_ends_the_pause() -> None:
|
||||
clock = _Clock()
|
||||
backoff = ModelBackoff(clock=clock)
|
||||
for _ in range(6):
|
||||
clock.now += 200.0
|
||||
backoff.record_failure("model_auth_failed", "p:1", "chat")
|
||||
assert backoff.state().until is not None
|
||||
|
||||
backoff.reset()
|
||||
|
||||
assert backoff.allowance() is None
|
||||
assert backoff.state().until is None
|
||||
# A new revision of the profile is a new configuration as well.
|
||||
backoff.record_failure("model_auth_failed", "p:1", "chat")
|
||||
backoff.record_failure("model_auth_failed", "p:2", "chat")
|
||||
assert backoff.state().failures == 1
|
||||
|
||||
|
||||
def test_a_probe_that_never_reports_does_not_hold_the_queue_forever() -> None:
|
||||
clock = _Clock()
|
||||
backoff = ModelBackoff(clock=clock)
|
||||
backoff.record_failure("model_timeout", "p:1", "chat")
|
||||
clock.now = 5.0
|
||||
assert backoff.allowance() == 1
|
||||
assert backoff.allowance() == 0
|
||||
clock.now = 35.0
|
||||
assert backoff.allowance() == 1
|
||||
|
||||
|
||||
def test_memory_processing_shows_the_pause_and_a_profile_save_ends_it(tmp_path) -> None:
|
||||
async def scenario() -> None:
|
||||
app = create_app(
|
||||
Settings(
|
||||
data_dir=tmp_path / "data",
|
||||
source_home=tmp_path,
|
||||
api_token=SecretStr("local-token"),
|
||||
),
|
||||
secret_store=InMemorySecretStore(),
|
||||
)
|
||||
headers = {"Authorization": "Bearer local-token", "Origin": "http://127.0.0.1:5173"}
|
||||
async with (
|
||||
app.router.lifespan_context(app),
|
||||
AsyncClient(
|
||||
transport=ASGITransport(app=app), base_url="http://test", headers=headers
|
||||
) as client,
|
||||
):
|
||||
created = await client.post(
|
||||
"/api/v1/model-profiles",
|
||||
json={
|
||||
"name": "Provider",
|
||||
"provider_kind": "openai_compatible",
|
||||
"base_url": "https://models.example.com/v1",
|
||||
"model": "memory-model",
|
||||
"embedding_model": "embedding-model",
|
||||
"embedding_base_url": "https://models.example.com/v1",
|
||||
"api_key": "api-route-secret",
|
||||
},
|
||||
)
|
||||
assert created.status_code == 200
|
||||
backoff: ModelBackoff = app.state.model_backoff
|
||||
backoff.record_failure("model_auth_failed", "p:1", "chat")
|
||||
|
||||
paused = (await client.get("/api/v1/memory-processing")).json()
|
||||
assert paused["llm_backoff_error"] == "model_auth_failed"
|
||||
until = datetime.fromisoformat(paused["llm_backoff_until"])
|
||||
assert until > datetime.now(UTC)
|
||||
assert paused["suspended"] == 0
|
||||
|
||||
saved = await client.patch(
|
||||
f"/api/v1/model-profiles/{created.json()['profile_id']}",
|
||||
json={"expected_revision": created.json()["revision"], "model": "other-model"},
|
||||
)
|
||||
assert saved.status_code == 200
|
||||
|
||||
cleared = (await client.get("/api/v1/memory-processing")).json()
|
||||
assert cleared["llm_backoff_until"] is None
|
||||
assert cleared["llm_backoff_error"] is None
|
||||
assert backoff.allowance() is None
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
# --- Every write retries itself: provider failures wait, others retry 3 times ---
|
||||
|
||||
|
||||
class _FakeSdk:
|
||||
"""Stands in for the SDK write path, with its real replay rule.
|
||||
|
||||
Like `HyMemoryClient`, it keeps one outcome per request id and replays a
|
||||
failed one without calling the model. Every write goes out through the
|
||||
breaker's transport, so the provider sees exactly what MemHub sends.
|
||||
"""
|
||||
|
||||
def __init__(self, transport: BackedOffModelTransport | None = None) -> None:
|
||||
self.transport = transport
|
||||
self.outcomes: dict[str, str] = {}
|
||||
self.written: list[str] = []
|
||||
self.request_ids: list[str] = []
|
||||
self.fail_with: str | None = None
|
||||
|
||||
async def process_sdk_job(self, records, *, request_id, before_hy_invocation):
|
||||
await before_hy_invocation()
|
||||
self.request_ids.append(request_id)
|
||||
if request_id in self.outcomes:
|
||||
if self.outcomes[request_id] == "failed":
|
||||
raise RuntimeError("hy_sdk_write_failed")
|
||||
raise AssertionError("a succeeded id is never sent again")
|
||||
if self.fail_with is not None:
|
||||
self.outcomes[request_id] = "failed"
|
||||
raise RuntimeError(self.fail_with)
|
||||
if self.transport is not None:
|
||||
try:
|
||||
response = await self.transport.handle_async_request(httpx.Request("POST", CHAT))
|
||||
except httpx.TransportError:
|
||||
self.outcomes[request_id] = "unknown"
|
||||
raise
|
||||
if response.status_code >= 400:
|
||||
self.outcomes[request_id] = "failed"
|
||||
raise RuntimeError("hy_sdk_write_failed")
|
||||
self.outcomes[request_id] = "succeeded"
|
||||
self.written.extend(record.evidence_id for record in records)
|
||||
return SimpleNamespace(raw_ids=(f"raw-{request_id}",), memory_ids=(f"m-{request_id}",))
|
||||
|
||||
async def inquire_write_outcome(self, request_id):
|
||||
if self.outcomes.get(request_id) != "succeeded":
|
||||
return None
|
||||
return SimpleNamespace(raw_ids=(f"raw-{request_id}",), memory_ids=(f"m-{request_id}",))
|
||||
|
||||
|
||||
class _Evidence:
|
||||
def get(self, evidence_id: str):
|
||||
return SimpleNamespace(evidence_id=evidence_id, source_id="cursor", user_id="local")
|
||||
|
||||
|
||||
def _ordered_request(index: int, *, source: str = "cursor") -> WriteRequest:
|
||||
return WriteRequest(
|
||||
internal_session_id="sess-1",
|
||||
batch_identity=BatchIdentity(
|
||||
internal_session_id="sess-1", first_round=index + 1, last_round=index + 1
|
||||
),
|
||||
evidence_ids=[f"e-{index:03d}"],
|
||||
idempotency_key=f"idem-{index:03d}",
|
||||
source_id=source,
|
||||
event_at=datetime(2026, 10, 1, tzinfo=UTC) + timedelta(minutes=index),
|
||||
)
|
||||
|
||||
|
||||
def _jobs(path) -> list[tuple[str, str, int, int, str | None]]:
|
||||
with sqlite3.connect(path) as database:
|
||||
return database.execute(
|
||||
"SELECT idempotency_key,state,sdk_attempt,retries,last_error "
|
||||
"FROM memory_write_jobs ORDER BY queue_seq"
|
||||
).fetchall()
|
||||
|
||||
|
||||
def test_an_outage_with_200_queued_jobs_ends_with_every_job_written_once_in_order(
|
||||
tmp_path,
|
||||
) -> None:
|
||||
"""The provider refuses every request for five minutes, then recovers.
|
||||
|
||||
No job is lost, skipped, written twice or left `unknown`; the order is
|
||||
the order of the conversation; the provider sees a handful of probes
|
||||
during the outage, not one request per job.
|
||||
"""
|
||||
clock = _Clock()
|
||||
backoff = ModelBackoff(clock=clock)
|
||||
calls: list[tuple[float, int]] = []
|
||||
|
||||
def provider(_request: httpx.Request) -> httpx.Response:
|
||||
status = 401 if clock.now < 300.0 else 200
|
||||
calls.append((clock.now, status))
|
||||
return httpx.Response(status)
|
||||
|
||||
sdk = _FakeSdk(
|
||||
BackedOffModelTransport(backoff, profile_id="p:1", inner=httpx.MockTransport(provider))
|
||||
)
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
for index in range(200):
|
||||
queue.submit(_ordered_request(index))
|
||||
drain = DurableWriteDrain(
|
||||
queue,
|
||||
execute=_never_execute,
|
||||
owner="outage",
|
||||
model_claim_allowance=backoff.allowance,
|
||||
)
|
||||
|
||||
async def tick() -> None:
|
||||
await asyncio.gather(
|
||||
*(
|
||||
execute_job(
|
||||
job_id,
|
||||
queue=queue,
|
||||
supervisor=sdk,
|
||||
evidence=_Evidence(),
|
||||
owner="outage",
|
||||
model_failures=backoff.attempt,
|
||||
)
|
||||
for job_id in drain.claim(limit=10)
|
||||
)
|
||||
)
|
||||
|
||||
for second in range(800):
|
||||
clock.now = float(second)
|
||||
asyncio.run(tick())
|
||||
|
||||
jobs = _jobs(path)
|
||||
assert [state for _, state, *_ in jobs] == ["completed"] * 200
|
||||
assert sdk.written == [f"e-{index:03d}" for index in range(200)]
|
||||
outage = [at for at, status in calls if status == 401]
|
||||
# One request at 0 s, then one probe per pause: 5, 15, 35, 75, 155 s.
|
||||
assert outage == [0.0, 5.0, 15.0, 35.0, 75.0, 155.0]
|
||||
# The first job was tried seven times; each try had its own SDK id, so
|
||||
# the recovered provider was really called, not a replayed failure.
|
||||
assert jobs[0][2:4] == (6, 0)
|
||||
assert sdk.request_ids[:3] == ["idem-000", "idem-000#r1", "idem-000#r2"]
|
||||
assert all(retries == 0 for *_, retries, _ in jobs)
|
||||
|
||||
|
||||
def test_a_failure_of_the_job_itself_is_retried_three_times_then_skipped(tmp_path) -> None:
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
first = queue.submit(_ordered_request(0))
|
||||
second = queue.submit(_ordered_request(1))
|
||||
sdk = _FakeSdk()
|
||||
drain = DurableWriteDrain(queue, execute=_never_execute, owner="skip")
|
||||
now = datetime.now(UTC)
|
||||
|
||||
def run(at: datetime) -> tuple[str, ...]:
|
||||
claimed = drain.claim(limit=1, now=at)
|
||||
for job_id in claimed:
|
||||
asyncio.run(
|
||||
execute_job(
|
||||
job_id,
|
||||
queue=queue,
|
||||
supervisor=sdk,
|
||||
evidence=_Evidence(),
|
||||
owner="skip",
|
||||
model_failures=ModelBackoff().attempt,
|
||||
)
|
||||
)
|
||||
return claimed
|
||||
|
||||
sdk.fail_with = "model_content_json_parse_failed"
|
||||
assert run(now) == (first.job_id,)
|
||||
# Each retry waits longer: 30 s, 60 s, 120 s. Until then the claim skips
|
||||
# the job, and the job after it in the conversation waits behind it.
|
||||
assert run(now + timedelta(seconds=20)) == ()
|
||||
assert run(now + timedelta(seconds=40)) == (first.job_id,)
|
||||
assert run(now + timedelta(seconds=150)) == (first.job_id,)
|
||||
assert run(now + timedelta(seconds=400)) == (first.job_id,)
|
||||
assert _jobs(path)[0][1:] == ("failed", 3, 3, "model_content_json_parse_failed")
|
||||
assert queue.get("idem-000").error_code == "model_content_json_parse_failed"
|
||||
|
||||
sdk.fail_with = None
|
||||
assert run(now + timedelta(seconds=401)) == (second.job_id,)
|
||||
assert _jobs(path)[1][1] == "completed"
|
||||
assert sdk.request_ids == [
|
||||
"idem-000",
|
||||
"idem-000#r1",
|
||||
"idem-000#r2",
|
||||
"idem-000#r3",
|
||||
"idem-001",
|
||||
]
|
||||
|
||||
|
||||
def test_a_write_interrupted_mid_call_is_rerun_under_a_new_request_id(tmp_path) -> None:
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
record = queue.submit(_ordered_request(0))
|
||||
drain = DurableWriteDrain(queue, execute=_never_execute, owner="crashed")
|
||||
assert drain.claim(limit=1) == (record.job_id,)
|
||||
assert queue.mark_invoked(record.job_id, owner="crashed")
|
||||
# The process stops; on restart the lease has lapsed.
|
||||
with sqlite3.connect(path) as database:
|
||||
database.execute(
|
||||
"UPDATE memory_write_jobs SET lease_expires_at=? WHERE job_id=?",
|
||||
("2000-01-01T00:00:00+00:00", record.job_id),
|
||||
)
|
||||
assert queue.reclaim_expired() == 1
|
||||
assert queue.get("idem-000").state == "unknown"
|
||||
# Still running on its thread in this process: left alone.
|
||||
assert queue.recover_unknown(in_flight=lambda request_id: request_id == "idem-000") == 0
|
||||
|
||||
assert queue.recover_unknown() == 1
|
||||
assert _jobs(path)[0][1:] == ("pending", 1, 1, "durable_write_interrupted")
|
||||
assert queue.recover_unknown() == 0
|
||||
|
||||
sdk = _FakeSdk()
|
||||
later = datetime.now(UTC) + timedelta(seconds=31)
|
||||
rerun = DurableWriteDrain(queue, execute=_never_execute, owner="restarted")
|
||||
assert rerun.claim(limit=1, now=later) == (record.job_id,)
|
||||
asyncio.run(
|
||||
execute_job(
|
||||
record.job_id, queue=queue, supervisor=sdk, evidence=_Evidence(), owner="restarted"
|
||||
)
|
||||
)
|
||||
assert sdk.request_ids == ["idem-000#r1"]
|
||||
assert queue.get("idem-000").state == "completed"
|
||||
|
||||
|
||||
def test_a_write_that_did_land_is_completed_from_its_receipt_not_rerun(tmp_path) -> None:
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
record = queue.submit(_ordered_request(0))
|
||||
with sqlite3.connect(path) as database:
|
||||
database.execute(
|
||||
"UPDATE memory_write_jobs SET state='unknown',hy_invoked=1,"
|
||||
"last_error='durable_work_ceiling_exceeded' WHERE job_id=?",
|
||||
(record.job_id,),
|
||||
)
|
||||
sdk = _FakeSdk()
|
||||
sdk.outcomes["idem-000"] = "succeeded"
|
||||
assert queue.recover_unknown() == 1
|
||||
drain = DurableWriteDrain(queue, execute=_never_execute, owner="landed")
|
||||
assert drain.claim(limit=1, now=datetime.now(UTC) + timedelta(seconds=31)) == (record.job_id,)
|
||||
asyncio.run(
|
||||
execute_job(
|
||||
record.job_id, queue=queue, supervisor=sdk, evidence=_Evidence(), owner="landed"
|
||||
)
|
||||
)
|
||||
assert sdk.request_ids == []
|
||||
completed = queue.get("idem-000")
|
||||
assert completed.state == "completed"
|
||||
assert completed.sdk_memory_ids == ("m-idem-000",)
|
||||
|
||||
|
||||
def test_unknown_jobs_from_before_the_upgrade_are_handled_once(tmp_path) -> None:
|
||||
from one_memory.memory_tasks import NON_RETRYABLE_POST_WRITE_CODES
|
||||
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path)
|
||||
plain = queue.submit(_ordered_request(0, source="a"))
|
||||
verified = queue.submit(_ordered_request(1, source="b"))
|
||||
looping = queue.submit(_ordered_request(2, source="c"))
|
||||
with sqlite3.connect(path) as database:
|
||||
for job, code, retries in (
|
||||
(plain, "hy_sdk_write_failed", 0),
|
||||
(verified, NON_RETRYABLE_POST_WRITE_CODES[0], 0),
|
||||
(looping, "durable_write_interrupted", 3),
|
||||
):
|
||||
database.execute(
|
||||
"UPDATE memory_write_jobs SET state='unknown',hy_invoked=1,attempts=1,"
|
||||
"last_error=?,retries=? WHERE job_id=?",
|
||||
(code, retries, job.job_id),
|
||||
)
|
||||
|
||||
assert queue.recover_unknown() == 3
|
||||
assert queue.recover_unknown() == 0
|
||||
# Reopening the queue (a restart) changes nothing either.
|
||||
assert DurableWriteQueue(path).recover_unknown() == 0
|
||||
states = {key: (state, attempt) for key, state, attempt, _, _ in _jobs(path)}
|
||||
assert states == {
|
||||
"idem-000": ("pending", 1),
|
||||
# The write landed against an authority Core could not verify.
|
||||
"idem-001": ("failed", 0),
|
||||
# A crash loop ends instead of spinning.
|
||||
"idem-002": ("failed", 0),
|
||||
}
|
||||
|
||||
|
||||
def test_concurrent_writes_are_all_charged_with_a_provider_failure() -> None:
|
||||
"""The breaker cannot tell whose request failed, so every write in flight waits.
|
||||
|
||||
A write the provider did fail must never spend its retries on the outage;
|
||||
a write that failed beside it for its own reason waits for the provider
|
||||
once, and is retried with the rest.
|
||||
"""
|
||||
backoff = ModelBackoff(clock=_Clock())
|
||||
with backoff.attempt() as first, backoff.attempt() as second:
|
||||
backoff.record_failure("model_server_error", "p:1", "chat")
|
||||
with backoff.attempt() as later:
|
||||
pass
|
||||
assert (first.failure, second.failure, later.failure) == (
|
||||
"model_server_error",
|
||||
"model_server_error",
|
||||
None,
|
||||
)
|
||||
|
||||
|
||||
def test_the_manual_retry_endpoint_is_gone(tmp_path) -> None:
|
||||
async def scenario() -> int:
|
||||
app = create_app(
|
||||
Settings(
|
||||
data_dir=tmp_path / "data",
|
||||
source_home=tmp_path,
|
||||
api_token=SecretStr("local-token"),
|
||||
),
|
||||
secret_store=InMemorySecretStore(),
|
||||
)
|
||||
async with (
|
||||
app.router.lifespan_context(app),
|
||||
AsyncClient(
|
||||
transport=ASGITransport(app=app),
|
||||
base_url="http://test",
|
||||
headers={"Authorization": "Bearer local-token"},
|
||||
) as client,
|
||||
):
|
||||
response = await client.post(
|
||||
"/api/v1/memory-processing/retry-unknown", json={"confirm": True}
|
||||
)
|
||||
return response.status_code
|
||||
|
||||
assert asyncio.run(scenario()) in {404, 405}
|
||||
@@ -334,7 +334,9 @@ def test_another_users_memory_id_answers_404_for_read_update_and_delete(
|
||||
|
||||
assert (await client.get(path)).status_code == 404
|
||||
assert (await client.put(path, json={"content": "stolen"})).status_code == 404
|
||||
assert (await client.delete(path)).status_code == 204
|
||||
refused = await client.delete(path)
|
||||
assert refused.status_code == 404
|
||||
assert refused.json()["detail"] == "memory_not_found"
|
||||
assert sdk.memories[wife_memory]["content"] == "wife drinks coffee"
|
||||
|
||||
own = await client.get(path, params={"user_id": "wife"})
|
||||
@@ -343,6 +345,36 @@ def test_another_users_memory_id_answers_404_for_read_update_and_delete(
|
||||
assert updated.status_code == 200
|
||||
assert (await client.delete(path, params={"user_id": "wife"})).status_code == 204
|
||||
assert wife_memory not in sdk.memories
|
||||
# A second delete (a double click) finds nothing: 404, already gone.
|
||||
again = await client.delete(path, params={"user_id": "wife"})
|
||||
assert again.status_code == 404
|
||||
assert again.json()["detail"] == "memory_not_found"
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_a_delete_that_loses_the_race_answers_404_like_a_missing_id(tmp_path, monkeypatch) -> None:
|
||||
"""Read as the caller's own, then gone before the SDK deletes it."""
|
||||
|
||||
async def scenario(app, sdk, client):
|
||||
await _write_both_users(app, sdk, client)
|
||||
[mine] = [
|
||||
memory_id
|
||||
for memory_id, row in sdk.memories.items()
|
||||
if row["content"] == "local drinks tea"
|
||||
]
|
||||
original = sdk.delete
|
||||
|
||||
def raced(memory_id, kind=None, variant=None):
|
||||
sdk.memories.pop(memory_id, None)
|
||||
return original(memory_id, kind=kind, variant=variant)
|
||||
|
||||
sdk.delete = raced # type: ignore[method-assign]
|
||||
response = await client.delete(f"/api/v2/memories/{mine}")
|
||||
missing = await client.delete("/api/v2/memories/no-such-memory")
|
||||
assert _observed(response) == _observed(missing)
|
||||
assert response.status_code == 404
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
@@ -426,6 +458,52 @@ async def _true() -> bool:
|
||||
return True
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
@pytest.mark.parametrize("user", [None, "wife"])
|
||||
def test_every_read_echoes_the_user_it_was_read_for(tmp_path, monkeypatch, user) -> None:
|
||||
"""Each read names the user it answered for: the one asked for, else `local`."""
|
||||
|
||||
async def scenario(app, sdk, client):
|
||||
await _write_both_users(app, sdk, client)
|
||||
monkeypatch.setattr(
|
||||
"one_memory.api._recall_model_ready", lambda _app: _true(), raising=True
|
||||
)
|
||||
app.state.recall_target = _sdk_recall_target(app, "local")
|
||||
expected = user or "local"
|
||||
params = {} if user is None else {"user_id": user}
|
||||
body = {} if user is None else {"user_id": user}
|
||||
reads = [
|
||||
await client.get("/api/v2/memory-library", params=params),
|
||||
await client.get("/api/v2/memory-library", params={**params, "query": "route"}),
|
||||
await client.get("/api/v2/memory-library/layers", params=params),
|
||||
await client.get("/api/v2/user-profile", params=params),
|
||||
await client.get("/api/v2/home-summary", params=params),
|
||||
await client.get("/api/v2/memories/graph", params=params),
|
||||
await client.get("/api/v2/raw", params=params),
|
||||
await client.post(
|
||||
"/api/v2/context/search", json={"query": "drinks", "kinds": ["memory"], **body}
|
||||
),
|
||||
await client.post("/api/v1/recall", json={"query": "drinks", **body}),
|
||||
await client.post("/api/v2/memories/ask", json={"question": "what drink", **body}),
|
||||
]
|
||||
for response in reads:
|
||||
assert response.status_code == 200, (response.request.url, response.text)
|
||||
assert response.json()["user_id"] == expected, response.request.url
|
||||
|
||||
async with client.stream(
|
||||
"POST", "/api/v2/memories/ask/stream", json={"question": "what drink", **body}
|
||||
) as streamed:
|
||||
events = [
|
||||
json.loads(line[5:])
|
||||
async for line in streamed.aiter_lines()
|
||||
if line.startswith("data:")
|
||||
]
|
||||
assert events[0]["type"] == "recall"
|
||||
assert events[0]["user_id"] == expected
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
@pytest.mark.parametrize(
|
||||
("method", "path", "kwargs"),
|
||||
@@ -569,6 +647,26 @@ def test_a_hermes_root_scan_never_reads_its_profiles(tmp_path) -> None:
|
||||
assert _scan_texts(wife, tmp_path) == ["I am the wife", "hello", "wife's transcript"]
|
||||
|
||||
|
||||
def test_a_profiles_directory_is_skipped_at_any_depth(tmp_path) -> None:
|
||||
"""A root above the Hermes home still never reads `.hermes/profiles/**`."""
|
||||
home = tmp_path / "home"
|
||||
hermes, _wife = _hermes_homes(home)
|
||||
(hermes / "sessions").mkdir()
|
||||
(hermes / "sessions" / "own.jsonl").write_text(
|
||||
json.dumps({"type": "message", "role": "user", "content": "own transcript"}) + "\n"
|
||||
)
|
||||
nested = hermes / "sessions" / "profiles" / "kid" / "sessions"
|
||||
nested.mkdir(parents=True)
|
||||
(nested / "k.jsonl").write_text(
|
||||
json.dumps({"type": "message", "role": "user", "content": "kid's transcript"}) + "\n"
|
||||
)
|
||||
|
||||
above = _scan_texts(home, tmp_path)
|
||||
assert "I am the wife" not in above and "kid's transcript" not in above
|
||||
assert set(above) == {"I am the husband", "hi", "own transcript"}
|
||||
assert "kid's transcript" not in _scan_texts(hermes, tmp_path)
|
||||
|
||||
|
||||
def _jobs(database: Path) -> list[sqlite3.Row]:
|
||||
with sqlite3.connect(database) as db:
|
||||
db.row_factory = sqlite3.Row
|
||||
@@ -719,6 +817,87 @@ def test_a_migrated_home_with_the_old_grant_triggers_can_connect_and_revoke(tmp_
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
async def _connect(client, user_id: str, root: Path | None = None, **extra):
|
||||
body: dict[str, object] = {"source_ids": ["hermes"], "request_id": str(uuid4()), **extra}
|
||||
if user_id != "local":
|
||||
body["user_id"] = user_id
|
||||
if root is not None:
|
||||
body["root"] = str(root)
|
||||
return await client.post("/api/v1/sources/connect", json=body)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_a_hermes_root_that_is_the_profiles_container_is_refused(tmp_path) -> None:
|
||||
async def scenario():
|
||||
home = tmp_path / "home"
|
||||
hermes, _wife = _hermes_homes(home)
|
||||
app = application(tmp_path, home=home)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
for user in ("wife", "local"):
|
||||
refused = await _connect(client, user, hermes / "profiles")
|
||||
assert refused.status_code == 422, refused.text
|
||||
assert refused.json()["detail"] == "hermes_root_is_profiles_container"
|
||||
assert app.state.source_grant_store.grants() == ()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_two_users_on_one_hermes_root_or_nested_roots_are_refused(tmp_path) -> None:
|
||||
async def scenario():
|
||||
home = tmp_path / "home"
|
||||
hermes, wife = _hermes_homes(home)
|
||||
(wife / "sessions").mkdir()
|
||||
link = home / "hermes-link"
|
||||
link.symlink_to(hermes, target_is_directory=True)
|
||||
app = application(tmp_path, home=home)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
location = (await client.post("/api/v1/sources/hermes/location")).json()
|
||||
local = await _connect(client, "local", source_roots={"hermes": location["root_id"]})
|
||||
assert local.status_code == 200, local.text
|
||||
# Same root, the same root through a symlink, a root inside it
|
||||
# that is not `profiles/<name>`, and a root above it.
|
||||
for root in (hermes, link, wife / "sessions", home):
|
||||
refused = await _connect(client, "kid", root)
|
||||
assert refused.status_code == 409, (root, refused.text)
|
||||
assert refused.json()["detail"] == "hermes_root_overlaps_other_user"
|
||||
# The standard layout: `local` on ~/.hermes, wife on its profile.
|
||||
accepted = await _connect(client, "wife", wife)
|
||||
assert accepted.status_code == 200, accepted.text
|
||||
# Another user on wife's own root is refused too.
|
||||
twin = await _connect(client, "wife2", wife)
|
||||
assert twin.status_code == 409
|
||||
# Once wife's grant is revoked her root is free again.
|
||||
await client.post("/api/v1/sources/hermes/revoke", params={"user_id": "wife"})
|
||||
assert (await _connect(client, "wife2", wife)).status_code == 200
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_local_cannot_take_another_users_hermes_root(tmp_path) -> None:
|
||||
"""The check runs for `local`'s own connect too, not only for other users."""
|
||||
|
||||
async def scenario():
|
||||
home = tmp_path / "home"
|
||||
hermes, _wife = _hermes_homes(home)
|
||||
other = home / "elsewhere" / ".hermes"
|
||||
_hermes_state_db(other, [(1, "s", "user", "elsewhere")])
|
||||
app = application(tmp_path, home=home)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
# wife took ~/.hermes itself, which is `local`'s default root.
|
||||
taken = await _connect(client, "wife", hermes)
|
||||
assert taken.status_code == 200, taken.text
|
||||
location = (await client.post("/api/v1/sources/hermes/location")).json()
|
||||
refused = await _connect(client, "local", source_roots={"hermes": location["root_id"]})
|
||||
assert refused.status_code == 409, refused.text
|
||||
assert refused.json()["detail"] == "hermes_root_overlaps_other_user"
|
||||
# `local` on a root of its own that does not overlap is fine.
|
||||
assert (await _connect(client, "local", other)).status_code == 200
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_only_hermes_takes_another_user_or_a_custom_root(tmp_path) -> None:
|
||||
async def scenario():
|
||||
@@ -852,7 +1031,7 @@ def test_another_users_id_answers_exactly_like_an_id_that_does_not_exist(
|
||||
other = await client.request(method, theirs, **kwargs)
|
||||
absent = await client.request(method, missing, **kwargs)
|
||||
assert _observed(other) == _observed(absent), (method, theirs)
|
||||
assert other.status_code == (204 if method == "DELETE" else 404)
|
||||
assert other.status_code == 404
|
||||
assert sdk.memories == before
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
@@ -259,3 +259,52 @@ def test_recall_route_falls_back_and_keeps_the_raw_grant(tmp_path, monkeypatch):
|
||||
# A host granted recall alone may not read original conversations.
|
||||
assert recall_only["mode"] == "raw" and recall_only["hits"] == []
|
||||
assert len(searched) == 1
|
||||
|
||||
|
||||
def test_the_total_recall_budget_stays_under_the_brokers_wait():
|
||||
from one_memory.recall_fallback import recall_total_timeout_seconds
|
||||
|
||||
assert recall_total_timeout_seconds(800) == pytest.approx(3.8)
|
||||
# The broker gives Core 15 seconds; Core answers before it gives up.
|
||||
assert recall_total_timeout_seconds(60_000) == 12.0
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_a_recall_that_cannot_finish_in_time_is_a_clear_timeout(tmp_path, monkeypatch):
|
||||
"""A recall stuck behind the SDK answers 504 `recall_timeout`, not nothing (#231)."""
|
||||
from one_memory import recall_fallback
|
||||
|
||||
async def stuck(*_args, **_kwargs):
|
||||
await asyncio.Event().wait()
|
||||
|
||||
monkeypatch.setattr(recall_fallback, "recall_with_fallback", stuck)
|
||||
monkeypatch.setattr(recall_fallback, "recall_total_timeout_seconds", lambda _ms: 0.2)
|
||||
|
||||
async def scenario():
|
||||
app = create_app(
|
||||
Settings(
|
||||
data_dir=tmp_path / "data",
|
||||
source_home=tmp_path,
|
||||
api_token=SecretStr("recall-test"),
|
||||
imports_enabled=False,
|
||||
automatic_memory_processing=False,
|
||||
live_sync_enabled=False,
|
||||
),
|
||||
secret_store=InMemorySecretStore(),
|
||||
)
|
||||
async with (
|
||||
app.router.lifespan_context(app),
|
||||
AsyncClient(
|
||||
transport=ASGITransport(app=app),
|
||||
base_url="http://test",
|
||||
headers={"Authorization": "Bearer recall-test"},
|
||||
) as client,
|
||||
):
|
||||
return await asyncio.wait_for(
|
||||
client.post("/api/v1/recall", json={"query": "数据库"}), 5
|
||||
)
|
||||
|
||||
response = asyncio.run(scenario())
|
||||
|
||||
assert response.status_code == 504
|
||||
assert response.json()["detail"] == "recall_timeout"
|
||||
@@ -1124,13 +1124,13 @@ def test_pre_upgrade_unknown_writes_are_settled_at_cutover_not_retried(store):
|
||||
surviving = DurableWriteQueue(store.database_path, sdk_authority=True)
|
||||
assert surviving.unknown_jobs() == ()
|
||||
# Never requeued: they may already have written.
|
||||
assert surviving.retry_unknown_confirmed(unknown[0]) == 0
|
||||
assert surviving.recover_unknown() == 0
|
||||
assert surviving.readmit_parked() == 0
|
||||
assert PENDING not in {state for state, _, _ in rows.values()}
|
||||
|
||||
|
||||
def test_retrying_unknown_writes_works_after_the_legacy_tables_are_dropped(store):
|
||||
"""The retry button raised `no such table: memory_tasks` after an upgrade."""
|
||||
def test_recovering_unknown_writes_works_after_the_legacy_tables_are_dropped(store):
|
||||
"""The old retry button raised `no such table: memory_tasks` after an upgrade."""
|
||||
from one_memory.memory_server.durable_queue import PENDING, UNKNOWN
|
||||
from one_memory.target_schema import drop_legacy_tables
|
||||
|
||||
@@ -1162,14 +1162,10 @@ def test_retrying_unknown_writes_works_after_the_legacy_tables_are_dropped(store
|
||||
surviving = DurableWriteQueue(store.database_path, sdk_authority=sdk_authority)
|
||||
make_unknown("model_content_outcome_unknown", 1)
|
||||
assert surviving.resolve_unknown_from_linked_output(identifier) is False
|
||||
assert surviving.retry_unknown_confirmed(identifier) == 1
|
||||
assert surviving.recover_unknown() == 1
|
||||
assert surviving.get("after-upgrade").state == PENDING
|
||||
|
||||
make_unknown("hy_memory_timeout", 1)
|
||||
since = datetime.now(UTC)
|
||||
assert surviving.retry_timed_out_unknown_after_reconcile(older_than=since) == 1
|
||||
make_unknown("model_content_outcome_unknown", 1)
|
||||
assert surviving.retry_first_unknown_after_reconcile(older_than=since) == 1
|
||||
with main_database(store.database_path).write() as database, database:
|
||||
database.execute("UPDATE memory_write_jobs SET retries=0 WHERE job_id=?", (identifier,))
|
||||
if sdk_authority:
|
||||
# The SDK-authority queue never brings a dropped table back.
|
||||
assert legacy_tables_present(store.database_path) == ()
|
||||
|
||||
@@ -91,7 +91,7 @@ def test_bundle_contains_management_sdk_config_and_verified_manifest(tmp_path):
|
||||
assert manifest["bundle_id"] == "migration-1"
|
||||
assert manifest["sdk_namespace"] == "governed-v2"
|
||||
assert manifest["runtime_build"] == "build-13"
|
||||
assert manifest["snapshot_sdk_version"] == "2.0.11"
|
||||
assert manifest["snapshot_sdk_version"] == "2.0.12"
|
||||
paths = {item["path"]: item for item in manifest["files"]}
|
||||
assert {
|
||||
"management/one-memory.sqlite3",
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
"""The App's display language, and the SDK prompt language it implies.
|
||||
|
||||
The owner's rule: zh-Hans, or no explicit choice, passes no language, so the
|
||||
SDK keeps detecting zh/en exactly as before; every other code (en included)
|
||||
is passed as `MemoryConfig.language` (hy-memory 2.0.11). A change rebuilds the
|
||||
The owner's rule (#243): every display language, zh-Hans and en included, is
|
||||
passed as `MemoryConfig.language` (hy-memory 2.0.11); only an unset language
|
||||
passes none, so the SDK keeps detecting zh/en exactly as before. A change rebuilds the
|
||||
handles the way a mode or feature change does, and the language never touches
|
||||
the vector collection identity.
|
||||
"""
|
||||
@@ -28,8 +28,8 @@ from one_memory.secrets import InMemorySecretStore
|
||||
from one_memory.ui_language import UI_LANGUAGES, normalize_ui_language, sdk_language
|
||||
|
||||
TOKEN = "ui-language-test"
|
||||
#: Every code except zh-Hans is passed to the SDK.
|
||||
PASSED = ("en", "zh-Hant", "ja", "es", "de", "pt", "fr", "ko", "ru", "id")
|
||||
#: Every code is passed to the SDK, zh-Hans included.
|
||||
PASSED = ("en", "zh-Hans", "zh-Hant", "ja", "es", "de", "pt", "fr", "ko", "ru", "id")
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
@@ -51,7 +51,7 @@ def _config(tmp_path: Path, language: str | None):
|
||||
|
||||
|
||||
def test_eleven_codes_and_the_legacy_chinese_code() -> None:
|
||||
assert ("en", "zh-Hans", *PASSED[1:]) == UI_LANGUAGES
|
||||
assert PASSED == UI_LANGUAGES
|
||||
assert len(UI_LANGUAGES) == 11
|
||||
# Earlier App builds stored zh-CN; it is zh-Hans now.
|
||||
assert normalize_ui_language("zh-CN") == "zh-Hans"
|
||||
@@ -60,19 +60,20 @@ def test_eleven_codes_and_the_legacy_chinese_code() -> None:
|
||||
assert normalize_ui_language(None) is None
|
||||
|
||||
|
||||
def test_only_zh_hans_and_no_choice_leave_the_sdk_to_detect() -> None:
|
||||
def test_only_an_unset_language_leaves_the_sdk_to_detect() -> None:
|
||||
assert sdk_language(None) is None
|
||||
assert sdk_language("zh-Hans") is None
|
||||
assert sdk_language("zh-CN") is None
|
||||
assert sdk_language("system") is None
|
||||
assert sdk_language("xx") is None
|
||||
# zh-Hans is passed like every other code; the legacy zh-CN maps to it.
|
||||
assert sdk_language("zh-Hans") == "zh-Hans"
|
||||
assert sdk_language("zh-CN") == "zh-Hans"
|
||||
for code in PASSED:
|
||||
assert sdk_language(code) == code
|
||||
|
||||
|
||||
def test_zh_hans_and_no_choice_build_the_config_of_before(tmp_path) -> None:
|
||||
before = repr(dataclasses.asdict(_config(tmp_path, None)))
|
||||
def test_no_language_builds_the_config_of_before(tmp_path) -> None:
|
||||
assert _config(tmp_path, None).language is None
|
||||
assert repr(dataclasses.asdict(_config(tmp_path, sdk_language("zh-Hans")))) == before
|
||||
assert _config(tmp_path, sdk_language("zh-Hans")).language == "zh-Hans"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("code", PASSED)
|
||||
@@ -282,17 +283,31 @@ def test_ui_language_api_get_put_and_status(tmp_path, monkeypatch) -> None:
|
||||
assert current["languages"] == list(UI_LANGUAGES)
|
||||
assert reads.current_language() is None and writes.sdk_language() is None
|
||||
|
||||
# zh-Hans: recorded, but the SDK still detects zh/en; nothing rebuilt.
|
||||
# zh-Hans is passed explicitly: Chinese prompts, handles rebuilt (#243).
|
||||
body = (
|
||||
await client.put("/api/v1/settings/ui-language", json={"ui_language": "zh-Hans"})
|
||||
).json()
|
||||
assert body == {**current, "ui_language": "zh-Hans", "rebuilt": False}
|
||||
assert retired == [] and reads.current_language() is None
|
||||
assert body == {
|
||||
**current,
|
||||
"ui_language": "zh-Hans",
|
||||
"sdk_language": "zh-Hans",
|
||||
"rebuilt": True,
|
||||
}
|
||||
assert reads.current_language() == "zh-Hans" and writes.sdk_language() == "zh-Hans"
|
||||
assert retired == ["zh-Hans"]
|
||||
processing = (await client.get("/api/v1/memory-processing")).json()
|
||||
assert processing["pipeline"]["sdk_language"] == "zh-Hans"
|
||||
|
||||
# The same value again changes nothing.
|
||||
body = (
|
||||
await client.put("/api/v1/settings/ui-language", json={"ui_language": "zh-Hans"})
|
||||
).json()
|
||||
assert body["rebuilt"] is False and retired == ["zh-Hans"]
|
||||
|
||||
body = (await client.put("/api/v1/settings/ui-language", json={"ui_language": "ja"})).json()
|
||||
assert body["sdk_language"] == "ja" and body["rebuilt"] is True
|
||||
assert reads.current_language() == "ja" and writes.sdk_language() == "ja"
|
||||
assert retired == ["ja"]
|
||||
assert retired == ["zh-Hans", "ja"]
|
||||
|
||||
# An explicit English choice forces English prompts.
|
||||
body = (await client.put("/api/v1/settings/ui-language", json={"ui_language": "en"})).json()
|
||||
@@ -302,7 +317,7 @@ def test_ui_language_api_get_put_and_status(tmp_path, monkeypatch) -> None:
|
||||
assert processing["pipeline"]["ui_language"] == "en"
|
||||
assert processing["pipeline"]["sdk_language"] == "en"
|
||||
|
||||
# Back to "follow the system": auto again.
|
||||
# Unset (CLI `settings language system`): auto again.
|
||||
body = (await client.put("/api/v1/settings/ui-language", json={"ui_language": None})).json()
|
||||
assert body["sdk_language"] == "auto" and body["rebuilt"] is True
|
||||
assert reads.current_language() is None
|
||||
|
||||
Generated
+4
-4
@@ -991,7 +991,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "hy-memory"
|
||||
version = "2.0.11"
|
||||
version = "2.0.12"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "chromadb" },
|
||||
@@ -1008,9 +1008,9 @@ dependencies = [
|
||||
{ name = "scikit-learn" },
|
||||
{ name = "tiktoken" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/01/93/aec7d243259f6a315f38e8ebc21a9eea153a48fc13648fda149be7672788/hy_memory-2.0.11.tar.gz", hash = "sha256:96d6bd606d61052f127cd274aa6f8de8e7151d6250f1e89e4ef5b90e18ec7cb0", size = 1245738, upload-time = "2026-11-01T14:15:02.474Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/44/19/bcd0de44ae2011ee1ad70579996aad72e18f0ca292df22d1545db05db850/hy_memory-2.0.12.tar.gz", hash = "sha256:c1231596b4cf12595775541f81680d4819461cb69e4aa4dc449f876d4c3b6742", size = 1246969, upload-time = "2026-11-02T06:30:37.448Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/1a/85/1f5e6a13507a0cfcdce26f67c590df00d327a0f7d5c5b5483c3bd0fa2f85/hy_memory-2.0.11-py3-none-any.whl", hash = "sha256:c0173444a83c1197a49acaccb123f8286cea205b4838c1e83d106974f5bad259", size = 1315871, upload-time = "2026-11-01T14:14:58.764Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/75/9e/b64c7643c1bad1ddca4623a39bb62423bed0c22e7fd0d86be1ff73b2a17a/hy_memory-2.0.12-py3-none-any.whl", hash = "sha256:7a2332288197fafb89d5037eac7fef97df4c80794ead18159d21da84cde431df", size = 1316922, upload-time = "2026-11-02T06:30:35.105Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1837,7 +1837,7 @@ requires-dist = [
|
||||
{ name = "fastapi", specifier = ">=0.141.1" },
|
||||
{ name = "filelock", specifier = ">=3.32.4" },
|
||||
{ name = "httpx", specifier = ">=0.28.1" },
|
||||
{ name = "hy-memory", specifier = "==2.0.11" },
|
||||
{ name = "hy-memory", specifier = "==2.0.12" },
|
||||
{ name = "keyring", specifier = ">=25.7.0" },
|
||||
{ name = "numpy", specifier = "==2.5.2" },
|
||||
{ name = "onnxruntime", specifier = "==1.23.2" },
|
||||
|
||||
@@ -15,9 +15,13 @@ it("bakes fallback release identity when the packaging pipeline omits it", async
|
||||
expect(config).toMatch(
|
||||
/__ONE_MEMORY_BUILD_NUMBER__: JSON\.stringify\(\s*String\(releaseBuildNumber\)/,
|
||||
);
|
||||
expect(config).toMatch(
|
||||
/process\.env\.ONE_MEMORY_COMMIT\?\.trim\(\) \|\| readSourceCommit\(\)/,
|
||||
expect(config).toContain(
|
||||
"const releaseCommit = resolveReleaseCommit(process.env.ONE_MEMORY_COMMIT)",
|
||||
);
|
||||
expect(config).toMatch(/if \(!trimmed\) return readSourceCommit\(\);/);
|
||||
// A supplied commit must be the full 40-character form the runtime accepts.
|
||||
expect(config).toContain("/^[0-9a-f]{40}$/i.test(trimmed)");
|
||||
expect(config).toContain("return trimmed.toLowerCase();");
|
||||
expect(config).toMatch(
|
||||
/execFileSync\("git", \["rev-parse", "HEAD"\],[\s\S]*?encoding: "utf8"/,
|
||||
);
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import type { LanguageSnapshot } from "@one-memory/i18n";
|
||||
import type { GuiLocale, LanguageSnapshot } from "@one-memory/i18n";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { coreUiLanguage, installCoreLanguageSync } from "./core-language-sync";
|
||||
|
||||
@@ -21,11 +21,16 @@ function source(initial: LanguageSnapshot) {
|
||||
const settle = () => new Promise((resolve) => setTimeout(resolve, 0));
|
||||
|
||||
describe("Core language sync", () => {
|
||||
it("tells Core null for system and the code for an explicit choice", () => {
|
||||
expect(coreUiLanguage("system")).toBeNull();
|
||||
expect(coreUiLanguage("zh-Hans")).toBe("zh-Hans");
|
||||
expect(coreUiLanguage("en")).toBe("en");
|
||||
expect(coreUiLanguage("ja")).toBe("ja");
|
||||
it("tells Core the language the GUI shows, system included", () => {
|
||||
const shown = (
|
||||
preference: LanguageSnapshot["preference"],
|
||||
locale: GuiLocale,
|
||||
) => coreUiLanguage({ preference, locale, revision: 0 });
|
||||
expect(shown("system", "zh-Hans")).toBe("zh-Hans");
|
||||
expect(shown("system", "en")).toBe("en");
|
||||
expect(shown("zh-Hans", "zh-Hans")).toBe("zh-Hans");
|
||||
expect(shown("en", "en")).toBe("en");
|
||||
expect(shown("ja", "ja")).toBe("ja");
|
||||
});
|
||||
|
||||
it("sends the stored choice once at launch, then every change", async () => {
|
||||
@@ -38,29 +43,33 @@ describe("Core language sync", () => {
|
||||
await settle();
|
||||
language.change({ preference: "system", locale: "en", revision: 2 });
|
||||
await settle();
|
||||
expect(send.mock.calls).toEqual([["es"], ["ja"], [null]]);
|
||||
expect(send.mock.calls).toEqual([["es"], ["ja"], ["en"]]);
|
||||
dispose();
|
||||
language.change({ preference: "de", locale: "de", revision: 3 });
|
||||
await settle();
|
||||
expect(send).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
|
||||
it("an untouched install sends null, which leaves the SDK on auto", async () => {
|
||||
it("an untouched install sends the system language the GUI shows", async () => {
|
||||
const language = source({
|
||||
preference: "system",
|
||||
locale: "ja",
|
||||
locale: "zh-Hans",
|
||||
revision: 0,
|
||||
});
|
||||
const send = vi.fn(async () => {});
|
||||
installCoreLanguageSync({ language, send });
|
||||
await settle();
|
||||
expect(send.mock.calls).toEqual([[null]]);
|
||||
expect(send.mock.calls).toEqual([["zh-Hans"]]);
|
||||
// Choosing the language already shown changes nothing Core sees.
|
||||
language.change({ preference: "zh-Hans", locale: "zh-Hans", revision: 1 });
|
||||
await settle();
|
||||
expect(send).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("reports a failed send and retries it with the next change", async () => {
|
||||
const language = source({ preference: "fr", locale: "fr", revision: 0 });
|
||||
const send = vi
|
||||
.fn<(value: string | null) => Promise<void>>()
|
||||
.fn<(value: string) => Promise<void>>()
|
||||
.mockRejectedValueOnce(new Error("core_unavailable"))
|
||||
.mockResolvedValue(undefined);
|
||||
const reportError = vi.fn();
|
||||
|
||||
@@ -1,35 +1,37 @@
|
||||
import type { GuiLocale, LanguagePreference } from "@one-memory/i18n";
|
||||
import type { GuiLocale, LanguageSnapshot } from "@one-memory/i18n";
|
||||
import type { NativeLanguageSource } from "./native-language";
|
||||
|
||||
/** What Core is told: a code, or null for "follow the system". */
|
||||
export function coreUiLanguage(
|
||||
preference: LanguagePreference,
|
||||
): GuiLocale | null {
|
||||
return preference === "system" ? null : preference;
|
||||
/**
|
||||
* What Core is told: the language the GUI shows. "Follow the system" sends
|
||||
* the language it resolved to, so the memories match the interface a person
|
||||
* sees; the App never leaves the SDK to detect zh/en (#243).
|
||||
*/
|
||||
export function coreUiLanguage(snapshot: LanguageSnapshot): GuiLocale {
|
||||
return snapshot.locale;
|
||||
}
|
||||
|
||||
/**
|
||||
* Keep Core's `ui_language` setting on the display language a person chose.
|
||||
* Keep Core's `ui_language` setting on the display language the GUI shows.
|
||||
*
|
||||
* Core builds every SDK client with it: zh-Hans and "follow the system" pass
|
||||
* no language (the SDK keeps detecting zh/en), any other choice sets the SDK
|
||||
* prompt language. Sent once when installed, so a choice made before this
|
||||
* existed reaches Core, then on every committed change; a value Core already
|
||||
* has is not sent again. Best effort: a failed send is reported and retried
|
||||
* with the next change or launch, and never blocks the GUI.
|
||||
* Core builds every SDK client with it and passes it, zh-Hans included, as
|
||||
* the SDK prompt language. Sent once when installed, so a choice made before
|
||||
* this existed, or the system language followed at this launch, reaches
|
||||
* Core; then on every committed change. A value Core already has is not sent
|
||||
* again. Best effort: a failed send is reported and retried with the next
|
||||
* change or launch, and never blocks the GUI.
|
||||
*/
|
||||
export function installCoreLanguageSync(options: {
|
||||
language: NativeLanguageSource;
|
||||
send(uiLanguage: GuiLocale | null): Promise<void>;
|
||||
send(uiLanguage: GuiLocale): Promise<void>;
|
||||
reportError?(error: unknown): void;
|
||||
}): () => void {
|
||||
let closed = false;
|
||||
let sent: GuiLocale | null | undefined;
|
||||
let sent: GuiLocale | undefined;
|
||||
let queue: Promise<void> = Promise.resolve();
|
||||
const push = () => {
|
||||
queue = queue.then(async () => {
|
||||
if (closed) return;
|
||||
const value = coreUiLanguage(options.language.read().preference);
|
||||
const value = coreUiLanguage(options.language.read());
|
||||
if (value === sent) return;
|
||||
try {
|
||||
await options.send(value);
|
||||
|
||||
@@ -526,7 +526,22 @@ describe("hourly isolated desktop launch", () => {
|
||||
expect(resolvePackagedBuildNumber("-1")).toBeUndefined();
|
||||
expect(resolvePackagedBuildNumber(undefined, "3907")).toBe(3907);
|
||||
expect(resolvePackagedBuildNumber("3908", "3907")).toBe(3908);
|
||||
expect(resolvePackagedCommit(undefined, "388fd0e9")).toBe("388fd0e9");
|
||||
expect(resolvePackagedCommit("not-a-sha", "388fd0e9")).toBe("388fd0e9");
|
||||
});
|
||||
|
||||
it("resolves only a full 40-character commit and lowercases it", () => {
|
||||
// Every downstream check (packaged-runtime install, local install, release
|
||||
// download, release scripts) requires 40 lowercase hex characters.
|
||||
const embedded = "388fd0e9b3c41a2d5e6f7a8b9c0d1e2f3a4b5c6d";
|
||||
expect(resolvePackagedCommit(undefined, embedded)).toBe(embedded);
|
||||
expect(resolvePackagedCommit(embedded.toUpperCase())).toBe(embedded);
|
||||
expect(
|
||||
resolvePackagedCommit(` ${embedded.toUpperCase()} `, undefined),
|
||||
).toBe(embedded);
|
||||
expect(resolvePackagedCommit("388fd0e9", embedded)).toBe(embedded);
|
||||
expect(resolvePackagedCommit("388fd0e9")).toBeUndefined();
|
||||
expect(resolvePackagedCommit("not-a-sha", embedded)).toBe(embedded);
|
||||
expect(resolvePackagedCommit(`${embedded}0`, embedded)).toBe(embedded);
|
||||
expect(resolvePackagedCommit("g".repeat(40), embedded)).toBe(embedded);
|
||||
expect(resolvePackagedCommit(undefined, "unknown")).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -143,6 +143,12 @@ export function resolvePackagedBuildNumber(
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* The packaged-runtime install, local install, release download and release
|
||||
* scripts all require a full 40-character lowercase commit. Normalise here so
|
||||
* that one format crosses this boundary: an uppercase full SHA is lowercased,
|
||||
* and a short or malformed value is skipped so it cannot shadow the next source.
|
||||
*/
|
||||
export function resolvePackagedCommit(
|
||||
environmentValue: unknown,
|
||||
embeddedValue?: unknown,
|
||||
@@ -150,7 +156,7 @@ export function resolvePackagedCommit(
|
||||
for (const value of [environmentValue, embeddedValue]) {
|
||||
if (typeof value !== "string") continue;
|
||||
const trimmed = value.trim();
|
||||
if (/^[0-9a-f]{7,40}$/i.test(trimmed)) return trimmed;
|
||||
if (/^[0-9a-f]{40}$/i.test(trimmed)) return trimmed.toLowerCase();
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
@@ -622,9 +622,11 @@ async function initialize(isQuitting: () => boolean): Promise<void> {
|
||||
throw new Error("host_permission_store_unavailable");
|
||||
}
|
||||
if (isQuitting()) return;
|
||||
const hosts = (await recallHosts(app.getPath("home"))).filter(
|
||||
(host) => host.hostId !== "openclaw",
|
||||
);
|
||||
const scanHosts = async () =>
|
||||
(await recallHosts(app.getPath("home"))).filter(
|
||||
(host) => host.hostId !== "openclaw",
|
||||
);
|
||||
const hosts = await scanHosts();
|
||||
readDiagnosticSummary = async () => {
|
||||
const core = coreSupervisor;
|
||||
if (!core) return {};
|
||||
@@ -915,10 +917,14 @@ async function initialize(isQuitting: () => boolean): Promise<void> {
|
||||
});
|
||||
ipcMain.handle("one-memory:recall-hosts", async (event) => {
|
||||
assertAllowedRenderer(event.senderFrame?.url);
|
||||
// A Harness profile can appear at any time, and which ones exist is only
|
||||
// knowable from the disk. In place, so `requireHost` sees it too. One
|
||||
// directory read per list, next to an IPC round trip per host below.
|
||||
hosts.splice(0, hosts.length, ...(await scanHosts()));
|
||||
if (connectionClient?.host) {
|
||||
const inspector = createConnectionClient({ dataDir });
|
||||
const rows: unknown[] = [];
|
||||
for (const host of hosts) {
|
||||
for (const host of [...hosts]) {
|
||||
const inspected = await inspector.host?.({
|
||||
hostId: host.hostId,
|
||||
operation: "inspect",
|
||||
|
||||
@@ -11,8 +11,20 @@ const { resolveBuildNumber, resolveBuildVersion } =
|
||||
};
|
||||
const releaseVersion = resolveBuildVersion(process.env);
|
||||
const releaseBuildNumber = resolveBuildNumber(process.env);
|
||||
const releaseCommit =
|
||||
process.env.ONE_MEMORY_COMMIT?.trim() || readSourceCommit();
|
||||
const releaseCommit = resolveReleaseCommit(process.env.ONE_MEMORY_COMMIT);
|
||||
|
||||
// The runtime accepts only a full 40-character commit, so an explicitly
|
||||
// supplied malformed value fails the build instead of being baked and skipped.
|
||||
function resolveReleaseCommit(explicit: string | undefined): string {
|
||||
const trimmed = explicit?.trim();
|
||||
if (!trimmed) return readSourceCommit();
|
||||
if (!/^[0-9a-f]{40}$/i.test(trimmed)) {
|
||||
throw new Error(
|
||||
`ONE_MEMORY_COMMIT must be a full 40-character hex commit, got ${JSON.stringify(trimmed)}`,
|
||||
);
|
||||
}
|
||||
return trimmed.toLowerCase();
|
||||
}
|
||||
|
||||
function readSourceCommit(): string {
|
||||
try {
|
||||
|
||||
@@ -0,0 +1,185 @@
|
||||
// @vitest-environment jsdom
|
||||
|
||||
/**
|
||||
* #230: DeepSeek Harness has no single host, so the app page used to grey the
|
||||
* switch out with `unsupported_host` and offered no other way to grant recall.
|
||||
* The one switch now covers every detected profile.
|
||||
*/
|
||||
|
||||
import type { SourceResponse } from "@one-memory/contracts";
|
||||
import {
|
||||
cleanup,
|
||||
fireEvent,
|
||||
render,
|
||||
screen,
|
||||
waitFor,
|
||||
within,
|
||||
} from "@testing-library/react";
|
||||
import { afterEach, beforeEach, expect, it, vi } from "vitest";
|
||||
import { ApplicationDetail } from "./ApplicationDetail";
|
||||
import { readConnectionPreferences } from "./connection-preferences";
|
||||
import { resetConnectionReconcile } from "./connection-reconcile";
|
||||
import { harnessBridge } from "./harness-profiles.test-fixture";
|
||||
|
||||
const source = {
|
||||
source_id: "deepseek-harness",
|
||||
display_name: "DeepSeek Harness",
|
||||
authorized: true,
|
||||
sync_enabled: true,
|
||||
capabilities: {
|
||||
historical_import: "implemented",
|
||||
incremental_import: "implemented",
|
||||
live_sync: "implemented",
|
||||
recall: "implemented",
|
||||
installer: "implemented",
|
||||
verification: "real_replayed",
|
||||
},
|
||||
} as SourceResponse;
|
||||
|
||||
function renderDetail() {
|
||||
return render(
|
||||
<ApplicationDetail
|
||||
source={source}
|
||||
sources={[source]}
|
||||
stale={false}
|
||||
setSync={vi.fn()}
|
||||
refresh={vi.fn().mockResolvedValue({ sources: [source] })}
|
||||
onBack={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
}
|
||||
|
||||
function profileItems() {
|
||||
return within(
|
||||
screen.getByRole("list", { name: "DeepSeek Harness 的 Profile" }),
|
||||
)
|
||||
.getAllByRole("listitem")
|
||||
.map((item) => item.textContent);
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
window.localStorage.clear();
|
||||
resetConnectionReconcile();
|
||||
});
|
||||
afterEach(() => {
|
||||
cleanup();
|
||||
window.oneMemory = undefined;
|
||||
window.localStorage.clear();
|
||||
resetConnectionReconcile();
|
||||
});
|
||||
|
||||
it("lists every detected profile with its own state", async () => {
|
||||
harnessBridge({ desktop: "approved", web: "not-installed" });
|
||||
renderDetail();
|
||||
await waitFor(() =>
|
||||
expect(profileItems()).toEqual(["desktop:已接入", "web:未接入"]),
|
||||
);
|
||||
const insertion = screen.getByRole("switch", { name: "一键插入应用" });
|
||||
expect(insertion.hasAttribute("disabled")).toBe(false);
|
||||
expect(screen.queryByText(/当前安装不支持一键插入/)).toBeNull();
|
||||
});
|
||||
|
||||
it("turns every profile on with the one switch and remembers the choice", async () => {
|
||||
const { api, state } = harnessBridge({
|
||||
desktop: "not-installed",
|
||||
web: "not-installed",
|
||||
});
|
||||
renderDetail();
|
||||
const insertion = await screen.findByRole("switch", { name: "一键插入应用" });
|
||||
await waitFor(() => expect(profileItems()).toHaveLength(2));
|
||||
expect(insertion.getAttribute("aria-checked")).toBe("false");
|
||||
fireEvent.click(insertion);
|
||||
await waitFor(() =>
|
||||
expect(profileItems()).toEqual(["desktop:已接入", "web:已接入"]),
|
||||
);
|
||||
expect(api.installRecall.mock.calls.map(([hostId]) => hostId)).toEqual([
|
||||
"deepseek-harness:desktop",
|
||||
"deepseek-harness:web",
|
||||
]);
|
||||
expect([...state.values()]).toEqual(["approved", "approved"]);
|
||||
expect(insertion.getAttribute("aria-checked")).toBe("true");
|
||||
// Not "未接通": the self-checks the grants recorded were read back.
|
||||
await waitFor(() =>
|
||||
expect(
|
||||
screen.getByRole("button", { name: /已接通|未接通|检测中/ }).textContent,
|
||||
).toBe("已接通"),
|
||||
);
|
||||
// Remembered, so a profile found on a later launch is connected too.
|
||||
expect(readConnectionPreferences().hostSourceIds).toEqual([
|
||||
"deepseek-harness",
|
||||
]);
|
||||
});
|
||||
|
||||
it("names the profile that failed and keeps the others on", async () => {
|
||||
const { state } = harnessBridge(
|
||||
{ desktop: "not-installed", web: "not-installed" },
|
||||
{ failInstall: ["web"] },
|
||||
);
|
||||
renderDetail();
|
||||
const insertion = await screen.findByRole("switch", { name: "一键插入应用" });
|
||||
await waitFor(() => expect(profileItems()).toHaveLength(2));
|
||||
fireEvent.click(insertion);
|
||||
const alert = await screen.findByRole("alert");
|
||||
expect(alert.textContent).toContain("以下 Profile 未能接入:web");
|
||||
expect(alert.textContent).toContain("应用控制通道暂不可用");
|
||||
await waitFor(() =>
|
||||
expect(profileItems()).toEqual(["desktop:已接入", "web:未接入"]),
|
||||
);
|
||||
expect(state.get("deepseek-harness:desktop")).toBe("approved");
|
||||
expect(insertion.getAttribute("aria-checked")).toBe("true");
|
||||
});
|
||||
|
||||
it("turns every linked profile off with the one switch", async () => {
|
||||
const { api, state } = harnessBridge({
|
||||
desktop: "approved",
|
||||
web: "approved",
|
||||
});
|
||||
renderDetail();
|
||||
const insertion = await screen.findByRole("switch", { name: "一键插入应用" });
|
||||
await waitFor(() =>
|
||||
expect(insertion.getAttribute("aria-checked")).toBe("true"),
|
||||
);
|
||||
fireEvent.click(insertion);
|
||||
await waitFor(() =>
|
||||
expect(profileItems()).toEqual(["desktop:未接入", "web:未接入"]),
|
||||
);
|
||||
expect(api.uninstallRecall).toHaveBeenCalledTimes(2);
|
||||
expect([...state.values()]).toEqual(["not-installed", "not-installed"]);
|
||||
expect(insertion.getAttribute("aria-checked")).toBe("false");
|
||||
expect(readConnectionPreferences().hostSourceIds).toEqual([]);
|
||||
});
|
||||
|
||||
it("names the profile whose connection could not be removed", async () => {
|
||||
harnessBridge(
|
||||
{ desktop: "approved", web: "approved" },
|
||||
{
|
||||
failUninstall: ["desktop"],
|
||||
},
|
||||
);
|
||||
renderDetail();
|
||||
const insertion = await screen.findByRole("switch", { name: "一键插入应用" });
|
||||
await waitFor(() =>
|
||||
expect(insertion.getAttribute("aria-checked")).toBe("true"),
|
||||
);
|
||||
fireEvent.click(insertion);
|
||||
// Removal failed but revocation went through: the grant is gone, the file
|
||||
// is left for the person to check.
|
||||
expect((await screen.findByRole("alert")).textContent).toContain(
|
||||
"已撤销以下 Profile 的记忆权限,但连接文件没有全部移除,请核对:desktop",
|
||||
);
|
||||
});
|
||||
|
||||
it("says how to get a profile when none is detected", async () => {
|
||||
harnessBridge({});
|
||||
renderDetail();
|
||||
expect(
|
||||
await screen.findByText(
|
||||
"还没有检测到 DeepSeek Harness 的 Profile。创建 Profile 后再回到此页。",
|
||||
),
|
||||
).toBeTruthy();
|
||||
expect(
|
||||
screen
|
||||
.getByRole("switch", { name: "一键插入应用" })
|
||||
.hasAttribute("disabled"),
|
||||
).toBe(true);
|
||||
});
|
||||
@@ -2,15 +2,21 @@ import type { SourceResponse } from "@one-memory/contracts";
|
||||
import type { MessageKey } from "@one-memory/i18n";
|
||||
import { useCallback, useEffect, useRef, useState } from "react";
|
||||
import { AppMark } from "./app-mark";
|
||||
import { connectionHostId, preferenceHasSource } from "./connection-hosts";
|
||||
import {
|
||||
connectionHostId,
|
||||
isProfileSource,
|
||||
preferenceHasSource,
|
||||
profileName,
|
||||
} from "./connection-hosts";
|
||||
import {
|
||||
readConnectionPreferences,
|
||||
writeConnectionPreference,
|
||||
} from "./connection-preferences";
|
||||
import { noteUserHostChoice } from "./connection-reconcile";
|
||||
import type { RecallLinkCheck } from "./global";
|
||||
import type { RecallHostStatus, RecallLinkCheck } from "./global";
|
||||
import {
|
||||
disconnectHost,
|
||||
disconnectProfiles,
|
||||
enableHostConnection,
|
||||
hostCapabilities,
|
||||
hostConnectionApproved,
|
||||
@@ -19,8 +25,11 @@ import {
|
||||
hostWriteReason,
|
||||
matchHost,
|
||||
probeHost,
|
||||
profileHosts,
|
||||
profileWriteFailures,
|
||||
refreshRecallHosts,
|
||||
setHostMemoryPermission,
|
||||
sourceConnectionApproved,
|
||||
UNSUPPORTED_NOTE,
|
||||
useRecallHosts,
|
||||
} from "./host-state";
|
||||
@@ -63,6 +72,24 @@ function connectionLabel(input: {
|
||||
return "未开启关联";
|
||||
}
|
||||
|
||||
/** The recorded self-check across the profiles that may use memory: wired
|
||||
* only when every one of them is, otherwise the first one that is not. */
|
||||
function profilesProbeState(profiles: RecallHostStatus[]) {
|
||||
const linked = profiles.filter(hostConnectionApproved);
|
||||
if (!linked.length) return null;
|
||||
for (const host of linked) {
|
||||
const state = host.selfCheck?.state ?? null;
|
||||
if (state !== "wired") return state;
|
||||
}
|
||||
return "wired";
|
||||
}
|
||||
|
||||
function profileList(hosts: readonly ProfileHost[]) {
|
||||
return hosts.map((host) => profileName(host.hostId)).join(", ");
|
||||
}
|
||||
|
||||
type ProfileHost = Pick<RecallHostStatus, "hostId">;
|
||||
|
||||
function SourceIcon({ source }: { source: SourceResponse }) {
|
||||
return (
|
||||
<AppMark
|
||||
@@ -215,7 +242,10 @@ export function ApplicationDetail({
|
||||
if (!mounted.current || version !== epoch.current) return;
|
||||
setRead({ sourceId: id, state: "ready" });
|
||||
setCheckedAt(new Date().toISOString());
|
||||
if (matched?.installed) void probeLink(matched.hostId);
|
||||
// A per-profile source has no single host to probe; its status comes
|
||||
// from each profile's recorded self-check instead.
|
||||
if (matched?.installed && !isProfileSource(id))
|
||||
void probeLink(matched.hostId);
|
||||
else setLink(null);
|
||||
} catch {
|
||||
if (mounted.current && version === epoch.current)
|
||||
@@ -292,10 +322,14 @@ export function ApplicationDetail({
|
||||
}
|
||||
const current = read.sourceId === source.source_id ? read : null;
|
||||
const host = matchHost(hosts, source.source_id);
|
||||
const profileSource = isProfileSource(source.source_id);
|
||||
const profiles = profileHosts(hosts, source.source_id);
|
||||
const loading = !current || current.state === "loading";
|
||||
const readFailed = current?.state === "error";
|
||||
const approved = hostConnectionApproved(host);
|
||||
const unsupported = hostConnectionUnsupported(host);
|
||||
const approved = profileSource
|
||||
? sourceConnectionApproved(hosts, source.source_id)
|
||||
: hostConnectionApproved(host);
|
||||
const unsupported = !profileSource && hostConnectionUnsupported(host);
|
||||
const insertionChecked = insertionIntent ?? approved;
|
||||
const memoryChecked = memoryIntent ?? host?.permission?.state === "approved";
|
||||
const probing =
|
||||
@@ -305,9 +339,10 @@ export function ApplicationDetail({
|
||||
loading ||
|
||||
readFailed ||
|
||||
(stale && current?.state !== "ready");
|
||||
const probeState =
|
||||
link?.probe.state ??
|
||||
(linkPending || linkFailed ? null : (host?.selfCheck?.state ?? null));
|
||||
const probeState = profileSource
|
||||
? profilesProbeState(profiles)
|
||||
: (link?.probe.state ??
|
||||
(linkPending || linkFailed ? null : (host?.selfCheck?.state ?? null)));
|
||||
const stateText = connectionLabel({
|
||||
approved,
|
||||
insertionOn: insertionChecked,
|
||||
@@ -326,10 +361,43 @@ export function ApplicationDetail({
|
||||
? { reason: failureNote(installError.failure) }
|
||||
: {}),
|
||||
});
|
||||
async function applyProfiles(enabled: boolean) {
|
||||
if (enabled) {
|
||||
try {
|
||||
await enableHostConnection(source.source_id);
|
||||
} finally {
|
||||
// Each grant records its self-check on the desktop, and the status row
|
||||
// reads those; nothing else re-reads them while this page is open.
|
||||
await refreshRecallHosts().catch(() => {});
|
||||
}
|
||||
return;
|
||||
}
|
||||
const results = await disconnectProfiles(source.source_id);
|
||||
const failed = results.filter((item) => item.outcome.state === "failed");
|
||||
const retained = results.filter(
|
||||
(item) =>
|
||||
item.outcome.state === "revoked" || item.outcome.state === "review",
|
||||
);
|
||||
if (retained.length)
|
||||
setInstallError({
|
||||
key: "appProfilesRevokedRetained",
|
||||
params: { profiles: profileList(retained) },
|
||||
});
|
||||
const first = failed[0]?.outcome;
|
||||
if (first?.state === "failed")
|
||||
throw Object.assign(Error("disconnect_failed"), {
|
||||
profiles: failed.map((item) => profileName(item.hostId)),
|
||||
failure: {
|
||||
reason: first.reason,
|
||||
rereadFailed: first.rereadFailed,
|
||||
} satisfies HostFailure,
|
||||
});
|
||||
}
|
||||
async function applyInsertion(enabled: boolean) {
|
||||
const bridge = window.oneMemory;
|
||||
if (!bridge) return;
|
||||
setInstallError("");
|
||||
if (profileSource) return applyProfiles(enabled);
|
||||
if (enabled) {
|
||||
await enableHostConnection(source.source_id);
|
||||
void probeLink(hostId);
|
||||
@@ -385,6 +453,30 @@ export function ApplicationDetail({
|
||||
}
|
||||
// A failed turn-on keeps the stored choice on; a failed turn-off
|
||||
// shows the grant that is really still there after the re-read.
|
||||
// Per-profile sources name the profiles that failed; the others
|
||||
// keep what they got.
|
||||
const failedProfiles =
|
||||
profileWriteFailures(error) ??
|
||||
(error as { profiles?: string[] }).profiles ??
|
||||
null;
|
||||
if (failedProfiles) {
|
||||
setInstallError({
|
||||
key: wanted
|
||||
? "appProfilesConnectFailed"
|
||||
: "appProfilesDisconnectFailed",
|
||||
params: { profiles: failedProfiles.join(", ") },
|
||||
failure: wanted
|
||||
? {
|
||||
reason: hostWriteReason(error),
|
||||
rereadFailed: hostRereadFailed(error),
|
||||
}
|
||||
: ((error as { failure?: HostFailure }).failure ?? {
|
||||
reason: null,
|
||||
rereadFailed: false,
|
||||
}),
|
||||
});
|
||||
continue;
|
||||
}
|
||||
setInstallError(
|
||||
wanted
|
||||
? {
|
||||
@@ -561,6 +653,31 @@ export function ApplicationDetail({
|
||||
<div>
|
||||
<h2>{text("一键插入应用")}</h2>
|
||||
<p>{text("安装记忆连接,并默认允许此应用使用已授权记忆。")}</p>
|
||||
{profileSource &&
|
||||
(profiles.length ? (
|
||||
<>
|
||||
<ul
|
||||
className="r6-app-profiles"
|
||||
aria-label={t("appProfilesLabel", {
|
||||
name: source.display_name,
|
||||
})}
|
||||
>
|
||||
{profiles.map((profile) => (
|
||||
<li key={profile.hostId}>
|
||||
{t(
|
||||
hostConnectionApproved(profile)
|
||||
? "appProfileOn"
|
||||
: "appProfileOff",
|
||||
{ profile: profileName(profile.hostId) },
|
||||
)}
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
<p>{t("appProfilesHint")}</p>
|
||||
</>
|
||||
) : (
|
||||
<p>{t("appProfilesNone", { name: source.display_name })}</p>
|
||||
))}
|
||||
{unsupported && (
|
||||
<p>{`${source.display_name} ${text(UNSUPPORTED_NOTE)}`}</p>
|
||||
)}
|
||||
@@ -578,6 +695,7 @@ export function ApplicationDetail({
|
||||
stale ||
|
||||
appLocationNote !== null ||
|
||||
unsupported ||
|
||||
(profileSource && !profiles.length) ||
|
||||
source.capabilities?.installer !== "implemented" ||
|
||||
source.capabilities?.recall !== "implemented" ||
|
||||
!hostCapabilities().install
|
||||
|
||||
@@ -31,6 +31,8 @@ import {
|
||||
hostBridgeAvailable,
|
||||
hostConnectionApproved,
|
||||
hostConnectionUnknown,
|
||||
sourceConnectionApproved,
|
||||
sourceConnectionIncomplete,
|
||||
useRecallHosts,
|
||||
} from "./host-state";
|
||||
import { subscribePoll } from "./poll-scheduler";
|
||||
@@ -737,12 +739,7 @@ export function ApplicationsHome({
|
||||
failure.kind === "host"
|
||||
? preferences.hostSourceIds.some((id) =>
|
||||
sameConnectionHost(id, failure.sourceId),
|
||||
) &&
|
||||
!hosts.some(
|
||||
(host) =>
|
||||
hostConnectionApproved(host) &&
|
||||
hostMatchesSource(failure.sourceId, host.hostId),
|
||||
)
|
||||
) && sourceConnectionIncomplete(hosts, failure.sourceId)
|
||||
: preferences.syncSourceIds.includes(failure.sourceId) &&
|
||||
!imports.sources.some(
|
||||
(source) =>
|
||||
@@ -754,14 +751,22 @@ export function ApplicationsHome({
|
||||
const autoFailureText = autoFailures
|
||||
.map((failure) =>
|
||||
failure.kind === "host"
|
||||
? t("apps.autoHostFailed", {
|
||||
name: nameOf(failure.sourceId),
|
||||
reason: failure.reason
|
||||
? t("apps.failureReason", {
|
||||
reason: localizeFailureReason(failure.reason),
|
||||
})
|
||||
: "",
|
||||
})
|
||||
? t(
|
||||
failure.profiles
|
||||
? "apps.autoHostProfilesFailed"
|
||||
: "apps.autoHostFailed",
|
||||
{
|
||||
name: nameOf(failure.sourceId),
|
||||
...(failure.profiles
|
||||
? { profiles: failure.profiles.join(", ") }
|
||||
: {}),
|
||||
reason: failure.reason
|
||||
? t("apps.failureReason", {
|
||||
reason: localizeFailureReason(failure.reason),
|
||||
})
|
||||
: "",
|
||||
},
|
||||
)
|
||||
: t("apps.autoSyncFailed", {
|
||||
name: nameOf(failure.sourceId),
|
||||
reason: localizeFailureReason(failure.reason),
|
||||
@@ -1181,8 +1186,9 @@ export function ApplicationsHome({
|
||||
t,
|
||||
callState === "unavailable",
|
||||
) || sourceSyncActivity(source.source_id, syncHealth, locale, t);
|
||||
const host = hosts.find((item) =>
|
||||
sourceMatchesHost(source.source_id, item.hostId),
|
||||
const recallAllowed = sourceConnectionApproved(
|
||||
hosts,
|
||||
source.source_id,
|
||||
);
|
||||
return (
|
||||
<div className="r6-home-app-row" key={source.source_id}>
|
||||
@@ -1193,8 +1199,7 @@ export function ApplicationsHome({
|
||||
<p>
|
||||
{source.sync_enabled
|
||||
? t("apps.syncEnabled")
|
||||
: host?.installed &&
|
||||
host.permission?.state === "approved"
|
||||
: recallAllowed
|
||||
? t("apps.recallAllowed")
|
||||
: source.authorized
|
||||
? t("apps.historyAllowed")
|
||||
|
||||
@@ -4,7 +4,6 @@ import type { GovernedMemory, MemoryDetail } from "@one-memory/contracts";
|
||||
import {
|
||||
act,
|
||||
cleanup,
|
||||
fireEvent,
|
||||
render,
|
||||
screen,
|
||||
waitFor,
|
||||
@@ -75,7 +74,6 @@ const {
|
||||
askMemoriesStream,
|
||||
setExtractionWindow,
|
||||
manualExtract,
|
||||
retryUnknownMemoryProcessing,
|
||||
getGraphWidth,
|
||||
decideGraphWidth,
|
||||
getModelProfiles,
|
||||
@@ -89,7 +87,6 @@ const {
|
||||
askMemoriesStream: vi.fn(),
|
||||
setExtractionWindow: vi.fn(),
|
||||
manualExtract: vi.fn(),
|
||||
retryUnknownMemoryProcessing: vi.fn(),
|
||||
getGraphWidth: vi.fn(),
|
||||
decideGraphWidth: vi.fn(),
|
||||
getModelProfiles: vi.fn(),
|
||||
@@ -107,7 +104,6 @@ vi.mock("./api", () => ({
|
||||
askMemoriesStream,
|
||||
setExtractionWindow,
|
||||
manualExtract,
|
||||
retryUnknownMemoryProcessing,
|
||||
getGraphWidth,
|
||||
decideGraphWidth,
|
||||
getModelProfiles,
|
||||
@@ -159,7 +155,6 @@ describe("MemorySpace", () => {
|
||||
in_flight_count: 0,
|
||||
seconds_per_completion: null,
|
||||
});
|
||||
retryUnknownMemoryProcessing.mockResolvedValue({ requeued: 0 });
|
||||
getGraphWidth.mockResolvedValue({ mismatch: null, memory_count: 1 });
|
||||
decideGraphWidth.mockResolvedValue({
|
||||
archived_path: null,
|
||||
@@ -394,11 +389,9 @@ describe("MemorySpace", () => {
|
||||
).toBeTruthy();
|
||||
});
|
||||
|
||||
it("shows unknown results once and only retries after confirmation", async () => {
|
||||
const confirm = vi
|
||||
.spyOn(window, "confirm")
|
||||
.mockReturnValueOnce(false)
|
||||
.mockReturnValueOnce(true);
|
||||
it("leaves unknown outcomes to Core: no manual retry, no alarm", async () => {
|
||||
// An unknown outcome is a transient "checking" state Core settles by
|
||||
// itself within seconds; there is nothing for the reader to do.
|
||||
getMemoryProcessing.mockResolvedValue({
|
||||
pending: 0,
|
||||
fresh_pending: 0,
|
||||
@@ -420,20 +413,104 @@ describe("MemorySpace", () => {
|
||||
in_flight_count: 0,
|
||||
seconds_per_completion: 20,
|
||||
});
|
||||
retryUnknownMemoryProcessing.mockResolvedValue({ requeued: 17 });
|
||||
render(<MemorySpace />);
|
||||
|
||||
expect(
|
||||
await screen.findByText("有 17 段对话的整理结果不确定"),
|
||||
).toBeTruthy();
|
||||
fireEvent.click(screen.getByRole("button", { name: "重新整理" }));
|
||||
expect(retryUnknownMemoryProcessing).not.toHaveBeenCalled();
|
||||
fireEvent.click(screen.getByRole("button", { name: "重新整理" }));
|
||||
await waitFor(() =>
|
||||
expect(retryUnknownMemoryProcessing).toHaveBeenCalledOnce(),
|
||||
);
|
||||
expect(confirm).toHaveBeenCalledTimes(2);
|
||||
confirm.mockRestore();
|
||||
expect(await screen.findByText("整理状态")).toBeTruthy();
|
||||
expect(screen.queryByText("整理需要处理")).toBeNull();
|
||||
expect(screen.queryByText(/整理结果不确定/)).toBeNull();
|
||||
expect(screen.queryByRole("button", { name: "重新整理" })).toBeNull();
|
||||
});
|
||||
|
||||
describe("模型熔断提示", () => {
|
||||
function backoff(
|
||||
error: string | null,
|
||||
pending = 4,
|
||||
): Record<string, unknown> {
|
||||
return {
|
||||
pending,
|
||||
running: 0,
|
||||
fresh_pending: pending,
|
||||
backlog_pending: 0,
|
||||
recent_completions: [],
|
||||
waiting_for_model: 0,
|
||||
completed: 1,
|
||||
failed: 0,
|
||||
blocked_stages: [],
|
||||
outside_window_pending: 0,
|
||||
extraction_window_preset: "3m",
|
||||
manual_extract_batches_planned: 0,
|
||||
manual_extract_batches_current: 0,
|
||||
active_queue: 0,
|
||||
in_flight_count: 0,
|
||||
seconds_per_completion: null,
|
||||
suspended: 0,
|
||||
llm_backoff_until: error ? "2026-11-02T10:00:00+00:00" : null,
|
||||
llm_backoff_error: error,
|
||||
};
|
||||
}
|
||||
|
||||
it("names the mapped reason and the queued count, and links to the model", async () => {
|
||||
const onConfigureModel = vi.fn();
|
||||
getMemoryProcessing.mockResolvedValue(backoff("model_auth_failed"));
|
||||
render(
|
||||
<MemorySpace
|
||||
onConnect={() => {}}
|
||||
onConfigureModel={onConfigureModel}
|
||||
/>,
|
||||
);
|
||||
|
||||
const banner = await screen.findByRole("status", {
|
||||
name: "模型暂停处理",
|
||||
});
|
||||
expect(banner.textContent).toContain(
|
||||
"模型配置有问题(API Key 无效或没有权限),记忆暂停处理,已有 4 条在排队",
|
||||
);
|
||||
await userEvent.click(
|
||||
within(banner).getByRole("button", { name: "配置整理模型" }),
|
||||
);
|
||||
expect(onConfigureModel).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("reads an unrecognised code as a plain connection failure", async () => {
|
||||
getMemoryProcessing.mockResolvedValue(backoff("model_something_new", 1));
|
||||
render(<MemorySpace onConnect={() => {}} onConfigureModel={() => {}} />);
|
||||
|
||||
const banner = await screen.findByRole("status", {
|
||||
name: "模型暂停处理",
|
||||
});
|
||||
expect(banner.textContent).toContain("(连接模型服务失败)");
|
||||
expect(banner.textContent).toContain("已有 1 条在排队");
|
||||
});
|
||||
|
||||
it("shows nothing while the breaker is closed", async () => {
|
||||
getMemoryProcessing.mockResolvedValue(backoff(null));
|
||||
render(<MemorySpace onConnect={() => {}} onConfigureModel={() => {}} />);
|
||||
|
||||
expect(await screen.findByText("正在整理记忆")).toBeTruthy();
|
||||
expect(screen.queryByRole("status", { name: "模型暂停处理" })).toBeNull();
|
||||
expect(screen.queryByText(/模型配置有问题/)).toBeNull();
|
||||
});
|
||||
|
||||
it("goes away by itself once a refresh clears the pause", async () => {
|
||||
vi.useFakeTimers({ shouldAdvanceTime: true });
|
||||
getMemoryProcessing.mockResolvedValue(backoff("model_timeout"));
|
||||
render(<MemorySpace onConnect={() => {}} onConfigureModel={() => {}} />);
|
||||
|
||||
const banner = await screen.findByRole("status", {
|
||||
name: "模型暂停处理",
|
||||
});
|
||||
expect(banner.textContent).toContain("模型响应超时");
|
||||
|
||||
getMemoryProcessing.mockResolvedValue(backoff(null));
|
||||
await act(async () => {
|
||||
await vi.advanceTimersByTimeAsync(3_500);
|
||||
});
|
||||
await waitFor(() =>
|
||||
expect(
|
||||
screen.queryByRole("status", { name: "模型暂停处理" }),
|
||||
).toBeNull(),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
it("splits the queue into new and catch-up rather than one total", async () => {
|
||||
|
||||
@@ -37,7 +37,6 @@ import {
|
||||
getSdkMemoryGraph,
|
||||
getSdkMemoryLibrary,
|
||||
manualExtract,
|
||||
retryUnknownMemoryProcessing,
|
||||
setExtractionWindow,
|
||||
updateSdkMemory,
|
||||
} from "./api";
|
||||
@@ -46,7 +45,6 @@ import { useGuiLanguage } from "./gui-language";
|
||||
import { MemoryDeleteDialog } from "./MemoryDeleteDialog";
|
||||
import { MemoryMap } from "./MemoryMap";
|
||||
import { subscribePoll } from "./poll-scheduler";
|
||||
import { R6Button } from "./R6Controls";
|
||||
import { SdkMemoryLibrary } from "./SdkMemoryLibrary";
|
||||
import { SdkRawHistory } from "./SdkRawHistory";
|
||||
import {
|
||||
@@ -65,6 +63,7 @@ import type { MemoryArrival } from "./memory-arrival-state";
|
||||
import {
|
||||
blockReasonLabel,
|
||||
extractionWindowPresetLabel,
|
||||
llmBackoffReasonLabel,
|
||||
modelBlock,
|
||||
splitBlockedStages,
|
||||
taskKindLabel,
|
||||
@@ -691,44 +690,15 @@ export function MemorySpace({
|
||||
}
|
||||
}
|
||||
|
||||
async function retryUnknown() {
|
||||
if (
|
||||
busy ||
|
||||
!window.confirm(
|
||||
t(
|
||||
"memory:重新整理可能再次调用模型。只会重跑结果不确定的对话,是否继续?",
|
||||
),
|
||||
)
|
||||
)
|
||||
return;
|
||||
setBusy(true);
|
||||
setError(null);
|
||||
const trace = traceUiAction("memory", "retry");
|
||||
try {
|
||||
const receipt = await retryUnknownMemoryProcessing(trace.traceId);
|
||||
trace.done("succeeded", { count: receipt.requeued });
|
||||
setProcessing(await getMemoryProcessing());
|
||||
} catch (reason) {
|
||||
trace.done(writeTraceStatus(reason), {
|
||||
reason: isDefiniteRefusal(reason)
|
||||
? "memory_unknown_retry_rejected"
|
||||
: "memory_unknown_retry_result_unknown",
|
||||
});
|
||||
setError(message(reason, t));
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
}
|
||||
|
||||
// What the reader owes attention, and what the pipeline is already handling
|
||||
// on its own. The second half is counted, never listed: an internal guard
|
||||
// that clears itself on the next pass is not something to act on. See
|
||||
// `worthNamingBlock`.
|
||||
// `worthNamingBlock`. An unknown outcome is in the second half too: Core
|
||||
// settles it by itself within seconds.
|
||||
const blocked = processing
|
||||
? splitBlockedStages(processing.blocked_stages)
|
||||
: { named: [], retrying: 0 };
|
||||
const processingNeedsAttention =
|
||||
blocked.named.length > 0 || (processing?.unknown ?? 0) > 0;
|
||||
const processingNeedsAttention = blocked.named.length > 0;
|
||||
|
||||
return (
|
||||
<section
|
||||
@@ -933,19 +903,6 @@ export function MemorySpace({
|
||||
</span>
|
||||
) : null}
|
||||
</p>
|
||||
{(processing.unknown ?? 0) > 0 ? (
|
||||
<div className="memory-processing-unknown" role="status">
|
||||
<span>
|
||||
{tn(
|
||||
"memory:有 {count} 段对话的整理结果不确定",
|
||||
processing.unknown ?? 0,
|
||||
)}
|
||||
</span>
|
||||
<R6Button disabled={busy} onClick={() => void retryUnknown()}>
|
||||
{t("memory:重新整理")}
|
||||
</R6Button>
|
||||
</div>
|
||||
) : null}
|
||||
<ExtractionWindowControls
|
||||
processing={processing}
|
||||
busy={busy}
|
||||
@@ -963,6 +920,10 @@ export function MemorySpace({
|
||||
</details>
|
||||
) : null}
|
||||
|
||||
<ModelBackoffBanner
|
||||
processing={processing}
|
||||
onConfigureModel={onConfigureModel}
|
||||
/>
|
||||
<ModelPrompt
|
||||
processing={processing}
|
||||
hasMemories={(sdkLibrary.page?.total ?? 0) > 0}
|
||||
@@ -1232,6 +1193,49 @@ function ModelPrompt({
|
||||
);
|
||||
}
|
||||
|
||||
/** Core's circuit breaker paused model work because the provider keeps
|
||||
* failing (bad key, wrong URL, missing model, provider down).
|
||||
*
|
||||
* Nothing is lost and nothing needs retrying by hand: the queued work stays
|
||||
* pending and Core keeps probing on its own, so this only says what is wrong
|
||||
* and where to fix it. It is driven by the polled summary alone and goes away
|
||||
* on the first read that no longer carries `llm_backoff_error`. */
|
||||
function ModelBackoffBanner({
|
||||
processing,
|
||||
onConfigureModel,
|
||||
}: {
|
||||
processing: MemoryProcessingSummary | null;
|
||||
onConfigureModel(): void;
|
||||
}) {
|
||||
const { t, tn } = useGuiLanguage();
|
||||
const code = processing?.llm_backoff_error;
|
||||
if (!processing || !code) return null;
|
||||
return (
|
||||
<p
|
||||
className="model-prompt"
|
||||
role="status"
|
||||
aria-label={t("memory:模型暂停处理")}
|
||||
>
|
||||
<span>
|
||||
{tn(
|
||||
"memory:模型配置有问题({reason}),记忆暂停处理,已有 {count} 条在排队",
|
||||
processing.pending,
|
||||
{
|
||||
reason: t(
|
||||
`memory:${llmBackoffReasonLabel(code)}` as Parameters<
|
||||
typeof t
|
||||
>[0],
|
||||
),
|
||||
},
|
||||
)}
|
||||
</span>
|
||||
<button type="button" onClick={onConfigureModel}>
|
||||
{t("memory:配置整理模型")}
|
||||
</button>
|
||||
</p>
|
||||
);
|
||||
}
|
||||
|
||||
function MemoryInspector({
|
||||
detail,
|
||||
busy,
|
||||
|
||||
@@ -13,9 +13,18 @@ vi.mock("axios", () => ({
|
||||
delete: del,
|
||||
interceptors: { request: { use: vi.fn() } },
|
||||
}),
|
||||
isAxiosError: (value: unknown) =>
|
||||
typeof value === "object" && value !== null && "isAxiosError" in value,
|
||||
},
|
||||
}));
|
||||
|
||||
function httpError(status: number, detail: string) {
|
||||
return Object.assign(new Error(`http_${status}`), {
|
||||
isAxiosError: true,
|
||||
response: { status, data: { detail } },
|
||||
});
|
||||
}
|
||||
|
||||
import {
|
||||
deleteSdkMemory,
|
||||
getHomeSummary,
|
||||
@@ -100,6 +109,18 @@ describe("paged memory library client", () => {
|
||||
expect(del).toHaveBeenLastCalledWith("/api/v2/memories/c1", undefined);
|
||||
});
|
||||
|
||||
it("treats a delete answered 404 memory_not_found as already gone", async () => {
|
||||
del.mockRejectedValueOnce(httpError(404, "memory_not_found"));
|
||||
await expect(deleteSdkMemory("gone")).resolves.toBeUndefined();
|
||||
|
||||
const other = httpError(404, "source_not_found");
|
||||
del.mockRejectedValueOnce(other);
|
||||
await expect(deleteSdkMemory("x")).rejects.toBe(other);
|
||||
const failed = httpError(503, "hy_memory_delete_failed");
|
||||
del.mockRejectedValueOnce(failed);
|
||||
await expect(deleteSdkMemory("x")).rejects.toBe(failed);
|
||||
});
|
||||
|
||||
it("forwards cancellation for inactive or superseded views", async () => {
|
||||
const controller = new AbortController();
|
||||
get.mockResolvedValueOnce({
|
||||
|
||||
+19
-14
@@ -763,9 +763,25 @@ export async function deleteSdkMemory(
|
||||
...(traceId ? { headers: { "X-One-Memory-Trace-Id": traceId } } : {}),
|
||||
...(params ? { params } : {}),
|
||||
};
|
||||
await client.delete(
|
||||
`/api/v2/memories/${encodeURIComponent(memoryId)}`,
|
||||
Object.keys(config).length ? config : undefined,
|
||||
try {
|
||||
await client.delete(
|
||||
`/api/v2/memories/${encodeURIComponent(memoryId)}`,
|
||||
Object.keys(config).length ? config : undefined,
|
||||
);
|
||||
} catch (reason) {
|
||||
// 404 `memory_not_found`: already gone (a second click, or another
|
||||
// window deleted it first). The memory is not there, which is what the
|
||||
// person asked for, so it is not an error.
|
||||
if (isMemoryNotFound(reason)) return;
|
||||
throw reason;
|
||||
}
|
||||
}
|
||||
|
||||
function isMemoryNotFound(reason: unknown): boolean {
|
||||
return (
|
||||
axios.isAxiosError(reason) &&
|
||||
reason.response?.status === 404 &&
|
||||
apiErrorCode(reason) === "memory_not_found"
|
||||
);
|
||||
}
|
||||
|
||||
@@ -985,17 +1001,6 @@ export async function getModelUsage(
|
||||
return value;
|
||||
}
|
||||
|
||||
export async function retryUnknownMemoryProcessing(
|
||||
traceId?: string,
|
||||
): Promise<{ requeued: number }> {
|
||||
const response = await client.post<{ reconciled: number; requeued: number }>(
|
||||
"/api/v1/memory-processing/retry-unknown",
|
||||
{ confirm: true },
|
||||
traceId ? { headers: { "X-One-Memory-Trace-Id": traceId } } : undefined,
|
||||
);
|
||||
return response.data;
|
||||
}
|
||||
|
||||
export async function getExtractionSettings(): Promise<ExtractionSettings> {
|
||||
const response = await client.get<ExtractionSettings>(
|
||||
"/api/v1/extraction-settings",
|
||||
|
||||
@@ -118,6 +118,14 @@
|
||||
margin: 0;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
.r6-app-profiles {
|
||||
font-size: 13px;
|
||||
color: #637786;
|
||||
line-height: 1.8;
|
||||
margin: 4px 0;
|
||||
padding-left: 18px;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
.r6-app-detail-actions {
|
||||
display: flex;
|
||||
gap: 12px;
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
/** Mirrors desktop `SOURCE_HOST`. OpenClaw and DeepSeek Harness profiles
|
||||
* are not accepted by one-click insertion. */
|
||||
/** Mirrors desktop `SOURCE_HOST`. OpenClaw is not accepted by one-click
|
||||
* insertion; DeepSeek Harness has no single host and goes through
|
||||
* `PROFILE_SOURCES` instead. */
|
||||
export const SOURCE_HOST: Readonly<Record<string, string>> = {
|
||||
cursor: "cursor",
|
||||
"claude-code": "claude-code",
|
||||
@@ -14,6 +15,23 @@ export const SOURCE_HOST: Readonly<Record<string, string>> = {
|
||||
|
||||
export const ONE_CLICK_SOURCES = new Set(Object.keys(SOURCE_HOST));
|
||||
|
||||
/** Sources configured per profile. Each profile is its own host
|
||||
* (`deepseek-harness:<profile>`, with its own patch, credential and grant), so
|
||||
* the app's one switch covers every profile the desktop detected. */
|
||||
export const PROFILE_SOURCES: ReadonlySet<string> = new Set([
|
||||
"deepseek-harness",
|
||||
]);
|
||||
|
||||
export function isProfileSource(sourceId: string) {
|
||||
return PROFILE_SOURCES.has(sourceId);
|
||||
}
|
||||
|
||||
/** `deepseek-harness:web` -> `web`. */
|
||||
export function profileName(hostId: string) {
|
||||
const colon = hostId.indexOf(":");
|
||||
return colon < 0 ? hostId : hostId.slice(colon + 1);
|
||||
}
|
||||
|
||||
export function connectionHostId(sourceId: string): string | null {
|
||||
return SOURCE_HOST[sourceId] ?? null;
|
||||
}
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
import type { SourceResponse } from "@one-memory/contracts";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { connectionHostId, sameConnectionHost } from "./connection-hosts";
|
||||
import {
|
||||
connectionHostId,
|
||||
isProfileSource,
|
||||
sameConnectionHost,
|
||||
} from "./connection-hosts";
|
||||
import {
|
||||
readConnectionPreferences,
|
||||
writeConnectionPreference,
|
||||
@@ -8,11 +12,13 @@ import {
|
||||
import {
|
||||
cachedRecallHosts,
|
||||
enableHostConnection,
|
||||
hostConnectionApproved,
|
||||
hostConnectionUnsupported,
|
||||
hostWriteInFlight,
|
||||
hostWriteReason,
|
||||
matchHost,
|
||||
profileHosts,
|
||||
profileWriteFailures,
|
||||
sourceConnectionIncomplete,
|
||||
} from "./host-state";
|
||||
import { onClientReset, queryClient } from "./query-client";
|
||||
import { latestSource, sourceWriteInFlight } from "./source-state";
|
||||
@@ -37,6 +43,8 @@ export type AutoConnectFailure = {
|
||||
kind: "sync" | "host";
|
||||
sourceId: string;
|
||||
reason: string | null;
|
||||
/** Which profiles of a per-profile source it was; the others are kept. */
|
||||
profiles?: string[];
|
||||
};
|
||||
export type AutoConnectState = {
|
||||
running: string[];
|
||||
@@ -53,8 +61,10 @@ queryClient.setQueryDefaults(AUTO_KEY, {
|
||||
});
|
||||
|
||||
let flight: Promise<AutoConnectFailure[]> | null = null;
|
||||
const seenProfiles = new Map<string, Set<string>>();
|
||||
onClientReset(() => {
|
||||
flight = null;
|
||||
seenProfiles.clear();
|
||||
});
|
||||
|
||||
function autoState() {
|
||||
@@ -148,8 +158,26 @@ export async function reconcileConnectionPreferences(
|
||||
function hostNeedsApply(sourceId: string) {
|
||||
const hosts = cachedRecallHosts();
|
||||
if (hosts === undefined) return false;
|
||||
const host = matchHost(hosts, sourceId);
|
||||
return !hostConnectionApproved(host) && !hostConnectionUnsupported(host);
|
||||
return sourceConnectionIncomplete(hosts, sourceId);
|
||||
}
|
||||
|
||||
/** `attempted` holds one apply per source, which is right until a per-profile
|
||||
* source grows a profile: that one was never tried, so the remembered switch
|
||||
* gets one more apply. A profile that fails is not retried on every poll. */
|
||||
function rearmNewProfiles(sourceIds: string[]) {
|
||||
const hosts = cachedRecallHosts();
|
||||
if (hosts === undefined) return;
|
||||
for (const sourceId of sourceIds) {
|
||||
if (!isProfileSource(sourceId)) continue;
|
||||
const current = profileHosts(hosts, sourceId).map((host) => host.hostId);
|
||||
const seen = seenProfiles.get(sourceId);
|
||||
seenProfiles.set(sourceId, new Set([...(seen ?? []), ...current]));
|
||||
if (!seen || current.every((hostId) => seen.has(hostId))) continue;
|
||||
updateAuto((state) => ({
|
||||
...state,
|
||||
attempted: state.attempted.filter((key) => key !== hostKey(sourceId)),
|
||||
}));
|
||||
}
|
||||
}
|
||||
|
||||
/** Whether a remembered host is still waiting for (or in) its one apply. */
|
||||
@@ -191,6 +219,7 @@ export function autoReconcile(
|
||||
if (hostConnectionUnsupported(matchHost(hosts, id)))
|
||||
writeConnectionPreference("host", id, false);
|
||||
const preferences = readConnectionPreferences();
|
||||
rearmNewProfiles(preferences.hostSourceIds);
|
||||
const { attempted } = autoState();
|
||||
const only = options.only;
|
||||
const wantedHosts = only
|
||||
@@ -256,6 +285,7 @@ export function retryAutoReconcile(options: ReconcileOptions) {
|
||||
|
||||
export function resetConnectionReconcile() {
|
||||
flight = null;
|
||||
seenProfiles.clear();
|
||||
queryClient.removeQueries({ queryKey: AUTO_KEY });
|
||||
}
|
||||
|
||||
@@ -301,12 +331,15 @@ async function runReconcile(
|
||||
// Per-app: other hosts keep going. Unsupported never succeeds on retry.
|
||||
if (error instanceof Error && error.message === "host_unsupported")
|
||||
writeConnectionPreference("host", sourceId, false);
|
||||
else
|
||||
else {
|
||||
const profiles = profileWriteFailures(error);
|
||||
failures.push({
|
||||
kind: "host",
|
||||
sourceId,
|
||||
reason: hostWriteReason(error),
|
||||
...(profiles ? { profiles } : {}),
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
return failures;
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
/** A fake desktop bridge holding DeepSeek Harness profile hosts, for tests. */
|
||||
|
||||
import { vi } from "vitest";
|
||||
import type { RecallHostStatus, RecallPermissionStatus } from "./global";
|
||||
|
||||
type Bridge = NonNullable<typeof window.oneMemory>;
|
||||
type Grant =
|
||||
| "not-installed"
|
||||
| "install-only"
|
||||
| "approved"
|
||||
| "revoked"
|
||||
| "unsupported";
|
||||
|
||||
function permission(hostId: string, state: Grant): RecallPermissionStatus {
|
||||
return {
|
||||
hostId,
|
||||
state,
|
||||
reason: null,
|
||||
receipt:
|
||||
state === "not-installed" || state === "unsupported"
|
||||
? null
|
||||
: {
|
||||
hostId,
|
||||
installationId: `install-${hostId}`,
|
||||
revision: 1,
|
||||
status: state,
|
||||
allowedTools: state === "approved" ? ["om_recall"] : [],
|
||||
allowedScopes: state === "approved" ? ["all-apps"] : [],
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** A desktop that knows a set of Harness profiles and keeps their state. */
|
||||
export function harnessBridge(
|
||||
initial: Record<string, Grant>,
|
||||
options: { failInstall?: string[]; failUninstall?: string[] } = {},
|
||||
) {
|
||||
const state = new Map(
|
||||
Object.entries(initial).map(([profile, grant]) => [
|
||||
`deepseek-harness:${profile}`,
|
||||
grant,
|
||||
]),
|
||||
);
|
||||
const hosts = (): RecallHostStatus[] =>
|
||||
[...state].map(([hostId, grant]) => ({
|
||||
hostId,
|
||||
label: `DeepSeek Harness · ${hostId.split(":")[1]}`,
|
||||
present: true,
|
||||
installed: grant === "install-only" || grant === "approved",
|
||||
permission: permission(hostId, grant),
|
||||
// Recorded by the desktop when the grant lands, as on a real install.
|
||||
selfCheck:
|
||||
grant === "approved"
|
||||
? { state: "wired", detail: "", probed_at: "2026-11-02" }
|
||||
: null,
|
||||
skillDrift: null,
|
||||
}));
|
||||
const failing = (hostId: string, list: string[] = []) =>
|
||||
list.some((profile) => hostId === `deepseek-harness:${profile}`);
|
||||
const api = {
|
||||
getRecallHosts: vi.fn(async () => hosts()),
|
||||
getRecallPermission: vi.fn(async (hostId: string) =>
|
||||
permission(hostId, state.get(hostId) ?? "not-installed"),
|
||||
),
|
||||
previewRecallInstall: vi.fn(async (hostId: string) => ({
|
||||
hostId,
|
||||
hostLabel: hostId,
|
||||
files: [],
|
||||
fingerprint: `fp-${hostId}`,
|
||||
})),
|
||||
installRecall: vi.fn(async (hostId: string, _approved?: string) => {
|
||||
if (failing(hostId, options.failInstall))
|
||||
throw Error(
|
||||
"Error invoking remote method 'one-memory:install-recall': Error: host_control_unavailable",
|
||||
);
|
||||
state.set(hostId, "install-only");
|
||||
return {
|
||||
hostId,
|
||||
installed: true,
|
||||
permission: permission(hostId, "install-only"),
|
||||
};
|
||||
}),
|
||||
setRecallPermission: vi.fn(async (hostId: string, _input: unknown) => {
|
||||
state.set(hostId, "approved");
|
||||
return permission(hostId, "approved");
|
||||
}),
|
||||
revokeRecallPermission: vi.fn(async (hostId: string, _input: unknown) => {
|
||||
state.set(hostId, "revoked");
|
||||
return permission(hostId, "revoked");
|
||||
}),
|
||||
previewRecallUninstall: vi.fn(async (hostId: string) => ({
|
||||
hostId,
|
||||
hostLabel: hostId,
|
||||
files: [],
|
||||
review: [],
|
||||
fingerprint: `fp-${hostId}`,
|
||||
})),
|
||||
uninstallRecall: vi.fn(async (hostId: string, _approved?: string) => {
|
||||
if (failing(hostId, options.failUninstall))
|
||||
throw Error(
|
||||
"Error invoking remote method 'one-memory:uninstall-recall': Error: host_control_unavailable",
|
||||
);
|
||||
state.set(hostId, "not-installed");
|
||||
return {
|
||||
hostId,
|
||||
installed: false,
|
||||
permission: permission(hostId, "not-installed"),
|
||||
};
|
||||
}),
|
||||
repairRecallCommands: vi.fn(async () => []),
|
||||
};
|
||||
window.oneMemory = api as unknown as Bridge;
|
||||
return {
|
||||
api,
|
||||
state,
|
||||
addProfile(profile: string) {
|
||||
state.set(`deepseek-harness:${profile}`, "not-installed");
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,210 @@
|
||||
// @vitest-environment jsdom
|
||||
|
||||
/**
|
||||
* DeepSeek Harness is configured per profile: every `deepseek-harness:<profile>`
|
||||
* is its own host with its own patch, credential and grant. The app page has
|
||||
* one switch for all of them (#230). These pin what that switch does: on
|
||||
* installs and grants every detected profile, off removes and revokes every
|
||||
* linked one, a failing profile is named without undoing the others, and a
|
||||
* profile that appears later is picked up by the remembered switch.
|
||||
*/
|
||||
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { saveConnectionPreferences } from "./connection-preferences";
|
||||
import {
|
||||
autoReconcile,
|
||||
resetConnectionReconcile,
|
||||
} from "./connection-reconcile";
|
||||
import { harnessBridge } from "./harness-profiles.test-fixture";
|
||||
import {
|
||||
disconnectProfiles,
|
||||
enableHostConnection,
|
||||
hostWriteReason,
|
||||
ProfileWriteError,
|
||||
profileHosts,
|
||||
refreshRecallHosts,
|
||||
sourceConnectionApproved,
|
||||
sourceConnectionIncomplete,
|
||||
} from "./host-state";
|
||||
|
||||
afterEach(() => {
|
||||
window.oneMemory = undefined;
|
||||
window.localStorage.clear();
|
||||
resetConnectionReconcile();
|
||||
});
|
||||
|
||||
describe("DeepSeek Harness profile hosts", () => {
|
||||
it("lists only the profiles of the per-profile source", async () => {
|
||||
harnessBridge({ desktop: "not-installed", web: "approved" });
|
||||
const hosts = await refreshRecallHosts();
|
||||
expect(
|
||||
profileHosts(hosts, "deepseek-harness").map((host) => host.hostId),
|
||||
).toEqual(["deepseek-harness:desktop", "deepseek-harness:web"]);
|
||||
expect(profileHosts(hosts, "cursor")).toEqual([]);
|
||||
// One approved profile reads as on; the other is still to do.
|
||||
expect(sourceConnectionApproved(hosts, "deepseek-harness")).toBe(true);
|
||||
expect(sourceConnectionIncomplete(hosts, "deepseek-harness")).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("turning the DeepSeek Harness switch on", () => {
|
||||
it("installs and grants every profile that is not approved yet", async () => {
|
||||
const { api, state } = harnessBridge({
|
||||
desktop: "not-installed",
|
||||
headless: "revoked",
|
||||
web: "approved",
|
||||
});
|
||||
await refreshRecallHosts();
|
||||
await enableHostConnection("deepseek-harness");
|
||||
expect(api.installRecall.mock.calls.map(([hostId]) => hostId)).toEqual([
|
||||
"deepseek-harness:desktop",
|
||||
"deepseek-harness:headless",
|
||||
]);
|
||||
expect(api.setRecallPermission).toHaveBeenCalledWith(
|
||||
"deepseek-harness:desktop",
|
||||
expect.objectContaining({
|
||||
allowedTools: ["om_recall", "om_conversations", "om_diagnostic_trace"],
|
||||
allowedScopes: ["all-apps", "source:deepseek-harness"],
|
||||
userConfirmed: true,
|
||||
}),
|
||||
);
|
||||
expect([...state.values()]).toEqual(["approved", "approved", "approved"]);
|
||||
});
|
||||
|
||||
it("keeps the profiles that worked and names the one that failed", async () => {
|
||||
const { api, state } = harnessBridge(
|
||||
{ desktop: "not-installed", web: "not-installed" },
|
||||
{ failInstall: ["web"] },
|
||||
);
|
||||
await refreshRecallHosts();
|
||||
const error = await enableHostConnection("deepseek-harness").catch(
|
||||
(reason: unknown) => reason,
|
||||
);
|
||||
expect(error).toBeInstanceOf(ProfileWriteError);
|
||||
expect((error as ProfileWriteError).profiles).toEqual(["web"]);
|
||||
expect((error as Error).message).toContain("host_control_unavailable");
|
||||
expect(state.get("deepseek-harness:desktop")).toBe("approved");
|
||||
expect(state.get("deepseek-harness:web")).toBe("not-installed");
|
||||
expect(api.setRecallPermission).toHaveBeenCalledTimes(1);
|
||||
// The failure re-read lands the real state of every profile.
|
||||
const hosts = await refreshRecallHosts();
|
||||
expect(sourceConnectionApproved(hosts, "deepseek-harness")).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("turning the DeepSeek Harness switch off", () => {
|
||||
it("removes and revokes every linked profile and leaves unlinked ones alone", async () => {
|
||||
const { api, state } = harnessBridge({
|
||||
desktop: "approved",
|
||||
headless: "not-installed",
|
||||
web: "approved",
|
||||
});
|
||||
await refreshRecallHosts();
|
||||
const results = await disconnectProfiles("deepseek-harness");
|
||||
expect(results.map((item) => [item.hostId, item.outcome.state])).toEqual([
|
||||
["deepseek-harness:desktop", "removed"],
|
||||
["deepseek-harness:web", "removed"],
|
||||
]);
|
||||
expect(api.uninstallRecall).toHaveBeenCalledTimes(2);
|
||||
expect(state.get("deepseek-harness:desktop")).toBe("not-installed");
|
||||
expect(state.get("deepseek-harness:web")).toBe("not-installed");
|
||||
});
|
||||
|
||||
it("revokes a profile whose files cannot be removed and still reaches the rest", async () => {
|
||||
const { api, state } = harnessBridge(
|
||||
{ desktop: "approved", web: "approved" },
|
||||
{ failUninstall: ["desktop"] },
|
||||
);
|
||||
await refreshRecallHosts();
|
||||
const results = await disconnectProfiles("deepseek-harness");
|
||||
expect(results.map((item) => item.outcome.state)).toEqual([
|
||||
"revoked",
|
||||
"removed",
|
||||
]);
|
||||
expect(api.revokeRecallPermission).toHaveBeenCalledWith(
|
||||
"deepseek-harness:desktop",
|
||||
expect.objectContaining({ userConfirmed: true }),
|
||||
);
|
||||
expect(state.get("deepseek-harness:desktop")).toBe("revoked");
|
||||
expect(state.get("deepseek-harness:web")).toBe("not-installed");
|
||||
});
|
||||
});
|
||||
|
||||
describe("a profile created after the switch was turned on", () => {
|
||||
it("is installed and granted by the remembered switch while the app is running", async () => {
|
||||
saveConnectionPreferences({
|
||||
syncSourceIds: [],
|
||||
hostSourceIds: ["deepseek-harness"],
|
||||
});
|
||||
const harness = harnessBridge({ desktop: "not-installed" });
|
||||
await refreshRecallHosts();
|
||||
expect(await autoReconcile({ sources: [] })).toEqual([]);
|
||||
expect(harness.api.installRecall).toHaveBeenCalledTimes(1);
|
||||
|
||||
// One attempt was already spent on this source; the new profile gets its own.
|
||||
harness.addProfile("web");
|
||||
await refreshRecallHosts();
|
||||
expect(await autoReconcile({ sources: [] })).toEqual([]);
|
||||
expect(harness.api.installRecall.mock.calls.at(-1)).toEqual([
|
||||
"deepseek-harness:web",
|
||||
"fp-deepseek-harness:web",
|
||||
]);
|
||||
expect(harness.state.get("deepseek-harness:web")).toBe("approved");
|
||||
});
|
||||
|
||||
it("does not retry a failing profile on every poll", async () => {
|
||||
saveConnectionPreferences({
|
||||
syncSourceIds: [],
|
||||
hostSourceIds: ["deepseek-harness"],
|
||||
});
|
||||
const harness = harnessBridge(
|
||||
{ desktop: "approved", web: "not-installed" },
|
||||
{ failInstall: ["web"] },
|
||||
);
|
||||
await refreshRecallHosts();
|
||||
expect(await autoReconcile({ sources: [] })).toEqual([
|
||||
{
|
||||
kind: "host",
|
||||
sourceId: "deepseek-harness",
|
||||
reason: "应用控制通道暂不可用",
|
||||
profiles: ["web"],
|
||||
},
|
||||
]);
|
||||
await refreshRecallHosts();
|
||||
expect(await autoReconcile({ sources: [] })).toBeNull();
|
||||
expect(harness.api.installRecall).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("stays out while the switch is off", async () => {
|
||||
const harness = harnessBridge({ desktop: "not-installed" });
|
||||
await refreshRecallHosts();
|
||||
expect(await autoReconcile({ sources: [] })).toBeNull();
|
||||
harness.addProfile("web");
|
||||
await refreshRecallHosts();
|
||||
expect(await autoReconcile({ sources: [] })).toBeNull();
|
||||
expect(harness.api.installRecall).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe("a profile whose name MemHub cannot grant", () => {
|
||||
it("is named with the real reason and does not hold up the others", async () => {
|
||||
const { state } = harnessBridge({
|
||||
My_Profile: "unsupported",
|
||||
web: "not-installed",
|
||||
});
|
||||
await refreshRecallHosts();
|
||||
const error = await enableHostConnection("deepseek-harness").catch(
|
||||
(reason: unknown) => reason,
|
||||
);
|
||||
expect((error as ProfileWriteError).profiles).toEqual(["My_Profile"]);
|
||||
expect(hostWriteReason(error)).toBe("appProfileNameUnsupported");
|
||||
expect(state.get("deepseek-harness:web")).toBe("approved");
|
||||
// Nothing is left for the remembered switch to keep trying.
|
||||
expect(
|
||||
sourceConnectionIncomplete(
|
||||
await refreshRecallHosts(),
|
||||
"deepseek-harness",
|
||||
),
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -1,5 +1,10 @@
|
||||
import { MutationObserver, useQuery } from "@tanstack/react-query";
|
||||
import { connectionHostId, hostMatchesSource } from "./connection-hosts";
|
||||
import {
|
||||
connectionHostId,
|
||||
hostMatchesSource,
|
||||
isProfileSource,
|
||||
profileName,
|
||||
} from "./connection-hosts";
|
||||
import type {
|
||||
RecallHostStatus,
|
||||
RecallPermissionStatus,
|
||||
@@ -226,6 +231,7 @@ export function matchHost(
|
||||
|
||||
const REASONS: Record<string, string> = {
|
||||
host_unsupported: "它没有 MCP 通道,无法发放可撤销的记忆凭证",
|
||||
profile_name_unsupported: "appProfileNameUnsupported",
|
||||
unsupported_host: "当前安装不支持一键插入",
|
||||
connection_client_unavailable: "桌面端的连接组件不可用",
|
||||
noncanonical_data_dir_host_integration_disabled: "测试实例不接入应用",
|
||||
@@ -350,18 +356,77 @@ export function hostWriteInFlight() {
|
||||
return queryClient.isMutating({ mutationKey: HOST_WRITE }) > 0;
|
||||
}
|
||||
|
||||
/** A per-profile source writes to every profile, so it waits on each of them. */
|
||||
function sourceHost(sourceId: string) {
|
||||
if (isProfileSource(sourceId))
|
||||
return [
|
||||
sourceId,
|
||||
...profileHosts(cachedRecallHosts(), sourceId).map((host) => host.hostId),
|
||||
];
|
||||
return [connectionHostId(sourceId) ?? sourceId];
|
||||
}
|
||||
|
||||
/** Every detected profile host of a per-profile source, e.g. DeepSeek Harness. */
|
||||
export function profileHosts(
|
||||
hosts: readonly RecallHostStatus[] | undefined,
|
||||
sourceId: string,
|
||||
) {
|
||||
if (!isProfileSource(sourceId)) return [];
|
||||
return (hosts ?? []).filter((host) => host.hostId.startsWith(`${sourceId}:`));
|
||||
}
|
||||
|
||||
/** The app switch reads as on: its host, or for a per-profile source any
|
||||
* profile, may use memory. A failed profile is reported, not hidden. */
|
||||
export function sourceConnectionApproved(
|
||||
hosts: readonly RecallHostStatus[] | undefined,
|
||||
sourceId: string,
|
||||
) {
|
||||
if (isProfileSource(sourceId))
|
||||
return profileHosts(hosts, sourceId).some(hostConnectionApproved);
|
||||
return hostConnectionApproved(matchHost(hosts, sourceId));
|
||||
}
|
||||
|
||||
/** Something the switch would still do: a host, or any profile, that is
|
||||
* neither approved nor unsupported. */
|
||||
export function sourceConnectionIncomplete(
|
||||
hosts: readonly RecallHostStatus[] | undefined,
|
||||
sourceId: string,
|
||||
) {
|
||||
const pending = (host: RecallHostStatus | null) =>
|
||||
!hostConnectionApproved(host) && !hostConnectionUnsupported(host);
|
||||
if (isProfileSource(sourceId))
|
||||
return profileHosts(hosts, sourceId).some(pending);
|
||||
return pending(matchHost(hosts, sourceId));
|
||||
}
|
||||
|
||||
/** Some profiles of a per-profile source could not be written; the others
|
||||
* were kept. The message keeps the first cause so `hostWriteReason` names it. */
|
||||
export class ProfileWriteError extends Error {
|
||||
readonly profiles: string[];
|
||||
constructor(profiles: string[], cause: unknown) {
|
||||
super(
|
||||
`profile_write_failed ${cause instanceof Error ? cause.message : ""}`.trim(),
|
||||
);
|
||||
this.profiles = profiles;
|
||||
}
|
||||
}
|
||||
|
||||
/** The profile names a failed write left behind, if it was a profile write. */
|
||||
export function profileWriteFailures(error: unknown): string[] | null {
|
||||
return error instanceof ProfileWriteError ? error.profiles : null;
|
||||
}
|
||||
|
||||
type Connected = {
|
||||
hostId: string;
|
||||
installed: boolean;
|
||||
permission?: RecallPermissionStatus;
|
||||
};
|
||||
|
||||
async function connect(sourceId: string): Promise<Connected | null> {
|
||||
const hostId = connectionHostId(sourceId);
|
||||
type Bridge = NonNullable<ReturnType<typeof bridge>>;
|
||||
|
||||
async function connect(sourceId: string): Promise<Connected[]> {
|
||||
const profiles = isProfileSource(sourceId);
|
||||
const hostId = profiles ? sourceId : connectionHostId(sourceId);
|
||||
const api = bridge();
|
||||
if (
|
||||
!hostId ||
|
||||
@@ -371,12 +436,65 @@ async function connect(sourceId: string): Promise<Connected | null> {
|
||||
!api.setRecallPermission
|
||||
)
|
||||
throw Error("unsupported_host");
|
||||
const existing = matchHost(await refreshRecallHosts(), sourceId);
|
||||
if (hostConnectionUnsupported(existing)) throw Error("host_unsupported");
|
||||
if (hostConnectionApproved(existing)) {
|
||||
await api.repairRecallCommands?.();
|
||||
return null;
|
||||
const hosts = await refreshRecallHosts();
|
||||
if (profiles) return connectProfiles(api, sourceId, hosts);
|
||||
const result = await grantHost(
|
||||
api,
|
||||
hostId,
|
||||
sourceId,
|
||||
matchHost(hosts, sourceId),
|
||||
);
|
||||
await api.repairRecallCommands?.();
|
||||
return result ? [result] : [];
|
||||
}
|
||||
|
||||
/** One switch, every detected profile. A profile that fails does not undo
|
||||
* the ones that worked; the error names it. Profiles already approved are
|
||||
* left alone, so turning the switch on again only reaches what is missing. */
|
||||
async function connectProfiles(
|
||||
api: Bridge,
|
||||
sourceId: string,
|
||||
hosts: RecallHostStatus[],
|
||||
): Promise<Connected[]> {
|
||||
const profiles = profileHosts(hosts, sourceId);
|
||||
if (!profiles.length) throw Error("profiles_not_found");
|
||||
const connected: Connected[] = [];
|
||||
const failed: string[] = [];
|
||||
let cause: unknown;
|
||||
for (const host of profiles) {
|
||||
try {
|
||||
const result = await grantHost(api, host.hostId, sourceId, host);
|
||||
if (result) connected.push(result);
|
||||
} catch (error) {
|
||||
failed.push(profileName(host.hostId));
|
||||
// Harness speaks MCP, so "unsupported" can only mean the id grammar
|
||||
// refused this name; "no MCP channel" would send the person astray.
|
||||
cause ??= hostConnectionUnsupported(host)
|
||||
? Error("profile_name_unsupported")
|
||||
: error;
|
||||
}
|
||||
}
|
||||
if (failed.length < profiles.length) await api.repairRecallCommands?.();
|
||||
if (failed.length) throw new ProfileWriteError(failed, cause);
|
||||
return connected;
|
||||
}
|
||||
|
||||
/** Installs and grants one host; null when it was already approved. */
|
||||
async function grantHost(
|
||||
api: Bridge,
|
||||
hostId: string,
|
||||
sourceId: string,
|
||||
existing: RecallHostStatus | null,
|
||||
): Promise<Connected | null> {
|
||||
if (
|
||||
!api.previewRecallInstall ||
|
||||
!api.installRecall ||
|
||||
!api.getRecallPermission ||
|
||||
!api.setRecallPermission
|
||||
)
|
||||
throw Error("unsupported_host");
|
||||
if (hostConnectionUnsupported(existing)) throw Error("host_unsupported");
|
||||
if (hostConnectionApproved(existing)) return null;
|
||||
const preview = await api.previewRecallInstall(hostId);
|
||||
const installed = await api.installRecall(hostId, preview.fingerprint);
|
||||
let permission =
|
||||
@@ -394,19 +512,19 @@ async function connect(sourceId: string): Promise<Connected | null> {
|
||||
}
|
||||
if (permission.receipt?.status !== "approved")
|
||||
throw Error("permission_not_approved");
|
||||
await api.repairRecallCommands?.();
|
||||
return { hostId, installed: true, permission };
|
||||
}
|
||||
|
||||
function landConnected(result: Connected | null) {
|
||||
if (result)
|
||||
function landConnected(results: Connected[]) {
|
||||
for (const result of results)
|
||||
landHost(result.hostId, {
|
||||
installed: result.installed,
|
||||
permission: result.permission,
|
||||
});
|
||||
}
|
||||
|
||||
/** Installs and grants one host; resolves only once the grant is approved. */
|
||||
/** Installs and grants one host (or every profile of a per-profile source);
|
||||
* resolves only once the grants are approved. */
|
||||
export const enableHostConnection = hostWrite(
|
||||
sourceHost,
|
||||
connect,
|
||||
@@ -479,6 +597,22 @@ async function disconnect(hostId: string): Promise<Disconnected> {
|
||||
}
|
||||
}
|
||||
|
||||
function landDisconnected({
|
||||
hostId,
|
||||
outcome,
|
||||
}: {
|
||||
hostId: string;
|
||||
outcome: DisconnectOutcome;
|
||||
}) {
|
||||
if (outcome.state === "removed")
|
||||
landHost(hostId, {
|
||||
installed: false,
|
||||
...(outcome.permission ? { permission: outcome.permission } : {}),
|
||||
});
|
||||
else if (outcome.state !== "failed" && outcome.permission)
|
||||
landHost(hostId, { permission: outcome.permission });
|
||||
}
|
||||
|
||||
export const disconnectHost = hostWrite(
|
||||
(hostId: string) => [hostId],
|
||||
async (
|
||||
@@ -489,14 +623,48 @@ export const disconnectHost = hostWrite(
|
||||
const rereadFailed = !(await rereadOnce());
|
||||
return { hostId, outcome: { ...result, rereadFailed } };
|
||||
},
|
||||
({ hostId, outcome }) => {
|
||||
if (outcome.state === "removed")
|
||||
landHost(hostId, {
|
||||
installed: false,
|
||||
...(outcome.permission ? { permission: outcome.permission } : {}),
|
||||
landDisconnected,
|
||||
);
|
||||
|
||||
export type ProfileDisconnect = { hostId: string; outcome: DisconnectOutcome };
|
||||
|
||||
/** Linked to MemHub at all: installed, or holding a receipt that is not yet
|
||||
* revoked. A profile that never was has nothing to remove. */
|
||||
function profileLinked(host: RecallHostStatus) {
|
||||
const status = host.permission?.receipt?.status;
|
||||
return host.installed || (status !== undefined && status !== "revoked");
|
||||
}
|
||||
|
||||
/** Turn a per-profile source off: each linked profile is removed and its grant
|
||||
* revoked, exactly as one host would be. One failure does not stop the rest;
|
||||
* every outcome is returned so the page can name the profile it concerns. */
|
||||
export const disconnectProfiles = hostWrite(
|
||||
sourceHost,
|
||||
async (sourceId: string): Promise<ProfileDisconnect[]> => {
|
||||
const profiles = profileHosts(await refreshRecallHosts(), sourceId).filter(
|
||||
profileLinked,
|
||||
);
|
||||
const results: ProfileDisconnect[] = [];
|
||||
let anyFailed = false;
|
||||
for (const host of profiles) {
|
||||
const outcome = await disconnect(host.hostId);
|
||||
anyFailed ||= outcome.state === "failed";
|
||||
results.push({
|
||||
hostId: host.hostId,
|
||||
outcome:
|
||||
outcome.state === "failed"
|
||||
? { ...outcome, rereadFailed: false }
|
||||
: outcome,
|
||||
});
|
||||
else if (outcome.state !== "failed" && outcome.permission)
|
||||
landHost(hostId, { permission: outcome.permission });
|
||||
}
|
||||
if (anyFailed && !(await rereadOnce()))
|
||||
for (const result of results)
|
||||
if (result.outcome.state === "failed")
|
||||
result.outcome = { ...result.outcome, rereadFailed: true };
|
||||
return results;
|
||||
},
|
||||
(results) => {
|
||||
for (const result of results) landDisconnected(result);
|
||||
},
|
||||
);
|
||||
|
||||
|
||||
@@ -165,6 +165,25 @@ export function modelBlock(
|
||||
);
|
||||
}
|
||||
|
||||
/** Why Core's circuit breaker paused model work, from `llm_backoff_error`.
|
||||
*
|
||||
* Said as what is wrong with the model, never as the code: the reader's next
|
||||
* step is the model settings, and the code alone does not say which part of
|
||||
* them to check. Unknown codes read as a plain connection failure. */
|
||||
export function llmBackoffReasonLabel(code: string): string {
|
||||
return LLM_BACKOFF_REASON_LABELS[code] ?? "连接模型服务失败";
|
||||
}
|
||||
|
||||
const LLM_BACKOFF_REASON_LABELS: Record<string, string> = {
|
||||
model_auth_failed: "API Key 无效或没有权限",
|
||||
model_not_found: "找不到模型或地址",
|
||||
model_rate_limited: "请求过于频繁,被服务商限流",
|
||||
model_timeout: "模型响应超时",
|
||||
model_server_error: "模型服务出错",
|
||||
model_unreachable: "无法连接模型服务",
|
||||
model_transport_failed: "连接模型服务失败",
|
||||
};
|
||||
|
||||
/** What went wrong reading a source, said without the code.
|
||||
*
|
||||
* These are `ScanIssueCode` plus the one access-level refusal, and they land
|
||||
|
||||
@@ -91,7 +91,14 @@ describe("single state layer", () => {
|
||||
});
|
||||
|
||||
it("reads and writes host connections only through host-state", () => {
|
||||
expect(offenders(hostUses, ["host-state.ts", "global.d.ts"])).toEqual([]);
|
||||
expect(
|
||||
offenders(hostUses, [
|
||||
"host-state.ts",
|
||||
"global.d.ts",
|
||||
// Test support: it is the fake bridge, and only tests import it.
|
||||
"harness-profiles.test-fixture.ts",
|
||||
]),
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it("reads and writes sources only through source-state", () => {
|
||||
|
||||
@@ -88,7 +88,6 @@ describe("presentation adapters", () => {
|
||||
"批量处理结果已保留,但列表刷新失败,请重新读取记忆库。",
|
||||
"批量处理结果已保留,但列表刷新失败,稍后会自动重新读取。",
|
||||
"召回范围已保存,但列表刷新失败。请稍后重新打开记忆页。",
|
||||
"重新整理可能再次调用模型。只会重跑结果不确定的对话,是否继续?",
|
||||
"原始对话暂时无法读取",
|
||||
"没有匹配的原始对话",
|
||||
"还没有导入的原始对话",
|
||||
|
||||
@@ -282,7 +282,7 @@ Support is declared per immutable GitHub Release manifest:
|
||||
- Linux x64/arm64 Runtime on native GHA runners.
|
||||
- macOS arm64/x64 App/Runtime: Developer ID, notarization, native acceptance.
|
||||
- Windows is not supported.
|
||||
- `hy-memory==2.0.11` is the current stable SDK dependency; the release
|
||||
- `hy-memory==2.0.12` is the current stable SDK dependency; the release
|
||||
lock and [`../MEMORY.md`](../MEMORY.md) are authoritative when the pin changes.
|
||||
|
||||
See [`INSTALLATION.md`](INSTALLATION.md).
|
||||
|
||||
@@ -69,8 +69,14 @@ node apps/cli/dist/one-memory.cjs sources list
|
||||
```
|
||||
|
||||
- 只有 `hermes` 能带 `--user-id`(非 `local`)或 `--root`;其他来源带了会退 2,什么都不发。
|
||||
- 用户 ID 不必等于 profile 名:profile 名里有 `:` 等字符时另起一个,例如
|
||||
`--user-id wife-work --root ~/.hermes/profiles/wife:work`。
|
||||
- `--root` 里的 `~` 由 CLI 展开为主目录,相对路径按当前目录解析成绝对路径。
|
||||
- 扫描 `~/.hermes` 时会排除 `profiles/**`,所以 `local` 不会读到 wife 的 `state.db`。
|
||||
- 扫描时任何层级名为 `profiles` 的目录都会被排除,所以 `local` 不会读到 wife 的 `state.db`,
|
||||
即使授权的根在 `~/.hermes` 之上。
|
||||
- `--root` 不能是 `profiles` 目录本身(`hermes_root_is_profiles_container`),也不能与另一个
|
||||
用户已接入的 Hermes 目录相同、包含它或在它之内(`hermes_root_overlaps_other_user`,解析
|
||||
symlink 后比较);`~/.hermes` 给 `local`、`~/.hermes/profiles/<name>` 给另一个用户是允许的。
|
||||
- App 不变:它只读写 `local`,其他来源也都固定是 `local`。
|
||||
|
||||
## 3. 看导入进度
|
||||
@@ -140,7 +146,11 @@ node apps/cli/dist/one-memory.cjs write status <request_id> --user-id wife
|
||||
|
||||
- 用户 ID 只能是 1–64 位字母、数字、`.`、`_`、`-`;其他写法退 2,不连 Runtime。
|
||||
- 不带 `--user-id`(或写 `--user-id local`)时请求里不出现 `user_id`,和以前完全一样。
|
||||
- 用户之间互相隔离:读、改、删另一个用户的记忆 ID 返回「未找到」;write 的 request id 也只属于提交它的用户。
|
||||
- 用户之间互相隔离:读、改、删另一个用户的记忆 ID 返回「未找到」(HTTP 404 `memory_not_found`,
|
||||
与不存在的 ID 完全一样);write 的 request id 也只属于提交它的用户。
|
||||
- `memories delete <id> --yes` 遇到「未找到」时按已删除处理:提示「未找到记忆 <id>(可能已删除)」,
|
||||
退出 0;`--json` 为 `state: "committed"` 加 `already_gone: true`。App 里重复点删除也不再报错。
|
||||
- 读接口(`memories list`、`profile`、`recall` 等背后的 HTTP 响应)都带 `user_id`,即实际读的是哪个用户。
|
||||
|
||||
### 画像版本(`profile --versions`)
|
||||
|
||||
@@ -177,7 +187,7 @@ node apps/cli/dist/one-memory.cjs settings features reset
|
||||
```
|
||||
|
||||
- 全局设置,不分用户;没设置的功能保持 SDK 默认值,什么都不设时与以前完全一样(App 不变)。
|
||||
- 可设:`graph_rerank`(图谱重排,仅 ultra)、`same_series_timeline`(需 graph_rerank)、`episode_recall`(片段召回,pro 即可)、`session_episodes`(片段写入)、`episode_summary`(片段摘要,每个片段一次模型调用)、`intention`(默认开)、`few_shot`。
|
||||
- 可设:`graph_rerank`(图谱重排,仅 ultra)、`same_series_timeline`(需 graph_rerank)、`episode_recall`(片段召回,pro 即可)、`session_episodes`(片段写入)、`episode_summary`(片段摘要,每个片段一次模型调用)、`intention`(默认开)、`intention_dedup`(意图判重,默认开;关闭后检索和写入都不判重、不删除意图)、`few_shot`;以及数值 `intention_dedup_threshold`(意图判重的余弦阈值,默认 0.95,可设 0.8–1.0,例如 `settings features set intention_dedup_threshold=0.9 --yes`)。
|
||||
- 片段召回只对带片段的记忆有用:同时开 `session_episodes`,之后提取的记忆才按会话归入片段。
|
||||
- 在当前模式下不生效的功能照样保存,并注明原因(如 `graph_rerank` 在 pro 下「需要 ultra」)。
|
||||
- 修改会重建 SDK 客户端:正在写的一批按旧设置写完,之后的写入和检索用新设置;不迁移数据,不改集合名。
|
||||
@@ -189,14 +199,14 @@ node apps/cli/dist/one-memory.cjs settings features reset
|
||||
```bash
|
||||
node apps/cli/dist/one-memory.cjs settings language # 界面语言和 SDK 提示语言
|
||||
node apps/cli/dist/one-memory.cjs settings language ja # 设为日语,立即生效
|
||||
node apps/cli/dist/one-memory.cjs settings language system # 跟随系统(不指定 SDK 语言)
|
||||
node apps/cli/dist/one-memory.cjs settings language system # 清除设置(不指定 SDK 语言)
|
||||
```
|
||||
|
||||
- 可选 11 种:`en`、`zh-Hans`、`zh-Hant`、`ja`、`es`、`de`、`pt`、`fr`、`ko`、`ru`、`id`,以及 `system`;旧代码 `zh-CN` 按 `zh-Hans` 处理。
|
||||
- App 切换语言时会自动设置(Desktop 启动时也会同步一次已保存的选择)。全局设置,存于 `config.json` 的 `ui_language`;从未选择时不写入。
|
||||
- `zh-Hans` 和 `system` 不向 SDK 传语言,SDK 照旧自动识别中文或英文;其余语言(包括 `en`)作为 `MemoryConfig.language` 传给 SDK,提取等提示使用该语言。
|
||||
- App 切换语言时会自动设置;Desktop 每次启动也会同步一次界面实际显示的语言(App 选“跟随系统”时,发送它按系统语言解析出的语言,例如 `zh-Hans`)。全局设置,存于 `config.json` 的 `ui_language`;从未设置时不写入。
|
||||
- 所有语言(包括 `zh-Hans` 和 `en`)都作为 `MemoryConfig.language` 传给 SDK,提取等提示使用该语言;只有未设置(`system`,或只用 CLI/API、从未打开 App 的安装)时不向 SDK 传语言,SDK 照旧自动识别中文或英文。
|
||||
- 修改会像功能开关一样重建 SDK 客户端;不迁移数据,不改集合名(需要 hy-memory 2.0.11 的 `MemoryConfig.language`)。
|
||||
- `status --json` 的 `pipeline.ui_language` / `pipeline.sdk_language`(`auto` 或语言代码)。API:`GET`/`PUT /api/v1/settings/ui-language`。
|
||||
- `status --json` 的 `pipeline.ui_language` / `pipeline.sdk_language`(`auto` 或语言代码;`zh-Hans` 用户显示 `zh-Hans`)。API:`GET`/`PUT /api/v1/settings/ui-language`。
|
||||
|
||||
## 8. 召回(recall)
|
||||
|
||||
|
||||
@@ -5,6 +5,11 @@ Post-cutover hosts use SDK-native Context v2 at
|
||||
`POST /api/v2/context/search`. A v2 response is never converted into v1 by
|
||||
inventing revisions, Evidence IDs, paths, or adoption state.
|
||||
|
||||
Since MemHub 1.0.6 the v2 response also carries `user_id`, the user whose
|
||||
memories were searched (`local` unless the request named one). The Host
|
||||
Bridge validator accepts it as an optional field, so a response from an older
|
||||
Core without it still validates; any other unknown field still fails closed.
|
||||
|
||||
## Transport
|
||||
|
||||
- The Broker `/context` route (used by host hooks and `recall-cli --context`;
|
||||
|
||||
@@ -32,6 +32,7 @@ prompt and no extra call.
|
||||
| `session_episodes` | `recall.session_episode_enabled` (`MEMORY_SESSION_EPISODE_ENABLED`) | off | pro or ultra | none |
|
||||
| `episode_summary` | `recall.episode_summary_enabled` (`MEMORY_EPISODE_SUMMARY_ENABLED`) | off | pro or ultra | LLM: one call per closed episode of 3+ memories |
|
||||
| `intention` | `llm.enable_intention` (`MEMORY_ENABLE_INTENTION`) | **on** | pro or ultra | none (off shortens the prompt) |
|
||||
| `intention_dedup` | `recall.intention_dedup_enabled` (`MEMORY_INTENTION_DEDUP_ENABLED`) | **on** | pro or ultra | none |
|
||||
| `few_shot` | `llm.few_shot_enabled` (`MEMORY_FEW_SHOT_ENABLED`) | off | pro or ultra | tokens (extraction and reconcile) |
|
||||
|
||||
- **Graph rerank (usage A).** It propagates support over the graph edges
|
||||
@@ -52,8 +53,32 @@ prompt and no extra call.
|
||||
write half of episodes, and both are off by default. Usage B only helps once
|
||||
episodes are being written. The SDK's session narrative (L3, `finalize_session`)
|
||||
is a separate feature and is not exposed (section 2).
|
||||
- **`intention`** is the only feature that is on by default. Turning it off
|
||||
stops L7 intention memories from being extracted.
|
||||
- **`intention`** is on by default. Turning it off stops L7 intention
|
||||
memories from being extracted.
|
||||
- **`intention_dedup`** (hy-memory 2.0.12) is on by default, which is what
|
||||
every earlier version did. A search treats intentions of one user and agent
|
||||
with the same `memory_at` day, the same `valid_until` day and a cosine
|
||||
similarity at or above `intention_dedup_threshold` as copies: it returns the
|
||||
earliest and deletes the others from the store. Off, no intention dedup
|
||||
runs, at read time or within a write batch, and nothing is hidden, dropped
|
||||
or deleted.
|
||||
|
||||
### Parameters
|
||||
|
||||
Numeric settings beside the features. Each tunes one feature and acts only
|
||||
while that feature acts.
|
||||
|
||||
| Name | SDK field (env) | Default | Range | Tunes |
|
||||
|---|---|---|---|---|
|
||||
| `intention_dedup_threshold` | `recall.intention_dedup_threshold` (`MEMORY_INTENTION_DEDUP_THRESHOLD`, else `MEMORY_DEDUP_THRESHOLD`) | 0.95 | 0.8–1.0 inclusive | `intention_dedup` (read time only) |
|
||||
|
||||
Why 0.8–1.0: with dedup on, every intention at or above the threshold is
|
||||
deleted from the store at read time, and that cannot be undone. Below about
|
||||
0.8 the embedding models MemHub ships or connects score different plans on
|
||||
one topic as that close, so a lower value would delete intentions that are
|
||||
not copies. Cosine similarity cannot exceed 1.0, and 1.0 keeps dedup for
|
||||
exact copies only. Someone who wants no dedup turns `intention_dedup` off
|
||||
rather than picking a threshold.
|
||||
|
||||
### Left out, and why
|
||||
|
||||
@@ -81,11 +106,12 @@ prompt and no extra call.
|
||||
holds only the features they set:
|
||||
|
||||
```json
|
||||
{"memory_mode": "pro", "memory_features": {"episode_recall": true, "session_episodes": true}}
|
||||
{"memory_mode": "pro", "memory_features": {"episode_recall": true, "intention_dedup_threshold": 0.9}}
|
||||
```
|
||||
|
||||
The values are strict booleans and unknown keys are refused, so a hand-edited
|
||||
file with an unknown feature fails to load (`config_invalid`), like any other
|
||||
Feature values are strict booleans, a parameter is a number in its range, and
|
||||
unknown keys are refused. So a hand-edited file with an unknown feature or an
|
||||
out-of-range threshold fails to load (`config_invalid`), like any other
|
||||
malformed setting.
|
||||
|
||||
## 3. How a setting reaches the SDK
|
||||
@@ -104,7 +130,8 @@ malformed setting.
|
||||
- Each client's features are fixed when it is built, like its mode. Both the
|
||||
write handles (`SdkWriteCoordinator`) and the read handle
|
||||
(`ActiveProfileReadHandle`) are keyed by (profile, revision, mode,
|
||||
features).
|
||||
features). The features part of the key is the sorted (name, value) pairs
|
||||
a person set, parameters included, so a new threshold is a new key.
|
||||
- A change runs `retire_stale_handles`, the #218 use count. An idle write
|
||||
handle closes at once, and a write in flight finishes with the features it
|
||||
started with. The next read builds a client with the new key. The old read
|
||||
@@ -135,6 +162,22 @@ malformed setting.
|
||||
"description": "Usage A: propagate support over knowledge-graph edges ..."
|
||||
}
|
||||
],
|
||||
"parameters": [
|
||||
{
|
||||
"name": "intention_dedup_threshold",
|
||||
"value": 0.95,
|
||||
"configured": null,
|
||||
"default": 0.95,
|
||||
"minimum": 0.8,
|
||||
"maximum": 1.0,
|
||||
"active": true,
|
||||
"reason": null,
|
||||
"requires_feature": "intention_dedup",
|
||||
"sdk_field": "recall.intention_dedup_threshold",
|
||||
"env": "MEMORY_INTENTION_DEDUP_THRESHOLD",
|
||||
"description": "Cosine similarity at or above which ..."
|
||||
}
|
||||
],
|
||||
"non_default": {"graph_rerank": true}
|
||||
}
|
||||
```
|
||||
@@ -143,27 +186,32 @@ malformed setting.
|
||||
- `configured` is null when unset.
|
||||
- `default` is the SDK default: its code default, or its own env var when set.
|
||||
- `active` is false with a `reason` when the feature does nothing right now.
|
||||
The reason is one of `disabled`, `requires_pro` (lite), `requires_ultra`, or
|
||||
`requires_graph_rerank` (for `same_series_timeline`).
|
||||
The reason is one of `disabled`, `requires_pro` (lite), `requires_ultra`,
|
||||
`requires_graph_rerank` (for `same_series_timeline`), or
|
||||
`requires_intention_dedup` (for the threshold while `intention_dedup` is off).
|
||||
- `cost` is one of `none`, `extra_query`, `llm_call`, `prompt_tokens`.
|
||||
- `features` always lists every feature, in the table's order.
|
||||
- `parameters` lists every parameter. `value` is what clients are built with;
|
||||
`minimum` and `maximum` are the range a `PUT` accepts. A parameter is
|
||||
active exactly when the feature it tunes is.
|
||||
|
||||
## 5. `PUT /api/v1/settings/memory-features`
|
||||
|
||||
The body has any of the feature names as keys:
|
||||
|
||||
- `true` or `false` sets that feature.
|
||||
- A JSON number sets a parameter, if it is in range (inclusive).
|
||||
- `null` resets it to the default.
|
||||
- An absent key is left as it is.
|
||||
- Anything else answers `422`, including a non-boolean, `"true"`, `1`, or an
|
||||
unknown name.
|
||||
- Anything else answers `422`, including a non-boolean feature, `"true"`, `1`,
|
||||
an unknown name, or a threshold outside 0.8–1.0, a string, or a boolean.
|
||||
|
||||
```json
|
||||
{"graph_rerank": true, "episode_recall": true, "few_shot": null}
|
||||
{"graph_rerank": true, "intention_dedup_threshold": 0.9, "few_shot": null}
|
||||
```
|
||||
|
||||
The answer is the state of section 4, plus `changed`: the features whose
|
||||
effective value changed, in the table's order. When the set of configured
|
||||
The answer is the state of section 4, plus `changed`: the features and then
|
||||
the parameters whose effective value changed, in the tables' order. When the set of configured
|
||||
features changes, the SDK clients are rebuilt (section 3). That includes a
|
||||
reset to a value equal to the default, after which the client carries no
|
||||
override at all.
|
||||
@@ -172,10 +220,11 @@ override at all.
|
||||
|
||||
`pipeline` in `GET /api/v1/memory-processing` (memory-mode contract §7) gains
|
||||
`features`, which lists the features whose effective value differs from the
|
||||
default. It is `{}` while every feature is at its default:
|
||||
default, and the parameters (as numbers) likewise. It is `{}` while every
|
||||
setting is at its default:
|
||||
|
||||
```json
|
||||
{"pipeline": {"mode": "pro", "...": "...", "features": {"episode_recall": true, "intention": false}}}
|
||||
{"pipeline": {"mode": "pro", "...": "...", "features": {"episode_recall": true, "intention_dedup_threshold": 0.9}}}
|
||||
```
|
||||
|
||||
`memhub status --json` and `memhub progress --json` carry it as
|
||||
@@ -188,14 +237,19 @@ to the memory-mode line.
|
||||
memhub settings features # list them
|
||||
memhub settings features set episode_recall=true session_episodes=true # plan only
|
||||
memhub settings features set episode_recall=true session_episodes=true --yes
|
||||
memhub settings features set intention_dedup=false --yes # no intention dedup
|
||||
memhub settings features set intention_dedup_threshold=0.9 --yes # a looser dedup
|
||||
memhub settings features reset few_shot # back to the default
|
||||
memhub settings features reset # every set feature
|
||||
```
|
||||
|
||||
`set` prints its plan and needs `--yes`, like `settings mode`, because some
|
||||
features spend model calls. `reset` only returns features to their defaults,
|
||||
so it applies at once. Unknown names or values other than `true` / `false` are
|
||||
a usage error (exit 2) before the Runtime is asked.
|
||||
so it applies at once. Unknown names, feature values other than `true` /
|
||||
`false`, and a threshold that is not a plain decimal in 0.8–1.0 are a usage
|
||||
error (exit 2) before the Runtime is asked. Against an older Runtime that does
|
||||
not list the threshold, `set` answers `rejected` /
|
||||
`memory_feature_unsupported` without sending it.
|
||||
|
||||
## 8. Rules kept
|
||||
|
||||
@@ -216,6 +270,8 @@ a usage error (exit 2) before the Runtime is asked.
|
||||
| session_episodes | 片段写入:同一会话中提取的记忆归入同一片段。 | Episodes per conversation: memories extracted from one conversation share one episode. | Episodios por conversación: las memorias extraídas de una conversación comparten un episodio. |
|
||||
| episode_summary | 片段摘要:片段结束时总结一次(每个片段一次模型调用)。 | Episode summaries: each finished episode is summarized once (one model call per episode). | Resúmenes de episodio: cada episodio terminado se resume una vez (una llamada al modelo por episodio). |
|
||||
| intention | 意图提取:随事实一起记住你提到的计划和意图。 | Intentions: remembers the plans and intentions you mention, along with facts. | Intenciones: recuerda los planes e intenciones que mencionas, junto con los hechos. |
|
||||
| intention_dedup | 意图去重:同一天里几乎相同的意图只保留最早一条,其余删除。 | Intention dedup: of near-identical intentions from one day, keeps the earliest and deletes the rest. | Deduplicación de intenciones: de las intenciones casi idénticas de un mismo día, conserva la más antigua y borra las demás. |
|
||||
| intention_dedup_threshold | 去重阈值:越高,越少的意图被视为重复。 | Dedup threshold: the higher it is, the fewer intentions count as copies. | Umbral de deduplicación: cuanto más alto, menos intenciones cuentan como copias. |
|
||||
| few_shot | 提取示例:在提取提示中加入示例,可帮助较弱的模型(输入更长)。 | Extraction examples: adds examples to the extraction prompt, which can help a weaker model (longer input). | Ejemplos de extracción: añade ejemplos a la instrucción de extracción, lo que puede ayudar a un modelo más débil (entrada más larga). |
|
||||
| default | 默认 | Default | Predeterminado |
|
||||
| requires_ultra | 当前模式下不生效,需要 ultra。 | Has no effect in this mode; needs ultra. | No tiene efecto en este modo; requiere ultra. |
|
||||
|
||||
@@ -20,6 +20,20 @@ cursor。旧文件后续追加也必须能被再次发现,不能只有一个
|
||||
用户授权是读取前置条件。环境变量只用于发现候选根,不能代替授权。每个候选
|
||||
文件必须在打开时再次验证 realpath 归属、symlink、普通文件类型和大小预算。
|
||||
|
||||
多用户(仅 Hermes):一个授权根属于一个 `user_id`。Hermes 的扫描、检测和
|
||||
持续同步共用同一次遍历,**任何层级**名为 `profiles` 的目录都不进入(发现
|
||||
helper 的 `excluded_directory_names`),所以授权根即使在 `~/.hermes` 之上,
|
||||
也不会读到 `.hermes/profiles/<name>`。接入时 Core 拒绝:
|
||||
|
||||
- 根的最后一段是 `profiles`(profiles 容器本身):422
|
||||
`hermes_root_is_profiles_container`;
|
||||
- 根(解析 symlink 后)与另一个用户未撤销的 Hermes 授权根相同、包含它或在
|
||||
它之内:409 `hermes_root_overlaps_other_user`。唯一例外是一方正好是另一方
|
||||
的 `profiles/<name>`(`local` 在 `~/.hermes`、另一个用户在
|
||||
`~/.hermes/profiles/<name>`),因为上面的排除已把两者隔开。
|
||||
|
||||
本机 `local` 的接入(`/sources/connect`、`/sources/{id}/authorize`)也走同一检查。
|
||||
|
||||
## Imported source record
|
||||
|
||||
Adapter 输出的 `ScannedEvidence` 是兼容命名的导入中间表示。原文持久化与 raw
|
||||
|
||||
@@ -40,7 +40,7 @@ Preview 不会弹出给用户;开关点击本身被视为确认。关闭时通
|
||||
| [Qwen Code](qwen-code.md) | 开放 | MCP + Skill | 无自动 Hook | `$QWEN_HOME/projects` 扫描 |
|
||||
| [Pi](pi.md) | 开放 | CLI Skill,不使用 MCP | JS Capture Extension | Pi session 扫描 + Extension 实时入口 |
|
||||
| [OpenClaw](openclaw.md) | 正式产品冻结 | 当前没有;仅有未开放的底层 MCP 写入器 | 没有 Hook、Plugin 或 Prefetch | Adapter 也被冻结,开关不可用 |
|
||||
| [DeepSeek Harness](deepseek-harness.md) | 详情页排除,来源管理页按 Profile 接入 | Profile 级 MCP Patch | 无自动 Hook | `$DSH_HOME/sessions` 扫描 |
|
||||
| [DeepSeek Harness](deepseek-harness.md) | 开放;一个开关覆盖全部已检测 Profile | Profile 级 MCP Patch | 无自动 Hook | `$DSH_HOME/sessions` 扫描 |
|
||||
|
||||
## 同步开关的统一精确语义
|
||||
|
||||
|
||||
@@ -1,28 +1,45 @@
|
||||
# DeepSeek Harness 一键插入支持
|
||||
|
||||
更新日期:2026-10-18
|
||||
更新日期:2026-11-02
|
||||
|
||||
## 结论
|
||||
|
||||
DeepSeek Harness 的 Profile 级 MCP 安装器已经实现,Profile Scoped Host ID(`deepseek-harness:<profile>`)在权限层按不透明标签处理,可正常走完 Preview 与安装。
|
||||
DeepSeek Harness 按 Profile 配置 MCP:每个 Profile 是一个独立 Host(`deepseek-harness:<profile>`),有自己的 `cordis.patch.yml`、Credential 与 Grant。应用详情页只有一个“一键插入应用”开关,它一次覆盖桌面检测到的全部 Profile:打开时逐个安装并授权,关闭时逐个移除并撤销。
|
||||
|
||||
## 接入方式和产品状态
|
||||
|
||||
| 项目 | DeepSeek Harness 当前实现 |
|
||||
|---|---|
|
||||
| 应用详情页一键插入 | 有意排除,因为无法选择 Profile |
|
||||
| 来源管理页 Profile 安装 | 可用;每个 Profile 独立 Patch、Credential 与 Grant |
|
||||
| 应用详情页一键插入 | 可用;一个开关覆盖全部已检测 Profile,并列出每个 Profile 的状态 |
|
||||
| 首次导入批量连接 | 排除;在应用详情页打开开关 |
|
||||
| Harness 查询 MemHub | 通过 Profile 级 Cordis MCP Client |
|
||||
| Skill | 没有 |
|
||||
| Prompt 前自动 Prefetch | 没有 |
|
||||
| Capture Hook/Plugin | 没有 |
|
||||
| Harness 新会话进入 MemHub | Core 扫描 `$DSH_HOME/sessions` |
|
||||
| Harness 新会话进入 MemHub | Core 扫描 `$DSH_HOME/sessions`,按 `session[.vN].jsonl[.zstd]` 识别会话文件 |
|
||||
|
||||
会话文件按写入时的格式版本命名:首版为 `session.jsonl` / `session.jsonl.zstd`,v3/v4 起为 `session.v3.jsonl.zstd` / `session.v4.jsonl.zstd`。来源扫描接受带版本号的名称;解析前再读文件头 `version` 字段,只导入已验证的 v0/v3/v4,未来未知版本不会被解析:整个文件以 `unsupported_schema` 报告,且不留下读取进度,将来支持该版本的发布会从头读取(错误导入比暂时漏导更难回滚)。`tool/result` 的输出无论写成 `tool-result` 块还是纯 `text` 块,都按同一条裁剪与凭据清洗路径导入。
|
||||
|
||||
## 两个开关分别做什么
|
||||
|
||||
### “一键插入应用”
|
||||
### 打开“一键插入应用”
|
||||
|
||||
应用详情页不可用。来源管理页对每个 Profile 写 `cordis.patch.yml`、创建独立 Credential 并授权;Host ID 形如 `deepseek-harness:<profile>`,冒号只作为标签存在,权限层的安装目录按 Host ID 摘要生成,不参与路径拼接。
|
||||
应用详情页对每个已检测、尚未获批的 Profile:
|
||||
|
||||
1. 预览并写入该 Profile 的 `cordis.patch.yml` 受管块,创建独立 Credential;
|
||||
2. 授予 `om_recall`、`om_conversations`、`om_diagnostic_trace`,范围为 `all-apps` 与 `source:deepseek-harness`(与其他应用的一键插入相同)。
|
||||
|
||||
已获批的 Profile 不会重装。某个 Profile 失败时,其他 Profile 的结果保留,页面提示失败的 Profile 名称;开关只要有一个 Profile 获批就显示为开。Host ID 里的冒号只作为标签存在,权限层的安装目录按 Host ID 摘要生成,不参与路径拼接。
|
||||
|
||||
### 关闭“一键插入应用”
|
||||
|
||||
对每个已接入的 Profile 先撤销 Credential,再删除受管块(与其他应用的关闭语义相同)。受管块被改过、无法删除时仍会撤销授权,页面列出需要核对的 Profile;撤销也失败的 Profile 会单独列出,它仍可使用记忆。从未接入的 Profile 不受影响。
|
||||
|
||||
### 之后新建的 Profile
|
||||
|
||||
桌面每次刷新应用接入状态时都会重新枚举 Profile(`$DSH_HOME/profiles/<profile>/cordis.yml`),应用首页打开时每 3 秒刷新一次;常驻的 Runtime 收到未知的 Profile Host ID 时也会重新读盘。开关的选择会被记住:开关为开时,新出现的 Profile 由应用首页自动安装并授权,不需要重启 MemHub。
|
||||
|
||||
每个新 Profile 触发一次自动尝试;失败时应用首页会指明 Profile,不会每次轮询都重试,下次启动或点击重试时再试一次。开关为关时不会自动接入任何 Profile。
|
||||
|
||||
### 打开“同步开关”
|
||||
|
||||
@@ -32,12 +49,10 @@ DeepSeek Harness 的 Profile 级 MCP 安装器已经实现,Profile Scoped Host
|
||||
|
||||
Core 停止扫描和 pending Job claim,running Job可结束,已有 Memory 保留。它不会修改任何 Profile 的 `cordis.patch.yml` 或 Credential。
|
||||
|
||||
### 修复一键安装后的组合语义
|
||||
### 组合语义
|
||||
|
||||
未来修复 Profile Host ID 后:
|
||||
|
||||
- 某个 Profile 的一键插入只决定该 Profile 是否能查询 MemHub;
|
||||
- 全局同步开关决定 MemHub 是否继续读取整个 DeepSeek Harness 的 Session;
|
||||
- 一键插入决定已检测 Profile 是否能查询 MemHub;
|
||||
- 同步开关决定 MemHub 是否继续读取整个 DeepSeek Harness 的 Session;
|
||||
- 没有自动 Prefetch,也没有 Capture Hook。
|
||||
|
||||
## Profile 粒度
|
||||
@@ -58,7 +73,7 @@ deepseek-harness:headless
|
||||
...
|
||||
```
|
||||
|
||||
理论上每个 Profile 拥有独立 Patch、Journal、Credential 和 Grant;Core 会话同步仍按统一的 `deepseek-harness` 来源扫描 `$DSH_HOME/sessions`。
|
||||
每个 Profile 拥有独立 Patch、Journal、Credential 和 Grant;Core 会话同步仍按统一的 `deepseek-harness` 来源扫描 `$DSH_HOME/sessions`。
|
||||
|
||||
## MCP Patch
|
||||
|
||||
@@ -85,13 +100,15 @@ DeepSeek Harness 不安装 Skill,也不安装 Capture Hook;新会话进入 M
|
||||
|
||||
## UI 状态
|
||||
|
||||
- 应用详情页排除 DeepSeek Harness,因为单一来源页无法选择 Profile。
|
||||
- 首次导入批量连接也排除它。
|
||||
- 来源管理页会列出每个 Profile,并显示独立安装按钮。
|
||||
- 应用详情页的“一键插入应用”下方列出每个已检测 Profile 及其“已接入 / 未接入”状态;没有检测到 Profile 时开关不可用,并提示创建 Profile 后回到此页。
|
||||
- Host ID 只接受小写字母、数字和连字符(最长 32 个字符)。名称不符合的 Profile 仍会列出,但无法授权,页面会指明它和原因,不影响其他 Profile。
|
||||
- 连接状态取各已接入 Profile 记录的自检结果:全部接通才显示“已接通”。
|
||||
- 首次导入批量连接排除 DeepSeek Harness。
|
||||
- CLI 的 `memhub host install deepseek-harness:<profile>` 只安装,不授权(状态停在 `install-only`);授权入口只有应用详情页的开关,CLI 没有授权命令。
|
||||
|
||||
## 已修复:Profile Host ID 校验
|
||||
|
||||
此前来源管理页安装会在写入 `cordis.patch.yml` 和创建 Credential 之前失败,原因是:
|
||||
此前 Profile 安装会在写入 `cordis.patch.yml` 和创建 Credential 之前失败,原因是:
|
||||
|
||||
```text
|
||||
系统生成 hostId = deepseek-harness:<profile>
|
||||
@@ -101,18 +118,12 @@ HostGrantStore 也不接受冒号
|
||||
|
||||
Host ID 是不可解释的标签:安装目录对它取摘要,不会把它拼进路径。现在权限层与 Grant 存储都接受一个可选的 `:<profile>` 段,因此 Profile 安装可以正常走完。这不是 DeepSeek Harness 不支持 MCP 的问题,而是 Profile Host ID 格式与权限校验不一致,现已对齐。
|
||||
|
||||
## 修复后的权限语义风险
|
||||
## 已修复:应用详情页返回 `unsupported_host`
|
||||
|
||||
来源管理页当前默认授权选择可能包含:
|
||||
|
||||
- `om_recall`、`om_conversations`、`om_diagnostic_trace`(`om_ask` 已随 `MemHub_ask` 移除,旧授权仍可读取但不再授权任何工具);
|
||||
- `all-apps`;
|
||||
- 所有已授权 Source 和 Workspace。
|
||||
|
||||
这比应用详情页通用授权更宽。Host ID 修复后仍应确认是否希望每个 Profile 默认获得如此宽的权限。
|
||||
此前应用详情页按“一个来源对应一个 Host”接入,而 DeepSeek Harness 没有不带 Profile 的 Host,开关因此置灰并返回 `unsupported_host`,安装后的 Profile 也只能停在 `install-only`(#230)。现在开关直接作用于全部 `deepseek-harness:<profile>` Host,复用与其他应用相同的安装、授权、撤销与卸载接口。
|
||||
|
||||
## 重复安装与卸载
|
||||
|
||||
底层 Patch 安装是幂等的:已有受管块时替换,不追加第二份。相同 Row ID 出现在 MemHub 哨兵块外时默认拒绝接管。
|
||||
|
||||
正常卸载设计是先撤销该 Profile Credential,再删除受管块并保留其他 Patch;如果删完没有其他 Row,恢复 `[]`。用户修改过受管块时不强删。
|
||||
卸载先撤销该 Profile Credential,再删除受管块并保留其他 Patch;如果删完没有其他 Row,恢复 `[]`。用户修改过受管块时不强删。
|
||||
@@ -118,6 +118,9 @@ export interface SdkContextSearchResponse {
|
||||
used_characters: number;
|
||||
truncated: boolean;
|
||||
total_ms: number;
|
||||
/** Whose memories were searched; `local` unless the request named a user.
|
||||
* Absent from a Core older than 1.0.6. */
|
||||
user_id?: string;
|
||||
}
|
||||
|
||||
/** Native Host Bridge wire response. Schemas stay distinct and are never adapted. */
|
||||
|
||||
Loaded 100 of 128 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user