Compare commits
22
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4e1748f65a | ||
|
|
8d0a069950 | ||
|
|
cdb44658cf | ||
|
|
2ab04291a9 | ||
|
|
80d3b55723 | ||
|
|
b5ec0861f0 | ||
|
|
47e938da1b | ||
|
|
f5ea0af75d | ||
|
|
1c2f48579f | ||
|
|
d29615abeb | ||
|
|
0965519e16 | ||
|
|
7e9f0d98d8 | ||
|
|
0ff4538f36 | ||
|
|
a9709788d8 | ||
|
|
c1e5916b51 | ||
|
|
97de5480bd | ||
|
|
97d9309ca8 | ||
|
|
b1b3e0a291 | ||
|
|
53507c7abe | ||
|
|
5a1d252787 | ||
|
|
8895fea775 | ||
|
|
78882f3300 |
No files matched your search
@@ -94,6 +94,29 @@ authority on first launch.
|
||||
`hy_node_permissions` proof table. Evidence, governed memories, model
|
||||
profiles, SDK stores, and source/host grants are preserved.
|
||||
|
||||
## Hy Memory 2.0.9 — 2026-11-01
|
||||
|
||||
- Published
|
||||
[`hy-memory==2.0.9`](https://pypi.org/project/hy-memory/2.0.9/).
|
||||
- Placeholder values no longer overwrite the user's basic profile (L0). A
|
||||
small extraction model filled `basic_info` with `{"name": "用户"}`,
|
||||
`The user` or `N/A` when the user had said nothing about themselves, and
|
||||
each placeholder superseded the real value, so the L0 chain read
|
||||
费天翔 → Fei → 用户 → The user. Two defences:
|
||||
- `upsert_basic_profile` drops placeholder values: empty, `用户`,
|
||||
`the user`, `user`, `n/a`, `na`, `none`, `null`, `unknown`, `未提供`,
|
||||
`未知`, `不详`, `-`, `?` and the field's own name, compared
|
||||
case-insensitively with whitespace and punctuation stripped. When nothing
|
||||
is left it writes no new version and no new L0 head. Placeholders already
|
||||
stored in the chain no longer override earlier real values when the
|
||||
profile is assembled, so `get_user_profile` of an affected user shows the
|
||||
last real value again without any migration. This also stops an
|
||||
assistant quoting an old memory ("The user's name is 用户.") from being
|
||||
written back.
|
||||
- The extraction prompts (zh and en) state the rule: only what the user
|
||||
said in a `user` turn, never the field name or a placeholder, `{}`
|
||||
otherwise, with a small-talk counter-example.
|
||||
|
||||
## Hy Memory 2.0.8 — 2026-10-28
|
||||
|
||||
- Published
|
||||
|
||||
@@ -682,6 +682,23 @@ Output:
|
||||
```
|
||||
John's occupation and location are HIS attributes, so they go into `memory` with "John" as the subject. They must NEVER land in `basic_info.occupation` / `basic_info.location` — `basic_info` is exclusively the user's own profile. Only `name` (Tim, from the user message's `name` field) belongs there.
|
||||
|
||||
## Example 11 — Counter-example: small talk has no basic_info
|
||||
|
||||
New Messages:
|
||||
[{"role": "user", "content": "Hi"},
|
||||
{"role": "assistant", "content": "Hello! The user's name is The user. How can I help?"}]
|
||||
Memory Date: 2025-09-03
|
||||
|
||||
Output:
|
||||
```json
|
||||
{
|
||||
"memory": [],
|
||||
"intentions": [],
|
||||
"basic_info": {}
|
||||
}
|
||||
```
|
||||
The user stated nothing about themselves. The assistant's line is not a user turn, and "The user" is a placeholder, not a name. Output `{}` — never `{"name": "The user"}` or `{"name": "user"}`.
|
||||
|
||||
---
|
||||
"""
|
||||
|
||||
@@ -749,6 +766,14 @@ Rules:
|
||||
fields (e.g. name, age, location, occupation, employer) **explicitly stated**
|
||||
in the NEW user messages. Never infer, never invent. If no field is explicitly
|
||||
stated, output `{{}}`.
|
||||
* HARD RULE: fill a field ONLY if the user stated it in a `user` turn.
|
||||
Anything said only by the assistant (including the assistant quoting an
|
||||
earlier memory such as "The user's name is X") does NOT count. If nothing
|
||||
qualifies, output `"basic_info": {{}}`.
|
||||
* NEVER write the field name itself or a placeholder as a value ("user",
|
||||
"The user", "用户", "name", "N/A", "unknown", "none", "-"). Omit the field.
|
||||
* Negative example: small talk such as `[user]: Hi` / `[assistant]: How can
|
||||
I help?`, or `[user]: Reply with exactly: PONG` → `"basic_info": {{}}`.
|
||||
* Exception for `name`: the `name` field on the user's own message, or a leading
|
||||
line "The user's name is X" / "用户名叫X", is an authoritative source for
|
||||
`name` — even though the user never said it out loud.
|
||||
@@ -817,6 +842,14 @@ Rules:
|
||||
fields (e.g. name, age, location, occupation, employer) **explicitly stated**
|
||||
in the NEW user messages. Never infer, never invent. If no field is explicitly
|
||||
stated, output `{{}}`.
|
||||
* HARD RULE: fill a field ONLY if the user stated it in a `user` turn.
|
||||
Anything said only by the assistant (including the assistant quoting an
|
||||
earlier memory such as "The user's name is X") does NOT count. If nothing
|
||||
qualifies, output `"basic_info": {{}}`.
|
||||
* NEVER write the field name itself or a placeholder as a value ("user",
|
||||
"The user", "用户", "name", "N/A", "unknown", "none", "-"). Omit the field.
|
||||
* Negative example: small talk such as `[user]: Hi` / `[assistant]: How can
|
||||
I help?`, or `[user]: Reply with exactly: PONG` → `"basic_info": {{}}`.
|
||||
* Exception for `name`: the `name` field on the user's own message, or a leading
|
||||
line "The user's name is X" / "用户名叫X", is an authoritative source for
|
||||
`name` — even though the user never said it out loud.
|
||||
|
||||
@@ -365,6 +365,13 @@ EXTRACT_PROMPT_ZH = """## 最近 k 条消息
|
||||
* basic_info:**只装用户本人的画像**。
|
||||
仅当 NEW user messages 中**明确**陈述稳定画像字段(如 姓名、年龄、所在地、职业、雇主)时才填写。
|
||||
绝不推断、绝不编造。没有任何字段被明确陈述时输出 `{{}}`。
|
||||
* **硬性规则**:只有用户在 `user` 回合里**明确说出**的字段才能填写。只在
|
||||
assistant 回合出现的内容(包括 assistant 引用旧记忆,如"The user's name is X"
|
||||
/ "用户名是X")一律不算。没有符合条件的字段时,必须输出 `"basic_info": {{}}`。
|
||||
* **绝不**把字段名本身或占位符当值写入("用户"、"The user"、"user"、"name"、
|
||||
"姓名"、"N/A"、"unknown"、"未知"、"未提供"、"-"),宁可省略该字段。
|
||||
* 负例:寒暄如 `[user]: 你好` / `[assistant]: 我可以帮你。`,或
|
||||
`[user]: Reply with exactly: PONG` → `"basic_info": {{}}`。
|
||||
* `name` 的例外:用户自己那条消息上的 `name` 字段,或文首形如
|
||||
"用户名叫X" / "The user's name is X" 的那一行,都是 `name` 的权威来源
|
||||
——即使用户本人从未开口自报姓名。
|
||||
@@ -440,6 +447,13 @@ EXTRACT_PROMPT_ZH_NO_INTENTION = """## 最近 k 条消息
|
||||
* basic_info:**只装用户本人的画像**。
|
||||
仅当 NEW user messages 中**明确**陈述稳定画像字段(如 姓名、年龄、所在地、职业、雇主)时才填写。
|
||||
绝不推断、绝不编造。没有任何字段被明确陈述时输出 `{{}}`。
|
||||
* **硬性规则**:只有用户在 `user` 回合里**明确说出**的字段才能填写。只在
|
||||
assistant 回合出现的内容(包括 assistant 引用旧记忆,如"The user's name is X"
|
||||
/ "用户名是X")一律不算。没有符合条件的字段时,必须输出 `"basic_info": {{}}`。
|
||||
* **绝不**把字段名本身或占位符当值写入("用户"、"The user"、"user"、"name"、
|
||||
"姓名"、"N/A"、"unknown"、"未知"、"未提供"、"-"),宁可省略该字段。
|
||||
* 负例:寒暄如 `[user]: 你好` / `[assistant]: 我可以帮你。`,或
|
||||
`[user]: Reply with exactly: PONG` → `"basic_info": {{}}`。
|
||||
* `name` 的例外:用户自己那条消息上的 `name` 字段,或文首形如
|
||||
"用户名叫X" / "The user's name is X" 的那一行,都是 `name` 的权威来源
|
||||
——即使用户本人从未开口自报姓名。
|
||||
@@ -669,6 +683,23 @@ Memory Date:2025-09-02
|
||||
```
|
||||
职业和所在地是 **John 的**属性,所以进 `memory` 并以"John"作主体,**绝不能**落到 `basic_info` 的 `occupation` / `location` —— `basic_info` 专属于用户本人。只有 `name`(Tim,来自用户消息的 `name` 字段)才该放进去。注意输出语言随输入语言(此例为英文)。
|
||||
|
||||
## 示例 11 —— 反例:寒暄没有 basic_info
|
||||
|
||||
新消息:
|
||||
[{"role": "user", "content": "你好"},
|
||||
{"role": "assistant", "content": "你好!我可以帮你。之前记得 The user's name is 用户."}]
|
||||
Memory Date:2025-09-03
|
||||
|
||||
输出:
|
||||
```json
|
||||
{
|
||||
"memory": [],
|
||||
"intentions": [],
|
||||
"basic_info": {}
|
||||
}
|
||||
```
|
||||
用户没有说任何关于自己的信息。assistant 那句不是 user 回合,"用户" 也是占位符而不是名字。必须输出 `{}`——**绝不能**输出 `{"name": "用户"}`。
|
||||
|
||||
---
|
||||
"""
|
||||
|
||||
|
||||
@@ -26,6 +26,8 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import re
|
||||
import unicodedata
|
||||
import uuid
|
||||
import weakref
|
||||
from dataclasses import dataclass
|
||||
@@ -208,6 +210,76 @@ def _sanitize_kv(
|
||||
return result
|
||||
|
||||
|
||||
# ================================================================
|
||||
# 占位符守卫(P0-1)
|
||||
# ================================================================
|
||||
# 弱模型(如 qwen3.5-2b)在用户什么都没说时,会把 schema 字段说明里的词当值
|
||||
# 填进 basic_info:{"name": "用户"} / "The user" / "N/A"。每个占位符都会
|
||||
# supersede 掉真实值,演化链变成 费天翔 → Fei → 用户 → The user。
|
||||
# 规则:占位符永远不入库;读侧组装 KV 时占位符也不覆盖已有真实值。
|
||||
|
||||
_PLACEHOLDER_VALUES = frozenset({
|
||||
"用户", "该用户", "本用户", "用户本人",
|
||||
"the user", "the user's", "user", "a user", "this user",
|
||||
"n/a", "na", "none", "null", "nil", "unknown", "unspecified",
|
||||
"not provided", "not specified", "not mentioned",
|
||||
"未提供", "未知", "不详", "未说明", "未提及", "无",
|
||||
"-", "?",
|
||||
})
|
||||
|
||||
# 内置字段的中文标签:弱模型也会把字段说明的中文词当值。
|
||||
_FIELD_LABELS: Dict[str, tuple] = {
|
||||
"name": ("姓名", "名字", "用户名", "用户姓名", "用户名字"),
|
||||
"age": ("年龄", "用户年龄"),
|
||||
"location": ("所在地", "地点", "城市", "用户所在地"),
|
||||
"occupation": ("职业", "工作", "用户职业"),
|
||||
"employer": ("雇主", "公司", "用户雇主"),
|
||||
}
|
||||
|
||||
# 去掉标点/符号后再比较,让 "The user." / "用户。" / "N/A" 都命中。
|
||||
# "-" / "?" 本身就是纯标点,剥完为空串,同样判为占位符。
|
||||
_PUNCT_RE = re.compile(r"[\s\.,;:!?'\"`~()\[\]{}<>/\\|_\-\u3000-\u303f\uff00-\uff0f\uff1a-\uff20]+")
|
||||
|
||||
|
||||
def _placeholder_key(value: str) -> str:
|
||||
s = unicodedata.normalize("NFKC", str(value or "")).strip().lower()
|
||||
s = s.replace("\u2019", "'")
|
||||
s = _PUNCT_RE.sub(" ", s)
|
||||
return " ".join(s.split())
|
||||
|
||||
|
||||
_PLACEHOLDER_KEYS = frozenset(_placeholder_key(v) for v in _PLACEHOLDER_VALUES)
|
||||
|
||||
|
||||
def is_placeholder_value(field_name: str, value: Any) -> bool:
|
||||
"""value 是否为占位符(空 / 用户 / The user / N/A / 字段名本身 等)。
|
||||
|
||||
大小写不敏感、首尾空白与标点不敏感。数字(如 age=30)永远不是占位符。
|
||||
"""
|
||||
if value is None:
|
||||
return True
|
||||
if isinstance(value, bool):
|
||||
return False
|
||||
if isinstance(value, (int, float)):
|
||||
return False
|
||||
key = _placeholder_key(value)
|
||||
if not key:
|
||||
return True
|
||||
if key in _PLACEHOLDER_KEYS:
|
||||
return True
|
||||
field_key = _placeholder_key(field_name)
|
||||
if field_key and key in (field_key, field_key.replace("_", " ")):
|
||||
return True
|
||||
for label in _FIELD_LABELS.get(field_name, ()):
|
||||
if key == _placeholder_key(label):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _drop_placeholders(kv: Dict[str, Any]) -> Dict[str, Any]:
|
||||
return {k: v for k, v in kv.items() if not is_placeholder_value(k, v)}
|
||||
|
||||
|
||||
# ================================================================
|
||||
# 公共入口:upsert_basic_profile
|
||||
# ================================================================
|
||||
@@ -287,6 +359,8 @@ async def _upsert_basic_profile_unlocked(
|
||||
# L0 Basic Info 从输入、唯一性到落盘都只属于 user_id。
|
||||
|
||||
new_kv = _sanitize_kv(kv or {}, allowed_fields=allowed_fields)
|
||||
# 单调性守卫:占位符永远不覆盖已有值,过滤后为空就不写新版本、不产生新链头。
|
||||
new_kv = _drop_placeholders(new_kv)
|
||||
if not new_kv:
|
||||
return BasicProfileUpsertResult(
|
||||
success=True, changed=False, reason="no valid fields after sanitize"
|
||||
@@ -513,5 +587,8 @@ def _assemble_full_kv_from_nodes(
|
||||
for k, v in kv.items():
|
||||
if allowed_set is not None and k not in allowed_set:
|
||||
continue
|
||||
# 历史上已落盘的占位符版本(如 name=用户)不覆盖更早的真实值。
|
||||
if is_placeholder_value(k, v):
|
||||
continue
|
||||
full_kv[k] = v
|
||||
return full_kv
|
||||
+2
-2
@@ -13,8 +13,8 @@
|
||||
接口先补 SDK。Runtime writers 停止后的 App 启动迁移可直接操作底层存储,
|
||||
但不得把迁移专用方法加入公共 Client。
|
||||
3. MemHub **不存 memory 实体**;**可以存 `memory_id`** 做组织/回执。原文进入 **SDK raw**。
|
||||
4. SDK raw 只保留滚动三天;一条 raw 对应一个完整 turn,只含真人 user 与
|
||||
assistant,tool 不持久化。提取后的高层 memory 不受该窗口影响。
|
||||
4. SDK raw 全部保留,任何模式(lite / pro / ultra)都不删除;一条 raw 对应一个
|
||||
完整 turn,只含真人 user 与 assistant,tool 不持久化。
|
||||
5. `source_raw_ids` 是 SDK 内部溯源关系,不是产品 Evidence 合同;记忆详情不提供
|
||||
“原始依据”或按 raw ID 展开原文的入口。
|
||||
|
||||
|
||||
+2
-2
@@ -18,7 +18,7 @@ If you run into a problem, please open an Issue with the version, platform,repro
|
||||
|
||||
## 发行与平台合同
|
||||
|
||||
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.8`。MemHub 源码版本为 `1.0.5`
|
||||
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.9`。MemHub 源码版本为 `1.0.5`
|
||||
(build 36),GitHub 仓库为
|
||||
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
|
||||
`memhub-cli`。当前代码架构见
|
||||
@@ -118,7 +118,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
|
||||
|
||||
## Hy Memory
|
||||
|
||||
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.8`。SDK 配置、权限边界及 search
|
||||
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.9`。SDK 配置、权限边界及 search
|
||||
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
|
||||
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
|
||||
|
||||
|
||||
@@ -2352,7 +2352,7 @@
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-hy-memory",
|
||||
"name": "hy-memory",
|
||||
"versionInfo": "2.0.8",
|
||||
"versionInfo": "2.0.9",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
"filesAnalyzed": false,
|
||||
"licenseConcluded": "Apache-2.0",
|
||||
|
||||
@@ -9,7 +9,7 @@ Per-package license texts are not reproduced here yet; the license
|
||||
column is filled in as part of release review, and the SBOM records
|
||||
`NOASSERTION` until then rather than guessing.
|
||||
|
||||
## hy-memory 2.0.8
|
||||
## hy-memory 2.0.9
|
||||
|
||||
License: Apache-2.0
|
||||
|
||||
|
||||
@@ -315,6 +315,8 @@ describe("catalog exit codes", () => {
|
||||
"models",
|
||||
"memories",
|
||||
"profile",
|
||||
"settings",
|
||||
"evolve",
|
||||
"graph-width",
|
||||
"host",
|
||||
"recall",
|
||||
|
||||
@@ -162,10 +162,54 @@ it("never emits a secret from a malformed manually written event", async () => {
|
||||
command: "logs",
|
||||
state: "empty",
|
||||
events: 0,
|
||||
invalid_lines: 1,
|
||||
invalid_lines: 0,
|
||||
unrecognized_lines: 1,
|
||||
});
|
||||
});
|
||||
|
||||
// After an upgrade the files still hold an older build's records. They are
|
||||
// whole JSON lines the reader leaves out, not damage, so the text output must
|
||||
// not warn about them; only a line that is not JSON is called damaged.
|
||||
it("warns only about lines that are not JSON, not about older records", async () => {
|
||||
const dataDir = await fixture();
|
||||
const older = {
|
||||
schema_version: 0,
|
||||
service: "desktop",
|
||||
component: "desktop.lifecycle",
|
||||
event: "desktop.started",
|
||||
};
|
||||
await writeFile(
|
||||
path.join(dataDir, "logs", "diagnostic-desktop.jsonl"),
|
||||
`${JSON.stringify(older)}\n${JSON.stringify({ ...older, schema_version: 1 })}\n`,
|
||||
{ mode: 0o600 },
|
||||
);
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
vi.spyOn(process.stdout, "write").mockImplementation((value) => {
|
||||
stdout += String(value);
|
||||
return true;
|
||||
});
|
||||
vi.spyOn(process.stderr, "write").mockImplementation((value) => {
|
||||
stderr += String(value);
|
||||
return true;
|
||||
});
|
||||
|
||||
expect(await run(["logs"], { ONE_MEMORY_DATA_DIR: dataDir })).toBe(0);
|
||||
expect(stdout).toContain("core.start_failed");
|
||||
expect(stdout).not.toContain("损坏");
|
||||
expect(stderr).toBe("");
|
||||
|
||||
await writeFile(
|
||||
path.join(dataDir, "logs", "diagnostic-cli.jsonl"),
|
||||
'{"schema_version": 1, "event": \n',
|
||||
{ mode: 0o600 },
|
||||
);
|
||||
stdout = "";
|
||||
expect(await run(["logs"], { ONE_MEMORY_DATA_DIR: dataDir })).toBe(0);
|
||||
expect(stdout).toContain("跳过 1 行损坏的诊断记录。");
|
||||
expect(stderr).toBe("memhub: 跳过 1 行损坏的诊断记录。\n");
|
||||
});
|
||||
|
||||
it("rejects invalid filters without touching the log directory", async () => {
|
||||
const dataDir = path.join(
|
||||
await mkdtemp(path.join(os.tmpdir(), "om-cli-no-log-")),
|
||||
|
||||
+323
-13
@@ -188,10 +188,24 @@ import {
|
||||
removeManagedCliLink,
|
||||
rollbackManagedCliLink,
|
||||
} from "./managed-cli-link.js";
|
||||
import {
|
||||
type EvolveDocument,
|
||||
evolveExitCode,
|
||||
type ModeDocument,
|
||||
modeExitCode,
|
||||
parseEvolveCommand,
|
||||
parseSettingsCommand,
|
||||
renderEvolveText,
|
||||
renderModeText,
|
||||
runEvolveCommand,
|
||||
runModeCommand,
|
||||
} from "./memory-mode-flow.js";
|
||||
import {
|
||||
addMemory,
|
||||
DEFAULT_USER_ID,
|
||||
isMemoryId,
|
||||
isRequestId,
|
||||
isUserId,
|
||||
MEMORY_KINDS,
|
||||
MEMORY_SCENES,
|
||||
MEMORY_VARIANTS,
|
||||
@@ -209,6 +223,7 @@ import {
|
||||
renderMemoryAddText,
|
||||
renderMemoryDetailText,
|
||||
renderMemoryPageText,
|
||||
userIdParameters,
|
||||
validMemoryText,
|
||||
} from "./memory-report.js";
|
||||
import {
|
||||
@@ -322,6 +337,8 @@ export interface RunControls {
|
||||
watchSignal?: AbortSignal;
|
||||
watchWait?: (milliseconds: number, signal: AbortSignal) => Promise<void>;
|
||||
watchNow?: () => string;
|
||||
/** `memhub evolve --wait` polls on this; tests inject an instant one. */
|
||||
sleep?: (milliseconds: number) => Promise<void>;
|
||||
readStdin?: () => Promise<Buffer | null>;
|
||||
stdinIsTTY?: boolean;
|
||||
/** Injectable onboarding prompter so the wizard can be driven in tests
|
||||
@@ -538,6 +555,23 @@ async function runCommand(
|
||||
environment = { ...environment, ONE_MEMORY_DATA_DIR: lifted.dataDir };
|
||||
}
|
||||
}
|
||||
// `--user-id <id>` is global to the memory commands: it names whose
|
||||
// memories are read or written. It is lifted before dispatch so each
|
||||
// command's own parser keeps seeing exactly the arguments it always did.
|
||||
// `sources connect` gives the flag its own meaning and parses it itself.
|
||||
let userId: string | undefined;
|
||||
if (USER_ID_COMMANDS.has(arguments_[0] ?? "")) {
|
||||
const lifted = liftUserIdOption(arguments_);
|
||||
if (lifted === null) {
|
||||
return reject(
|
||||
findCommand(arguments_[0] ?? ""),
|
||||
asJson,
|
||||
"--user-id 无效:只能是 1–64 位字母、数字、点、下划线或连字符。",
|
||||
);
|
||||
}
|
||||
arguments_ = lifted.arguments_;
|
||||
userId = lifted.userId;
|
||||
}
|
||||
environment = await environmentWithManagedDataDir(environment);
|
||||
const command = arguments_[0];
|
||||
if (command === undefined) return reject(undefined, asJson, "缺少命令。");
|
||||
@@ -776,9 +810,11 @@ async function runCommand(
|
||||
const attachment = await attachToRuntimeOrNull(environment);
|
||||
if (attachment === null) return runtimeUnavailable(descriptor, asJson);
|
||||
if (parsed.kind === "status") {
|
||||
const receipt = await readMemoryWrite(attachment, parsed.id).catch(
|
||||
() => undefined,
|
||||
);
|
||||
const receipt = await readMemoryWrite(
|
||||
attachment,
|
||||
parsed.id,
|
||||
userId,
|
||||
).catch(() => undefined);
|
||||
if (receipt === undefined) {
|
||||
return emitFailure(asJson, "write status", "memory_write_read_failed");
|
||||
}
|
||||
@@ -812,6 +848,7 @@ async function runCommand(
|
||||
? {}
|
||||
: { memory_at: parsed.memory_at }),
|
||||
...(parsed.priority === undefined ? {} : { priority: parsed.priority }),
|
||||
...(userId === undefined ? {} : { user_id: userId }),
|
||||
});
|
||||
if (outcome.state !== "accepted") {
|
||||
emit(
|
||||
@@ -957,9 +994,12 @@ async function runCommand(
|
||||
state: "empty",
|
||||
events: 0,
|
||||
invalid_lines: report.invalid_lines,
|
||||
unrecognized_lines: report.unrecognized_lines,
|
||||
})}\n`
|
||||
: renderLogText(report.events, report.invalid_lines),
|
||||
);
|
||||
// Only lines that are not JSON are damage. Records an older build wrote
|
||||
// are left out without a warning: after an upgrade they are expected.
|
||||
if (report.invalid_lines) {
|
||||
process.stderr.write(
|
||||
`${PRODUCT}: 跳过 ${report.invalid_lines} 行损坏的诊断记录。\n`,
|
||||
@@ -1582,6 +1622,7 @@ async function runCommand(
|
||||
limit: parsed.limit,
|
||||
character_budget: parsed.characterBudget,
|
||||
timeout_ms: parsed.timeoutMs,
|
||||
...userIdParameters(userId),
|
||||
})
|
||||
: ({
|
||||
schema_version: 2,
|
||||
@@ -1780,6 +1821,16 @@ async function runCommand(
|
||||
}
|
||||
if (named === "sources" && arguments_[1] !== "list") {
|
||||
const parsed = parseSourcesCommand(arguments_.slice(1));
|
||||
if (
|
||||
!parsed &&
|
||||
arguments_.some((item) => item === "--user-id" || item === "--root")
|
||||
) {
|
||||
return reject(
|
||||
descriptor,
|
||||
asJson,
|
||||
"--user-id 和 --root 只能用于接入单个 hermes 来源(Hermes profile),用户 ID 只能是 1–64 位字母、数字、点、下划线或连字符。",
|
||||
);
|
||||
}
|
||||
if (!parsed) return usage();
|
||||
if (parsed.kind === "receipt") {
|
||||
const connected = await connectRuntime(environment);
|
||||
@@ -1981,12 +2032,17 @@ async function runCommand(
|
||||
return sourceProgressExitCode(document);
|
||||
}
|
||||
if (named === "profile") {
|
||||
const profileFlags = arguments_.slice(1);
|
||||
if (
|
||||
arguments_.filter((item) => item === "--json").length > 1 ||
|
||||
arguments_.slice(1).some((value) => value !== "--json")
|
||||
profileFlags.filter((item) => item === "--json").length > 1 ||
|
||||
profileFlags.filter((item) => item === "--versions").length > 1 ||
|
||||
profileFlags.some((value) => value !== "--json" && value !== "--versions")
|
||||
) {
|
||||
return usage();
|
||||
}
|
||||
if (profileFlags.includes("--versions")) {
|
||||
return await profileVersionsCommand(environment, { asJson, userId });
|
||||
}
|
||||
const connected = await connectRuntime(environment, {
|
||||
readerTimeoutMs: 15_000,
|
||||
});
|
||||
@@ -2004,7 +2060,11 @@ async function runCommand(
|
||||
);
|
||||
return connected.exitCode;
|
||||
}
|
||||
const outcome = await connected.reader<unknown>("/api/v2/user-profile");
|
||||
const profileUser = userIdParameters(userId);
|
||||
const outcome =
|
||||
profileUser.user_id === undefined
|
||||
? await connected.reader<unknown>("/api/v2/user-profile")
|
||||
: await connected.reader<unknown>("/api/v2/user-profile", profileUser);
|
||||
const value =
|
||||
outcome.state === "ok"
|
||||
? (outcome.data as {
|
||||
@@ -2109,7 +2169,7 @@ async function runCommand(
|
||||
}
|
||||
const { reader, operations } = connected;
|
||||
if (parsed.kind === "list") {
|
||||
const document = await readMemoryPage(reader, parsed.query);
|
||||
const document = await readMemoryPage(reader, parsed.query, userId);
|
||||
emit(
|
||||
asJson
|
||||
? `${JSON.stringify(document)}\n`
|
||||
@@ -2122,6 +2182,7 @@ async function runCommand(
|
||||
reader,
|
||||
parsed.memoryId,
|
||||
parsed.namespace,
|
||||
userId,
|
||||
);
|
||||
emit(
|
||||
asJson
|
||||
@@ -2133,8 +2194,8 @@ async function runCommand(
|
||||
if (parsed.kind === "add" || parsed.kind === "add-status") {
|
||||
const document =
|
||||
parsed.kind === "add"
|
||||
? await addMemory(operations, text ?? "", parsed.requestId)
|
||||
: await readMemoryAddStatus(reader, parsed.requestId);
|
||||
? await addMemory(operations, text ?? "", parsed.requestId, userId)
|
||||
: await readMemoryAddStatus(reader, parsed.requestId, userId);
|
||||
emit(
|
||||
asJson
|
||||
? `${JSON.stringify(document)}\n`
|
||||
@@ -2149,9 +2210,13 @@ async function runCommand(
|
||||
content: text,
|
||||
...namespaceParameters(parsed.namespace),
|
||||
...(parsed.task === undefined ? {} : { task: parsed.task }),
|
||||
...userIdParameters(userId),
|
||||
})
|
||||
: await operations.delete<unknown>(
|
||||
withQuery(memoryPath, namespaceParameters(parsed.namespace)),
|
||||
withQuery(memoryPath, {
|
||||
...namespaceParameters(parsed.namespace),
|
||||
...userIdParameters(userId),
|
||||
}),
|
||||
);
|
||||
const document = {
|
||||
schema_version: 2,
|
||||
@@ -2177,6 +2242,55 @@ async function runCommand(
|
||||
);
|
||||
return outcome.state === "committed" ? EXIT_OK : EXIT_FAILED;
|
||||
}
|
||||
if (named === "settings") {
|
||||
const parsed = parseSettingsCommand(arguments_.slice(1));
|
||||
if (!parsed) return usage();
|
||||
const connected = await connectRuntime(environment);
|
||||
if (!connected.ok) {
|
||||
const document: ModeDocument = {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
command: "settings mode",
|
||||
state: "unavailable",
|
||||
code: connected.code,
|
||||
};
|
||||
emit(asJson ? `${JSON.stringify(document)}\n` : renderModeText(document));
|
||||
return connected.exitCode;
|
||||
}
|
||||
const document = await runModeCommand(
|
||||
parsed,
|
||||
connected.reader,
|
||||
connected.operations,
|
||||
);
|
||||
emit(asJson ? `${JSON.stringify(document)}\n` : renderModeText(document));
|
||||
return modeExitCode(document);
|
||||
}
|
||||
if (named === "evolve") {
|
||||
const parsed = parseEvolveCommand(arguments_.slice(1));
|
||||
if (!parsed) return usage();
|
||||
const connected = await connectRuntime(environment);
|
||||
if (!connected.ok) {
|
||||
const document: EvolveDocument = {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
command: "evolve",
|
||||
state: "unavailable",
|
||||
code: connected.code,
|
||||
};
|
||||
emit(
|
||||
asJson ? `${JSON.stringify(document)}\n` : renderEvolveText(document),
|
||||
);
|
||||
return connected.exitCode;
|
||||
}
|
||||
const document = await runEvolveCommand(parsed, {
|
||||
userId: userIdParameters(userId).user_id,
|
||||
reader: connected.reader,
|
||||
operations: connected.operations,
|
||||
sleep: controls.sleep,
|
||||
});
|
||||
emit(asJson ? `${JSON.stringify(document)}\n` : renderEvolveText(document));
|
||||
return evolveExitCode(document);
|
||||
}
|
||||
if (named === "graph-width") {
|
||||
const parsed = parseGraphWidthCommand(arguments_.slice(1));
|
||||
if (!parsed) return usage();
|
||||
@@ -3675,13 +3789,21 @@ async function connectSources(
|
||||
asJson,
|
||||
);
|
||||
}
|
||||
if (parsed.importHistory) {
|
||||
// Another user's Hermes profile is granted and imported in one connect
|
||||
// request: only that request can carry a user and a root.
|
||||
if (
|
||||
parsed.importHistory ||
|
||||
parsed.userId !== undefined ||
|
||||
parsed.root !== undefined
|
||||
) {
|
||||
const document = await connectSourcesWithImport(reader, operations, {
|
||||
sourceIds: parsed.sourceIds,
|
||||
confirmOverrides: parsed.confirmOverrides,
|
||||
...(parsed.requestId === undefined
|
||||
? {}
|
||||
: { requestId: parsed.requestId }),
|
||||
...(parsed.userId === undefined ? {} : { userId: parsed.userId }),
|
||||
...(parsed.root === undefined ? {} : { root: parsed.root }),
|
||||
});
|
||||
emitFlowDocument(document, asJson);
|
||||
return flowExitCode(document);
|
||||
@@ -3834,6 +3956,108 @@ async function emitModelReceipt(
|
||||
|
||||
/** Attach to a verified Runtime, telling a stopped one (exit 3) from one
|
||||
* that is up but unusable (exit 1). */
|
||||
/**
|
||||
* `memhub profile --versions`: the long-term profile's L0 version chains,
|
||||
* latest first, so a value that replaced a better one can be found by id.
|
||||
*/
|
||||
async function profileVersionsCommand(
|
||||
environment: NodeJS.ProcessEnv,
|
||||
options: { asJson: boolean; userId?: string },
|
||||
): Promise<number> {
|
||||
const fail = (state: string, code: string, exitCode: number): number => {
|
||||
emit(
|
||||
options.asJson
|
||||
? `${JSON.stringify({ schema_version: 2, product: PRODUCT, state, code })}\n`
|
||||
: `画像版本读取失败:${code}\n`,
|
||||
);
|
||||
return exitCode;
|
||||
};
|
||||
const connected = await connectRuntime(environment, {
|
||||
readerTimeoutMs: 15_000,
|
||||
});
|
||||
if (!connected.ok) {
|
||||
return fail("unavailable", connected.code, connected.exitCode);
|
||||
}
|
||||
const outcome = await connected.reader<unknown>(
|
||||
"/api/v2/user-profile/versions",
|
||||
userIdParameters(options.userId),
|
||||
);
|
||||
const chains = outcome.state === "ok" ? profileChains(outcome.data) : null;
|
||||
if (outcome.state !== "ok" || chains === null) {
|
||||
return fail(
|
||||
outcome.state === "ok" ? "invalid" : outcome.state,
|
||||
outcome.state === "ok" ? "response_invalid" : outcome.code,
|
||||
EXIT_FAILED,
|
||||
);
|
||||
}
|
||||
if (options.asJson) {
|
||||
emit(
|
||||
`${JSON.stringify({
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
state: "ok",
|
||||
...(outcome.data as object),
|
||||
})}\n`,
|
||||
);
|
||||
return EXIT_OK;
|
||||
}
|
||||
if (chains.length === 0) {
|
||||
emit("尚未形成用户画像版本。\n");
|
||||
return EXIT_OK;
|
||||
}
|
||||
const lines: string[] = [];
|
||||
for (const chain of chains) {
|
||||
lines.push(`画像 ${chain.memory_id}(${chain.status})`);
|
||||
for (const version of chain.versions) {
|
||||
const when =
|
||||
typeof version.gmt_created === "number"
|
||||
? new Date(version.gmt_created * 1000).toISOString()
|
||||
: "时间未知";
|
||||
lines.push(
|
||||
` ${version.current ? "*" : " "} ${version.memory_id} ${when} ${version.content}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
lines.push(
|
||||
"* 表示当前生效的版本;删除错误版本:memhub memories delete <memory-id> --yes",
|
||||
);
|
||||
emit(`${lines.join("\n")}\n`);
|
||||
return EXIT_OK;
|
||||
}
|
||||
|
||||
type ProfileChainView = {
|
||||
memory_id: string;
|
||||
status: string;
|
||||
versions: {
|
||||
memory_id: string;
|
||||
content: string;
|
||||
gmt_created?: number | null;
|
||||
current: boolean;
|
||||
}[];
|
||||
};
|
||||
|
||||
function profileChains(value: unknown): ProfileChainView[] | null {
|
||||
if (!value || typeof value !== "object") return null;
|
||||
const chains = (value as { chains?: unknown }).chains;
|
||||
if (!Array.isArray(chains)) return null;
|
||||
const valid = chains.every(
|
||||
(chain) =>
|
||||
chain &&
|
||||
typeof chain === "object" &&
|
||||
typeof (chain as ProfileChainView).memory_id === "string" &&
|
||||
typeof (chain as ProfileChainView).status === "string" &&
|
||||
Array.isArray((chain as ProfileChainView).versions) &&
|
||||
(chain as ProfileChainView).versions.every(
|
||||
(version) =>
|
||||
version &&
|
||||
typeof version.memory_id === "string" &&
|
||||
typeof version.content === "string" &&
|
||||
typeof version.current === "boolean",
|
||||
),
|
||||
);
|
||||
return valid ? (chains as ProfileChainView[]) : null;
|
||||
}
|
||||
|
||||
async function connectRuntime(
|
||||
environment: NodeJS.ProcessEnv,
|
||||
options?: { readerTimeoutMs?: number },
|
||||
@@ -4101,6 +4325,8 @@ const SDK_MIGRATION_FAILURES: Record<string, string> = {
|
||||
"核对发现新版本还读不到你的全部记忆,没有切换数据源。",
|
||||
sdk_migration_model_profile_required:
|
||||
"迁移需要先有可用的模型配置,用来核对新版本能否读回记忆。请先用 memhub models setup 完成配置,再运行 memhub restart 重试。",
|
||||
sdk_migration_vector_profile_missing:
|
||||
"记忆的向量来自一个已删除的模型配置,当前配置读不回它们,所以没有迁移。请重新添加并启用那个模型配置,再运行 memhub restart 重试。",
|
||||
};
|
||||
|
||||
const MIGRATION_ISSUES_URL =
|
||||
@@ -5131,14 +5357,23 @@ type SourcesCommand =
|
||||
importHistory: boolean;
|
||||
confirmOverrides: boolean;
|
||||
requestId?: string;
|
||||
/** `--user-id`: whose grant (Hermes only). Absent for `local`. */
|
||||
userId?: string;
|
||||
/** `--root`: an absolute root other than the candidate (Hermes only). */
|
||||
root?: string;
|
||||
}
|
||||
| { kind: "all"; importHistory: boolean; confirmOverrides: boolean }
|
||||
| { kind: "receipt"; requestId: string };
|
||||
|
||||
const SOURCE_ID_SHAPE = /^[a-z0-9][a-z0-9-]{0,127}$/;
|
||||
|
||||
/** Only Hermes keeps one history per profile, so only Hermes may be
|
||||
* connected for another user or from another root. */
|
||||
const MULTI_USER_SOURCES = new Set(["hermes"]);
|
||||
|
||||
/** `sources connect <id> [<id> ...] | --all-detected [--import]
|
||||
* [--confirm-overrides] [--request-id <uuid>]` and `sources receipt <id>`.
|
||||
* [--confirm-overrides] [--request-id <uuid>]
|
||||
* [--user-id <id>] [--root <path>]` and `sources receipt <id>`.
|
||||
*
|
||||
* Sources may be named positionally or with repeated `--source` (the old
|
||||
* `init --source` spelling is rewritten to this command). */
|
||||
@@ -5154,6 +5389,8 @@ function parseSourcesCommand(rest: readonly string[]): SourcesCommand | null {
|
||||
const sourceIds: string[] = [];
|
||||
const switches = new Set<string>();
|
||||
let requestId: string | undefined;
|
||||
let userId: string | undefined;
|
||||
let root: string | undefined;
|
||||
for (let index = 1; index < tokens.length; index += 1) {
|
||||
const token = tokens[index];
|
||||
if (["--import", "--confirm-overrides", "--all-detected"].includes(token)) {
|
||||
@@ -5161,6 +5398,21 @@ function parseSourcesCommand(rest: readonly string[]): SourcesCommand | null {
|
||||
switches.add(token);
|
||||
continue;
|
||||
}
|
||||
if (token === "--user-id" || token === "--root") {
|
||||
const value = tokens[index + 1];
|
||||
if (value === undefined || value.startsWith("--")) return null;
|
||||
if (token === "--user-id") {
|
||||
if (userId !== undefined || !isUserId(value)) return null;
|
||||
userId = value;
|
||||
} else {
|
||||
if (root !== undefined) return null;
|
||||
const expanded = expandRootPath(value);
|
||||
if (expanded === null) return null;
|
||||
root = expanded;
|
||||
}
|
||||
index += 1;
|
||||
continue;
|
||||
}
|
||||
if (token === "--source" || token === "--request-id") {
|
||||
const value = tokens[index + 1];
|
||||
if (value === undefined || value.startsWith("--")) return null;
|
||||
@@ -5184,10 +5436,23 @@ function parseSourcesCommand(rest: readonly string[]): SourcesCommand | null {
|
||||
const confirmOverrides = switches.has("--confirm-overrides");
|
||||
if (
|
||||
requestId !== undefined &&
|
||||
(!OPERATION_ID.test(requestId) || !importHistory)
|
||||
(!OPERATION_ID.test(requestId) ||
|
||||
!(importHistory || userId !== undefined || root !== undefined))
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
// `--user-id local` is the default and changes nothing.
|
||||
if (userId === DEFAULT_USER_ID) userId = undefined;
|
||||
if (userId !== undefined || root !== undefined) {
|
||||
// One Hermes profile per connect: the user and root name one grant.
|
||||
if (
|
||||
switches.has("--all-detected") ||
|
||||
sourceIds.length !== 1 ||
|
||||
!MULTI_USER_SOURCES.has(sourceIds[0])
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
if (switches.has("--all-detected")) {
|
||||
// One names exact sources, the other means "whatever is detected".
|
||||
if (sourceIds.length > 0 || requestId !== undefined) return null;
|
||||
@@ -5200,9 +5465,24 @@ function parseSourcesCommand(rest: readonly string[]): SourcesCommand | null {
|
||||
importHistory,
|
||||
confirmOverrides,
|
||||
...(requestId === undefined ? {} : { requestId }),
|
||||
...(userId === undefined ? {} : { userId }),
|
||||
...(root === undefined ? {} : { root }),
|
||||
};
|
||||
}
|
||||
|
||||
/** `--root` as an absolute path: a leading `~` is the home directory, a
|
||||
* relative path is taken from the working directory. Core expands nothing. */
|
||||
function expandRootPath(value: string): string | null {
|
||||
if (value.length === 0 || /[\0\r\n]/.test(value)) return null;
|
||||
const expanded =
|
||||
value === "~"
|
||||
? os.homedir()
|
||||
: value.startsWith("~/")
|
||||
? path.join(os.homedir(), value.slice(2))
|
||||
: value;
|
||||
return path.resolve(expanded);
|
||||
}
|
||||
|
||||
function parseGraphWidthCommand(
|
||||
rest: readonly string[],
|
||||
): { decision: GraphWidthDecision; confirmed: boolean } | null {
|
||||
@@ -6596,6 +6876,36 @@ const OWN_DATA_DIR_COMMANDS = new Set([
|
||||
|
||||
/** Remove one `--data-dir <absolute path>` from argv; null when it is
|
||||
* repeated, missing its value, or not an absolute path. */
|
||||
/** Memory commands that honour the global `--user-id`. */
|
||||
const USER_ID_COMMANDS = new Set([
|
||||
"memories",
|
||||
"recall",
|
||||
"profile",
|
||||
"write",
|
||||
"evolve",
|
||||
]);
|
||||
|
||||
/**
|
||||
* Take `--user-id <id>` out of the arguments. The default user (`local`) is
|
||||
* returned as undefined, so a default request names no user at all and stays
|
||||
* byte-identical to one sent before multi-user support.
|
||||
*/
|
||||
function liftUserIdOption(
|
||||
arguments_: readonly string[],
|
||||
): { arguments_: string[]; userId?: string } | null {
|
||||
const index = arguments_.indexOf("--user-id");
|
||||
if (index === -1) return { arguments_: [...arguments_] };
|
||||
if (arguments_.indexOf("--user-id", index + 1) !== -1) return null;
|
||||
const value = arguments_[index + 1];
|
||||
if (value === undefined || value.startsWith("--") || !isUserId(value)) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
arguments_: [...arguments_.slice(0, index), ...arguments_.slice(index + 2)],
|
||||
...(value === DEFAULT_USER_ID ? {} : { userId: value }),
|
||||
};
|
||||
}
|
||||
|
||||
function liftDataDirOption(
|
||||
arguments_: readonly string[],
|
||||
): { arguments_: string[]; dataDir?: string } | null {
|
||||
|
||||
@@ -68,6 +68,12 @@ const JSON_OPTION: CommandOption = {
|
||||
flag: "--json",
|
||||
summary: "以 JSON 输出到 stdout,供脚本与 Agent 消费。",
|
||||
};
|
||||
/** Global to the memory commands: whose memories are read or written. */
|
||||
const USER_ID_OPTION: CommandOption = {
|
||||
flag: "--user-id <id>",
|
||||
summary:
|
||||
"读写哪个用户的记忆,默认 local(App 用的就是 local)。1–64 位字母、数字、点、下划线或连字符;其他用户的记忆来自用 sources connect hermes --user-id 接入的 Hermes profile。",
|
||||
};
|
||||
const WATCH_OPTION: CommandOption = {
|
||||
flag: "--watch",
|
||||
summary: "串行持续刷新;Ctrl-C 只停止观察,不停止 Runtime。",
|
||||
@@ -252,15 +258,25 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
{
|
||||
name: "sources",
|
||||
summary:
|
||||
"列出来源,直接接入来源,或按 request id 查接入回执。connect 不加 --import 只授权(处理保持冻结:不导入历史、不扫描新会话);加 --import 同时导入历史并开启持续同步,返回 request id。目录来自环境变量覆盖的来源必须加 --confirm-overrides。",
|
||||
"列出来源,直接接入来源,或按 request id 查接入回执。connect 不加 --import 只授权(处理保持冻结:不导入历史、不扫描新会话);加 --import 同时导入历史并开启持续同步,返回 request id。目录来自环境变量覆盖的来源必须加 --confirm-overrides。Hermes 的每个 profile 是一个独立用户:用 --user-id 和 --root 接入,授权并导入历史(会自动排除 ~/.hermes/profiles/**,各用户互不读取)。",
|
||||
usage:
|
||||
"memhub sources list [--json]\n memhub sources connect <id> [<id> ...] [--import] [--confirm-overrides] [--request-id <uuid>] [--json]\n memhub sources connect --all-detected [--import] [--confirm-overrides] [--json]\n memhub sources receipt <request-id> [--json]",
|
||||
"memhub sources list [--json]\n memhub sources connect <id> [<id> ...] [--import] [--confirm-overrides] [--request-id <uuid>] [--json]\n memhub sources connect hermes --user-id <id> --root <path> [--import] [--request-id <uuid>] [--json]\n memhub sources connect --all-detected [--import] [--confirm-overrides] [--json]\n memhub sources receipt <request-id> [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--import",
|
||||
summary:
|
||||
"接入后立即导入历史会话并开启持续同步;不加时只授权,处理保持冻结。",
|
||||
},
|
||||
{
|
||||
flag: "--user-id <id>",
|
||||
summary:
|
||||
"只用于 hermes:这条接入属于哪个用户(默认 local)。和 --root 一起接入一个 Hermes profile,例如 --user-id wife --root ~/.hermes/profiles/wife;其他来源只能是 local。",
|
||||
},
|
||||
{
|
||||
flag: "--root <path>",
|
||||
summary:
|
||||
"只用于 hermes:读取的根目录(~ 会展开为主目录)。默认是 Hermes 自己的目录 ~/.hermes。",
|
||||
},
|
||||
{
|
||||
flag: "--all-detected",
|
||||
summary:
|
||||
@@ -373,7 +389,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
summary:
|
||||
"列出、搜索、查看、添加、修改或物理删除记忆(对话、工作、编码三个场景)。add 和 App 的「添加记忆」是同一个入口;delete 不可逆,需 --yes。",
|
||||
usage:
|
||||
"memhub memories list [--scene <all|chat|working|coding>] [--source <id>] [--query <text>] [--status <state>] [--offset <n>] [--limit <n>] [--json]\n memhub memories show <memory-id> [--kind <kind>] [--variant <variant>] [--json]\n memhub memories add <text> [--request-id <uuid>] [--json]\n memhub memories add --content-file <path> [--request-id <uuid>] [--json]\n memhub memories add-status <request-id> [--json]\n memhub memories update <memory-id> --content-file <path> [--kind <kind>] [--variant <variant>] [--task <text>] [--json]\n memhub memories delete <memory-id> [--kind <kind>] [--variant <variant>] [--yes] [--json]",
|
||||
"memhub memories list [--scene <all|chat|working|coding>] [--source <id>] [--query <text>] [--status <state>] [--offset <n>] [--limit <n>] [--user-id <id>] [--json]\n memhub memories show <memory-id> [--kind <kind>] [--variant <variant>] [--user-id <id>] [--json]\n memhub memories add <text> [--request-id <uuid>] [--user-id <id>] [--json]\n memhub memories add --content-file <path> [--request-id <uuid>] [--user-id <id>] [--json]\n memhub memories add-status <request-id> [--user-id <id>] [--json]\n memhub memories update <memory-id> --content-file <path> [--kind <kind>] [--variant <variant>] [--task <text>] [--user-id <id>] [--json]\n memhub memories delete <memory-id> [--kind <kind>] [--variant <variant>] [--yes] [--user-id <id>] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--scene <all|chat|working|coding>",
|
||||
@@ -411,15 +427,62 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
"add 的幂等键;用同一个 id 重试只会读回第一次的结果,不会重复添加。不给时自动生成并输出。",
|
||||
},
|
||||
DESTRUCTIVE_CONFIRM_OPTION,
|
||||
USER_ID_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "profile",
|
||||
summary: "读取 SDK 组装后的用户长期画像(L0 + 已启用的长期画像补充)。",
|
||||
usage: "memhub profile [--json]",
|
||||
options: [JSON_OPTION],
|
||||
summary:
|
||||
"读取 SDK 组装后的用户长期画像(L0 + 已启用的长期画像补充);--versions 列出 L0 画像的版本链(最新在前,* 为当前版本)。",
|
||||
usage: "memhub profile [--versions] [--user-id <id>] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--versions",
|
||||
summary:
|
||||
"列出 L0 基本信息的版本链(get_user_basic_info),用于找出覆盖了正确值的错误版本。",
|
||||
},
|
||||
USER_ID_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "settings",
|
||||
summary:
|
||||
"查看或切换记忆模式(lite / pro / ultra)。lite 不调用模型,只在原始对话里检索;pro 由模型提取记忆(System 1);ultra 在 pro 之上每晚自动进化(System 2,使用知识图谱)。任何模式都保留全部原始对话。切换不迁移数据:已处理的不会重做,排队中的和新的按新模式处理;需 --yes,不加时只显示计划。环境变量 ONE_MEMORY_GRAPH_STORE 会强制 ultra。",
|
||||
usage: "memhub settings mode [lite|pro|ultra] [--yes] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "mode [lite|pro|ultra]",
|
||||
summary: "不带模式只显示当前模式;带模式时先显示切换计划。",
|
||||
},
|
||||
CONFIRM_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "evolve",
|
||||
summary:
|
||||
"立即为一个用户运行记忆进化(System 2,仅 ultra 模式)。在后台运行并返回运行 ID;--wait 等待结果。同一用户同时只运行一次;每晚的自动进化由 Runtime 按设置安排。",
|
||||
usage:
|
||||
"memhub evolve [--user-id <id>] [--agent-id <id>] [--max-batches <n>] [--wait] [--json]",
|
||||
options: [
|
||||
USER_ID_OPTION,
|
||||
{
|
||||
flag: "--agent-id <id>",
|
||||
summary:
|
||||
"进化哪个 agent 的记忆,默认 all-apps(MemHub 写入记忆用的 agent)。",
|
||||
},
|
||||
{
|
||||
flag: "--max-batches <n>",
|
||||
summary: "每轮最多处理的批次数(1–1000),默认由 SDK 决定。",
|
||||
},
|
||||
{ flag: "--wait", summary: "等待这次进化结束再返回。" },
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
@@ -475,7 +538,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
summary:
|
||||
"从原始对话与已沉淀记忆统一检索上下文;本地只读,不调用模型、不触发补索引。文本输出标明每条是「原始对话摘录」还是「记忆」(JSON 里是 kind: raw|memory)。",
|
||||
usage:
|
||||
"memhub recall <query> [--kind <raw|memory>] [--source <id>] [--limit <n>] [--character-budget <n>] [--timeout-ms <n>] [--json]\n memhub recall --query-file <path> [...]\n memhub recall --stdin [...]",
|
||||
"memhub recall <query> [--kind <raw|memory>] [--source <id>] [--limit <n>] [--character-budget <n>] [--timeout-ms <n>] [--user-id <id>] [--json]\n memhub recall --query-file <path> [...]\n memhub recall --stdin [...]",
|
||||
options: [
|
||||
{
|
||||
flag: "--query-file <path>",
|
||||
@@ -502,6 +565,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
flag: "--timeout-ms <n>",
|
||||
summary: "Core 查询时限 50..30000 毫秒,默认 2000。",
|
||||
},
|
||||
USER_ID_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
@@ -650,7 +714,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
summary:
|
||||
"提交一段 OpenAI 格式的对话(messages)到写入队列,异步加工成记忆;或按 request id 查询状态。request id 是幂等键:不给时自动生成并输出,重试请带上同一个。新增一条结论式记忆请用 memories add。",
|
||||
usage:
|
||||
"memhub write submit --messages-file <path> --session <id> [--source <id>] [--request-id <id>] [--memory-at <RFC3339>] [--live] [--json]\n memhub write status <request-id> [--json]",
|
||||
"memhub write submit --messages-file <path> --session <id> [--source <id>] [--request-id <id>] [--memory-at <RFC3339>] [--live] [--user-id <id>] [--json]\n memhub write status <request-id> [--user-id <id>] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--messages-file <path>",
|
||||
@@ -666,6 +730,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
},
|
||||
{ flag: "--memory-at <RFC3339>", summary: "对话发生时间,必须带时区。" },
|
||||
{ flag: "--live", summary: "优先处理。" },
|
||||
USER_ID_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
|
||||
@@ -76,6 +76,8 @@ const COMMANDS = [
|
||||
"models",
|
||||
"memories",
|
||||
"profile",
|
||||
"settings",
|
||||
"evolve",
|
||||
"graph-width",
|
||||
"host",
|
||||
"recall",
|
||||
|
||||
@@ -10,6 +10,8 @@ export interface ContextSearchInput {
|
||||
limit: number;
|
||||
character_budget: number;
|
||||
timeout_ms: number;
|
||||
/** Whose memories to search. Omitted for the default user (`local`). */
|
||||
user_id?: string;
|
||||
}
|
||||
|
||||
/** The CLI's JSON document mirrors Core's SDK-native context schema v2. */
|
||||
|
||||
@@ -12,11 +12,18 @@ const READ_ONLY_PATHS = [
|
||||
"/api/v1/live-sync",
|
||||
"/api/v1/memory-processing",
|
||||
"/api/v1/model-profiles",
|
||||
"/api/v1/source-grants",
|
||||
"/api/v1/sources",
|
||||
"/api/v2/home-summary",
|
||||
"/api/v2/memory-library",
|
||||
"/api/v2/user-profile",
|
||||
"/api/v2/user-profile/versions",
|
||||
"/api/v1/settings/memory-mode",
|
||||
"/api/v2/evolve",
|
||||
] as const;
|
||||
/** One evolve run's receipt (`memhub evolve --wait`). */
|
||||
const EVOLVE_RUN =
|
||||
/^\/api\/v2\/evolve\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
const SDK_MEMORY_DETAIL = /^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}$/;
|
||||
/** Status of one `POST /api/v2/memories` addition, by its request id. */
|
||||
const MEMORY_ADDITION =
|
||||
@@ -219,6 +226,7 @@ export function isReadOnlyPath(path: string): boolean {
|
||||
if (IMPORT_JOB.test(path)) return true;
|
||||
if (OPERATION_RECEIPT.test(path)) return true;
|
||||
if (MEMORY_ADDITION.test(path)) return true;
|
||||
if (EVOLVE_RUN.test(path)) return true;
|
||||
if (!SDK_MEMORY_DETAIL.test(path)) return false;
|
||||
const segment = path.slice("/api/v2/memories/".length);
|
||||
return !(RESERVED_MEMORY_SEGMENTS as readonly string[]).includes(segment);
|
||||
@@ -310,6 +318,8 @@ export async function submitMemoryWrite(
|
||||
messages: readonly OpenAIMessage[];
|
||||
memory_at?: string;
|
||||
priority?: "live" | "normal";
|
||||
/** Whose memory this is. Omitted for the default user (`local`). */
|
||||
user_id?: string;
|
||||
},
|
||||
): Promise<MemoryWriteOutcome> {
|
||||
let response: { status: number; data: unknown };
|
||||
@@ -327,6 +337,9 @@ export async function submitMemoryWrite(
|
||||
...(request.priority === undefined
|
||||
? {}
|
||||
: { priority: request.priority }),
|
||||
...(request.user_id === undefined || request.user_id === "local"
|
||||
? {}
|
||||
: { user_id: request.user_id }),
|
||||
},
|
||||
{
|
||||
headers: { Authorization: `Bearer ${attachment.apiToken}` },
|
||||
@@ -469,11 +482,16 @@ export async function deleteModelProfile(
|
||||
export async function readMemoryWrite(
|
||||
attachment: RuntimeAttachmentLike,
|
||||
requestId: string,
|
||||
/** A receipt belongs to the user that submitted it; omit for `local`. */
|
||||
userId?: string,
|
||||
): Promise<MemoryWriteReceipt | null> {
|
||||
try {
|
||||
const response = await axios.get<MemoryWriteReceipt>(
|
||||
`${baseUrlOf(attachment)}/api/v2/memory-writes/${encodeURIComponent(requestId)}`,
|
||||
{
|
||||
...(userId === undefined || userId === "local"
|
||||
? {}
|
||||
: { params: { user_id: userId } }),
|
||||
headers: { Authorization: `Bearer ${attachment.apiToken}` },
|
||||
timeout: 10_000,
|
||||
proxy: false,
|
||||
|
||||
@@ -6,9 +6,10 @@ const WRITE_PATHS = [
|
||||
] as const;
|
||||
/** `POST /api/v2/memories` adds one memory (the app's 添加记忆). */
|
||||
const MEMORY_ADD_PATH = "/api/v2/memories";
|
||||
/** One SDK memory; delete names its namespace in the query string. */
|
||||
/** One SDK memory; delete names its namespace, and a non-default user, in
|
||||
* the query string. */
|
||||
const SDK_MEMORY_PATH =
|
||||
/^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}(?:\?kind=(?:chat|experience|fact)(?:&variant=(?:working|coding))?)?$/;
|
||||
/^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}(?:\?(?:kind=(?:chat|experience|fact)(?:&variant=(?:working|coding))?(?:&user_id=[A-Za-z0-9._-]{1,64})?|user_id=[A-Za-z0-9._-]{1,64}))?$/;
|
||||
const MODEL_PROBE_PATH =
|
||||
/^\/api\/v1\/model-profiles\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\/probe$/;
|
||||
const MODEL_PROFILE_PATH =
|
||||
@@ -25,6 +26,9 @@ const SOURCE_IMPORT_JOB_PATH = "/api/v1/import-jobs";
|
||||
const IMPORT_JOB_ACTION_PATH =
|
||||
/^\/api\/v1\/import-jobs\/[A-Za-z0-9][\w.-]{0,127}\/(?:pause|resume|cancel)$/;
|
||||
const GRAPH_WIDTH_PATH = "/api/v1/graph-width";
|
||||
/** `memhub evolve` starts a run; `memhub settings mode <m> --yes` sets the mode. */
|
||||
const EVOLVE_PATH = "/api/v2/evolve";
|
||||
const MEMORY_MODE_PATH = "/api/v1/settings/memory-mode";
|
||||
const ONBOARDING_COMPLETE_PATH = "/api/v1/onboarding/complete";
|
||||
const MAX_BYTES = 4 * 1024 * 1024;
|
||||
|
||||
@@ -96,6 +100,7 @@ export function createCoreOperations(attachment: {
|
||||
!SOURCE_AUTHORIZE_PATH.test(path) &&
|
||||
!SOURCE_REVOKE_PATH.test(path) &&
|
||||
path !== GRAPH_WIDTH_PATH &&
|
||||
path !== EVOLVE_PATH &&
|
||||
path !== ONBOARDING_COMPLETE_PATH &&
|
||||
!MODEL_PROFILE_ACTIVATE.test(path)
|
||||
) {
|
||||
@@ -131,7 +136,10 @@ export function createCoreOperations(attachment: {
|
||||
body: Record<string, unknown>,
|
||||
signal?: AbortSignal,
|
||||
) {
|
||||
if (!SDK_MEMORY_PATH.test(path) || path.includes("?")) {
|
||||
if (
|
||||
path !== MEMORY_MODE_PATH &&
|
||||
(!SDK_MEMORY_PATH.test(path) || path.includes("?"))
|
||||
) {
|
||||
throw new Error("core_operation_path_not_allowed");
|
||||
}
|
||||
let response: { status: number; data: unknown };
|
||||
|
||||
@@ -110,9 +110,18 @@ export async function connectSourcesWithImport(
|
||||
sourceIds: readonly string[];
|
||||
confirmOverrides: boolean;
|
||||
requestId?: string;
|
||||
/** Whose grant this is. Omitted for the default user (`local`). */
|
||||
userId?: string;
|
||||
/** An absolute root other than the source's candidate (a Hermes
|
||||
* profile). Core reads its identity itself, so the source's candidate
|
||||
* root id and revision are not sent for it. */
|
||||
root?: string;
|
||||
},
|
||||
): Promise<FlowDocument> {
|
||||
const sourceIds = input.sourceIds;
|
||||
// A second user's grant is not a first run, and a custom root has no
|
||||
// candidate the app showed: both are a plain connect.
|
||||
const profile = input.root !== undefined || input.userId !== undefined;
|
||||
if (
|
||||
sourceIds.length < 1 ||
|
||||
sourceIds.length > 32 ||
|
||||
@@ -134,9 +143,14 @@ export async function connectSourcesWithImport(
|
||||
return invalid("source_unavailable");
|
||||
}
|
||||
const items = selected.map(([candidate]) => candidate);
|
||||
const needsOverride = items
|
||||
.filter((item) => item.requires_override_confirmation)
|
||||
.map((item) => item.source_id);
|
||||
// A custom root is the caller's own choice, not the environment override
|
||||
// the confirmation exists for.
|
||||
const needsOverride =
|
||||
input.root !== undefined
|
||||
? []
|
||||
: items
|
||||
.filter((item) => item.requires_override_confirmation)
|
||||
.map((item) => item.source_id);
|
||||
if (needsOverride.length > 0 && !input.confirmOverrides) {
|
||||
return {
|
||||
schema_version: 1,
|
||||
@@ -154,18 +168,22 @@ export async function connectSourcesWithImport(
|
||||
request = buildSourceImportRequest({
|
||||
sourceIds: [...sourceIds],
|
||||
sourceRoots: Object.fromEntries(
|
||||
items.map((item) => [item.source_id, item.candidate_root_id]),
|
||||
items.flatMap((item) =>
|
||||
input.root === undefined
|
||||
? [[item.source_id, item.candidate_root_id]]
|
||||
: [],
|
||||
),
|
||||
),
|
||||
sourceCandidateRevisions: Object.fromEntries(
|
||||
items.flatMap((item) =>
|
||||
item.candidate_revision === undefined
|
||||
item.candidate_revision === undefined || input.root !== undefined
|
||||
? []
|
||||
: [[item.source_id, item.candidate_revision]],
|
||||
),
|
||||
),
|
||||
confirmEnvironmentOverrides: needsOverride.length > 0,
|
||||
fullRescan: false,
|
||||
onboarding: true,
|
||||
onboarding: !profile,
|
||||
enableSync: true,
|
||||
requestId,
|
||||
});
|
||||
@@ -175,7 +193,11 @@ export async function connectSourcesWithImport(
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
const result = await operations.post<unknown>(request.path, request.body);
|
||||
const result = await operations.post<unknown>(request.path, {
|
||||
...request.body,
|
||||
...(input.userId === undefined ? {} : { user_id: input.userId }),
|
||||
...(input.root === undefined ? {} : { root: input.root }),
|
||||
});
|
||||
if (result.state === "rejected") {
|
||||
return rejected(requestId, result.code);
|
||||
}
|
||||
|
||||
@@ -107,7 +107,11 @@ export async function readLogReport(
|
||||
query: LogQuery,
|
||||
): Promise<{
|
||||
events: DiagnosticEventV1[];
|
||||
/** Lines that are not JSON: the only ones `memhub logs` calls damaged. */
|
||||
invalid_lines: number;
|
||||
/** Whole JSON lines the reader leaves out, such as an older build's
|
||||
* records. Nothing is wrong with them, so the text output says nothing. */
|
||||
unrecognized_lines: number;
|
||||
failure?: "log_directory_unreadable";
|
||||
}> {
|
||||
// `readDiagnosticEvents` treats an unreadable logs path as "no events". That
|
||||
@@ -119,6 +123,7 @@ export async function readLogReport(
|
||||
return {
|
||||
events: [],
|
||||
invalid_lines: 0,
|
||||
unrecognized_lines: 0,
|
||||
failure: "log_directory_unreadable",
|
||||
};
|
||||
}
|
||||
@@ -129,6 +134,7 @@ export async function readLogReport(
|
||||
return {
|
||||
events: [],
|
||||
invalid_lines: 0,
|
||||
unrecognized_lines: 0,
|
||||
failure: "log_directory_unreadable",
|
||||
};
|
||||
}
|
||||
|
||||
@@ -0,0 +1,309 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { run } from "./cli.js";
|
||||
import { modeEffects, parseSettingsCommand } from "./memory-mode-flow.js";
|
||||
import { composeStatusReport, renderStatusText } from "./status-report.js";
|
||||
|
||||
const transport = vi.hoisted(() => ({
|
||||
get: vi.fn(),
|
||||
post: vi.fn(),
|
||||
put: vi.fn(),
|
||||
patch: vi.fn(),
|
||||
delete: vi.fn(),
|
||||
}));
|
||||
vi.mock("axios", () => ({
|
||||
default: {
|
||||
create: vi.fn(() => transport),
|
||||
get: vi.fn(),
|
||||
post: vi.fn(),
|
||||
isAxiosError: () => false,
|
||||
},
|
||||
}));
|
||||
|
||||
const runtime = vi.hoisted(() => ({
|
||||
inspectRuntimeStatus: vi.fn(),
|
||||
discoverRuntimeAttachment: vi.fn(),
|
||||
}));
|
||||
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
|
||||
...runtime,
|
||||
}));
|
||||
|
||||
const RUN_ID = "11111111-2222-4333-8444-555555555555";
|
||||
|
||||
function modeState(mode: string, configured = mode) {
|
||||
return {
|
||||
mode,
|
||||
configured_mode: configured,
|
||||
forced_by_env: false,
|
||||
modes: ["lite", "pro", "ultra"],
|
||||
graph_build: { state: "idle", users: {} },
|
||||
};
|
||||
}
|
||||
|
||||
function receipt(state: string, extra: Record<string, unknown> = {}) {
|
||||
return {
|
||||
run_id: RUN_ID,
|
||||
user_id: "wife",
|
||||
agent_id: "all-apps",
|
||||
state,
|
||||
trigger: "manual",
|
||||
started_at: "2026-11-01T03:00:00Z",
|
||||
rounds: state === "running" ? 0 : 2,
|
||||
tasks_processed: state === "running" ? 0 : 5,
|
||||
batch_failures: 0,
|
||||
capped: false,
|
||||
elapsed_ms: 10,
|
||||
results: { schemas: ["never printed"] },
|
||||
...extra,
|
||||
};
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
for (const method of Object.values(transport)) method.mockReset();
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "running",
|
||||
code: "runtime_verified",
|
||||
instanceId: "instance-1",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
version: "1.2.3",
|
||||
});
|
||||
runtime.discoverRuntimeAttachment.mockResolvedValue({
|
||||
instanceId: "instance-1",
|
||||
apiToken: "synthetic-runtime-token",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
servicePid: 4242,
|
||||
health: {
|
||||
status: "ok",
|
||||
service: "one-memory-core",
|
||||
version: "1.2.3",
|
||||
instance_id: "instance-1",
|
||||
},
|
||||
});
|
||||
transport.get.mockImplementation(async (url: string) =>
|
||||
url === "/api/v1/settings/memory-mode"
|
||||
? { status: 200, data: modeState("pro") }
|
||||
: { status: 404, data: {} },
|
||||
);
|
||||
});
|
||||
|
||||
describe("memhub settings mode", () => {
|
||||
it("shows the current mode without changing it", async () => {
|
||||
const result = await capturedRun(["settings", "mode", "--json"]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(JSON.parse(result.stdout).mode.mode).toBe("pro");
|
||||
expect(transport.put).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("prints the plan and changes nothing without --yes", async () => {
|
||||
const result = await capturedRun(["settings", "mode", "lite"]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.put).not.toHaveBeenCalled();
|
||||
expect(result.stdout).toContain("更高模式生成的记忆仍保存着");
|
||||
expect(result.stdout).toContain("memhub settings mode lite --yes");
|
||||
const json = JSON.parse(
|
||||
(await capturedRun(["settings", "mode", "ultra", "--json"])).stdout,
|
||||
);
|
||||
expect(json.state).toBe("preview");
|
||||
expect(json.applied).toBe(false);
|
||||
expect(json.effects).toEqual(["existing_memories_added_to_graph"]);
|
||||
});
|
||||
|
||||
it("changes the mode with --yes and reports what changed", async () => {
|
||||
transport.put.mockResolvedValue({
|
||||
status: 200,
|
||||
data: {
|
||||
...modeState("lite"),
|
||||
previous_mode: "pro",
|
||||
effects: ["higher_mode_memories_hidden"],
|
||||
},
|
||||
});
|
||||
const result = await capturedRun(["settings", "mode", "lite", "--yes"]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.put).toHaveBeenCalledWith(
|
||||
"/api/v1/settings/memory-mode",
|
||||
{ mode: "lite" },
|
||||
undefined,
|
||||
);
|
||||
expect(result.stdout).toContain("已切换");
|
||||
expect(result.stdout).toContain("原始对话:全部保留");
|
||||
});
|
||||
|
||||
it("does nothing for the mode already set, and refuses bad input", async () => {
|
||||
const same = await capturedRun(["settings", "mode", "pro", "--yes"]);
|
||||
expect(same.code).toBe(0);
|
||||
expect(same.stdout).toContain("已经是这个模式");
|
||||
expect(transport.put).not.toHaveBeenCalled();
|
||||
for (const bad of [
|
||||
["settings"],
|
||||
["settings", "mode", "max"],
|
||||
["settings", "mode", "--yes"],
|
||||
["settings", "mode", "lite", "pro"],
|
||||
["settings", "language", "en"],
|
||||
]) {
|
||||
expect((await capturedRun(bad)).code, bad.join(" ")).toBe(2);
|
||||
}
|
||||
});
|
||||
|
||||
it("plans the same effects Core reports", () => {
|
||||
expect(modeEffects("lite", "pro")).toEqual([
|
||||
"earlier_raw_not_extracted",
|
||||
"lite_search_stops",
|
||||
]);
|
||||
expect(modeEffects("ultra", "lite")).toEqual([
|
||||
"higher_mode_memories_hidden",
|
||||
"evolve_stopped",
|
||||
]);
|
||||
expect(modeEffects("pro", "pro")).toEqual([]);
|
||||
expect(parseSettingsCommand(["mode", "ultra", "--yes"])).toEqual({
|
||||
kind: "set",
|
||||
mode: "ultra",
|
||||
confirmed: true,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("memhub evolve", () => {
|
||||
it("starts a run for the named user and returns at once", async () => {
|
||||
transport.post.mockResolvedValue({ status: 202, data: receipt("running") });
|
||||
const result = await capturedRun([
|
||||
"evolve",
|
||||
"--user-id",
|
||||
"wife",
|
||||
"--agent-id",
|
||||
"all-apps",
|
||||
"--max-batches",
|
||||
"3",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
"/api/v2/evolve",
|
||||
{ user_id: "wife", agent_id: "all-apps", max_batches: 3 },
|
||||
undefined,
|
||||
);
|
||||
const document = JSON.parse(result.stdout);
|
||||
expect(document.state).toBe("running");
|
||||
expect(document.run.results).toBeUndefined();
|
||||
});
|
||||
|
||||
it("follows the run with --wait until it ends", async () => {
|
||||
transport.post.mockResolvedValue({ status: 202, data: receipt("running") });
|
||||
const reads = [receipt("running"), receipt("succeeded")];
|
||||
transport.get.mockImplementation(async (url: string) =>
|
||||
url === `/api/v2/evolve/${RUN_ID}`
|
||||
? { status: 200, data: reads.shift() }
|
||||
: { status: 404, data: {} },
|
||||
);
|
||||
const result = await capturedRun(["evolve", "--wait"], {
|
||||
sleep: async () => undefined,
|
||||
});
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
"/api/v2/evolve",
|
||||
{},
|
||||
undefined,
|
||||
);
|
||||
expect(result.stdout).toContain("进化完成");
|
||||
expect(result.stdout).toContain("2 轮");
|
||||
});
|
||||
|
||||
it("names the mode when Core refuses outside ultra", async () => {
|
||||
transport.post.mockResolvedValue({
|
||||
status: 409,
|
||||
data: { detail: "memory_mode_not_ultra" },
|
||||
});
|
||||
const result = await capturedRun(["evolve"]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(result.stdout).toContain("memhub settings mode ultra --yes");
|
||||
expect((await capturedRun(["evolve", "--max-batches", "0"])).code).toBe(2);
|
||||
expect((await capturedRun(["evolve", "--now"])).code).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe("status pipeline", () => {
|
||||
it("reports pipeline.mode and the last evolve from memory-processing", async () => {
|
||||
const processing = {
|
||||
pending: 0,
|
||||
fresh_pending: 0,
|
||||
backlog_pending: 0,
|
||||
outside_window_pending: 0,
|
||||
running: 0,
|
||||
waiting_for_model: 0,
|
||||
completed: 3,
|
||||
failed: 0,
|
||||
blocked_stages: [],
|
||||
pipeline: {
|
||||
mode: "ultra",
|
||||
configured_mode: "ultra",
|
||||
forced_by_env: false,
|
||||
last_evolve_at: "2026-11-01T03:00:05+08:00",
|
||||
graph_build: { state: "succeeded", users: {} },
|
||||
evolve: {
|
||||
mode: "ultra",
|
||||
schedule: { enabled: true, time: "03:00" },
|
||||
running: [],
|
||||
users: {
|
||||
local: {
|
||||
last_run_at: "2026-11-01T03:00:05+08:00",
|
||||
last_result: { state: "succeeded", tasks_processed: 4 },
|
||||
last_error: null,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
const report = await composeStatusReport({
|
||||
dataDir: "/tmp/isolated",
|
||||
runtime: { state: "running", code: "ok", host: "127.0.0.1", port: 1 },
|
||||
background: null,
|
||||
reader: (async (path: string) =>
|
||||
path === "/api/v1/memory-processing"
|
||||
? { state: "ok", data: processing }
|
||||
: { state: "unavailable", code: "x" }) as never,
|
||||
});
|
||||
expect(report.pipeline?.mode).toBe("ultra");
|
||||
expect(report.pipeline?.last_evolve_at).toBe("2026-11-01T03:00:05+08:00");
|
||||
expect(report.pipeline?.evolve_users.local?.state).toBe("succeeded");
|
||||
const text = renderStatusText(report, "MemHub");
|
||||
expect(text).toContain("记忆模式:ultra");
|
||||
expect(text).toContain("每日 03:00 自动进化");
|
||||
});
|
||||
});
|
||||
|
||||
async function capturedRun(
|
||||
arguments_: string[],
|
||||
controls: Parameters<typeof run>[2] = {},
|
||||
) {
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
const stdoutSpy = vi
|
||||
.spyOn(process.stdout, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stdout += String(chunk);
|
||||
return true;
|
||||
});
|
||||
const stderrSpy = vi
|
||||
.spyOn(process.stderr, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stderr += String(chunk);
|
||||
return true;
|
||||
});
|
||||
try {
|
||||
return {
|
||||
code: await run(
|
||||
arguments_,
|
||||
{ ONE_MEMORY_DATA_DIR: "/synthetic/data" },
|
||||
controls,
|
||||
),
|
||||
stdout,
|
||||
stderr,
|
||||
};
|
||||
} finally {
|
||||
stdoutSpy.mockRestore();
|
||||
stderrSpy.mockRestore();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,372 @@
|
||||
/**
|
||||
* `memhub settings mode` and `memhub evolve`.
|
||||
*
|
||||
* The mode (lite / pro / ultra) decides what the SDK does with every
|
||||
* conversation, and switching it migrates nothing: what one mode made is not
|
||||
* shown by a lower one, and raw kept by lite is not extracted until it is
|
||||
* imported again. So a change prints its plan first and needs `--yes`, like
|
||||
* every other command whose effect a person cannot simply take back.
|
||||
*
|
||||
* `memhub evolve` starts System 2 (ultra only) and, with `--wait`, follows
|
||||
* the run until it ends.
|
||||
*/
|
||||
|
||||
import type { CoreReader } from "./core-client.js";
|
||||
import type { CoreOperations, CorePutOperations } from "./core-operations.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
export const MEMORY_MODES = ["lite", "pro", "ultra"] as const;
|
||||
export type MemoryMode = (typeof MEMORY_MODES)[number];
|
||||
|
||||
export type ModeEffect =
|
||||
| "earlier_raw_not_extracted"
|
||||
| "higher_mode_memories_hidden"
|
||||
| "existing_memories_added_to_graph"
|
||||
| "evolve_stopped"
|
||||
| "lite_search_stops"
|
||||
| "graph_backfill_already_running";
|
||||
|
||||
const MODE_SUMMARY: Record<MemoryMode, string> = {
|
||||
lite: "lite:不调用模型,保留全部原始对话,只在原始对话里检索",
|
||||
pro: "pro:模型从对话中提取记忆(System 1)",
|
||||
ultra: "ultra:pro + 每晚自动进化(System 2,使用知识图谱)",
|
||||
};
|
||||
|
||||
const EFFECT_TEXT: Record<ModeEffect, string> = {
|
||||
earlier_raw_not_extracted:
|
||||
"lite 下已处理完的对话不会再被提取;只有尚未处理的和之后的新对话会由模型整理成记忆,检索立即改为记忆检索",
|
||||
higher_mode_memories_hidden:
|
||||
"更高模式生成的记忆仍保存着,但在这个模式下不会显示",
|
||||
existing_memories_added_to_graph:
|
||||
"已有记忆会在后台加入知识图谱(每个用户一次)",
|
||||
evolve_stopped: "每晚的自动进化(System 2)停止",
|
||||
lite_search_stops: "lite 只在原始对话里的检索立即停止,改为检索记忆",
|
||||
graph_backfill_already_running: "知识图谱的补建已在进行中",
|
||||
};
|
||||
|
||||
/** The same rule Core applies (`memory_mode_effects`), for the plan. */
|
||||
export function modeEffects(
|
||||
previous: MemoryMode,
|
||||
next: MemoryMode,
|
||||
): ModeEffect[] {
|
||||
if (previous === next) return [];
|
||||
const rank: Record<MemoryMode, number> = { lite: 0, pro: 1, ultra: 2 };
|
||||
const effects: ModeEffect[] = [];
|
||||
if (previous === "lite") {
|
||||
effects.push("earlier_raw_not_extracted", "lite_search_stops");
|
||||
}
|
||||
if (rank[next] < rank[previous]) effects.push("higher_mode_memories_hidden");
|
||||
if (previous === "ultra") effects.push("evolve_stopped");
|
||||
if (next === "ultra") effects.push("existing_memories_added_to_graph");
|
||||
return effects;
|
||||
}
|
||||
|
||||
export type ModeCommand =
|
||||
| { kind: "show" }
|
||||
| { kind: "set"; mode: MemoryMode; confirmed: boolean };
|
||||
|
||||
/** `settings mode [lite|pro|ultra] [--yes]` after `settings`; null = usage. */
|
||||
export function parseSettingsCommand(
|
||||
values: readonly string[],
|
||||
): ModeCommand | null {
|
||||
const rest = values.filter((value) => value !== "--json");
|
||||
if (rest[0] !== "mode") return null;
|
||||
const tail = rest.slice(1);
|
||||
const confirmed = tail.includes("--yes");
|
||||
const positional = tail.filter((value) => value !== "--yes");
|
||||
if (tail.filter((value) => value === "--yes").length > 1) return null;
|
||||
if (positional.length === 0) return confirmed ? null : { kind: "show" };
|
||||
if (positional.length !== 1) return null;
|
||||
const mode = positional[0] as MemoryMode;
|
||||
if (!(MEMORY_MODES as readonly string[]).includes(mode)) return null;
|
||||
return { kind: "set", mode, confirmed };
|
||||
}
|
||||
|
||||
export interface ModeState {
|
||||
mode: MemoryMode;
|
||||
configured_mode: MemoryMode;
|
||||
forced_by_env: boolean;
|
||||
graph_build?: { state?: string } | null;
|
||||
previous_mode?: MemoryMode;
|
||||
effects?: ModeEffect[];
|
||||
}
|
||||
|
||||
function modeState(value: unknown): ModeState | null {
|
||||
if (!value || typeof value !== "object") return null;
|
||||
const item = value as Record<string, unknown>;
|
||||
if (
|
||||
!(MEMORY_MODES as readonly unknown[]).includes(item.mode) ||
|
||||
!(MEMORY_MODES as readonly unknown[]).includes(item.configured_mode) ||
|
||||
typeof item.forced_by_env !== "boolean"
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return item as unknown as ModeState;
|
||||
}
|
||||
|
||||
export type ModeDocument = {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
command: "settings mode";
|
||||
} & (
|
||||
| { state: "ok"; mode: ModeState }
|
||||
| {
|
||||
state: "preview";
|
||||
mode: ModeState;
|
||||
target: MemoryMode;
|
||||
effects: ModeEffect[];
|
||||
applied: false;
|
||||
next_step: string;
|
||||
}
|
||||
| { state: "unchanged"; mode: ModeState }
|
||||
| { state: "committed"; mode: ModeState; effects: ModeEffect[] }
|
||||
| { state: "rejected" | "unknown" | "unavailable"; code: string }
|
||||
);
|
||||
|
||||
const BASE = {
|
||||
schema_version: 1 as const,
|
||||
product: PRODUCT,
|
||||
command: "settings mode" as const,
|
||||
};
|
||||
|
||||
export async function runModeCommand(
|
||||
command: ModeCommand,
|
||||
reader: CoreReader,
|
||||
operations: CorePutOperations,
|
||||
): Promise<ModeDocument> {
|
||||
const read = await reader<unknown>("/api/v1/settings/memory-mode");
|
||||
if (read.state !== "ok") {
|
||||
return { ...BASE, state: "unavailable", code: read.code };
|
||||
}
|
||||
const current = modeState(read.data);
|
||||
if (current === null) {
|
||||
return { ...BASE, state: "unavailable", code: "response_invalid" };
|
||||
}
|
||||
if (command.kind === "show") return { ...BASE, state: "ok", mode: current };
|
||||
if (current.configured_mode === command.mode) {
|
||||
return { ...BASE, state: "unchanged", mode: current };
|
||||
}
|
||||
if (!command.confirmed) {
|
||||
return {
|
||||
...BASE,
|
||||
state: "preview",
|
||||
mode: current,
|
||||
target: command.mode,
|
||||
effects: modeEffects(current.mode, command.mode),
|
||||
applied: false,
|
||||
next_step: `${PRODUCT} settings mode ${command.mode} --yes`,
|
||||
};
|
||||
}
|
||||
const result = await operations.put<unknown>("/api/v1/settings/memory-mode", {
|
||||
mode: command.mode,
|
||||
});
|
||||
if (result.state !== "committed") {
|
||||
return { ...BASE, state: result.state, code: result.code };
|
||||
}
|
||||
const changed = modeState(result.data);
|
||||
if (changed === null) {
|
||||
return { ...BASE, state: "unknown", code: "response_invalid" };
|
||||
}
|
||||
return {
|
||||
...BASE,
|
||||
state: "committed",
|
||||
mode: changed,
|
||||
effects: changed.effects ?? [],
|
||||
};
|
||||
}
|
||||
|
||||
export function modeExitCode(document: ModeDocument): number {
|
||||
return ["ok", "preview", "unchanged", "committed"].includes(document.state)
|
||||
? 0
|
||||
: 1;
|
||||
}
|
||||
|
||||
export function renderModeText(document: ModeDocument): string {
|
||||
if ("code" in document) {
|
||||
return `记忆模式未能${document.state === "unavailable" ? "读取" : "修改"}:${document.code}\n`;
|
||||
}
|
||||
const mode = document.mode;
|
||||
const lines = [`当前模式:${MODE_SUMMARY[mode.mode]}`];
|
||||
if (mode.forced_by_env) {
|
||||
lines.push(
|
||||
`设置里是 ${mode.configured_mode};环境变量 ONE_MEMORY_GRAPH_STORE 强制使用 ultra。`,
|
||||
);
|
||||
}
|
||||
lines.push("原始对话:全部保留(任何模式都不删除)");
|
||||
if (document.state === "unchanged") {
|
||||
lines.push("已经是这个模式,未做改动。");
|
||||
}
|
||||
if (document.state === "preview") {
|
||||
lines.push(`将切换到:${MODE_SUMMARY[document.target]}`);
|
||||
for (const effect of document.effects)
|
||||
lines.push(`- ${EFFECT_TEXT[effect]}`);
|
||||
lines.push(`未执行任何改动。确认后运行:${document.next_step}`);
|
||||
}
|
||||
if (document.state === "committed") {
|
||||
lines.push("已切换。");
|
||||
for (const effect of document.effects)
|
||||
lines.push(`- ${EFFECT_TEXT[effect]}`);
|
||||
}
|
||||
return `${lines.join("\n")}\n`;
|
||||
}
|
||||
|
||||
export type EvolveCommand = {
|
||||
agentId?: string;
|
||||
maxBatches?: number;
|
||||
wait: boolean;
|
||||
};
|
||||
|
||||
/** `evolve [--agent-id <id>] [--max-batches <n>] [--wait]`; null = usage. */
|
||||
export function parseEvolveCommand(
|
||||
values: readonly string[],
|
||||
): EvolveCommand | null {
|
||||
const parsed: EvolveCommand = { wait: false };
|
||||
const rest = values.filter((value) => value !== "--json");
|
||||
for (let index = 0; index < rest.length; index += 1) {
|
||||
const flag = rest[index];
|
||||
if (flag === "--wait" && !parsed.wait) {
|
||||
parsed.wait = true;
|
||||
} else if (flag === "--agent-id" && parsed.agentId === undefined) {
|
||||
const value = rest[index + 1];
|
||||
if (!value || !/^[A-Za-z0-9._:-]{1,128}$/.test(value)) return null;
|
||||
parsed.agentId = value;
|
||||
index += 1;
|
||||
} else if (flag === "--max-batches" && parsed.maxBatches === undefined) {
|
||||
const value = Number(rest[index + 1]);
|
||||
if (!Number.isInteger(value) || value < 1 || value > 1000) return null;
|
||||
parsed.maxBatches = value;
|
||||
index += 1;
|
||||
} else {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
export type EvolveDocument = {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
command: "evolve";
|
||||
} & (
|
||||
| { state: "running" | "succeeded" | "failed"; run: Record<string, unknown> }
|
||||
| { state: "rejected" | "unknown" | "unavailable"; code: string }
|
||||
);
|
||||
|
||||
const RUN_ID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
|
||||
function evolveRun(value: unknown): Record<string, unknown> | null {
|
||||
if (!value || typeof value !== "object") return null;
|
||||
const run = value as Record<string, unknown>;
|
||||
if (
|
||||
typeof run.run_id !== "string" ||
|
||||
!RUN_ID.test(run.run_id) ||
|
||||
!["running", "succeeded", "failed"].includes(String(run.state))
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return run;
|
||||
}
|
||||
|
||||
export async function runEvolveCommand(
|
||||
command: EvolveCommand,
|
||||
options: {
|
||||
userId?: string;
|
||||
reader: CoreReader;
|
||||
operations: CoreOperations;
|
||||
sleep?: (ms: number) => Promise<void>;
|
||||
pollMs?: number;
|
||||
},
|
||||
): Promise<EvolveDocument> {
|
||||
const base = {
|
||||
schema_version: 1 as const,
|
||||
product: PRODUCT,
|
||||
command: "evolve" as const,
|
||||
};
|
||||
const body: Record<string, unknown> = {};
|
||||
if (options.userId !== undefined) body.user_id = options.userId;
|
||||
if (command.agentId !== undefined) body.agent_id = command.agentId;
|
||||
if (command.maxBatches !== undefined) body.max_batches = command.maxBatches;
|
||||
const started = await options.operations.post<unknown>(
|
||||
"/api/v2/evolve",
|
||||
body,
|
||||
);
|
||||
if (started.state !== "committed") {
|
||||
return { ...base, state: started.state, code: started.code };
|
||||
}
|
||||
let run = evolveRun(started.data);
|
||||
if (run === null)
|
||||
return { ...base, state: "unknown", code: "response_invalid" };
|
||||
const sleep =
|
||||
options.sleep ??
|
||||
((ms: number) => new Promise((done) => setTimeout(done, ms)));
|
||||
while (command.wait && run.state === "running") {
|
||||
await sleep(options.pollMs ?? 2_000);
|
||||
const read = await options.reader<unknown>(
|
||||
`/api/v2/evolve/${run.run_id}`,
|
||||
options.userId === undefined ? {} : { user_id: options.userId },
|
||||
);
|
||||
if (read.state !== "ok") {
|
||||
return { ...base, state: "unavailable", code: read.code };
|
||||
}
|
||||
const next = evolveRun(read.data);
|
||||
if (next === null)
|
||||
return { ...base, state: "unknown", code: "response_invalid" };
|
||||
run = next;
|
||||
}
|
||||
return {
|
||||
...base,
|
||||
state: run.state as "running" | "succeeded" | "failed",
|
||||
run: projectRun(run),
|
||||
};
|
||||
}
|
||||
|
||||
/** Counts and codes; SDK result bodies stay in the Core receipt. */
|
||||
function projectRun(run: Record<string, unknown>): Record<string, unknown> {
|
||||
const keep = [
|
||||
"run_id",
|
||||
"user_id",
|
||||
"agent_id",
|
||||
"state",
|
||||
"trigger",
|
||||
"started_at",
|
||||
"finished_at",
|
||||
"rounds",
|
||||
"tasks_processed",
|
||||
"batch_failures",
|
||||
"cycle_status",
|
||||
"should_continue",
|
||||
"continuation_reason",
|
||||
"remaining_clusters",
|
||||
"remaining_actionable_clusters",
|
||||
"remaining_fresh_facts",
|
||||
"capped",
|
||||
"elapsed_ms",
|
||||
"error_code",
|
||||
"failure_kind",
|
||||
];
|
||||
return Object.fromEntries(
|
||||
keep.filter((key) => key in run).map((key) => [key, run[key]]),
|
||||
);
|
||||
}
|
||||
|
||||
export function evolveExitCode(document: EvolveDocument): number {
|
||||
return document.state === "running" || document.state === "succeeded" ? 0 : 1;
|
||||
}
|
||||
|
||||
export function renderEvolveText(document: EvolveDocument): string {
|
||||
if ("code" in document) {
|
||||
return document.code === "memory_mode_not_ultra"
|
||||
? "进化只在 ultra 模式下可用。切换:memhub settings mode ultra --yes\n"
|
||||
: `进化未能开始:${document.code}\n`;
|
||||
}
|
||||
const run = document.run;
|
||||
if (document.state === "running") {
|
||||
return `进化已开始(${run.run_id}),在后台运行;加 --wait 等待结果。\n`;
|
||||
}
|
||||
if (document.state === "failed") {
|
||||
return `进化失败(${run.run_id}):${String(run.error_code ?? "unknown")}\n`;
|
||||
}
|
||||
return `进化完成(${run.run_id}):${run.rounds} 轮,处理 ${run.tasks_processed} 项${
|
||||
run.cycle_status ? `,周期 ${run.cycle_status}` : ""
|
||||
}${run.capped ? ",已到轮数上限,可再次运行继续" : ""}\n`;
|
||||
}
|
||||
@@ -14,6 +14,25 @@ import { type CoreReader, RESERVED_MEMORY_SEGMENTS } from "./core-client.js";
|
||||
import type { CoreOperations } from "./core-operations.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
/** A MemHub user id, the same shape Core accepts. `local` is the default. */
|
||||
const USER_ID = /^[A-Za-z0-9._-]{1,64}$/;
|
||||
export const DEFAULT_USER_ID = "local";
|
||||
|
||||
export function isUserId(value: string): boolean {
|
||||
return USER_ID.test(value);
|
||||
}
|
||||
|
||||
/**
|
||||
* The `user_id` to send, or nothing. The default user is never named on the
|
||||
* wire, so a request without `--user-id` stays exactly what it was before
|
||||
* multi-user support and still works against an older Core.
|
||||
*/
|
||||
export function userIdParameters(userId?: string): { user_id?: string } {
|
||||
return userId === undefined || userId === DEFAULT_USER_ID
|
||||
? {}
|
||||
: { user_id: userId };
|
||||
}
|
||||
|
||||
/** Output budget: a long memory is shortened visibly, never dropped silently. */
|
||||
const CONTENT_BUDGET = 240;
|
||||
|
||||
@@ -101,10 +120,12 @@ export interface MemoryDetailDocument {
|
||||
export async function readMemoryPage(
|
||||
reader: CoreReader,
|
||||
query: MemoryQuery,
|
||||
userId?: string,
|
||||
): Promise<MemoryPageDocument> {
|
||||
const parameters: Record<string, string | number> = {
|
||||
offset: query.offset,
|
||||
limit: query.limit,
|
||||
...userIdParameters(userId),
|
||||
};
|
||||
if (query.status !== undefined) parameters.statuses = query.status;
|
||||
if (query.source !== undefined) parameters.source = query.source;
|
||||
@@ -156,8 +177,12 @@ export async function readMemoryDetail(
|
||||
reader: CoreReader,
|
||||
memoryId: string,
|
||||
namespace: MemoryNamespace = {},
|
||||
userId?: string,
|
||||
): Promise<MemoryDetailDocument> {
|
||||
const parameters = namespaceParameters(namespace);
|
||||
const parameters = {
|
||||
...namespaceParameters(namespace),
|
||||
...userIdParameters(userId),
|
||||
};
|
||||
const outcome = await reader<unknown>(
|
||||
`/api/v2/memories/${memoryId}`,
|
||||
Object.keys(parameters).length > 0 ? parameters : undefined,
|
||||
@@ -236,6 +261,7 @@ export async function addMemory(
|
||||
operations: CoreOperations,
|
||||
text: string,
|
||||
requestId: string = randomUUID(),
|
||||
userId?: string,
|
||||
): Promise<MemoryAddDocument> {
|
||||
const base = {
|
||||
schema_version: 2 as const,
|
||||
@@ -259,7 +285,10 @@ export async function addMemory(
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
const result = await operations.post<unknown>(request.path, request.body);
|
||||
const result = await operations.post<unknown>(request.path, {
|
||||
...request.body,
|
||||
...userIdParameters(userId),
|
||||
});
|
||||
if (result.state !== "committed") {
|
||||
return { ...base, state: result.state, code: result.code };
|
||||
}
|
||||
@@ -273,6 +302,7 @@ export async function addMemory(
|
||||
export async function readMemoryAddStatus(
|
||||
reader: CoreReader,
|
||||
requestId: string,
|
||||
userId?: string,
|
||||
): Promise<MemoryAddDocument> {
|
||||
const base = {
|
||||
schema_version: 2 as const,
|
||||
@@ -283,7 +313,11 @@ export async function readMemoryAddStatus(
|
||||
if (!UUID.test(requestId)) {
|
||||
return { ...base, state: "invalid", code: "invalid_request_id" };
|
||||
}
|
||||
const outcome = await reader<unknown>(memoryAddStatusPath(requestId));
|
||||
const user = userIdParameters(userId);
|
||||
const outcome =
|
||||
user.user_id === undefined
|
||||
? await reader<unknown>(memoryAddStatusPath(requestId))
|
||||
: await reader<unknown>(memoryAddStatusPath(requestId), user);
|
||||
if (outcome.state === "not_found") {
|
||||
return { ...base, state: "not_found", code: "memory_add_not_found" };
|
||||
}
|
||||
|
||||
@@ -0,0 +1,166 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { run } from "./cli.js";
|
||||
|
||||
const transport = vi.hoisted(() => ({
|
||||
get: vi.fn(),
|
||||
post: vi.fn(),
|
||||
put: vi.fn(),
|
||||
patch: vi.fn(),
|
||||
delete: vi.fn(),
|
||||
}));
|
||||
vi.mock("axios", () => ({
|
||||
default: {
|
||||
create: vi.fn(() => transport),
|
||||
get: vi.fn(),
|
||||
post: vi.fn(),
|
||||
isAxiosError: () => false,
|
||||
},
|
||||
}));
|
||||
|
||||
const runtime = vi.hoisted(() => ({
|
||||
inspectRuntimeStatus: vi.fn(),
|
||||
discoverRuntimeAttachment: vi.fn(),
|
||||
}));
|
||||
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
|
||||
...runtime,
|
||||
}));
|
||||
|
||||
const VERSIONS = {
|
||||
user_id: "wife",
|
||||
request_id: "r-versions",
|
||||
elapsed_ms: 2,
|
||||
chains: [
|
||||
{
|
||||
memory_id: "l0-head",
|
||||
content: "The user's name is Fei.",
|
||||
status: "active",
|
||||
memory_at: null,
|
||||
gmt_created: 1_790_000_100,
|
||||
basic_info: { name: "Fei" },
|
||||
versions: [
|
||||
{
|
||||
memory_id: "l0-head",
|
||||
content: "The user's name is Fei.",
|
||||
memory_at: null,
|
||||
gmt_created: 1_790_000_100,
|
||||
current: true,
|
||||
},
|
||||
{
|
||||
memory_id: "l0-old",
|
||||
content: "The user's name is 费天翔.",
|
||||
memory_at: null,
|
||||
gmt_created: 1_790_000_000,
|
||||
current: false,
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
for (const method of Object.values(transport)) method.mockReset();
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "running",
|
||||
code: "runtime_verified",
|
||||
instanceId: "instance-1",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
version: "1.2.3",
|
||||
});
|
||||
runtime.discoverRuntimeAttachment.mockResolvedValue({
|
||||
instanceId: "instance-1",
|
||||
apiToken: "synthetic-runtime-token",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
servicePid: 4242,
|
||||
health: {
|
||||
status: "ok",
|
||||
service: "one-memory-core",
|
||||
version: "1.2.3",
|
||||
instance_id: "instance-1",
|
||||
},
|
||||
});
|
||||
transport.get.mockImplementation(async (url: string) =>
|
||||
url === "/api/v2/user-profile/versions"
|
||||
? { status: 200, data: VERSIONS }
|
||||
: { status: 404, data: {} },
|
||||
);
|
||||
});
|
||||
|
||||
describe("memhub profile --versions", () => {
|
||||
it("reads the L0 version chain for the named user", async () => {
|
||||
const result = await capturedRun([
|
||||
"profile",
|
||||
"--versions",
|
||||
"--user-id",
|
||||
"wife",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.get).toHaveBeenCalledWith(
|
||||
"/api/v2/user-profile/versions",
|
||||
{ params: { user_id: "wife" } },
|
||||
);
|
||||
const document = JSON.parse(result.stdout);
|
||||
expect(document.state).toBe("ok");
|
||||
expect(document.chains[0].versions).toHaveLength(2);
|
||||
});
|
||||
|
||||
it("marks the current version in text and names no user for local", async () => {
|
||||
const result = await capturedRun(["profile", "--versions"]);
|
||||
expect(result.code).toBe(0);
|
||||
const [url, config] = transport.get.mock.calls[0] ?? [];
|
||||
expect(url).toBe("/api/v2/user-profile/versions");
|
||||
expect(config?.params?.user_id).toBeUndefined();
|
||||
expect(result.stdout).toContain("* l0-head");
|
||||
expect(result.stdout).toContain(" l0-old");
|
||||
expect(result.stdout).toContain("费天翔");
|
||||
});
|
||||
|
||||
it("refuses an unknown flag and a repeated --versions", async () => {
|
||||
expect((await capturedRun(["profile", "--version-chain"])).code).toBe(2);
|
||||
expect(
|
||||
(await capturedRun(["profile", "--versions", "--versions"])).code,
|
||||
).toBe(2);
|
||||
expect(transport.get).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("reports a malformed answer as invalid", async () => {
|
||||
transport.get.mockResolvedValue({ status: 200, data: { chains: "x" } });
|
||||
const result = await capturedRun(["profile", "--versions", "--json"]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(JSON.parse(result.stdout).code).toBe("response_invalid");
|
||||
});
|
||||
});
|
||||
|
||||
async function capturedRun(arguments_: string[]) {
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
const stdoutSpy = vi
|
||||
.spyOn(process.stdout, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stdout += String(chunk);
|
||||
return true;
|
||||
});
|
||||
const stderrSpy = vi
|
||||
.spyOn(process.stderr, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stderr += String(chunk);
|
||||
return true;
|
||||
});
|
||||
try {
|
||||
return {
|
||||
code: await run(arguments_, {
|
||||
ONE_MEMORY_DATA_DIR: "/synthetic/data",
|
||||
}),
|
||||
stdout,
|
||||
stderr,
|
||||
};
|
||||
} finally {
|
||||
stdoutSpy.mockRestore();
|
||||
stderrSpy.mockRestore();
|
||||
}
|
||||
}
|
||||
@@ -14,6 +14,11 @@ import {
|
||||
validateProcessing,
|
||||
validateSources,
|
||||
} from "./core-responses.js";
|
||||
import {
|
||||
describePipeline,
|
||||
type PipelineSummary,
|
||||
pipelineLine,
|
||||
} from "./status-report.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
type ReadState =
|
||||
@@ -38,6 +43,21 @@ export interface SourcesDocument {
|
||||
state: ReadState["state"];
|
||||
code?: string;
|
||||
sources: Array<Record<string, unknown>>;
|
||||
/** Every grant of every user (`GET /api/v1/source-grants`). Absent when
|
||||
* Core is too old to list them; `sources` is the default user's view. */
|
||||
grants?: SourceGrant[];
|
||||
}
|
||||
|
||||
/** One source grant. Only Hermes has grants for users other than `local`,
|
||||
* one per Hermes profile root. */
|
||||
export interface SourceGrant {
|
||||
user_id: string;
|
||||
source_id: string;
|
||||
root_label: string | null;
|
||||
root_path: string | null;
|
||||
authorized: boolean;
|
||||
revoked_at: string | null;
|
||||
sync_enabled: boolean;
|
||||
}
|
||||
|
||||
export interface SyncDocument {
|
||||
@@ -69,6 +89,8 @@ export interface ProgressDocument {
|
||||
summary_state: ReadState;
|
||||
summary: Record<string, unknown> | null;
|
||||
};
|
||||
/** The SDK mode and last evolve per user; absent on an older Core. */
|
||||
pipeline?: PipelineSummary | null;
|
||||
}
|
||||
|
||||
export async function readSources(
|
||||
@@ -86,14 +108,66 @@ export async function readSources(
|
||||
sources: [],
|
||||
};
|
||||
}
|
||||
const grants = await readSourceGrants(reader);
|
||||
return {
|
||||
schema_version: SCHEMA_VERSION,
|
||||
product: PRODUCT,
|
||||
state: "ok",
|
||||
sources: result.data.map(projectSource),
|
||||
// `/api/v1/sources` is the default user's view of every source.
|
||||
sources: result.data.map((source) => ({
|
||||
...projectSource(source),
|
||||
user_id: "local",
|
||||
})),
|
||||
...(grants === null ? {} : { grants }),
|
||||
};
|
||||
}
|
||||
|
||||
/** The grants of every user, or null when Core cannot list them (an older
|
||||
* Core answers 404): the source list is then shown as before. */
|
||||
async function readSourceGrants(
|
||||
reader: CoreReader,
|
||||
): Promise<SourceGrant[] | null> {
|
||||
let outcome: ReadOutcome<unknown>;
|
||||
try {
|
||||
outcome = await reader<unknown>("/api/v1/source-grants");
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
if (outcome.state !== "ok") return null;
|
||||
const value = outcome.data;
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) return null;
|
||||
const items = (value as { grants?: unknown }).grants;
|
||||
if (!Array.isArray(items)) return null;
|
||||
const grants: SourceGrant[] = [];
|
||||
const text = (candidate: unknown): string | null =>
|
||||
typeof candidate === "string" && candidate.length <= 4096
|
||||
? candidate
|
||||
: null;
|
||||
for (const item of items) {
|
||||
if (!item || typeof item !== "object" || Array.isArray(item)) return null;
|
||||
const grant = item as Record<string, unknown>;
|
||||
if (
|
||||
typeof grant.user_id !== "string" ||
|
||||
!/^[A-Za-z0-9._-]{1,64}$/.test(grant.user_id) ||
|
||||
typeof grant.source_id !== "string" ||
|
||||
!/^[a-z0-9][a-z0-9-]{0,127}$/.test(grant.source_id) ||
|
||||
typeof grant.authorized !== "boolean"
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
grants.push({
|
||||
user_id: grant.user_id,
|
||||
source_id: grant.source_id,
|
||||
root_label: text(grant.root_label),
|
||||
root_path: text(grant.root_path),
|
||||
authorized: grant.authorized,
|
||||
revoked_at: text(grant.revoked_at),
|
||||
sync_enabled: grant.sync_enabled === true,
|
||||
});
|
||||
}
|
||||
return grants;
|
||||
}
|
||||
|
||||
export async function readSync(reader: CoreReader): Promise<SyncDocument> {
|
||||
const result = validated(
|
||||
await reader<unknown>("/api/v1/live-sync"),
|
||||
@@ -216,6 +290,10 @@ export async function readProgress(
|
||||
summary:
|
||||
processing.state === "ok" ? projectProcessing(processing.data) : null,
|
||||
},
|
||||
pipeline:
|
||||
processingRead.state === "ok"
|
||||
? describePipeline(processingRead.data)
|
||||
: null,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -231,6 +309,22 @@ export function renderSourcesText(document: SourcesDocument): string {
|
||||
);
|
||||
}
|
||||
if (document.sources.length === 0) lines.push("(没有来源)");
|
||||
// The list above is the default user's (`local`). Another user's grant,
|
||||
// a Hermes profile, is listed with its user so it is not mistaken for it.
|
||||
const others = (document.grants ?? []).filter(
|
||||
(grant) => grant.user_id !== "local",
|
||||
);
|
||||
if (others.length > 0) {
|
||||
lines.push("其他用户的连接:");
|
||||
for (const grant of others) {
|
||||
const name =
|
||||
document.sources.find((source) => source.source_id === grant.source_id)
|
||||
?.display_name ?? grant.source_id;
|
||||
lines.push(
|
||||
`- ${name}(用户 ${grant.user_id}):${grant.authorized ? "已连接" : grant.revoked_at ? "已撤销" : "未连接"};处理${grant.sync_enabled ? "已开启" : "已冻结"}${grant.root_path ? `;目录 ${grant.root_path}` : ""}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
lines.push("");
|
||||
return lines.join("\n");
|
||||
}
|
||||
@@ -270,6 +364,18 @@ export function renderProgressText(document: ProgressDocument): string {
|
||||
? `记忆整理:待处理 ${document.generation.summary.pending} · 进行中 ${document.generation.summary.running} · 等待模型 ${document.generation.summary.waiting_for_model} · 失败 ${document.generation.summary.failed}`
|
||||
: "记忆整理:暂时无法读取",
|
||||
);
|
||||
if (document.pipeline) {
|
||||
lines.push(`记忆模式:${pipelineLine(document.pipeline)}`);
|
||||
for (const [userId, user] of Object.entries(
|
||||
document.pipeline.evolve_users,
|
||||
)) {
|
||||
lines.push(
|
||||
` 进化 ${userId}:${user.last_run_at ?? "尚未进化"}${
|
||||
user.last_error ? `(失败:${user.last_error})` : ""
|
||||
}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
if (document.imports.output_truncated) {
|
||||
lines.push("这里只显示汇总;完整记录请使用 --json。");
|
||||
}
|
||||
|
||||
@@ -179,12 +179,72 @@ describe("CLI-07/10 source and synchronization reads", () => {
|
||||
sync_enabled: true,
|
||||
sync_completeness: "additions_only",
|
||||
});
|
||||
expect(requestedPaths()).toEqual(["/api/v1/sources"]);
|
||||
// The grants list answers 404 here, as an older Core does: the source
|
||||
// list is shown as before, every row the default user's.
|
||||
expect(report.sources[0].user_id).toBe("local");
|
||||
expect(report.grants).toBeUndefined();
|
||||
expect(requestedPaths()).toEqual([
|
||||
"/api/v1/sources",
|
||||
"/api/v1/source-grants",
|
||||
]);
|
||||
expect(requestedPaths().some((path) => path.includes("detect"))).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
it("shows every user's grant, a Hermes profile with its user", async () => {
|
||||
const grants = {
|
||||
schema_version: 1,
|
||||
grants: [
|
||||
{
|
||||
user_id: "local",
|
||||
source_id: "codex",
|
||||
root_label: ".codex",
|
||||
root_path: "/home/synthetic/.codex",
|
||||
authorized: true,
|
||||
revoked_at: null,
|
||||
sync_enabled: true,
|
||||
},
|
||||
{
|
||||
user_id: "wife",
|
||||
source_id: "hermes",
|
||||
root_label: "wife",
|
||||
root_path: "/home/synthetic/.hermes/profiles/wife",
|
||||
authorized: true,
|
||||
revoked_at: null,
|
||||
sync_enabled: true,
|
||||
},
|
||||
],
|
||||
};
|
||||
transport.get.mockImplementation(async (url: string) =>
|
||||
url === "/api/v1/source-grants"
|
||||
? { status: 200, data: grants }
|
||||
: (RESPONSES.get(url) ?? { status: 404, data: {} }),
|
||||
);
|
||||
let streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["sources", "list", "--json"], {})).toBe(0);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
const report = JSON.parse(streams.stdout());
|
||||
expect(report.grants).toEqual(grants.grants);
|
||||
expect(report.sources[0]).toMatchObject({
|
||||
source_id: "codex",
|
||||
user_id: "local",
|
||||
});
|
||||
streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["sources", "list"], {})).toBe(0);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(streams.stdout()).toContain("其他用户的连接:");
|
||||
expect(streams.stdout()).toContain(
|
||||
"hermes(用户 wife):已连接;处理已开启;目录 /home/synthetic/.hermes/profiles/wife",
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps degraded synchronization distinct and prints its error code", async () => {
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
@@ -552,6 +612,7 @@ describe("CLI-19 progress and issue reads", () => {
|
||||
requestedPaths().every((path) =>
|
||||
[
|
||||
"/api/v1/sources",
|
||||
"/api/v1/source-grants",
|
||||
"/api/v1/live-sync",
|
||||
"/api/v1/import-jobs",
|
||||
"/api/v1/memory-processing",
|
||||
|
||||
@@ -57,6 +57,9 @@ export interface StatusDocument {
|
||||
broker?: BackgroundHealth["broker"];
|
||||
};
|
||||
sections: Record<string, Section>;
|
||||
/** R10: the SDK mode and System 2 state; null when the Runtime is not read
|
||||
* or is older than the setting. */
|
||||
pipeline?: PipelineSummary | null;
|
||||
summary: {
|
||||
state: "ok" | "partial";
|
||||
degraded: string[];
|
||||
@@ -64,6 +67,88 @@ export interface StatusDocument {
|
||||
};
|
||||
}
|
||||
|
||||
export interface PipelineSummary {
|
||||
mode: "lite" | "pro" | "ultra";
|
||||
configured_mode: "lite" | "pro" | "ultra";
|
||||
forced_by_env: boolean;
|
||||
last_evolve_at: string | null;
|
||||
evolve_schedule: { enabled: boolean; time: string } | null;
|
||||
evolve_running: number;
|
||||
evolve_users: Record<
|
||||
string,
|
||||
{
|
||||
last_run_at: string | null;
|
||||
last_error: string | null;
|
||||
state: string | null;
|
||||
}
|
||||
>;
|
||||
graph_build: string | null;
|
||||
}
|
||||
|
||||
const MODES = new Set(["lite", "pro", "ultra"]);
|
||||
|
||||
/** Projection of `pipeline` in `/api/v1/memory-processing`; null if absent or malformed. */
|
||||
export function describePipeline(value: unknown): PipelineSummary | null {
|
||||
if (!value || typeof value !== "object") return null;
|
||||
const pipeline = (value as { pipeline?: unknown }).pipeline;
|
||||
if (!pipeline || typeof pipeline !== "object") return null;
|
||||
const item = pipeline as Record<string, unknown>;
|
||||
if (
|
||||
!MODES.has(String(item.mode)) ||
|
||||
!MODES.has(String(item.configured_mode))
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const evolve =
|
||||
item.evolve && typeof item.evolve === "object"
|
||||
? (item.evolve as Record<string, unknown>)
|
||||
: {};
|
||||
const schedule =
|
||||
evolve.schedule && typeof evolve.schedule === "object"
|
||||
? (evolve.schedule as Record<string, unknown>)
|
||||
: null;
|
||||
const users: PipelineSummary["evolve_users"] = {};
|
||||
if (evolve.users && typeof evolve.users === "object") {
|
||||
for (const [userId, raw] of Object.entries(
|
||||
evolve.users as Record<string, unknown>,
|
||||
)) {
|
||||
const user =
|
||||
raw && typeof raw === "object" ? (raw as Record<string, unknown>) : {};
|
||||
const result =
|
||||
user.last_result && typeof user.last_result === "object"
|
||||
? (user.last_result as Record<string, unknown>)
|
||||
: {};
|
||||
users[userId] = {
|
||||
last_run_at:
|
||||
typeof user.last_run_at === "string" ? user.last_run_at : null,
|
||||
last_error:
|
||||
typeof user.last_error === "string" ? user.last_error : null,
|
||||
state: typeof result.state === "string" ? result.state : null,
|
||||
};
|
||||
}
|
||||
}
|
||||
const graph =
|
||||
item.graph_build && typeof item.graph_build === "object"
|
||||
? (item.graph_build as Record<string, unknown>)
|
||||
: {};
|
||||
return {
|
||||
mode: item.mode as PipelineSummary["mode"],
|
||||
configured_mode: item.configured_mode as PipelineSummary["mode"],
|
||||
forced_by_env: item.forced_by_env === true,
|
||||
last_evolve_at:
|
||||
typeof item.last_evolve_at === "string" ? item.last_evolve_at : null,
|
||||
evolve_schedule:
|
||||
schedule &&
|
||||
typeof schedule.enabled === "boolean" &&
|
||||
typeof schedule.time === "string"
|
||||
? { enabled: schedule.enabled, time: schedule.time }
|
||||
: null,
|
||||
evolve_running: Array.isArray(evolve.running) ? evolve.running.length : 0,
|
||||
evolve_users: users,
|
||||
graph_build: typeof graph.state === "string" ? graph.state : null,
|
||||
};
|
||||
}
|
||||
|
||||
const SECTION_NAMES = [
|
||||
"sources",
|
||||
"sync",
|
||||
@@ -97,6 +182,7 @@ export async function composeStatusReport(options: {
|
||||
attachFailure?: string;
|
||||
}): Promise<StatusDocument> {
|
||||
const sections: Record<string, Section> = {};
|
||||
let pipeline: PipelineSummary | null = null;
|
||||
if (options.reader === null) {
|
||||
const state: SectionState =
|
||||
options.attachFailure === undefined ? "not_checked" : "unavailable";
|
||||
@@ -118,6 +204,8 @@ export async function composeStatusReport(options: {
|
||||
validateProcessing,
|
||||
describeProcessing,
|
||||
);
|
||||
pipeline =
|
||||
processing.state === "ok" ? describePipeline(processing.data) : null;
|
||||
sections.model = section(model, validateModelProfiles, describeModel);
|
||||
sections.memories = section(
|
||||
memories,
|
||||
@@ -169,6 +257,7 @@ export async function composeStatusReport(options: {
|
||||
},
|
||||
bridge,
|
||||
sections,
|
||||
pipeline,
|
||||
summary: {
|
||||
state:
|
||||
degraded.length === 0 && notChecked.length === 0 ? "ok" : "partial",
|
||||
@@ -205,6 +294,7 @@ export function renderStatusText(
|
||||
`${SECTION_LABELS[name]}:${sectionLine(report.sections[name])}`,
|
||||
);
|
||||
}
|
||||
if (report.pipeline) lines.push(`记忆模式:${pipelineLine(report.pipeline)}`);
|
||||
if (report.summary.state === "partial") {
|
||||
const notes = [
|
||||
report.summary.degraded.length > 0
|
||||
@@ -220,6 +310,28 @@ export function renderStatusText(
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
export function pipelineLine(pipeline: PipelineSummary): string {
|
||||
const parts: string[] = [pipeline.mode];
|
||||
if (pipeline.forced_by_env)
|
||||
parts.push("(ONE_MEMORY_GRAPH_STORE 强制 ultra)");
|
||||
if (pipeline.mode === "ultra") {
|
||||
const schedule = pipeline.evolve_schedule;
|
||||
parts.push(
|
||||
schedule?.enabled
|
||||
? `,每日 ${schedule.time} 自动进化`
|
||||
: ",自动进化已关闭",
|
||||
);
|
||||
parts.push(
|
||||
pipeline.last_evolve_at
|
||||
? `,上次进化 ${pipeline.last_evolve_at}`
|
||||
: ",尚未进化",
|
||||
);
|
||||
if (pipeline.evolve_running > 0)
|
||||
parts.push(`,进行中 ${pipeline.evolve_running}`);
|
||||
}
|
||||
return parts.join("");
|
||||
}
|
||||
|
||||
function runtimeLine(report: StatusDocument): string {
|
||||
if (report.runtime.state !== "running") {
|
||||
return report.runtime.state === "stopped" ? "未运行" : "暂时无法确认";
|
||||
|
||||
@@ -0,0 +1,405 @@
|
||||
import { mkdtemp, writeFile } from "node:fs/promises";
|
||||
import os, { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { run } from "./cli.js";
|
||||
|
||||
const transport = vi.hoisted(() => ({
|
||||
get: vi.fn(),
|
||||
post: vi.fn(),
|
||||
put: vi.fn(),
|
||||
patch: vi.fn(),
|
||||
delete: vi.fn(),
|
||||
}));
|
||||
const direct = vi.hoisted(() => ({ get: vi.fn(), post: vi.fn() }));
|
||||
vi.mock("axios", () => ({
|
||||
default: {
|
||||
create: vi.fn(() => transport),
|
||||
get: direct.get,
|
||||
post: direct.post,
|
||||
isAxiosError: () => false,
|
||||
},
|
||||
}));
|
||||
|
||||
const runtime = vi.hoisted(() => ({
|
||||
inspectRuntimeStatus: vi.fn(),
|
||||
discoverRuntimeAttachment: vi.fn(),
|
||||
}));
|
||||
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
|
||||
...runtime,
|
||||
}));
|
||||
|
||||
const REQUEST = "33333333-3333-4333-8333-333333333333";
|
||||
const HERMES = {
|
||||
source_id: "hermes",
|
||||
display_name: "Hermes",
|
||||
storage_kinds: ["sqlite"],
|
||||
available: true,
|
||||
authorized: true,
|
||||
detected: null,
|
||||
root_label: ".hermes",
|
||||
candidate_path: "~/.hermes",
|
||||
candidate_kind: "default",
|
||||
candidate_revision: 0,
|
||||
candidate_root_id: "hermes-root-identity",
|
||||
imported_messages: 0,
|
||||
imported_conversations: 0,
|
||||
capabilities: {
|
||||
historical_import: "implemented",
|
||||
incremental_import: "implemented",
|
||||
live_sync: "implemented",
|
||||
recall: "implemented",
|
||||
installer: "not_implemented",
|
||||
verification: "fixture_only",
|
||||
},
|
||||
sync_completeness: "additions_only",
|
||||
location_state: "found",
|
||||
};
|
||||
const CODEX = {
|
||||
...HERMES,
|
||||
source_id: "codex",
|
||||
display_name: "Codex",
|
||||
candidate_root_id: "codex-root-identity",
|
||||
};
|
||||
|
||||
function addReceipt() {
|
||||
return {
|
||||
evidence_id: "44444444-4444-4444-8444-444444444444",
|
||||
text: "Prefers tea",
|
||||
scope: "all-apps",
|
||||
created_at: "2026-10-27T00:00:00Z",
|
||||
state: "pending",
|
||||
task_id: null,
|
||||
memory_ids: [],
|
||||
error: null,
|
||||
};
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
for (const method of [
|
||||
...Object.values(transport),
|
||||
...Object.values(direct),
|
||||
]) {
|
||||
method.mockReset();
|
||||
}
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "running",
|
||||
code: "runtime_verified",
|
||||
instanceId: "instance-1",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
version: "1.2.3",
|
||||
});
|
||||
runtime.discoverRuntimeAttachment.mockResolvedValue({
|
||||
instanceId: "instance-1",
|
||||
apiToken: "synthetic-runtime-token",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
servicePid: 4242,
|
||||
health: {
|
||||
status: "ok",
|
||||
service: "one-memory-core",
|
||||
version: "1.2.3",
|
||||
instance_id: "instance-1",
|
||||
},
|
||||
});
|
||||
transport.get.mockImplementation(async (url: string) => {
|
||||
if (url === "/api/v1/sources") {
|
||||
return { status: 200, data: [HERMES, CODEX] };
|
||||
}
|
||||
if (url === "/api/v2/user-profile") {
|
||||
return {
|
||||
status: 200,
|
||||
data: { request_id: "r-1", profile: null, elapsed_ms: 1 },
|
||||
};
|
||||
}
|
||||
return { status: 404, data: {} };
|
||||
});
|
||||
transport.post.mockImplementation(
|
||||
async (url: string, body: Record<string, unknown>) =>
|
||||
url === "/api/v1/sources/connect"
|
||||
? {
|
||||
status: 200,
|
||||
data: {
|
||||
request_id: body.request_id,
|
||||
accepted_source_ids: ["hermes"],
|
||||
failed_sources: {},
|
||||
jobs: [],
|
||||
},
|
||||
}
|
||||
: url === "/api/v2/memories"
|
||||
? { status: 201, data: addReceipt() }
|
||||
: { status: 404, data: {} },
|
||||
);
|
||||
transport.put.mockResolvedValue({ status: 200, data: {} });
|
||||
transport.delete.mockResolvedValue({ status: 204, data: "" });
|
||||
direct.post.mockResolvedValue({
|
||||
status: 202,
|
||||
data: {
|
||||
request_id: REQUEST,
|
||||
state: "accepted",
|
||||
created_at: "T1",
|
||||
updated_at: "T1",
|
||||
},
|
||||
});
|
||||
direct.get.mockResolvedValue({
|
||||
data: {
|
||||
request_id: REQUEST,
|
||||
state: "accepted",
|
||||
created_at: "T1",
|
||||
updated_at: "T1",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
describe("--user-id on the memory commands", () => {
|
||||
it("passes the user as a query parameter to memory reads and deletes", async () => {
|
||||
await capturedRun(["memories", "list", "--user-id", "wife", "--json"]);
|
||||
expect(transport.get).toHaveBeenCalledWith("/api/v2/memory-library", {
|
||||
params: { offset: 0, limit: 100, user_id: "wife" },
|
||||
});
|
||||
// Global: the flag may come before the subcommand too.
|
||||
await capturedRun(["memories", "--user-id", "wife", "show", "m-1"]);
|
||||
expect(transport.get).toHaveBeenCalledWith("/api/v2/memories/m-1", {
|
||||
params: { user_id: "wife" },
|
||||
});
|
||||
await capturedRun(["memories", "add-status", REQUEST, "--user-id", "wife"]);
|
||||
expect(transport.get).toHaveBeenCalledWith(
|
||||
`/api/v2/memories/additions/${REQUEST}`,
|
||||
{ params: { user_id: "wife" } },
|
||||
);
|
||||
await capturedRun(["profile", "--user-id", "wife", "--json"]);
|
||||
expect(transport.get).toHaveBeenCalledWith("/api/v2/user-profile", {
|
||||
params: { user_id: "wife" },
|
||||
});
|
||||
await capturedRun([
|
||||
"memories",
|
||||
"delete",
|
||||
"fact-1",
|
||||
"--kind",
|
||||
"fact",
|
||||
"--yes",
|
||||
"--user-id",
|
||||
"wife",
|
||||
]);
|
||||
expect(transport.delete).toHaveBeenCalledWith(
|
||||
"/api/v2/memories/fact-1?kind=fact&user_id=wife",
|
||||
undefined,
|
||||
);
|
||||
await capturedRun([
|
||||
"memories",
|
||||
"delete",
|
||||
"m-1",
|
||||
"--yes",
|
||||
"--user-id",
|
||||
"wife",
|
||||
]);
|
||||
expect(transport.delete).toHaveBeenCalledWith(
|
||||
"/api/v2/memories/m-1?user_id=wife",
|
||||
undefined,
|
||||
);
|
||||
await capturedRun(["write", "status", REQUEST, "--user-id", "wife"]);
|
||||
expect(direct.get.mock.calls[0][0]).toBe(
|
||||
`http://127.0.0.1:8765/api/v2/memory-writes/${REQUEST}`,
|
||||
);
|
||||
expect(direct.get.mock.calls[0][1].params).toEqual({ user_id: "wife" });
|
||||
});
|
||||
|
||||
it("passes the user in the body of memory writes and searches", async () => {
|
||||
await capturedRun(["memories", "add", "Prefers tea", "--user-id", "wife"]);
|
||||
expect(transport.post.mock.calls[0][0]).toBe("/api/v2/memories");
|
||||
expect(transport.post.mock.calls[0][1]).toMatchObject({
|
||||
text: "Prefers tea",
|
||||
scope: "all-apps",
|
||||
user_id: "wife",
|
||||
});
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "om-user-id-"));
|
||||
const content = path.join(directory, "content.txt");
|
||||
await writeFile(content, "Prefers green tea\n");
|
||||
await capturedRun([
|
||||
"memories",
|
||||
"update",
|
||||
"m-1",
|
||||
"--content-file",
|
||||
content,
|
||||
"--user-id",
|
||||
"wife",
|
||||
]);
|
||||
expect(transport.put.mock.calls[0][0]).toBe("/api/v2/memories/m-1");
|
||||
expect(transport.put.mock.calls[0][1]).toMatchObject({ user_id: "wife" });
|
||||
await capturedRun(["recall", "tea", "--user-id", "wife", "--json"]);
|
||||
const search = transport.post.mock.calls.find(
|
||||
([url]) => url === "/api/v2/context/search",
|
||||
);
|
||||
expect(search?.[1]).toMatchObject({ query: "tea", user_id: "wife" });
|
||||
const messages = path.join(directory, "messages.json");
|
||||
await writeFile(
|
||||
messages,
|
||||
JSON.stringify([{ role: "user", content: "I like tea" }]),
|
||||
);
|
||||
await capturedRun([
|
||||
"write",
|
||||
"submit",
|
||||
"--messages-file",
|
||||
messages,
|
||||
"--session",
|
||||
"s-1",
|
||||
"--user-id",
|
||||
"wife",
|
||||
]);
|
||||
expect(direct.post.mock.calls[0][0]).toBe(
|
||||
"http://127.0.0.1:8765/api/v2/memory-writes",
|
||||
);
|
||||
expect(direct.post.mock.calls[0][1]).toMatchObject({
|
||||
session_id: "s-1",
|
||||
user_id: "wife",
|
||||
});
|
||||
});
|
||||
|
||||
it("names no user by default or for --user-id local", async () => {
|
||||
for (const extra of [[], ["--user-id", "local"]]) {
|
||||
transport.get.mockClear();
|
||||
transport.post.mockClear();
|
||||
await capturedRun(["memories", "list", ...extra, "--json"]);
|
||||
expect(transport.get).toHaveBeenCalledWith("/api/v2/memory-library", {
|
||||
params: { offset: 0, limit: 100 },
|
||||
});
|
||||
await capturedRun(["profile", ...extra]);
|
||||
expect(transport.get).toHaveBeenCalledWith(
|
||||
"/api/v2/user-profile",
|
||||
undefined,
|
||||
);
|
||||
await capturedRun(["memories", "add", "Prefers tea", ...extra]);
|
||||
expect(transport.post.mock.calls[0][1]).not.toHaveProperty("user_id");
|
||||
await capturedRun(["recall", "tea", ...extra, "--json"]);
|
||||
const search = transport.post.mock.calls.find(
|
||||
([url]) => url === "/api/v2/context/search",
|
||||
);
|
||||
expect(search?.[1]).not.toHaveProperty("user_id");
|
||||
}
|
||||
});
|
||||
|
||||
it("refuses a malformed user id before contacting Core", async () => {
|
||||
for (const value of ["bad:id", "", "x".repeat(65), "a b", "--json"]) {
|
||||
const result = await capturedRun([
|
||||
"memories",
|
||||
"list",
|
||||
"--user-id",
|
||||
value,
|
||||
]);
|
||||
expect(result.code).toBe(2);
|
||||
}
|
||||
expect(
|
||||
(await capturedRun(["profile", "--user-id", "a", "--user-id", "b"])).code,
|
||||
).toBe(2);
|
||||
expect((await capturedRun(["recall", "tea", "--user-id"])).code).toBe(2);
|
||||
expect(transport.get).not.toHaveBeenCalled();
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe("sources connect for a Hermes profile", () => {
|
||||
it("connects hermes for another user from its own root", async () => {
|
||||
const result = await capturedRun([
|
||||
"sources",
|
||||
"connect",
|
||||
"hermes",
|
||||
"--user-id",
|
||||
"wife",
|
||||
"--root",
|
||||
"~/.hermes/profiles/wife",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.post).toHaveBeenCalledTimes(1);
|
||||
const [url, body] = transport.post.mock.calls[0];
|
||||
expect(url).toBe("/api/v1/sources/connect");
|
||||
expect(body).toMatchObject({
|
||||
source_ids: ["hermes"],
|
||||
// Core reads the custom root's identity itself.
|
||||
source_roots: {},
|
||||
source_candidate_revisions: {},
|
||||
onboarding: false,
|
||||
user_id: "wife",
|
||||
root: path.join(os.homedir(), ".hermes/profiles/wife"),
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps a plain connect unchanged and accepts --import with a profile", async () => {
|
||||
await capturedRun(["sources", "connect", "hermes", "--import", "--json"]);
|
||||
const plain = transport.post.mock.calls[0][1];
|
||||
expect(plain).not.toHaveProperty("user_id");
|
||||
expect(plain).not.toHaveProperty("root");
|
||||
expect(plain).toMatchObject({
|
||||
source_roots: { hermes: "hermes-root-identity" },
|
||||
onboarding: true,
|
||||
});
|
||||
transport.post.mockClear();
|
||||
const result = await capturedRun([
|
||||
"sources",
|
||||
"connect",
|
||||
"hermes",
|
||||
"--import",
|
||||
"--user-id",
|
||||
"wife",
|
||||
"--root",
|
||||
"/profiles/wife",
|
||||
"--request-id",
|
||||
REQUEST,
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.post.mock.calls[0][1]).toMatchObject({
|
||||
request_id: REQUEST,
|
||||
user_id: "wife",
|
||||
root: "/profiles/wife",
|
||||
});
|
||||
});
|
||||
|
||||
it("refuses --user-id and --root for any source but hermes", async () => {
|
||||
for (const argv of [
|
||||
["sources", "connect", "codex", "--user-id", "wife"],
|
||||
["sources", "connect", "codex", "--root", "/somewhere", "--import"],
|
||||
["sources", "connect", "hermes", "codex", "--user-id", "wife"],
|
||||
["sources", "connect", "--all-detected", "--user-id", "wife"],
|
||||
["sources", "connect", "hermes", "--user-id", "bad:id"],
|
||||
]) {
|
||||
const result = await capturedRun(argv);
|
||||
expect(result.code).toBe(2);
|
||||
expect(result.stderr).toContain("hermes");
|
||||
}
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
async function capturedRun(arguments_: string[]) {
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
const stdoutSpy = vi
|
||||
.spyOn(process.stdout, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stdout += String(chunk);
|
||||
return true;
|
||||
});
|
||||
const stderrSpy = vi
|
||||
.spyOn(process.stderr, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stderr += String(chunk);
|
||||
return true;
|
||||
});
|
||||
try {
|
||||
return {
|
||||
code: await run(arguments_, {
|
||||
ONE_MEMORY_DATA_DIR: "/synthetic/data",
|
||||
}),
|
||||
stdout,
|
||||
stderr,
|
||||
};
|
||||
} finally {
|
||||
stdoutSpy.mockRestore();
|
||||
stderrSpy.mockRestore();
|
||||
}
|
||||
}
|
||||
@@ -12,7 +12,7 @@ dependencies = [
|
||||
"fastapi>=0.141.1",
|
||||
"filelock>=3.32.4",
|
||||
# Public SDK version validated against MemHub's permission and recall contracts.
|
||||
"hy-memory==2.0.8",
|
||||
"hy-memory==2.0.9",
|
||||
# chromadb arrives through the HY SDK; locked here so an unrelated upgrade
|
||||
# cannot move the SDK's store. Built-in embedding consumes numpy directly.
|
||||
"chromadb==1.5.9",
|
||||
|
||||
+1356
-117
File diff suppressed because it is too large.
Load diff
@@ -584,6 +584,7 @@ def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
|
||||
}
|
||||
else:
|
||||
from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE
|
||||
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot
|
||||
from one_memory.sdk_data_plane_migration import (
|
||||
MemoryAuthority,
|
||||
SdkDataPlaneMigrationStore,
|
||||
@@ -595,14 +596,14 @@ def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
|
||||
|
||||
authority = SdkDataPlaneMigrationStore(database_path).authority()
|
||||
if authority is MemoryAuthority.SDK:
|
||||
profile = read_active_profile(database_path)
|
||||
profile = read_active_profile(database_path) or builtin_retrieval_snapshot()
|
||||
leftovers = bool(
|
||||
legacy_tables_present(database_path)
|
||||
or abandoned_staging_present(
|
||||
data_dir=settings.data_dir,
|
||||
namespace=HY_MEMORY_NAMESPACE,
|
||||
)
|
||||
or (profile is not None and legacy_vector_collections(settings.data_dir, profile))
|
||||
or legacy_vector_collections(settings.data_dir, profile)
|
||||
)
|
||||
state = "required" if leftovers else "not_required"
|
||||
else:
|
||||
|
||||
@@ -5,6 +5,9 @@ from typing import Annotated, Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||
|
||||
from one_memory.sdk_search_options import SdkSearchOptions
|
||||
from one_memory.user_scope import DEFAULT_USER_ID, UserId
|
||||
|
||||
ContextKind = Literal["raw", "memory"]
|
||||
ContextSearchChannel = Literal["lexical", "semantic"]
|
||||
ContextSemanticStatus = Literal["not_searched", "completed", "failed", "timed_out"]
|
||||
@@ -47,17 +50,24 @@ class ContextSearchRequest(BaseModel):
|
||||
return self
|
||||
|
||||
|
||||
class SdkContextSearchRequest(ContextSearchRequest):
|
||||
class SdkContextSearchRequest(ContextSearchRequest, SdkSearchOptions):
|
||||
"""`POST /api/v2/context/search`: the three SDK recall lanes are set apart.
|
||||
|
||||
`limit` is the normal lane's top-k and the cap on returned items;
|
||||
`schema_limit` and `intention_limit` are the SDK's schema (L6) and
|
||||
proactive intention (L7) lanes. A caller's values are passed through.
|
||||
|
||||
The `SdkSearchOptions` fields reach the memory lane's `client.search`;
|
||||
the raw lane takes the floor and the source, session, agent and time
|
||||
filters (never wider than the caller's raw authority). Absent, both
|
||||
lanes make the call they always made.
|
||||
"""
|
||||
|
||||
limit: int = Field(default=10, ge=1, le=20, strict=True)
|
||||
schema_limit: int = Field(default=3, ge=0, le=20, strict=True)
|
||||
intention_limit: int = Field(default=1, ge=0, le=20, strict=True)
|
||||
#: Whose memories to search. The App never sends it and reads `local`.
|
||||
user_id: UserId = DEFAULT_USER_ID
|
||||
|
||||
|
||||
class ContextMessage(BaseModel):
|
||||
|
||||
@@ -24,13 +24,57 @@ from one_memory.hy_runtime_config import HY_MEMORY_READER
|
||||
from one_memory.hy_sdk_runtime import sdk_relevance_floor
|
||||
from one_memory.hy_sdk_types import (
|
||||
HySdkMemoryRecord,
|
||||
parse_sdk_lane_search,
|
||||
parse_sdk_raw_search,
|
||||
parse_sdk_search,
|
||||
)
|
||||
from one_memory.imports.storage import EvidenceStore
|
||||
from one_memory.sdk_search_options import SdkSearchFilters, SearchLane
|
||||
from one_memory.semantic_deadline import SemanticMemoryHit
|
||||
from one_memory.user_scope import owns
|
||||
|
||||
CALL_EVENT_RESPONSE_BUDGET_MS = 250
|
||||
|
||||
#: Pass-through filters the raw lane honours. Scene, layers, reader and the
|
||||
#: schema/working floors describe memory lanes; the raw lane is always the
|
||||
#: chat store's L1 raw layer.
|
||||
_RAW_LANE_FILTERS = ("created_after", "session_ids", "agent_ids", "exclude_sources", "rerank")
|
||||
|
||||
|
||||
def raw_lane_sources(
|
||||
allowed: Sequence[str] | frozenset[str] | None,
|
||||
requested: Sequence[str] | None,
|
||||
) -> list[str] | None:
|
||||
"""The raw lane's source filter: the caller's, never wider than its authority.
|
||||
|
||||
`allowed` empty or None means every source (the call Core always made).
|
||||
None back means nothing is left to search.
|
||||
"""
|
||||
allowed_list = list(allowed or ())
|
||||
if requested is None:
|
||||
return allowed_list
|
||||
chosen = [
|
||||
source for source in dict.fromkeys(requested) if not allowed_list or source in allowed_list
|
||||
]
|
||||
return chosen or None
|
||||
|
||||
|
||||
def raw_lane_overrides(value: SdkSearchFilters) -> dict[str, object]:
|
||||
"""The caller's raw-lane overrides: its floor and its row filters."""
|
||||
overrides = value.filter_kwargs()
|
||||
found = {name: overrides[name] for name in _RAW_LANE_FILTERS if name in overrides}
|
||||
floor = value.lane_floor("raw")
|
||||
if floor is not None:
|
||||
found["min_score"] = floor
|
||||
return found
|
||||
|
||||
|
||||
def hit_namespace(hit: SemanticMemoryHit) -> dict[str, str]:
|
||||
"""The `get` namespace of a hit: nothing for chat, kind and variant otherwise."""
|
||||
if hit.kind in {"experience", "fact"} and hit.variant is not None:
|
||||
return {"kind": hit.kind, "variant": hit.variant}
|
||||
return {}
|
||||
|
||||
|
||||
def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
"""SDK-native context: no Evidence/Governance ids or fabricated revisions."""
|
||||
@@ -66,26 +110,36 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
client = await client_handle.client()
|
||||
except Exception:
|
||||
raise HTTPException(503, "sdk_context_unavailable") from None
|
||||
memory_mode = getattr(request.app.state, "memory_mode", None)
|
||||
lite = callable(memory_mode) and memory_mode() == "lite"
|
||||
|
||||
async def raw_lane() -> tuple[SdkRawContextItem, ...]:
|
||||
if "raw" not in value.kinds or not authority.raw_allowed:
|
||||
return ()
|
||||
sources = raw_lane_sources(authority.raw_sources, value.include_sources)
|
||||
if sources is None:
|
||||
return ()
|
||||
raw = await asyncio.to_thread(
|
||||
client.search,
|
||||
value.query,
|
||||
user_ids=["local"],
|
||||
agent_ids=[],
|
||||
include_sources=list(authority.raw_sources or ()),
|
||||
limit=value.limit,
|
||||
min_score=0.0,
|
||||
schema_min_score=0.0,
|
||||
schema_limit=0,
|
||||
intention_limit=0,
|
||||
scene="chat",
|
||||
reader=HY_MEMORY_READER,
|
||||
layers=["l1_raw"],
|
||||
**{
|
||||
"user_ids": [value.user_id],
|
||||
"agent_ids": [],
|
||||
"include_sources": sources,
|
||||
"limit": value.limit,
|
||||
"min_score": 0.0,
|
||||
"schema_min_score": 0.0,
|
||||
"schema_limit": 0,
|
||||
"intention_limit": 0,
|
||||
"scene": "chat",
|
||||
"reader": HY_MEMORY_READER,
|
||||
"layers": ["l1_raw"],
|
||||
**raw_lane_overrides(value),
|
||||
},
|
||||
)
|
||||
hits = tuple(
|
||||
hit for hit in parse_sdk_raw_search(raw) if value.passes_floor("raw", hit.score)
|
||||
)
|
||||
hits = parse_sdk_raw_search(raw)
|
||||
return tuple(
|
||||
SdkRawContextItem(
|
||||
raw_id=hit.raw_id,
|
||||
@@ -103,27 +157,48 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
)
|
||||
|
||||
async def memory_lane() -> tuple[SdkMemoryContextItem, ...]:
|
||||
if "memory" not in value.kinds or not authority.memory_allowed:
|
||||
# lite extracts nothing: its search answers raw turns, which the
|
||||
# raw lane already reads under the raw authority.
|
||||
if "memory" not in value.kinds or not authority.memory_allowed or lite:
|
||||
return ()
|
||||
overrides = value.search_overrides()
|
||||
raw = await asyncio.to_thread(
|
||||
client.search,
|
||||
value.query,
|
||||
user_ids=["local"],
|
||||
agent_ids=[],
|
||||
limit=value.limit,
|
||||
min_score=sdk_relevance_floor(),
|
||||
schema_min_score=sdk_relevance_floor(),
|
||||
schema_limit=value.schema_limit,
|
||||
intention_limit=value.intention_limit,
|
||||
scene="normal",
|
||||
reader=HY_MEMORY_READER,
|
||||
**{
|
||||
"user_ids": [value.user_id],
|
||||
"agent_ids": [],
|
||||
"limit": value.limit,
|
||||
"min_score": sdk_relevance_floor(),
|
||||
"schema_min_score": sdk_relevance_floor(),
|
||||
"schema_limit": value.schema_limit,
|
||||
"intention_limit": value.intention_limit,
|
||||
"scene": "normal",
|
||||
"reader": HY_MEMORY_READER,
|
||||
**overrides,
|
||||
},
|
||||
)
|
||||
hits = (
|
||||
tuple(
|
||||
hit
|
||||
for hit in parse_sdk_lane_search(
|
||||
raw, error_code="hy_memory_search_shape_unknown"
|
||||
)
|
||||
if value.passes_floor(cast(SearchLane, hit.lane), hit.score)
|
||||
)
|
||||
if overrides
|
||||
else parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
|
||||
)
|
||||
hits = parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
|
||||
items: list[SdkMemoryContextItem] = []
|
||||
for hit in hits:
|
||||
record = await asyncio.to_thread(client.get, hit.memory_id)
|
||||
record = await asyncio.to_thread(client.get, hit.memory_id, **hit_namespace(hit))
|
||||
memory = None if record is None else HySdkMemoryRecord.from_sdk(record)
|
||||
if memory is None or memory.status != "active":
|
||||
# `get` takes no user: a hit is kept only if it is the caller's.
|
||||
if (
|
||||
memory is None
|
||||
or memory.status != "active"
|
||||
or not owns(memory.user_id, value.user_id)
|
||||
):
|
||||
continue
|
||||
items.append(
|
||||
SdkMemoryContextItem(
|
||||
|
||||
@@ -71,6 +71,16 @@ def graph_store_enabled() -> bool:
|
||||
return raw != "none"
|
||||
|
||||
|
||||
def effective_memory_mode(configured: str = "pro") -> str:
|
||||
"""The SDK mode clients are built with: the setting, unless dev forces ultra.
|
||||
|
||||
`ONE_MEMORY_GRAPH_STORE` (any value but empty or `none`) predates the
|
||||
setting and keeps working for development: it forces ultra whatever the
|
||||
configured mode. Without it the configured mode (default pro) applies.
|
||||
"""
|
||||
return "ultra" if graph_store_enabled() else configured
|
||||
|
||||
|
||||
def inspect_graph_width_mismatch(config: Any) -> GraphWidthMismatch | None:
|
||||
"""Return a mismatch without renaming or deleting anything."""
|
||||
db_path = Path(config.graph_store.db_path)
|
||||
@@ -83,11 +93,12 @@ def inspect_profile_graph_width(
|
||||
*,
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
mode: str | None = None,
|
||||
) -> GraphWidthMismatch | None:
|
||||
"""Inspect the graph path and width selected by the active profile."""
|
||||
del api_key
|
||||
return _inspect_graph_width(
|
||||
_profile_graph_path(data_dir),
|
||||
_profile_graph_path(data_dir, mode),
|
||||
_profile_embedding_width(profile),
|
||||
)
|
||||
|
||||
@@ -98,11 +109,12 @@ def decide_profile_graph_width(
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
decision: GraphWidthDecision,
|
||||
mode: str | None = None,
|
||||
) -> GraphWidthOutcome:
|
||||
"""Apply the user's decision to the active profile's graph."""
|
||||
del api_key
|
||||
config = _GraphAdminConfig(
|
||||
graph_store=_GraphStoreConfig(str(_profile_graph_path(data_dir))),
|
||||
graph_store=_GraphStoreConfig(str(_profile_graph_path(data_dir, mode))),
|
||||
embedder=_EmbeddingWidthConfig(_profile_embedding_width(profile)),
|
||||
)
|
||||
return apply_graph_width_decision(
|
||||
@@ -240,8 +252,9 @@ def _inspect_graph_width(
|
||||
)
|
||||
|
||||
|
||||
def _profile_graph_path(data_dir: Path) -> Path:
|
||||
name = "graph" if graph_store_enabled() else "graph-disabled"
|
||||
def _profile_graph_path(data_dir: Path, mode: str | None = None) -> Path:
|
||||
ultra = mode == "ultra" if mode is not None else graph_store_enabled()
|
||||
name = "graph" if ultra else "graph-disabled"
|
||||
return data_dir / "hy-memory" / HY_MEMORY_NAMESPACE / name
|
||||
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ from pathlib import Path
|
||||
from typing import Literal, cast
|
||||
|
||||
HY_MEMORY_DISTRIBUTION = "hy-memory"
|
||||
HY_MEMORY_VERSION = "2.0.8"
|
||||
HY_MEMORY_VERSION = "2.0.9"
|
||||
HY_MEMORY_NAMESPACE = "governed-v2"
|
||||
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
|
||||
|
||||
|
||||
@@ -497,7 +497,7 @@ def _ensure_coding_schema(path: Path) -> None:
|
||||
CodingMemoryStore,
|
||||
)
|
||||
|
||||
# Relies on the pinned SDK's internals (hy-memory==2.0.8 is pinned exactly).
|
||||
# Relies on the pinned SDK's internals (hy-memory==2.0.9 is pinned exactly).
|
||||
store = CodingMemoryStore.__new__(CodingMemoryStore)
|
||||
store._db_path = str(path)
|
||||
store._conn = None
|
||||
|
||||
@@ -31,7 +31,7 @@ from one_memory.embedding_assets import (
|
||||
)
|
||||
from one_memory.hy_graph_admin import (
|
||||
GraphWidthMismatchError,
|
||||
graph_store_enabled,
|
||||
effective_memory_mode,
|
||||
inspect_graph_width_mismatch,
|
||||
)
|
||||
from one_memory.hy_sdk_types import HySdkRuntimeError
|
||||
@@ -170,11 +170,14 @@ class SdkClientHandle:
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
http_transport_factory: Callable[[], Any] | None = None,
|
||||
mode: str | None = None,
|
||||
) -> None:
|
||||
self._profile = profile
|
||||
self._api_key = api_key
|
||||
self._data_dir = data_dir
|
||||
self._http_transport_factory = http_transport_factory
|
||||
#: The SDK mode this handle's client is built with, fixed for its life.
|
||||
self.mode = mode or effective_memory_mode()
|
||||
self._lease: ClientLease | None = None
|
||||
self._lock = asyncio.Lock()
|
||||
|
||||
@@ -189,6 +192,7 @@ class SdkClientHandle:
|
||||
api_key=self._api_key,
|
||||
data_dir=self._data_dir,
|
||||
http_transport_factory=self._http_transport_factory,
|
||||
mode=self.mode,
|
||||
)
|
||||
return self._lease.client
|
||||
|
||||
@@ -217,6 +221,33 @@ def builtin_retrieval_snapshot() -> ModelProfileSnapshot:
|
||||
)
|
||||
|
||||
|
||||
def retrieval_view(profile: ModelProfileSnapshot) -> ModelProfileSnapshot:
|
||||
"""The read side of `profile`: its embedding space, and no generation.
|
||||
|
||||
Reads must open the collection writes fill (#242), and the collection is
|
||||
named by the embedding only. Generation stays the built-in placeholder, so
|
||||
a read never needs the generation model. A remote embedding still needs
|
||||
the profile's API key, which `build_sdk_config` uses as the embedding key.
|
||||
"""
|
||||
provider = profile.embedding_provider_kind or profile.provider_kind
|
||||
endpoint: str | None = None
|
||||
if provider is not ProviderKind.LOCAL_ONNX:
|
||||
endpoint = profile.embedding_base_url
|
||||
if not endpoint:
|
||||
raise HySdkRuntimeError("hy_sdk_model_endpoint_missing")
|
||||
return builtin_retrieval_snapshot().model_copy(
|
||||
update={
|
||||
"profile_id": profile.profile_id,
|
||||
"revision": profile.revision,
|
||||
"embedding_provider_kind": provider,
|
||||
"embedding_model": profile.embedding_model,
|
||||
"embedding_base_url": endpoint,
|
||||
"embedding_dimensions": profile.embedding_dimensions,
|
||||
"has_api_key": profile.has_api_key,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def sdk_relevance_floor() -> float:
|
||||
raw = os.environ.get("ONE_MEMORY_HY_RELEVANCE_FLOOR", "").strip()
|
||||
if not raw:
|
||||
@@ -249,10 +280,8 @@ def build_sdk_config(
|
||||
embedding_key = None
|
||||
dimensions = profile.embedding_dimensions or BUILTIN_EMBEDDING_DIMENSIONS
|
||||
else:
|
||||
embedding_endpoint = _openai_endpoint(
|
||||
embedding_provider,
|
||||
profile.embedding_base_url or profile.base_url,
|
||||
)
|
||||
# Never the generation endpoint: the collection name hashes this one.
|
||||
embedding_endpoint = _openai_endpoint(embedding_provider, profile.embedding_base_url)
|
||||
embedding_key = "ollama" if embedding_provider is ProviderKind.OLLAMA else api_key
|
||||
if embedding_provider is ProviderKind.OPENAI_COMPATIBLE and not embedding_key:
|
||||
raise HySdkRuntimeError("hy_sdk_embedding_api_key_missing")
|
||||
@@ -327,9 +356,12 @@ def acquire_client(
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
http_transport_factory: Callable[[], Any] | None = None,
|
||||
mode: str | None = None,
|
||||
) -> ClientLease:
|
||||
"""Construct a public SDK client and return its explicit runtime lease.
|
||||
|
||||
`mode` is the SDK pipeline mode (lite / pro / ultra); None keeps the
|
||||
pre-setting default (ultra when `ONE_MEMORY_GRAPH_STORE` is set, else pro).
|
||||
Waits while `hold_stores_offline` holds this home's stores.
|
||||
"""
|
||||
key = data_dir.resolve()
|
||||
@@ -343,6 +375,7 @@ def acquire_client(
|
||||
api_key=api_key,
|
||||
data_dir=data_dir,
|
||||
http_transport_factory=http_transport_factory,
|
||||
mode=mode or effective_memory_mode(),
|
||||
)
|
||||
finally:
|
||||
with _STORE_GATE:
|
||||
@@ -381,6 +414,7 @@ def _acquire_client(
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
http_transport_factory: Callable[[], Any] | None,
|
||||
mode: str = "pro",
|
||||
) -> ClientLease:
|
||||
config = build_sdk_config(profile, api_key=api_key, data_dir=data_dir)
|
||||
config.http_transport_factory = http_transport_factory
|
||||
@@ -398,7 +432,7 @@ def _acquire_client(
|
||||
with constructing:
|
||||
client = HyMemoryClient(
|
||||
config=config,
|
||||
mode="ultra" if graph_store_enabled() else "pro",
|
||||
mode=mode,
|
||||
runtime=runtime,
|
||||
embedding_service=embedding_service,
|
||||
)
|
||||
@@ -467,7 +501,7 @@ def vector_collection_bases(profile: ModelProfileSnapshot) -> tuple[str, tuple[s
|
||||
endpoint, dimensions = None, profile.embedding_dimensions or BUILTIN_EMBEDDING_DIMENSIONS
|
||||
legacy_endpoint = "local-onnx"
|
||||
else:
|
||||
endpoint = _openai_endpoint(provider, profile.embedding_base_url or profile.base_url)
|
||||
endpoint = _openai_endpoint(provider, profile.embedding_base_url)
|
||||
dimensions = profile.embedding_dimensions
|
||||
legacy_endpoint = endpoint
|
||||
legacy_model = profile.embedding_model or "bge-small-zh-v1.5"
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from collections.abc import Mapping
|
||||
from datetime import UTC, datetime
|
||||
from typing import Literal, NamedTuple, cast
|
||||
@@ -391,6 +392,8 @@ class HySdkRawRecord(BaseModel):
|
||||
memory_at: int | None = None
|
||||
gmt_created: int | None = None
|
||||
custom: dict[str, object] = Field(default_factory=dict)
|
||||
#: active, or shadow once pro extracted it; None when the SDK gave none.
|
||||
status: str | None = None
|
||||
|
||||
@classmethod
|
||||
def from_sdk(cls, raw: object) -> HySdkRawRecord:
|
||||
@@ -417,11 +420,34 @@ class HySdkRawRecord(BaseModel):
|
||||
raw.get("gmt_created") if isinstance(raw.get("gmt_created"), int) else None
|
||||
),
|
||||
custom=dict(custom),
|
||||
status=str(raw["status"]) if isinstance(raw.get("status"), str) else None,
|
||||
)
|
||||
except (KeyError, TypeError, ValueError) as error:
|
||||
raise HySdkReadError("hy_memory_raw_shape_unknown") from error
|
||||
|
||||
|
||||
_RAW_TURN_LINE = re.compile(r"^\[(user|assistant|tool|system)\]:\s?(.*)$")
|
||||
|
||||
|
||||
def raw_turn_messages(content: str) -> tuple[tuple[str, str], ...]:
|
||||
"""The `(role, text)` messages of one SDK raw turn.
|
||||
|
||||
The SDK stores a turn as `[user]: …\\n[assistant]: …`; a line without a
|
||||
role prefix continues the message above it. Content that names no role
|
||||
at all is one message of unknown role (`""`).
|
||||
"""
|
||||
messages: list[list[str]] = []
|
||||
for line in content.splitlines():
|
||||
match = _RAW_TURN_LINE.match(line)
|
||||
if match is not None:
|
||||
messages.append([match.group(1), match.group(2)])
|
||||
elif messages:
|
||||
messages[-1][1] += "\n" + line
|
||||
else:
|
||||
messages.append(["", line])
|
||||
return tuple((role, text) for role, text in messages)
|
||||
|
||||
|
||||
class HySdkRawHit(HySdkRawRecord):
|
||||
score: float = Field(ge=0, le=1)
|
||||
|
||||
@@ -516,6 +542,96 @@ class HySdkUserProfile(BaseModel):
|
||||
)
|
||||
|
||||
|
||||
class HySdkProfileVersion(BaseModel):
|
||||
"""One version on an L0 basic-info chain."""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
memory_id: str
|
||||
content: str
|
||||
#: Conversation time as the SDK renders it on a chain (epoch or text).
|
||||
memory_at: int | str | None = None
|
||||
gmt_created: int | None = None
|
||||
#: The chain head: the version the profile shows now.
|
||||
current: bool = False
|
||||
|
||||
|
||||
class HySdkProfileChain(BaseModel):
|
||||
"""One L0 basic-info head and its versions, latest first."""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
memory_id: str
|
||||
content: str
|
||||
status: str
|
||||
memory_at: int | None = None
|
||||
gmt_created: int | None = None
|
||||
#: The structured fields this head holds (`basic_info_kv`).
|
||||
basic_info: dict[str, object] = Field(default_factory=dict)
|
||||
versions: tuple[HySdkProfileVersion, ...]
|
||||
|
||||
|
||||
class HySdkProfileVersions(BaseModel):
|
||||
"""`get_user_basic_info`: the long-term profile's L0 version chains."""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
request_id: str
|
||||
chains: tuple[HySdkProfileChain, ...]
|
||||
elapsed_ms: float = Field(ge=0)
|
||||
|
||||
@classmethod
|
||||
def from_sdk(cls, raw: object) -> HySdkProfileVersions:
|
||||
if not isinstance(raw, Mapping):
|
||||
raise HySdkReadError("hy_memory_profile_versions_shape_unknown")
|
||||
request_id = raw.get("request_id")
|
||||
rows = raw.get("memories")
|
||||
elapsed_ms = raw.get("elapsed_ms")
|
||||
if (
|
||||
not isinstance(request_id, str)
|
||||
or not isinstance(rows, list)
|
||||
or not isinstance(elapsed_ms, int | float)
|
||||
or isinstance(elapsed_ms, bool)
|
||||
):
|
||||
raise HySdkReadError("hy_memory_profile_versions_shape_unknown")
|
||||
chains: list[HySdkProfileChain] = []
|
||||
for row in rows:
|
||||
if not isinstance(row, Mapping) or not isinstance(row.get("memory_id"), str):
|
||||
raise HySdkReadError("hy_memory_profile_versions_shape_unknown")
|
||||
head_id = str(row["memory_id"])
|
||||
chain = row.get("evolution_chain")
|
||||
entries = chain if isinstance(chain, list) and chain else [{**row, "node_id": head_id}]
|
||||
versions = tuple(
|
||||
HySdkProfileVersion(
|
||||
memory_id=str(entry.get("node_id") or entry.get("memory_id")),
|
||||
content=str(entry.get("content") or ""),
|
||||
memory_at=(
|
||||
entry.get("memory_at")
|
||||
if isinstance(entry.get("memory_at"), int | str)
|
||||
and not isinstance(entry.get("memory_at"), bool)
|
||||
else None
|
||||
),
|
||||
gmt_created=_epoch(entry.get("gmt_created")),
|
||||
current=str(entry.get("node_id") or entry.get("memory_id")) == head_id,
|
||||
)
|
||||
for entry in entries
|
||||
if isinstance(entry, Mapping) and (entry.get("node_id") or entry.get("memory_id"))
|
||||
)
|
||||
basic_info = row.get("basic_info_kv")
|
||||
chains.append(
|
||||
HySdkProfileChain(
|
||||
memory_id=head_id,
|
||||
content=str(row.get("content") or ""),
|
||||
status=str(row.get("status") or "active"),
|
||||
memory_at=_epoch(row.get("memory_at")),
|
||||
gmt_created=_epoch(row.get("gmt_created")),
|
||||
basic_info=dict(basic_info) if isinstance(basic_info, Mapping) else {},
|
||||
versions=versions,
|
||||
)
|
||||
)
|
||||
return cls(request_id=request_id, chains=tuple(chains), elapsed_ms=float(elapsed_ms))
|
||||
|
||||
|
||||
def parse_sdk_search(raw: object, *, error_code: str) -> tuple[SemanticMemoryHit, ...]:
|
||||
rows = _reply_rows(raw, error_code)
|
||||
found: list[SemanticMemoryHit] = []
|
||||
@@ -542,6 +658,120 @@ def parse_sdk_search(raw: object, *, error_code: str) -> tuple[SemanticMemoryHit
|
||||
return tuple(found)
|
||||
|
||||
|
||||
def parse_sdk_lane_search(raw: object, *, error_code: str) -> tuple[SemanticMemoryHit, ...]:
|
||||
"""Every scene's search reply as hits that name their lane and namespace.
|
||||
|
||||
`parse_sdk_search` reads the chat reply only and forgets which lane a hit
|
||||
came from. A caller may now pick the scene and a floor per lane, so this
|
||||
reads all three reply shapes:
|
||||
|
||||
- chat: `memories` as `{schema, proactive, normal}` (or one list, normal);
|
||||
- working/coding: `facts` and `experiences` of that variant;
|
||||
- all: one fused `memories` list whose rows carry `scene`, `kind` and
|
||||
their own lane's score (`lane_score`). The fused `score` is a rank
|
||||
fusion (1/(60+rank)), not a relevance, so the lane score is reported.
|
||||
|
||||
Highest score first, first occurrence of an id kept.
|
||||
"""
|
||||
if not isinstance(raw, Mapping):
|
||||
raise HySdkReadError(error_code)
|
||||
scene = raw.get("scene")
|
||||
found: list[SemanticMemoryHit] = []
|
||||
if scene == "all":
|
||||
rows = raw.get("memories")
|
||||
if not isinstance(rows, list) or any(not isinstance(row, Mapping) for row in rows):
|
||||
raise HySdkReadError(error_code)
|
||||
for row in rows:
|
||||
hit = _lane_hit(row, scene=str(row.get("scene") or "chat"), score_key="lane_score")
|
||||
if hit is not None:
|
||||
found.append(hit)
|
||||
elif scene in ("productivity", "working", "coding"):
|
||||
variant = raw.get("variant") if raw.get("variant") in {"working", "coding"} else None
|
||||
lane = variant or (scene if scene in {"working", "coding"} else "working")
|
||||
for key, kind in (("facts", "fact"), ("experiences", "experience")):
|
||||
rows = raw.get(key) or []
|
||||
if not isinstance(rows, list) or any(not isinstance(row, Mapping) for row in rows):
|
||||
raise HySdkReadError(error_code)
|
||||
for row in rows:
|
||||
hit = _lane_hit({**row, "kind": kind}, scene=lane, score_key="score")
|
||||
if hit is not None:
|
||||
found.append(hit)
|
||||
else:
|
||||
memories = raw.get("memories")
|
||||
if isinstance(memories, Mapping):
|
||||
if set(memories) != {"schema", "proactive", "normal"}:
|
||||
raise HySdkReadError(error_code)
|
||||
channels = tuple(memories.items())
|
||||
elif isinstance(memories, list):
|
||||
channels = (("normal", memories),)
|
||||
else:
|
||||
raise HySdkReadError(error_code)
|
||||
for lane, rows in channels:
|
||||
if not isinstance(rows, list) or any(not isinstance(row, Mapping) for row in rows):
|
||||
raise HySdkReadError(error_code)
|
||||
for row in rows:
|
||||
hit = _lane_hit(row, scene="chat", score_key="score", lane=str(lane))
|
||||
if hit is not None:
|
||||
found.append(hit)
|
||||
ordered: list[SemanticMemoryHit] = []
|
||||
seen: set[tuple[str, str, str | None]] = set()
|
||||
for hit in sorted(found, key=lambda value: value.score, reverse=True):
|
||||
identity = (hit.memory_id, hit.kind, hit.variant)
|
||||
if identity in seen:
|
||||
continue
|
||||
seen.add(identity)
|
||||
ordered.append(hit)
|
||||
return tuple(ordered)
|
||||
|
||||
|
||||
def _lane_hit(
|
||||
row: Mapping[str, object],
|
||||
*,
|
||||
scene: str,
|
||||
score_key: str,
|
||||
lane: str | None = None,
|
||||
) -> SemanticMemoryHit | None:
|
||||
kind = str(row.get("kind")) if row.get("kind") in {"experience", "fact"} else "chat"
|
||||
if kind == "chat":
|
||||
memory_id = row.get("memory_id") or row.get("id")
|
||||
variant = None
|
||||
hit_lane = lane or "normal"
|
||||
else:
|
||||
id_key = "experience_id" if kind == "experience" else "fact_id"
|
||||
memory_id = row.get("memory_id") or row.get(id_key)
|
||||
variant = scene if scene in {"working", "coding"} else None
|
||||
if variant is None:
|
||||
return None
|
||||
hit_lane = variant
|
||||
if not isinstance(memory_id, str) or not memory_id:
|
||||
return None
|
||||
value = row.get(score_key)
|
||||
if not isinstance(value, int | float) or isinstance(value, bool):
|
||||
value = row.get("score")
|
||||
score = (
|
||||
max(0.0, min(1.0, float(value)))
|
||||
if isinstance(value, int | float) and not isinstance(value, bool)
|
||||
else 0.0
|
||||
)
|
||||
if kind != "chat":
|
||||
return SemanticMemoryHit(
|
||||
memory_id=memory_id, score=score, lane=hit_lane, kind=kind, variant=variant
|
||||
)
|
||||
content = row.get("content")
|
||||
member_ids, formatting_complete = _formatted_member_ids(row)
|
||||
return SemanticMemoryHit(
|
||||
memory_id=memory_id,
|
||||
score=score,
|
||||
formatted_content=(
|
||||
content.strip() if isinstance(content, str) and content.strip() else None
|
||||
),
|
||||
member_ids=member_ids,
|
||||
sdk_formatted=True,
|
||||
formatting_complete=formatting_complete,
|
||||
lane=hit_lane,
|
||||
)
|
||||
|
||||
|
||||
class LibraryHit(NamedTuple):
|
||||
"""One keyword match in a library search window, with its namespace."""
|
||||
|
||||
|
||||
@@ -31,6 +31,16 @@ class ScanIssueCode(StrEnum):
|
||||
#: import it. Named for the reason, so a reader can tell a setting from
|
||||
#: a record the parser could not use.
|
||||
SUBAGENT_TRACE_DISABLED = "subagent_trace_disabled"
|
||||
#: A host's own notice wearing the user's role — Hermes posts a finished
|
||||
#: background process, and its gateway marks the post as not written by a
|
||||
#: human. Nothing in it is the person's, so it never reaches extraction.
|
||||
INJECTED_MACHINE_NOTICE = "injected_machine_notice"
|
||||
#: The host could not transcribe a voice message and left a placeholder
|
||||
#: naming the audio file. The person did speak, but the placeholder holds
|
||||
#: only a path, which an extractor would otherwise read as a fact.
|
||||
VOICE_PLACEHOLDER = "voice_placeholder"
|
||||
#: A health check such as "Reply with exactly: PONG": no one's words.
|
||||
PROBE = "probe"
|
||||
SOURCE_BUSY = "source_busy"
|
||||
UNSUPPORTED_SCHEMA = "unsupported_schema"
|
||||
BUDGET_EXCEEDED = "budget_exceeded"
|
||||
|
||||
@@ -27,6 +27,7 @@ def discovered_off_loop(
|
||||
*,
|
||||
suffixes: frozenset[str],
|
||||
max_targets: int,
|
||||
excluded_top_directories: frozenset[str] = frozenset(),
|
||||
) -> AsyncIterator[SourceFileDiscoveryEvent]:
|
||||
"""The same walk, for callers that have an event loop to keep free.
|
||||
|
||||
@@ -41,6 +42,7 @@ def discovered_off_loop(
|
||||
grant,
|
||||
suffixes=suffixes,
|
||||
max_targets=max_targets,
|
||||
excluded_top_directories=excluded_top_directories,
|
||||
),
|
||||
scope_key=grant.source_id,
|
||||
)
|
||||
@@ -51,12 +53,15 @@ def discover_source_files(
|
||||
*,
|
||||
suffixes: frozenset[str],
|
||||
max_targets: int,
|
||||
excluded_top_directories: frozenset[str] = frozenset(),
|
||||
) -> Iterator[SourceFileDiscoveryEvent]:
|
||||
"""Metadata only: one lstat per candidate, no open and no per-file grant lookup.
|
||||
|
||||
The grant and root identity are checked once before the walk. Each listed
|
||||
file keeps the grant's check, which runs again when that file is opened;
|
||||
the open itself refuses symlinks and any inode other than the one listed.
|
||||
`excluded_top_directories` are directories directly under the root that
|
||||
are never entered, such as another user's Hermes profiles.
|
||||
"""
|
||||
grant.check_root()
|
||||
inspected = 0
|
||||
@@ -64,8 +69,12 @@ def discover_source_files(
|
||||
grant.root,
|
||||
followlinks=False,
|
||||
):
|
||||
at_root = Path(directory) == grant.root
|
||||
child_directories[:] = sorted(
|
||||
child for child in child_directories if not (Path(directory) / child).is_symlink()
|
||||
child
|
||||
for child in child_directories
|
||||
if not (Path(directory) / child).is_symlink()
|
||||
and not (at_root and child in excluded_top_directories)
|
||||
)
|
||||
for filename in sorted(filenames):
|
||||
if Path(filename).suffix.lower() not in suffixes:
|
||||
|
||||
@@ -252,6 +252,18 @@ class SourceGrantStore:
|
||||
).fetchone()
|
||||
return self._to_receipt(row) if row else None
|
||||
|
||||
def grants(self) -> tuple[tuple[SourceGrantReceipt, str], ...]:
|
||||
"""Every grant of every user, with its granted root path.
|
||||
|
||||
One source may hold several grants, one per `(user_id, source_id)`:
|
||||
a Hermes root for `local` and a Hermes profile for another user.
|
||||
"""
|
||||
with self._connect() as connection:
|
||||
rows = connection.execute(
|
||||
"SELECT * FROM source_grants ORDER BY source_id, user_id"
|
||||
).fetchall()
|
||||
return tuple((self._to_receipt(row), str(row["requested_path"])) for row in rows)
|
||||
|
||||
def require_receipt(self, *, user_id: str, source_id: str) -> SourceGrantReceipt:
|
||||
receipt = self.receipt(user_id=user_id, source_id=source_id)
|
||||
if receipt is None:
|
||||
|
||||
@@ -28,6 +28,7 @@ from one_memory.imports.adapters import (
|
||||
)
|
||||
from one_memory.imports.jobs import ImportJob, ImportJobStore, ImportStatus
|
||||
from one_memory.imports.supervisor import ImportSupervisor
|
||||
from one_memory.user_scope import DEFAULT_USER_ID, lane_key, split_lane
|
||||
|
||||
#: How long an automatic retry will wait for a wedged worker to notice cancel.
|
||||
ABANDON_TIMEOUT_SECONDS = 5.0
|
||||
@@ -64,7 +65,12 @@ class LiveSyncHealth(BaseModel):
|
||||
|
||||
|
||||
class PollingLiveSyncSupervisor:
|
||||
"""Near-live sync that reuses each adapter's persisted incremental cursor."""
|
||||
"""Near-live sync that reuses each adapter's persisted incremental cursor.
|
||||
|
||||
A "source" here is a lane key (`user_scope.lane_key`): the bare source id
|
||||
for `local`, `source@user` for another user's grant. Cursors, jobs and
|
||||
locks are therefore per `(user_id, source_id)`.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
@@ -92,8 +98,22 @@ class PollingLiveSyncSupervisor:
|
||||
}
|
||||
self._locks = {source_id: asyncio.Lock() for source_id in self.source_ids}
|
||||
self._task: asyncio.Task[None] | None = None
|
||||
self._group: asyncio.TaskGroup | None = None
|
||||
self._observers: set[asyncio.Task[None]] = set()
|
||||
|
||||
def ensure_source(self, source_id: str) -> None:
|
||||
"""Follow a lane granted after start, such as a new Hermes profile."""
|
||||
if source_id in self._locks:
|
||||
return
|
||||
self.source_ids = (*self.source_ids, source_id)
|
||||
self._health.setdefault(
|
||||
source_id,
|
||||
LiveSyncHealth(source_id=source_id, mode=self.mode, state=LiveSyncState.IDLE),
|
||||
)
|
||||
self._locks[source_id] = asyncio.Lock()
|
||||
if self._group is not None:
|
||||
self._group.create_task(self._poll_source(source_id))
|
||||
|
||||
def _set_health(self, source_id: str, value: LiveSyncHealth) -> None:
|
||||
previous = self._health.get(source_id)
|
||||
self._health[source_id] = value
|
||||
@@ -152,7 +172,12 @@ class PollingLiveSyncSupervisor:
|
||||
self._watched_sources = frozenset(source_ids)
|
||||
|
||||
def all_health(self) -> tuple[LiveSyncHealth, ...]:
|
||||
return tuple(self.health(source_id) for source_id in self.source_ids)
|
||||
"""`local`'s lanes only: what the App has always been shown."""
|
||||
return tuple(
|
||||
self.health(source_id)
|
||||
for source_id in self.source_ids
|
||||
if split_lane(source_id)[0] == DEFAULT_USER_ID
|
||||
)
|
||||
|
||||
async def initialize_source(
|
||||
self, source_id: str, *, baseline_missing_files: bool = True
|
||||
@@ -164,11 +189,12 @@ class PollingLiveSyncSupervisor:
|
||||
baselines = (
|
||||
await adapter.live_baseline() if isinstance(adapter, _LiveBaselineAdapter) else {}
|
||||
)
|
||||
user_id, bare_source_id = split_lane(source_id)
|
||||
await in_the_background(
|
||||
partial(
|
||||
self.job_store.ensure_live_cursors,
|
||||
user_id="local",
|
||||
source_id=source_id,
|
||||
user_id=user_id,
|
||||
source_id=bare_source_id,
|
||||
baselines=baselines,
|
||||
baseline_missing_files=baseline_missing_files,
|
||||
)
|
||||
@@ -228,7 +254,7 @@ class PollingLiveSyncSupervisor:
|
||||
(
|
||||
job
|
||||
for job in reversed(source_jobs)
|
||||
if job.source_id == source_id
|
||||
if lane_key(job.user_id, job.source_id) == source_id
|
||||
and job.operation == "sync"
|
||||
and job.status
|
||||
in {
|
||||
@@ -277,13 +303,13 @@ class PollingLiveSyncSupervisor:
|
||||
competing = [
|
||||
job
|
||||
for job in source_jobs
|
||||
if job.source_id == source_id
|
||||
if lane_key(job.user_id, job.source_id) == source_id
|
||||
and job.operation != "sync"
|
||||
and job.status
|
||||
in {ImportStatus.PENDING, ImportStatus.RUNNING, ImportStatus.INTERRUPTED}
|
||||
]
|
||||
bounded_history_incomplete = any(
|
||||
job.source_id == source_id
|
||||
lane_key(job.user_id, job.source_id) == source_id
|
||||
and job.operation == "history"
|
||||
and job.history_after is not None
|
||||
and job.status is not ImportStatus.COMPLETED
|
||||
@@ -379,11 +405,12 @@ class PollingLiveSyncSupervisor:
|
||||
await close()
|
||||
|
||||
async def _live_cursors(self, source_id: str, adapter: object) -> dict[str, SourceCursor]:
|
||||
user_id, bare_source_id = split_lane(source_id)
|
||||
cursors = await in_the_background(
|
||||
partial(
|
||||
self.job_store.get_source_cursors,
|
||||
user_id="local",
|
||||
source_id=source_id,
|
||||
user_id=user_id,
|
||||
source_id=bare_source_id,
|
||||
operation="sync",
|
||||
)
|
||||
)
|
||||
@@ -395,8 +422,8 @@ class PollingLiveSyncSupervisor:
|
||||
return await in_the_background(
|
||||
partial(
|
||||
self.job_store.ensure_live_cursors,
|
||||
user_id="local",
|
||||
source_id=source_id,
|
||||
user_id=user_id,
|
||||
source_id=bare_source_id,
|
||||
baselines=baselines,
|
||||
)
|
||||
)
|
||||
@@ -520,9 +547,13 @@ class PollingLiveSyncSupervisor:
|
||||
async def _run(self) -> None:
|
||||
# A source owns its cadence; one large catch-up cannot delay every
|
||||
# other source. The group keeps shutdown responsible for every poller.
|
||||
async with asyncio.TaskGroup() as group:
|
||||
for source_id in self.source_ids:
|
||||
group.create_task(self._poll_source(source_id))
|
||||
try:
|
||||
async with asyncio.TaskGroup() as group:
|
||||
self._group = group
|
||||
for source_id in self.source_ids:
|
||||
group.create_task(self._poll_source(source_id))
|
||||
finally:
|
||||
self._group = None
|
||||
|
||||
async def _poll_source(self, source_id: str) -> None:
|
||||
while True:
|
||||
|
||||
@@ -3,7 +3,7 @@ import logging
|
||||
import math
|
||||
import sqlite3
|
||||
from collections import Counter
|
||||
from collections.abc import Callable
|
||||
from collections.abc import AsyncIterator, Callable
|
||||
from contextlib import suppress
|
||||
from contextvars import ContextVar
|
||||
from dataclasses import dataclass, field
|
||||
@@ -227,14 +227,17 @@ class ImportRunner:
|
||||
heartbeat = asyncio.create_task(self._keep_alive(job_id=job_id, runner_token=runner_token))
|
||||
pending = _PendingSkips()
|
||||
writes = _PendingWrites()
|
||||
stream = adapter.scan(
|
||||
ScanOptions(
|
||||
cursors=cursors,
|
||||
reuse_listing=(
|
||||
job.operation == "history"
|
||||
or (job.operation == "sync" and job.source_id == "workbuddy")
|
||||
),
|
||||
)
|
||||
stream = _owned_by(
|
||||
adapter.scan(
|
||||
ScanOptions(
|
||||
cursors=cursors,
|
||||
reuse_listing=(
|
||||
job.operation == "history"
|
||||
or (job.operation == "sync" and job.source_id == "workbuddy")
|
||||
),
|
||||
)
|
||||
),
|
||||
job.user_id,
|
||||
)
|
||||
slice_cursors = dict(cursors)
|
||||
next_event: asyncio.Future[ScanEvent] | None = None
|
||||
@@ -924,3 +927,30 @@ async def _settled_work[T](
|
||||
with suppress(asyncio.CancelledError, Exception):
|
||||
task.result()
|
||||
raise
|
||||
|
||||
|
||||
def _owned_by(stream: AsyncIterator[ScanEvent], user_id: str) -> AsyncIterator[ScanEvent]:
|
||||
"""Stamp every scanned record with the user whose root is being read.
|
||||
|
||||
Adapters know nothing of users and emit `local`. Which user a record
|
||||
belongs to is decided by the grant the job reads (a Hermes profile's
|
||||
`state.db` is that profile's user), so the job's user is the record's.
|
||||
`local` passes the stream through untouched.
|
||||
"""
|
||||
if user_id == "local":
|
||||
return stream
|
||||
return _restamped(stream, user_id)
|
||||
|
||||
|
||||
async def _restamped(stream: AsyncIterator[ScanEvent], user_id: str) -> AsyncIterator[ScanEvent]:
|
||||
try:
|
||||
async for event in stream:
|
||||
if isinstance(event, ScannedEvidence) and event.evidence.user_id != user_id:
|
||||
event = event.model_copy(
|
||||
update={"evidence": event.evidence.model_copy(update={"user_id": user_id})}
|
||||
)
|
||||
yield event
|
||||
finally:
|
||||
close = getattr(stream, "aclose", None)
|
||||
if close is not None:
|
||||
await close()
|
||||
@@ -1,4 +1,5 @@
|
||||
import json
|
||||
import re
|
||||
import sqlite3
|
||||
from collections.abc import AsyncIterator, Iterator
|
||||
from functools import partial
|
||||
@@ -87,6 +88,7 @@ class HermesSourceAdapter:
|
||||
self.grant,
|
||||
suffixes=frozenset({".jsonl", ".db"}),
|
||||
max_targets=10_000,
|
||||
excluded_top_directories=frozenset({HERMES_PROFILES_DIRECTORY}),
|
||||
):
|
||||
if isinstance(item, AuthorizedSourceFile) and _is_hermes_target(
|
||||
item.relative_path.as_posix()
|
||||
@@ -100,6 +102,7 @@ class HermesSourceAdapter:
|
||||
self.grant,
|
||||
suffixes=frozenset({".jsonl", ".db"}),
|
||||
max_targets=options.max_scan_targets or 100_000,
|
||||
excluded_top_directories=frozenset({HERMES_PROFILES_DIRECTORY}),
|
||||
):
|
||||
yield ScanProgressEvent(
|
||||
phase=ScanPhase.DISCOVER,
|
||||
@@ -214,7 +217,11 @@ def parse_hermes_rollout_record(
|
||||
message.get("content"),
|
||||
allowed_block_types=frozenset({"text", "input_text", "output_text"}),
|
||||
)
|
||||
text = strip_injected_memory(text)
|
||||
text = strip_injected_memory(text or "")
|
||||
if role == "user":
|
||||
text, injected = gateway_user_text(text)
|
||||
if injected is not None:
|
||||
return ParsedSkip(reason_code=injected, conversation_id=conversation_id)
|
||||
if not text:
|
||||
return ParsedSkip(
|
||||
reason_code=ScanIssueCode.NO_TEXT,
|
||||
@@ -242,8 +249,17 @@ def parse_hermes_rollout_record(
|
||||
)
|
||||
|
||||
|
||||
#: Hermes keeps each extra profile under `<root>/profiles/<name>/`, with its
|
||||
#: own `state.db` and sessions. A profile is another person's history: it is
|
||||
#: read only through its own grant (its own MemHub user), never as part of the
|
||||
#: root above it. Hermes' own export skips this directory for the same reason.
|
||||
HERMES_PROFILES_DIRECTORY = "profiles"
|
||||
|
||||
|
||||
def _is_hermes_target(relative_path: str) -> bool:
|
||||
path = PurePosixPath(relative_path)
|
||||
if path.parts and path.parts[0] == HERMES_PROFILES_DIRECTORY:
|
||||
return False
|
||||
return path.name == "state.db" or (path.suffix == ".jsonl" and "sessions" in path.parts)
|
||||
|
||||
|
||||
@@ -389,6 +405,15 @@ def _read_state_db(
|
||||
# `tool` rows after it hold only what came back.
|
||||
calls = _tool_calls_text(row["tool_calls"]) if role == "assistant" else None
|
||||
content = strip_injected_memory(str(row["content"] or ""))
|
||||
if role == "user":
|
||||
content, injected = gateway_user_text(content)
|
||||
if injected is not None:
|
||||
yield ScanSkip(
|
||||
cursor=cursor,
|
||||
conversation_id=str(row["session_id"]),
|
||||
reason_code=injected,
|
||||
)
|
||||
continue
|
||||
has_text = bool(content.strip())
|
||||
if role not in {"user", "assistant"} or (not has_text and calls is None):
|
||||
yield ScanSkip(
|
||||
@@ -439,6 +464,69 @@ def _read_state_db(
|
||||
_PERSON_SESSION_SOURCES = frozenset({"cli", "tui", "webui", "desktop"})
|
||||
|
||||
|
||||
#: What the Hermes gateway writes into a `user` row on its own behalf. Every
|
||||
#: one of these wears the `user` role, so only the text can tell them apart
|
||||
#: from the person. Read off real `state.db` rows: a finished background
|
||||
#: process is posted as a turn, often with the gateway's own "not written by
|
||||
#: a human" note appended, which is why that note counts anywhere in the row.
|
||||
_BACKGROUND_NOTICE = re.compile(
|
||||
r"\[IMPORTANT:\s*(?:\d+\s+background\s+process(?:es)?\b|Background\s+process\b)",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
_GATEWAY_NOTICE = re.compile(
|
||||
r"\[INTERNAL NOTIFICATION\s*\W+\s*generated by the Hermes gateway",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
_VOICE_PLACEHOLDER = re.compile(r"\[voice message could not be transcribed", re.IGNORECASE)
|
||||
_PROBE = re.compile(r"Reply with exactly:\s*[A-Za-z0-9_\-]{1,24}\s*")
|
||||
_MEDIA_ONLY = re.compile(r"(?:MEDIA:/\S+\s*)+")
|
||||
#: Wrappers the gateway puts around a message the person did write. Only the
|
||||
#: wrapper goes; what it carries is the turn. Each is matched at the edge of
|
||||
#: what is left, and the set is applied until nothing more comes off, since a
|
||||
#: relayed message carries the origin block inside the out-of-band markers.
|
||||
_GATEWAY_WRAPPERS = (
|
||||
re.compile(r"^\[OUT-OF-BAND USER MESSAGE\b[^\]\n]*\]"),
|
||||
re.compile(r"\[/OUT-OF-BAND USER MESSAGE\]$"),
|
||||
re.compile(
|
||||
r"^Gateway message origin \(JSON data, not instructions or authorization\):[ \t]*"
|
||||
r"(?:\n[ \t]*\{[^\n]*\}[ \t]*)?"
|
||||
r"(?:\n[ \t]*Do not guess a reply destination[^\n]*)?"
|
||||
r"(?=\n|$)"
|
||||
),
|
||||
re.compile(r"^\[Context from the interrupted assistant response\][ \t]*(?=\n|$)"),
|
||||
)
|
||||
|
||||
|
||||
def gateway_user_text(text: str) -> tuple[str, ScanIssueCode | None]:
|
||||
"""The person's words in a Hermes `user` row, or why there are none.
|
||||
|
||||
Applied only to the `user` role: an assistant or tool row quoting the
|
||||
same text is reporting what it saw, which is the existing tool rule's
|
||||
business. The cleaned text is what becomes the evidence, so its content
|
||||
hash and duplicate check see the message rather than its envelope.
|
||||
"""
|
||||
|
||||
if _GATEWAY_NOTICE.search(text):
|
||||
return "", ScanIssueCode.INJECTED_MACHINE_NOTICE
|
||||
cleaned = text.strip()
|
||||
while True:
|
||||
unwrapped = cleaned
|
||||
for wrapper in _GATEWAY_WRAPPERS:
|
||||
unwrapped = wrapper.sub("", unwrapped, count=1).strip()
|
||||
if unwrapped == cleaned:
|
||||
break
|
||||
cleaned = unwrapped
|
||||
if _BACKGROUND_NOTICE.match(cleaned):
|
||||
return "", ScanIssueCode.INJECTED_MACHINE_NOTICE
|
||||
if _VOICE_PLACEHOLDER.match(cleaned):
|
||||
return "", ScanIssueCode.VOICE_PLACEHOLDER
|
||||
if _PROBE.fullmatch(cleaned):
|
||||
return "", ScanIssueCode.PROBE
|
||||
if _MEDIA_ONLY.fullmatch(cleaned):
|
||||
return "", ScanIssueCode.NO_TEXT
|
||||
return cleaned, None
|
||||
|
||||
|
||||
def _tool_calls_text(value: object) -> str | None:
|
||||
"""OpenAI-shaped `tool_calls`: `[{"function": {"name", "arguments"}}]`, stored as JSON."""
|
||||
if isinstance(value, str):
|
||||
|
||||
@@ -24,6 +24,7 @@ from one_memory.service_levels import (
|
||||
ServiceLevelScheduler,
|
||||
ServiceResource,
|
||||
)
|
||||
from one_memory.user_scope import lane_key, split_lane
|
||||
|
||||
|
||||
class UnknownSourceError(KeyError):
|
||||
@@ -82,36 +83,45 @@ class ImportSupervisor:
|
||||
self._tasks: dict[str, asyncio.Task[None]] = {}
|
||||
self._drains: dict[str, asyncio.Task[None]] = {}
|
||||
|
||||
def register_adapter(self, adapter: SourceAdapter) -> None:
|
||||
self.adapters[adapter.descriptor.source_id] = adapter
|
||||
def register_adapter(self, adapter: SourceAdapter, *, user_id: str = "local") -> None:
|
||||
"""Adapters are keyed by lane: the source id for `local`, `source@user`
|
||||
for another user's grant of the same source."""
|
||||
self.adapters[lane_key(user_id, adapter.descriptor.source_id)] = adapter
|
||||
|
||||
def unregister_adapter(self, source_id: str) -> None:
|
||||
self.adapters.pop(source_id, None)
|
||||
|
||||
@staticmethod
|
||||
def lane_of(job: ImportJob) -> str:
|
||||
return lane_key(job.user_id, job.source_id)
|
||||
|
||||
def active_job_ids(self) -> frozenset[str]:
|
||||
return frozenset(job_id for job_id, task in self._tasks.items() if not task.done())
|
||||
|
||||
def create_job(
|
||||
self, source_id: str, *, operation: Literal["history", "sync"] = "history"
|
||||
) -> ImportJob:
|
||||
"""`source_id` is a lane key: another user's lane creates that user's job."""
|
||||
if self.source_enabled is not None and not self.source_enabled(source_id):
|
||||
raise SourceAccessError(SourceAccessErrorCode.GRANT_REQUIRED)
|
||||
return self.job_store.create(user_id="local", source_id=source_id, operation=operation)
|
||||
user_id, bare_source_id = split_lane(source_id)
|
||||
return self.job_store.create(user_id=user_id, source_id=bare_source_id, operation=operation)
|
||||
|
||||
def enqueue(self, job_id: str) -> ImportJob:
|
||||
if job_id in self._drains:
|
||||
raise InvalidJobTransitionError("import_cleanup_in_progress")
|
||||
job = self.job_store.require(job_id)
|
||||
if job.source_id not in self.adapters:
|
||||
raise UnknownSourceError(job.source_id)
|
||||
if self.source_enabled is not None and not self.source_enabled(job.source_id):
|
||||
lane = self.lane_of(job)
|
||||
if lane not in self.adapters:
|
||||
raise UnknownSourceError(lane)
|
||||
if self.source_enabled is not None and not self.source_enabled(lane):
|
||||
return job
|
||||
existing = self._tasks.get(job_id)
|
||||
if existing is None or existing.done():
|
||||
self._tasks[job_id] = asyncio.create_task(
|
||||
self._execute(
|
||||
job_id,
|
||||
source_id=job.source_id,
|
||||
source_id=lane,
|
||||
operation=job.operation,
|
||||
)
|
||||
)
|
||||
@@ -130,8 +140,10 @@ class ImportSupervisor:
|
||||
# pending history. Each resumed runner will yield after one bounded slice.
|
||||
for status in (ImportStatus.INTERRUPTED, ImportStatus.PENDING):
|
||||
for job in self.job_store.list_jobs(status=status):
|
||||
if job.source_id in self.adapters:
|
||||
if self.source_enabled is not None and not self.source_enabled(job.source_id):
|
||||
if self.lane_of(job) in self.adapters:
|
||||
if self.source_enabled is not None and not self.source_enabled(
|
||||
self.lane_of(job)
|
||||
):
|
||||
continue
|
||||
if job.status is not ImportStatus.PENDING:
|
||||
self.job_store.queue_for_resume(job.job_id)
|
||||
@@ -227,7 +239,7 @@ class ImportSupervisor:
|
||||
}:
|
||||
return
|
||||
if self.source_enabled is not None and not await in_the_background(
|
||||
self.source_enabled, job.source_id
|
||||
self.source_enabled, self.lane_of(job)
|
||||
):
|
||||
return
|
||||
runner = ImportRunner(
|
||||
@@ -238,7 +250,7 @@ class ImportSupervisor:
|
||||
sdk_queue=self.sdk_queue,
|
||||
sdk_authority=self.sdk_authority,
|
||||
)
|
||||
adapter = self.adapters[job.source_id]
|
||||
adapter = self.adapters[self.lane_of(job)]
|
||||
if self.prepare_adapter is not None:
|
||||
adapter = await self.prepare_adapter(job, adapter)
|
||||
|
||||
@@ -273,7 +285,7 @@ class ImportSupervisor:
|
||||
and retry_attempt < self.max_history_retries
|
||||
and (
|
||||
self.source_enabled is None
|
||||
or await in_the_background(self.source_enabled, current.source_id)
|
||||
or await in_the_background(self.source_enabled, self.lane_of(current))
|
||||
)
|
||||
):
|
||||
delay = self.retry_base_seconds * (2**retry_attempt)
|
||||
|
||||
@@ -7,9 +7,9 @@ between pages. The matches are kept here under an opaque ``window_id`` that
|
||||
the client passes back with the next offset.
|
||||
|
||||
A window is bound to the filters it was searched with (query, source,
|
||||
statuses, scene). It expires after ``ttl_seconds`` without being read, and the
|
||||
store keeps at most ``max_windows`` windows, dropping the least recently read
|
||||
first.
|
||||
statuses, scene) and to the user whose library it searched. It expires after
|
||||
``ttl_seconds`` without being read, and the store keeps at most
|
||||
``max_windows`` windows, dropping the least recently read first.
|
||||
Only memory identities are kept, never content: each page is read fresh from
|
||||
the SDK, so an edit or a delete shows on the next page read.
|
||||
"""
|
||||
@@ -28,13 +28,24 @@ from one_memory.hy_sdk_types import LibraryHit
|
||||
DEFAULT_WINDOW_TTL_SECONDS = 300.0
|
||||
DEFAULT_MAX_WINDOWS = 32
|
||||
|
||||
WindowKey = tuple[str, str | None, tuple[str, ...], str]
|
||||
#: (query, source, statuses, scene, user, other filters as sorted name/values pairs)
|
||||
WindowKey = tuple[
|
||||
str, str | None, tuple[str, ...], str, str, tuple[tuple[str, tuple[str, ...]], ...]
|
||||
]
|
||||
|
||||
|
||||
def window_key(
|
||||
query: str, source: str | None, statuses: list[str], scene: str = "all"
|
||||
query: str,
|
||||
source: str | None,
|
||||
statuses: list[str],
|
||||
scene: str = "all",
|
||||
user_id: str = "local",
|
||||
filters: dict[str, list[str]] | None = None,
|
||||
) -> WindowKey:
|
||||
return (query, source, tuple(sorted(set(statuses))), scene)
|
||||
extra = tuple(
|
||||
sorted((name, tuple(sorted(set(values)))) for name, values in (filters or {}).items())
|
||||
)
|
||||
return (query, source, tuple(sorted(set(statuses))), scene, user_id, extra)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
|
||||
@@ -11,10 +11,11 @@ import asyncio
|
||||
import sqlite3
|
||||
from collections.abc import Sequence
|
||||
from datetime import UTC, datetime
|
||||
from functools import partial
|
||||
from typing import Any, Literal, cast
|
||||
from uuid import UUID
|
||||
|
||||
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request
|
||||
from fastapi import APIRouter, BackgroundTasks, HTTPException, Query, Request
|
||||
from fastapi.params import Depends
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||
|
||||
@@ -34,6 +35,7 @@ from one_memory.memory_server.durable_queue import DurableWriteQueue
|
||||
from one_memory.memory_server.unified_entry import UnifiedWriteIntake
|
||||
from one_memory.memory_tasks import MemoryTaskStatus, MemoryTaskStore
|
||||
from one_memory.secrets import SecretStoreUnavailableError
|
||||
from one_memory.user_scope import DEFAULT_USER_ID, USER_ID_PATTERN, UserId
|
||||
|
||||
SOURCE_ID = "one-memory-manual"
|
||||
MemoryAddState = Literal[
|
||||
@@ -53,6 +55,8 @@ class MemoryAddInput(BaseModel):
|
||||
request_id: UUID
|
||||
text: str = Field(min_length=1, max_length=12_000)
|
||||
scope: Literal["all-apps"]
|
||||
#: Whose memory this is. The App never sends it and writes `local`.
|
||||
user_id: UserId = DEFAULT_USER_ID
|
||||
|
||||
@field_validator("text")
|
||||
@classmethod
|
||||
@@ -108,7 +112,11 @@ class MemoryAdditionStore:
|
||||
|
||||
def save(self, addition: MemoryAddInput) -> EvidenceRecord:
|
||||
now = datetime.now(UTC)
|
||||
if addition.user_id != DEFAULT_USER_ID and not self.sdk_authority:
|
||||
# Legacy authority keeps one person's library only.
|
||||
raise ValueError("memory_add_user_requires_sdk_authority")
|
||||
value = EvidenceInput(
|
||||
user_id=addition.user_id,
|
||||
source_id=SOURCE_ID,
|
||||
conversation_id=f"note:{addition.request_id}",
|
||||
message_id=str(addition.request_id),
|
||||
@@ -200,7 +208,7 @@ class MemoryAdditionStore:
|
||||
)
|
||||
return self.require(value.evidence_id)
|
||||
|
||||
def require(self, evidence_id: str) -> EvidenceRecord:
|
||||
def require(self, evidence_id: str, *, user_id: str = DEFAULT_USER_ID) -> EvidenceRecord:
|
||||
if self.sdk_authority:
|
||||
if self.write_queue is None:
|
||||
raise KeyError(evidence_id)
|
||||
@@ -219,11 +227,11 @@ class MemoryAdditionStore:
|
||||
(item for item in payload.input_records if item.evidence_id == evidence_id),
|
||||
None,
|
||||
)
|
||||
if record is None:
|
||||
if record is None or record.user_id != user_id:
|
||||
raise KeyError(evidence_id)
|
||||
return record
|
||||
record = self.evidence.get(evidence_id)
|
||||
if record is None or record.user_id != "local" or record.source_id != SOURCE_ID:
|
||||
if record is None or record.user_id != user_id or record.source_id != SOURCE_ID:
|
||||
raise KeyError(evidence_id)
|
||||
metadata = record.origin_meta.get(EvidenceOrigin.MANUAL_IMPORT)
|
||||
if metadata is None or metadata.raw_meta.get("input_mode") != "direct_memory_note":
|
||||
@@ -237,8 +245,8 @@ class MemoryAdditionStore:
|
||||
raise ValueError("memory_add_source_revoked")
|
||||
return record
|
||||
|
||||
def receipt(self, evidence_id: str) -> MemoryAddReceipt:
|
||||
record = self.require(evidence_id)
|
||||
def receipt(self, evidence_id: str, *, user_id: str = DEFAULT_USER_ID) -> MemoryAddReceipt:
|
||||
record = self.require(evidence_id, user_id=user_id)
|
||||
if self.sdk_authority:
|
||||
assert self.write_queue is not None
|
||||
with main_database(self.write_queue.database_path).read() as database:
|
||||
@@ -300,10 +308,15 @@ class MemoryAdditionStore:
|
||||
error=error,
|
||||
)
|
||||
|
||||
def receipt_for_request(self, request_id: UUID) -> MemoryAddReceipt:
|
||||
def receipt_for_request(
|
||||
self, request_id: UUID, *, user_id: str = DEFAULT_USER_ID
|
||||
) -> MemoryAddReceipt:
|
||||
identity = str(request_id)
|
||||
if self.sdk_authority:
|
||||
# The Evidence id carries the user, so another user's request id
|
||||
# names a different note and reads as not found.
|
||||
probe = EvidenceInput(
|
||||
user_id=user_id,
|
||||
source_id=SOURCE_ID,
|
||||
conversation_id=f"note:{identity}",
|
||||
message_id=identity,
|
||||
@@ -316,16 +329,16 @@ class MemoryAdditionStore:
|
||||
)
|
||||
},
|
||||
)
|
||||
return self.receipt(probe.evidence_id)
|
||||
return self.receipt(probe.evidence_id, user_id=user_id)
|
||||
with main_database(self.evidence.database_path).read() as db:
|
||||
row = db.execute(
|
||||
"SELECT evidence_id FROM evidence WHERE user_id='local' AND source_id=? "
|
||||
"SELECT evidence_id FROM evidence WHERE user_id=? AND source_id=? "
|
||||
"AND conversation_id=? AND message_id=?",
|
||||
(SOURCE_ID, f"note:{identity}", identity),
|
||||
(user_id, SOURCE_ID, f"note:{identity}", identity),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
raise KeyError(identity)
|
||||
return self.receipt(str(row[0]))
|
||||
return self.receipt(str(row[0]), user_id=user_id)
|
||||
|
||||
|
||||
def memory_add_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
@@ -376,14 +389,20 @@ def memory_add_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
record = await asyncio.to_thread(additions.save, addition)
|
||||
except ValueError as error:
|
||||
raise HTTPException(409, str(error)) from error
|
||||
saved = await asyncio.to_thread(additions.receipt, record.evidence_id)
|
||||
saved = await asyncio.to_thread(
|
||||
partial(additions.receipt, record.evidence_id, user_id=addition.user_id)
|
||||
)
|
||||
background.add_task(schedule, additions, record)
|
||||
return saved
|
||||
|
||||
@router.get("/additions/{request_id}", response_model=MemoryAddReceipt)
|
||||
def memory_add_status(request_id: UUID, request: Request) -> MemoryAddReceipt:
|
||||
def memory_add_status(
|
||||
request_id: UUID,
|
||||
request: Request,
|
||||
user_id: str = Query(default=DEFAULT_USER_ID, pattern=USER_ID_PATTERN),
|
||||
) -> MemoryAddReceipt:
|
||||
try:
|
||||
return store(request).receipt_for_request(request_id)
|
||||
return store(request).receipt_for_request(request_id, user_id=user_id)
|
||||
except KeyError as error:
|
||||
raise HTTPException(404, "memory_add_not_found") from error
|
||||
except ValueError as error:
|
||||
|
||||
@@ -0,0 +1,618 @@
|
||||
"""System 2 for ultra: evolve runs, their receipts, the nightly schedule, graph backfill.
|
||||
|
||||
hy-memory 2.0.9's `client.evolve(user_id, agent_id, max_batches)` is ultra
|
||||
only, holds no lock, and has no scheduler: MemHub owns all three.
|
||||
|
||||
- A run is asynchronous: `start` answers a receipt at once and the run goes
|
||||
on in the background (evolve makes model calls across the user's whole
|
||||
library and can take minutes, longer than any HTTP timeout). One run per
|
||||
user at a time: asking again while one runs returns that run. A run holds
|
||||
the write coordinator's per-user exclusivity, so it never overlaps that
|
||||
user's write batch, and loops while the SDK answers `should_continue`, up
|
||||
to `EVOLVE_MAX_ROUNDS`.
|
||||
- The schedule runs every user that has memories once a day at the
|
||||
configured local time, and catches up after the machine slept through it.
|
||||
- Switching to ultra backfills each user's existing facts into the graph
|
||||
once (`build_memory_graph`).
|
||||
|
||||
What is kept on disk is counts and stable codes only, never memory text:
|
||||
`<data_dir>/evolve-state.json`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
from collections.abc import Awaitable, Callable, Iterable, Mapping
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from tempfile import NamedTemporaryFile
|
||||
from typing import Any, Literal, Protocol
|
||||
from uuid import uuid4
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from one_memory.diagnostic_log import diagnostic_event
|
||||
from one_memory.memory_server.config_state import EvolveSchedule
|
||||
from one_memory.memory_server.durable_queue import safe_error_code
|
||||
from one_memory.sdk_write_processor import PRODUCT_AGENT_ID
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: Rounds one run makes while the SDK says there is more to do.
|
||||
EVOLVE_MAX_ROUNDS = 20
|
||||
#: How often the schedule looks at the clock; also bounds catch-up after wake.
|
||||
SCHEDULE_CHECK_SECONDS = 300.0
|
||||
#: Receipts kept in memory for `GET /api/v2/evolve/{run_id}`.
|
||||
_KEPT_RUNS = 64
|
||||
_STABLE_CODE = re.compile(r"[a-z][a-z0-9_]{0,95}")
|
||||
_COUNTS = (
|
||||
"remaining_clusters",
|
||||
"remaining_actionable_clusters",
|
||||
"remaining_fresh_facts",
|
||||
)
|
||||
|
||||
RunState = Literal["running", "succeeded", "failed"]
|
||||
Trigger = Literal["manual", "scheduled"]
|
||||
|
||||
|
||||
class EvolveModeError(RuntimeError):
|
||||
"""Evolve asked for outside ultra."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__("memory_mode_not_ultra")
|
||||
|
||||
|
||||
class EvolveRunReceipt(BaseModel):
|
||||
"""One evolve run. Counts and codes; `results` only while in memory."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
run_id: str
|
||||
user_id: str
|
||||
agent_id: str
|
||||
max_batches: int | None = None
|
||||
trigger: Trigger = "manual"
|
||||
state: RunState = "running"
|
||||
started_at: datetime
|
||||
finished_at: datetime | None = None
|
||||
rounds: int = 0
|
||||
tasks_processed: int = 0
|
||||
batch_failures: int = 0
|
||||
cycle_status: str | None = None
|
||||
should_continue: bool | None = None
|
||||
continuation_reason: str | None = None
|
||||
remaining_clusters: int | None = None
|
||||
remaining_actionable_clusters: int | None = None
|
||||
remaining_fresh_facts: int | None = None
|
||||
#: True when the round cap stopped a run the SDK would have continued.
|
||||
capped: bool = False
|
||||
elapsed_ms: float = 0.0
|
||||
error_code: str | None = None
|
||||
failure_kind: str | None = None
|
||||
#: The last round's SDK `request_id`, for matching SDK logs.
|
||||
request_id: str | None = None
|
||||
#: The last round's SDK `results` and `cross_domain_sweeper`. Kept only
|
||||
#: in this receipt, never written to disk or diagnostics.
|
||||
results: Any = None
|
||||
cross_domain_sweeper: Any = None
|
||||
|
||||
def summary(self) -> dict[str, object]:
|
||||
"""What is persisted and shown in status: no SDK result bodies."""
|
||||
return self.model_dump(mode="json", exclude={"results", "cross_domain_sweeper"})
|
||||
|
||||
|
||||
class UserEvolveState(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
last_run_at: datetime | None = None
|
||||
last_result: dict[str, object] | None = None
|
||||
last_error: str | None = None
|
||||
|
||||
|
||||
class GraphBuildUser(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
state: Literal["pending", "running", "succeeded", "failed"] = "pending"
|
||||
memories_scanned: int | None = None
|
||||
nodes: int | None = None
|
||||
error_code: str | None = None
|
||||
|
||||
|
||||
class GraphBuildState(BaseModel):
|
||||
"""The one-time backfill of existing facts into the graph after → ultra."""
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
state: Literal["idle", "running", "succeeded", "failed"] = "idle"
|
||||
started_at: datetime | None = None
|
||||
finished_at: datetime | None = None
|
||||
users: dict[str, GraphBuildUser] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class EvolveStateDocument(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
schema_version: int = Field(default=1, ge=1, le=1)
|
||||
users: dict[str, UserEvolveState] = Field(default_factory=dict)
|
||||
#: When the schedule first ran in ultra: the "last run" of a user that
|
||||
#: never ran, so enabling ultra at noon does not evolve everyone at once.
|
||||
schedule_anchor: datetime | None = None
|
||||
graph_build: GraphBuildState = Field(default_factory=GraphBuildState)
|
||||
|
||||
|
||||
class EvolveStateStore:
|
||||
"""The only writer of `<data_dir>/evolve-state.json`."""
|
||||
|
||||
def __init__(self, data_dir: Path) -> None:
|
||||
self.path = data_dir / "evolve-state.json"
|
||||
self._lock = threading.RLock()
|
||||
self._document = self._read()
|
||||
|
||||
def document(self) -> EvolveStateDocument:
|
||||
with self._lock:
|
||||
return self._document.model_copy(deep=True)
|
||||
|
||||
def user(self, user_id: str) -> UserEvolveState:
|
||||
with self._lock:
|
||||
return self._document.users.get(user_id, UserEvolveState()).model_copy()
|
||||
|
||||
def record_run(self, receipt: EvolveRunReceipt) -> None:
|
||||
with self._lock:
|
||||
users = dict(self._document.users)
|
||||
users[receipt.user_id] = UserEvolveState(
|
||||
last_run_at=receipt.started_at,
|
||||
last_result=receipt.summary(),
|
||||
last_error=receipt.error_code,
|
||||
)
|
||||
self._save(self._document.model_copy(update={"users": users}))
|
||||
|
||||
def anchor(self, now: datetime) -> datetime:
|
||||
"""The schedule anchor, set to `now` the first time it is asked."""
|
||||
with self._lock:
|
||||
if self._document.schedule_anchor is None:
|
||||
self._save(self._document.model_copy(update={"schedule_anchor": now}))
|
||||
assert self._document.schedule_anchor is not None
|
||||
return self._document.schedule_anchor
|
||||
|
||||
def graph_build(self) -> GraphBuildState:
|
||||
with self._lock:
|
||||
return self._document.graph_build.model_copy(deep=True)
|
||||
|
||||
def set_graph_build(self, state: GraphBuildState) -> None:
|
||||
with self._lock:
|
||||
self._save(self._document.model_copy(update={"graph_build": state}))
|
||||
|
||||
def _save(self, document: EvolveStateDocument) -> None:
|
||||
payload = json.dumps(document.model_dump(mode="json"), indent=2, sort_keys=True) + "\n"
|
||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||
temporary: Path | None = None
|
||||
try:
|
||||
with NamedTemporaryFile(
|
||||
mode="w",
|
||||
encoding="utf-8",
|
||||
dir=self.path.parent,
|
||||
prefix=".evolve-state.",
|
||||
suffix=".tmp",
|
||||
delete=False,
|
||||
) as stream:
|
||||
temporary = Path(stream.name)
|
||||
stream.write(payload)
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
os.chmod(temporary, 0o600)
|
||||
os.replace(temporary, self.path)
|
||||
except OSError:
|
||||
if temporary is not None:
|
||||
temporary.unlink(missing_ok=True)
|
||||
# Status is best-effort: a run must not fail because its record did.
|
||||
logger.warning("evolve_state_write_failed")
|
||||
self._document = document
|
||||
|
||||
def _read(self) -> EvolveStateDocument:
|
||||
try:
|
||||
return EvolveStateDocument.model_validate_json(self.path.read_text(encoding="utf-8"))
|
||||
except FileNotFoundError:
|
||||
return EvolveStateDocument()
|
||||
except (OSError, ValueError):
|
||||
# A damaged status file only loses history; start a fresh one.
|
||||
logger.warning("evolve_state_unreadable")
|
||||
return EvolveStateDocument()
|
||||
|
||||
|
||||
class EvolveCoordinator(Protocol):
|
||||
"""The write coordinator surface evolve and the graph backfill use."""
|
||||
|
||||
def exclusive_for_evolve(self, user_id: str) -> Any: ...
|
||||
|
||||
def use_client(self) -> Any: ...
|
||||
|
||||
|
||||
class EvolveService:
|
||||
"""Start, serialize and record evolve runs."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
coordinator: EvolveCoordinator,
|
||||
state: EvolveStateStore,
|
||||
memory_mode: Callable[[], str],
|
||||
max_rounds: int = EVOLVE_MAX_ROUNDS,
|
||||
now: Callable[[], datetime] = lambda: datetime.now(UTC),
|
||||
) -> None:
|
||||
self.coordinator = coordinator
|
||||
self.state = state
|
||||
self.memory_mode = memory_mode
|
||||
self.max_rounds = max_rounds
|
||||
self._now = now
|
||||
self._runs: dict[str, EvolveRunReceipt] = {}
|
||||
self._tasks: dict[str, asyncio.Task[None]] = {}
|
||||
self._active: dict[str, str] = {}
|
||||
|
||||
def start(
|
||||
self,
|
||||
user_id: str,
|
||||
*,
|
||||
agent_id: str | None = None,
|
||||
max_batches: int | None = None,
|
||||
trigger: Trigger = "manual",
|
||||
) -> EvolveRunReceipt:
|
||||
"""Start a run, or return the one already running for this user."""
|
||||
if self.memory_mode() != "ultra":
|
||||
raise EvolveModeError()
|
||||
running = self._active.get(user_id)
|
||||
if running is not None:
|
||||
return self._runs[running]
|
||||
receipt = EvolveRunReceipt(
|
||||
run_id=str(uuid4()),
|
||||
user_id=user_id,
|
||||
agent_id=agent_id or PRODUCT_AGENT_ID,
|
||||
max_batches=max_batches,
|
||||
trigger=trigger,
|
||||
started_at=self._now(),
|
||||
)
|
||||
self._remember(receipt)
|
||||
self._active[user_id] = receipt.run_id
|
||||
self._tasks[receipt.run_id] = asyncio.create_task(
|
||||
self._run(receipt.run_id), name="one-memory-evolve"
|
||||
)
|
||||
return receipt
|
||||
|
||||
def get(self, run_id: str) -> EvolveRunReceipt | None:
|
||||
return self._runs.get(run_id)
|
||||
|
||||
def running(self) -> tuple[EvolveRunReceipt, ...]:
|
||||
return tuple(self._runs[run_id] for run_id in self._active.values())
|
||||
|
||||
async def wait(self, run_id: str) -> EvolveRunReceipt | None:
|
||||
task = self._tasks.get(run_id)
|
||||
if task is not None:
|
||||
await asyncio.shield(task)
|
||||
return self._runs.get(run_id)
|
||||
|
||||
async def shutdown(self) -> None:
|
||||
tasks = tuple(self._tasks.values())
|
||||
for task in tasks:
|
||||
task.cancel()
|
||||
await asyncio.gather(*tasks, return_exceptions=True)
|
||||
self._tasks.clear()
|
||||
|
||||
def _remember(self, receipt: EvolveRunReceipt) -> None:
|
||||
self._runs[receipt.run_id] = receipt
|
||||
while len(self._runs) > _KEPT_RUNS:
|
||||
oldest = next(iter(self._runs))
|
||||
if oldest in self._active.values():
|
||||
break
|
||||
del self._runs[oldest]
|
||||
|
||||
async def _run(self, run_id: str) -> None:
|
||||
receipt = self._runs[run_id]
|
||||
diagnostic_event(
|
||||
level="info",
|
||||
component="memory.evolve",
|
||||
event="memory_evolve.started",
|
||||
outcome="started",
|
||||
details={"trigger": receipt.trigger, "max_batches": receipt.max_batches},
|
||||
)
|
||||
try:
|
||||
async with (
|
||||
self.coordinator.exclusive_for_evolve(receipt.user_id),
|
||||
self.coordinator.use_client() as client,
|
||||
):
|
||||
while receipt.rounds < self.max_rounds:
|
||||
result = await asyncio.to_thread(
|
||||
client.evolve,
|
||||
receipt.user_id,
|
||||
agent_id=receipt.agent_id,
|
||||
max_batches=receipt.max_batches,
|
||||
)
|
||||
receipt = _folded(receipt, result)
|
||||
self._runs[run_id] = receipt
|
||||
if receipt.state == "failed" or not receipt.should_continue:
|
||||
break
|
||||
else:
|
||||
receipt = receipt.model_copy(update={"capped": True})
|
||||
if receipt.state == "running":
|
||||
receipt = receipt.model_copy(update={"state": "succeeded"})
|
||||
except asyncio.CancelledError:
|
||||
receipt = receipt.model_copy(
|
||||
update={"state": "failed", "error_code": "memory_evolve_cancelled"}
|
||||
)
|
||||
raise
|
||||
except Exception as error: # noqa: BLE001 - recorded as a stable code
|
||||
code = safe_error_code(error)
|
||||
receipt = receipt.model_copy(
|
||||
update={
|
||||
"state": "failed",
|
||||
"error_code": _stable(code, "hy_memory_evolve_failed"),
|
||||
"failure_kind": type(error).__name__,
|
||||
}
|
||||
)
|
||||
finally:
|
||||
receipt = receipt.model_copy(update={"finished_at": self._now()})
|
||||
self._runs[run_id] = receipt
|
||||
self._active.pop(receipt.user_id, None)
|
||||
self._tasks.pop(run_id, None)
|
||||
await asyncio.to_thread(self.state.record_run, receipt)
|
||||
diagnostic_event(
|
||||
level="info" if receipt.state == "succeeded" else "warning",
|
||||
component="memory.evolve",
|
||||
event="memory_evolve.finished",
|
||||
outcome="succeeded" if receipt.state == "succeeded" else "failed",
|
||||
details={
|
||||
"trigger": receipt.trigger,
|
||||
"rounds": receipt.rounds,
|
||||
"tasks_processed": receipt.tasks_processed,
|
||||
"batch_failures": receipt.batch_failures,
|
||||
"capped": receipt.capped,
|
||||
"elapsed_ms": round(receipt.elapsed_ms, 1),
|
||||
"error_code": receipt.error_code,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _folded(receipt: EvolveRunReceipt, result: object) -> EvolveRunReceipt:
|
||||
"""One more evolve round's result added to the run."""
|
||||
if not isinstance(result, Mapping):
|
||||
return receipt.model_copy(
|
||||
update={"state": "failed", "error_code": "hy_memory_evolve_shape_unknown"}
|
||||
)
|
||||
failures = result.get("batch_failures")
|
||||
update: dict[str, object] = {
|
||||
"rounds": receipt.rounds + 1,
|
||||
"tasks_processed": receipt.tasks_processed + _int(result.get("tasks_processed")),
|
||||
"batch_failures": receipt.batch_failures
|
||||
+ (len(failures) if isinstance(failures, list | tuple) else _int(failures)),
|
||||
"elapsed_ms": receipt.elapsed_ms + _float(result.get("elapsed_ms")),
|
||||
"should_continue": result.get("should_continue") is True,
|
||||
"cycle_status": _text(result.get("cycle_status")),
|
||||
"continuation_reason": _text(result.get("continuation_reason")),
|
||||
"request_id": _text(result.get("request_id")),
|
||||
"results": result.get("results"),
|
||||
"cross_domain_sweeper": result.get("cross_domain_sweeper"),
|
||||
}
|
||||
for name in _COUNTS:
|
||||
value = result.get(name)
|
||||
update[name] = value if isinstance(value, int) and not isinstance(value, bool) else None
|
||||
if result.get("success") is not True:
|
||||
code = result.get("error_code")
|
||||
kind = result.get("failure_kind")
|
||||
update.update(
|
||||
state="failed",
|
||||
should_continue=False,
|
||||
error_code=(
|
||||
code
|
||||
if isinstance(code, str) and _STABLE_CODE.fullmatch(code)
|
||||
else "hy_memory_evolve_rejected"
|
||||
),
|
||||
failure_kind=kind if isinstance(kind, str) and _STABLE_CODE.fullmatch(kind) else None,
|
||||
)
|
||||
return receipt.model_copy(update=update)
|
||||
|
||||
|
||||
def _stable(code: str, fallback: str) -> str:
|
||||
return code if _STABLE_CODE.fullmatch(code) else fallback
|
||||
|
||||
|
||||
def _int(value: object) -> int:
|
||||
return value if isinstance(value, int) and not isinstance(value, bool) else 0
|
||||
|
||||
|
||||
def _float(value: object) -> float:
|
||||
if isinstance(value, int | float) and not isinstance(value, bool):
|
||||
return float(value)
|
||||
return 0.0
|
||||
|
||||
|
||||
def _text(value: object) -> str | None:
|
||||
return value[:128] if isinstance(value, str) else None
|
||||
|
||||
|
||||
def evolve_due(
|
||||
now: datetime,
|
||||
schedule: EvolveSchedule,
|
||||
last_run_at: datetime | None,
|
||||
anchor: datetime,
|
||||
) -> bool:
|
||||
"""Whether a user's nightly evolve is due at `now` (aware, local time).
|
||||
|
||||
Due once the day's slot (`schedule.time`, local) has passed and the last
|
||||
run (or, for a user who never ran, the anchor) is older than that slot.
|
||||
A last run more than 24 hours old is due at once as well: a machine asleep
|
||||
at the slot catches up when it wakes, even before the next day's slot.
|
||||
"""
|
||||
if not schedule.enabled:
|
||||
return False
|
||||
local = now.astimezone()
|
||||
hour, minute = (int(part) for part in schedule.time.split(":"))
|
||||
slot = local.replace(hour=hour, minute=minute, second=0, microsecond=0)
|
||||
reference = last_run_at or anchor
|
||||
if local >= slot and reference < slot:
|
||||
return True
|
||||
return last_run_at is not None and local - last_run_at > timedelta(hours=24)
|
||||
|
||||
|
||||
class EvolveScheduler:
|
||||
"""Run evolve nightly for every user that has memories (ultra only)."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
service: EvolveService,
|
||||
schedule: Callable[[], EvolveSchedule],
|
||||
users_with_memories: Callable[[], Awaitable[Iterable[str]]],
|
||||
now: Callable[[], datetime] = lambda: datetime.now().astimezone(),
|
||||
check_seconds: float = SCHEDULE_CHECK_SECONDS,
|
||||
) -> None:
|
||||
self.service = service
|
||||
self.schedule = schedule
|
||||
self.users_with_memories = users_with_memories
|
||||
self._now = now
|
||||
self.check_seconds = check_seconds
|
||||
self._task: asyncio.Task[None] | None = None
|
||||
|
||||
def start(self) -> None:
|
||||
if self._task is None or self._task.done():
|
||||
self._task = asyncio.create_task(self._loop(), name="one-memory-evolve-schedule")
|
||||
|
||||
async def tick(self) -> tuple[str, ...]:
|
||||
"""Run every due user once, one after another; the users run."""
|
||||
schedule = self.schedule()
|
||||
if not schedule.enabled or self.service.memory_mode() != "ultra":
|
||||
return ()
|
||||
now = self._now()
|
||||
anchor = await asyncio.to_thread(self.service.state.anchor, now)
|
||||
ran: list[str] = []
|
||||
for user_id in await self.users_with_memories():
|
||||
last = self.service.state.user(user_id).last_run_at
|
||||
if not evolve_due(now, schedule, last, anchor):
|
||||
continue
|
||||
try:
|
||||
receipt = self.service.start(user_id, trigger="scheduled")
|
||||
except EvolveModeError:
|
||||
break
|
||||
await self.service.wait(receipt.run_id)
|
||||
ran.append(user_id)
|
||||
if ran:
|
||||
diagnostic_event(
|
||||
level="info",
|
||||
component="memory.evolve",
|
||||
event="memory_evolve.schedule_ran",
|
||||
outcome="succeeded",
|
||||
details={"users": len(ran)},
|
||||
)
|
||||
return tuple(ran)
|
||||
|
||||
async def _loop(self) -> None:
|
||||
while True:
|
||||
try:
|
||||
await self.tick()
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception as error: # noqa: BLE001 - the loop must survive
|
||||
logger.warning("evolve_schedule_tick_failed error_type=%s", type(error).__name__)
|
||||
await asyncio.sleep(self.check_seconds)
|
||||
|
||||
async def shutdown(self) -> None:
|
||||
if self._task is None:
|
||||
return
|
||||
self._task.cancel()
|
||||
await asyncio.gather(self._task, return_exceptions=True)
|
||||
self._task = None
|
||||
|
||||
|
||||
class MemoryGraphBackfill:
|
||||
"""After a switch to ultra, project each user's existing facts into the graph once."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
coordinator: EvolveCoordinator,
|
||||
state: EvolveStateStore,
|
||||
now: Callable[[], datetime] = lambda: datetime.now(UTC),
|
||||
) -> None:
|
||||
self.coordinator = coordinator
|
||||
self.state = state
|
||||
self._now = now
|
||||
self._task: asyncio.Task[None] | None = None
|
||||
|
||||
@property
|
||||
def running(self) -> bool:
|
||||
return self._task is not None and not self._task.done()
|
||||
|
||||
def start(self, user_ids: Iterable[str]) -> GraphBuildState:
|
||||
if self.running:
|
||||
return self.state.graph_build()
|
||||
users = tuple(dict.fromkeys(user_ids))
|
||||
state = GraphBuildState(
|
||||
state="running",
|
||||
started_at=self._now(),
|
||||
users={user_id: GraphBuildUser() for user_id in users},
|
||||
)
|
||||
self.state.set_graph_build(state)
|
||||
self._task = asyncio.create_task(self._run(users), name="one-memory-graph-backfill")
|
||||
return state
|
||||
|
||||
async def wait(self) -> None:
|
||||
if self._task is not None:
|
||||
await asyncio.shield(self._task)
|
||||
|
||||
async def _run(self, users: tuple[str, ...]) -> None:
|
||||
state = self.state.graph_build()
|
||||
for user_id in users:
|
||||
state.users[user_id] = GraphBuildUser(state="running")
|
||||
await asyncio.to_thread(self.state.set_graph_build, state.model_copy(deep=True))
|
||||
try:
|
||||
async with (
|
||||
self.coordinator.exclusive_for_evolve(user_id),
|
||||
self.coordinator.use_client() as client,
|
||||
):
|
||||
result = await asyncio.to_thread(
|
||||
client.build_memory_graph,
|
||||
user_id,
|
||||
agent_id=None,
|
||||
limit=10000,
|
||||
clear_first=False,
|
||||
)
|
||||
ok = isinstance(result, Mapping) and result.get("success") is True
|
||||
state.users[user_id] = GraphBuildUser(
|
||||
state="succeeded" if ok else "failed",
|
||||
memories_scanned=_int(result.get("memories_scanned"))
|
||||
if isinstance(result, Mapping)
|
||||
else None,
|
||||
nodes=_int(result.get("nodes")) if isinstance(result, Mapping) else None,
|
||||
error_code=None if ok else "hy_memory_graph_build_rejected",
|
||||
)
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception as error: # noqa: BLE001 - recorded as a stable code
|
||||
code = safe_error_code(error)
|
||||
state.users[user_id] = GraphBuildUser(
|
||||
state="failed",
|
||||
error_code=_stable(code, "hy_memory_graph_build_failed"),
|
||||
)
|
||||
failed = any(user.state == "failed" for user in state.users.values())
|
||||
state.state = "failed" if failed else "succeeded"
|
||||
state.finished_at = self._now()
|
||||
await asyncio.to_thread(self.state.set_graph_build, state)
|
||||
diagnostic_event(
|
||||
level="warning" if failed else "info",
|
||||
component="memory.evolve",
|
||||
event="memory_graph_backfill.finished",
|
||||
outcome="failed" if failed else "succeeded",
|
||||
details={
|
||||
"users": len(state.users),
|
||||
"failed_users": sum(user.state == "failed" for user in state.users.values()),
|
||||
},
|
||||
)
|
||||
|
||||
async def shutdown(self) -> None:
|
||||
if self._task is None:
|
||||
return
|
||||
self._task.cancel()
|
||||
await asyncio.gather(self._task, return_exceptions=True)
|
||||
self._task = None
|
||||
@@ -13,6 +13,7 @@ import threading
|
||||
from collections.abc import Mapping
|
||||
from pathlib import Path
|
||||
from tempfile import NamedTemporaryFile
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, StrictBool, field_validator
|
||||
|
||||
@@ -27,6 +28,31 @@ class SourceConfig(BaseModel):
|
||||
sync_enabled: StrictBool = False
|
||||
|
||||
|
||||
#: The SDK pipeline mode (hy-memory `HyMemoryClient(mode=...)`):
|
||||
#: lite keeps raw conversations only, with no model call; pro extracts
|
||||
#: memories (System 1); ultra adds the knowledge graph and evolve (System 2).
|
||||
MemoryMode = Literal["lite", "pro", "ultra"]
|
||||
MEMORY_MODES: tuple[MemoryMode, ...] = ("lite", "pro", "ultra")
|
||||
_SCHEDULE_TIME = re.compile(r"^(?:[01][0-9]|2[0-3]):[0-5][0-9]$")
|
||||
|
||||
|
||||
class EvolveSchedule(BaseModel):
|
||||
"""When ultra runs evolve for every user: once a day, local time."""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid", strict=True)
|
||||
|
||||
enabled: StrictBool = True
|
||||
#: `HH:MM`, 24-hour, the machine's local time.
|
||||
time: str = "03:00"
|
||||
|
||||
@field_validator("time")
|
||||
@classmethod
|
||||
def valid_time(cls, value: str) -> str:
|
||||
if _SCHEDULE_TIME.fullmatch(value) is None:
|
||||
raise ValueError("invalid_evolve_schedule_time")
|
||||
return value
|
||||
|
||||
|
||||
class ConfigDocument(BaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid", strict=True)
|
||||
|
||||
@@ -39,6 +65,10 @@ class ConfigDocument(BaseModel):
|
||||
#: needs no re-entry of a key.
|
||||
sedimentation_enabled: StrictBool = True
|
||||
extraction_window_preset: ExtractionWindowPreset = ExtractionWindowPreset.THREE_MONTHS
|
||||
#: The SDK mode every client is built with. `ONE_MEMORY_GRAPH_STORE` (dev)
|
||||
#: still forces ultra over it; see `hy_graph_admin.effective_memory_mode`.
|
||||
memory_mode: MemoryMode = "pro"
|
||||
evolve_schedule: EvolveSchedule = Field(default_factory=EvolveSchedule)
|
||||
|
||||
@field_validator("sources")
|
||||
@classmethod
|
||||
@@ -61,6 +91,9 @@ class ConfigFileError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
_WRITTEN_WHEN_SET = frozenset({"memory_mode", "evolve_schedule"})
|
||||
|
||||
|
||||
class ConfigStore:
|
||||
"""The only writer of ``<data_dir>/config.json`` inside Core."""
|
||||
|
||||
@@ -107,6 +140,32 @@ class ConfigStore:
|
||||
self._document = updated
|
||||
return updated
|
||||
|
||||
def memory_mode(self) -> MemoryMode:
|
||||
with self._lock:
|
||||
return self._document.memory_mode
|
||||
|
||||
def set_memory_mode(self, mode: MemoryMode) -> ConfigDocument:
|
||||
if mode not in MEMORY_MODES:
|
||||
raise ValueError("invalid_memory_mode")
|
||||
with self._lock:
|
||||
updated = self._document.model_copy(update={"memory_mode": mode})
|
||||
self._write(updated)
|
||||
self._document = updated
|
||||
return updated
|
||||
|
||||
def evolve_schedule(self) -> EvolveSchedule:
|
||||
with self._lock:
|
||||
return self._document.evolve_schedule
|
||||
|
||||
def set_evolve_schedule(self, schedule: EvolveSchedule) -> ConfigDocument:
|
||||
if not isinstance(schedule, EvolveSchedule):
|
||||
raise ValueError("invalid_evolve_schedule")
|
||||
with self._lock:
|
||||
updated = self._document.model_copy(update={"evolve_schedule": schedule})
|
||||
self._write(updated)
|
||||
self._document = updated
|
||||
return updated
|
||||
|
||||
def migrate_extraction_window_preset(self, preset: ExtractionWindowPreset) -> ConfigDocument:
|
||||
"""Adopt the legacy SQL value only when config.json has no value."""
|
||||
if not isinstance(preset, ExtractionWindowPreset):
|
||||
@@ -152,9 +211,12 @@ class ConfigStore:
|
||||
raise ConfigFileError("config_invalid") from error
|
||||
|
||||
def _write(self, document: ConfigDocument) -> None:
|
||||
# Settings added after 1.0.5 are written only once a person chose
|
||||
# them, so an untouched install's file stays what older builds read.
|
||||
unset = {name for name in _WRITTEN_WHEN_SET if name not in document.model_fields_set}
|
||||
payload = (
|
||||
json.dumps(
|
||||
document.model_dump(mode="json"),
|
||||
document.model_dump(mode="json", exclude=unset),
|
||||
ensure_ascii=False,
|
||||
indent=2,
|
||||
sort_keys=True,
|
||||
|
||||
@@ -75,7 +75,11 @@ def batch_id(*, internal_session_id: str, first_round: int, last_round: int) ->
|
||||
INTERNAL_SESSION_ID_MAX = 128
|
||||
|
||||
|
||||
def internal_session_id_for(conversation_id: str, source_id: str) -> str:
|
||||
def internal_session_id_for(
|
||||
conversation_id: str,
|
||||
source_id: str,
|
||||
user_id: str = "local",
|
||||
) -> str:
|
||||
"""Normalise a conversation identity to the durable ``internal_session_id``.
|
||||
|
||||
This is the single derivation shared by every Evidence-first intake path.
|
||||
@@ -86,7 +90,17 @@ def internal_session_id_for(conversation_id: str, source_id: str) -> str:
|
||||
per-session serialisation split). Only a value that the contract would have
|
||||
rejected before it ever reached the queue is replaced by the versioned
|
||||
session id, which is deterministic for a given (source, conversation).
|
||||
|
||||
Another user's conversation is always derived with the user in it: two
|
||||
Hermes profiles can hold the same session id, and sharing a session key
|
||||
would serialise and batch one person's turns behind the other's. `local`
|
||||
keeps the identity it always had.
|
||||
"""
|
||||
if user_id != "local":
|
||||
return session_id(
|
||||
harness_id=source_id,
|
||||
external_session_id=f"user:{user_id}|{conversation_id}",
|
||||
)
|
||||
if len(conversation_id) <= INTERNAL_SESSION_ID_MAX:
|
||||
return conversation_id
|
||||
derived = session_id(harness_id=source_id, external_session_id=conversation_id)
|
||||
|
||||
@@ -54,6 +54,7 @@ from one_memory.memory_tasks import (
|
||||
)
|
||||
from one_memory.sdk_migration_log import error_chain_text
|
||||
from one_memory.service_levels import ServiceLevel, service_level_context
|
||||
from one_memory.user_scope import DEFAULT_USER_ID, lane_key, other_user_lane_prefix
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -345,6 +346,27 @@ EVENT_TIME_WINDOW = timedelta(minutes=10)
|
||||
DIRECT_WRITE_SOURCES = frozenset({"one-memory-manual"})
|
||||
|
||||
|
||||
def request_user_id(request: WriteRequest) -> str:
|
||||
"""Whose write this is: its frozen input's user, `local` for id-only rows."""
|
||||
users = {record.user_id for record in request.input_records}
|
||||
if len(users) > 1:
|
||||
raise ValueError("durable_write_multiple_users")
|
||||
return next(iter(users), DEFAULT_USER_ID)
|
||||
|
||||
|
||||
def request_source_key(request: WriteRequest) -> str:
|
||||
"""The FIFO/claim key of a write: its source, per user.
|
||||
|
||||
`local` keeps the bare source id it always had. Another user's Hermes
|
||||
lane is `hermes@user`, so one person's running job never holds the other
|
||||
person's queue and the claim gate answers for that user's grant.
|
||||
"""
|
||||
return lane_key(
|
||||
request_user_id(request),
|
||||
request.source_id or request.internal_session_id,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class DurableWriteRecord:
|
||||
job_id: str
|
||||
@@ -862,7 +884,7 @@ class DurableWriteQueue:
|
||||
job_id,
|
||||
queue_seq,
|
||||
request.internal_session_id,
|
||||
request.source_id or request.internal_session_id,
|
||||
request_source_key(request),
|
||||
request.batch_identity.batch_id,
|
||||
key,
|
||||
payload,
|
||||
@@ -929,7 +951,9 @@ class DurableWriteQueue:
|
||||
f"{hashlib.sha256(file_key.encode()).hexdigest()}:"
|
||||
f"{hashlib.sha256(record.conversation_id.encode()).hexdigest()}"
|
||||
)
|
||||
session_key = internal_session_id_for(record.conversation_id, record.source_id)
|
||||
session_key = internal_session_id_for(
|
||||
record.conversation_id, record.source_id, record.user_id
|
||||
)
|
||||
written = connection.execute(
|
||||
"SELECT 1 FROM memory_write_jobs AS job, json_each(job.evidence_ids_json) AS ref "
|
||||
"WHERE job.session_key=? AND job.state!='assembling' AND ref.value=? LIMIT 1",
|
||||
@@ -998,7 +1022,7 @@ class DurableWriteQueue:
|
||||
str(uuid4()),
|
||||
queue_seq,
|
||||
session_key,
|
||||
record.source_id,
|
||||
lane_key(record.user_id, record.source_id),
|
||||
assembly_key,
|
||||
assembly_key,
|
||||
payload_json,
|
||||
@@ -1034,10 +1058,13 @@ class DurableWriteQueue:
|
||||
`flush` (history at end of file) completes the last turn and writes a
|
||||
short batch; without it (live polling) those stay in the row.
|
||||
"""
|
||||
# Two users' roots can hold a file with the same relative key (each
|
||||
# Hermes profile has its own `state.db`): only this user's rows.
|
||||
assembly_prefix = f"assembly:{user_id}:{source_id}:"
|
||||
rows = connection.execute(
|
||||
"SELECT * FROM memory_write_jobs WHERE state='assembling' "
|
||||
"AND source_key=? ORDER BY queue_seq",
|
||||
(source_id,),
|
||||
"AND source_key=? AND substr(idempotency_key,1,?)=? ORDER BY queue_seq",
|
||||
(lane_key(user_id, source_id), len(assembly_prefix), assembly_prefix),
|
||||
).fetchall()
|
||||
finalized: list[str] = []
|
||||
for row in rows:
|
||||
@@ -1160,7 +1187,7 @@ class DurableWriteQueue:
|
||||
Same records, same key: a re-read returns the job already written.
|
||||
"""
|
||||
first = records[0]
|
||||
session_id = internal_session_id_for(first.conversation_id, first.source_id)
|
||||
session_id = internal_session_id_for(first.conversation_id, first.source_id, first.user_id)
|
||||
material = ":".join(f"{item.evidence_id}:{item.content_hash}" for item in records)
|
||||
request_id = f"turn:v2:{hashlib.sha256(material.encode()).hexdigest()}"
|
||||
existing = connection.execute(
|
||||
@@ -1205,7 +1232,7 @@ class DurableWriteQueue:
|
||||
job_id,
|
||||
queue_seq,
|
||||
session_id,
|
||||
first.source_id,
|
||||
lane_key(first.user_id, first.source_id),
|
||||
request.batch_identity.batch_id,
|
||||
request_id,
|
||||
json.dumps(
|
||||
@@ -2494,6 +2521,12 @@ async def execute_job(
|
||||
source_id = request.internal_session_id
|
||||
if source_ids and source_ids != {source_id}:
|
||||
raise RuntimeError("durable_write_multiple_sources")
|
||||
users = {getattr(record, "user_id", DEFAULT_USER_ID) for record in records}
|
||||
if len(users) > 1:
|
||||
raise RuntimeError("durable_write_multiple_users")
|
||||
# The gates answer per user: another user's Hermes job asks about
|
||||
# that user's grant, not `local`'s.
|
||||
source_id = lane_key(next(iter(users), DEFAULT_USER_ID), source_id)
|
||||
|
||||
async def mark() -> bool:
|
||||
return await in_the_background(queue.mark_invoked, job_id, owner=owner)
|
||||
@@ -2530,6 +2563,8 @@ async def execute_job(
|
||||
in {
|
||||
"model_content_source_disabled",
|
||||
"model_sedimentation_disabled",
|
||||
# Evolve holds this user; the job runs once it is done.
|
||||
"memory_evolve_in_progress",
|
||||
}
|
||||
and not hy_invoked
|
||||
):
|
||||
@@ -2799,6 +2834,19 @@ class DurableWriteDrain:
|
||||
source_clause = f"AND candidate.source_key IN ({placeholders}) "
|
||||
source_params = tuple(sorted(enabled_sources))
|
||||
direct_sources = ",".join("?" for _ in DIRECT_WRITE_SOURCES)
|
||||
# Another user's hand-added memories are `one-memory-manual@user`:
|
||||
# the same lane ahead of history, still serial per user.
|
||||
direct_lanes = " OR ".join(
|
||||
"substr(candidate.source_key,1,?)=?" for _ in DIRECT_WRITE_SOURCES
|
||||
)
|
||||
direct_lane_params = tuple(
|
||||
value
|
||||
for source in sorted(DIRECT_WRITE_SOURCES)
|
||||
for value in (
|
||||
len(other_user_lane_prefix(source)),
|
||||
other_user_lane_prefix(source),
|
||||
)
|
||||
)
|
||||
# Same eligibility as the window claim below (free lease, no
|
||||
# running job of its source, nothing earlier in its session), but
|
||||
# not bound to the window: a note must not wait for history.
|
||||
@@ -2808,7 +2856,7 @@ class DurableWriteDrain:
|
||||
"WHERE candidate.state=? "
|
||||
"AND (candidate.lease_expires_at IS NULL OR candidate.lease_expires_at < ?) "
|
||||
+ source_clause
|
||||
+ f"AND candidate.source_key IN ({direct_sources}) "
|
||||
+ f"AND (candidate.source_key IN ({direct_sources}) OR {direct_lanes}) "
|
||||
"AND NOT EXISTS ("
|
||||
"SELECT 1 FROM memory_write_jobs AS held "
|
||||
"WHERE held.source_key=candidate.source_key AND held.state=? "
|
||||
@@ -2826,6 +2874,7 @@ class DurableWriteDrain:
|
||||
now.isoformat(),
|
||||
*source_params,
|
||||
*sorted(DIRECT_WRITE_SOURCES),
|
||||
*direct_lane_params,
|
||||
RUNNING,
|
||||
now.isoformat(),
|
||||
*_FIFO_BLOCKING_STATES,
|
||||
|
||||
@@ -19,10 +19,11 @@ does the writing. That separation is D52.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any, Literal, cast
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from fastapi import APIRouter, HTTPException, Query, Request
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from one_memory.imports.models import (
|
||||
@@ -43,6 +44,24 @@ from one_memory.memory_server.durable_queue import (
|
||||
)
|
||||
from one_memory.memory_server.registry import WriteRegistry, WriteState
|
||||
from one_memory.memory_server.submit import WriteConflictError
|
||||
from one_memory.user_scope import DEFAULT_USER_ID, USER_ID_PATTERN, UserId
|
||||
|
||||
#: Durable keys of another user's writes start with this. A `local` request
|
||||
#: id never may, so `local` cannot name another user's receipt.
|
||||
_USER_KEY_PREFIX = "user:"
|
||||
|
||||
|
||||
def durable_request_key(request_id: str, user_id: str) -> str:
|
||||
"""The queue key of a caller's request id, namespaced per user.
|
||||
|
||||
Two users choosing the same request id must not collide: the key is the
|
||||
queue's idempotency key and the SDK's `request_id`. `local` keeps its own
|
||||
request id unchanged.
|
||||
"""
|
||||
if user_id == DEFAULT_USER_ID:
|
||||
return request_id
|
||||
digest = hashlib.sha256(request_id.encode()).hexdigest()
|
||||
return f"{_USER_KEY_PREFIX}{user_id}:{digest}"
|
||||
|
||||
|
||||
class MemoryWriteReceipt(BaseModel):
|
||||
@@ -71,6 +90,7 @@ class SdkMemoryWriteInput(BaseModel):
|
||||
messages: list[OpenAIMessage] = Field(min_length=1, max_length=200)
|
||||
memory_at: datetime | None = None
|
||||
priority: Literal["live", "normal"] = "normal"
|
||||
user_id: UserId = DEFAULT_USER_ID
|
||||
|
||||
|
||||
def _registry(
|
||||
@@ -124,6 +144,9 @@ def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
|
||||
) -> MemoryWriteReceipt:
|
||||
if not any(message.role == "user" for message in payload.messages):
|
||||
raise HTTPException(422, "messages_require_user_turn")
|
||||
if payload.user_id == DEFAULT_USER_ID and payload.request_id.startswith(_USER_KEY_PREFIX):
|
||||
raise HTTPException(422, "request_id_reserved_prefix")
|
||||
request_key = durable_request_key(payload.request_id, payload.user_id)
|
||||
observed_at = payload.memory_at or datetime(1970, 1, 1, tzinfo=UTC)
|
||||
written_by = {
|
||||
"user": WrittenBy.PERSON,
|
||||
@@ -133,6 +156,7 @@ def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
|
||||
records = tuple(
|
||||
materialize_write_input(
|
||||
EvidenceInput(
|
||||
user_id=payload.user_id,
|
||||
source_id=payload.source_id,
|
||||
conversation_id=payload.session_id,
|
||||
message_id=f"{payload.request_id}:{index}",
|
||||
@@ -157,6 +181,7 @@ def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
|
||||
internal_session_id = internal_session_id_for(
|
||||
payload.session_id,
|
||||
payload.source_id,
|
||||
payload.user_id,
|
||||
)
|
||||
contract = WriteRequest(
|
||||
internal_session_id=internal_session_id,
|
||||
@@ -168,7 +193,7 @@ def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
|
||||
),
|
||||
evidence_ids=[record.evidence_id for record in records],
|
||||
input_records=records,
|
||||
idempotency_key=payload.request_id,
|
||||
idempotency_key=request_key,
|
||||
priority=payload.priority,
|
||||
event_at=payload.memory_at,
|
||||
time_quality="exact" if payload.memory_at is not None else "approximate",
|
||||
@@ -178,7 +203,7 @@ def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
|
||||
except (WriteConflictError, ValueError) as error:
|
||||
raise HTTPException(409, str(error)) from error
|
||||
return MemoryWriteReceipt(
|
||||
request_id=record.request_id,
|
||||
request_id=payload.request_id,
|
||||
state=record.state,
|
||||
created_at=record.created_at,
|
||||
updated_at=record.updated_at,
|
||||
@@ -186,16 +211,24 @@ def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
|
||||
)
|
||||
|
||||
@instance.get("/{request_id}", response_model=MemoryWriteReceipt)
|
||||
def status(request_id: str, request: Request) -> MemoryWriteReceipt:
|
||||
def status(
|
||||
request_id: str,
|
||||
request: Request,
|
||||
user_id: str = Query(default=DEFAULT_USER_ID, pattern=USER_ID_PATTERN),
|
||||
) -> MemoryWriteReceipt:
|
||||
# A receipt belongs to the user that submitted it. Another user's key
|
||||
# is namespaced, and `local` may not name a namespaced key directly.
|
||||
if user_id == DEFAULT_USER_ID and request_id.startswith(_USER_KEY_PREFIX):
|
||||
raise HTTPException(404, "unknown request id")
|
||||
registry = _registry(request, write=False)
|
||||
try:
|
||||
record = registry.get(request_id)
|
||||
record = registry.get(durable_request_key(request_id, user_id))
|
||||
except DurableQueueSchemaUnavailableError as error:
|
||||
raise HTTPException(503, str(error)) from error
|
||||
if record is None:
|
||||
raise HTTPException(404, "unknown request id")
|
||||
return MemoryWriteReceipt(
|
||||
request_id=record.request_id,
|
||||
request_id=request_id,
|
||||
state=record.state,
|
||||
created_at=record.created_at,
|
||||
updated_at=record.updated_at,
|
||||
|
||||
@@ -271,7 +271,9 @@ def build_request_from_evidence(
|
||||
# Keep the exact conversation id when it fits the 128-char contract;
|
||||
# otherwise derive the versioned session id, so a long branch id does
|
||||
# not reject the whole history import.
|
||||
internal_session_id=internal_session_id_for(seed.conversation_id, seed.source_id),
|
||||
internal_session_id=internal_session_id_for(
|
||||
seed.conversation_id, seed.source_id, seed.user_id
|
||||
),
|
||||
first_round=first_round,
|
||||
last_round=last_round,
|
||||
evidence_ids=list(ordered),
|
||||
|
||||
@@ -28,6 +28,7 @@ from one_memory.memory_server.contracts import (
|
||||
internal_session_id_for,
|
||||
)
|
||||
from one_memory.memory_server.durable_queue import DurableWriteQueue, DurableWriteRecord
|
||||
from one_memory.user_scope import lane_key
|
||||
|
||||
WritePriority = Literal["live", "normal"]
|
||||
|
||||
@@ -74,7 +75,7 @@ class UnifiedWriteIntake:
|
||||
if priority not in {"live", "normal"}:
|
||||
raise ValueError("priority must be 'live' or 'normal'")
|
||||
if self.source_writes_permitted is not None and not self.source_writes_permitted(
|
||||
record.source_id
|
||||
lane_key(record.user_id, record.source_id)
|
||||
):
|
||||
# Freeze is the product's one Harness processing decision. Refuse
|
||||
# new intake so a frozen source creates no new durable write row;
|
||||
@@ -106,7 +107,9 @@ class UnifiedWriteIntake:
|
||||
last_round: int,
|
||||
priority: WritePriority,
|
||||
) -> WriteRequest:
|
||||
session_id = internal_session_id_for(record.conversation_id, record.source_id)
|
||||
session_id = internal_session_id_for(
|
||||
record.conversation_id, record.source_id, record.user_id
|
||||
)
|
||||
event_at, time_quality = _job_time((record,))
|
||||
request_key = (
|
||||
f"entry:{record.evidence_id}"
|
||||
@@ -145,14 +148,14 @@ class UnifiedWriteIntake:
|
||||
):
|
||||
raise ValueError("records must belong to one conversation")
|
||||
if self.source_writes_permitted is not None and not self.source_writes_permitted(
|
||||
first.source_id
|
||||
lane_key(first.user_id, first.source_id)
|
||||
):
|
||||
raise SourceWriteDisabledError(first.evidence_id)
|
||||
material = ":".join(
|
||||
f"{record.evidence_id}:{record.revision}:{record.content_hash}" for record in records
|
||||
)
|
||||
request_id = f"turn:v1:{sha256(material.encode()).hexdigest()}"
|
||||
session_id = internal_session_id_for(first.conversation_id, first.source_id)
|
||||
session_id = internal_session_id_for(first.conversation_id, first.source_id, first.user_id)
|
||||
event_at, time_quality = _job_time(records)
|
||||
durable = self.queue.submit(
|
||||
WriteRequest(
|
||||
|
||||
@@ -160,7 +160,7 @@ def pending(db: sqlite3.Connection, *, limit: int = 100) -> list[str]:
|
||||
|
||||
|
||||
def _trigger_sql() -> list[tuple[str, str]]:
|
||||
"""(table it depends on, statement) for each trigger."""
|
||||
"""(table it is on, statement) for each trigger."""
|
||||
return [
|
||||
(
|
||||
"evidence",
|
||||
@@ -303,9 +303,61 @@ def install_triggers(db: sqlite3.Connection) -> None:
|
||||
for table, statement in _trigger_sql():
|
||||
if table not in present:
|
||||
continue
|
||||
if not _body_tables(statement) <= present:
|
||||
# SQLite resolves a trigger body only when it fires, so a body
|
||||
# naming a dropped table fails the write it rides on (#233: the
|
||||
# SDK cutover drops `governed_memory_evidence`, and every grant
|
||||
# write then failed). Not installed, and an old copy is removed.
|
||||
cursor.execute(f"DROP TRIGGER IF EXISTS {_trigger_name(statement)}")
|
||||
continue
|
||||
cursor.execute(statement)
|
||||
|
||||
|
||||
def _trigger_name(statement: str) -> str:
|
||||
return statement.split("IF NOT EXISTS", 1)[1].split()[0]
|
||||
|
||||
|
||||
def _body_tables(statement: str) -> set[str]:
|
||||
"""Tables the trigger body reads or writes besides the one it is on."""
|
||||
_head, _begin, body = statement.partition("BEGIN")
|
||||
words = body.replace("(", " ").replace(")", " ").split()
|
||||
return {
|
||||
name
|
||||
for keyword, name in zip(words, words[1:], strict=False)
|
||||
if keyword.upper() in {"FROM", "JOIN", "INTO"}
|
||||
}
|
||||
|
||||
|
||||
def drop_stale_triggers(database_path: Path) -> tuple[str, ...]:
|
||||
"""Drop this module's triggers whose body names a table that is gone.
|
||||
|
||||
Startup on SDK authority does not install the cache (it is read only on
|
||||
legacy authority), but a home migrated by an earlier build still carries
|
||||
the triggers it installed. One whose body reads a dropped table fails
|
||||
every write to the table it is on, so it is dropped here. Idempotent.
|
||||
"""
|
||||
with main_database(database_path).write() as db, db:
|
||||
cursor = db.cursor()
|
||||
cursor.row_factory = sqlite3.Row
|
||||
present = {
|
||||
str(row["name"])
|
||||
for row in cursor.execute("SELECT name FROM sqlite_master WHERE type='table'")
|
||||
}
|
||||
installed = {
|
||||
str(row["name"])
|
||||
for row in cursor.execute("SELECT name FROM sqlite_master WHERE type='trigger'")
|
||||
}
|
||||
dropped = tuple(
|
||||
name
|
||||
for _table, statement in _trigger_sql()
|
||||
if (name := _trigger_name(statement)) in installed
|
||||
and not _body_tables(statement) <= present
|
||||
)
|
||||
for name in dropped:
|
||||
cursor.execute(f"DROP TRIGGER IF EXISTS {name}")
|
||||
return dropped
|
||||
|
||||
|
||||
def startup(database_path: Path) -> None:
|
||||
"""Idempotent: create the derived tables and triggers. No version change.
|
||||
|
||||
|
||||
@@ -564,7 +564,7 @@ def _embedding_request(
|
||||
secret: str | None,
|
||||
) -> tuple[str, str, dict[str, str], dict[str, object]]:
|
||||
provider = profile.embedding_provider_kind or profile.provider_kind
|
||||
base_url = (profile.embedding_base_url or profile.base_url or "").rstrip("/")
|
||||
base_url = (profile.embedding_base_url or "").rstrip("/")
|
||||
if provider is ProviderKind.OLLAMA:
|
||||
return (
|
||||
"POST",
|
||||
|
||||
@@ -97,7 +97,6 @@ class ModelProfileInput(BaseModel):
|
||||
)
|
||||
embedding_provider, resolved_embedding_base_url = resolve_embedding_endpoint(
|
||||
provider_kind=self.provider_kind,
|
||||
base_url=base_url,
|
||||
embedding_model=embedding_model,
|
||||
embedding_provider_kind=embedding_provider_kind,
|
||||
embedding_base_url=embedding_base_url,
|
||||
@@ -107,6 +106,12 @@ class ModelProfileInput(BaseModel):
|
||||
ProviderKind.ANTHROPIC,
|
||||
}:
|
||||
raise ValueError("embedding provider must be OpenAI-compatible, Ollama, or local ONNX")
|
||||
if (
|
||||
embedding_provider is not ProviderKind.LOCAL_ONNX
|
||||
and resolved_embedding_base_url is None
|
||||
):
|
||||
# `ModelProfileStore.create` refuses it with its own code.
|
||||
return self
|
||||
if (
|
||||
embedding_provider is not ProviderKind.LOCAL_ONNX
|
||||
and resolved_embedding_base_url is not None
|
||||
@@ -308,11 +313,12 @@ class ModelProfileStore:
|
||||
f"""
|
||||
UPDATE {table}
|
||||
SET embedding_provider_kind = provider_kind,
|
||||
embedding_base_url = base_url
|
||||
embedding_base_url = COALESCE(embedding_base_url, base_url)
|
||||
WHERE embedding_model IS NOT NULL
|
||||
AND embedding_provider_kind IS NULL
|
||||
"""
|
||||
)
|
||||
_pin_embedding_endpoints(connection, table)
|
||||
connection.execute(
|
||||
"""
|
||||
INSERT OR IGNORE INTO model_profile_versions (
|
||||
@@ -355,11 +361,11 @@ class ModelProfileStore:
|
||||
)
|
||||
embedding_provider_kind, embedding_base_url = resolve_embedding_endpoint(
|
||||
provider_kind=value.provider_kind,
|
||||
base_url=base_url,
|
||||
embedding_model=defaulted_model,
|
||||
embedding_provider_kind=defaulted_provider,
|
||||
embedding_base_url=defaulted_base_url,
|
||||
)
|
||||
require_embedding_base_url(embedding_provider_kind, embedding_base_url)
|
||||
try:
|
||||
with self._connect(writing=True) as connection:
|
||||
connection.execute(
|
||||
@@ -612,7 +618,6 @@ class ModelProfileStore:
|
||||
)
|
||||
embedding_provider_kind, embedding_base_url = resolve_embedding_endpoint(
|
||||
provider_kind=current.provider_kind,
|
||||
base_url=base_url,
|
||||
embedding_model=embedding_model,
|
||||
embedding_provider_kind=(
|
||||
patch.embedding_provider_kind
|
||||
@@ -627,6 +632,7 @@ class ModelProfileStore:
|
||||
)
|
||||
if embedding_provider_kind in {ProviderKind.HY_DEFAULT, ProviderKind.ANTHROPIC}:
|
||||
raise ValueError("embedding provider must be OpenAI-compatible, Ollama, or local ONNX")
|
||||
require_embedding_base_url(embedding_provider_kind, embedding_base_url)
|
||||
if (
|
||||
embedding_provider_kind is not ProviderKind.LOCAL_ONNX
|
||||
and embedding_base_url is not None
|
||||
@@ -1167,21 +1173,77 @@ def attach_disable_thinking(
|
||||
def resolve_embedding_endpoint(
|
||||
*,
|
||||
provider_kind: ProviderKind,
|
||||
base_url: str | None,
|
||||
embedding_model: str | None,
|
||||
embedding_provider_kind: ProviderKind | None,
|
||||
embedding_base_url: str | None,
|
||||
) -> tuple[ProviderKind | None, str | None]:
|
||||
"""The embedding provider and the endpoint exactly as given.
|
||||
|
||||
The endpoint never follows the generation `base_url`: the collection name
|
||||
hashes it, so a proxy or region edit of the generation endpoint moved
|
||||
writes to a new, empty collection, and providers without embeddings were
|
||||
sent embedding requests. `require_embedding_base_url` refuses a remote
|
||||
embedding without one.
|
||||
"""
|
||||
if embedding_model is None:
|
||||
return None, None
|
||||
if embedding_provider_kind is ProviderKind.LOCAL_ONNX:
|
||||
return ProviderKind.LOCAL_ONNX, None
|
||||
if embedding_provider_kind is None:
|
||||
return provider_kind, embedding_base_url or base_url
|
||||
return (
|
||||
embedding_provider_kind,
|
||||
embedding_base_url or DEFAULT_BASE_URLS.get(embedding_provider_kind),
|
||||
)
|
||||
return embedding_provider_kind or provider_kind, embedding_base_url
|
||||
|
||||
|
||||
class EmbeddingBaseUrlRequiredError(ValueError):
|
||||
def __init__(self) -> None:
|
||||
super().__init__("embedding_base_url_required")
|
||||
|
||||
|
||||
def require_embedding_base_url(provider: ProviderKind | None, base_url: str | None) -> None:
|
||||
"""A remote embedding must name its endpoint; the UI may offer the generation one."""
|
||||
if provider not in {None, ProviderKind.LOCAL_ONNX} and not base_url:
|
||||
raise EmbeddingBaseUrlRequiredError()
|
||||
|
||||
|
||||
def pinned_embedding_base_url(
|
||||
*,
|
||||
provider_kind: ProviderKind | str,
|
||||
base_url: str | None,
|
||||
embedding_model: str | None,
|
||||
embedding_provider_kind: ProviderKind | str | None,
|
||||
embedding_base_url: str | None,
|
||||
) -> str | None:
|
||||
"""The endpoint a stored profile's embedding resolved to before it was pinned.
|
||||
|
||||
The Runtime used `embedding_base_url or base_url`, and the provider default
|
||||
only when both were empty (no collection could have existed then). Writing
|
||||
that value keeps the endpoint, and so the collection name, byte-identical.
|
||||
"""
|
||||
provider = ProviderKind(embedding_provider_kind or provider_kind)
|
||||
if embedding_model is None or embedding_base_url or provider is ProviderKind.LOCAL_ONNX:
|
||||
return embedding_base_url
|
||||
return base_url or DEFAULT_BASE_URLS.get(provider)
|
||||
|
||||
|
||||
def _pin_embedding_endpoints(connection: sqlite3.Connection, table: str) -> None:
|
||||
"""Write the resolved endpoint into every remote embedding left without one."""
|
||||
rows = connection.execute(
|
||||
f"SELECT rowid, provider_kind, base_url, embedding_model, embedding_provider_kind "
|
||||
f"FROM {table} WHERE embedding_model IS NOT NULL AND embedding_base_url IS NULL "
|
||||
"AND COALESCE(embedding_provider_kind, provider_kind) != 'local_onnx'"
|
||||
).fetchall()
|
||||
for row in rows:
|
||||
connection.execute(
|
||||
f"UPDATE {table} SET embedding_base_url = ? WHERE rowid = ?",
|
||||
(
|
||||
pinned_embedding_base_url(
|
||||
provider_kind=row["provider_kind"],
|
||||
base_url=row["base_url"],
|
||||
embedding_model=row["embedding_model"],
|
||||
embedding_provider_kind=row["embedding_provider_kind"],
|
||||
embedding_base_url=None,
|
||||
),
|
||||
row["rowid"],
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def validate_model_base_url(value: str) -> None:
|
||||
|
||||
@@ -14,6 +14,7 @@ from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from one_memory.hy_sdk_types import HySdkMemoryRecord
|
||||
from one_memory.memory_governance import GovernedMemory, MemoryStatus
|
||||
from one_memory.sdk_search_options import SdkSearchOptions
|
||||
from one_memory.semantic_deadline import (
|
||||
SemanticMemoryHit,
|
||||
SemanticSearch,
|
||||
@@ -22,6 +23,7 @@ from one_memory.semantic_deadline import (
|
||||
normalize_semantic_hits,
|
||||
search_within,
|
||||
)
|
||||
from one_memory.user_scope import DEFAULT_USER_ID, UserId
|
||||
|
||||
|
||||
def recall_blend_mode() -> str:
|
||||
@@ -49,7 +51,15 @@ class TruncationBudget(StrEnum):
|
||||
TIMEOUT = "timeout_budget"
|
||||
|
||||
|
||||
class RecallRequest(BaseModel):
|
||||
class RecallRequest(SdkSearchOptions):
|
||||
"""`POST /api/v1/recall`.
|
||||
|
||||
The `SdkSearchOptions` fields, `schema_limit` and `intention_limit` pass
|
||||
through to the SDK memory search (and the floor and row filters to the
|
||||
raw fallback). Absent, recall makes the search it always made: the
|
||||
relevance floor, schema and intention lanes sized to `limit`.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
query: str = Field(min_length=1, max_length=2_000)
|
||||
@@ -67,6 +77,19 @@ class RecallRequest(BaseModel):
|
||||
#: Wall-clock budget for candidate gather + score. Not a substitute for
|
||||
#: raising the old newest-500 page; it is the third parallel budget.
|
||||
timeout_ms: int = Field(default=2_000, ge=50, le=30_000)
|
||||
#: Whose memories to recall. Omitted, it is `local`, the App's library.
|
||||
user_id: UserId = DEFAULT_USER_ID
|
||||
#: The SDK's schema (L6) and proactive intention (L7) lanes.
|
||||
schema_limit: int | None = Field(default=None, ge=0, le=20)
|
||||
intention_limit: int | None = Field(default=None, ge=0, le=20)
|
||||
|
||||
def search_overrides(self) -> Mapping[str, object]:
|
||||
values = dict(super().search_overrides())
|
||||
if self.schema_limit is not None:
|
||||
values["schema_limit"] = self.schema_limit
|
||||
if self.intention_limit is not None:
|
||||
values["intention_limit"] = self.intention_limit
|
||||
return values
|
||||
|
||||
|
||||
class RecallHit(BaseModel):
|
||||
|
||||
@@ -6,7 +6,7 @@ import asyncio
|
||||
import json
|
||||
from collections.abc import AsyncIterator, Awaitable, Callable
|
||||
from contextlib import aclosing
|
||||
from typing import Any, Literal, Protocol
|
||||
from typing import Any, Literal, Protocol, cast
|
||||
|
||||
import httpx
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
@@ -14,7 +14,7 @@ from pydantic import BaseModel, ConfigDict
|
||||
from one_memory.http_client import create_async_client
|
||||
from one_memory.hy_runtime_config import HY_MEMORY_READER
|
||||
from one_memory.hy_sdk_runtime import sdk_relevance_floor
|
||||
from one_memory.hy_sdk_types import HySdkMemoryRecord, parse_sdk_search
|
||||
from one_memory.hy_sdk_types import HySdkMemoryRecord, parse_sdk_lane_search, parse_sdk_search
|
||||
from one_memory.memory_qa import (
|
||||
MemoryAnswerError,
|
||||
_DeferredSecretStore,
|
||||
@@ -31,7 +31,9 @@ from one_memory.model_profiles import (
|
||||
ProviderKind,
|
||||
attach_disable_thinking,
|
||||
)
|
||||
from one_memory.sdk_search_options import SdkSearchFilters, SearchLane
|
||||
from one_memory.secrets import SecretStore, SecretStoreUnavailableError
|
||||
from one_memory.user_scope import owns
|
||||
|
||||
_NO_GROUNDING = "现有记忆不足以回答这个问题。"
|
||||
_GROUNDING_LIMIT = 6
|
||||
@@ -95,29 +97,51 @@ class SdkMemoryAnswerer:
|
||||
question: str,
|
||||
*,
|
||||
user_id: str = "local",
|
||||
filters: SdkSearchFilters | None = None,
|
||||
) -> tuple[SdkAskMemory, ...]:
|
||||
"""The user's active memories nearest the question.
|
||||
|
||||
`filters` are a caller's pass-through search options; absent, the
|
||||
search is the one ask always made.
|
||||
"""
|
||||
client = await self.client_handle.client()
|
||||
overrides = filters.search_overrides() if filters is not None else {}
|
||||
raw = await asyncio.to_thread(
|
||||
client.search,
|
||||
question,
|
||||
user_ids=[user_id],
|
||||
agent_ids=[],
|
||||
limit=12,
|
||||
min_score=sdk_relevance_floor(),
|
||||
schema_min_score=sdk_relevance_floor(),
|
||||
schema_limit=12,
|
||||
intention_limit=12,
|
||||
scene="normal",
|
||||
reader=HY_MEMORY_READER,
|
||||
**{
|
||||
"user_ids": [user_id],
|
||||
"agent_ids": [],
|
||||
"limit": 12,
|
||||
"min_score": sdk_relevance_floor(),
|
||||
"schema_min_score": sdk_relevance_floor(),
|
||||
"schema_limit": 12,
|
||||
"intention_limit": 12,
|
||||
"scene": "normal",
|
||||
"reader": HY_MEMORY_READER,
|
||||
**overrides,
|
||||
},
|
||||
)
|
||||
hits = parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
|
||||
if filters is not None and overrides:
|
||||
hits = tuple(
|
||||
hit
|
||||
for hit in parse_sdk_lane_search(raw, error_code="hy_memory_search_shape_unknown")
|
||||
if filters.passes_floor(cast(SearchLane, hit.lane), hit.score)
|
||||
)
|
||||
else:
|
||||
hits = parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
|
||||
found: list[SdkAskMemory] = []
|
||||
seen: set[str] = set()
|
||||
for hit in hits:
|
||||
if hit.memory_id in seen:
|
||||
continue
|
||||
record = await _get_memory(client, hit.memory_id)
|
||||
if record is None or record.status != "active" or not record.is_latest:
|
||||
if (
|
||||
record is None
|
||||
or record.status != "active"
|
||||
or not record.is_latest
|
||||
or not owns(record.user_id, user_id)
|
||||
):
|
||||
continue
|
||||
seen.add(record.memory_id)
|
||||
found.append(_wire_memory(record))
|
||||
@@ -128,8 +152,9 @@ class SdkMemoryAnswerer:
|
||||
question: str,
|
||||
*,
|
||||
user_id: str = "local",
|
||||
filters: SdkSearchFilters | None = None,
|
||||
) -> SdkMemoryAnswer:
|
||||
candidates = await self.retrieve(question, user_id=user_id)
|
||||
candidates = await self.retrieve(question, user_id=user_id, filters=filters)
|
||||
if not candidates:
|
||||
return SdkMemoryAnswer(answer=_NO_GROUNDING, memories=(), mode="empty")
|
||||
profile = await self._active_profile()
|
||||
@@ -160,8 +185,9 @@ class SdkMemoryAnswerer:
|
||||
question: str,
|
||||
*,
|
||||
user_id: str = "local",
|
||||
filters: SdkSearchFilters | None = None,
|
||||
) -> AsyncIterator[dict[str, object]]:
|
||||
candidates = await self.retrieve(question, user_id=user_id)
|
||||
candidates = await self.retrieve(question, user_id=user_id, filters=filters)
|
||||
await self._require_current(candidates)
|
||||
yield {"type": "recall", "memories": candidates}
|
||||
if not candidates:
|
||||
|
||||
@@ -5,8 +5,8 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
import logging
|
||||
import re
|
||||
from collections.abc import AsyncIterator
|
||||
from contextlib import AsyncExitStack, asynccontextmanager, suppress
|
||||
from collections.abc import AsyncIterator, Callable
|
||||
from contextlib import AsyncExitStack, asynccontextmanager
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
@@ -33,16 +33,6 @@ from one_memory.semantic_deadline import SemanticMemoryHit
|
||||
_STABLE_CODE = re.compile(r"[a-z][a-z0-9_]{0,95}")
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
RAW_RETENTION_DAYS = 3
|
||||
RAW_RETENTION_CHECK_SECONDS = 6 * 60 * 60
|
||||
_RAW_STATUSES = (
|
||||
"active",
|
||||
"superseded",
|
||||
"shadow",
|
||||
"negated",
|
||||
"conflicted",
|
||||
"archived",
|
||||
)
|
||||
|
||||
|
||||
class SdkMemoryRuntime:
|
||||
@@ -56,74 +46,17 @@ class SdkMemoryRuntime:
|
||||
write_queue: DurableWriteQueue,
|
||||
extraction_settings: ExtractionSettingsStore,
|
||||
data_dir: Path,
|
||||
memory_mode: Callable[[], str] = lambda: "pro",
|
||||
) -> None:
|
||||
self.coordinator = coordinator
|
||||
#: The SDK mode, for the graph store's path (`hy_graph_admin`).
|
||||
self.memory_mode = memory_mode
|
||||
self.read_handle = read_handle
|
||||
self.write_queue = write_queue
|
||||
self.extraction_settings = extraction_settings
|
||||
self.data_dir = data_dir
|
||||
self._raw_retention_task: asyncio.Task[None] | None = None
|
||||
|
||||
def start_raw_retention(self) -> None:
|
||||
if self._raw_retention_task is None or self._raw_retention_task.done():
|
||||
self._raw_retention_task = asyncio.create_task(
|
||||
self._raw_retention_loop(),
|
||||
name="one-memory-sdk-raw-retention",
|
||||
)
|
||||
|
||||
async def prune_expired_raw(self, *, now: datetime | None = None) -> int:
|
||||
cutoff = (now or datetime.now(UTC)) - timedelta(days=RAW_RETENTION_DAYS)
|
||||
cutoff_timestamp = int(cutoff.timestamp())
|
||||
client = await self.read_handle.client()
|
||||
deleted = 0
|
||||
while True:
|
||||
page = await asyncio.to_thread(
|
||||
client.list_memories,
|
||||
user_id="local",
|
||||
layers=["l1_raw"],
|
||||
statuses=list(_RAW_STATUSES),
|
||||
order="asc",
|
||||
limit=500,
|
||||
offset=0,
|
||||
)
|
||||
bucket = page.get("vdb") if isinstance(page, dict) else None
|
||||
rows = bucket.get("memories") if isinstance(bucket, dict) else None
|
||||
if not isinstance(rows, list) or not rows:
|
||||
return deleted
|
||||
expired = [
|
||||
row
|
||||
for row in rows
|
||||
if isinstance(row, dict)
|
||||
and isinstance(row.get("memory_id"), str)
|
||||
and int(row.get("memory_at") or row.get("gmt_created") or 0)
|
||||
< cutoff_timestamp
|
||||
]
|
||||
if not expired:
|
||||
return deleted
|
||||
removed_page = 0
|
||||
for row in expired:
|
||||
result = await asyncio.to_thread(client.delete, row["memory_id"])
|
||||
if isinstance(result, dict) and result.get("success") is True:
|
||||
count = int(result.get("deleted_count") or 0)
|
||||
deleted += count
|
||||
removed_page += count
|
||||
if removed_page == 0:
|
||||
return deleted
|
||||
|
||||
async def _raw_retention_loop(self) -> None:
|
||||
while True:
|
||||
await asyncio.sleep(RAW_RETENTION_CHECK_SECONDS)
|
||||
try:
|
||||
removed = await self.prune_expired_raw()
|
||||
if removed:
|
||||
logger.info("sdk_raw_retention_deleted count=%d", removed)
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception as error:
|
||||
logger.warning(
|
||||
"sdk_raw_retention_failed error_type=%s",
|
||||
type(error).__name__,
|
||||
)
|
||||
# MemHub keeps every raw the SDK stores, in every mode: there is no
|
||||
# retention window (owner decision; the SDK itself never deletes raw).
|
||||
|
||||
async def semantic_recall(
|
||||
self,
|
||||
@@ -154,11 +87,10 @@ class SdkMemoryRuntime:
|
||||
limit: int = 12,
|
||||
) -> tuple[tuple[str, float], ...]:
|
||||
return tuple(
|
||||
(hit.memory_id, hit.score)
|
||||
for hit in await self.semantic_recall(query, limit=limit)
|
||||
(hit.memory_id, hit.score) for hit in await self.semantic_recall(query, limit=limit)
|
||||
)
|
||||
|
||||
async def graph_snapshot(self, *, limit: int = 300) -> HyGraphSnapshot:
|
||||
async def graph_snapshot(self, *, limit: int = 300, user_id: str = "local") -> HyGraphSnapshot:
|
||||
try:
|
||||
await self.coordinator.active_profile_credentials()
|
||||
except RuntimeError as error:
|
||||
@@ -173,7 +105,7 @@ class SdkMemoryRuntime:
|
||||
return await asyncio.to_thread(
|
||||
read_graph_snapshot,
|
||||
client,
|
||||
"local",
|
||||
user_id,
|
||||
"*",
|
||||
limit,
|
||||
)
|
||||
@@ -224,6 +156,7 @@ class SdkMemoryRuntime:
|
||||
profile,
|
||||
api_key=secret,
|
||||
data_dir=self.data_dir,
|
||||
mode=self.memory_mode(),
|
||||
)
|
||||
|
||||
async def decide_graph_width(
|
||||
@@ -240,6 +173,7 @@ class SdkMemoryRuntime:
|
||||
api_key=secret,
|
||||
data_dir=self.data_dir,
|
||||
decision=decision,
|
||||
mode=self.memory_mode(),
|
||||
)
|
||||
|
||||
@asynccontextmanager
|
||||
@@ -378,12 +312,7 @@ class SdkMemoryRuntime:
|
||||
return None
|
||||
|
||||
async def shutdown(self) -> None:
|
||||
if self._raw_retention_task is None:
|
||||
return
|
||||
self._raw_retention_task.cancel()
|
||||
with suppress(asyncio.CancelledError):
|
||||
await self._raw_retention_task
|
||||
self._raw_retention_task = None
|
||||
return None
|
||||
|
||||
|
||||
def _agent_ids(raw: object) -> tuple[str, ...]:
|
||||
|
||||
@@ -23,6 +23,7 @@ from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot
|
||||
from one_memory.imports.models import EvidenceRecord
|
||||
from one_memory.imports.storage import EvidenceStore
|
||||
from one_memory.memory_server.contracts import (
|
||||
@@ -51,6 +52,7 @@ from one_memory.sdk_migration_bundle import inspect_sdk_migration_bundle
|
||||
from one_memory.sdk_migration_verify import (
|
||||
MemoryReadbackFailed,
|
||||
ReadbackReport,
|
||||
expected_memory_ids,
|
||||
legacy_memory_ids_from,
|
||||
legacy_memory_ids_from_bundle,
|
||||
read_active_profile,
|
||||
@@ -61,6 +63,8 @@ from one_memory.sdk_vector_reattach import (
|
||||
ReattachedCollection,
|
||||
legacy_vector_collections,
|
||||
reattach_legacy_vectors,
|
||||
stored_memory_ids,
|
||||
vectors_belong_to_another_space,
|
||||
)
|
||||
from one_memory.target_schema import drop_legacy_tables, legacy_tables_present
|
||||
|
||||
@@ -259,6 +263,7 @@ def prepare_legacy_write_queue_for_cutover(database_path: Path) -> tuple[int, in
|
||||
internal_session_id = internal_session_id_for(
|
||||
source.conversation_id,
|
||||
source.source_id,
|
||||
source.user_id,
|
||||
)
|
||||
queue.submit(
|
||||
WriteRequest(
|
||||
@@ -456,13 +461,17 @@ def resume_legacy_cleanup(*, data_dir: Path, database_path: Path) -> tuple[str,
|
||||
bundle_id=marker.backup_bundle_id,
|
||||
)
|
||||
from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE
|
||||
from one_memory.memory_visibility import drop_stale_triggers
|
||||
|
||||
recover_abandoned_staging(
|
||||
data_dir=data_dir,
|
||||
database_path=database_path,
|
||||
namespace=HY_MEMORY_NAMESPACE,
|
||||
)
|
||||
return drop_legacy_tables(database_path)
|
||||
dropped = drop_legacy_tables(database_path)
|
||||
# The visibility cache's grant triggers read a table just dropped (#233).
|
||||
drop_stale_triggers(database_path)
|
||||
return dropped
|
||||
|
||||
|
||||
MIGRATION_PREFIX = "sdk-authority-v2"
|
||||
@@ -699,13 +708,14 @@ def run_startup_migration(
|
||||
step["authority"] = authority.value
|
||||
log.migration_id = authority_migration
|
||||
with log.step("preflight.profile_read") as step:
|
||||
profile = read_active_profile(database_path)
|
||||
step["profile_present"] = profile is not None
|
||||
active = read_active_profile(database_path)
|
||||
step["profile_present"] = active is not None
|
||||
# No active profile means no generation model, not no embedding: the
|
||||
# vectors are the built-in model's, which is also what the Runtime reads.
|
||||
profile = active if active is not None else builtin_retrieval_snapshot()
|
||||
if authority is MemoryAuthority.SDK:
|
||||
with log.step("preflight.leftovers_scan") as step:
|
||||
vectors_waiting = profile is not None and bool(
|
||||
legacy_vector_collections(data_dir, profile)
|
||||
)
|
||||
vectors_waiting = bool(legacy_vector_collections(data_dir, profile))
|
||||
leftovers = bool(legacy_tables_present(database_path)) or abandoned_staging_present(
|
||||
data_dir=data_dir, namespace=namespace
|
||||
)
|
||||
@@ -716,7 +726,6 @@ def run_startup_migration(
|
||||
reattached: tuple[ReattachedCollection, ...] = ()
|
||||
readback: ReadbackReport | None = None
|
||||
if vectors_waiting:
|
||||
assert profile is not None
|
||||
progress("vectors")
|
||||
with log.step("vectors.reattach") as step:
|
||||
reattached = reattach_legacy_vectors(data_dir, profile)
|
||||
@@ -768,12 +777,12 @@ def run_startup_migration(
|
||||
step["chat"] = len(legacy.chat)
|
||||
step["productivity"] = len(legacy.productivity)
|
||||
step["users"] = len(legacy.users)
|
||||
if profile is None and not legacy.empty:
|
||||
# Without the embedding profile the Runtime's collections cannot be
|
||||
# named, so neither reattached nor read back. Legacy stays intact.
|
||||
log.check("preflight.model_profile_check", "sdk_migration_model_profile_required")
|
||||
raise DataPlaneMigrationError("sdk_migration_model_profile_required")
|
||||
log.check("preflight.model_profile_check", None)
|
||||
if active is None and not legacy.empty and vectors_belong_to_another_space(data_dir, profile):
|
||||
# The memories live in the space of a profile that is no longer active,
|
||||
# so the built-in model cannot read them. Legacy stays intact.
|
||||
log.check("preflight.vector_space_check", "sdk_migration_vector_profile_missing")
|
||||
raise DataPlaneMigrationError("sdk_migration_vector_profile_missing")
|
||||
log.check("preflight.vector_space_check", None)
|
||||
with capture_fence(data_dir):
|
||||
# Builds through 19 left a staged SDK copy beside the original store.
|
||||
# Settling it before the bundle means the bundle holds the original
|
||||
@@ -829,17 +838,23 @@ def run_startup_migration(
|
||||
step.update(preparation.as_document())
|
||||
reattached = ()
|
||||
readback = None
|
||||
if profile is not None:
|
||||
progress("vectors")
|
||||
with log.step("vectors.reattach") as step:
|
||||
reattached = reattach_legacy_vectors(data_dir, profile)
|
||||
step["collections"] = len(reattached)
|
||||
if not legacy.empty:
|
||||
assert profile is not None
|
||||
# Before reattach touches the store, and after the staging
|
||||
# recovery above has settled which store is the original.
|
||||
with log.step("verify.stored_ids") as step:
|
||||
expected, missing = expected_memory_ids(legacy, stored_memory_ids(data_dir))
|
||||
step["chat_expected"] = len(expected.chat)
|
||||
step["chat_missing_before_migration"] = missing
|
||||
progress("vectors")
|
||||
with log.step("vectors.reattach") as step:
|
||||
reattached = reattach_legacy_vectors(data_dir, profile)
|
||||
step["collections"] = len(reattached)
|
||||
if not expected.empty:
|
||||
progress("verify")
|
||||
with log.step("verify.readback") as step:
|
||||
with client_for(profile) as client:
|
||||
readback = verify_readback(client, legacy)
|
||||
readback = verify_readback(
|
||||
client, expected, chat_missing_before_migration=missing
|
||||
)
|
||||
step.update(readback.as_document())
|
||||
if not readback.passed:
|
||||
store.set_phase(migration_id, phase="readback_failed", state="blocked")
|
||||
|
||||
@@ -14,12 +14,12 @@ import json
|
||||
import sqlite3
|
||||
from collections.abc import Iterator
|
||||
from contextlib import closing, contextmanager
|
||||
from dataclasses import dataclass
|
||||
from dataclasses import dataclass, replace
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.model_profiles import ModelProfileSnapshot
|
||||
from one_memory.model_profiles import ModelProfileSnapshot, pinned_embedding_base_url
|
||||
from one_memory.sdk_data_plane_migration import DataPlaneMigrationError
|
||||
|
||||
_PRODUCTIVITY_KINDS = frozenset({"coding", "working"})
|
||||
@@ -47,6 +47,8 @@ class ReadbackReport:
|
||||
chat_readable: int
|
||||
productivity_expected: int
|
||||
productivity_readable: int
|
||||
#: Confirmed in the mirror, gone from the SDK before this migration began.
|
||||
chat_missing_before_migration: int = 0
|
||||
|
||||
@property
|
||||
def passed(self) -> bool:
|
||||
@@ -63,6 +65,7 @@ class ReadbackReport:
|
||||
"chat_readable": self.chat_readable,
|
||||
"productivity_expected": self.productivity_expected,
|
||||
"productivity_readable": self.productivity_readable,
|
||||
"chat_missing_before_migration": self.chat_missing_before_migration,
|
||||
"passed": self.passed,
|
||||
}
|
||||
|
||||
@@ -115,6 +118,27 @@ def read_legacy_memory_ids(connection: sqlite3.Connection) -> LegacyMemoryIds:
|
||||
)
|
||||
|
||||
|
||||
def expected_memory_ids(
|
||||
legacy: LegacyMemoryIds, stored: frozenset[str]
|
||||
) -> tuple[LegacyMemoryIds, int]:
|
||||
"""What the readback must find, and how many confirmed chats it need not.
|
||||
|
||||
A 1.0.4 library can hold confirmed rows whose SDK node the SDK had already
|
||||
merged or deleted: no migration could read those back (658 confirmed, 657
|
||||
in the store, and the whole upgrade failed). So chat is held to what the
|
||||
store held before this run, and anything in there must still read back.
|
||||
|
||||
If none of the confirmed chats is in the store, that is not history: the
|
||||
store is empty or is the wrong one. Nothing is excused then, so the
|
||||
readback still fails on every id instead of checking nothing just before
|
||||
the mirror is dropped.
|
||||
"""
|
||||
chat = tuple(memory_id for memory_id in legacy.chat if memory_id in stored)
|
||||
if not chat:
|
||||
return legacy, 0
|
||||
return replace(legacy, chat=chat), len(legacy.chat) - len(chat)
|
||||
|
||||
|
||||
def legacy_memory_ids_from(database_path: Path) -> LegacyMemoryIds:
|
||||
with main_database(database_path).read() as connection:
|
||||
return read_legacy_memory_ids(connection)
|
||||
@@ -160,7 +184,14 @@ def read_active_profile(database_path: Path) -> ModelProfileSnapshot | None:
|
||||
model=row["model"],
|
||||
embedding_model=row["embedding_model"],
|
||||
embedding_provider_kind=row["embedding_provider_kind"],
|
||||
embedding_base_url=row["embedding_base_url"],
|
||||
# Read-only, beside a Core that may not have pinned it yet.
|
||||
embedding_base_url=pinned_embedding_base_url(
|
||||
provider_kind=row["provider_kind"],
|
||||
base_url=row["base_url"],
|
||||
embedding_model=row["embedding_model"],
|
||||
embedding_provider_kind=row["embedding_provider_kind"],
|
||||
embedding_base_url=row["embedding_base_url"],
|
||||
),
|
||||
embedding_dimensions=row["embedding_dimensions"],
|
||||
timeout_seconds=row["timeout_seconds"],
|
||||
max_concurrency=row["max_concurrency"],
|
||||
@@ -179,14 +210,18 @@ def runtime_sdk_client(
|
||||
from one_memory.model_profiles import ModelProfileStore
|
||||
from one_memory.secrets import KeyringSecretStore, SecretStoreUnavailableError
|
||||
|
||||
try:
|
||||
secret = ModelProfileStore(database_path).resolve_snapshot_secret(
|
||||
profile.profile_id, profile.revision, KeyringSecretStore()
|
||||
)
|
||||
except SecretStoreUnavailableError:
|
||||
raise DataPlaneMigrationError("sdk_migration_secure_storage_unavailable") from None
|
||||
if profile.has_api_key and secret is None:
|
||||
raise DataPlaneMigrationError("sdk_migration_model_secret_unavailable")
|
||||
secret: str | None = None
|
||||
# Without a key there is nothing to resolve, and the built-in retrieval
|
||||
# snapshot has no stored version to look one up by.
|
||||
if profile.has_api_key:
|
||||
try:
|
||||
secret = ModelProfileStore(database_path).resolve_snapshot_secret(
|
||||
profile.profile_id, profile.revision, KeyringSecretStore()
|
||||
)
|
||||
except SecretStoreUnavailableError:
|
||||
raise DataPlaneMigrationError("sdk_migration_secure_storage_unavailable") from None
|
||||
if secret is None:
|
||||
raise DataPlaneMigrationError("sdk_migration_model_secret_unavailable")
|
||||
lease = acquire_client(profile, api_key=secret, data_dir=data_dir)
|
||||
try:
|
||||
yield lease.client
|
||||
@@ -194,7 +229,9 @@ def runtime_sdk_client(
|
||||
lease.close()
|
||||
|
||||
|
||||
def verify_readback(client: Any, legacy: LegacyMemoryIds) -> ReadbackReport:
|
||||
def verify_readback(
|
||||
client: Any, legacy: LegacyMemoryIds, *, chat_missing_before_migration: int = 0
|
||||
) -> ReadbackReport:
|
||||
listed: set[str] = set()
|
||||
for user_id in legacy.users or ("local",):
|
||||
offset = 0
|
||||
@@ -228,5 +265,6 @@ def verify_readback(client: Any, legacy: LegacyMemoryIds) -> ReadbackReport:
|
||||
chat_readable=sum(1 for memory_id in legacy.chat if readable(memory_id)),
|
||||
productivity_expected=len(legacy.productivity),
|
||||
productivity_readable=sum(1 for memory_id in legacy.productivity if readable(memory_id)),
|
||||
chat_missing_before_migration=chat_missing_before_migration,
|
||||
)
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
"""The read client, kept in the embedding space the active profile writes to.
|
||||
|
||||
Every read used to open the built-in 512-dim collection while writes followed
|
||||
the active profile, so a custom embedding read an empty space (#242). This
|
||||
handle opens the active profile's `retrieval_view` instead, and the built-in
|
||||
space only when no profile is active.
|
||||
|
||||
A profile change builds a new client and retires the old one; a retired client
|
||||
is closed once no task still uses it, as `SdkWriteCoordinator` does for writes
|
||||
(#218). A use is the task that asked for the client: callers keep their
|
||||
`client = await handle.client()` shape, and the use ends with that task (one
|
||||
HTTP request, one retention pass).
|
||||
|
||||
The memory mode (lite / pro / ultra) is fixed per SDK client, so a mode change
|
||||
rebuilds and retires the same way: handles are keyed by (profile, mode). The
|
||||
mode never names a collection; it is a separate client parameter.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from one_memory.hy_graph_admin import effective_memory_mode
|
||||
from one_memory.hy_sdk_runtime import (
|
||||
BUILTIN_RETRIEVAL_PROFILE_ID,
|
||||
SdkClientHandle,
|
||||
builtin_retrieval_snapshot,
|
||||
retrieval_view,
|
||||
)
|
||||
from one_memory.hy_sdk_types import HySdkRuntimeError
|
||||
from one_memory.model_profiles import ModelProfileStore, ProviderKind
|
||||
from one_memory.secrets import SecretStore, SecretStoreUnavailableError
|
||||
|
||||
_BUILTIN = (BUILTIN_RETRIEVAL_PROFILE_ID, 0)
|
||||
|
||||
|
||||
class ActiveProfileReadHandle(SdkClientHandle):
|
||||
"""An `SdkClientHandle` whose profile follows the active one."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
profiles: ModelProfileStore,
|
||||
secrets: SecretStore,
|
||||
data_dir: Path,
|
||||
memory_mode: Callable[[], str] = effective_memory_mode,
|
||||
) -> None:
|
||||
super().__init__(builtin_retrieval_snapshot(), api_key=None, data_dir=data_dir)
|
||||
self.profiles = profiles
|
||||
self.secrets = secrets
|
||||
self.data_dir = data_dir
|
||||
self.memory_mode = memory_mode
|
||||
self._handles: dict[tuple[str, int, str], SdkClientHandle] = {}
|
||||
#: Tasks in flight per handle; a retired handle closes at zero.
|
||||
self._uses: dict[SdkClientHandle, int] = {}
|
||||
self._retired: set[SdkClientHandle] = set()
|
||||
self._closing: set[asyncio.Task[None]] = set()
|
||||
self._handle_lock = asyncio.Lock()
|
||||
|
||||
async def client(self) -> Any:
|
||||
profile = await asyncio.to_thread(self.profiles.active_identity) or _BUILTIN
|
||||
identity = (*profile, self.memory_mode())
|
||||
async with self._handle_lock:
|
||||
handle = self._handles.get(identity)
|
||||
if handle is None:
|
||||
# Only on a change of profile: resolving a key can reach the
|
||||
# system keychain, so it is not done on every read.
|
||||
handle = await asyncio.to_thread(self._build, identity)
|
||||
self._handles[identity] = handle
|
||||
self._retired.update(value for key, value in self._handles.items() if key != identity)
|
||||
self._handles = {identity: handle}
|
||||
unused = {value for value in self._retired if not self._uses.get(value)}
|
||||
self._retired -= unused
|
||||
self._hold(handle)
|
||||
await asyncio.gather(*(value.close() for value in unused))
|
||||
return await handle.client()
|
||||
|
||||
async def close(self) -> None:
|
||||
async with self._handle_lock:
|
||||
handles = (*self._handles.values(), *self._retired)
|
||||
self._handles, self._retired, self._uses = {}, set(), {}
|
||||
await asyncio.gather(*(handle.close() for handle in handles), *self._closing)
|
||||
|
||||
def _hold(self, handle: SdkClientHandle) -> None:
|
||||
"""Count a use of `handle` until the asking task finishes."""
|
||||
task = asyncio.current_task()
|
||||
if task is None: # pragma: no cover - always called from a coroutine
|
||||
return
|
||||
self._uses[handle] = self._uses.get(handle, 0) + 1
|
||||
task.add_done_callback(lambda _task: self._release(handle))
|
||||
|
||||
def _release(self, handle: SdkClientHandle) -> None:
|
||||
remaining = self._uses.get(handle, 0) - 1
|
||||
if remaining > 0:
|
||||
self._uses[handle] = remaining
|
||||
return
|
||||
self._uses.pop(handle, None)
|
||||
if handle in self._retired:
|
||||
self._retired.discard(handle)
|
||||
closing = asyncio.get_running_loop().create_task(handle.close())
|
||||
self._closing.add(closing)
|
||||
closing.add_done_callback(self._closing.discard)
|
||||
|
||||
def _build(self, key: tuple[str, int, str]) -> SdkClientHandle:
|
||||
profile_id, revision, mode = key
|
||||
identity = (profile_id, revision)
|
||||
if identity == _BUILTIN:
|
||||
return SdkClientHandle(
|
||||
builtin_retrieval_snapshot(), api_key=None, data_dir=self.data_dir, mode=mode
|
||||
)
|
||||
profile = self.profiles.snapshot(*identity)
|
||||
view = retrieval_view(profile)
|
||||
secret: str | None = None
|
||||
# Only a remote OpenAI-compatible embedding sends the shared key.
|
||||
if view.embedding_provider_kind is ProviderKind.OPENAI_COMPATIBLE:
|
||||
try:
|
||||
secret = self.profiles.resolve_snapshot_secret(*identity, self.secrets)
|
||||
except SecretStoreUnavailableError:
|
||||
raise HySdkRuntimeError("secure_storage_unavailable") from None
|
||||
if secret is None:
|
||||
raise HySdkRuntimeError("model_secret_unavailable")
|
||||
return SdkClientHandle(view, api_key=secret, data_dir=self.data_dir, mode=mode)
|
||||
@@ -0,0 +1,179 @@
|
||||
"""SDK search parameters a host may pass through to `client.search`.
|
||||
|
||||
`POST /api/v2/context/search`, `POST /api/v1/recall` and the v2 ask routes
|
||||
used to fix every SDK search knob, so a host that wanted a stricter relevance
|
||||
floor filtered the answer itself. These fields are the public `search`
|
||||
parameters of hy-memory 2.0.9 that are safe to hand a caller. Every field is
|
||||
optional: absent, the route keeps exactly the call it made before.
|
||||
|
||||
Never exposed: `user_ids` (a route searches its one scoped `user_id`),
|
||||
`embed_override` and `query_embedding` (they bypass the configured embedder),
|
||||
and `request_id` (Core's own correlation id).
|
||||
|
||||
A caller's floor is also applied to what Core returns, not only handed to the
|
||||
SDK: a hit below the floor that names its lane is dropped (`passes_floor`).
|
||||
With `min_score` alone that floor covers every lane, so `min_score=0.9`
|
||||
answers only results scoring at least 0.9.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from typing import Annotated, Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, StrictBool, model_validator
|
||||
|
||||
#: The SDK's `MemoryLayer` values (hy-memory 2.0.9).
|
||||
MemoryLayerName = Literal[
|
||||
"l0_basic_info",
|
||||
"l1_raw",
|
||||
"l2_fact",
|
||||
"l3_summary",
|
||||
"l4_identity",
|
||||
"l5_knowledge",
|
||||
"l6_schema",
|
||||
"l7_intention",
|
||||
]
|
||||
MEMORY_LAYERS: tuple[str, ...] = (
|
||||
"l0_basic_info",
|
||||
"l1_raw",
|
||||
"l2_fact",
|
||||
"l3_summary",
|
||||
"l4_identity",
|
||||
"l5_knowledge",
|
||||
"l6_schema",
|
||||
"l7_intention",
|
||||
)
|
||||
#: The SDK's `MemoryStatus` values.
|
||||
MEMORY_STATUSES: tuple[str, ...] = (
|
||||
"active",
|
||||
"superseded",
|
||||
"negated",
|
||||
"conflicted",
|
||||
"shadow",
|
||||
"archived",
|
||||
)
|
||||
SearchScene = Literal["chat", "working", "coding", "all"]
|
||||
#: Which recall lane produced a hit, for the floor that applies to it.
|
||||
SearchLane = Literal["normal", "schema", "proactive", "working", "coding", "raw"]
|
||||
|
||||
Score = Annotated[float, Field(ge=0.0, le=1.0)]
|
||||
SourceName = Annotated[str, Field(min_length=1, max_length=128)]
|
||||
ScopeId = Annotated[str, Field(min_length=1, max_length=512)]
|
||||
|
||||
|
||||
class SdkSearchFilters(BaseModel):
|
||||
"""Filters and floors every search route accepts (recall, context, ask)."""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
#: Chat lane floor; with nothing more specific, every lane's.
|
||||
min_score: Score | None = None
|
||||
#: Schema (L6) lane floor; defaults to `min_score` when only that is set.
|
||||
schema_min_score: Score | None = None
|
||||
#: False skips the SDK's reranking stages for this call.
|
||||
rerank: StrictBool | None = None
|
||||
include_sources: tuple[SourceName, ...] | None = Field(default=None, max_length=64)
|
||||
exclude_sources: tuple[SourceName, ...] | None = Field(default=None, max_length=64)
|
||||
#: Unix seconds: only memories created at or after it.
|
||||
created_after: float | None = Field(default=None, ge=0)
|
||||
session_ids: tuple[ScopeId, ...] | None = Field(default=None, max_length=64)
|
||||
agent_ids: tuple[ScopeId, ...] | None = Field(default=None, max_length=64)
|
||||
#: Only these layers (pro/ultra; lite always searches raw).
|
||||
layers: tuple[MemoryLayerName, ...] | None = Field(default=None, min_length=1, max_length=8)
|
||||
|
||||
def filter_kwargs(self) -> dict[str, object]:
|
||||
"""The SDK keyword arguments this request sets, and only those."""
|
||||
values: dict[str, object] = {}
|
||||
for name in (
|
||||
"min_score",
|
||||
"schema_min_score",
|
||||
"rerank",
|
||||
"created_after",
|
||||
):
|
||||
value = getattr(self, name)
|
||||
if value is not None:
|
||||
values[name] = value
|
||||
for name in ("include_sources", "exclude_sources", "session_ids", "agent_ids", "layers"):
|
||||
value = getattr(self, name)
|
||||
if value is not None:
|
||||
values[name] = list(dict.fromkeys(value))
|
||||
if self.min_score is not None and self.schema_min_score is None:
|
||||
values["schema_min_score"] = self.min_score
|
||||
return values
|
||||
|
||||
def lane_floor(self, lane: SearchLane) -> float | None:
|
||||
"""The caller's floor for one lane, or None when the caller set none."""
|
||||
if lane == "schema":
|
||||
return self.schema_min_score if self.schema_min_score is not None else self.min_score
|
||||
return self.min_score
|
||||
|
||||
def passes_floor(self, lane: SearchLane, score: float) -> bool:
|
||||
floor = self.lane_floor(lane)
|
||||
return floor is None or score >= floor
|
||||
|
||||
def search_overrides(self) -> Mapping[str, object]:
|
||||
return self.filter_kwargs()
|
||||
|
||||
|
||||
class SdkSearchOptions(SdkSearchFilters):
|
||||
"""Every SDK search knob a recall or context caller may set."""
|
||||
|
||||
#: working/coding fact-layer floor (SDK default 0.30).
|
||||
working_fact_min_score: Score | None = None
|
||||
#: working/coding experience-layer floor (SDK default fact floor + 0.10).
|
||||
working_experience_min_score: Score | None = None
|
||||
reader: Literal["legacy", "hybrid"] | None = None
|
||||
#: chat (the default), working, coding, or all three fused.
|
||||
scene: SearchScene | None = None
|
||||
#: agentic spends one model call to classify and rewrite the query.
|
||||
search_mode: Literal["default", "agentic"] | None = None
|
||||
#: working/coding lanes only; `branch` needs `workspace_id`.
|
||||
workspace_id: ScopeId | None = None
|
||||
branch: ScopeId | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def branch_needs_workspace(self) -> SdkSearchOptions:
|
||||
if self.branch is not None and self.workspace_id is None:
|
||||
raise ValueError("branch requires workspace_id")
|
||||
return self
|
||||
|
||||
def search_overrides(self) -> Mapping[str, object]:
|
||||
values = self.filter_kwargs()
|
||||
for name in (
|
||||
"working_fact_min_score",
|
||||
"working_experience_min_score",
|
||||
"reader",
|
||||
"scene",
|
||||
"search_mode",
|
||||
"workspace_id",
|
||||
"branch",
|
||||
):
|
||||
value = getattr(self, name)
|
||||
if value is not None:
|
||||
values[name] = value
|
||||
if self.min_score is not None and self.working_fact_min_score is None:
|
||||
values["working_fact_min_score"] = self.min_score
|
||||
if self.min_score is not None and self.working_experience_min_score is None:
|
||||
values["working_experience_min_score"] = self.min_score
|
||||
return values
|
||||
|
||||
def lane_floor(self, lane: SearchLane) -> float | None:
|
||||
if lane in {"working", "coding"}:
|
||||
explicit = [
|
||||
value
|
||||
for value in (self.working_fact_min_score, self.working_experience_min_score)
|
||||
if value is not None
|
||||
]
|
||||
# Facts and experiences share one score field on a fused hit, so
|
||||
# the looser of the two floors is the one Core can enforce.
|
||||
return min(explicit) if explicit else self.min_score
|
||||
return super().lane_floor(lane)
|
||||
|
||||
|
||||
def split_csv(values: list[str] | None) -> list[str]:
|
||||
"""Query values given repeated (`a&a`) or comma separated (`a,b`)."""
|
||||
found: list[str] = []
|
||||
for value in values or ():
|
||||
found.extend(part.strip() for part in value.split(",") if part.strip())
|
||||
return list(dict.fromkeys(found))
|
||||
@@ -9,6 +9,7 @@ one. Runtime code only reaches vectors through the SDK. This module runs from
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import sqlite3
|
||||
from collections.abc import Iterator
|
||||
from contextlib import closing
|
||||
@@ -21,6 +22,8 @@ from one_memory.model_profiles import ModelProfileSnapshot
|
||||
from one_memory.sdk_data_plane_migration import DataPlaneMigrationError
|
||||
|
||||
_BATCH = 500
|
||||
#: Every name a MemHub build gave an embedding space's base collection.
|
||||
_SPACE_NAME = re.compile(r"^(?:memhub_[0-9a-f]{16}|one_memory_[0-9a-f]{12})(?:_|$)")
|
||||
|
||||
|
||||
def open_offline_chroma(directory: Path) -> Any:
|
||||
@@ -51,13 +54,20 @@ def _vector_directory(data_dir: Path) -> Path:
|
||||
return data_dir / "hy-memory" / "governed-v2" / "vector"
|
||||
|
||||
|
||||
def _stored_names(data_dir: Path) -> tuple[str, ...]:
|
||||
def _stored_names(data_dir: Path, *, filled: bool = False) -> tuple[str, ...]:
|
||||
"""Collection names; with `filled`, only those holding at least one row."""
|
||||
database = _vector_directory(data_dir) / "chroma.sqlite3"
|
||||
if not database.is_file():
|
||||
return ()
|
||||
with closing(sqlite3.connect(f"file:{database}?mode=ro", uri=True)) as connection:
|
||||
try:
|
||||
rows = connection.execute("SELECT name FROM collections").fetchall()
|
||||
rows = connection.execute(
|
||||
"SELECT DISTINCT collections.name FROM collections "
|
||||
"JOIN segments ON segments.collection = collections.id "
|
||||
"JOIN embeddings ON embeddings.segment_id = segments.id"
|
||||
if filled
|
||||
else "SELECT name FROM collections"
|
||||
).fetchall()
|
||||
except sqlite3.Error:
|
||||
return ()
|
||||
return tuple(sorted(str(row[0]) for row in rows))
|
||||
@@ -80,6 +90,59 @@ def legacy_vector_collections(
|
||||
return tuple(pairs)
|
||||
|
||||
|
||||
def vectors_belong_to_another_space(data_dir: Path, profile: ModelProfileSnapshot) -> bool:
|
||||
"""Vectors exist, but only in embedding spaces `profile` does not name.
|
||||
|
||||
Counted rows, not names: every Runtime opens its read collection with
|
||||
`get_or_create`, so an empty collection under the expected base proves
|
||||
nothing. Read-only, so a preflight can ask before anything changes.
|
||||
"""
|
||||
current, legacy_bases = vector_collection_bases(profile)
|
||||
known = (current, *legacy_bases)
|
||||
filled = _stored_names(data_dir, filled=True)
|
||||
return not any(
|
||||
name == base or name.startswith(f"{base}_") for name in filled for base in known
|
||||
) and any(_SPACE_NAME.match(name) for name in filled)
|
||||
|
||||
|
||||
def stored_memory_ids(data_dir: Path) -> frozenset[str]:
|
||||
"""Every id in the vector store as found, across all embedding spaces.
|
||||
|
||||
All of them on purpose. A memory stranded under a space the Runtime will
|
||||
not read must count as present, so the readback then fails on it instead
|
||||
of excusing it. Both the point id and the SDK's `node_id` are kept: the
|
||||
SDK derives a point id from a node id that is not a UUID.
|
||||
|
||||
Callers must have stopped every SDK writer. A store that cannot be read
|
||||
raises: "unknown" must never be taken for "absent".
|
||||
"""
|
||||
if not (_vector_directory(data_dir) / "chroma.sqlite3").is_file():
|
||||
return frozenset()
|
||||
found: set[str] = set()
|
||||
try:
|
||||
client = open_offline_chroma(_vector_directory(data_dir))
|
||||
try:
|
||||
for collection in client.list_collections():
|
||||
offset = 0
|
||||
while True:
|
||||
page = collection.get(include=["metadatas"], limit=_BATCH, offset=offset)
|
||||
ids = list(page["ids"])
|
||||
if not ids:
|
||||
break
|
||||
found.update(ids)
|
||||
found.update(
|
||||
str(metadata["node_id"])
|
||||
for metadata in page["metadatas"] or ()
|
||||
if metadata and metadata.get("node_id")
|
||||
)
|
||||
offset += len(ids)
|
||||
finally:
|
||||
client.close()
|
||||
except Exception as error:
|
||||
raise DataPlaneMigrationError("sdk_migration_vector_store_unreadable") from error
|
||||
return frozenset(found)
|
||||
|
||||
|
||||
def _records(collection: Any) -> Iterator[dict[str, Any]]:
|
||||
offset = 0
|
||||
while True:
|
||||
|
||||
@@ -9,7 +9,8 @@ from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from one_memory.hy_sdk_runtime import SdkClientHandle
|
||||
from one_memory.hy_graph_admin import effective_memory_mode
|
||||
from one_memory.hy_sdk_runtime import SdkClientHandle, builtin_retrieval_snapshot
|
||||
from one_memory.imports.models import EvidenceContentKind, EvidenceOrigin, EvidenceRecord
|
||||
from one_memory.input_sanitization import sanitize_model_input
|
||||
from one_memory.model_profiles import ModelProfileSnapshot, ModelProfileStore
|
||||
@@ -31,8 +32,11 @@ class SdkWriteReceipt:
|
||||
class SdkWriteProcessor:
|
||||
"""Turn one durable job into exactly one public SDK write call."""
|
||||
|
||||
def __init__(self, client_handle: SdkClientHandle) -> None:
|
||||
def __init__(self, client_handle: SdkClientHandle, *, lite: bool = False) -> None:
|
||||
self._client_handle = client_handle
|
||||
#: lite makes no model call: `add_extracted` would still call one, so a
|
||||
#: direct memory note is stored as one raw user turn through `add`.
|
||||
self._lite = lite
|
||||
|
||||
async def process_sdk_job(
|
||||
self,
|
||||
@@ -64,7 +68,7 @@ class SdkWriteProcessor:
|
||||
# waits and retries instead of being left `unknown`.
|
||||
raise RuntimeError("hy_memory_client_unavailable") from error
|
||||
|
||||
if extracted:
|
||||
if extracted and not self._lite:
|
||||
contents = [sanitize_model_input(record.verbatim) for record in records]
|
||||
await before_hy_invocation()
|
||||
result = await asyncio.to_thread(
|
||||
@@ -81,12 +85,15 @@ class SdkWriteProcessor:
|
||||
else:
|
||||
messages = [
|
||||
{
|
||||
"role": record.role,
|
||||
"role": "user" if extracted else record.role,
|
||||
"content": sanitize_model_input(record.verbatim),
|
||||
}
|
||||
for record in records
|
||||
if record.content_kind is EvidenceContentKind.RAW_CONVERSATION
|
||||
and record.role in _MESSAGE_ROLES
|
||||
if extracted
|
||||
or (
|
||||
record.content_kind is EvidenceContentKind.RAW_CONVERSATION
|
||||
and record.role in _MESSAGE_ROLES
|
||||
)
|
||||
]
|
||||
if not messages:
|
||||
raise RuntimeError("hy_memory_task_without_inputs")
|
||||
@@ -138,6 +145,15 @@ class SdkWriteCoordinator:
|
||||
from under a write still running on it: the SDK cleared its stores and
|
||||
the SharedRuntime closed mid-`add`, the job failed with AttributeError
|
||||
after HY was invoked, and it was left `unknown`.
|
||||
|
||||
Handles are keyed by profile and memory mode, so a mode change retires
|
||||
handles the same way. In lite no generation model is needed: without an
|
||||
active profile the built-in local embedder writes raw turns.
|
||||
|
||||
`exclusive_for_evolve(user_id)` keeps evolve and that user's writes
|
||||
apart: it waits for the user's write in flight, and while it runs that
|
||||
user's jobs are refused before the SDK is called
|
||||
(`memory_evolve_in_progress`), so the drain puts them back.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
@@ -148,13 +164,22 @@ class SdkWriteCoordinator:
|
||||
data_dir: Any,
|
||||
transport_factory: Callable[[ModelProfileSnapshot], Callable[[], Any] | None],
|
||||
sedimentation_enabled: Callable[[], bool],
|
||||
memory_mode: Callable[[], str] = effective_memory_mode,
|
||||
) -> None:
|
||||
self.profiles = profiles
|
||||
self.secrets = secrets
|
||||
self.data_dir = data_dir
|
||||
self.transport_factory = transport_factory
|
||||
self.sedimentation_enabled = sedimentation_enabled
|
||||
self._handles: dict[tuple[str, int], SdkClientHandle] = {}
|
||||
self.memory_mode = memory_mode
|
||||
self._handles: dict[tuple[str, int, str], SdkClientHandle] = {}
|
||||
#: Users evolve (or the graph backfill) is running for, the writes in
|
||||
#: flight per user, and one exclusive run per user at a time. Writes
|
||||
#: of one user still run side by side; only evolve waits for them.
|
||||
self._evolving: set[str] = set()
|
||||
self._writing: dict[str, int] = {}
|
||||
self._writes_idle = asyncio.Condition()
|
||||
self._exclusive: dict[str, asyncio.Lock] = {}
|
||||
#: Uses in flight per handle; a retired handle closes at zero.
|
||||
self._uses: dict[SdkClientHandle, int] = {}
|
||||
self._retired: set[SdkClientHandle] = set()
|
||||
@@ -169,16 +194,66 @@ class SdkWriteCoordinator:
|
||||
) -> SdkWriteReceipt:
|
||||
if not self.sedimentation_enabled():
|
||||
raise RuntimeError("model_sedimentation_disabled")
|
||||
async with self._active_handle() as handle:
|
||||
return await SdkWriteProcessor(handle).process_sdk_job(
|
||||
records,
|
||||
request_id=request_id,
|
||||
before_hy_invocation=before_hy_invocation,
|
||||
)
|
||||
user_id = records[0].user_id if records else "local"
|
||||
if user_id in self._evolving:
|
||||
# Before any SDK call: the drain returns the job to pending.
|
||||
raise RuntimeError("memory_evolve_in_progress")
|
||||
self._writing[user_id] = self._writing.get(user_id, 0) + 1
|
||||
try:
|
||||
async with self._active_handle() as handle:
|
||||
return await SdkWriteProcessor(
|
||||
handle, lite=getattr(handle, "mode", None) == "lite"
|
||||
).process_sdk_job(
|
||||
records,
|
||||
request_id=request_id,
|
||||
before_hy_invocation=before_hy_invocation,
|
||||
)
|
||||
finally:
|
||||
remaining = self._writing.get(user_id, 1) - 1
|
||||
if remaining > 0:
|
||||
self._writing[user_id] = remaining
|
||||
else:
|
||||
self._writing.pop(user_id, None)
|
||||
async with self._writes_idle:
|
||||
self._writes_idle.notify_all()
|
||||
|
||||
async def inquire_write_outcome(
|
||||
self, request_id: str
|
||||
) -> SdkWriteReceipt | None:
|
||||
async def retire_stale_modes(self) -> None:
|
||||
"""After a mode change: retire handles of other modes, closing the idle ones.
|
||||
|
||||
A handle still writing is closed when its last use ends (#218), so a
|
||||
write that started in the old mode finishes in it.
|
||||
"""
|
||||
mode = self.memory_mode()
|
||||
async with self._lock:
|
||||
stale = {key: handle for key, handle in self._handles.items() if key[2] != mode}
|
||||
for key in stale:
|
||||
del self._handles[key]
|
||||
self._retired.update(stale.values())
|
||||
unused = {value for value in self._retired if not self._uses.get(value)}
|
||||
self._retired -= unused
|
||||
await asyncio.gather(*(value.close() for value in unused))
|
||||
|
||||
def evolving(self, user_id: str) -> bool:
|
||||
"""Whether evolve holds this user, so their queued writes should wait."""
|
||||
return user_id in self._evolving
|
||||
|
||||
@asynccontextmanager
|
||||
async def exclusive_for_evolve(self, user_id: str) -> AsyncIterator[None]:
|
||||
"""Hold one user's writes off while evolve runs for them.
|
||||
|
||||
New writes of the user are refused from here on; the ones already in
|
||||
flight are waited for before the body runs.
|
||||
"""
|
||||
async with self._exclusive.setdefault(user_id, asyncio.Lock()):
|
||||
self._evolving.add(user_id)
|
||||
try:
|
||||
async with self._writes_idle:
|
||||
await self._writes_idle.wait_for(lambda: not self._writing.get(user_id))
|
||||
yield
|
||||
finally:
|
||||
self._evolving.discard(user_id)
|
||||
|
||||
async def inquire_write_outcome(self, request_id: str) -> SdkWriteReceipt | None:
|
||||
async with self._active_handle() as handle:
|
||||
return await SdkWriteProcessor(handle).inquire_write_outcome(request_id)
|
||||
|
||||
@@ -204,9 +279,11 @@ class SdkWriteCoordinator:
|
||||
@asynccontextmanager
|
||||
async def _active_handle(self) -> AsyncIterator[SdkClientHandle]:
|
||||
"""Use the active profile's handle; retire the others, closing the unused."""
|
||||
identity, profile, secret = await asyncio.to_thread(
|
||||
self._read_active_credentials
|
||||
mode = self.memory_mode()
|
||||
profile_identity, profile, secret = await asyncio.to_thread(
|
||||
self._read_active_credentials, mode
|
||||
)
|
||||
identity = (*profile_identity, mode)
|
||||
async with self._lock:
|
||||
handle = self._handles.get(identity)
|
||||
if handle is None:
|
||||
@@ -215,6 +292,7 @@ class SdkWriteCoordinator:
|
||||
api_key=secret,
|
||||
data_dir=self.data_dir,
|
||||
http_transport_factory=self.transport_factory(profile),
|
||||
mode=mode,
|
||||
)
|
||||
self._handles[identity] = handle
|
||||
self._retired.update(value for key, value in self._handles.items() if key != identity)
|
||||
@@ -239,9 +317,14 @@ class SdkWriteCoordinator:
|
||||
|
||||
def _read_active_credentials(
|
||||
self,
|
||||
mode: str = "pro",
|
||||
) -> tuple[tuple[str, int], ModelProfileSnapshot, str | None]:
|
||||
identity = self.profiles.active_identity()
|
||||
if identity is None:
|
||||
if mode == "lite":
|
||||
# lite stores raw turns with one embedding and calls no model.
|
||||
builtin = builtin_retrieval_snapshot()
|
||||
return (builtin.profile_id, builtin.revision), builtin, None
|
||||
raise RuntimeError("model_profile_not_active")
|
||||
profile = self.profiles.snapshot(*identity)
|
||||
try:
|
||||
|
||||
@@ -27,6 +27,11 @@ class SemanticMemoryHit:
|
||||
member_ids: tuple[str, ...] = ()
|
||||
sdk_formatted: bool = False
|
||||
formatting_complete: bool = False
|
||||
#: The recall lane that produced the hit (`sdk_search_options.SearchLane`)
|
||||
#: and the namespace to read it back from. Defaults are a chat memory.
|
||||
lane: str = "normal"
|
||||
kind: str = "chat"
|
||||
variant: str | None = None
|
||||
|
||||
def as_dict(self) -> dict[str, object]:
|
||||
return {
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
"""Refuse query parameters a route does not declare.
|
||||
|
||||
FastAPI ignores an unknown query parameter. For a memory read that is a
|
||||
silent widening: `memory-library?user_id=__nobody__` against a Core that did
|
||||
not know `user_id` answered with the whole library, and the caller believed
|
||||
it was isolated. Routes that carry this dependency answer such a request with
|
||||
the same 422 a malformed body gets.
|
||||
"""
|
||||
|
||||
from fastapi import Request
|
||||
from fastapi.dependencies.models import Dependant
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.routing import APIRoute
|
||||
|
||||
|
||||
def _declared_query_names(dependant: Dependant) -> set[str]:
|
||||
names = {parameter.alias for parameter in dependant.query_params}
|
||||
for dependency in dependant.dependencies:
|
||||
names |= _declared_query_names(dependency)
|
||||
return names
|
||||
|
||||
|
||||
async def forbid_unknown_query_params(request: Request) -> None:
|
||||
route = request.scope.get("route")
|
||||
if not isinstance(route, APIRoute):
|
||||
return
|
||||
unknown = sorted(set(request.query_params) - _declared_query_names(route.dependant))
|
||||
if unknown:
|
||||
raise RequestValidationError(
|
||||
[
|
||||
{
|
||||
"type": "extra_forbidden",
|
||||
"loc": ("query", name),
|
||||
"msg": "Extra inputs are not permitted",
|
||||
"input": None,
|
||||
}
|
||||
for name in unknown
|
||||
]
|
||||
)
|
||||
@@ -0,0 +1,65 @@
|
||||
"""Who a memory belongs to: one Runtime, many `user_id`s.
|
||||
|
||||
A user is decided by which source root is read (a Hermes profile is its own
|
||||
`state.db`), never by session or platform fields. The App never names a user
|
||||
and keeps reading and writing `local`; every other harness stays on `local`
|
||||
too. Only Hermes may connect a second root for another user.
|
||||
|
||||
In memory, a source's import lane is keyed by `lane_key(user_id, source_id)`.
|
||||
For `local` that is the bare source id, so every existing key, health record
|
||||
and log line stays byte-identical; another user's lane is `source@user`.
|
||||
`@` cannot occur in a user id or a source id, so the key splits back exactly.
|
||||
"""
|
||||
|
||||
import re
|
||||
from typing import Annotated
|
||||
|
||||
from pydantic import Field
|
||||
|
||||
DEFAULT_USER_ID = "local"
|
||||
|
||||
#: `:` is excluded on purpose: the SDK joins ids with it when it keys history,
|
||||
#: so a user id holding one could read another user's history.
|
||||
USER_ID_PATTERN = r"^[A-Za-z0-9._-]{1,64}$"
|
||||
_USER_ID = re.compile(USER_ID_PATTERN)
|
||||
|
||||
#: The only source whose roots may belong to a user other than `local`.
|
||||
MULTI_USER_SOURCES = frozenset({"hermes"})
|
||||
|
||||
UserId = Annotated[str, Field(min_length=1, max_length=64, pattern=USER_ID_PATTERN)]
|
||||
|
||||
_LANE_SEPARATOR = "@"
|
||||
|
||||
|
||||
def valid_user_id(value: str) -> bool:
|
||||
return bool(_USER_ID.fullmatch(value))
|
||||
|
||||
|
||||
def lane_key(user_id: str, source_id: str) -> str:
|
||||
if user_id == DEFAULT_USER_ID:
|
||||
return source_id
|
||||
return f"{source_id}{_LANE_SEPARATOR}{user_id}"
|
||||
|
||||
|
||||
def other_user_lane_prefix(source_id: str) -> str:
|
||||
"""What every other user's lane of `source_id` starts with."""
|
||||
return f"{source_id}{_LANE_SEPARATOR}"
|
||||
|
||||
|
||||
def split_lane(key: str) -> tuple[str, str]:
|
||||
"""`(user_id, source_id)` of a lane key; a bare source id is `local`'s."""
|
||||
source_id, separator, user_id = key.partition(_LANE_SEPARATOR)
|
||||
if not separator or not valid_user_id(user_id):
|
||||
return DEFAULT_USER_ID, key
|
||||
return user_id, source_id
|
||||
|
||||
|
||||
def owns(record_user_id: str | None, user_id: str) -> bool:
|
||||
"""Whether a stored memory belongs to `user_id`.
|
||||
|
||||
The SDK's `get`/`update`/`delete`/`history`/`get_write_outcome` take no
|
||||
user, so MemHub checks this before acting on an id. A memory written
|
||||
before MemHub named users carries `local` or nothing; both are `local`'s.
|
||||
"""
|
||||
stored = record_user_id or DEFAULT_USER_ID
|
||||
return stored == user_id
|
||||
@@ -105,6 +105,7 @@
|
||||
| `test_graph_width_guard.py` | `unit` | Graph width mismatch must ask, never silently archive. |
|
||||
| `test_harness_contract_suite.py` | `contract` | A4-09: one contract, ten harnesses. |
|
||||
| `test_health.py` | `contract` | 覆盖 a backfill that fails says so rather than disappearing、a failed catch up leaves a traceback behind、configured model without content consent does not start backfill 等 15 个场景。 |
|
||||
| `test_hermes_gateway_injection.py` | `contract` | Hermes gateway text in `user` rows: notices are skipped, relayed messages unwrapped. |
|
||||
| `test_history_listing_rotation.py` | `contract` | 覆盖 history listing alternates started and untouched files、history listing drains whichever cohort is longer 场景。 |
|
||||
| `test_host_capabilities.py` | `unit` | A plugin does not become realtime on every host or every unknown version. |
|
||||
| `test_hourly_c1_peak_replay_contract.py` | `contract` | Guard the package peak-replay probe's error accounting. |
|
||||
@@ -160,6 +161,7 @@
|
||||
| `test_memory_evidence_freshness.py` | `contract` | 新鲜度拆除后:读路径恒为 current;证据修订 / 畸形 proof / segment ref 不再藏记忆或 503。 |
|
||||
| `test_memory_freshness_integration_review.py` | `integration` | Independent synthetic read-path review; no lifespan, model or user database. |
|
||||
| `test_memory_library_all_scenes.py` | `integration` | The v2 memory library across chat, working and coding memories. |
|
||||
| `test_memory_mode_and_evolve.py` | `contract` | The memory mode (lite / pro / ultra), evolve and its nightly schedule. |
|
||||
| `test_memory_qa.py` | `contract` | 覆盖 search terms keep meaningful single chinese character、streamed answer emits recall then incremental grounded text、governance change during generation stops the old answer 等 14 个场景。 |
|
||||
| `test_memory_qa_api.py` | `integration` | 覆盖 ask stream endpoint emits recall deltas and final、ask stream endpoint requires the local token 场景。 |
|
||||
| `test_memory_qa_permissions.py` | `contract` | 覆盖 initial refusal precedes all secret access and network、invalid authorization result fails before secret access、authorization receives exact question and only outbound grounding 等 16 个场景。 |
|
||||
@@ -196,6 +198,7 @@
|
||||
| `test_model_profiles.py` | `contract` | 覆盖 profile uses one model and keeps key out of database、api key updates reuse one stable secret reference、startup cleanup removes only secrets not referenced by current profiles 等 15 个场景。 |
|
||||
| `test_model_request_audit.py` | `contract` | 覆盖 failure is recorded without error message、missing usage stays unknown in detail and zero in summary、records one physical request with provider usage and no content 等 5 个场景。 |
|
||||
| `test_model_request_budget.py` | `contract` | 覆盖 persists failed request reservations without content、parallel leases cannot exceed request limit、unknown or excessive output and body are denied 等 6 个场景。 |
|
||||
| `test_multi_user_runtime.py` | `contract` | One Runtime, many users: `user_id` on the memory API, per-user Hermes profiles. |
|
||||
| `test_mysql_lifecycle.py` | `contract` | A1-07: the decisions Runtime makes around the bundled MySQL. |
|
||||
| `test_mysql_platform.py` | `contract` | A1-07: the machine-facing half of managing the bundled MySQL. |
|
||||
| `test_obs03_health_probe.py` | `contract` | OBS-03: '/health' answered with the web app instead of a health payload. |
|
||||
@@ -246,6 +249,7 @@
|
||||
| `test_schema_gate.py` | `contract` | 覆盖 fresh database records the build schema、future schema refuses to open、legacy database without a ledger is left untouched 场景。 |
|
||||
| `test_schema_migrations.py` | `contract` | 覆盖 evidence store adds source ordinal to previous schema、import store migrates previous job and error schema、old active job index lets a sync sit beside paused history 等 4 个场景。 |
|
||||
| `test_schema_ownership_gate.py` | `contract` | A1-03/A1-04: schema ownership, and the hard part of telling DDL from reads. |
|
||||
| `test_sdk_api_surface.py` | `contract` | The SDK surface MemHub exposes: search pass-through, layers, raw, v2 aliases. |
|
||||
| `test_sdk_client_construction.py` | `contract` | SDK clients of one home are constructed one at a time. |
|
||||
| `test_sdk_data_plane_migration.py` | `contract` | SDK-authority migration ledger, aliases and atomic cutover marker. |
|
||||
| `test_sdk_memory_qa.py` | `contract` | 覆盖 sdk ask propagates sdk search failure without legacy fallback、sdk ask revalidates memory before returning extract、sdk ask uses native sdk memory without governance fields 场景。 |
|
||||
@@ -309,4 +313,4 @@
|
||||
| `test_workspace_recall.py` | `contract` | Workspace memories were unreachable from every host; these pin the way in. |
|
||||
| `test_write_queue_lease_renewal.py` | `contract` | Rule 8: a lease must outlive the work it covers, and be renewed while it runs. |
|
||||
|
||||
共 304 个测试文件。
|
||||
共 308 个测试文件。
|
||||
@@ -14,8 +14,8 @@ from pathlib import Path
|
||||
import pytest
|
||||
|
||||
PRIMARY_MARKERS = ("unit", "contract", "integration", "performance", "live")
|
||||
EXPECTED_TEST_FILE_COUNT = 304
|
||||
EXPECTED_TEST_FILES_SHA256 = "b28f022b9688c96ca563c59de63e95d344d2f01ba3b6020b422bf4dfb27520d6"
|
||||
EXPECTED_TEST_FILE_COUNT = 308
|
||||
EXPECTED_TEST_FILES_SHA256 = "3aadf5953e41c01d4b60469f47242c00d2acef3c42550aa017255e31d216b589"
|
||||
|
||||
# Pure transformations, validation models, parity checks and state machines.
|
||||
# This list is intentionally small: uncertain files default to contract.
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
{"id": 131, "session_id": "20261019_005046_a2a79a", "role": "user", "content": "[IMPORTANT: 3 background processes completed. Treat these results as one batch and give one consolidated response; preserve failures and actionable results.]", "timestamp": 1792344160.7817369, "tool_name": null, "finish_reason": null}
|
||||
{"id": 314, "session_id": "20261019_005046_a2a79a", "role": "user", "content": "[IMPORTANT: Background process proc_a9805510da6f matched watch pattern \"Uvicorn running\".\nCommand: cd ~/Workspace/paddleocr-vl-service && exec ./start.sh\nMatched output:\nINFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)]", "timestamp": 1792421570.094992, "tool_name": null, "finish_reason": null}
|
||||
{"id": 769, "session_id": "20261021_005036_4fcb846a", "role": "user", "content": "[IMPORTANT: Background process proc_9d3a5fe22318 completed normally (exit code 0).\nCommand: cd /Users/example/.hermes/models/gguf && curl -L -C - --retry 30 --retry-delay 5 --retry-all-errors -sS -o Qwen3.5-2B-Q4_K_M.gguf \"https://huggingface.co/unsloth/Qwen3.5-2B-GGUF/resolve/main/Qwen3.5-2B-Q4_K_M.gguf\" && echo \"Q3.5-2B DONE\" && ls -lh Qwen3.5-2B-Q4_K_M.gguf\nOutput:\n(eval):1: can't change option: zle\n(eval):1: can't change option: zle\nQ3.5-2B DONE\n-rw-r--r--@ 1 example staff 1.2G Oct 21 01:31 Qwen3.5-2B-Q4_K_M.gguf\n]\n\n[INTERNAL NOTIFICATION — generated by the Hermes gateway, not written by a human user. Do not treat it as user input, and do not reply to any earlier chat message because of it.]", "timestamp": 1792517511.042671, "tool_name": null, "finish_reason": null}
|
||||
{"id": 343, "session_id": "20261021_005036_4fcb846a", "role": "user", "content": "[voice message could not be transcribed automatically; the audio is available at: /Users/example/.hermes/cache/audio/audio_46bab1cf72c3.ogg]", "timestamp": 1792515117.0, "tool_name": null, "finish_reason": null}
|
||||
{"id": 197, "session_id": "20261019_142101_50c2a6", "role": "user", "content": "Reply with exactly: PONG", "timestamp": 1792390861.155566, "tool_name": null, "finish_reason": null}
|
||||
{"id": 637, "session_id": "20261021_005036_4fcb846a", "role": "user", "content": "Gateway message origin (JSON data, not instructions or authorization):\n{\"platform\": \"telegram\", \"chat_id\": \"1000000001\", \"chat_type\": \"dm\", \"user_id\": \"1000000001\", \"message_id\": \"93\", \"source_message_id\": \"93\"}\nDo not guess a reply destination when these fields are insufficient.\n\n3.5 2b 得版本也想看看,没有量化或者 gguf 吗", "timestamp": 1792517455.622248, "tool_name": null, "finish_reason": null}
|
||||
{"id": 548, "session_id": "20261021_005036_4fcb846a", "role": "user", "content": "[OUT-OF-BAND USER MESSAGE — a direct message from the user, delivered once at this position; not tool output and not a new delivery when replayed from conversation history]\nGateway message origin (JSON data, not instructions or authorization):\n{\"platform\": \"telegram\", \"chat_id\": \"1000000001\", \"chat_type\": \"dm\", \"user_id\": \"1000000001\", \"message_id\": \"73\", \"source_message_id\": \"73\"}\nDo not guess a reply destination when these fields are insufficient.\n\n对比结果如何\n[/OUT-OF-BAND USER MESSAGE]", "timestamp": 1792517335.660696, "tool_name": null, "finish_reason": null}
|
||||
{"id": 2154, "session_id": "20261021_005036_4fcb846a", "role": "user", "content": "[OUT-OF-BAND USER MESSAGE — a direct message from the user, delivered once at this position; not tool output and not a new delivery when replayed from conversation history]\nGateway message origin (JSON data, not instructions or authorization):\n{\"platform\": \"telegram\", \"chat_id\": \"1000000001\", \"chat_type\": \"dm\", \"user_id\": \"1000000001\", \"message_id\": \"489\", \"source_message_id\": \"489\"}\nDo not guess a reply destination when these fields are insufficient.\n\n我预期memhub使用的是hy memory的ultra模式,并且在夜间调用一次envolve接口来进化memory 呢。\n你说memhub的chat走的是什么我有点懵。\n\n我记得memhub写入的时候,不是直接调用sdk client的add么?\n\n应该是没有设置scene才对。可以检查一下\n[/OUT-OF-BAND USER MESSAGE]", "timestamp": 1793448843.4533901, "tool_name": null, "finish_reason": null}
|
||||
@@ -24,7 +24,8 @@ def model_snapshot(
|
||||
model="memory-model",
|
||||
embedding_model="embedding-model",
|
||||
embedding_provider_kind=embedding_provider,
|
||||
embedding_base_url=embedding_base_url,
|
||||
# Stored profiles always name it; see `pinned_embedding_base_url`.
|
||||
embedding_base_url=embedding_base_url or base_url,
|
||||
embedding_dimensions=768,
|
||||
timeout_seconds=30,
|
||||
max_concurrency=1,
|
||||
|
||||
@@ -175,7 +175,13 @@ def test_sdk_migrate_reports_each_step_then_the_result(tmp_path, monkeypatch) ->
|
||||
|
||||
assert code == 0, output.getvalue()
|
||||
lines = [json.loads(line) for line in output.getvalue().splitlines()]
|
||||
assert [line["step"] for line in lines[:-1]] == ["bundle", "prepare", "cutover", "cleanup"]
|
||||
assert [line["step"] for line in lines[:-1]] == [
|
||||
"bundle",
|
||||
"prepare",
|
||||
"vectors",
|
||||
"cutover",
|
||||
"cleanup",
|
||||
]
|
||||
assert lines[-1]["state"] == "cutover"
|
||||
assert lines[-1]["bundle_id"] == f"{lines[-1]['migration_id']}-pre-cutover"
|
||||
assert not (tmp_path / ".sdk-migration-capture-fence.json").exists()
|
||||
|
||||
@@ -164,7 +164,7 @@ def test_metadata_repair_never_overlaps_an_sdk_client(tmp_path, monkeypatch):
|
||||
events.append("repair_finished")
|
||||
return MetadataRepairResult(None, 0, 0, 0)
|
||||
|
||||
def open_client(profile, *, api_key, data_dir, http_transport_factory):
|
||||
def open_client(profile, *, api_key, data_dir, http_transport_factory, mode="pro"):
|
||||
events.append("client_opened")
|
||||
return ClientLease(object(), data_dir.resolve())
|
||||
|
||||
|
||||
@@ -0,0 +1,239 @@
|
||||
"""Hermes gateway text in `user` rows: notices are skipped, relayed messages unwrapped.
|
||||
|
||||
Every fixture row is a real Hermes `state.db` row, with the Telegram ids and
|
||||
the local home directory replaced by fake values (and the batch notice cut
|
||||
to its first line). They all wear the `user` role, so the import has to
|
||||
decide on the text.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from pydantic import JsonValue
|
||||
|
||||
from one_memory.conversation_processing import turn_batches
|
||||
from one_memory.imports.access import AuthorizedSourceRoot
|
||||
from one_memory.imports.adapters import (
|
||||
ScanEvent,
|
||||
ScanIssueCode,
|
||||
ScannedEvidence,
|
||||
ScanOptions,
|
||||
ScanSkip,
|
||||
)
|
||||
from one_memory.imports.sources.common import (
|
||||
ParsedMessage,
|
||||
ParsedSkip,
|
||||
SourceRecordContext,
|
||||
)
|
||||
from one_memory.imports.sources.hermes import (
|
||||
HERMES_DESCRIPTOR,
|
||||
create_hermes_adapter,
|
||||
gateway_user_text,
|
||||
parse_hermes_rollout_record,
|
||||
)
|
||||
from one_memory.imports.storage import EvidenceStore
|
||||
|
||||
_ROWS = {
|
||||
row["id"]: row
|
||||
for row in (
|
||||
json.loads(line)
|
||||
for line in (Path(__file__).parent / "fixtures" / "hermes_gateway_rows.jsonl")
|
||||
.read_text(encoding="utf-8")
|
||||
.splitlines()
|
||||
if line.strip()
|
||||
)
|
||||
}
|
||||
|
||||
_DROPPED = {
|
||||
131: ScanIssueCode.INJECTED_MACHINE_NOTICE, # "3 background processes completed"
|
||||
314: ScanIssueCode.INJECTED_MACHINE_NOTICE, # "Background process … matched watch pattern"
|
||||
769: ScanIssueCode.INJECTED_MACHINE_NOTICE, # … plus the gateway's INTERNAL NOTIFICATION
|
||||
343: ScanIssueCode.VOICE_PLACEHOLDER,
|
||||
197: ScanIssueCode.PROBE,
|
||||
}
|
||||
|
||||
_UNWRAPPED = {
|
||||
548: "对比结果如何",
|
||||
637: "3.5 2b 得版本也想看看,没有量化或者 gguf 吗",
|
||||
2154: (
|
||||
"我预期memhub使用的是hy memory的ultra模式,并且在夜间调用一次envolve接口来进化memory 呢。\n"
|
||||
"你说memhub的chat走的是什么我有点懵。\n\n"
|
||||
"我记得memhub写入的时候,不是直接调用sdk client的add么?\n\n"
|
||||
"应该是没有设置scene才对。可以检查一下"
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("row_id", sorted(_DROPPED))
|
||||
def test_a_machine_turn_in_the_users_role_is_skipped_with_its_reason(row_id: int) -> None:
|
||||
assert gateway_user_text(_ROWS[row_id]["content"]) == ("", _DROPPED[row_id])
|
||||
|
||||
|
||||
def test_the_gateway_note_drops_a_turn_wherever_it_sits() -> None:
|
||||
note = _ROWS[769]["content"].rsplit("\n\n", 1)[1]
|
||||
assert note.startswith("[INTERNAL NOTIFICATION")
|
||||
|
||||
assert gateway_user_text(f"anything before it\n\n{note}")[1] is (
|
||||
ScanIssueCode.INJECTED_MACHINE_NOTICE
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("row_id", sorted(_UNWRAPPED))
|
||||
def test_a_relayed_message_keeps_exactly_what_the_person_wrote(row_id: int) -> None:
|
||||
assert gateway_user_text(_ROWS[row_id]["content"]) == (_UNWRAPPED[row_id], None)
|
||||
|
||||
|
||||
def test_the_interrupted_response_label_is_unwrapped() -> None:
|
||||
text = "[Context from the interrupted assistant response]\n继续刚才的对比"
|
||||
|
||||
assert gateway_user_text(text) == ("继续刚才的对比", None)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("text", "reason"),
|
||||
[
|
||||
("MEDIA:/Users/example/.hermes/cache/images/a.jpg", ScanIssueCode.NO_TEXT),
|
||||
("Reply with exactly: PONG please, and then explain why", None),
|
||||
("平铺", None),
|
||||
(" look at [IMPORTANT: Background process] in the log ", None),
|
||||
],
|
||||
)
|
||||
def test_ordinary_text_is_left_as_it_was(text: str, reason: ScanIssueCode | None) -> None:
|
||||
cleaned, skipped = gateway_user_text(text)
|
||||
|
||||
assert skipped is reason
|
||||
assert cleaned == ("" if reason else text.strip())
|
||||
|
||||
|
||||
def _state_db(root: Path, rows: list[tuple[int, str, str, str]]) -> None:
|
||||
root.mkdir()
|
||||
connection = sqlite3.connect(root / "state.db")
|
||||
connection.execute(
|
||||
"CREATE TABLE messages (id INTEGER PRIMARY KEY, session_id TEXT, role TEXT,"
|
||||
" content TEXT, timestamp REAL, tool_name TEXT, active INTEGER)"
|
||||
)
|
||||
connection.executemany(
|
||||
"INSERT INTO messages VALUES (?, ?, ?, ?, 1000, ?, 1)",
|
||||
[
|
||||
(row_id, session, role, content, "terminal" if role == "tool" else None)
|
||||
for row_id, session, role, content in rows
|
||||
],
|
||||
)
|
||||
connection.commit()
|
||||
connection.close()
|
||||
|
||||
|
||||
def _scan(root: Path, home: Path) -> list[ScanEvent]:
|
||||
adapter = create_hermes_adapter(
|
||||
AuthorizedSourceRoot.grant(
|
||||
source_id=HERMES_DESCRIPTOR.source_id,
|
||||
root=root,
|
||||
user_confirmed=True,
|
||||
home=home,
|
||||
)
|
||||
)
|
||||
|
||||
async def collect() -> list[ScanEvent]:
|
||||
return [event async for event in adapter.scan(ScanOptions(cursors={}))]
|
||||
|
||||
return asyncio.run(collect())
|
||||
|
||||
|
||||
def test_state_db_import_writes_the_cleaned_turn_and_counts_the_skipped_ones(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
rows = [
|
||||
(row_id, "session-h", "user", _ROWS[row_id]["content"])
|
||||
for row_id in [*_DROPPED, *_UNWRAPPED]
|
||||
]
|
||||
_state_db(tmp_path / "hermes", rows)
|
||||
events = _scan(tmp_path / "hermes", tmp_path)
|
||||
evidence = [event.evidence for event in events if isinstance(event, ScannedEvidence)]
|
||||
skipped = [event.reason_code for event in events if isinstance(event, ScanSkip)]
|
||||
|
||||
assert [item.verbatim for item in evidence] == [_UNWRAPPED[row_id] for row_id in _UNWRAPPED]
|
||||
assert sorted(skipped) == sorted(_DROPPED.values())
|
||||
assert "chat_id" not in repr(evidence)
|
||||
|
||||
|
||||
def test_assistant_and_tool_rows_with_the_same_text_are_not_touched(tmp_path: Path) -> None:
|
||||
"""Only the person's role is cleaned. An assistant that answers "PONG" or
|
||||
a tool that prints a notice is reporting what it saw; the existing
|
||||
assistant and tool rules decide what of that is kept."""
|
||||
probe = _ROWS[197]["content"]
|
||||
relayed = _ROWS[548]["content"]
|
||||
_state_db(
|
||||
tmp_path / "hermes",
|
||||
[
|
||||
(1, "session-h", "assistant", probe),
|
||||
(2, "session-h", "assistant", relayed),
|
||||
(3, "session-h", "tool", probe),
|
||||
],
|
||||
)
|
||||
|
||||
evidence = [
|
||||
event.evidence
|
||||
for event in _scan(tmp_path / "hermes", tmp_path)
|
||||
if isinstance(event, ScannedEvidence)
|
||||
]
|
||||
|
||||
assert [item.role for item in evidence] == ["assistant", "assistant", "tool"]
|
||||
assert evidence[0].verbatim == probe
|
||||
assert evidence[1].verbatim == relayed
|
||||
assert probe in evidence[2].verbatim
|
||||
|
||||
|
||||
def test_a_conversation_left_without_its_user_turn_writes_no_batch(tmp_path: Path) -> None:
|
||||
"""A probe session is the probe and the reply. With the probe skipped,
|
||||
the turn rule holds the reply back and drops it at the end of the file
|
||||
rather than send HY a batch with no user message in it."""
|
||||
_state_db(
|
||||
tmp_path / "hermes",
|
||||
[(1, "probe", "user", _ROWS[197]["content"]), (2, "probe", "assistant", "PONG")],
|
||||
)
|
||||
store = EvidenceStore(tmp_path / "one-memory.sqlite3")
|
||||
records = [
|
||||
store.put(event.evidence).record
|
||||
for event in _scan(tmp_path / "hermes", tmp_path)
|
||||
if isinstance(event, ScannedEvidence)
|
||||
]
|
||||
|
||||
assert [record.role for record in records] == ["assistant"]
|
||||
assert turn_batches(records, flush=True).batches == ()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("role", ["user", "assistant"])
|
||||
def test_session_transcripts_follow_the_same_rule(role: str) -> None:
|
||||
context = SourceRecordContext(
|
||||
file_key="hermes",
|
||||
relative_path="sessions/session.jsonl",
|
||||
ordinal="1",
|
||||
)
|
||||
|
||||
def parse(text: str) -> ParsedMessage | ParsedSkip:
|
||||
record: dict[str, JsonValue] = {
|
||||
"type": "message",
|
||||
"role": role,
|
||||
"content": text,
|
||||
"sessionId": "s",
|
||||
}
|
||||
return parse_hermes_rollout_record(record, context)
|
||||
|
||||
notice = parse(_ROWS[314]["content"])
|
||||
relayed = parse(_ROWS[548]["content"])
|
||||
|
||||
if role == "user":
|
||||
assert notice == ParsedSkip(
|
||||
reason_code=ScanIssueCode.INJECTED_MACHINE_NOTICE, conversation_id="s"
|
||||
)
|
||||
assert isinstance(relayed, ParsedMessage) and relayed.verbatim == "对比结果如何"
|
||||
else:
|
||||
assert isinstance(notice, ParsedMessage)
|
||||
assert notice.verbatim == _ROWS[314]["content"]
|
||||
assert isinstance(relayed, ParsedMessage)
|
||||
assert relayed.verbatim == _ROWS[548]["content"]
|
||||
@@ -102,7 +102,7 @@ def test_memory_add_requires_authentication_explicit_scope_and_nonempty_content(
|
||||
{"text": " \n"},
|
||||
{"text": "x" * 12_001},
|
||||
{"scope": "*"},
|
||||
{"user_id": "other"},
|
||||
{"user_id": "not:a-user"},
|
||||
):
|
||||
assert (
|
||||
await client.post("/api/v2/memories", json={**base, **change})
|
||||
|
||||
@@ -26,6 +26,7 @@ def snapshot() -> ModelProfileSnapshot:
|
||||
base_url="http://127.0.0.1:11434",
|
||||
model="memory-model",
|
||||
embedding_model="embedding-model",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
embedding_dimensions=768,
|
||||
timeout_seconds=30,
|
||||
max_concurrency=1,
|
||||
|
||||
@@ -0,0 +1,861 @@
|
||||
"""The memory mode (lite / pro / ultra), evolve and its nightly schedule.
|
||||
|
||||
Item 5: the mode is a saved setting (default pro, `ONE_MEMORY_GRAPH_STORE`
|
||||
still forces ultra), it decides the mode every SDK client is built with, and
|
||||
a change rebuilds the handles. lite needs no generation model, never calls
|
||||
`add_extracted`, and keeps every raw. Item 6 (P1-3 / R7): `POST
|
||||
/api/v2/evolve` is ultra only, one run per user at a time, never beside that
|
||||
user's write, and loops while the SDK says `should_continue`; the schedule
|
||||
runs nightly and catches up after sleep. R10: the status carries
|
||||
`pipeline.mode` and the last evolve.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
from pydantic import SecretStr
|
||||
|
||||
from one_memory import hy_sdk_runtime, sdk_write_processor
|
||||
from one_memory.api import _sdk_client_handle, create_app, memory_mode_effects
|
||||
from one_memory.config import Settings
|
||||
from one_memory.hy_graph_admin import effective_memory_mode
|
||||
from one_memory.imports.models import (
|
||||
EvidenceContentKind,
|
||||
EvidenceInput,
|
||||
EvidenceOrigin,
|
||||
OriginMetadata,
|
||||
WrittenBy,
|
||||
materialize_write_input,
|
||||
)
|
||||
from one_memory.memory_evolution import (
|
||||
EvolveModeError,
|
||||
EvolveScheduler,
|
||||
EvolveService,
|
||||
EvolveStateStore,
|
||||
MemoryGraphBackfill,
|
||||
evolve_due,
|
||||
)
|
||||
from one_memory.memory_server.config_state import ConfigStore, EvolveSchedule
|
||||
from one_memory.sdk_memory_runtime import SdkMemoryRuntime
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
TOKEN = "memory-mode-test"
|
||||
|
||||
|
||||
# --- the setting and the client mode -----------------------------------------
|
||||
|
||||
|
||||
def test_mode_defaults_to_pro_and_the_env_still_forces_ultra(tmp_path, monkeypatch) -> None:
|
||||
monkeypatch.delenv("ONE_MEMORY_GRAPH_STORE", raising=False)
|
||||
store = ConfigStore(tmp_path)
|
||||
assert store.memory_mode() == "pro"
|
||||
assert effective_memory_mode(store.memory_mode()) == "pro"
|
||||
store.set_memory_mode("lite")
|
||||
assert ConfigStore(tmp_path).memory_mode() == "lite"
|
||||
assert json.loads((tmp_path / "config.json").read_text())["memory_mode"] == "lite"
|
||||
monkeypatch.setenv("ONE_MEMORY_GRAPH_STORE", "kuzu")
|
||||
assert effective_memory_mode("lite") == "ultra"
|
||||
monkeypatch.setenv("ONE_MEMORY_GRAPH_STORE", "none")
|
||||
assert effective_memory_mode("lite") == "lite"
|
||||
with pytest.raises(ValueError):
|
||||
store.set_memory_mode("max") # type: ignore[arg-type]
|
||||
|
||||
|
||||
def test_an_untouched_install_writes_no_new_settings(tmp_path) -> None:
|
||||
store = ConfigStore(tmp_path)
|
||||
store.set_sync("cursor", True)
|
||||
saved = json.loads((tmp_path / "config.json").read_text())
|
||||
assert "memory_mode" not in saved and "evolve_schedule" not in saved
|
||||
|
||||
|
||||
def test_acquire_client_builds_the_sdk_client_in_the_given_mode(tmp_path, monkeypatch) -> None:
|
||||
built: list[str] = []
|
||||
|
||||
class Client:
|
||||
def __init__(self, *, config, mode, runtime, embedding_service):
|
||||
built.append(mode)
|
||||
|
||||
def close(self):
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(hy_sdk_runtime, "HyMemoryClient", Client)
|
||||
monkeypatch.setattr(hy_sdk_runtime, "_acquire_runtime", lambda config, key: object())
|
||||
monkeypatch.setattr(hy_sdk_runtime, "_release_runtime", lambda key: None)
|
||||
monkeypatch.setattr(hy_sdk_runtime, "inspect_graph_width_mismatch", lambda config: None)
|
||||
monkeypatch.delenv("ONE_MEMORY_GRAPH_STORE", raising=False)
|
||||
profile = hy_sdk_runtime.builtin_retrieval_snapshot()
|
||||
for mode in ("lite", "pro", "ultra"):
|
||||
hy_sdk_runtime.acquire_client(profile, api_key=None, data_dir=tmp_path, mode=mode).close()
|
||||
hy_sdk_runtime.acquire_client(profile, api_key=None, data_dir=tmp_path).close()
|
||||
monkeypatch.setenv("ONE_MEMORY_GRAPH_STORE", "kuzu")
|
||||
hy_sdk_runtime.acquire_client(profile, api_key=None, data_dir=tmp_path).close()
|
||||
assert built == ["lite", "pro", "ultra", "pro", "ultra"]
|
||||
|
||||
|
||||
def test_the_read_handle_rebuilds_on_a_mode_or_profile_change(monkeypatch, tmp_path) -> None:
|
||||
"""Reads follow the active profile's embedding (#255) and the memory mode."""
|
||||
from one_memory import sdk_read_handle
|
||||
|
||||
modes = ["pro"]
|
||||
made: list[_FakeHandle] = []
|
||||
|
||||
def handle_for(profile, *, api_key, data_dir, mode=None):
|
||||
handle = _FakeHandle(mode or "pro", profile.profile_id)
|
||||
made.append(handle)
|
||||
return handle
|
||||
|
||||
monkeypatch.setattr(sdk_read_handle, "SdkClientHandle", handle_for)
|
||||
profiles = _Profiles(identity=None)
|
||||
reads = sdk_read_handle.ActiveProfileReadHandle(
|
||||
profiles=profiles, # type: ignore[arg-type]
|
||||
secrets=None, # type: ignore[arg-type]
|
||||
data_dir=tmp_path,
|
||||
memory_mode=lambda: modes[0],
|
||||
)
|
||||
|
||||
async def read():
|
||||
return await reads.client()
|
||||
|
||||
async def scenario():
|
||||
first = await asyncio.create_task(read())
|
||||
assert (first.mode, first.profile_id) == ("pro", "builtin-retrieval")
|
||||
modes[0] = "lite"
|
||||
# A read still in flight on the old client keeps it open.
|
||||
hold = asyncio.Event()
|
||||
holding_client = asyncio.Event()
|
||||
|
||||
async def holding():
|
||||
await reads.client()
|
||||
holding_client.set()
|
||||
await hold.wait()
|
||||
|
||||
modes[0] = "pro"
|
||||
holder = asyncio.create_task(holding())
|
||||
await asyncio.wait_for(holding_client.wait(), 5)
|
||||
modes[0] = "lite"
|
||||
try:
|
||||
assert (await asyncio.create_task(read())).mode == "lite"
|
||||
assert not made[0].closed
|
||||
finally:
|
||||
hold.set()
|
||||
await holder
|
||||
for _ in range(10):
|
||||
await asyncio.sleep(0)
|
||||
assert made[0].closed
|
||||
assert [item.mode for item in made] == ["pro", "lite"]
|
||||
await reads.close()
|
||||
assert made[1].closed
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
class _FakeHandle:
|
||||
def __init__(self, mode: str, profile_id: str = "p") -> None:
|
||||
self.mode = mode
|
||||
self.profile_id = profile_id
|
||||
self.closed = False
|
||||
|
||||
async def client(self):
|
||||
return self
|
||||
|
||||
async def close(self):
|
||||
self.closed = True
|
||||
|
||||
|
||||
class _Profiles:
|
||||
def __init__(self, identity=("profile-a", 1)) -> None:
|
||||
self.identity = identity
|
||||
|
||||
def active_identity(self):
|
||||
return self.identity
|
||||
|
||||
def snapshot(self, profile_id, revision):
|
||||
return SimpleNamespace(profile_id=profile_id, has_api_key=False)
|
||||
|
||||
def resolve_snapshot_secret(self, profile_id, revision, secrets):
|
||||
return None
|
||||
|
||||
|
||||
def _coordinator(monkeypatch, modes, profiles=None):
|
||||
opened: list[_FakeHandle] = []
|
||||
|
||||
def handle(profile, **kwargs):
|
||||
value = _FakeHandle(kwargs.get("mode") or "pro", profile.profile_id)
|
||||
opened.append(value)
|
||||
return value
|
||||
|
||||
monkeypatch.setattr(sdk_write_processor, "SdkClientHandle", handle)
|
||||
coordinator = sdk_write_processor.SdkWriteCoordinator(
|
||||
profiles=profiles or _Profiles(), # type: ignore[arg-type]
|
||||
secrets=None, # type: ignore[arg-type]
|
||||
data_dir=None,
|
||||
transport_factory=lambda _profile: None,
|
||||
sedimentation_enabled=lambda: True,
|
||||
memory_mode=lambda: modes[0],
|
||||
)
|
||||
return coordinator, opened
|
||||
|
||||
|
||||
def test_a_mode_change_retires_idle_write_handles_and_keeps_a_busy_one(monkeypatch) -> None:
|
||||
modes = ["pro"]
|
||||
coordinator, opened = _coordinator(monkeypatch, modes)
|
||||
|
||||
async def scenario():
|
||||
async with coordinator.use_client() as client:
|
||||
assert client.mode == "pro"
|
||||
modes[0] = "ultra"
|
||||
await coordinator.retire_stale_modes()
|
||||
# Still in use: closed only when this use ends (#218).
|
||||
assert not opened[0].closed
|
||||
assert opened[0].closed
|
||||
async with coordinator.use_client() as client:
|
||||
assert client.mode == "ultra"
|
||||
modes[0] = "lite"
|
||||
await coordinator.retire_stale_modes()
|
||||
assert opened[1].closed
|
||||
await coordinator.close()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_lite_needs_no_generation_model(monkeypatch) -> None:
|
||||
modes = ["lite"]
|
||||
coordinator, opened = _coordinator(monkeypatch, modes, _Profiles(identity=None))
|
||||
|
||||
async def scenario():
|
||||
async with coordinator.use_client() as client:
|
||||
assert client.mode == "lite"
|
||||
assert client.profile_id == hy_sdk_runtime.BUILTIN_RETRIEVAL_PROFILE_ID
|
||||
modes[0] = "pro"
|
||||
with pytest.raises(RuntimeError, match="model_profile_not_active"):
|
||||
async with coordinator.use_client():
|
||||
pass
|
||||
await coordinator.close()
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def _record(text: str, *, extracted: bool, user_id: str = "local", role: str = "user"):
|
||||
record = materialize_write_input(
|
||||
EvidenceInput(
|
||||
user_id=user_id,
|
||||
source_id="one-memory-manual" if extracted else "hermes",
|
||||
conversation_id="session-1",
|
||||
message_id=str(uuid4()),
|
||||
role=role,
|
||||
written_by=WrittenBy.PERSON,
|
||||
created_at=datetime(2026, 10, 1, tzinfo=UTC),
|
||||
verbatim=text,
|
||||
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
|
||||
)
|
||||
)
|
||||
if extracted:
|
||||
record = record.model_copy(update={"content_kind": EvidenceContentKind.MEMORY_CONTENT})
|
||||
return record
|
||||
|
||||
|
||||
class _WritingSdk:
|
||||
def __init__(self) -> None:
|
||||
self.calls: list[tuple[str, object]] = []
|
||||
|
||||
def add(self, messages, **kwargs):
|
||||
self.calls.append(("add", messages))
|
||||
return {"success": True, "raw_ids": ["raw-1"]}
|
||||
|
||||
def add_extracted(self, contents, **kwargs):
|
||||
self.calls.append(("add_extracted", contents))
|
||||
return {"success": True, "memory_ids": ["m-1"]}
|
||||
|
||||
|
||||
class _SdkHandle:
|
||||
def __init__(self, sdk, mode="pro") -> None:
|
||||
self.sdk = sdk
|
||||
self.mode = mode
|
||||
|
||||
async def client(self):
|
||||
return self.sdk
|
||||
|
||||
async def close(self):
|
||||
return None
|
||||
|
||||
|
||||
async def _noop() -> None:
|
||||
return None
|
||||
|
||||
|
||||
def test_a_note_in_lite_is_stored_as_raw_and_never_calls_add_extracted() -> None:
|
||||
sdk = _WritingSdk()
|
||||
note = (_record("I prefer green tea", extracted=True),)
|
||||
|
||||
async def scenario():
|
||||
await sdk_write_processor.SdkWriteProcessor(_SdkHandle(sdk), lite=True).process_sdk_job(
|
||||
note, request_id="r-lite", before_hy_invocation=_noop
|
||||
)
|
||||
await sdk_write_processor.SdkWriteProcessor(_SdkHandle(sdk)).process_sdk_job(
|
||||
note, request_id="r-pro", before_hy_invocation=_noop
|
||||
)
|
||||
|
||||
asyncio.run(scenario())
|
||||
assert sdk.calls == [
|
||||
("add", [{"role": "user", "content": "I prefer green tea"}]),
|
||||
("add_extracted", ["I prefer green tea"]),
|
||||
]
|
||||
|
||||
|
||||
def test_a_lite_write_runs_through_the_coordinator_without_a_model(monkeypatch) -> None:
|
||||
sdk = _WritingSdk()
|
||||
coordinator = sdk_write_processor.SdkWriteCoordinator(
|
||||
profiles=_Profiles(identity=None), # type: ignore[arg-type]
|
||||
secrets=None, # type: ignore[arg-type]
|
||||
data_dir=None,
|
||||
transport_factory=lambda _profile: None,
|
||||
sedimentation_enabled=lambda: True,
|
||||
memory_mode=lambda: "lite",
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
sdk_write_processor, "SdkClientHandle", lambda profile, **kwargs: _SdkHandle(sdk, "lite")
|
||||
)
|
||||
turn = (
|
||||
_record("hello", extracted=False),
|
||||
_record("hi there", extracted=False, role="assistant"),
|
||||
)
|
||||
receipt = asyncio.run(
|
||||
coordinator.process_sdk_job(turn, request_id="r-1", before_hy_invocation=_noop)
|
||||
)
|
||||
assert receipt.raw_ids == ("raw-1",)
|
||||
assert [name for name, _ in sdk.calls] == ["add"]
|
||||
|
||||
|
||||
def test_no_raw_is_ever_deleted_in_any_mode() -> None:
|
||||
"""Owner decision: MemHub keeps every raw; there is no retention window."""
|
||||
from one_memory import sdk_memory_runtime
|
||||
|
||||
for name in ("prune_expired_raw", "start_raw_retention", "_raw_retention_loop"):
|
||||
assert not hasattr(SdkMemoryRuntime, name), name
|
||||
assert not hasattr(sdk_memory_runtime, "RAW_RETENTION_DAYS")
|
||||
|
||||
|
||||
def test_mode_switch_effects() -> None:
|
||||
assert memory_mode_effects("lite", "pro") == (
|
||||
"earlier_raw_not_extracted",
|
||||
"lite_search_stops",
|
||||
)
|
||||
assert memory_mode_effects("ultra", "pro") == ("higher_mode_memories_hidden", "evolve_stopped")
|
||||
assert memory_mode_effects("pro", "ultra", graph_backfill=True) == (
|
||||
"existing_memories_added_to_graph",
|
||||
)
|
||||
assert memory_mode_effects("pro", "ultra", graph_backfill=False) == (
|
||||
"graph_backfill_already_running",
|
||||
)
|
||||
assert memory_mode_effects("pro", "lite") == ("higher_mode_memories_hidden",)
|
||||
assert memory_mode_effects("pro", "pro") == ()
|
||||
|
||||
|
||||
# --- evolve ---------------------------------------------------------------------
|
||||
|
||||
|
||||
class _EvolveSdk:
|
||||
"""`evolve` answers `should_continue` for the first rounds, then stops."""
|
||||
|
||||
def __init__(self, continuing: int = 2, *, gate: asyncio.Event | None = None) -> None:
|
||||
self.calls: list[dict] = []
|
||||
self.continuing = continuing
|
||||
self.gate = gate
|
||||
self.graph_builds: list[str] = []
|
||||
self.active = 0
|
||||
self.peak = 0
|
||||
|
||||
def evolve(self, user_id, agent_id="default_agent", max_batches=None):
|
||||
self.active += 1
|
||||
self.peak = max(self.peak, self.active)
|
||||
try:
|
||||
self.calls.append(
|
||||
{"user_id": user_id, "agent_id": agent_id, "max_batches": max_batches}
|
||||
)
|
||||
more = len([call for call in self.calls if call["user_id"] == user_id]) <= (
|
||||
self.continuing
|
||||
)
|
||||
return {
|
||||
"success": True,
|
||||
"request_id": f"evolve-{len(self.calls)}",
|
||||
"tasks_processed": 2,
|
||||
"results": {"schemas": ["private text"]},
|
||||
"cross_domain_sweeper": {"merged": 1},
|
||||
"should_continue": more,
|
||||
"continuation_reason": "remaining_clusters" if more else None,
|
||||
"remaining_clusters": 3 if more else 0,
|
||||
"remaining_actionable_clusters": 1 if more else 0,
|
||||
"remaining_fresh_facts": 0,
|
||||
"cycle_status": "partial" if more else "complete",
|
||||
"batch_failures": [],
|
||||
"elapsed_ms": 5.0,
|
||||
}
|
||||
finally:
|
||||
self.active -= 1
|
||||
|
||||
def build_memory_graph(self, user_id, *, agent_id=None, limit=10000, clear_first=False):
|
||||
self.graph_builds.append(user_id)
|
||||
return {"success": True, "memories_scanned": 4, "nodes": 4}
|
||||
|
||||
|
||||
class _Coordinator:
|
||||
"""The real per-user exclusivity over a stand-in client."""
|
||||
|
||||
def __init__(self, sdk, monkeypatch) -> None:
|
||||
self.inner, _ = _coordinator(monkeypatch, ["ultra"])
|
||||
self.sdk = sdk
|
||||
|
||||
def exclusive_for_evolve(self, user_id):
|
||||
return self.inner.exclusive_for_evolve(user_id)
|
||||
|
||||
@asynccontextmanager
|
||||
async def use_client(self):
|
||||
yield self.sdk
|
||||
|
||||
|
||||
def test_evolve_refuses_outside_ultra(tmp_path, monkeypatch) -> None:
|
||||
service = EvolveService(
|
||||
coordinator=_Coordinator(_EvolveSdk(), monkeypatch),
|
||||
state=EvolveStateStore(tmp_path),
|
||||
memory_mode=lambda: "pro",
|
||||
)
|
||||
|
||||
async def scenario():
|
||||
with pytest.raises(EvolveModeError):
|
||||
service.start("local")
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_evolve_loops_while_should_continue_and_records_counts_only(tmp_path, monkeypatch) -> None:
|
||||
sdk = _EvolveSdk(continuing=2)
|
||||
state = EvolveStateStore(tmp_path)
|
||||
service = EvolveService(
|
||||
coordinator=_Coordinator(sdk, monkeypatch), state=state, memory_mode=lambda: "ultra"
|
||||
)
|
||||
|
||||
async def scenario():
|
||||
started = service.start("wife", max_batches=2)
|
||||
assert started.state == "running"
|
||||
done = await service.wait(started.run_id)
|
||||
assert done is not None
|
||||
return done
|
||||
|
||||
done = asyncio.run(scenario())
|
||||
assert done.state == "succeeded" and done.rounds == 3 and done.tasks_processed == 6
|
||||
assert done.cycle_status == "complete" and done.should_continue is False
|
||||
assert {call["agent_id"] for call in sdk.calls} == {"all-apps"}
|
||||
assert {call["max_batches"] for call in sdk.calls} == {2}
|
||||
saved = (tmp_path / "evolve-state.json").read_text()
|
||||
assert "private text" not in saved
|
||||
assert json.loads(saved)["users"]["wife"]["last_result"]["rounds"] == 3
|
||||
|
||||
|
||||
def test_evolve_stops_at_the_round_cap(tmp_path, monkeypatch) -> None:
|
||||
sdk = _EvolveSdk(continuing=100)
|
||||
service = EvolveService(
|
||||
coordinator=_Coordinator(sdk, monkeypatch),
|
||||
state=EvolveStateStore(tmp_path),
|
||||
memory_mode=lambda: "ultra",
|
||||
max_rounds=4,
|
||||
)
|
||||
|
||||
async def scenario():
|
||||
return await service.wait(service.start("local").run_id)
|
||||
|
||||
done = asyncio.run(scenario())
|
||||
assert done is not None and done.rounds == 4 and done.capped is True
|
||||
assert done.state == "succeeded"
|
||||
|
||||
|
||||
def test_one_run_per_user_and_never_beside_that_users_write(tmp_path, monkeypatch) -> None:
|
||||
sdk = _EvolveSdk(continuing=0)
|
||||
coordinator = _Coordinator(sdk, monkeypatch)
|
||||
service = EvolveService(
|
||||
coordinator=coordinator, state=EvolveStateStore(tmp_path), memory_mode=lambda: "ultra"
|
||||
)
|
||||
writing = asyncio.Event()
|
||||
release = asyncio.Event()
|
||||
|
||||
async def fake_write(self, records, *, request_id, before_hy_invocation):
|
||||
writing.set()
|
||||
await release.wait()
|
||||
return sdk_write_processor.SdkWriteReceipt()
|
||||
|
||||
monkeypatch.setattr(sdk_write_processor.SdkWriteProcessor, "process_sdk_job", fake_write)
|
||||
record = (_record("hello", extracted=False, user_id="wife"),)
|
||||
|
||||
async def scenario():
|
||||
inner = coordinator.inner
|
||||
write = asyncio.create_task(
|
||||
inner.process_sdk_job(record, request_id="w", before_hy_invocation=_noop)
|
||||
)
|
||||
await writing.wait()
|
||||
first = service.start("wife")
|
||||
again = service.start("wife")
|
||||
assert again.run_id == first.run_id
|
||||
other = service.start("local")
|
||||
await service.wait(other.run_id)
|
||||
# The other user is not held; wife's run waits for wife's write.
|
||||
assert sdk.calls and {call["user_id"] for call in sdk.calls} == {"local"}
|
||||
assert inner.evolving("wife")
|
||||
with pytest.raises(RuntimeError, match="memory_evolve_in_progress"):
|
||||
await inner.process_sdk_job(record, request_id="w2", before_hy_invocation=_noop)
|
||||
release.set()
|
||||
await write
|
||||
done = await service.wait(first.run_id)
|
||||
assert done is not None and done.state == "succeeded"
|
||||
assert not inner.evolving("wife")
|
||||
assert sdk.peak == 1
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_a_failed_round_records_its_code(tmp_path, monkeypatch) -> None:
|
||||
class Failing(_EvolveSdk):
|
||||
def evolve(self, user_id, agent_id="default_agent", max_batches=None):
|
||||
return {
|
||||
"success": False,
|
||||
"error_code": "schema_llm_failed",
|
||||
"failure_kind": "llm_error",
|
||||
"error_message": "the provider said something private",
|
||||
}
|
||||
|
||||
state = EvolveStateStore(tmp_path)
|
||||
service = EvolveService(
|
||||
coordinator=_Coordinator(Failing(), monkeypatch), state=state, memory_mode=lambda: "ultra"
|
||||
)
|
||||
done = asyncio.run(service.wait(_start(service, "local")))
|
||||
assert done is not None and done.state == "failed"
|
||||
assert done.error_code == "schema_llm_failed" and done.failure_kind == "llm_error"
|
||||
assert state.user("local").last_error == "schema_llm_failed"
|
||||
assert "private" not in (tmp_path / "evolve-state.json").read_text()
|
||||
|
||||
|
||||
def _start(service: EvolveService, user_id: str) -> str:
|
||||
async def begin() -> str:
|
||||
return service.start(user_id).run_id
|
||||
|
||||
loop = asyncio.new_event_loop()
|
||||
try:
|
||||
run_id = loop.run_until_complete(begin())
|
||||
loop.run_until_complete(service.wait(run_id))
|
||||
finally:
|
||||
loop.close()
|
||||
return run_id
|
||||
|
||||
|
||||
# --- the nightly schedule -------------------------------------------------------
|
||||
|
||||
LOCAL = datetime.now().astimezone().tzinfo
|
||||
|
||||
|
||||
def _at(day: int, hour: int, minute: int = 0) -> datetime:
|
||||
return datetime(2026, 11, day, hour, minute, tzinfo=LOCAL)
|
||||
|
||||
|
||||
def test_evolve_is_due_after_the_slot_once_a_day_and_catches_up_after_sleep() -> None:
|
||||
schedule = EvolveSchedule()
|
||||
anchor = _at(1, 12)
|
||||
# Never ran: not at noon on the day ultra was enabled, then at 03:00 next day.
|
||||
assert not evolve_due(_at(1, 13), schedule, None, anchor)
|
||||
assert not evolve_due(_at(2, 2, 59), schedule, None, anchor)
|
||||
assert evolve_due(_at(2, 3, 0), schedule, None, anchor)
|
||||
# Ran at 03:00: not again that day, due again the next night.
|
||||
assert not evolve_due(_at(2, 22), schedule, _at(2, 3, 1), anchor)
|
||||
assert evolve_due(_at(3, 3, 5), schedule, _at(2, 3, 1), anchor)
|
||||
# Asleep at 03:00: a run older than the slot is due on wake, the same day.
|
||||
assert evolve_due(_at(3, 9), schedule, _at(2, 3, 1), anchor)
|
||||
# Last run more than 24h ago is due even before today's slot.
|
||||
assert evolve_due(_at(4, 1), schedule, _at(2, 3, 1), anchor)
|
||||
assert not evolve_due(_at(3, 1), schedule, _at(2, 3, 1), anchor)
|
||||
# Moved and turned off.
|
||||
assert evolve_due(_at(2, 23, 30), EvolveSchedule(time="23:30"), _at(2, 3), anchor)
|
||||
assert not evolve_due(_at(3, 9), EvolveSchedule(enabled=False), _at(1, 3), anchor)
|
||||
|
||||
|
||||
def test_the_scheduler_runs_each_due_user_once_in_ultra_only(tmp_path, monkeypatch) -> None:
|
||||
sdk = _EvolveSdk(continuing=0)
|
||||
state = EvolveStateStore(tmp_path)
|
||||
modes = ["pro"]
|
||||
clock = [_at(1, 12)]
|
||||
service = EvolveService(
|
||||
coordinator=_Coordinator(sdk, monkeypatch),
|
||||
state=state,
|
||||
memory_mode=lambda: modes[0],
|
||||
now=lambda: clock[0],
|
||||
)
|
||||
schedule = [EvolveSchedule()]
|
||||
|
||||
async def users():
|
||||
return ("local", "wife")
|
||||
|
||||
scheduler = EvolveScheduler(
|
||||
service=service,
|
||||
schedule=lambda: schedule[0],
|
||||
users_with_memories=users,
|
||||
now=lambda: clock[0],
|
||||
)
|
||||
|
||||
async def scenario():
|
||||
assert await scheduler.tick() == () # pro: nothing
|
||||
modes[0] = "ultra"
|
||||
assert await scheduler.tick() == () # anchored at noon
|
||||
clock[0] = _at(2, 3, 2)
|
||||
assert await scheduler.tick() == ("local", "wife")
|
||||
assert await scheduler.tick() == () # once a night
|
||||
clock[0] = _at(3, 8) # slept through 03:00
|
||||
assert await scheduler.tick() == ("local", "wife")
|
||||
schedule[0] = EvolveSchedule(enabled=False)
|
||||
clock[0] = _at(5, 8)
|
||||
assert await scheduler.tick() == ()
|
||||
|
||||
asyncio.run(scenario())
|
||||
assert [call["user_id"] for call in sdk.calls] == ["local", "wife", "local", "wife"]
|
||||
assert state.user("wife").last_result["trigger"] == "scheduled"
|
||||
|
||||
|
||||
def test_the_graph_backfill_runs_once_per_user(tmp_path, monkeypatch) -> None:
|
||||
sdk = _EvolveSdk()
|
||||
state = EvolveStateStore(tmp_path)
|
||||
backfill = MemoryGraphBackfill(coordinator=_Coordinator(sdk, monkeypatch), state=state)
|
||||
|
||||
async def scenario():
|
||||
backfill.start(["local", "wife", "local"])
|
||||
await backfill.wait()
|
||||
|
||||
asyncio.run(scenario())
|
||||
assert sdk.graph_builds == ["local", "wife"]
|
||||
built = state.graph_build()
|
||||
assert built.state == "succeeded"
|
||||
assert {user: value.state for user, value in built.users.items()} == {
|
||||
"local": "succeeded",
|
||||
"wife": "succeeded",
|
||||
}
|
||||
|
||||
|
||||
# --- the HTTP surface -----------------------------------------------------------
|
||||
|
||||
|
||||
def application(tmp_path: Path):
|
||||
settings = Settings(
|
||||
data_dir=tmp_path / "data",
|
||||
source_home=tmp_path,
|
||||
api_token=SecretStr(TOKEN),
|
||||
imports_enabled=False,
|
||||
automatic_memory_processing=False,
|
||||
live_sync_enabled=False,
|
||||
)
|
||||
return create_app(settings, secret_store=InMemorySecretStore())
|
||||
|
||||
|
||||
def client_for(app) -> AsyncClient:
|
||||
return AsyncClient(
|
||||
transport=ASGITransport(app=app),
|
||||
base_url="http://test",
|
||||
headers={"Authorization": f"Bearer {TOKEN}"},
|
||||
)
|
||||
|
||||
|
||||
class _ReadSdk(_EvolveSdk):
|
||||
def memory_stats(self, *, user_id, statuses=None):
|
||||
return {"total": 1, "scenes": {}}
|
||||
|
||||
|
||||
def run_app(tmp_path, monkeypatch, scenario) -> None:
|
||||
monkeypatch.delenv("ONE_MEMORY_GRAPH_STORE", raising=False)
|
||||
|
||||
async def main() -> None:
|
||||
app = application(tmp_path)
|
||||
sdk = _ReadSdk(continuing=1)
|
||||
|
||||
@asynccontextmanager
|
||||
async def write_client():
|
||||
yield sdk
|
||||
|
||||
async with app.router.lifespan_context(app):
|
||||
monkeypatch.setattr(app.state.sdk_write_coordinator, "use_client", write_client)
|
||||
async with client_for(app) as client:
|
||||
await scenario(app, sdk, client)
|
||||
|
||||
asyncio.run(main())
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_mode_api_get_put_and_its_effects(tmp_path, monkeypatch) -> None:
|
||||
async def scenario(app, sdk, client):
|
||||
current = (await client.get("/api/v1/settings/memory-mode")).json()
|
||||
assert current["mode"] == current["configured_mode"] == "pro"
|
||||
assert current["forced_by_env"] is False
|
||||
assert app.state.sdk_write_coordinator.memory_mode() == "pro"
|
||||
|
||||
lite = await client.put("/api/v1/settings/memory-mode", json={"mode": "lite"})
|
||||
assert lite.status_code == 200, lite.text
|
||||
body = lite.json()
|
||||
assert body["mode"] == "lite" and body["previous_mode"] == "pro"
|
||||
assert body["effects"] == ["higher_mode_memories_hidden"]
|
||||
assert app.state.sdk_write_coordinator.memory_mode() == "lite"
|
||||
assert _sdk_client_handle(app).memory_mode() == "lite"
|
||||
# lite claims queued work without a generation model.
|
||||
from one_memory.api import _sedimentation_claim_enabled
|
||||
|
||||
assert app.state.model_profile_store.active_identity() is None
|
||||
app.state.history_import_gate_released = True
|
||||
assert _sedimentation_claim_enabled(app, "one-memory-manual")
|
||||
|
||||
ultra = await client.put("/api/v1/settings/memory-mode", json={"mode": "ultra"})
|
||||
assert ultra.status_code == 200, ultra.text
|
||||
assert "existing_memories_added_to_graph" in ultra.json()["effects"]
|
||||
await app.state.memory_graph_backfill.wait()
|
||||
assert sdk.graph_builds == ["local"]
|
||||
reread = (await client.get("/api/v1/settings/memory-mode")).json()
|
||||
assert reread["graph_build"]["state"] == "succeeded"
|
||||
|
||||
for bad in ({"mode": "max"}, {"mode": "pro", "x": 1}):
|
||||
assert (await client.put("/api/v1/settings/memory-mode", json=bad)).status_code == 422
|
||||
assert (
|
||||
await client.get("/api/v1/settings/memory-mode", params={"user_id": "x"})
|
||||
).status_code == 422
|
||||
|
||||
monkeypatch.setenv("ONE_MEMORY_GRAPH_STORE", "kuzu")
|
||||
await client.put("/api/v1/settings/memory-mode", json={"mode": "pro"})
|
||||
forced = (await client.get("/api/v1/settings/memory-mode")).json()
|
||||
assert forced["mode"] == "ultra" and forced["configured_mode"] == "pro"
|
||||
assert forced["forced_by_env"] is True
|
||||
|
||||
run_app(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_evolve_api_is_ultra_only_async_and_scoped_per_user(tmp_path, monkeypatch) -> None:
|
||||
async def scenario(app, sdk, client):
|
||||
refused = await client.post("/api/v2/evolve", json={})
|
||||
assert refused.status_code == 409
|
||||
assert refused.json()["detail"] == "memory_mode_not_ultra"
|
||||
|
||||
await client.put("/api/v1/settings/memory-mode", json={"mode": "ultra"})
|
||||
await app.state.memory_graph_backfill.wait()
|
||||
started = await client.post(
|
||||
"/api/v2/evolve", json={"user_id": "wife", "agent_id": "all-apps", "max_batches": 2}
|
||||
)
|
||||
assert started.status_code == 202, started.text
|
||||
run = started.json()
|
||||
assert run["state"] == "running" and run["user_id"] == "wife"
|
||||
await app.state.evolve_service.wait(run["run_id"])
|
||||
path = f"/api/v2/evolve/{run['run_id']}"
|
||||
assert (await client.get(path)).status_code == 404
|
||||
done = (await client.get(path, params={"user_id": "wife"})).json()
|
||||
assert done["state"] == "succeeded" and done["rounds"] == 2
|
||||
assert sdk.calls[0] == {"user_id": "wife", "agent_id": "all-apps", "max_batches": 2}
|
||||
|
||||
overview = (await client.get("/api/v2/evolve", params={"user_id": "wife"})).json()
|
||||
assert set(overview["users"]) == {"wife"}
|
||||
assert overview["schedule"] == {"enabled": True, "time": "03:00"}
|
||||
|
||||
processing = (await client.get("/api/v1/memory-processing")).json()
|
||||
pipeline = processing["pipeline"]
|
||||
assert pipeline["mode"] == "ultra"
|
||||
assert pipeline["last_evolve_at"] is not None
|
||||
assert pipeline["evolve"]["users"]["wife"]["last_result"]["state"] == "succeeded"
|
||||
|
||||
for bad in (
|
||||
{"user_id": "a:b"},
|
||||
{"max_batches": 0},
|
||||
{"from_beginning": True},
|
||||
):
|
||||
assert (await client.post("/api/v2/evolve", json=bad)).status_code == 422
|
||||
|
||||
moved = await client.put(
|
||||
"/api/v1/settings/evolve-schedule", json={"time": "04:30", "enabled": False}
|
||||
)
|
||||
assert moved.json() == {"enabled": False, "time": "04:30"}
|
||||
assert (
|
||||
await client.put("/api/v1/settings/evolve-schedule", json={"time": "25:00"})
|
||||
).status_code == 422
|
||||
|
||||
run_app(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_a_job_written_in_lite_stays_done_and_a_pending_job_takes_the_new_mode(
|
||||
tmp_path, monkeypatch
|
||||
) -> None:
|
||||
"""lite to pro reprocesses nothing: the mode is read when a job runs."""
|
||||
from one_memory.memory_server.durable_queue import DurableWriteDrain, execute_job
|
||||
|
||||
modes = ["lite"]
|
||||
seen: list[str] = []
|
||||
|
||||
class Sdk:
|
||||
def __init__(self, mode):
|
||||
self.mode = mode
|
||||
|
||||
def add(self, messages, **kwargs):
|
||||
seen.append(self.mode)
|
||||
return {"success": True, "raw_ids": [f"raw-{len(seen)}"]}
|
||||
|
||||
monkeypatch.setattr(
|
||||
sdk_write_processor,
|
||||
"SdkClientHandle",
|
||||
lambda profile, **kwargs: _SdkHandle(Sdk(kwargs["mode"]), kwargs["mode"]),
|
||||
)
|
||||
coordinator = sdk_write_processor.SdkWriteCoordinator(
|
||||
profiles=_Profiles(), # type: ignore[arg-type]
|
||||
secrets=None, # type: ignore[arg-type]
|
||||
data_dir=None,
|
||||
transport_factory=lambda _profile: None,
|
||||
sedimentation_enabled=lambda: True,
|
||||
memory_mode=lambda: modes[0],
|
||||
)
|
||||
|
||||
async def never(_job_id):
|
||||
return None
|
||||
|
||||
async def drain_once(queue) -> int:
|
||||
drain = DurableWriteDrain(queue, execute=never, lease=timedelta(seconds=30))
|
||||
claimed = drain.claim(limit=8)
|
||||
for job_id in claimed:
|
||||
await execute_job(
|
||||
job_id,
|
||||
queue=queue,
|
||||
supervisor=coordinator,
|
||||
evidence=None, # type: ignore[arg-type]
|
||||
)
|
||||
return len(claimed)
|
||||
|
||||
async def scenario(app, _sdk, client):
|
||||
queue = app.state.memory_write_queue
|
||||
for request_id in ("turn-lite", "turn-pro"):
|
||||
written = await client.post(
|
||||
"/api/v2/memory-writes",
|
||||
json={
|
||||
"request_id": request_id,
|
||||
"session_id": f"s-{request_id}",
|
||||
"source_id": "one-memory-manual",
|
||||
"messages": [
|
||||
{"role": "user", "content": request_id},
|
||||
{"role": "assistant", "content": "ok"},
|
||||
],
|
||||
},
|
||||
)
|
||||
assert written.status_code == 202, written.text
|
||||
if request_id == "turn-lite":
|
||||
assert await drain_once(queue) == 1
|
||||
modes[0] = "pro"
|
||||
await coordinator.retire_stale_modes()
|
||||
assert await drain_once(queue) == 1
|
||||
assert await drain_once(queue) == 0
|
||||
assert queue.get("turn-lite").state == queue.get("turn-pro").state == "completed"
|
||||
assert seen == ["lite", "pro"]
|
||||
await coordinator.close()
|
||||
|
||||
run_app(tmp_path, monkeypatch, scenario)
|
||||
@@ -66,6 +66,7 @@ def _grounded_setup(tmp_path, *, memory_id: str = "memory-real-data"):
|
||||
base_url="http://127.0.0.1:11434",
|
||||
model="test-model",
|
||||
embedding_model="test-embedding",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
@@ -263,6 +264,7 @@ def test_answer_uses_only_grounded_confirmed_memory_citations(tmp_path) -> None:
|
||||
base_url="http://127.0.0.1:11434",
|
||||
model="test-model",
|
||||
embedding_model="test-embedding",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
@@ -468,6 +470,7 @@ def test_answer_rejects_model_citation_outside_retrieved_scope(tmp_path) -> None
|
||||
base_url="http://127.0.0.1:11434",
|
||||
model="test-model",
|
||||
embedding_model="test-embedding",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
@@ -530,6 +533,7 @@ def _compatible_gateway_setup(tmp_path):
|
||||
base_url="https://api.example.invalid/standard/v1",
|
||||
model="deepseek-v4-pro",
|
||||
embedding_model="text-embedding",
|
||||
embedding_base_url="https://api.example.invalid/standard/v1",
|
||||
api_key="probe-key",
|
||||
),
|
||||
secrets,
|
||||
|
||||
@@ -56,7 +56,7 @@ def test_sdk_ask_v2_stream_is_native(tmp_path, monkeypatch) -> None:
|
||||
)
|
||||
|
||||
class Answerer:
|
||||
async def answer_stream(self, _question):
|
||||
async def answer_stream(self, _question, *, user_id="local"):
|
||||
yield {"type": "recall", "memories": (memory,)}
|
||||
yield {"type": "delta", "text": "你要求真实数据。"}
|
||||
yield {
|
||||
|
||||
@@ -186,6 +186,7 @@ def _setup(tmp_path: Path, *, model: str = "available") -> _Setup:
|
||||
base_url="https://memory-qa.example.invalid/v1",
|
||||
model="synthetic-generation",
|
||||
embedding_model="synthetic-embedding",
|
||||
embedding_base_url="https://memory-qa.example.invalid/v1",
|
||||
api_key=SecretStr(_SECRET),
|
||||
),
|
||||
secrets,
|
||||
|
||||
@@ -25,6 +25,7 @@ def _stores(tmp_path):
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="hy-bot-phi4",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
|
||||
@@ -83,6 +83,7 @@ def test_task_freezes_active_profile_revision_and_is_idempotent(tmp_path) -> Non
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="qwen",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
@@ -240,6 +241,7 @@ def test_task_extraction_checkpoint_is_idempotent_and_recoverable(tmp_path) -> N
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="qwen",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
),
|
||||
@@ -293,6 +295,7 @@ def test_task_waits_for_model_and_retries_with_same_snapshot(tmp_path) -> None:
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="qwen",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
@@ -337,6 +340,7 @@ def test_blocked_stages_name_the_stuck_stage_and_reason(tmp_path) -> None:
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="qwen",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
),
|
||||
@@ -433,6 +437,7 @@ def test_task_duration_estimate_ignores_time_the_app_was_closed(tmp_path) -> Non
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="qwen",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
),
|
||||
@@ -537,6 +542,7 @@ def test_running_task_is_recovered_after_worker_restart(tmp_path) -> None:
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="qwen",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
),
|
||||
@@ -572,6 +578,7 @@ def test_superseded_profile_cancels_only_claimable_tasks(tmp_path) -> None:
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="phi",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
),
|
||||
@@ -601,6 +608,7 @@ def test_superseded_profile_cancels_only_claimable_tasks(tmp_path) -> None:
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="qwen",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
),
|
||||
@@ -630,6 +638,7 @@ def test_superseded_ingest_policy_cancels_only_old_conversation_tasks(tmp_path)
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="qwen",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
),
|
||||
@@ -676,6 +685,7 @@ def test_task_rejects_inactive_profile_and_request_id_reuse(tmp_path) -> None:
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="qwen",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
@@ -1700,6 +1710,7 @@ def _profile_and_tasks(tmp_path) -> tuple[MemoryTaskStore, ModelProfile]:
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="qwen",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
),
|
||||
@@ -2058,6 +2069,7 @@ def test_a_failed_task_below_the_ceiling_is_still_claimable(tmp_path) -> None:
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="qwen",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
),
|
||||
|
||||
@@ -55,6 +55,7 @@ def library(tmp_path):
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="hy-bot-phi4",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
@@ -611,3 +612,58 @@ def test_granting_a_source_repeatedly_does_not_raise(library):
|
||||
|
||||
with main_database_read(database) as db:
|
||||
assert memory.memory_id in pending(db)
|
||||
|
||||
|
||||
def test_grant_writes_still_work_after_the_cutover_drops_the_link_table(library):
|
||||
"""#233: the grant triggers read `governed_memory_evidence`.
|
||||
|
||||
The SDK cutover drops it, and SQLite resolves a trigger body only when it
|
||||
fires, so every grant, revoke and connect then failed with "no such
|
||||
table". A trigger whose body names a missing table is not installed, and
|
||||
the copy an earlier startup installed is dropped.
|
||||
"""
|
||||
database, *_ = library
|
||||
from one_memory.imports.candidates import RootCandidateKind
|
||||
from one_memory.imports.grants import SourceGrantStore
|
||||
from one_memory.memory_visibility import drop_stale_triggers
|
||||
|
||||
startup(database)
|
||||
with sqlite3.connect(database) as db:
|
||||
db.execute("DROP TABLE governed_memory_evidence")
|
||||
|
||||
def grant_triggers() -> set[str]:
|
||||
with sqlite3.connect(database) as db:
|
||||
return {
|
||||
str(row[0])
|
||||
for row in db.execute(
|
||||
"SELECT name FROM sqlite_master "
|
||||
"WHERE type='trigger' AND tbl_name='source_grants'"
|
||||
)
|
||||
}
|
||||
|
||||
assert grant_triggers() == {
|
||||
"memory_visibility_dirty_on_grants",
|
||||
"memory_visibility_dirty_on_grants_update",
|
||||
}
|
||||
# The `evidence` triggers read the dropped table too; the cutover drops
|
||||
# `evidence` as well, which takes them with it.
|
||||
assert {
|
||||
"memory_visibility_dirty_on_grants",
|
||||
"memory_visibility_dirty_on_grants_update",
|
||||
} <= set(drop_stale_triggers(database))
|
||||
assert grant_triggers() == set()
|
||||
assert drop_stale_triggers(database) == ()
|
||||
# Installing again on this shape skips them instead of bringing them back.
|
||||
startup(database)
|
||||
assert grant_triggers() == set()
|
||||
|
||||
folder = database.parent / "cursor"
|
||||
folder.mkdir(exist_ok=True)
|
||||
store = SourceGrantStore(database, home=database.parent)
|
||||
store.grant(
|
||||
user_id="local",
|
||||
source_id="cursor",
|
||||
root=folder,
|
||||
candidate_kind=RootCandidateKind.DEFAULT,
|
||||
)
|
||||
assert store.revoke(user_id="local", source_id="cursor").revoked_at is not None
|
||||
@@ -51,6 +51,7 @@ def library(tmp_path):
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="hy-bot-phi4",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
|
||||
@@ -28,6 +28,7 @@ def test_probe_still_measures_embedding_when_structured_output_fails(tmp_path) -
|
||||
base_url="http://127.0.0.1:11434",
|
||||
model="plain-model",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
@@ -63,6 +64,7 @@ def test_probe_records_the_real_embedding_width_on_the_profile(tmp_path) -> None
|
||||
base_url="http://127.0.0.1:11434",
|
||||
model="qwen3:8b",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
@@ -119,6 +121,7 @@ def test_openai_compatible_probe_checks_model_and_structured_output(tmp_path) ->
|
||||
base_url="https://models.example.com/v1",
|
||||
model="memory-model",
|
||||
embedding_model="embedding-model",
|
||||
embedding_base_url="https://models.example.com/v1",
|
||||
api_key=SecretStr(API_KEY),
|
||||
),
|
||||
secrets,
|
||||
@@ -260,6 +263,7 @@ def test_probe_reports_missing_model_without_sending_generation(tmp_path) -> Non
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="memory-model",
|
||||
embedding_model="embedding-model",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
|
||||
@@ -50,6 +50,7 @@ def test_model_profile_api_never_returns_key_and_enforces_revision(tmp_path) ->
|
||||
"base_url": "https://models.example.com/v1",
|
||||
"model": "memory-model",
|
||||
"embedding_model": "embedding-model",
|
||||
"embedding_base_url": "https://models.example.com/v1",
|
||||
"api_key": API_KEY,
|
||||
},
|
||||
)
|
||||
@@ -134,6 +135,7 @@ def test_model_profile_probe_route_returns_only_machine_results(tmp_path) -> Non
|
||||
"base_url": "https://models.example.com/v1",
|
||||
"model": "memory-model",
|
||||
"embedding_model": "embedding-model",
|
||||
"embedding_base_url": "https://models.example.com/v1",
|
||||
"api_key": API_KEY,
|
||||
},
|
||||
)
|
||||
@@ -244,6 +246,7 @@ def test_activation_survives_a_corpus_already_imported(tmp_path) -> None:
|
||||
"base_url": "https://models.example.com/v1",
|
||||
"model": "memory-model",
|
||||
"embedding_model": "embedding-model",
|
||||
"embedding_base_url": "https://models.example.com/v1",
|
||||
"api_key": API_KEY,
|
||||
},
|
||||
)
|
||||
|
||||
@@ -198,6 +198,7 @@ def test_active_profile_can_be_deleted_without_changing_other_profiles(tmp_path)
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="other",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
@@ -208,6 +209,7 @@ def test_active_profile_can_be_deleted_without_changing_other_profiles(tmp_path)
|
||||
base_url="https://models.example.com/v1",
|
||||
model="qwen",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="https://models.example.com/v1",
|
||||
api_key=SecretStr(API_KEY),
|
||||
),
|
||||
secrets,
|
||||
@@ -349,6 +351,7 @@ def _probed(store, secrets, *, structured: bool, generates: bool = True):
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="qwen",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
@@ -394,3 +397,102 @@ def test_a_model_that_cannot_generate_still_cannot_be_turned_on(tmp_path) -> Non
|
||||
|
||||
with pytest.raises(ModelProfileUnavailableError):
|
||||
store.activate(profile.profile_id, secrets)
|
||||
|
||||
|
||||
def _space(store: ModelProfileStore, profile_id: str) -> str:
|
||||
"""The embedding space's collection base, as the Runtime names it."""
|
||||
from one_memory.hy_sdk_runtime import vector_collection_bases
|
||||
|
||||
identity = (profile_id, store.require(profile_id, InMemorySecretStore()).revision)
|
||||
return vector_collection_bases(store.snapshot(*identity))[0]
|
||||
|
||||
|
||||
def test_a_remote_embedding_must_name_its_endpoint(tmp_path) -> None:
|
||||
"""The endpoint no longer follows the generation `base_url`."""
|
||||
from one_memory.model_profiles import EmbeddingBaseUrlRequiredError
|
||||
|
||||
store = ModelProfileStore(tmp_path / "one-memory.sqlite3")
|
||||
secrets = InMemorySecretStore()
|
||||
unnamed = ModelProfileInput(
|
||||
name="Remote",
|
||||
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
|
||||
base_url="https://models.example.com/v1",
|
||||
model="qwen",
|
||||
embedding_model="text-embedding-3-small",
|
||||
api_key=SecretStr(API_KEY),
|
||||
)
|
||||
with pytest.raises(EmbeddingBaseUrlRequiredError):
|
||||
store.create(unnamed, secrets)
|
||||
assert store.list(secrets) == []
|
||||
|
||||
named = store.create(
|
||||
unnamed.model_copy(update={"embedding_base_url": "https://models.example.com/v1"}),
|
||||
secrets,
|
||||
)
|
||||
with pytest.raises(EmbeddingBaseUrlRequiredError):
|
||||
store.update(
|
||||
named.profile_id,
|
||||
ModelProfilePatch(
|
||||
expected_revision=named.revision,
|
||||
embedding_provider_kind=ProviderKind.OLLAMA,
|
||||
embedding_base_url="",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
|
||||
|
||||
def test_stored_profiles_get_the_endpoint_they_resolved_to_and_keep_their_space(
|
||||
tmp_path,
|
||||
) -> None:
|
||||
"""An older build stored remote embeddings with a NULL endpoint and resolved
|
||||
it to the generation `base_url` at runtime. Opening the store writes that
|
||||
endpoint in once, so the collection, and the memories in it, stay put; a
|
||||
later edit of the generation endpoint no longer moves them.
|
||||
"""
|
||||
from one_memory.hy_sdk_runtime import vector_collection_bases
|
||||
|
||||
database = tmp_path / "one-memory.sqlite3"
|
||||
store = ModelProfileStore(database)
|
||||
secrets = InMemorySecretStore()
|
||||
profile = store.create(
|
||||
ModelProfileInput(
|
||||
name="Remote",
|
||||
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
|
||||
base_url="https://gateway.example.com/v1",
|
||||
model="qwen",
|
||||
embedding_model="text-embedding-3-small",
|
||||
embedding_base_url="https://gateway.example.com/v1",
|
||||
embedding_dimensions=1536,
|
||||
api_key=SecretStr(API_KEY),
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
identity = (profile.profile_id, profile.revision)
|
||||
# The space the older build's Runtime resolved: `embedding_base_url or base_url`.
|
||||
before = vector_collection_bases(store.snapshot(*identity))[0]
|
||||
with sqlite3.connect(database) as db:
|
||||
# The shape an older build stored.
|
||||
db.execute("UPDATE model_profiles SET embedding_base_url=NULL")
|
||||
db.execute("UPDATE model_profile_versions SET embedding_base_url=NULL")
|
||||
|
||||
migrated = ModelProfileStore(database)
|
||||
assert migrated.snapshot(*identity).embedding_base_url == "https://gateway.example.com/v1"
|
||||
assert _space(migrated, profile.profile_id) == before
|
||||
# Idempotent: opening again changes nothing.
|
||||
assert _space(ModelProfileStore(database), profile.profile_id) == before
|
||||
with sqlite3.connect(database) as db:
|
||||
assert db.execute(
|
||||
"SELECT COUNT(*) FROM model_profile_versions WHERE embedding_base_url IS NULL"
|
||||
).fetchone() == (0,)
|
||||
|
||||
edited = migrated.update(
|
||||
profile.profile_id,
|
||||
ModelProfilePatch(
|
||||
expected_revision=profile.revision,
|
||||
base_url="https://gateway.example.com/eu/v1",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
assert edited.base_url == "https://gateway.example.com/eu/v1"
|
||||
assert edited.embedding_base_url == "https://gateway.example.com/v1"
|
||||
assert _space(migrated, profile.profile_id) == before
|
||||
@@ -0,0 +1,858 @@
|
||||
"""One Runtime, many users: `user_id` on the memory API, per-user Hermes profiles.
|
||||
|
||||
A user is the root that is read (a Hermes profile's own `state.db`), never a
|
||||
session or platform field. The App never names a user and keeps `local`;
|
||||
every other harness stays on `local`. These tests drive the production app
|
||||
against a user-aware stand-in for the public SDK client, whose `get`,
|
||||
`update` and `delete` take no user exactly like hy-memory 2.0.9's.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import sqlite3
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
from pydantic import SecretStr
|
||||
|
||||
from one_memory.api import _sdk_recall_target, create_app
|
||||
from one_memory.config import Settings
|
||||
from one_memory.imports.access import AuthorizedSourceRoot
|
||||
from one_memory.imports.adapters import ScannedEvidence, ScanOptions
|
||||
from one_memory.imports.models import (
|
||||
EvidenceInput,
|
||||
EvidenceOrigin,
|
||||
OriginMetadata,
|
||||
WrittenBy,
|
||||
materialize_write_input,
|
||||
)
|
||||
from one_memory.imports.sources.hermes import HERMES_DESCRIPTOR, create_hermes_adapter
|
||||
from one_memory.imports.sources.registry import build_source_catalog
|
||||
from one_memory.memory_server.contracts import internal_session_id_for
|
||||
from one_memory.memory_server.durable_queue import (
|
||||
DurableWriteDrain,
|
||||
DurableWriteQueue,
|
||||
execute_job,
|
||||
)
|
||||
from one_memory.sdk_write_processor import SdkWriteProcessor
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
from one_memory.user_scope import lane_key, split_lane, valid_user_id
|
||||
|
||||
TOKEN = "multi-user-test"
|
||||
|
||||
|
||||
class UserAwareSdk:
|
||||
"""The public client surface MemHub calls, holding memories per user."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.memories: dict[str, dict[str, object]] = {}
|
||||
self.add_calls: list[dict[str, object]] = []
|
||||
|
||||
def _store(self, user_id: str, content: str, *, source: str, session_id: str) -> str:
|
||||
memory_id = f"memory-{len(self.memories) + 1}"
|
||||
self.memories[memory_id] = {
|
||||
"memory_id": memory_id,
|
||||
"content": content,
|
||||
"layer": "l2_fact",
|
||||
"status": "active",
|
||||
"user_id": user_id,
|
||||
"agent_id": "all-apps",
|
||||
"session_id": session_id,
|
||||
"source": source,
|
||||
"tags": [],
|
||||
"gmt_created": 1_790_000_000,
|
||||
}
|
||||
return memory_id
|
||||
|
||||
def add(self, messages, *, user_id, session_id, source, request_id, **_kwargs):
|
||||
self.add_calls.append({"user_id": user_id, "request_id": request_id})
|
||||
text = " ".join(item["content"] for item in messages if item["role"] == "user")
|
||||
memory_id = self._store(user_id, text, source=source, session_id=session_id)
|
||||
return {"success": True, "memory_ids": [memory_id]}
|
||||
|
||||
def add_extracted(self, contents, *, user_id, session_id, source, request_id, **_kwargs):
|
||||
self.add_calls.append({"user_id": user_id, "request_id": request_id})
|
||||
memory_id = self._store(user_id, " ".join(contents), source=source, session_id=session_id)
|
||||
return {"success": True, "memory_ids": [memory_id]}
|
||||
|
||||
def _rows(self, user_ids) -> list[dict[str, object]]:
|
||||
return [dict(row) for row in self.memories.values() if row["user_id"] in user_ids]
|
||||
|
||||
def list_memories(self, *, user_id, scene=None, offset=0, limit=100, **_kwargs):
|
||||
rows = [] if _kwargs.get("layers") == ["l1_raw"] else self._rows({user_id})
|
||||
page = rows[offset : offset + limit]
|
||||
if scene == "all":
|
||||
return {
|
||||
"scene": "all",
|
||||
"memories": [{**row, "scene": "chat", "kind": "chat"} for row in page],
|
||||
"total": len(rows),
|
||||
"totals": {"chat": len(rows), "working": 0, "coding": 0},
|
||||
"errors": {},
|
||||
"limit": limit,
|
||||
"offset": offset,
|
||||
"request_id": "list",
|
||||
"elapsed_ms": 1.0,
|
||||
}
|
||||
return {"vdb": {"memories": page, "total": len(rows), "limit": limit, "offset": offset}}
|
||||
|
||||
def memory_stats(self, *, user_id, statuses=None):
|
||||
return {"total": len(self._rows({user_id})), "request_id": "stats", "elapsed_ms": 1.0}
|
||||
|
||||
def search(self, query, *, user_ids, scene=None, **kwargs):
|
||||
rows = [] if kwargs.get("layers") == ["l1_raw"] else self._rows(set(user_ids))
|
||||
items = [{**row, "scene": "chat", "kind": "chat", "score": 0.9} for row in rows]
|
||||
if scene == "all":
|
||||
return {
|
||||
"request_id": "search",
|
||||
"scene": "all",
|
||||
"memories": items,
|
||||
"lanes": {"chat": len(items), "working": 0, "coding": 0},
|
||||
"errors": {},
|
||||
"elapsed_ms": 1.0,
|
||||
}
|
||||
return {"request_id": "search", "scene": scene or "normal", "memories": items}
|
||||
|
||||
def get(self, memory_id, kind=None, variant=None):
|
||||
row = self.memories.get(memory_id)
|
||||
return None if row is None else {**row, "kind": "chat", "variant": None}
|
||||
|
||||
def update(self, memory_id, content, kind=None, variant=None, task=None):
|
||||
if memory_id not in self.memories:
|
||||
return {"success": False, "error_message": f"memory_id '{memory_id}' not found"}
|
||||
self.memories[memory_id]["content"] = content
|
||||
return {"success": True, "memory_id": memory_id}
|
||||
|
||||
def delete(self, memory_id, kind=None, variant=None):
|
||||
if self.memories.pop(memory_id, None) is None:
|
||||
return {"success": False, "deleted_count": 0}
|
||||
return {"success": True, "deleted_count": 1}
|
||||
|
||||
def get_user_profile(self, *, user_id):
|
||||
return {
|
||||
"request_id": f"profile-{user_id}",
|
||||
"profile": {
|
||||
"memory_id": f"l0-{user_id}",
|
||||
"content": f"name is {user_id}",
|
||||
"layer": "L0_BASIC_INFO",
|
||||
"basic_info": {"name": user_id},
|
||||
},
|
||||
"elapsed_ms": 1.0,
|
||||
}
|
||||
|
||||
|
||||
class Handle:
|
||||
def __init__(self, sdk: UserAwareSdk) -> None:
|
||||
self.sdk = sdk
|
||||
|
||||
async def client(self) -> UserAwareSdk:
|
||||
return self.sdk
|
||||
|
||||
async def close(self) -> None:
|
||||
return None
|
||||
|
||||
|
||||
class Coordinator:
|
||||
"""The production write processor over the stand-in client."""
|
||||
|
||||
def __init__(self, sdk: UserAwareSdk) -> None:
|
||||
self.processor = SdkWriteProcessor(Handle(sdk)) # type: ignore[arg-type]
|
||||
|
||||
async def process_sdk_job(self, records, *, request_id, before_hy_invocation):
|
||||
return await self.processor.process_sdk_job(
|
||||
records, request_id=request_id, before_hy_invocation=before_hy_invocation
|
||||
)
|
||||
|
||||
|
||||
async def _never(_job_id: str) -> None:
|
||||
return None
|
||||
|
||||
|
||||
async def drain_all(queue: DurableWriteQueue, sdk: UserAwareSdk) -> None:
|
||||
"""Claim and run every pending job, ungated, the way the drain would."""
|
||||
drain = DurableWriteDrain(queue, execute=_never, lease=timedelta(seconds=30))
|
||||
while claimed := drain.claim(limit=8):
|
||||
for job_id in claimed:
|
||||
await execute_job(
|
||||
job_id,
|
||||
queue=queue,
|
||||
supervisor=Coordinator(sdk),
|
||||
evidence=None, # type: ignore[arg-type]
|
||||
)
|
||||
|
||||
|
||||
def application(tmp_path: Path, *, home: Path | None = None):
|
||||
"""The production app; with `home`, imports run over that home's sources."""
|
||||
settings = Settings(
|
||||
data_dir=tmp_path / "data",
|
||||
source_home=home or tmp_path,
|
||||
api_token=SecretStr(TOKEN),
|
||||
imports_enabled=home is not None,
|
||||
automatic_memory_processing=False,
|
||||
live_sync_enabled=False,
|
||||
)
|
||||
return create_app(
|
||||
settings,
|
||||
secret_store=InMemorySecretStore(),
|
||||
source_catalog=(
|
||||
build_source_catalog(home=home, environment={}, platform="darwin")
|
||||
if home is not None
|
||||
else None
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def client_for(app) -> AsyncClient:
|
||||
return AsyncClient(
|
||||
transport=ASGITransport(app=app),
|
||||
base_url="http://test",
|
||||
headers={"Authorization": f"Bearer {TOKEN}"},
|
||||
)
|
||||
|
||||
|
||||
def run_with_sdk(tmp_path, monkeypatch, scenario) -> None:
|
||||
async def main() -> None:
|
||||
app = application(tmp_path)
|
||||
sdk = UserAwareSdk()
|
||||
|
||||
@asynccontextmanager
|
||||
async def write_client():
|
||||
yield sdk
|
||||
|
||||
async with app.router.lifespan_context(app):
|
||||
app.state.sdk_client_handle = Handle(sdk)
|
||||
monkeypatch.setattr(app.state.sdk_write_coordinator, "use_client", write_client)
|
||||
async with client_for(app) as client:
|
||||
await scenario(app, sdk, client)
|
||||
|
||||
asyncio.run(main())
|
||||
|
||||
|
||||
def _messages(text: str) -> list[dict[str, str]]:
|
||||
return [{"role": "user", "content": text}, {"role": "assistant", "content": "noted"}]
|
||||
|
||||
|
||||
async def _write_both_users(app, sdk, client) -> dict[str, str]:
|
||||
"""`local` and `wife` each add one memory and submit one conversation."""
|
||||
receipts = {}
|
||||
for user, word in (("local", "tea"), ("wife", "coffee")):
|
||||
request_id = str(uuid4())
|
||||
body = {"request_id": request_id, "text": f"{user} drinks {word}", "scope": "all-apps"}
|
||||
if user != "local":
|
||||
body["user_id"] = user
|
||||
added = await client.post("/api/v2/memories", json=body)
|
||||
assert added.status_code == 201, added.text
|
||||
receipts[f"add:{user}"] = request_id
|
||||
write = {
|
||||
"request_id": "same-request-id",
|
||||
"session_id": "same-session",
|
||||
"source_id": "one-memory-manual",
|
||||
"messages": _messages(f"{user} walks the {word} route"),
|
||||
}
|
||||
if user != "local":
|
||||
write["user_id"] = user
|
||||
written = await client.post("/api/v2/memory-writes", json=write)
|
||||
assert written.status_code == 202, written.text
|
||||
assert written.json()["request_id"] == "same-request-id"
|
||||
await drain_all(app.state.memory_write_queue, sdk)
|
||||
return receipts
|
||||
|
||||
|
||||
def _contents(page: dict) -> set[str]:
|
||||
return {memory["content"] for memory in page["memories"]}
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_each_user_reads_only_their_own_library_search_profile_and_graph(
|
||||
tmp_path, monkeypatch
|
||||
) -> None:
|
||||
async def scenario(app, sdk, client):
|
||||
await _write_both_users(app, sdk, client)
|
||||
assert {call["user_id"] for call in sdk.add_calls} == {"local", "wife"}
|
||||
# The same caller request id from two users is two SDK requests.
|
||||
assert len({call["request_id"] for call in sdk.add_calls}) == len(sdk.add_calls) == 4
|
||||
|
||||
local = (await client.get("/api/v2/memory-library")).json()
|
||||
wife = (await client.get("/api/v2/memory-library", params={"user_id": "wife"})).json()
|
||||
assert _contents(local) == {"local drinks tea", "local walks the tea route"}
|
||||
assert _contents(wife) == {"wife drinks coffee", "wife walks the coffee route"}
|
||||
nobody = await client.get("/api/v2/memory-library", params={"user_id": "__nobody__"})
|
||||
assert nobody.status_code == 200 and nobody.json()["memories"] == []
|
||||
|
||||
searched = (
|
||||
await client.get("/api/v2/memory-library", params={"query": "route", "user_id": "wife"})
|
||||
).json()
|
||||
assert _contents(searched) == {"wife walks the coffee route"}
|
||||
|
||||
context = await client.post(
|
||||
"/api/v2/context/search",
|
||||
json={"query": "drinks", "kinds": ["memory"], "user_id": "wife"},
|
||||
)
|
||||
assert context.status_code == 200, context.text
|
||||
assert {item["content"] for item in context.json()["items"]} == {
|
||||
"wife drinks coffee",
|
||||
"wife walks the coffee route",
|
||||
}
|
||||
local_context = await client.post(
|
||||
"/api/v2/context/search", json={"query": "drinks", "kinds": ["memory"]}
|
||||
)
|
||||
assert all("wife" not in item["content"] for item in local_context.json()["items"])
|
||||
|
||||
profile = (await client.get("/api/v2/user-profile", params={"user_id": "wife"})).json()
|
||||
assert profile["profile"]["basic_info"] == {"name": "wife"}
|
||||
assert (await client.get("/api/v2/user-profile")).json()["profile"]["basic_info"] == {
|
||||
"name": "local"
|
||||
}
|
||||
|
||||
graph = (await client.get("/api/v2/memories/graph", params={"user_id": "wife"})).json()
|
||||
labels = {node["label"] for node in graph["nodes"] if node["node_type"] == "memory"}
|
||||
assert labels == {"wife drinks coffee", "wife walks the coffee route"}
|
||||
|
||||
summary = (await client.get("/api/v2/home-summary", params={"user_id": "wife"})).json()
|
||||
assert summary["memory_count"] == 2
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_another_users_memory_id_answers_404_for_read_update_and_delete(
|
||||
tmp_path, monkeypatch
|
||||
) -> None:
|
||||
async def scenario(app, sdk, client):
|
||||
await _write_both_users(app, sdk, client)
|
||||
[wife_memory] = [
|
||||
memory_id
|
||||
for memory_id, row in sdk.memories.items()
|
||||
if row["user_id"] == "wife" and row["content"] == "wife drinks coffee"
|
||||
]
|
||||
path = f"/api/v2/memories/{wife_memory}"
|
||||
|
||||
assert (await client.get(path)).status_code == 404
|
||||
assert (await client.put(path, json={"content": "stolen"})).status_code == 404
|
||||
assert (await client.delete(path)).status_code == 204
|
||||
assert sdk.memories[wife_memory]["content"] == "wife drinks coffee"
|
||||
|
||||
own = await client.get(path, params={"user_id": "wife"})
|
||||
assert own.status_code == 200 and own.json()["user_id"] == "wife"
|
||||
updated = await client.put(path, json={"content": "wife drinks tea", "user_id": "wife"})
|
||||
assert updated.status_code == 200
|
||||
assert (await client.delete(path, params={"user_id": "wife"})).status_code == 204
|
||||
assert wife_memory not in sdk.memories
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_receipts_belong_to_the_user_that_submitted_them(tmp_path, monkeypatch) -> None:
|
||||
async def scenario(app, sdk, client):
|
||||
receipts = await _write_both_users(app, sdk, client)
|
||||
wife_add = f"/api/v2/memories/additions/{receipts['add:wife']}"
|
||||
assert (await client.get(wife_add)).status_code == 404
|
||||
own = await client.get(wife_add, params={"user_id": "wife"})
|
||||
assert own.status_code == 200 and own.json()["state"] == "ready"
|
||||
local_add = f"/api/v2/memories/additions/{receipts['add:local']}"
|
||||
assert (await client.get(local_add, params={"user_id": "wife"})).status_code == 404
|
||||
|
||||
# Both users used `same-request-id`; each reads only their own receipt.
|
||||
local_write = await client.get("/api/v2/memory-writes/same-request-id")
|
||||
wife_write = await client.get(
|
||||
"/api/v2/memory-writes/same-request-id", params={"user_id": "wife"}
|
||||
)
|
||||
assert local_write.status_code == wife_write.status_code == 200
|
||||
assert local_write.json()["memory_ids"] != wife_write.json()["memory_ids"]
|
||||
assert (
|
||||
await client.get("/api/v2/memory-writes/same-request-id", params={"user_id": "kid"})
|
||||
).status_code == 404
|
||||
# `local` cannot name another user's namespaced queue key directly.
|
||||
with sqlite3.connect(app.state.settings.data_dir / "one-memory.sqlite3") as db:
|
||||
keys = [row[0] for row in db.execute("SELECT idempotency_key FROM memory_write_jobs")]
|
||||
wife_key = next(key for key in keys if key.startswith("user:wife:"))
|
||||
assert (await client.get(f"/api/v2/memory-writes/{wife_key}")).status_code == 404
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_omitting_user_id_keeps_local_and_recall_and_ask_pass_it_through(
|
||||
tmp_path, monkeypatch
|
||||
) -> None:
|
||||
async def scenario(app, sdk, client):
|
||||
await _write_both_users(app, sdk, client)
|
||||
seen: list[str] = []
|
||||
original = sdk.list_memories
|
||||
|
||||
def recording(**kwargs):
|
||||
seen.append(kwargs["user_id"])
|
||||
return original(**kwargs)
|
||||
|
||||
sdk.list_memories = recording # type: ignore[method-assign]
|
||||
# Built at startup over the real handle; rebuilt here over the stand-in.
|
||||
app.state.recall_target = _sdk_recall_target(app, "local")
|
||||
await client.get("/api/v2/memory-library")
|
||||
await client.get("/api/v2/raw")
|
||||
await client.get("/api/v2/home-summary")
|
||||
assert set(seen) == {"local"}
|
||||
|
||||
monkeypatch.setattr(
|
||||
"one_memory.api._recall_model_ready", lambda _app: _true(), raising=True
|
||||
)
|
||||
recalled = await client.post("/api/v1/recall", json={"query": "drinks", "user_id": "wife"})
|
||||
assert recalled.status_code == 200, recalled.text
|
||||
assert {hit["content"] for hit in recalled.json()["hits"]} == {
|
||||
"wife drinks coffee",
|
||||
"wife walks the coffee route",
|
||||
}
|
||||
local = await client.post("/api/v1/recall", json={"query": "drinks"})
|
||||
assert all("wife" not in hit["content"] for hit in local.json()["hits"])
|
||||
|
||||
answered = await client.post(
|
||||
"/api/v2/memories/ask", json={"question": "what does she drink", "user_id": "wife"}
|
||||
)
|
||||
assert answered.status_code == 200, answered.text
|
||||
assert {memory["content"] for memory in answered.json()["memories"]} <= {
|
||||
"wife drinks coffee",
|
||||
"wife walks the coffee route",
|
||||
}
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
async def _true() -> bool:
|
||||
return True
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
@pytest.mark.parametrize(
|
||||
("method", "path", "kwargs"),
|
||||
[
|
||||
("get", "/api/v2/memory-library", {"params": {"nobody": "1"}}),
|
||||
("get", "/api/v2/home-summary", {"params": {"userid": "wife"}}),
|
||||
("get", "/api/v2/user-profile", {"params": {"user": "wife"}}),
|
||||
("get", "/api/v2/raw", {"params": {"scope": "x"}}),
|
||||
("get", "/api/v2/memories/graph", {"params": {"scene": "all"}}),
|
||||
("get", "/api/v2/memories/some-id", {"params": {"owner": "wife"}}),
|
||||
("delete", "/api/v2/memories/some-id", {"params": {"owner": "wife"}}),
|
||||
("get", "/api/v2/memory-writes/some-id", {"params": {"owner": "wife"}}),
|
||||
("get", f"/api/v2/memories/additions/{uuid4()}", {"params": {"owner": "wife"}}),
|
||||
("post", "/api/v2/memories/ask", {"json": {"question": "q", "user": "wife"}}),
|
||||
("post", "/api/v1/recall", {"json": {"query": "q", "user": "wife"}}),
|
||||
("post", "/api/v1/recall", {"json": {"query": "q"}, "params": {"user_id": "wife"}}),
|
||||
("post", "/api/v2/context/search", {"json": {"query": "q", "owner": "wife"}}),
|
||||
(
|
||||
"post",
|
||||
"/api/v2/memories",
|
||||
{"json": {"request_id": str(uuid4()), "text": "t", "scope": "all-apps", "u": 1}},
|
||||
),
|
||||
("put", "/api/v2/memories/some-id", {"json": {"content": "c", "owner": "wife"}}),
|
||||
],
|
||||
)
|
||||
def test_unknown_query_and_body_parameters_answer_422(tmp_path, method, path, kwargs) -> None:
|
||||
async def scenario():
|
||||
app = application(tmp_path)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
response = await client.request(method.upper(), path, **kwargs)
|
||||
assert response.status_code == 422, response.text
|
||||
assert response.json()["detail"] == "request_validation_failed"
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
@pytest.mark.parametrize("bad", ["a:b", "", "x" * 65, "white space", "ü", "a/b"])
|
||||
def test_a_malformed_user_id_answers_422(tmp_path, bad) -> None:
|
||||
async def scenario():
|
||||
app = application(tmp_path)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
responses = [
|
||||
await client.get("/api/v2/memory-library", params={"user_id": bad}),
|
||||
await client.get("/api/v2/user-profile", params={"user_id": bad}),
|
||||
await client.post(
|
||||
"/api/v2/memories",
|
||||
json={
|
||||
"request_id": str(uuid4()),
|
||||
"text": "t",
|
||||
"scope": "all-apps",
|
||||
"user_id": bad,
|
||||
},
|
||||
),
|
||||
await client.post(
|
||||
"/api/v2/memory-writes",
|
||||
json={
|
||||
"request_id": "r",
|
||||
"session_id": "s",
|
||||
"messages": _messages("hello"),
|
||||
"user_id": bad,
|
||||
},
|
||||
),
|
||||
await client.post("/api/v1/recall", json={"query": "q", "user_id": bad}),
|
||||
]
|
||||
assert [response.status_code for response in responses] == [422] * len(responses)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_user_ids_and_lanes() -> None:
|
||||
assert valid_user_id("wife") and valid_user_id("A.b_c-9")
|
||||
assert not valid_user_id("a:b") and not valid_user_id("wife\n")
|
||||
assert lane_key("local", "hermes") == "hermes"
|
||||
assert lane_key("wife", "hermes") == "hermes@wife"
|
||||
assert split_lane("hermes@wife") == ("wife", "hermes")
|
||||
assert split_lane("hermes") == ("local", "hermes")
|
||||
# `local` keeps the session identity it always had; another user's never equals it.
|
||||
assert internal_session_id_for("s-1", "hermes") == "s-1"
|
||||
assert internal_session_id_for("s-1", "hermes", "wife") != "s-1"
|
||||
assert internal_session_id_for("s-1", "hermes", "wife") != internal_session_id_for(
|
||||
"s-1", "hermes", "kid"
|
||||
)
|
||||
|
||||
|
||||
def _hermes_state_db(root: Path, rows: list[tuple[int, str, str, str]]) -> None:
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
connection = sqlite3.connect(root / "state.db")
|
||||
connection.execute(
|
||||
"CREATE TABLE messages (id INTEGER PRIMARY KEY, session_id TEXT, role TEXT,"
|
||||
" content TEXT, timestamp REAL, tool_name TEXT, active INTEGER)"
|
||||
)
|
||||
connection.executemany(
|
||||
"INSERT INTO messages VALUES (?, ?, ?, ?, 1790000000, NULL, 1)",
|
||||
rows,
|
||||
)
|
||||
connection.commit()
|
||||
connection.close()
|
||||
|
||||
|
||||
def _hermes_homes(home: Path) -> tuple[Path, Path]:
|
||||
hermes = home / ".hermes"
|
||||
wife = hermes / "profiles" / "wife"
|
||||
# The same Hermes session id in both profiles: identities must not merge.
|
||||
_hermes_state_db(
|
||||
hermes,
|
||||
[(1, "session-1", "user", "I am the husband"), (2, "session-1", "assistant", "hi")],
|
||||
)
|
||||
_hermes_state_db(
|
||||
wife,
|
||||
[(1, "session-1", "user", "I am the wife"), (2, "session-1", "assistant", "hello")],
|
||||
)
|
||||
return hermes, wife
|
||||
|
||||
|
||||
def _scan_texts(root: Path, home: Path) -> list[str]:
|
||||
adapter = create_hermes_adapter(
|
||||
AuthorizedSourceRoot.grant(
|
||||
source_id=HERMES_DESCRIPTOR.source_id, root=root, user_confirmed=True, home=home
|
||||
)
|
||||
)
|
||||
|
||||
async def collect() -> list[str]:
|
||||
return [
|
||||
event.evidence.verbatim
|
||||
async for event in adapter.scan(ScanOptions(cursors={}))
|
||||
if isinstance(event, ScannedEvidence)
|
||||
]
|
||||
|
||||
return asyncio.run(collect())
|
||||
|
||||
|
||||
def test_a_hermes_root_scan_never_reads_its_profiles(tmp_path) -> None:
|
||||
hermes, wife = _hermes_homes(tmp_path)
|
||||
(hermes / "profiles" / "wife" / "sessions").mkdir()
|
||||
(hermes / "profiles" / "wife" / "sessions" / "s.jsonl").write_text(
|
||||
json.dumps({"type": "message", "role": "user", "content": "wife's transcript"}) + "\n"
|
||||
)
|
||||
|
||||
assert _scan_texts(hermes, tmp_path) == ["I am the husband", "hi"]
|
||||
assert _scan_texts(wife, tmp_path) == ["I am the wife", "hello", "wife's transcript"]
|
||||
|
||||
|
||||
def _jobs(database: Path) -> list[sqlite3.Row]:
|
||||
with sqlite3.connect(database) as db:
|
||||
db.row_factory = sqlite3.Row
|
||||
return db.execute(
|
||||
"SELECT session_key,source_key,idempotency_key,state,payload_json "
|
||||
"FROM memory_write_jobs ORDER BY queue_seq"
|
||||
).fetchall()
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_two_hermes_grants_each_read_their_own_root_into_separate_batches(tmp_path) -> None:
|
||||
async def scenario():
|
||||
home = tmp_path / "home"
|
||||
_hermes, wife = _hermes_homes(home)
|
||||
app = application(tmp_path, home=home)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
location = (await client.post("/api/v1/sources/hermes/location")).json()
|
||||
local = await client.post(
|
||||
"/api/v1/sources/connect",
|
||||
json={
|
||||
"source_ids": ["hermes"],
|
||||
"request_id": str(uuid4()),
|
||||
"source_roots": {"hermes": location["root_id"]},
|
||||
"enable_sync": True,
|
||||
},
|
||||
)
|
||||
assert local.status_code == 200, local.text
|
||||
profile = await client.post(
|
||||
"/api/v1/sources/connect",
|
||||
json={
|
||||
"source_ids": ["hermes"],
|
||||
"request_id": str(uuid4()),
|
||||
"user_id": "wife",
|
||||
"root": str(wife),
|
||||
},
|
||||
)
|
||||
assert profile.status_code == 200, profile.text
|
||||
[wife_job] = profile.json()["jobs"]
|
||||
assert wife_job["user_id"] == "wife"
|
||||
jobs = [*local.json()["jobs"], wife_job]
|
||||
for job in jobs:
|
||||
finished = await app.state.import_supervisor.wait(job["job_id"])
|
||||
assert finished.status == "completed", finished
|
||||
|
||||
rows = [row for row in _jobs(app.state.settings.data_dir / "one-memory.sqlite3")]
|
||||
assert rows, "history import wrote no batches"
|
||||
by_user: dict[str, list[str]] = {}
|
||||
for row in rows:
|
||||
records = json.loads(row["payload_json"])["input_records"]
|
||||
users = {record["user_id"] for record in records}
|
||||
assert len(users) == 1, "a batch mixed users"
|
||||
[user] = users
|
||||
by_user.setdefault(user, []).extend(record["verbatim"] for record in records)
|
||||
assert row["source_key"] == lane_key(user, "hermes")
|
||||
assert row["session_key"] == internal_session_id_for("session-1", "hermes", user)
|
||||
assert sorted(by_user["local"]) == ["I am the husband", "hi"]
|
||||
assert sorted(by_user["wife"]) == ["I am the wife", "hello"]
|
||||
|
||||
grants = (await client.get("/api/v1/source-grants")).json()["grants"]
|
||||
assert {(grant["user_id"], grant["source_id"]) for grant in grants} >= {
|
||||
("local", "hermes"),
|
||||
("wife", "hermes"),
|
||||
}
|
||||
# The App's view stays `local`'s.
|
||||
listed = (await client.get("/api/v1/import-jobs")).json()
|
||||
assert {job["user_id"] for job in listed} == {"local"}
|
||||
wife_jobs = (await client.get("/api/v1/import-jobs", params={"user_id": "wife"})).json()
|
||||
assert {job["user_id"] for job in wife_jobs} == {"wife"}
|
||||
assert all(
|
||||
item["source_id"] != "hermes@wife"
|
||||
for item in (await client.get("/api/v1/live-sync")).json()
|
||||
)
|
||||
|
||||
revoked = await client.post("/api/v1/sources/hermes/revoke", params={"user_id": "wife"})
|
||||
assert revoked.status_code == 200 and revoked.json()["revoked_at"] is not None
|
||||
local_grant = app.state.source_grant_store.receipt(user_id="local", source_id="hermes")
|
||||
assert local_grant.revoked_at is None
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_a_migrated_home_with_the_old_grant_triggers_can_connect_and_revoke(tmp_path) -> None:
|
||||
"""#233: a home an earlier build migrated kept two `source_grants` triggers
|
||||
whose body reads the dropped `governed_memory_evidence`. Every grant write
|
||||
then failed, so connecting and revoking answered 503. Startup drops them.
|
||||
"""
|
||||
stale = """
|
||||
CREATE TRIGGER memory_visibility_dirty_on_grants{suffix}
|
||||
AFTER {event} ON source_grants
|
||||
BEGIN
|
||||
INSERT INTO memory_visibility_dirty (memory_id)
|
||||
SELECT DISTINCT link.memory_id
|
||||
FROM governed_memory_evidence link
|
||||
JOIN evidence e ON e.evidence_id = link.evidence_id
|
||||
WHERE e.source_id = NEW.source_id
|
||||
ON CONFLICT(memory_id) DO NOTHING;
|
||||
END;"""
|
||||
|
||||
async def scenario():
|
||||
home = tmp_path / "home"
|
||||
_hermes, wife = _hermes_homes(home)
|
||||
app = application(tmp_path, home=home)
|
||||
async with app.router.lifespan_context(app):
|
||||
database = app.state.settings.data_dir / "one-memory.sqlite3"
|
||||
with sqlite3.connect(database) as db:
|
||||
db.execute("CREATE TABLE IF NOT EXISTS memory_visibility_dirty (memory_id TEXT)")
|
||||
db.execute(stale.format(suffix="", event="INSERT"))
|
||||
db.execute(stale.format(suffix="_update", event="UPDATE"))
|
||||
|
||||
app = application(tmp_path, home=home)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
location = (await client.post("/api/v1/sources/hermes/location")).json()
|
||||
local = await client.post(
|
||||
"/api/v1/sources/connect",
|
||||
json={
|
||||
"source_ids": ["hermes"],
|
||||
"request_id": str(uuid4()),
|
||||
"source_roots": {"hermes": location["root_id"]},
|
||||
},
|
||||
)
|
||||
assert local.status_code == 200, local.text
|
||||
profile = await client.post(
|
||||
"/api/v1/sources/connect",
|
||||
json={
|
||||
"source_ids": ["hermes"],
|
||||
"request_id": str(uuid4()),
|
||||
"user_id": "wife",
|
||||
"root": str(wife),
|
||||
},
|
||||
)
|
||||
assert profile.status_code == 200, profile.text
|
||||
for user_id in ("wife", "local"):
|
||||
revoked = await client.post(
|
||||
"/api/v1/sources/hermes/revoke", params={"user_id": user_id}
|
||||
)
|
||||
assert revoked.status_code == 200, revoked.text
|
||||
assert revoked.json()["revoked_at"] is not None
|
||||
with sqlite3.connect(database) as db:
|
||||
assert (
|
||||
db.execute(
|
||||
"SELECT COUNT(*) FROM sqlite_master "
|
||||
"WHERE type='trigger' AND tbl_name='source_grants'"
|
||||
).fetchone()[0]
|
||||
== 0
|
||||
)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_only_hermes_takes_another_user_or_a_custom_root(tmp_path) -> None:
|
||||
async def scenario():
|
||||
home = tmp_path / "home"
|
||||
(home / ".claude" / "projects").mkdir(parents=True)
|
||||
app = application(tmp_path, home=home)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
for extra in ({"user_id": "wife"}, {"root": str(home / ".claude" / "projects")}):
|
||||
response = await client.post(
|
||||
"/api/v1/sources/connect",
|
||||
json={
|
||||
"source_ids": ["claude-code"],
|
||||
"request_id": str(uuid4()),
|
||||
"source_roots": {},
|
||||
**extra,
|
||||
},
|
||||
)
|
||||
assert response.status_code == 422, response.text
|
||||
assert response.json()["detail"] == "user_id_requires_multi_user_source"
|
||||
revoke = await client.post(
|
||||
"/api/v1/sources/claude-code/revoke", params={"user_id": "wife"}
|
||||
)
|
||||
assert revoke.status_code == 422
|
||||
# Hermes too: another user must name their own root.
|
||||
rootless = await client.post(
|
||||
"/api/v1/sources/connect",
|
||||
json={
|
||||
"source_ids": ["hermes"],
|
||||
"request_id": str(uuid4()),
|
||||
"source_roots": {},
|
||||
"user_id": "wife",
|
||||
},
|
||||
)
|
||||
assert rootless.status_code == 422
|
||||
assert rootless.json()["detail"] == "source_root_required_for_user"
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def _record(user_id: str, message_id: str, role: str, text: str):
|
||||
return materialize_write_input(
|
||||
EvidenceInput(
|
||||
user_id=user_id,
|
||||
source_id="hermes",
|
||||
conversation_id="session-1",
|
||||
message_id=message_id,
|
||||
role=role,
|
||||
written_by=WrittenBy.PERSON if role == "user" else WrittenBy.AGENT,
|
||||
created_at=datetime(2026, 10, 1, tzinfo=UTC),
|
||||
verbatim=text,
|
||||
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def test_assembling_rows_and_batches_are_keyed_by_user(tmp_path) -> None:
|
||||
"""Two users' `state.db` share a relative file key and a session id.
|
||||
|
||||
Finishing one user's file must neither take nor drop the other user's
|
||||
staged turns, and the batches it writes hold one user each.
|
||||
"""
|
||||
path = tmp_path / "one-memory.sqlite3"
|
||||
queue = DurableWriteQueue(path, sdk_authority=True)
|
||||
file_key = "f" * 64
|
||||
from one_memory.database_access import main_database
|
||||
|
||||
with main_database(path).write(row_factory=sqlite3.Row) as connection, connection:
|
||||
for user in ("local", "wife"):
|
||||
for index, role in enumerate(("user", "assistant")):
|
||||
queue.stage_sdk_input(
|
||||
_record(user, f"m{index}", role, f"{user} {role}"),
|
||||
file_key=file_key,
|
||||
priority="normal",
|
||||
connection=connection,
|
||||
)
|
||||
queue.finalize_sdk_file(
|
||||
user_id="local", source_id="hermes", file_key=file_key, connection=connection
|
||||
)
|
||||
|
||||
rows = _jobs(path)
|
||||
pending = [row for row in rows if row["state"] == "pending"]
|
||||
assembling = [row for row in rows if row["state"] == "assembling"]
|
||||
assert [
|
||||
{record["user_id"] for record in json.loads(row["payload_json"])["input_records"]}
|
||||
for row in pending
|
||||
] == [{"local"}]
|
||||
assert [split_lane(row["source_key"]) for row in assembling] == [("wife", "hermes")]
|
||||
assert pending[0]["session_key"] != assembling[0]["session_key"]
|
||||
|
||||
|
||||
def _observed(response) -> tuple[int, bytes]:
|
||||
return response.status_code, response.content
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_another_users_id_answers_exactly_like_an_id_that_does_not_exist(
|
||||
tmp_path, monkeypatch
|
||||
) -> None:
|
||||
"""Comparing responses must not tell whether another user's id exists."""
|
||||
|
||||
async def scenario(app, sdk, client):
|
||||
receipts = await _write_both_users(app, sdk, client)
|
||||
[wife_memory] = [
|
||||
memory_id
|
||||
for memory_id, row in sdk.memories.items()
|
||||
if row["user_id"] == "wife" and row["content"] == "wife drinks coffee"
|
||||
]
|
||||
before = {key: dict(row) for key, row in sdk.memories.items()}
|
||||
with sqlite3.connect(app.state.settings.data_dir / "one-memory.sqlite3") as db:
|
||||
keys = [row[0] for row in db.execute("SELECT idempotency_key FROM memory_write_jobs")]
|
||||
wife_key = next(key for key in keys if key.startswith("user:wife:"))
|
||||
|
||||
pairs = [
|
||||
("GET", f"/api/v2/memories/{wife_memory}", "/api/v2/memories/no-such-memory", {}),
|
||||
(
|
||||
"PUT",
|
||||
f"/api/v2/memories/{wife_memory}",
|
||||
"/api/v2/memories/no-such-memory",
|
||||
{"json": {"content": "stolen"}},
|
||||
),
|
||||
("DELETE", f"/api/v2/memories/{wife_memory}", "/api/v2/memories/no-such-memory", {}),
|
||||
(
|
||||
"GET",
|
||||
f"/api/v2/memories/additions/{receipts['add:wife']}",
|
||||
f"/api/v2/memories/additions/{uuid4()}",
|
||||
{},
|
||||
),
|
||||
("GET", f"/api/v2/memory-writes/{wife_key}", "/api/v2/memory-writes/no-such-id", {}),
|
||||
]
|
||||
for method, theirs, missing, kwargs in pairs:
|
||||
other = await client.request(method, theirs, **kwargs)
|
||||
absent = await client.request(method, missing, **kwargs)
|
||||
assert _observed(other) == _observed(absent), (method, theirs)
|
||||
assert other.status_code == (204 if method == "DELETE" else 404)
|
||||
assert sdk.memories == before
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
@@ -173,12 +173,14 @@ def test_first_import_gate_waits_for_every_selected_history_job() -> None:
|
||||
jobs = [
|
||||
SimpleNamespace(
|
||||
operation="history",
|
||||
user_id="local",
|
||||
source_id="cursor",
|
||||
created_at=datetime(2026, 10, 18, tzinfo=UTC),
|
||||
status="completed",
|
||||
),
|
||||
SimpleNamespace(
|
||||
operation="history",
|
||||
user_id="local",
|
||||
source_id="codex",
|
||||
created_at=datetime(2026, 10, 18, tzinfo=UTC),
|
||||
status="running",
|
||||
|
||||
@@ -40,6 +40,7 @@ def library(tmp_path):
|
||||
provider_kind=ProviderKind.OLLAMA,
|
||||
model="hy-bot-phi4",
|
||||
embedding_model="nomic-embed-text",
|
||||
embedding_base_url="http://127.0.0.1:11434",
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
|
||||
@@ -210,7 +210,7 @@ def test_recall_route_falls_back_and_keeps_the_raw_grant(tmp_path, monkeypatch):
|
||||
async def no_model(_app):
|
||||
return False
|
||||
|
||||
async def raw(_app, query, *, limit, sources):
|
||||
async def raw(_app, query, *, limit, sources, user_id="local"):
|
||||
searched.append(sources)
|
||||
return (_raw("r1", "上周决定用 Postgres", source="cursor"),)
|
||||
|
||||
|
||||
@@ -0,0 +1,744 @@
|
||||
"""The SDK surface MemHub exposes: search pass-through, layers, raw, v2 aliases.
|
||||
|
||||
P2-1 (R8): `context/search`, `recall` and the v2 ask routes pass the public
|
||||
`client.search` knobs through, validated, and keep a caller's floor on what
|
||||
they return. P2-2 (R9): the library honours `layers`, `statuses`, `scene` and
|
||||
the source/session/agent filters, and counts memories per layer. P2-4: raw
|
||||
lists the SDK's L1 raw turns, active and shadow. P2-3: v2 aliases of the
|
||||
source and import routes, per user. Profile versions read the L0 chain.
|
||||
|
||||
Every route stays scoped to its one `user_id`; the stand-in client records
|
||||
each call so a test can see exactly what reached the SDK.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import sqlite3
|
||||
from contextlib import asynccontextmanager
|
||||
from pathlib import Path
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
from pydantic import SecretStr
|
||||
|
||||
from one_memory.api import create_app
|
||||
from one_memory.config import Settings
|
||||
from one_memory.imports.sources.registry import build_source_catalog
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
TOKEN = "sdk-surface-test"
|
||||
|
||||
CHAT_ROWS = {
|
||||
"local": [
|
||||
("m-high", "local likes green tea", 0.95, "l2_fact", "hermes"),
|
||||
("m-mid", "local walks at dawn", 0.6, "l2_fact", "codex"),
|
||||
],
|
||||
"wife": [("w-high", "wife likes coffee", 0.92, "l2_fact", "hermes")],
|
||||
}
|
||||
RAW_ROWS = {
|
||||
"local": [
|
||||
{
|
||||
"memory_id": "raw-1",
|
||||
"content": "[user]: 我叫费天翔\n[assistant]: 好的,记住了。",
|
||||
"status": "shadow",
|
||||
"session_id": "s-1",
|
||||
"source": "hermes",
|
||||
"memory_at": 1_790_000_000,
|
||||
"gmt_created": 1_790_000_100,
|
||||
},
|
||||
{
|
||||
"memory_id": "raw-2",
|
||||
"content": "[user]: 你好",
|
||||
"status": "active",
|
||||
"session_id": "s-2",
|
||||
"source": "codex",
|
||||
"memory_at": 1_790_100_000,
|
||||
"gmt_created": 1_790_100_100,
|
||||
},
|
||||
],
|
||||
"wife": [
|
||||
{
|
||||
"memory_id": "raw-w",
|
||||
"content": "[user]: I am the wife",
|
||||
"status": "active",
|
||||
"session_id": "s-w",
|
||||
"source": "hermes",
|
||||
"memory_at": 1_790_000_000,
|
||||
"gmt_created": 1_790_000_000,
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _chat(user_id: str, memory_id: str, content: str, layer: str, source: str) -> dict:
|
||||
return {
|
||||
"memory_id": memory_id,
|
||||
"content": content,
|
||||
"layer": layer,
|
||||
"status": "active",
|
||||
"user_id": user_id,
|
||||
"agent_id": "all-apps",
|
||||
"session_id": "s-1",
|
||||
"source": source,
|
||||
"tags": [],
|
||||
"gmt_created": 1_790_000_000,
|
||||
}
|
||||
|
||||
|
||||
class RecordingSdk:
|
||||
"""The public client surface, recording every call's keyword arguments."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.calls: list[tuple[str, dict]] = []
|
||||
|
||||
def _record(self, name: str, kwargs: dict) -> None:
|
||||
self.calls.append((name, dict(kwargs)))
|
||||
|
||||
def last(self, name: str) -> dict:
|
||||
return [kwargs for called, kwargs in self.calls if called == name][-1]
|
||||
|
||||
def all(self, name: str) -> list[dict]:
|
||||
return [kwargs for called, kwargs in self.calls if called == name]
|
||||
|
||||
# The SDK answers rows below a caller's floor in some lanes (a schema
|
||||
# floor, a fused rank score), so Core must filter what it returns.
|
||||
def search(self, query, **kwargs):
|
||||
self._record("search", {"query": query, **kwargs})
|
||||
[user_id] = kwargs["user_ids"]
|
||||
if kwargs.get("layers") == ["l1_raw"]:
|
||||
hits = [
|
||||
{**row, "user_id": user_id, "agent_id": "all-apps", "score": score}
|
||||
for row, score in zip(RAW_ROWS.get(user_id, []), (0.97, 0.3), strict=False)
|
||||
]
|
||||
return {"request_id": "s", "memories": hits}
|
||||
rows = [
|
||||
{**_chat(user_id, mid, text, layer, source), "score": score}
|
||||
for mid, text, score, layer, source in CHAT_ROWS.get(user_id, [])
|
||||
]
|
||||
scene = kwargs.get("scene")
|
||||
if scene == "all":
|
||||
fused = [
|
||||
{**row, "scene": "chat", "kind": "chat", "lane_score": row["score"], "score": 0.016}
|
||||
for row in rows
|
||||
]
|
||||
return {"request_id": "s", "scene": "all", "memories": fused, "lanes": {}, "errors": {}}
|
||||
if scene in {"working", "coding"}:
|
||||
return {
|
||||
"request_id": "s",
|
||||
"scene": "productivity",
|
||||
"variant": scene,
|
||||
"facts": [
|
||||
{"fact_id": "f-1", "statement": "use uv", "score": 0.91, "user_id": user_id}
|
||||
],
|
||||
"experiences": [],
|
||||
}
|
||||
return {
|
||||
"request_id": "s",
|
||||
"scene": "normal",
|
||||
"memories": {"normal": rows, "schema": [], "proactive": []},
|
||||
}
|
||||
|
||||
def get(self, memory_id, kind=None, variant=None):
|
||||
self._record("get", {"memory_id": memory_id, "kind": kind, "variant": variant})
|
||||
if kind == "fact":
|
||||
return {
|
||||
"kind": "fact",
|
||||
"variant": variant,
|
||||
"fact_id": memory_id,
|
||||
"statement": "use uv",
|
||||
"status": "active",
|
||||
"user_id": "local",
|
||||
}
|
||||
for user_id, rows in CHAT_ROWS.items():
|
||||
for mid, text, _score, layer, source in rows:
|
||||
if mid == memory_id:
|
||||
return {**_chat(user_id, mid, text, layer, source), "kind": "chat"}
|
||||
return None
|
||||
|
||||
def list_memories(self, **kwargs):
|
||||
self._record("list_memories", kwargs)
|
||||
user_id = kwargs["user_id"]
|
||||
layers = kwargs.get("layers")
|
||||
if layers == ["l1_raw"]:
|
||||
statuses = set(kwargs.get("statuses") or ["active"])
|
||||
rows = [
|
||||
{**row, "layer": "l1_raw", "user_id": user_id, "agent_id": "all-apps"}
|
||||
for row in RAW_ROWS.get(user_id, [])
|
||||
if row["status"] in statuses
|
||||
and (
|
||||
not kwargs.get("include_sources") or row["source"] in kwargs["include_sources"]
|
||||
)
|
||||
and (not kwargs.get("session_ids") or row["session_id"] in kwargs["session_ids"])
|
||||
]
|
||||
return {"vdb": {"memories": rows, "total": len(rows)}}
|
||||
rows = [_chat(user_id, *row[:2], *row[3:]) for row in CHAT_ROWS.get(user_id, [])]
|
||||
if layers:
|
||||
rows = [row for row in rows if row["layer"] in layers]
|
||||
if "l0_basic_info" in layers:
|
||||
rows.append(_chat(user_id, "l0-1", "name is Fei", "l0_basic_info", "hermes"))
|
||||
if kwargs.get("scene") == "all":
|
||||
return {
|
||||
"scene": "all",
|
||||
"memories": [{**row, "scene": "chat", "kind": "chat"} for row in rows],
|
||||
"total": len(rows),
|
||||
"totals": {"chat": len(rows), "working": 0, "coding": 0},
|
||||
"errors": {},
|
||||
}
|
||||
return {"vdb": {"memories": rows, "total": len(rows)}}
|
||||
|
||||
def memory_stats(self, *, user_id, statuses=None):
|
||||
return {"total": 0, "request_id": "stats", "elapsed_ms": 1.0}
|
||||
|
||||
def get_user_basic_info(self, *, user_id):
|
||||
self._record("get_user_basic_info", {"user_id": user_id})
|
||||
return {
|
||||
"request_id": f"basic-{user_id}",
|
||||
"elapsed_ms": 1.5,
|
||||
"memories": [
|
||||
{
|
||||
"memory_id": f"{user_id}-l0-head",
|
||||
"content": "The user's name is Fei.",
|
||||
"layer": "l0_basic_info",
|
||||
"status": "active",
|
||||
"memory_at": 1_790_000_100,
|
||||
"gmt_created": 1_790_000_100,
|
||||
"tags": [],
|
||||
"basic_info_kv": {"name": "Fei"},
|
||||
"evolution_chain": [
|
||||
{
|
||||
"node_id": f"{user_id}-l0-head",
|
||||
"content": "The user's name is Fei.",
|
||||
"memory_at": "2026-10-31 10:00",
|
||||
"gmt_created": 1_790_000_100,
|
||||
"layer": "l0_basic_info",
|
||||
},
|
||||
{
|
||||
"node_id": f"{user_id}-l0-old",
|
||||
"content": "The user's name is 费天翔.",
|
||||
"memory_at": "2026-10-30 10:00",
|
||||
"gmt_created": 1_790_000_000,
|
||||
"layer": "l0_basic_info",
|
||||
},
|
||||
],
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
class Handle:
|
||||
def __init__(self, sdk: RecordingSdk) -> None:
|
||||
self.sdk = sdk
|
||||
|
||||
async def client(self) -> RecordingSdk:
|
||||
return self.sdk
|
||||
|
||||
async def close(self) -> None:
|
||||
return None
|
||||
|
||||
|
||||
def application(tmp_path: Path, *, home: Path | None = None):
|
||||
settings = Settings(
|
||||
data_dir=tmp_path / "data",
|
||||
source_home=home or tmp_path,
|
||||
api_token=SecretStr(TOKEN),
|
||||
imports_enabled=home is not None,
|
||||
automatic_memory_processing=False,
|
||||
live_sync_enabled=False,
|
||||
)
|
||||
return create_app(
|
||||
settings,
|
||||
secret_store=InMemorySecretStore(),
|
||||
source_catalog=(
|
||||
build_source_catalog(home=home, environment={}, platform="darwin")
|
||||
if home is not None
|
||||
else None
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def client_for(app) -> AsyncClient:
|
||||
return AsyncClient(
|
||||
transport=ASGITransport(app=app),
|
||||
base_url="http://test",
|
||||
headers={"Authorization": f"Bearer {TOKEN}"},
|
||||
)
|
||||
|
||||
|
||||
async def _model_ready(_app) -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def run_with_sdk(tmp_path, monkeypatch, scenario) -> None:
|
||||
async def main() -> None:
|
||||
app = application(tmp_path)
|
||||
sdk = RecordingSdk()
|
||||
|
||||
@asynccontextmanager
|
||||
async def write_client():
|
||||
yield sdk
|
||||
|
||||
async with app.router.lifespan_context(app):
|
||||
app.state.sdk_client_handle = Handle(sdk)
|
||||
monkeypatch.setattr(app.state.sdk_write_coordinator, "use_client", write_client)
|
||||
monkeypatch.setattr("one_memory.api._recall_model_ready", _model_ready)
|
||||
async with client_for(app) as client:
|
||||
await scenario(app, sdk, client)
|
||||
|
||||
asyncio.run(main())
|
||||
|
||||
|
||||
PASS_THROUGH = {
|
||||
"min_score": 0.5,
|
||||
"schema_min_score": 0.6,
|
||||
"working_fact_min_score": 0.3,
|
||||
"working_experience_min_score": 0.45,
|
||||
"schema_limit": 4,
|
||||
"intention_limit": 2,
|
||||
"rerank": False,
|
||||
"reader": "legacy",
|
||||
"scene": "chat",
|
||||
"search_mode": "agentic",
|
||||
"include_sources": ["hermes", "codex"],
|
||||
"exclude_sources": ["cursor"],
|
||||
"created_after": 1_780_000_000,
|
||||
"session_ids": ["s-1"],
|
||||
"agent_ids": ["all-apps"],
|
||||
"workspace_id": "ws-1",
|
||||
"branch": "main",
|
||||
"layers": ["l2_fact", "l0_basic_info"],
|
||||
}
|
||||
#: How each field reaches `client.search` (lists stay lists).
|
||||
EXPECTED_SEARCH = {**PASS_THROUGH}
|
||||
|
||||
|
||||
def _memory_search(sdk: RecordingSdk) -> dict:
|
||||
return [call for call in sdk.all("search") if call.get("layers") != ["l1_raw"]][-1]
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_every_context_search_option_reaches_the_sdk_and_absent_ones_keep_the_old_call(
|
||||
tmp_path, monkeypatch
|
||||
) -> None:
|
||||
async def scenario(app, sdk, client):
|
||||
baseline = await client.post(
|
||||
"/api/v2/context/search", json={"query": "tea", "kinds": ["memory"]}
|
||||
)
|
||||
assert baseline.status_code == 200, baseline.text
|
||||
before = _memory_search(sdk)
|
||||
assert before["min_score"] == before["schema_min_score"] == 0.35
|
||||
assert before["scene"] == "normal" and before["reader"] == "hybrid"
|
||||
assert before["schema_limit"] == 3 and before["intention_limit"] == 1
|
||||
for absent in ("rerank", "search_mode", "layers", "session_ids", "created_after"):
|
||||
assert absent not in before
|
||||
|
||||
response = await client.post(
|
||||
"/api/v2/context/search",
|
||||
json={"query": "tea", "kinds": ["memory"], **PASS_THROUGH},
|
||||
)
|
||||
assert response.status_code == 200, response.text
|
||||
sent = _memory_search(sdk)
|
||||
for name, value in EXPECTED_SEARCH.items():
|
||||
assert sent[name] == value, name
|
||||
assert sent["user_ids"] == ["local"]
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_every_recall_and_ask_option_reaches_the_sdk(tmp_path, monkeypatch) -> None:
|
||||
async def scenario(app, sdk, client):
|
||||
recalled = await client.post(
|
||||
"/api/v1/recall", json={"query": "tea", "user_id": "wife", **PASS_THROUGH}
|
||||
)
|
||||
assert recalled.status_code == 200, recalled.text
|
||||
sent = _memory_search(sdk)
|
||||
for name, value in EXPECTED_SEARCH.items():
|
||||
assert sent[name] == value, name
|
||||
assert sent["user_ids"] == ["wife"]
|
||||
|
||||
ask_options = {
|
||||
name: value
|
||||
for name, value in PASS_THROUGH.items()
|
||||
if name
|
||||
not in {
|
||||
"working_fact_min_score",
|
||||
"working_experience_min_score",
|
||||
"schema_limit",
|
||||
"intention_limit",
|
||||
"reader",
|
||||
"scene",
|
||||
"search_mode",
|
||||
"workspace_id",
|
||||
"branch",
|
||||
}
|
||||
}
|
||||
asked = await client.post(
|
||||
"/api/v2/memories/ask", json={"question": "tea?", "user_id": "wife", **ask_options}
|
||||
)
|
||||
assert asked.status_code == 200, asked.text
|
||||
sent = _memory_search(sdk)
|
||||
for name, value in ask_options.items():
|
||||
assert sent[name] == value, name
|
||||
assert sent["user_ids"] == ["wife"] and sent["scene"] == "normal"
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_min_score_is_enforced_on_what_core_returns(tmp_path, monkeypatch) -> None:
|
||||
"""Acceptance item 9: `min_score=0.9` answers only results scoring ≥ 0.9."""
|
||||
|
||||
async def scenario(app, sdk, client):
|
||||
context = await client.post(
|
||||
"/api/v2/context/search", json={"query": "tea", "min_score": 0.9}
|
||||
)
|
||||
assert context.status_code == 200, context.text
|
||||
items = context.json()["items"]
|
||||
assert items and all(item["score"] >= 0.9 for item in items)
|
||||
assert {item.get("memory_id") or item.get("raw_id") for item in items} == {
|
||||
"m-high",
|
||||
"raw-1",
|
||||
}
|
||||
raw_call = sdk.last("search") if sdk.last("search").get("layers") else None
|
||||
assert raw_call is None or raw_call["min_score"] == 0.9
|
||||
|
||||
# The fused scene ranks by 1/(60+rank); the lane's own score is the floor's.
|
||||
fused = await client.post(
|
||||
"/api/v2/context/search",
|
||||
json={"query": "tea", "kinds": ["memory"], "scene": "all", "min_score": 0.9},
|
||||
)
|
||||
assert [item["memory_id"] for item in fused.json()["items"]] == ["m-high"]
|
||||
assert fused.json()["items"][0]["score"] == 0.95
|
||||
|
||||
recalled = await client.post("/api/v1/recall", json={"query": "tea", "min_score": 0.9})
|
||||
assert recalled.status_code == 200, recalled.text
|
||||
assert [hit["memory_id"] for hit in recalled.json()["hits"]] == ["m-high"]
|
||||
|
||||
# A working-scene fact is read back from its own namespace.
|
||||
working = await client.post(
|
||||
"/api/v2/context/search",
|
||||
json={"query": "uv", "kinds": ["memory"], "scene": "working", "min_score": 0.9},
|
||||
)
|
||||
assert [item["memory_id"] for item in working.json()["items"]] == ["f-1"]
|
||||
assert sdk.last("get") == {"memory_id": "f-1", "kind": "fact", "variant": "working"}
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
@pytest.mark.parametrize(
|
||||
"extra",
|
||||
[
|
||||
{"min_score": 1.5},
|
||||
{"min_score": -0.1},
|
||||
{"schema_min_score": "high"},
|
||||
{"scene": "normal"},
|
||||
{"search_mode": "fast"},
|
||||
{"reader": "vector"},
|
||||
{"layers": ["l9_nothing"]},
|
||||
{"layers": []},
|
||||
{"branch": "main"},
|
||||
{"rerank": "no"},
|
||||
{"created_after": -1},
|
||||
{"user_ids": ["wife"]},
|
||||
{"embed_override": {"model": "x"}},
|
||||
{"query_embedding": [0.1]},
|
||||
{"request_id": "r-1"},
|
||||
],
|
||||
)
|
||||
def test_out_of_range_and_unexposed_search_fields_answer_422(tmp_path, extra) -> None:
|
||||
async def scenario():
|
||||
app = application(tmp_path)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
for path, body in (
|
||||
("/api/v2/context/search", {"query": "q"}),
|
||||
("/api/v1/recall", {"query": "q"}),
|
||||
("/api/v2/memories/ask", {"question": "q"}),
|
||||
):
|
||||
response = await client.post(path, json={**body, **extra})
|
||||
assert response.status_code == 422, (path, response.text)
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_raw_lane_takes_the_callers_sources_only_inside_its_authority(
|
||||
tmp_path, monkeypatch
|
||||
) -> None:
|
||||
async def scenario(app, sdk, client):
|
||||
response = await client.post(
|
||||
"/api/v2/context/search",
|
||||
json={
|
||||
"query": "tea",
|
||||
"kinds": ["raw"],
|
||||
"include_sources": ["hermes"],
|
||||
"session_ids": ["s-1"],
|
||||
"created_after": 1_700_000_000,
|
||||
},
|
||||
)
|
||||
assert response.status_code == 200, response.text
|
||||
raw_call = sdk.last("search")
|
||||
assert raw_call["layers"] == ["l1_raw"]
|
||||
assert raw_call["include_sources"] == ["hermes"]
|
||||
assert raw_call["session_ids"] == ["s-1"]
|
||||
assert raw_call["created_after"] == 1_700_000_000
|
||||
# Memory-lane knobs never reach the raw search.
|
||||
assert raw_call["scene"] == "chat" and raw_call["schema_limit"] == 0
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_library_honours_layers_statuses_and_filters(tmp_path, monkeypatch) -> None:
|
||||
"""Acceptance item 10: `layer=l0_basic_info` returns only L0 records."""
|
||||
|
||||
async def scenario(app, sdk, client):
|
||||
page = await client.get("/api/v2/memory-library", params={"layer": "l0_basic_info"})
|
||||
assert page.status_code == 200, page.text
|
||||
assert {memory["layer"] for memory in page.json()["memories"]} == {"l0_basic_info"}
|
||||
sent = sdk.last("list_memories")
|
||||
assert sent["layers"] == ["l0_basic_info"] and sent["scene"] is None
|
||||
|
||||
await client.get(
|
||||
"/api/v2/memory-library",
|
||||
params=[
|
||||
("layers", "l2_fact,l6_schema"),
|
||||
("layers", "l7_intention"),
|
||||
("statuses", "active,superseded"),
|
||||
("session_ids", "s-1"),
|
||||
("agent_ids", "all-apps"),
|
||||
("exclude_sources", "cursor"),
|
||||
("user_id", "wife"),
|
||||
],
|
||||
)
|
||||
sent = sdk.last("list_memories")
|
||||
assert sent["layers"] == ["l2_fact", "l6_schema", "l7_intention"]
|
||||
assert sent["statuses"] == ["active", "superseded"]
|
||||
assert sent["session_ids"] == ["s-1"] and sent["agent_ids"] == ["all-apps"]
|
||||
assert sent["exclude_sources"] == ["cursor"] and sent["user_id"] == "wife"
|
||||
|
||||
# Without a layer the page is what it always was.
|
||||
await client.get("/api/v2/memory-library")
|
||||
sent = sdk.last("list_memories")
|
||||
assert sent["scene"] == "all" and "layers" not in sent
|
||||
|
||||
searched = await client.get(
|
||||
"/api/v2/memory-library", params={"query": "tea", "layers": "l2_fact"}
|
||||
)
|
||||
assert searched.status_code == 200, searched.text
|
||||
assert sdk.last("search")["layers"] == ["l2_fact"]
|
||||
|
||||
for params in (
|
||||
{"layers": "l9_nothing"},
|
||||
{"statuses": "gone"},
|
||||
{"layers": "l2_fact", "scene": "working"},
|
||||
):
|
||||
refused = await client.get("/api/v2/memory-library", params=params)
|
||||
assert refused.status_code == 422, (params, refused.text)
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_layer_counts_read_each_layer_for_one_user(tmp_path, monkeypatch) -> None:
|
||||
async def scenario(app, sdk, client):
|
||||
counts = await client.get("/api/v2/memory-library/layers", params={"user_id": "wife"})
|
||||
assert counts.status_code == 200, counts.text
|
||||
body = counts.json()
|
||||
assert body["layers"]["l1_raw"] == 1
|
||||
assert body["layers"]["l2_fact"] == 1
|
||||
assert body["layers"]["l0_basic_info"] == 1
|
||||
assert body["total"] == sum(body["layers"].values())
|
||||
calls = sdk.all("list_memories")
|
||||
assert {call["user_id"] for call in calls} == {"wife"}
|
||||
[raw_call] = [call for call in calls if call["layers"] == ["l1_raw"]]
|
||||
assert raw_call["statuses"] == ["active", "shadow"]
|
||||
assert (
|
||||
await client.get("/api/v2/memory-library/layers", params={"layer": "x"})
|
||||
).status_code == 422
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_raw_lists_active_and_shadow_turns_with_roles_and_hashes(tmp_path, monkeypatch) -> None:
|
||||
async def scenario(app, sdk, client):
|
||||
page = await client.get("/api/v2/raw")
|
||||
assert page.status_code == 200, page.text
|
||||
body = page.json()
|
||||
assert body["total"] == 2 and len(body["items"]) == 2
|
||||
sent = sdk.last("list_memories")
|
||||
assert sent["layers"] == ["l1_raw"] and sent["statuses"] == ["active", "shadow"]
|
||||
first = next(item for item in body["items"] if item["raw_id"] == "raw-1")
|
||||
assert first["status"] == "shadow"
|
||||
assert [message["role"] for message in first["messages"]] == ["user", "assistant"]
|
||||
assert first["messages"][0]["content"] == "我叫费天翔"
|
||||
assert first["role"] is None
|
||||
expected = hashlib.sha256(first["content"].encode()).hexdigest()
|
||||
assert first["content_hash"] == f"sha256:{expected}"
|
||||
assert first["session_id"] == "s-1" and first["source"] == "hermes"
|
||||
assert first["created"].startswith("2026-")
|
||||
second = next(item for item in body["items"] if item["raw_id"] == "raw-2")
|
||||
assert second["role"] == "user"
|
||||
|
||||
await client.get(
|
||||
"/api/v2/raw",
|
||||
params=[("source", "hermes,codex"), ("session_id", "s-1"), ("order", "asc")],
|
||||
)
|
||||
sent = sdk.last("list_memories")
|
||||
assert sent["include_sources"] == ["hermes", "codex"]
|
||||
assert sent["session_ids"] == ["s-1"] and sent["order"] == "asc"
|
||||
|
||||
windowed = (await client.get("/api/v2/raw", params={"after": 1_790_050_000})).json()
|
||||
assert windowed["time_filtered"] is True
|
||||
assert [item["raw_id"] for item in windowed["items"]] == ["raw-2"]
|
||||
|
||||
wife = (await client.get("/api/v2/raw", params={"user_id": "wife"})).json()
|
||||
assert [item["raw_id"] for item in wife["items"]] == ["raw-w"]
|
||||
assert sdk.last("list_memories")["user_id"] == "wife"
|
||||
assert (await client.get("/api/v2/raw", params={"statuses": "bogus"})).status_code == 422
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_profile_versions_read_the_l0_chain_for_one_user(tmp_path, monkeypatch) -> None:
|
||||
async def scenario(app, sdk, client):
|
||||
response = await client.get("/api/v2/user-profile/versions", params={"user_id": "wife"})
|
||||
assert response.status_code == 200, response.text
|
||||
body = response.json()
|
||||
assert body["user_id"] == "wife"
|
||||
assert sdk.last("get_user_basic_info") == {"user_id": "wife"}
|
||||
[chain] = body["chains"]
|
||||
assert chain["memory_id"] == "wife-l0-head"
|
||||
assert chain["basic_info"] == {"name": "Fei"}
|
||||
assert [(v["memory_id"], v["current"]) for v in chain["versions"]] == [
|
||||
("wife-l0-head", True),
|
||||
("wife-l0-old", False),
|
||||
]
|
||||
local = (await client.get("/api/v2/user-profile/versions")).json()
|
||||
assert local["user_id"] == "local"
|
||||
assert sdk.last("get_user_basic_info") == {"user_id": "local"}
|
||||
refused = await client.get("/api/v2/user-profile/versions", params={"owner": "wife"})
|
||||
assert refused.status_code == 422
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_graph_takes_layers_and_session_filters(tmp_path, monkeypatch) -> None:
|
||||
async def scenario(app, sdk, client):
|
||||
response = await client.get(
|
||||
"/api/v2/memories/graph",
|
||||
params={"layers": "l2_fact", "session_ids": "s-1", "user_id": "wife"},
|
||||
)
|
||||
assert response.status_code == 200, response.text
|
||||
sent = sdk.last("list_memories")
|
||||
assert sent["layers"] == ["l2_fact"] and sent["scene"] is None
|
||||
assert sent["session_ids"] == ["s-1"] and sent["user_id"] == "wife"
|
||||
|
||||
run_with_sdk(tmp_path, monkeypatch, scenario)
|
||||
|
||||
|
||||
def _hermes_state_db(root: Path, text: str) -> None:
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
connection = sqlite3.connect(root / "state.db")
|
||||
connection.execute(
|
||||
"CREATE TABLE messages (id INTEGER PRIMARY KEY, session_id TEXT, role TEXT,"
|
||||
" content TEXT, timestamp REAL, tool_name TEXT, active INTEGER)"
|
||||
)
|
||||
connection.executemany(
|
||||
"INSERT INTO messages VALUES (?, ?, ?, ?, 1790000000, NULL, 1)",
|
||||
[(1, "session-1", "user", text), (2, "session-1", "assistant", "ok")],
|
||||
)
|
||||
connection.commit()
|
||||
connection.close()
|
||||
|
||||
|
||||
@pytest.mark.fresh_sdk
|
||||
def test_v2_source_and_import_aliases_behave_like_v1_per_user(tmp_path) -> None:
|
||||
async def scenario():
|
||||
home = tmp_path / "home"
|
||||
hermes = home / ".hermes"
|
||||
wife = hermes / "profiles" / "wife"
|
||||
_hermes_state_db(hermes, "I am the husband")
|
||||
_hermes_state_db(wife, "I am the wife")
|
||||
app = application(tmp_path, home=home)
|
||||
async with app.router.lifespan_context(app), client_for(app) as client:
|
||||
location = (await client.post("/api/v1/sources/hermes/location")).json()
|
||||
local = await client.post(
|
||||
"/api/v1/sources/connect",
|
||||
json={
|
||||
"source_ids": ["hermes"],
|
||||
"request_id": str(uuid4()),
|
||||
"source_roots": {"hermes": location["root_id"]},
|
||||
"enable_sync": True,
|
||||
},
|
||||
)
|
||||
assert local.status_code == 200, local.text
|
||||
connected = await client.post(
|
||||
"/api/v1/sources/connect",
|
||||
json={
|
||||
"source_ids": ["hermes"],
|
||||
"request_id": str(uuid4()),
|
||||
"user_id": "wife",
|
||||
"root": str(wife),
|
||||
},
|
||||
)
|
||||
assert connected.status_code == 200, connected.text
|
||||
for job in (*local.json()["jobs"], *connected.json()["jobs"]):
|
||||
await app.state.import_supervisor.wait(job["job_id"])
|
||||
|
||||
v1 = (await client.get("/api/v1/sources")).json()
|
||||
v2 = (await client.get("/api/v2/sources")).json()
|
||||
assert v1 == v2
|
||||
wife_sources = (await client.get("/api/v2/sources", params={"user_id": "wife"})).json()
|
||||
assert [source["source_id"] for source in wife_sources] == ["hermes"]
|
||||
assert wife_sources[0]["authorized"] is True
|
||||
assert (await client.get("/api/v2/sources", params={"u": "x"})).status_code == 422
|
||||
|
||||
created = await client.post(
|
||||
"/api/v2/import-jobs",
|
||||
json={"source_id": "hermes", "operation": "sync", "user_id": "wife"},
|
||||
)
|
||||
assert created.status_code == 200, created.text
|
||||
wife_job = created.json()
|
||||
assert wife_job["user_id"] == "wife" and wife_job["source_id"] == "hermes"
|
||||
await app.state.import_supervisor.wait(wife_job["job_id"])
|
||||
refused = await client.post(
|
||||
"/api/v2/import-jobs",
|
||||
json={"source_id": "claude-code", "user_id": "wife"},
|
||||
)
|
||||
assert refused.status_code in {404, 422}
|
||||
unknown = await client.post(
|
||||
"/api/v2/import-jobs",
|
||||
json={"source_id": "hermes", "from_beginning": True},
|
||||
)
|
||||
assert unknown.status_code == 422
|
||||
|
||||
listed = (await client.get("/api/v2/import-jobs", params={"user_id": "wife"})).json()
|
||||
assert {job["user_id"] for job in listed} == {"wife"}
|
||||
assert (await client.get("/api/v2/import-jobs")).json() == (
|
||||
await client.get("/api/v1/import-jobs")
|
||||
).json()
|
||||
path = f"/api/v2/import-jobs/{wife_job['job_id']}"
|
||||
assert (await client.get(path)).status_code == 404
|
||||
own = await client.get(path, params={"user_id": "wife"})
|
||||
assert own.status_code == 200 and own.json()["job_id"] == wife_job["job_id"]
|
||||
|
||||
synced = await client.post("/api/v2/sources/hermes/sync", json={"user_id": "wife"})
|
||||
v1_synced = await client.post("/api/v1/sources/hermes/sync")
|
||||
assert synced.status_code == v1_synced.status_code, (synced.text, v1_synced.text)
|
||||
assert synced.status_code == 200
|
||||
assert synced.json()["user_id"] == "wife"
|
||||
assert v1_synced.json()["user_id"] == "local"
|
||||
other = await client.post("/api/v2/sources/claude-code/sync", json={"user_id": "wife"})
|
||||
assert other.status_code in {404, 422}
|
||||
assert (
|
||||
await client.post("/api/v2/sources/hermes/sync", json={"owner": "wife"})
|
||||
).status_code == 422
|
||||
|
||||
asyncio.run(scenario())
|
||||
@@ -6,7 +6,7 @@ import io
|
||||
import json
|
||||
import sqlite3
|
||||
from contextlib import closing
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
@@ -30,7 +30,6 @@ from one_memory.sdk_data_plane_migration import (
|
||||
MemoryAuthority,
|
||||
SdkDataPlaneMigrationStore,
|
||||
)
|
||||
from one_memory.sdk_memory_runtime import SdkMemoryRuntime
|
||||
from one_memory.sdk_migration_bundle import create_sdk_migration_bundle
|
||||
from one_memory.sdk_migration_runner import (
|
||||
PREPARED_PHASE,
|
||||
@@ -708,6 +707,7 @@ def test_cutover_waits_until_the_new_runtime_reads_every_chat_memory(store, acti
|
||||
"chat_readable": 1,
|
||||
"productivity_expected": 0,
|
||||
"productivity_readable": 0,
|
||||
"chat_missing_before_migration": 0,
|
||||
"passed": False,
|
||||
}
|
||||
assert store.authority() is MemoryAuthority.LEGACY
|
||||
@@ -774,18 +774,186 @@ def test_retired_coding_experiences_do_not_block_but_an_empty_store_does(store,
|
||||
assert (result.readback.productivity_expected, result.readback.productivity_readable) == (2, 1)
|
||||
|
||||
|
||||
def test_memories_without_a_model_profile_keep_legacy_authority(store, monkeypatch):
|
||||
def test_a_home_without_a_model_profile_migrates_on_the_builtin_embedding(store, monkeypatch):
|
||||
"""#234: no generation model was ever configured, and the upgrade stopped.
|
||||
|
||||
No profile means no LLM, not no embedding: the vectors are the built-in
|
||||
model's, the same space the Runtime reads with. They are reattached and
|
||||
read back through it, and the client is opened without resolving a key.
|
||||
"""
|
||||
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot, vector_collection_bases
|
||||
|
||||
monkeypatch.setattr("one_memory.sdk_migration_runner.read_active_profile", lambda _db: None)
|
||||
home = store.database_path.parent
|
||||
builtin = builtin_retrieval_snapshot()
|
||||
_current, legacy_bases = vector_collection_bases(builtin)
|
||||
client = _chroma(home)
|
||||
_fill(client, f"{legacy_bases[0]}_512", ["m1", "m2"])
|
||||
del client
|
||||
_confirmed(store, "m1", "m2")
|
||||
opened: list[str] = []
|
||||
|
||||
def open_client(profile):
|
||||
opened.append(profile.profile_id)
|
||||
return _reading({"m1", "m2"})(profile)
|
||||
|
||||
result = run_startup_migration(
|
||||
data_dir=home,
|
||||
runtime_build="21",
|
||||
sdk_version="2.0.5",
|
||||
namespace="governed-v2",
|
||||
open_client=open_client,
|
||||
)
|
||||
|
||||
assert result.state == "cutover"
|
||||
assert opened == [builtin.profile_id]
|
||||
assert [item.legacy for item in result.reattached] == [f"{legacy_bases[0]}_512"]
|
||||
assert result.readback is not None and result.readback.passed
|
||||
assert store.authority() is MemoryAuthority.SDK
|
||||
|
||||
|
||||
def test_the_builtin_readback_client_resolves_no_secret(tmp_path, monkeypatch):
|
||||
"""The built-in snapshot has no stored version and no key to look up."""
|
||||
from one_memory import sdk_migration_verify
|
||||
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot
|
||||
|
||||
leased: list[tuple[str, object]] = []
|
||||
|
||||
class Lease:
|
||||
client = object()
|
||||
|
||||
def close(self) -> None:
|
||||
leased.append(("closed", None))
|
||||
|
||||
def acquire(profile, *, api_key, data_dir):
|
||||
leased.append((profile.profile_id, api_key))
|
||||
return Lease()
|
||||
|
||||
def no_lookup(*_args, **_kwargs):
|
||||
raise AssertionError("a key-less profile must not touch the secret store")
|
||||
|
||||
monkeypatch.setattr("one_memory.hy_sdk_runtime.acquire_client", acquire)
|
||||
monkeypatch.setattr(
|
||||
"one_memory.model_profiles.ModelProfileStore.resolve_snapshot_secret", no_lookup
|
||||
)
|
||||
with sdk_migration_verify.runtime_sdk_client(
|
||||
tmp_path / "missing.sqlite3", tmp_path, builtin_retrieval_snapshot()
|
||||
) as client:
|
||||
assert client is Lease.client
|
||||
assert leased == [("builtin-retrieval", None), ("closed", None)]
|
||||
|
||||
|
||||
def test_vectors_left_by_a_deleted_profile_stop_the_upgrade_by_name(store, monkeypatch):
|
||||
"""No active profile, and the vectors sit in another profile's space.
|
||||
|
||||
The built-in model cannot read them, so the old abort was right for this
|
||||
one shape; it now says what is actually wrong, before anything changes.
|
||||
"""
|
||||
from one_memory.hy_sdk_runtime import vector_collection_bases
|
||||
from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind
|
||||
|
||||
monkeypatch.setattr("one_memory.sdk_migration_runner.read_active_profile", lambda _db: None)
|
||||
deleted = ModelProfileSnapshot(
|
||||
profile_id="deleted",
|
||||
revision=3,
|
||||
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
|
||||
base_url="https://models.example/v1",
|
||||
model="chat",
|
||||
embedding_provider_kind=ProviderKind.OPENAI_COMPATIBLE,
|
||||
embedding_base_url="https://models.example/v1",
|
||||
embedding_model="text-embedding-3-large",
|
||||
embedding_dimensions=1024,
|
||||
timeout_seconds=60,
|
||||
max_concurrency=1,
|
||||
context_window=None,
|
||||
has_api_key=True,
|
||||
)
|
||||
current, _legacy = vector_collection_bases(deleted)
|
||||
home = store.database_path.parent
|
||||
client = _chroma(home)
|
||||
_fill(client, f"{current}_1024", ["m1"])
|
||||
del client
|
||||
_confirmed(store, "m1")
|
||||
|
||||
with pytest.raises(DataPlaneMigrationError, match="sdk_migration_model_profile_required"):
|
||||
with pytest.raises(DataPlaneMigrationError, match="sdk_migration_vector_profile_missing"):
|
||||
run_startup_migration(
|
||||
data_dir=store.database_path.parent,
|
||||
data_dir=home,
|
||||
runtime_build="21",
|
||||
sdk_version="2.0.5",
|
||||
namespace="governed-v2",
|
||||
open_client=_reading({"m1"}),
|
||||
)
|
||||
assert store.authority() is MemoryAuthority.LEGACY
|
||||
assert not (home / "backups").exists()
|
||||
|
||||
|
||||
def test_confirmed_chats_the_sdk_had_already_merged_do_not_fail_the_upgrade(store, active_profile):
|
||||
"""#229/#240: 658 confirmed chats, 657 in the store, and the upgrade failed.
|
||||
|
||||
A 1.0.4 library can hold confirmed rows whose SDK node had been merged or
|
||||
deleted before the upgrade; no migration can read those back. The check
|
||||
holds chat to what the store held before this run and counts the rest.
|
||||
"""
|
||||
from one_memory.hy_sdk_runtime import vector_collection_bases
|
||||
|
||||
home = store.database_path.parent
|
||||
stored = [f"chat-{index:03d}" for index in range(657)]
|
||||
current, _legacy = vector_collection_bases(active_profile)
|
||||
client = _chroma(home)
|
||||
_fill(client, f"{current}_512", stored)
|
||||
del client
|
||||
_confirmed(store, *stored, "chat-merged-before-upgrade")
|
||||
|
||||
result = run_startup_migration(
|
||||
data_dir=home,
|
||||
runtime_build="21",
|
||||
sdk_version="2.0.5",
|
||||
namespace="governed-v2",
|
||||
open_client=_reading(set(stored)),
|
||||
)
|
||||
|
||||
assert result.state == "cutover"
|
||||
assert result.readback is not None
|
||||
assert result.readback.as_document() == {
|
||||
"chat_expected": 657,
|
||||
"chat_readable": 657,
|
||||
"productivity_expected": 0,
|
||||
"productivity_readable": 0,
|
||||
"chat_missing_before_migration": 1,
|
||||
"passed": True,
|
||||
}
|
||||
assert store.authority() is MemoryAuthority.SDK
|
||||
|
||||
|
||||
def test_a_chat_in_the_store_before_the_upgrade_must_still_read_back(store, active_profile):
|
||||
"""Only what was already gone is excused; losing a stored memory still fails."""
|
||||
from one_memory.hy_sdk_runtime import vector_collection_bases
|
||||
from one_memory.sdk_migration_verify import MemoryReadbackFailed
|
||||
|
||||
home = store.database_path.parent
|
||||
current, _legacy = vector_collection_bases(active_profile)
|
||||
client = _chroma(home)
|
||||
_fill(client, f"{current}_512", ["m1", "m2"])
|
||||
del client
|
||||
_confirmed(store, "m1", "m2", "merged")
|
||||
|
||||
with pytest.raises(MemoryReadbackFailed) as caught:
|
||||
run_startup_migration(
|
||||
data_dir=home,
|
||||
runtime_build="21",
|
||||
sdk_version="2.0.5",
|
||||
namespace="governed-v2",
|
||||
open_client=_reading({"m1"}),
|
||||
)
|
||||
assert caught.value.report.as_document() == {
|
||||
"chat_expected": 2,
|
||||
"chat_readable": 1,
|
||||
"productivity_expected": 0,
|
||||
"productivity_readable": 0,
|
||||
"chat_missing_before_migration": 1,
|
||||
"passed": False,
|
||||
}
|
||||
assert store.authority() is MemoryAuthority.LEGACY
|
||||
|
||||
|
||||
def test_a_build_20_home_gets_its_vectors_back_and_is_checked_against_its_backup(
|
||||
@@ -1072,7 +1240,13 @@ def test_host_capture_is_fenced_for_the_whole_migration(store):
|
||||
)
|
||||
|
||||
assert result.state == "cutover"
|
||||
assert fenced == {"bundle": True, "prepare": True, "cutover": True, "cleanup": True}
|
||||
assert fenced == {
|
||||
"bundle": True,
|
||||
"prepare": True,
|
||||
"vectors": True,
|
||||
"cutover": True,
|
||||
"cleanup": True,
|
||||
}
|
||||
assert not sdk_migration_capture_fenced(home)
|
||||
|
||||
|
||||
@@ -1175,57 +1349,6 @@ def test_raw_http_dto_uses_the_unified_memory_list_shape():
|
||||
assert page.raw[0].custom == {"role": "user"}
|
||||
|
||||
|
||||
def test_sdk_raw_retention_keeps_only_the_latest_three_days():
|
||||
now = datetime(2026, 10, 23, tzinfo=UTC)
|
||||
|
||||
class Client:
|
||||
def __init__(self):
|
||||
self.rows = [
|
||||
{
|
||||
"memory_id": "old",
|
||||
"memory_at": int((now - timedelta(days=4)).timestamp()),
|
||||
},
|
||||
{
|
||||
"memory_id": "recent",
|
||||
"memory_at": int((now - timedelta(days=2)).timestamp()),
|
||||
},
|
||||
]
|
||||
|
||||
def list_memories(self, **_kwargs):
|
||||
return {
|
||||
"vdb": {
|
||||
"memories": list(self.rows),
|
||||
"total": len(self.rows),
|
||||
"limit": 500,
|
||||
"offset": 0,
|
||||
}
|
||||
}
|
||||
|
||||
def delete(self, memory_id):
|
||||
before = len(self.rows)
|
||||
self.rows = [row for row in self.rows if row["memory_id"] != memory_id]
|
||||
return {
|
||||
"success": True,
|
||||
"deleted_count": before - len(self.rows),
|
||||
}
|
||||
|
||||
client = Client()
|
||||
|
||||
class Handle:
|
||||
async def client(self):
|
||||
return client
|
||||
|
||||
runtime = SdkMemoryRuntime(
|
||||
coordinator=None,
|
||||
read_handle=Handle(),
|
||||
write_queue=None,
|
||||
extraction_settings=None,
|
||||
data_dir=Path("."),
|
||||
)
|
||||
assert asyncio.run(runtime.prune_expired_raw(now=now)) == 1
|
||||
assert [row["memory_id"] for row in client.rows] == ["recent"]
|
||||
|
||||
|
||||
# The UI asks which memory data plane is authoritative and must get the
|
||||
# answer the API gates use.
|
||||
|
||||
@@ -1353,7 +1476,7 @@ def test_the_sdk_memory_map_draws_sdk_memories_without_legacy_tables(tmp_path) -
|
||||
original = app.state.sdk_client_handle
|
||||
app.state.sdk_client_handle = Handle()
|
||||
|
||||
async def graph_snapshot(*, limit):
|
||||
async def graph_snapshot(*, limit, user_id="local"):
|
||||
del limit
|
||||
return HyGraphSnapshot(
|
||||
nodes=(HyGraphNode(node_id="t1", content="topic", layer="TOPIC"),),
|
||||
|
||||
@@ -91,7 +91,7 @@ def test_bundle_contains_management_sdk_config_and_verified_manifest(tmp_path):
|
||||
assert manifest["bundle_id"] == "migration-1"
|
||||
assert manifest["sdk_namespace"] == "governed-v2"
|
||||
assert manifest["runtime_build"] == "build-13"
|
||||
assert manifest["snapshot_sdk_version"] == "2.0.8"
|
||||
assert manifest["snapshot_sdk_version"] == "2.0.9"
|
||||
paths = {item["path"]: item for item in manifest["files"]}
|
||||
assert {
|
||||
"management/one-memory.sqlite3",
|
||||
|
||||
@@ -306,7 +306,7 @@ def test_the_runner_names_the_step_and_sub_step_that_failed(tmp_path, monkeypatc
|
||||
("step.finished", "preflight.profile_read"),
|
||||
("step.finished", "preflight.resumable_identity"),
|
||||
("step.finished", "preflight.legacy_ids_scan"),
|
||||
("step.finished", "preflight.model_profile_check"),
|
||||
("step.finished", "preflight.vector_space_check"),
|
||||
("step.finished", "prepare.recover_abandoned_staging"),
|
||||
("step.finished", "bundle.ensure"),
|
||||
("step.finished", "bundle.restore_point"),
|
||||
|
||||
@@ -463,11 +463,17 @@ def test_a_run_killed_after_its_restore_point_is_rolled_back_on_the_next_start(h
|
||||
|
||||
|
||||
def test_a_preflight_failure_changes_nothing_and_needs_no_restore(home, monkeypatch):
|
||||
def unreadable(_data_dir, _profile):
|
||||
raise DataPlaneMigrationError("sdk_migration_vector_profile_missing")
|
||||
|
||||
monkeypatch.setattr("one_memory.sdk_migration_runner.read_active_profile", lambda _db: None)
|
||||
monkeypatch.setattr(
|
||||
"one_memory.sdk_migration_runner.vectors_belong_to_another_space", unreadable
|
||||
)
|
||||
before = _state(home)
|
||||
report: dict[str, object] = {}
|
||||
|
||||
with pytest.raises(DataPlaneMigrationError, match="sdk_migration_model_profile_required"):
|
||||
with pytest.raises(DataPlaneMigrationError, match="sdk_migration_vector_profile_missing"):
|
||||
_run(home, report=report)
|
||||
|
||||
assert report == {}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Model-free SDK search remains available without an active write profile."""
|
||||
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
@@ -95,3 +96,158 @@ async def test_search_without_profile_uses_public_sdk_search() -> None:
|
||||
assert options["user_ids"] == ["local"]
|
||||
assert options["agent_ids"] == []
|
||||
assert options["scene"] == "normal"
|
||||
|
||||
|
||||
def _activate(profiles, secrets, *, dimensions: int, model: str = "chat"):
|
||||
from pydantic import SecretStr
|
||||
|
||||
from one_memory.model_profiles import ModelCapabilities, ModelProfileInput
|
||||
|
||||
profile = profiles.create(
|
||||
ModelProfileInput(
|
||||
name=f"embedding-{dimensions}",
|
||||
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
|
||||
base_url="https://models.example/v1",
|
||||
model=model,
|
||||
embedding_provider_kind=ProviderKind.OPENAI_COMPATIBLE,
|
||||
embedding_base_url="https://models.example/v1",
|
||||
embedding_model="qwen3-embedding-4b",
|
||||
embedding_dimensions=dimensions,
|
||||
api_key=SecretStr("sk-test"),
|
||||
),
|
||||
secrets,
|
||||
)
|
||||
profiles.record_capabilities(
|
||||
profile.profile_id,
|
||||
expected_revision=profile.revision,
|
||||
capabilities=ModelCapabilities(
|
||||
endpoint_reachable=True,
|
||||
model_available=True,
|
||||
structured_output=True,
|
||||
embedding_available=True,
|
||||
),
|
||||
secrets=secrets,
|
||||
)
|
||||
profiles.activate(profile.profile_id, secrets)
|
||||
return profiles.snapshot(profile.profile_id, profiles.active_identity()[1])
|
||||
|
||||
|
||||
class _Recorder:
|
||||
"""Stands in for `acquire_client`: records the collection each client opens."""
|
||||
|
||||
def __init__(self, data_dir: Path) -> None:
|
||||
self.data_dir = data_dir
|
||||
self.opened: list[tuple[str, int | None, str | None]] = []
|
||||
self.closed: list[str] = []
|
||||
|
||||
def __call__(self, profile, *, api_key, data_dir, http_transport_factory=None, mode="pro"):
|
||||
from one_memory.hy_sdk_runtime import build_sdk_config
|
||||
|
||||
config = build_sdk_config(profile, api_key=api_key, data_dir=data_dir)
|
||||
name = config.vector_store.collection_name
|
||||
self.opened.append((name, config.vector_store.embedding_dims, api_key))
|
||||
return SimpleNamespace(
|
||||
client=SimpleNamespace(collection=name),
|
||||
close=lambda: self.closed.append(name),
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_reads_open_the_collection_the_active_profile_writes(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""#242: a 2560-dim profile wrote `memhub_<sha>_2560`, reads opened `_512`."""
|
||||
from one_memory.hy_sdk_runtime import build_sdk_config
|
||||
from one_memory.model_profiles import ModelProfileStore
|
||||
from one_memory.sdk_read_handle import ActiveProfileReadHandle
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
recorder = _Recorder(tmp_path)
|
||||
monkeypatch.setattr("one_memory.hy_sdk_runtime.acquire_client", recorder)
|
||||
profiles = ModelProfileStore(tmp_path / "one-memory.sqlite3")
|
||||
secrets = InMemorySecretStore()
|
||||
handle = ActiveProfileReadHandle(profiles=profiles, secrets=secrets, data_dir=tmp_path)
|
||||
|
||||
builtin = build_sdk_config(builtin_retrieval_snapshot(), api_key=None, data_dir=tmp_path)
|
||||
assert (await handle.client()).collection == builtin.vector_store.collection_name
|
||||
|
||||
written = _activate(profiles, secrets, dimensions=2560)
|
||||
writes = build_sdk_config(written, api_key="sk-test", data_dir=tmp_path)
|
||||
|
||||
async def read() -> str:
|
||||
return str((await handle.client()).collection)
|
||||
|
||||
assert await asyncio.create_task(read()) == writes.vector_store.collection_name
|
||||
assert recorder.opened[-1] == (writes.vector_store.collection_name, 2560, "sk-test")
|
||||
assert writes.vector_store.collection_name != builtin.vector_store.collection_name
|
||||
await handle.close()
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_a_profile_change_rebuilds_reads_without_closing_one_in_flight(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
from one_memory.model_profiles import ModelProfileStore
|
||||
from one_memory.sdk_read_handle import ActiveProfileReadHandle
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
recorder = _Recorder(tmp_path)
|
||||
monkeypatch.setattr("one_memory.hy_sdk_runtime.acquire_client", recorder)
|
||||
profiles = ModelProfileStore(tmp_path / "one-memory.sqlite3")
|
||||
secrets = InMemorySecretStore()
|
||||
handle = ActiveProfileReadHandle(profiles=profiles, secrets=secrets, data_dir=tmp_path)
|
||||
_activate(profiles, secrets, dimensions=1024)
|
||||
|
||||
holding = asyncio.Event()
|
||||
finish = asyncio.Event()
|
||||
|
||||
async def slow_read() -> str:
|
||||
client = await handle.client()
|
||||
holding.set()
|
||||
await finish.wait()
|
||||
return str(client.collection)
|
||||
|
||||
in_flight = asyncio.create_task(slow_read())
|
||||
await holding.wait()
|
||||
[(old, _, _)] = recorder.opened
|
||||
|
||||
_activate(profiles, secrets, dimensions=2560, model="chat-2")
|
||||
|
||||
async def read() -> str:
|
||||
return str((await handle.client()).collection)
|
||||
|
||||
new = await asyncio.create_task(read())
|
||||
assert new != old
|
||||
# The read that started on the old space still has its client.
|
||||
assert recorder.closed == []
|
||||
|
||||
finish.set()
|
||||
assert await in_flight == old
|
||||
for _ in range(5):
|
||||
await asyncio.sleep(0)
|
||||
assert recorder.closed == [old]
|
||||
await handle.close()
|
||||
assert recorder.closed == [old, new]
|
||||
|
||||
|
||||
def test_every_api_read_shares_the_active_profile_handle(tmp_path: Path) -> None:
|
||||
"""Library, search, recall, graph, profile, raw, ask and context all read
|
||||
through `_sdk_client_handle`, so it must be the active-profile handle."""
|
||||
from one_memory.api import _sdk_client_handle
|
||||
from one_memory.config import Settings
|
||||
from one_memory.model_profiles import ModelProfileStore
|
||||
from one_memory.sdk_read_handle import ActiveProfileReadHandle
|
||||
from one_memory.secrets import InMemorySecretStore
|
||||
|
||||
state = SimpleNamespace(
|
||||
sdk_client_handle=None,
|
||||
model_profile_store=ModelProfileStore(tmp_path / "one-memory.sqlite3"),
|
||||
secret_store=InMemorySecretStore(),
|
||||
settings=Settings(_env_file=None, data_dir=tmp_path),
|
||||
)
|
||||
app = SimpleNamespace(state=state)
|
||||
|
||||
handle = _sdk_client_handle(app) # type: ignore[arg-type]
|
||||
|
||||
assert isinstance(handle, ActiveProfileReadHandle)
|
||||
assert _sdk_client_handle(app) is handle is state.sdk_client_handle # type: ignore[arg-type]
|
||||
@@ -57,10 +57,10 @@ def client_for(app, headers=None):
|
||||
|
||||
|
||||
class Answerer:
|
||||
async def answer(self, question):
|
||||
async def answer(self, question, *, user_id="local"):
|
||||
return SdkMemoryAnswer(answer="真实数据。", memories=(MEMORY,), mode="extractive")
|
||||
|
||||
async def answer_stream(self, question):
|
||||
async def answer_stream(self, question, *, user_id="local"):
|
||||
yield {"type": "recall", "memories": (MEMORY,)}
|
||||
yield {
|
||||
"type": "final",
|
||||
|
||||
Generated
+4
-4
@@ -991,7 +991,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "hy-memory"
|
||||
version = "2.0.8"
|
||||
version = "2.0.9"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "chromadb" },
|
||||
@@ -1008,9 +1008,9 @@ dependencies = [
|
||||
{ name = "scikit-learn" },
|
||||
{ name = "tiktoken" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/21/15/83ca83b649ead238f186fdb33efdc73615fbba279efbbdca3857c1a86beb/hy_memory-2.0.8.tar.gz", hash = "sha256:0fa0ed1a1160b389a3d7c13a9fb6528b140d7ec23cf49e65f1be5974820a576b", size = 856025, upload-time = "2026-10-28T07:35:47.783Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/48/28/e01fd9a92cd090c8236df0dd06dae049c5de66345e5bd9653497ed56125e/hy_memory-2.0.9.tar.gz", hash = "sha256:5f55ea92044ced19a00b4e56c565215adedd3439a1bdb9cfe9ffe42387c8f1a1", size = 858323, upload-time = "2026-11-01T00:29:49.914Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/11/6a/c699cac0ff06bd089fceeff90446130763d8187e649dbdb0313014491a10/hy_memory-2.0.8-py3-none-any.whl", hash = "sha256:68bd789df24e51e64bb231e97a8d24ee89b4aa5f9ba14c3e0824589e9213bf7c", size = 921119, upload-time = "2026-10-28T07:35:45.571Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d5/44/4368555a408a58daf902c81b62184a3c99f12abd45d4f1144bf79f171a6f/hy_memory-2.0.9-py3-none-any.whl", hash = "sha256:95dbea232d2041f9b0ea3d08dd751603c2a5c0e4422b97e74f664e6ca913549e", size = 923250, upload-time = "2026-11-01T00:29:47.814Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1837,7 +1837,7 @@ requires-dist = [
|
||||
{ name = "fastapi", specifier = ">=0.141.1" },
|
||||
{ name = "filelock", specifier = ">=3.32.4" },
|
||||
{ name = "httpx", specifier = ">=0.28.1" },
|
||||
{ name = "hy-memory", specifier = "==2.0.8" },
|
||||
{ name = "hy-memory", specifier = "==2.0.9" },
|
||||
{ name = "keyring", specifier = ">=25.7.0" },
|
||||
{ name = "numpy", specifier = "==2.5.2" },
|
||||
{ name = "onnxruntime", specifier = "==1.23.2" },
|
||||
|
||||
@@ -133,6 +133,7 @@ const SAFE_STABLE_RUNTIME_ERROR_CODES = new Set([
|
||||
"sdk_migration_preflight_failed",
|
||||
"sdk_migration_memory_unreadable",
|
||||
"sdk_migration_model_profile_required",
|
||||
"sdk_migration_vector_profile_missing",
|
||||
"sdk_migration_model_secret_unavailable",
|
||||
"sdk_migration_secure_storage_unavailable",
|
||||
"sdk_migration_disk_space_insufficient",
|
||||
@@ -219,6 +220,8 @@ export const STABLE_RUNTIME_USER_MESSAGES: Record<string, string> = {
|
||||
"升级前核对发现,新版本还读不到你的全部记忆,所以没有切换数据源,旧数据保持原样可用。请把这个提示反馈给我们;原有记忆数据不会丢失。",
|
||||
sdk_migration_model_profile_required:
|
||||
"升级记忆数据格式需要先有可用的模型配置,用来核对新版本能否读回你的记忆。请在旧版本中完成模型配置后重启;你的记忆数据不会受到影响。",
|
||||
sdk_migration_vector_profile_missing:
|
||||
"你的记忆是用一个已删除的模型配置生成向量的,当前没有可用的模型配置能读回它们,所以没有升级。请在旧版本中重新添加那个模型配置并启用后重启;你的记忆数据不会受到影响。",
|
||||
sdk_migration_model_secret_unavailable:
|
||||
"模型密钥读不到(可能被系统钥匙串拒绝),无法核对新版本能否读回你的记忆。请允许 MemHub 访问钥匙串后重新启动;你的记忆数据不会受到影响。",
|
||||
sdk_migration_secure_storage_unavailable:
|
||||
|
||||
Loaded 100 of 116 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user