Compare commits

...
Author SHA1 Message Date
alvinfeiandClaude Opus 5.5 4e1748f65a feat(memhub): memory mode (lite/pro/ultra), evolve and its nightly schedule
- Mode: a saved setting (config.json memory_mode, default pro);
  ONE_MEMORY_GRAPH_STORE still forces ultra for development.
  - API: GET/PUT /api/v1/settings/memory-mode, which reports the switch's
    effects and the graph backfill.
  - CLI: memhub settings mode [lite|pro|ultra] [--yes], which prints the plan
    first.
- The mode drives every SDK client.
  - acquire_client passes it to HyMemoryClient. The read handle rebuilds on a
    change.
  - Write handles are keyed by (profile, revision, mode) and retired with the
    #218 use count, so a batch in flight finishes in the old mode.
  - Jobs read the mode when they run: a job completed in lite is never
    reprocessed; pending jobs take the new mode.
  - Mode never touches the vector collection identity.
- lite needs no generation model: the built-in embedder writes raw, and the
  claim gate admits lite without an active profile. lite never calls
  add_extracted (a note is one raw user turn through add). Recall serves the
  raw layer.
- → ultra: build_memory_graph once per known user, in the background, with a
  status (graph_build).
- Raw retention removed in every mode: MemHub deletes no raw (owner
  decision); the retention loop, its startup step and its test are gone.
- P1-3 / R7: POST /api/v2/evolve answers 202 with a run receipt;
  GET /api/v2/evolve[/{run_id}].
  - Ultra only: 409 memory_mode_not_ultra otherwise.
  - One run per user. Never beside that user's write: it waits for the write
    in flight, and holds that user's jobs (released pre-SDK) until it ends.
  - Loops while should_continue, up to 20 rounds.
  - Per-user last_run_at, last_result and last_error in evolve-state.json
    (counts and codes only).
  - CLI: memhub evolve [--user-id] [--agent-id] [--max-batches] [--wait].
- Nightly scheduler: 03:00 local by default, ultra only, with catch-up when
  the last run is older than the slot or 24h.
  - Users come from Core's records (local, grant holders, write-queue users)
    that the SDK holds memories for.
  - GET/PUT /api/v1/settings/evolve-schedule.
- R10: pipeline {mode, configured_mode, forced_by_env, evolve, last_evolve_at,
  graph_build} in /api/v1/memory-processing, memhub status --json and
  memhub progress.
- Contract for the App: docs/contracts/memory-mode-evolve-v1.md, including
  what SDK ultra additionally enables and zh/en/es copy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 19:09:19 +08:00
alvinfeiandClaude Opus 5.5 8d0a069950 feat(memhub): expose SDK search, layer, raw and profile-version surface
- P2-1 / R8: POST /api/v2/context/search and /api/v1/recall pass the public
  client.search knobs through, validated: min_score, schema_min_score,
  working_fact_min_score, working_experience_min_score, schema_limit,
  intention_limit, rerank, reader (legacy|hybrid), scene
  (chat|working|coding|all), search_mode (default|agentic),
  include_sources, exclude_sources, created_after, session_ids, agent_ids,
  workspace_id, branch (needs workspace_id), layers. The v2 ask routes take
  the floors and filters. Absent, every route makes the call it made before
  (relevance floor 0.35, scene normal, hybrid reader). A caller's floor is
  also enforced on what Core returns, per lane (the fused scene=all score is
  a rank, so the lane score is used): min_score=0.9 answers only >= 0.9.
  user_ids, embed_override, query_embedding and request_id are not exposed.
- P2-2 / R9: memory-library takes layers (or layer), statuses,
  exclude_sources, session_ids and agent_ids, repeated or comma separated;
  a layer filter lists the chat store and refuses scene=working|coding.
  New GET /api/v2/memory-library/layers counts memories per SDK layer.
  The memory graph takes layers, session_ids and exclude_sources.
- P2-4: /api/v2/raw lists layers=["l1_raw"] with statuses active+shadow (pro
  shadows a raw once extracted), filters by source, session_id, statuses
  and order, cuts after/before in Core, and returns each turn's messages
  with roles, a sha256 content_hash, status and created.
- P2-3: GET /api/v2/sources, POST /api/v2/sources/{id}/sync,
  POST /api/v2/import-jobs, GET /api/v2/import-jobs[/{id}] reuse the v1
  handlers on one user's lane (Hermes only for another user; another
  user's job answers 404).
- Profile: GET /api/v2/user-profile/versions (get_user_basic_info) returns
  the L0 version chains; memhub profile --versions prints them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 18:28:54 +08:00
feitianxiang cdb44658cf Merge pull request #255 from Tencent-Hunyuan/fix/upgrade-migration-issues
fix(memhub): upgrade migration without a model profile, tolerant readback, grant triggers after cutover, reads use the active embedding
2026-11-01 18:26:00 +08:00
alvinfeiandClaude Opus 5.5 2ab04291a9 fix(memhub): the embedding endpoint no longer follows the generation base_url
A remote embedding with no `embedding_base_url` resolved to the generation
`base_url` at runtime (`build_sdk_config`, `vector_collection_bases`, the
probe). The collection name hashes that endpoint, so a proxy or region edit
of the generation endpoint moved writes to a new, empty collection, and
providers without embeddings were sent embedding requests.

- `resolve_embedding_endpoint` no longer inherits; creating or updating a
  remote (non-local_onnx) embedding without an endpoint is refused with
  422 `embedding_base_url_required`. The runtime and probe read only
  `embedding_base_url`. The API key stays shared.
- Opening the profile store pins, once and idempotently, every stored
  remote embedding left without an endpoint to the one it resolved to
  (the generation base_url, or the provider default), so the collection
  name is byte-identical. `read_active_profile` applies the same rule for
  `sdk-migrate`, which may run before Core has opened the store.
- Web: settings write the prefilled endpoint instead of relying on the
  fallback; onboarding writes an empty custom field as the endpoint it
  means. CLI headless configs now get the 422 code.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 18:11:20 +08:00
alvinfeiandClaude Opus 5.5 80d3b55723 fix(memhub): reads use the active profile's embedding space
The read handle was always built from `builtin_retrieval_snapshot()`, while
writes follow the active profile, so a custom embedding (2560 dims, say)
wrote `memhub_<sha>_2560` and every read opened the empty built-in 512-dim
collection.

`_sdk_client_handle` now returns an `ActiveProfileReadHandle`. It opens a
`retrieval_view` of the active profile: the profile's embedding fields with
the built-in generation placeholder, so a read never needs the generation
model; a remote OpenAI-compatible embedding gets the profile's shared key.
With no active profile it stays on the built-in space.

A profile change builds a new client and retires the old one, closed once no
task still uses it, as `SdkWriteCoordinator` does (#218). A use is the task
that asked for the client, so every caller (library, search, recall and its
raw fallback, graph, profile, raw, home summary, ask, context, multi-user
routes) keeps its shape; each raw-retention pass runs in its own task.

Refs #242

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 17:50:21 +08:00
alvinfeiandClaude Opus 5.5 b5ec0861f0 fix(memhub): migrate without a model profile and check readback against the pre-migration store
#234: with no active model profile the migration aborted in preflight with
`sdk_migration_model_profile_required`. No profile means no generation
model, not no embedding: the vectors are the built-in model's, the space the
Runtime reads with. The runner, and `sdk-migration-status`, now use
`builtin_retrieval_snapshot()` for reattach, the collection bases and the
readback client; `runtime_sdk_client` resolves no secret for a key-less
profile. When legacy memories have vectors only under another embedding
space (a profile since deleted), the preflight stops with
`sdk_migration_vector_profile_missing` before anything changes.

#229/#240: the readback required every confirmed governed chat id to read
back, but a 1.0.4 library can hold confirmed rows whose SDK node was merged
or deleted before the upgrade (658 confirmed, 657 readable, upgrade failed).
The ids in the vector store are now read before reattach; chat is held to
the confirmed ids present there, and the rest is only counted as
`chat_missing_before_migration`. A memory present before the migration but
unreadable after it still fails, and if none of the confirmed chats is in
the store none is excused.

Fixes #234, Fixes #229, Fixes #240

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 17:15:51 +08:00
alvinfeiandClaude Opus 5.5 47e938da1b fix(memhub): drop grant triggers that read a table the cutover dropped
The two `source_grants` triggers of the visibility cache select from
`governed_memory_evidence`, which the SDK cutover drops. SQLite resolves a
trigger body only when it fires, so every grant insert or update on a
migrated home failed with "no such table": revoke, connect and the
multi-user Hermes connect all answered 503.

`install_triggers` now installs a trigger only when every table its body
names exists, and drops an old copy otherwise. Startup on SDK authority
(which does not install the cache) and the cutover cleanup drop the stale
copies an earlier build left behind.

Fixes #233

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 16:20:34 +08:00
feitianxiang f5ea0af75d Merge pull request #251 from Tencent-Hunyuan/fix/hermes-injected-content
feat(memhub): land #250 on main — multi-user Runtime, hy-memory 2.0.9
2026-11-01 14:39:43 +08:00
alvinfeiandClaude Opus 5.5 1c2f48579f Merge main into fix/hermes-injected-content to land #250
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 14:33:00 +08:00
feitianxiang d29615abeb Merge pull request #250 from Tencent-Hunyuan/feat/memhub-multi-user
feat(memhub): multi-user Runtime — user_id on memory APIs and CLI, per-user Hermes profiles, hy-memory 2.0.9
2026-11-01 14:25:05 +08:00
feitianxiang 0965519e16 Merge pull request #249 from Tencent-Hunyuan/fix/hermes-injected-content
fix(memhub): drop Hermes gateway notices and unwrap relayed messages on import; no false 'damaged log' warning
2026-11-01 14:24:05 +08:00
alvinfeiandClaude Opus 5.5 7e9f0d98d8 fix(memhub): another user's memory id answers like a missing one
DELETE of another user's id answered 404 while a missing id answered 204,
so comparing the two told that the id existed. It now answers 204 and
deletes nothing; GET and PUT answer the same 404 as a missing id, and
receipt and addition reads already matched. A test compares every by-id
route's response for another user's id with a nonexistent id's.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 10:23:10 +08:00
alvinfeiandClaude Opus 5.5 0ff4538f36 feat(memhub-cli): --user-id for memory commands, connect a Hermes profile
- Global --user-id <id> (default local) for memories list/show/add/
  add-status/update/delete, recall, profile and write submit/status; sent
  only when it is not the default, so default requests are unchanged.
- memhub sources connect hermes --user-id <id> --root <path> [--import]
  grants one Hermes profile to its own user; refused for other sources.
- memhub sources list shows the user of every grant (from
  GET /api/v1/source-grants, skipped on an older Core).
- Help catalog, contract tests and docs/cli/EXPERIENCE.md updated.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 10:12:31 +08:00
alvinfeiandClaude Opus 5.5 a9709788d8 feat(memhub): multi-user Core — user_id on memory APIs, per-user Hermes grants
One Runtime now serves many user_ids. A user is the root that is read,
never a session or platform field; the App never sends user_id and keeps
reading and writing `local`, and only Hermes may hold another user's grant.

- R1: every memory read/write route takes an optional user_id (query for
  GET/DELETE, body for POST/PUT), default `local`, pattern
  ^[A-Za-z0-9._-]{1,64}$ (no `:`).
- R2: user_id reaches the SDK: Evidence, durable payload and client.add
  carry it; reads filter by user. get/update/delete/history and
  get_write_outcome take no user in hy-memory 2.0.9, so Core reads the
  memory first and answers 404 for another user's id; memory-write keys
  are namespaced per user and addition receipts are keyed by the user's
  Evidence id.
- R3: grants, cursors, jobs, live sync and the claim gate work per
  (user_id, source_id) through lane keys (`hermes` for local,
  `hermes@wife` otherwise). POST /api/v1/sources/connect takes user_id and
  root (Hermes only, 422 otherwise); revoke takes ?user_id; new
  GET /api/v1/source-grants lists every user's grants. Another user's
  grant syncs while it stands; the App's per-source switch stays local's.
- R4: a Hermes root scan never enters profiles/** (discovery, history and
  live sync share the same walk).
- R5: session keys, assembling rows, FIFO source keys and request ids are
  per user, so two users' turns never share a batch.
- R6: memory routes refuse undeclared query parameters and body fields
  with 422.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 10:12:31 +08:00
alvinfeiandClaude Opus 5.5 c1e5916b51 build(memhub): pin hy-memory 2.0.9
Core pins and locks the published hy-memory 2.0.9 from PyPI; the
version constant, README, architecture notes, SBOM, notices and their
tests follow. CHANGELOG records 2.0.9 as published.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 09:12:21 +08:00
alvinfeiandClaude Opus 5.5 97de5480bd Merge github/main (hy-memory 2.0.9 release) into feat/memhub-multi-user
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 09:11:25 +08:00
feitianxiang 97d9309ca8 Merge pull request #248 from Tencent-Hunyuan/feat/sdk-2.0.9
fix(sdk): placeholder values never overwrite the basic profile (2.0.9)
2026-11-01 08:26:49 +08:00
alvinfeiandClaude Opus 5.5 b1b3e0a291 fix(sdk): drop the user-turn filter for basic_info
Owner decision on #248: the writer no longer filters basic_info against
the user turns. The placeholder guard (with healing on read) and the
prompt hard rules stay; the guard alone keeps an assistant quoting
"name is 用户" out of the profile.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 08:18:32 +08:00
alvinfeiandClaude Opus 5.5 53507c7abe fix(memhub): only call non-JSON diagnostic lines damaged in memhub logs
After an upgrade the diagnostic files still hold an older build's records.
They are valid JSON that the reader leaves out, but were counted as damaged,
so `memhub logs` warned "跳过 N 行损坏的诊断记录" and a support export
failed with diagnostic_export_invalid_lines. Count them apart as
`unrecognized_lines` (reported only in the JSON empty-state record) and
keep the warning for lines that are not JSON.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 05:10:19 +08:00
alvinfeiandClaude Opus 5.5 5a1d252787 fix(memhub): drop Hermes gateway notices and unwrap relayed messages on import
Hermes stores its gateway's own posts in `user` rows: finished background
process notices (often with the gateway's "not written by a human" note),
voice placeholders that only name an audio file, and "Reply with exactly:
PONG" probes. Relayed Telegram messages arrive wrapped in out-of-band
markers and a gateway-origin JSON block. All of it reached extraction as
the person's words.

The Hermes adapter now cleans `user` rows (state.db and session
transcripts) before evidence is built, so the content hash and dedup see
the cleaned text:
- skipped with new reason codes `injected_machine_notice`,
  `voice_placeholder` and `probe` (media-only rows as `no_text`), counted
  in `import.parse.issue_summary` like other parse skips;
- out-of-band markers, the gateway-origin block and the interrupted
  response label are removed, keeping only the person's message.

Assistant and tool rows are unchanged. Tests use real rows with the
Telegram ids and home paths replaced by fake values.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 05:10:19 +08:00
alvinfeiandClaude Opus 5.5 8895fea775 release: hy-memory 2.0.9
Placeholder values no longer overwrite the user's basic profile (L0). Not
yet published to PyPI.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 04:56:01 +08:00
alvinfeiandClaude Opus 5.5 78882f3300 fix(sdk): placeholder values never overwrite the basic profile
A weak extraction model filled basic_info with "用户", "The user" or "N/A"
when the user stated nothing, and each placeholder superseded the real L0
value. upsert_basic_profile now drops placeholder values and writes no new
version when nothing is left; profile assembly skips stored placeholders.
The writer keeps a basic_info field only when its value appears in a user
message of the request. The zh and en extraction prompts add a hard rule
and a small-talk counter-example.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-11-01 04:56:01 +08:00
116 changed files with 9990 additions and 550 deletions

No files matched your search

+23
View File
@@ -94,6 +94,29 @@ authority on first launch.
`hy_node_permissions` proof table. Evidence, governed memories, model
profiles, SDK stores, and source/host grants are preserved.
## Hy Memory 2.0.9 — 2026-11-01
- Published
[`hy-memory==2.0.9`](https://pypi.org/project/hy-memory/2.0.9/).
- Placeholder values no longer overwrite the user's basic profile (L0). A
small extraction model filled `basic_info` with `{"name": "用户"}`,
`The user` or `N/A` when the user had said nothing about themselves, and
each placeholder superseded the real value, so the L0 chain read
费天翔 → Fei → 用户 → The user. Two defences:
- `upsert_basic_profile` drops placeholder values: empty, `用户`,
`the user`, `user`, `n/a`, `na`, `none`, `null`, `unknown`, `未提供`,
`未知`, `不详`, `-`, `?` and the field's own name, compared
case-insensitively with whitespace and punctuation stripped. When nothing
is left it writes no new version and no new L0 head. Placeholders already
stored in the chain no longer override earlier real values when the
profile is assembled, so `get_user_profile` of an affected user shows the
last real value again without any migration. This also stops an
assistant quoting an old memory ("The user's name is 用户.") from being
written back.
- The extraction prompts (zh and en) state the rule: only what the user
said in a `user` turn, never the field name or a placeholder, `{}`
otherwise, with a small-talk counter-example.
## Hy Memory 2.0.8 — 2026-10-28
- Published
+33
View File
@@ -682,6 +682,23 @@ Output:
```
John's occupation and location are HIS attributes, so they go into `memory` with "John" as the subject. They must NEVER land in `basic_info.occupation` / `basic_info.location` — `basic_info` is exclusively the user's own profile. Only `name` (Tim, from the user message's `name` field) belongs there.
## Example 11 — Counter-example: small talk has no basic_info
New Messages:
[{"role": "user", "content": "Hi"},
{"role": "assistant", "content": "Hello! The user's name is The user. How can I help?"}]
Memory Date: 2025-09-03
Output:
```json
{
"memory": [],
"intentions": [],
"basic_info": {}
}
```
The user stated nothing about themselves. The assistant's line is not a user turn, and "The user" is a placeholder, not a name. Output `{}` — never `{"name": "The user"}` or `{"name": "user"}`.
---
"""
@@ -749,6 +766,14 @@ Rules:
fields (e.g. name, age, location, occupation, employer) **explicitly stated**
in the NEW user messages. Never infer, never invent. If no field is explicitly
stated, output `{{}}`.
* HARD RULE: fill a field ONLY if the user stated it in a `user` turn.
Anything said only by the assistant (including the assistant quoting an
earlier memory such as "The user's name is X") does NOT count. If nothing
qualifies, output `"basic_info": {{}}`.
* NEVER write the field name itself or a placeholder as a value ("user",
"The user", "用户", "name", "N/A", "unknown", "none", "-"). Omit the field.
* Negative example: small talk such as `[user]: Hi` / `[assistant]: How can
I help?`, or `[user]: Reply with exactly: PONG` → `"basic_info": {{}}`.
* Exception for `name`: the `name` field on the user's own message, or a leading
line "The user's name is X" / "用户名叫X", is an authoritative source for
`name` — even though the user never said it out loud.
@@ -817,6 +842,14 @@ Rules:
fields (e.g. name, age, location, occupation, employer) **explicitly stated**
in the NEW user messages. Never infer, never invent. If no field is explicitly
stated, output `{{}}`.
* HARD RULE: fill a field ONLY if the user stated it in a `user` turn.
Anything said only by the assistant (including the assistant quoting an
earlier memory such as "The user's name is X") does NOT count. If nothing
qualifies, output `"basic_info": {{}}`.
* NEVER write the field name itself or a placeholder as a value ("user",
"The user", "用户", "name", "N/A", "unknown", "none", "-"). Omit the field.
* Negative example: small talk such as `[user]: Hi` / `[assistant]: How can
I help?`, or `[user]: Reply with exactly: PONG` → `"basic_info": {{}}`.
* Exception for `name`: the `name` field on the user's own message, or a leading
line "The user's name is X" / "用户名叫X", is an authoritative source for
`name` — even though the user never said it out loud.
+31
View File
@@ -365,6 +365,13 @@ EXTRACT_PROMPT_ZH = """## 最近 k 条消息
* basic_info:**只装用户本人的画像**。
仅当 NEW user messages 中**明确**陈述稳定画像字段(如 姓名、年龄、所在地、职业、雇主)时才填写。
绝不推断、绝不编造。没有任何字段被明确陈述时输出 `{{}}`。
* **硬性规则**:只有用户在 `user` 回合里**明确说出**的字段才能填写。只在
assistant 回合出现的内容(包括 assistant 引用旧记忆,如"The user's name is X"
/ "用户名是X")一律不算。没有符合条件的字段时,必须输出 `"basic_info": {{}}`。
* **绝不**把字段名本身或占位符当值写入("用户"、"The user"、"user"、"name"、
"姓名"、"N/A"、"unknown"、"未知"、"未提供"、"-"),宁可省略该字段。
* 负例:寒暄如 `[user]: 你好` / `[assistant]: 我可以帮你。`,或
`[user]: Reply with exactly: PONG` → `"basic_info": {{}}`。
* `name` 的例外:用户自己那条消息上的 `name` 字段,或文首形如
"用户名叫X" / "The user's name is X" 的那一行,都是 `name` 的权威来源
——即使用户本人从未开口自报姓名。
@@ -440,6 +447,13 @@ EXTRACT_PROMPT_ZH_NO_INTENTION = """## 最近 k 条消息
* basic_info:**只装用户本人的画像**。
仅当 NEW user messages 中**明确**陈述稳定画像字段(如 姓名、年龄、所在地、职业、雇主)时才填写。
绝不推断、绝不编造。没有任何字段被明确陈述时输出 `{{}}`。
* **硬性规则**:只有用户在 `user` 回合里**明确说出**的字段才能填写。只在
assistant 回合出现的内容(包括 assistant 引用旧记忆,如"The user's name is X"
/ "用户名是X")一律不算。没有符合条件的字段时,必须输出 `"basic_info": {{}}`。
* **绝不**把字段名本身或占位符当值写入("用户"、"The user"、"user"、"name"、
"姓名"、"N/A"、"unknown"、"未知"、"未提供"、"-"),宁可省略该字段。
* 负例:寒暄如 `[user]: 你好` / `[assistant]: 我可以帮你。`,或
`[user]: Reply with exactly: PONG` → `"basic_info": {{}}`。
* `name` 的例外:用户自己那条消息上的 `name` 字段,或文首形如
"用户名叫X" / "The user's name is X" 的那一行,都是 `name` 的权威来源
——即使用户本人从未开口自报姓名。
@@ -669,6 +683,23 @@ Memory Date:2025-09-02
```
职业和所在地是 **John 的**属性,所以进 `memory` 并以"John"作主体,**绝不能**落到 `basic_info` 的 `occupation` / `location` —— `basic_info` 专属于用户本人。只有 `name`(Tim,来自用户消息的 `name` 字段)才该放进去。注意输出语言随输入语言(此例为英文)。
## 示例 11 —— 反例:寒暄没有 basic_info
新消息:
[{"role": "user", "content": "你好"},
{"role": "assistant", "content": "你好!我可以帮你。之前记得 The user's name is 用户."}]
Memory Date:2025-09-03
输出:
```json
{
"memory": [],
"intentions": [],
"basic_info": {}
}
```
用户没有说任何关于自己的信息。assistant 那句不是 user 回合,"用户" 也是占位符而不是名字。必须输出 `{}`——**绝不能**输出 `{"name": "用户"}`。
---
"""
+77
View File
@@ -26,6 +26,8 @@ from __future__ import annotations
import asyncio
import logging
import re
import unicodedata
import uuid
import weakref
from dataclasses import dataclass
@@ -208,6 +210,76 @@ def _sanitize_kv(
return result
# ================================================================
# 占位符守卫(P0-1)
# ================================================================
# 弱模型(如 qwen3.5-2b)在用户什么都没说时,会把 schema 字段说明里的词当值
# 填进 basic_info:{"name": "用户"} / "The user" / "N/A"。每个占位符都会
# supersede 掉真实值,演化链变成 费天翔 → Fei → 用户 → The user。
# 规则:占位符永远不入库;读侧组装 KV 时占位符也不覆盖已有真实值。
_PLACEHOLDER_VALUES = frozenset({
"用户", "该用户", "本用户", "用户本人",
"the user", "the user's", "user", "a user", "this user",
"n/a", "na", "none", "null", "nil", "unknown", "unspecified",
"not provided", "not specified", "not mentioned",
"未提供", "未知", "不详", "未说明", "未提及", "无",
"-", "?",
})
# 内置字段的中文标签:弱模型也会把字段说明的中文词当值。
_FIELD_LABELS: Dict[str, tuple] = {
"name": ("姓名", "名字", "用户名", "用户姓名", "用户名字"),
"age": ("年龄", "用户年龄"),
"location": ("所在地", "地点", "城市", "用户所在地"),
"occupation": ("职业", "工作", "用户职业"),
"employer": ("雇主", "公司", "用户雇主"),
}
# 去掉标点/符号后再比较,让 "The user." / "用户。" / "N/A" 都命中。
# "-" / "?" 本身就是纯标点,剥完为空串,同样判为占位符。
_PUNCT_RE = re.compile(r"[\s\.,;:!?'\"`~()\[\]{}<>/\\|_\-\u3000-\u303f\uff00-\uff0f\uff1a-\uff20]+")
def _placeholder_key(value: str) -> str:
s = unicodedata.normalize("NFKC", str(value or "")).strip().lower()
s = s.replace("\u2019", "'")
s = _PUNCT_RE.sub(" ", s)
return " ".join(s.split())
_PLACEHOLDER_KEYS = frozenset(_placeholder_key(v) for v in _PLACEHOLDER_VALUES)
def is_placeholder_value(field_name: str, value: Any) -> bool:
"""value 是否为占位符(空 / 用户 / The user / N/A / 字段名本身 等)。
大小写不敏感、首尾空白与标点不敏感。数字(如 age=30)永远不是占位符。
"""
if value is None:
return True
if isinstance(value, bool):
return False
if isinstance(value, (int, float)):
return False
key = _placeholder_key(value)
if not key:
return True
if key in _PLACEHOLDER_KEYS:
return True
field_key = _placeholder_key(field_name)
if field_key and key in (field_key, field_key.replace("_", " ")):
return True
for label in _FIELD_LABELS.get(field_name, ()):
if key == _placeholder_key(label):
return True
return False
def _drop_placeholders(kv: Dict[str, Any]) -> Dict[str, Any]:
return {k: v for k, v in kv.items() if not is_placeholder_value(k, v)}
# ================================================================
# 公共入口:upsert_basic_profile
# ================================================================
@@ -287,6 +359,8 @@ async def _upsert_basic_profile_unlocked(
# L0 Basic Info 从输入、唯一性到落盘都只属于 user_id。
new_kv = _sanitize_kv(kv or {}, allowed_fields=allowed_fields)
# 单调性守卫:占位符永远不覆盖已有值,过滤后为空就不写新版本、不产生新链头。
new_kv = _drop_placeholders(new_kv)
if not new_kv:
return BasicProfileUpsertResult(
success=True, changed=False, reason="no valid fields after sanitize"
@@ -513,5 +587,8 @@ def _assemble_full_kv_from_nodes(
for k, v in kv.items():
if allowed_set is not None and k not in allowed_set:
continue
# 历史上已落盘的占位符版本(如 name=用户)不覆盖更早的真实值。
if is_placeholder_value(k, v):
continue
full_kv[k] = v
return full_kv
+2 -2
View File
@@ -13,8 +13,8 @@
接口先补 SDK。Runtime writers 停止后的 App 启动迁移可直接操作底层存储,
但不得把迁移专用方法加入公共 Client。
3. MemHub **不存 memory 实体**;**可以存 `memory_id`** 做组织/回执。原文进入 **SDK raw**。
4. SDK raw 只保留滚动三天;一条 raw 对应一个完整 turn,只含真人 user 与
assistant,tool 不持久化。提取后的高层 memory 不受该窗口影响。
4. SDK raw 全部保留,任何模式(lite / pro / ultra)都不删除;一条 raw 对应一个
完整 turn,只含真人 user 与 assistant,tool 不持久化。
5. `source_raw_ids` 是 SDK 内部溯源关系,不是产品 Evidence 合同;记忆详情不提供
“原始依据”或按 raw ID 展开原文的入口。
+2 -2
View File
@@ -18,7 +18,7 @@ If you run into a problem, please open an Issue with the version, platform,repro
## 发行与平台合同
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.8`。MemHub 源码版本为 `1.0.5`
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.9`。MemHub 源码版本为 `1.0.5`
(build 36),GitHub 仓库为
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
`memhub-cli`。当前代码架构见
@@ -118,7 +118,7 @@ MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime
## Hy Memory
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.8`。SDK 配置、权限边界及 search
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.9`。SDK 配置、权限边界及 search
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
+1 -1
View File
@@ -2352,7 +2352,7 @@
{
"SPDXID": "SPDXRef-Package-hy-memory",
"name": "hy-memory",
"versionInfo": "2.0.8",
"versionInfo": "2.0.9",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"licenseConcluded": "Apache-2.0",
+1 -1
View File
@@ -9,7 +9,7 @@ Per-package license texts are not reproduced here yet; the license
column is filled in as part of release review, and the SBOM records
`NOASSERTION` until then rather than guessing.
## hy-memory 2.0.8
## hy-memory 2.0.9
License: Apache-2.0
@@ -315,6 +315,8 @@ describe("catalog exit codes", () => {
"models",
"memories",
"profile",
"settings",
"evolve",
"graph-width",
"host",
"recall",
+45 -1
View File
@@ -162,10 +162,54 @@ it("never emits a secret from a malformed manually written event", async () => {
command: "logs",
state: "empty",
events: 0,
invalid_lines: 1,
invalid_lines: 0,
unrecognized_lines: 1,
});
});
// After an upgrade the files still hold an older build's records. They are
// whole JSON lines the reader leaves out, not damage, so the text output must
// not warn about them; only a line that is not JSON is called damaged.
it("warns only about lines that are not JSON, not about older records", async () => {
const dataDir = await fixture();
const older = {
schema_version: 0,
service: "desktop",
component: "desktop.lifecycle",
event: "desktop.started",
};
await writeFile(
path.join(dataDir, "logs", "diagnostic-desktop.jsonl"),
`${JSON.stringify(older)}\n${JSON.stringify({ ...older, schema_version: 1 })}\n`,
{ mode: 0o600 },
);
let stdout = "";
let stderr = "";
vi.spyOn(process.stdout, "write").mockImplementation((value) => {
stdout += String(value);
return true;
});
vi.spyOn(process.stderr, "write").mockImplementation((value) => {
stderr += String(value);
return true;
});
expect(await run(["logs"], { ONE_MEMORY_DATA_DIR: dataDir })).toBe(0);
expect(stdout).toContain("core.start_failed");
expect(stdout).not.toContain("损坏");
expect(stderr).toBe("");
await writeFile(
path.join(dataDir, "logs", "diagnostic-cli.jsonl"),
'{"schema_version": 1, "event": \n',
{ mode: 0o600 },
);
stdout = "";
expect(await run(["logs"], { ONE_MEMORY_DATA_DIR: dataDir })).toBe(0);
expect(stdout).toContain("跳过 1 行损坏的诊断记录。");
expect(stderr).toBe("memhub: 跳过 1 行损坏的诊断记录。\n");
});
it("rejects invalid filters without touching the log directory", async () => {
const dataDir = path.join(
await mkdtemp(path.join(os.tmpdir(), "om-cli-no-log-")),
+323 -13
View File
@@ -188,10 +188,24 @@ import {
removeManagedCliLink,
rollbackManagedCliLink,
} from "./managed-cli-link.js";
import {
type EvolveDocument,
evolveExitCode,
type ModeDocument,
modeExitCode,
parseEvolveCommand,
parseSettingsCommand,
renderEvolveText,
renderModeText,
runEvolveCommand,
runModeCommand,
} from "./memory-mode-flow.js";
import {
addMemory,
DEFAULT_USER_ID,
isMemoryId,
isRequestId,
isUserId,
MEMORY_KINDS,
MEMORY_SCENES,
MEMORY_VARIANTS,
@@ -209,6 +223,7 @@ import {
renderMemoryAddText,
renderMemoryDetailText,
renderMemoryPageText,
userIdParameters,
validMemoryText,
} from "./memory-report.js";
import {
@@ -322,6 +337,8 @@ export interface RunControls {
watchSignal?: AbortSignal;
watchWait?: (milliseconds: number, signal: AbortSignal) => Promise<void>;
watchNow?: () => string;
/** `memhub evolve --wait` polls on this; tests inject an instant one. */
sleep?: (milliseconds: number) => Promise<void>;
readStdin?: () => Promise<Buffer | null>;
stdinIsTTY?: boolean;
/** Injectable onboarding prompter so the wizard can be driven in tests
@@ -538,6 +555,23 @@ async function runCommand(
environment = { ...environment, ONE_MEMORY_DATA_DIR: lifted.dataDir };
}
}
// `--user-id <id>` is global to the memory commands: it names whose
// memories are read or written. It is lifted before dispatch so each
// command's own parser keeps seeing exactly the arguments it always did.
// `sources connect` gives the flag its own meaning and parses it itself.
let userId: string | undefined;
if (USER_ID_COMMANDS.has(arguments_[0] ?? "")) {
const lifted = liftUserIdOption(arguments_);
if (lifted === null) {
return reject(
findCommand(arguments_[0] ?? ""),
asJson,
"--user-id 无效:只能是 1–64 位字母、数字、点、下划线或连字符。",
);
}
arguments_ = lifted.arguments_;
userId = lifted.userId;
}
environment = await environmentWithManagedDataDir(environment);
const command = arguments_[0];
if (command === undefined) return reject(undefined, asJson, "缺少命令。");
@@ -776,9 +810,11 @@ async function runCommand(
const attachment = await attachToRuntimeOrNull(environment);
if (attachment === null) return runtimeUnavailable(descriptor, asJson);
if (parsed.kind === "status") {
const receipt = await readMemoryWrite(attachment, parsed.id).catch(
() => undefined,
);
const receipt = await readMemoryWrite(
attachment,
parsed.id,
userId,
).catch(() => undefined);
if (receipt === undefined) {
return emitFailure(asJson, "write status", "memory_write_read_failed");
}
@@ -812,6 +848,7 @@ async function runCommand(
? {}
: { memory_at: parsed.memory_at }),
...(parsed.priority === undefined ? {} : { priority: parsed.priority }),
...(userId === undefined ? {} : { user_id: userId }),
});
if (outcome.state !== "accepted") {
emit(
@@ -957,9 +994,12 @@ async function runCommand(
state: "empty",
events: 0,
invalid_lines: report.invalid_lines,
unrecognized_lines: report.unrecognized_lines,
})}\n`
: renderLogText(report.events, report.invalid_lines),
);
// Only lines that are not JSON are damage. Records an older build wrote
// are left out without a warning: after an upgrade they are expected.
if (report.invalid_lines) {
process.stderr.write(
`${PRODUCT}: 跳过 ${report.invalid_lines} 行损坏的诊断记录。\n`,
@@ -1582,6 +1622,7 @@ async function runCommand(
limit: parsed.limit,
character_budget: parsed.characterBudget,
timeout_ms: parsed.timeoutMs,
...userIdParameters(userId),
})
: ({
schema_version: 2,
@@ -1780,6 +1821,16 @@ async function runCommand(
}
if (named === "sources" && arguments_[1] !== "list") {
const parsed = parseSourcesCommand(arguments_.slice(1));
if (
!parsed &&
arguments_.some((item) => item === "--user-id" || item === "--root")
) {
return reject(
descriptor,
asJson,
"--user-id 和 --root 只能用于接入单个 hermes 来源(Hermes profile),用户 ID 只能是 1–64 位字母、数字、点、下划线或连字符。",
);
}
if (!parsed) return usage();
if (parsed.kind === "receipt") {
const connected = await connectRuntime(environment);
@@ -1981,12 +2032,17 @@ async function runCommand(
return sourceProgressExitCode(document);
}
if (named === "profile") {
const profileFlags = arguments_.slice(1);
if (
arguments_.filter((item) => item === "--json").length > 1 ||
arguments_.slice(1).some((value) => value !== "--json")
profileFlags.filter((item) => item === "--json").length > 1 ||
profileFlags.filter((item) => item === "--versions").length > 1 ||
profileFlags.some((value) => value !== "--json" && value !== "--versions")
) {
return usage();
}
if (profileFlags.includes("--versions")) {
return await profileVersionsCommand(environment, { asJson, userId });
}
const connected = await connectRuntime(environment, {
readerTimeoutMs: 15_000,
});
@@ -2004,7 +2060,11 @@ async function runCommand(
);
return connected.exitCode;
}
const outcome = await connected.reader<unknown>("/api/v2/user-profile");
const profileUser = userIdParameters(userId);
const outcome =
profileUser.user_id === undefined
? await connected.reader<unknown>("/api/v2/user-profile")
: await connected.reader<unknown>("/api/v2/user-profile", profileUser);
const value =
outcome.state === "ok"
? (outcome.data as {
@@ -2109,7 +2169,7 @@ async function runCommand(
}
const { reader, operations } = connected;
if (parsed.kind === "list") {
const document = await readMemoryPage(reader, parsed.query);
const document = await readMemoryPage(reader, parsed.query, userId);
emit(
asJson
? `${JSON.stringify(document)}\n`
@@ -2122,6 +2182,7 @@ async function runCommand(
reader,
parsed.memoryId,
parsed.namespace,
userId,
);
emit(
asJson
@@ -2133,8 +2194,8 @@ async function runCommand(
if (parsed.kind === "add" || parsed.kind === "add-status") {
const document =
parsed.kind === "add"
? await addMemory(operations, text ?? "", parsed.requestId)
: await readMemoryAddStatus(reader, parsed.requestId);
? await addMemory(operations, text ?? "", parsed.requestId, userId)
: await readMemoryAddStatus(reader, parsed.requestId, userId);
emit(
asJson
? `${JSON.stringify(document)}\n`
@@ -2149,9 +2210,13 @@ async function runCommand(
content: text,
...namespaceParameters(parsed.namespace),
...(parsed.task === undefined ? {} : { task: parsed.task }),
...userIdParameters(userId),
})
: await operations.delete<unknown>(
withQuery(memoryPath, namespaceParameters(parsed.namespace)),
withQuery(memoryPath, {
...namespaceParameters(parsed.namespace),
...userIdParameters(userId),
}),
);
const document = {
schema_version: 2,
@@ -2177,6 +2242,55 @@ async function runCommand(
);
return outcome.state === "committed" ? EXIT_OK : EXIT_FAILED;
}
if (named === "settings") {
const parsed = parseSettingsCommand(arguments_.slice(1));
if (!parsed) return usage();
const connected = await connectRuntime(environment);
if (!connected.ok) {
const document: ModeDocument = {
schema_version: 1,
product: PRODUCT,
command: "settings mode",
state: "unavailable",
code: connected.code,
};
emit(asJson ? `${JSON.stringify(document)}\n` : renderModeText(document));
return connected.exitCode;
}
const document = await runModeCommand(
parsed,
connected.reader,
connected.operations,
);
emit(asJson ? `${JSON.stringify(document)}\n` : renderModeText(document));
return modeExitCode(document);
}
if (named === "evolve") {
const parsed = parseEvolveCommand(arguments_.slice(1));
if (!parsed) return usage();
const connected = await connectRuntime(environment);
if (!connected.ok) {
const document: EvolveDocument = {
schema_version: 1,
product: PRODUCT,
command: "evolve",
state: "unavailable",
code: connected.code,
};
emit(
asJson ? `${JSON.stringify(document)}\n` : renderEvolveText(document),
);
return connected.exitCode;
}
const document = await runEvolveCommand(parsed, {
userId: userIdParameters(userId).user_id,
reader: connected.reader,
operations: connected.operations,
sleep: controls.sleep,
});
emit(asJson ? `${JSON.stringify(document)}\n` : renderEvolveText(document));
return evolveExitCode(document);
}
if (named === "graph-width") {
const parsed = parseGraphWidthCommand(arguments_.slice(1));
if (!parsed) return usage();
@@ -3675,13 +3789,21 @@ async function connectSources(
asJson,
);
}
if (parsed.importHistory) {
// Another user's Hermes profile is granted and imported in one connect
// request: only that request can carry a user and a root.
if (
parsed.importHistory ||
parsed.userId !== undefined ||
parsed.root !== undefined
) {
const document = await connectSourcesWithImport(reader, operations, {
sourceIds: parsed.sourceIds,
confirmOverrides: parsed.confirmOverrides,
...(parsed.requestId === undefined
? {}
: { requestId: parsed.requestId }),
...(parsed.userId === undefined ? {} : { userId: parsed.userId }),
...(parsed.root === undefined ? {} : { root: parsed.root }),
});
emitFlowDocument(document, asJson);
return flowExitCode(document);
@@ -3834,6 +3956,108 @@ async function emitModelReceipt(
/** Attach to a verified Runtime, telling a stopped one (exit 3) from one
* that is up but unusable (exit 1). */
/**
* `memhub profile --versions`: the long-term profile's L0 version chains,
* latest first, so a value that replaced a better one can be found by id.
*/
async function profileVersionsCommand(
environment: NodeJS.ProcessEnv,
options: { asJson: boolean; userId?: string },
): Promise<number> {
const fail = (state: string, code: string, exitCode: number): number => {
emit(
options.asJson
? `${JSON.stringify({ schema_version: 2, product: PRODUCT, state, code })}\n`
: `画像版本读取失败:${code}\n`,
);
return exitCode;
};
const connected = await connectRuntime(environment, {
readerTimeoutMs: 15_000,
});
if (!connected.ok) {
return fail("unavailable", connected.code, connected.exitCode);
}
const outcome = await connected.reader<unknown>(
"/api/v2/user-profile/versions",
userIdParameters(options.userId),
);
const chains = outcome.state === "ok" ? profileChains(outcome.data) : null;
if (outcome.state !== "ok" || chains === null) {
return fail(
outcome.state === "ok" ? "invalid" : outcome.state,
outcome.state === "ok" ? "response_invalid" : outcome.code,
EXIT_FAILED,
);
}
if (options.asJson) {
emit(
`${JSON.stringify({
schema_version: 2,
product: PRODUCT,
state: "ok",
...(outcome.data as object),
})}\n`,
);
return EXIT_OK;
}
if (chains.length === 0) {
emit("尚未形成用户画像版本。\n");
return EXIT_OK;
}
const lines: string[] = [];
for (const chain of chains) {
lines.push(`画像 ${chain.memory_id}(${chain.status})`);
for (const version of chain.versions) {
const when =
typeof version.gmt_created === "number"
? new Date(version.gmt_created * 1000).toISOString()
: "时间未知";
lines.push(
` ${version.current ? "*" : " "} ${version.memory_id} ${when} ${version.content}`,
);
}
}
lines.push(
"* 表示当前生效的版本;删除错误版本:memhub memories delete <memory-id> --yes",
);
emit(`${lines.join("\n")}\n`);
return EXIT_OK;
}
type ProfileChainView = {
memory_id: string;
status: string;
versions: {
memory_id: string;
content: string;
gmt_created?: number | null;
current: boolean;
}[];
};
function profileChains(value: unknown): ProfileChainView[] | null {
if (!value || typeof value !== "object") return null;
const chains = (value as { chains?: unknown }).chains;
if (!Array.isArray(chains)) return null;
const valid = chains.every(
(chain) =>
chain &&
typeof chain === "object" &&
typeof (chain as ProfileChainView).memory_id === "string" &&
typeof (chain as ProfileChainView).status === "string" &&
Array.isArray((chain as ProfileChainView).versions) &&
(chain as ProfileChainView).versions.every(
(version) =>
version &&
typeof version.memory_id === "string" &&
typeof version.content === "string" &&
typeof version.current === "boolean",
),
);
return valid ? (chains as ProfileChainView[]) : null;
}
async function connectRuntime(
environment: NodeJS.ProcessEnv,
options?: { readerTimeoutMs?: number },
@@ -4101,6 +4325,8 @@ const SDK_MIGRATION_FAILURES: Record<string, string> = {
"核对发现新版本还读不到你的全部记忆,没有切换数据源。",
sdk_migration_model_profile_required:
"迁移需要先有可用的模型配置,用来核对新版本能否读回记忆。请先用 memhub models setup 完成配置,再运行 memhub restart 重试。",
sdk_migration_vector_profile_missing:
"记忆的向量来自一个已删除的模型配置,当前配置读不回它们,所以没有迁移。请重新添加并启用那个模型配置,再运行 memhub restart 重试。",
};
const MIGRATION_ISSUES_URL =
@@ -5131,14 +5357,23 @@ type SourcesCommand =
importHistory: boolean;
confirmOverrides: boolean;
requestId?: string;
/** `--user-id`: whose grant (Hermes only). Absent for `local`. */
userId?: string;
/** `--root`: an absolute root other than the candidate (Hermes only). */
root?: string;
}
| { kind: "all"; importHistory: boolean; confirmOverrides: boolean }
| { kind: "receipt"; requestId: string };
const SOURCE_ID_SHAPE = /^[a-z0-9][a-z0-9-]{0,127}$/;
/** Only Hermes keeps one history per profile, so only Hermes may be
* connected for another user or from another root. */
const MULTI_USER_SOURCES = new Set(["hermes"]);
/** `sources connect <id> [<id> ...] | --all-detected [--import]
* [--confirm-overrides] [--request-id <uuid>]` and `sources receipt <id>`.
* [--confirm-overrides] [--request-id <uuid>]
* [--user-id <id>] [--root <path>]` and `sources receipt <id>`.
*
* Sources may be named positionally or with repeated `--source` (the old
* `init --source` spelling is rewritten to this command). */
@@ -5154,6 +5389,8 @@ function parseSourcesCommand(rest: readonly string[]): SourcesCommand | null {
const sourceIds: string[] = [];
const switches = new Set<string>();
let requestId: string | undefined;
let userId: string | undefined;
let root: string | undefined;
for (let index = 1; index < tokens.length; index += 1) {
const token = tokens[index];
if (["--import", "--confirm-overrides", "--all-detected"].includes(token)) {
@@ -5161,6 +5398,21 @@ function parseSourcesCommand(rest: readonly string[]): SourcesCommand | null {
switches.add(token);
continue;
}
if (token === "--user-id" || token === "--root") {
const value = tokens[index + 1];
if (value === undefined || value.startsWith("--")) return null;
if (token === "--user-id") {
if (userId !== undefined || !isUserId(value)) return null;
userId = value;
} else {
if (root !== undefined) return null;
const expanded = expandRootPath(value);
if (expanded === null) return null;
root = expanded;
}
index += 1;
continue;
}
if (token === "--source" || token === "--request-id") {
const value = tokens[index + 1];
if (value === undefined || value.startsWith("--")) return null;
@@ -5184,10 +5436,23 @@ function parseSourcesCommand(rest: readonly string[]): SourcesCommand | null {
const confirmOverrides = switches.has("--confirm-overrides");
if (
requestId !== undefined &&
(!OPERATION_ID.test(requestId) || !importHistory)
(!OPERATION_ID.test(requestId) ||
!(importHistory || userId !== undefined || root !== undefined))
) {
return null;
}
// `--user-id local` is the default and changes nothing.
if (userId === DEFAULT_USER_ID) userId = undefined;
if (userId !== undefined || root !== undefined) {
// One Hermes profile per connect: the user and root name one grant.
if (
switches.has("--all-detected") ||
sourceIds.length !== 1 ||
!MULTI_USER_SOURCES.has(sourceIds[0])
) {
return null;
}
}
if (switches.has("--all-detected")) {
// One names exact sources, the other means "whatever is detected".
if (sourceIds.length > 0 || requestId !== undefined) return null;
@@ -5200,9 +5465,24 @@ function parseSourcesCommand(rest: readonly string[]): SourcesCommand | null {
importHistory,
confirmOverrides,
...(requestId === undefined ? {} : { requestId }),
...(userId === undefined ? {} : { userId }),
...(root === undefined ? {} : { root }),
};
}
/** `--root` as an absolute path: a leading `~` is the home directory, a
* relative path is taken from the working directory. Core expands nothing. */
function expandRootPath(value: string): string | null {
if (value.length === 0 || /[\0\r\n]/.test(value)) return null;
const expanded =
value === "~"
? os.homedir()
: value.startsWith("~/")
? path.join(os.homedir(), value.slice(2))
: value;
return path.resolve(expanded);
}
function parseGraphWidthCommand(
rest: readonly string[],
): { decision: GraphWidthDecision; confirmed: boolean } | null {
@@ -6596,6 +6876,36 @@ const OWN_DATA_DIR_COMMANDS = new Set([
/** Remove one `--data-dir <absolute path>` from argv; null when it is
* repeated, missing its value, or not an absolute path. */
/** Memory commands that honour the global `--user-id`. */
const USER_ID_COMMANDS = new Set([
"memories",
"recall",
"profile",
"write",
"evolve",
]);
/**
* Take `--user-id <id>` out of the arguments. The default user (`local`) is
* returned as undefined, so a default request names no user at all and stays
* byte-identical to one sent before multi-user support.
*/
function liftUserIdOption(
arguments_: readonly string[],
): { arguments_: string[]; userId?: string } | null {
const index = arguments_.indexOf("--user-id");
if (index === -1) return { arguments_: [...arguments_] };
if (arguments_.indexOf("--user-id", index + 1) !== -1) return null;
const value = arguments_[index + 1];
if (value === undefined || value.startsWith("--") || !isUserId(value)) {
return null;
}
return {
arguments_: [...arguments_.slice(0, index), ...arguments_.slice(index + 2)],
...(value === DEFAULT_USER_ID ? {} : { userId: value }),
};
}
function liftDataDirOption(
arguments_: readonly string[],
): { arguments_: string[]; dataDir?: string } | null {
+73 -8
View File
@@ -68,6 +68,12 @@ const JSON_OPTION: CommandOption = {
flag: "--json",
summary: "以 JSON 输出到 stdout,供脚本与 Agent 消费。",
};
/** Global to the memory commands: whose memories are read or written. */
const USER_ID_OPTION: CommandOption = {
flag: "--user-id <id>",
summary:
"读写哪个用户的记忆,默认 local(App 用的就是 local)。1–64 位字母、数字、点、下划线或连字符;其他用户的记忆来自用 sources connect hermes --user-id 接入的 Hermes profile。",
};
const WATCH_OPTION: CommandOption = {
flag: "--watch",
summary: "串行持续刷新;Ctrl-C 只停止观察,不停止 Runtime。",
@@ -252,15 +258,25 @@ export const COMMANDS: readonly CommandDescriptor[] = [
{
name: "sources",
summary:
"列出来源,直接接入来源,或按 request id 查接入回执。connect 不加 --import 只授权(处理保持冻结:不导入历史、不扫描新会话);加 --import 同时导入历史并开启持续同步,返回 request id。目录来自环境变量覆盖的来源必须加 --confirm-overrides。",
"列出来源,直接接入来源,或按 request id 查接入回执。connect 不加 --import 只授权(处理保持冻结:不导入历史、不扫描新会话);加 --import 同时导入历史并开启持续同步,返回 request id。目录来自环境变量覆盖的来源必须加 --confirm-overrides。Hermes 的每个 profile 是一个独立用户:用 --user-id 和 --root 接入,授权并导入历史(会自动排除 ~/.hermes/profiles/**,各用户互不读取)。",
usage:
"memhub sources list [--json]\n memhub sources connect <id> [<id> ...] [--import] [--confirm-overrides] [--request-id <uuid>] [--json]\n memhub sources connect --all-detected [--import] [--confirm-overrides] [--json]\n memhub sources receipt <request-id> [--json]",
"memhub sources list [--json]\n memhub sources connect <id> [<id> ...] [--import] [--confirm-overrides] [--request-id <uuid>] [--json]\n memhub sources connect hermes --user-id <id> --root <path> [--import] [--request-id <uuid>] [--json]\n memhub sources connect --all-detected [--import] [--confirm-overrides] [--json]\n memhub sources receipt <request-id> [--json]",
options: [
{
flag: "--import",
summary:
"接入后立即导入历史会话并开启持续同步;不加时只授权,处理保持冻结。",
},
{
flag: "--user-id <id>",
summary:
"只用于 hermes:这条接入属于哪个用户(默认 local)。和 --root 一起接入一个 Hermes profile,例如 --user-id wife --root ~/.hermes/profiles/wife;其他来源只能是 local。",
},
{
flag: "--root <path>",
summary:
"只用于 hermes:读取的根目录(~ 会展开为主目录)。默认是 Hermes 自己的目录 ~/.hermes。",
},
{
flag: "--all-detected",
summary:
@@ -373,7 +389,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
summary:
"列出、搜索、查看、添加、修改或物理删除记忆(对话、工作、编码三个场景)。add 和 App 的「添加记忆」是同一个入口;delete 不可逆,需 --yes。",
usage:
"memhub memories list [--scene <all|chat|working|coding>] [--source <id>] [--query <text>] [--status <state>] [--offset <n>] [--limit <n>] [--json]\n memhub memories show <memory-id> [--kind <kind>] [--variant <variant>] [--json]\n memhub memories add <text> [--request-id <uuid>] [--json]\n memhub memories add --content-file <path> [--request-id <uuid>] [--json]\n memhub memories add-status <request-id> [--json]\n memhub memories update <memory-id> --content-file <path> [--kind <kind>] [--variant <variant>] [--task <text>] [--json]\n memhub memories delete <memory-id> [--kind <kind>] [--variant <variant>] [--yes] [--json]",
"memhub memories list [--scene <all|chat|working|coding>] [--source <id>] [--query <text>] [--status <state>] [--offset <n>] [--limit <n>] [--user-id <id>] [--json]\n memhub memories show <memory-id> [--kind <kind>] [--variant <variant>] [--user-id <id>] [--json]\n memhub memories add <text> [--request-id <uuid>] [--user-id <id>] [--json]\n memhub memories add --content-file <path> [--request-id <uuid>] [--user-id <id>] [--json]\n memhub memories add-status <request-id> [--user-id <id>] [--json]\n memhub memories update <memory-id> --content-file <path> [--kind <kind>] [--variant <variant>] [--task <text>] [--user-id <id>] [--json]\n memhub memories delete <memory-id> [--kind <kind>] [--variant <variant>] [--yes] [--user-id <id>] [--json]",
options: [
{
flag: "--scene <all|chat|working|coding>",
@@ -411,15 +427,62 @@ export const COMMANDS: readonly CommandDescriptor[] = [
"add 的幂等键;用同一个 id 重试只会读回第一次的结果,不会重复添加。不给时自动生成并输出。",
},
DESTRUCTIVE_CONFIRM_OPTION,
USER_ID_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
},
{
name: "profile",
summary: "读取 SDK 组装后的用户长期画像(L0 + 已启用的长期画像补充)。",
usage: "memhub profile [--json]",
options: [JSON_OPTION],
summary:
"读取 SDK 组装后的用户长期画像(L0 + 已启用的长期画像补充);--versions 列出 L0 画像的版本链(最新在前,* 为当前版本)。",
usage: "memhub profile [--versions] [--user-id <id>] [--json]",
options: [
{
flag: "--versions",
summary:
"列出 L0 基本信息的版本链(get_user_basic_info),用于找出覆盖了正确值的错误版本。",
},
USER_ID_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
},
{
name: "settings",
summary:
"查看或切换记忆模式(lite / pro / ultra)。lite 不调用模型,只在原始对话里检索;pro 由模型提取记忆(System 1);ultra 在 pro 之上每晚自动进化(System 2,使用知识图谱)。任何模式都保留全部原始对话。切换不迁移数据:已处理的不会重做,排队中的和新的按新模式处理;需 --yes,不加时只显示计划。环境变量 ONE_MEMORY_GRAPH_STORE 会强制 ultra。",
usage: "memhub settings mode [lite|pro|ultra] [--yes] [--json]",
options: [
{
flag: "mode [lite|pro|ultra]",
summary: "不带模式只显示当前模式;带模式时先显示切换计划。",
},
CONFIRM_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
},
{
name: "evolve",
summary:
"立即为一个用户运行记忆进化(System 2,仅 ultra 模式)。在后台运行并返回运行 ID;--wait 等待结果。同一用户同时只运行一次;每晚的自动进化由 Runtime 按设置安排。",
usage:
"memhub evolve [--user-id <id>] [--agent-id <id>] [--max-batches <n>] [--wait] [--json]",
options: [
USER_ID_OPTION,
{
flag: "--agent-id <id>",
summary:
"进化哪个 agent 的记忆,默认 all-apps(MemHub 写入记忆用的 agent)。",
},
{
flag: "--max-batches <n>",
summary: "每轮最多处理的批次数(1–1000),默认由 SDK 决定。",
},
{ flag: "--wait", summary: "等待这次进化结束再返回。" },
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
},
{
@@ -475,7 +538,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
summary:
"从原始对话与已沉淀记忆统一检索上下文;本地只读,不调用模型、不触发补索引。文本输出标明每条是「原始对话摘录」还是「记忆」(JSON 里是 kind: raw|memory)。",
usage:
"memhub recall <query> [--kind <raw|memory>] [--source <id>] [--limit <n>] [--character-budget <n>] [--timeout-ms <n>] [--json]\n memhub recall --query-file <path> [...]\n memhub recall --stdin [...]",
"memhub recall <query> [--kind <raw|memory>] [--source <id>] [--limit <n>] [--character-budget <n>] [--timeout-ms <n>] [--user-id <id>] [--json]\n memhub recall --query-file <path> [...]\n memhub recall --stdin [...]",
options: [
{
flag: "--query-file <path>",
@@ -502,6 +565,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
flag: "--timeout-ms <n>",
summary: "Core 查询时限 50..30000 毫秒,默认 2000。",
},
USER_ID_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
@@ -650,7 +714,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
summary:
"提交一段 OpenAI 格式的对话(messages)到写入队列,异步加工成记忆;或按 request id 查询状态。request id 是幂等键:不给时自动生成并输出,重试请带上同一个。新增一条结论式记忆请用 memories add。",
usage:
"memhub write submit --messages-file <path> --session <id> [--source <id>] [--request-id <id>] [--memory-at <RFC3339>] [--live] [--json]\n memhub write status <request-id> [--json]",
"memhub write submit --messages-file <path> --session <id> [--source <id>] [--request-id <id>] [--memory-at <RFC3339>] [--live] [--user-id <id>] [--json]\n memhub write status <request-id> [--user-id <id>] [--json]",
options: [
{
flag: "--messages-file <path>",
@@ -666,6 +730,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
},
{ flag: "--memory-at <RFC3339>", summary: "对话发生时间,必须带时区。" },
{ flag: "--live", summary: "优先处理。" },
USER_ID_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
@@ -76,6 +76,8 @@ const COMMANDS = [
"models",
"memories",
"profile",
"settings",
"evolve",
"graph-width",
"host",
"recall",
+2
View File
@@ -10,6 +10,8 @@ export interface ContextSearchInput {
limit: number;
character_budget: number;
timeout_ms: number;
/** Whose memories to search. Omitted for the default user (`local`). */
user_id?: string;
}
/** The CLI's JSON document mirrors Core's SDK-native context schema v2. */
+18
View File
@@ -12,11 +12,18 @@ const READ_ONLY_PATHS = [
"/api/v1/live-sync",
"/api/v1/memory-processing",
"/api/v1/model-profiles",
"/api/v1/source-grants",
"/api/v1/sources",
"/api/v2/home-summary",
"/api/v2/memory-library",
"/api/v2/user-profile",
"/api/v2/user-profile/versions",
"/api/v1/settings/memory-mode",
"/api/v2/evolve",
] as const;
/** One evolve run's receipt (`memhub evolve --wait`). */
const EVOLVE_RUN =
/^\/api\/v2\/evolve\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const SDK_MEMORY_DETAIL = /^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}$/;
/** Status of one `POST /api/v2/memories` addition, by its request id. */
const MEMORY_ADDITION =
@@ -219,6 +226,7 @@ export function isReadOnlyPath(path: string): boolean {
if (IMPORT_JOB.test(path)) return true;
if (OPERATION_RECEIPT.test(path)) return true;
if (MEMORY_ADDITION.test(path)) return true;
if (EVOLVE_RUN.test(path)) return true;
if (!SDK_MEMORY_DETAIL.test(path)) return false;
const segment = path.slice("/api/v2/memories/".length);
return !(RESERVED_MEMORY_SEGMENTS as readonly string[]).includes(segment);
@@ -310,6 +318,8 @@ export async function submitMemoryWrite(
messages: readonly OpenAIMessage[];
memory_at?: string;
priority?: "live" | "normal";
/** Whose memory this is. Omitted for the default user (`local`). */
user_id?: string;
},
): Promise<MemoryWriteOutcome> {
let response: { status: number; data: unknown };
@@ -327,6 +337,9 @@ export async function submitMemoryWrite(
...(request.priority === undefined
? {}
: { priority: request.priority }),
...(request.user_id === undefined || request.user_id === "local"
? {}
: { user_id: request.user_id }),
},
{
headers: { Authorization: `Bearer ${attachment.apiToken}` },
@@ -469,11 +482,16 @@ export async function deleteModelProfile(
export async function readMemoryWrite(
attachment: RuntimeAttachmentLike,
requestId: string,
/** A receipt belongs to the user that submitted it; omit for `local`. */
userId?: string,
): Promise<MemoryWriteReceipt | null> {
try {
const response = await axios.get<MemoryWriteReceipt>(
`${baseUrlOf(attachment)}/api/v2/memory-writes/${encodeURIComponent(requestId)}`,
{
...(userId === undefined || userId === "local"
? {}
: { params: { user_id: userId } }),
headers: { Authorization: `Bearer ${attachment.apiToken}` },
timeout: 10_000,
proxy: false,
+11 -3
View File
@@ -6,9 +6,10 @@ const WRITE_PATHS = [
] as const;
/** `POST /api/v2/memories` adds one memory (the app's 添加记忆). */
const MEMORY_ADD_PATH = "/api/v2/memories";
/** One SDK memory; delete names its namespace in the query string. */
/** One SDK memory; delete names its namespace, and a non-default user, in
* the query string. */
const SDK_MEMORY_PATH =
/^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}(?:\?kind=(?:chat|experience|fact)(?:&variant=(?:working|coding))?)?$/;
/^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}(?:\?(?:kind=(?:chat|experience|fact)(?:&variant=(?:working|coding))?(?:&user_id=[A-Za-z0-9._-]{1,64})?|user_id=[A-Za-z0-9._-]{1,64}))?$/;
const MODEL_PROBE_PATH =
/^\/api\/v1\/model-profiles\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\/probe$/;
const MODEL_PROFILE_PATH =
@@ -25,6 +26,9 @@ const SOURCE_IMPORT_JOB_PATH = "/api/v1/import-jobs";
const IMPORT_JOB_ACTION_PATH =
/^\/api\/v1\/import-jobs\/[A-Za-z0-9][\w.-]{0,127}\/(?:pause|resume|cancel)$/;
const GRAPH_WIDTH_PATH = "/api/v1/graph-width";
/** `memhub evolve` starts a run; `memhub settings mode <m> --yes` sets the mode. */
const EVOLVE_PATH = "/api/v2/evolve";
const MEMORY_MODE_PATH = "/api/v1/settings/memory-mode";
const ONBOARDING_COMPLETE_PATH = "/api/v1/onboarding/complete";
const MAX_BYTES = 4 * 1024 * 1024;
@@ -96,6 +100,7 @@ export function createCoreOperations(attachment: {
!SOURCE_AUTHORIZE_PATH.test(path) &&
!SOURCE_REVOKE_PATH.test(path) &&
path !== GRAPH_WIDTH_PATH &&
path !== EVOLVE_PATH &&
path !== ONBOARDING_COMPLETE_PATH &&
!MODEL_PROFILE_ACTIVATE.test(path)
) {
@@ -131,7 +136,10 @@ export function createCoreOperations(attachment: {
body: Record<string, unknown>,
signal?: AbortSignal,
) {
if (!SDK_MEMORY_PATH.test(path) || path.includes("?")) {
if (
path !== MEMORY_MODE_PATH &&
(!SDK_MEMORY_PATH.test(path) || path.includes("?"))
) {
throw new Error("core_operation_path_not_allowed");
}
let response: { status: number; data: unknown };
+29 -7
View File
@@ -110,9 +110,18 @@ export async function connectSourcesWithImport(
sourceIds: readonly string[];
confirmOverrides: boolean;
requestId?: string;
/** Whose grant this is. Omitted for the default user (`local`). */
userId?: string;
/** An absolute root other than the source's candidate (a Hermes
* profile). Core reads its identity itself, so the source's candidate
* root id and revision are not sent for it. */
root?: string;
},
): Promise<FlowDocument> {
const sourceIds = input.sourceIds;
// A second user's grant is not a first run, and a custom root has no
// candidate the app showed: both are a plain connect.
const profile = input.root !== undefined || input.userId !== undefined;
if (
sourceIds.length < 1 ||
sourceIds.length > 32 ||
@@ -134,9 +143,14 @@ export async function connectSourcesWithImport(
return invalid("source_unavailable");
}
const items = selected.map(([candidate]) => candidate);
const needsOverride = items
.filter((item) => item.requires_override_confirmation)
.map((item) => item.source_id);
// A custom root is the caller's own choice, not the environment override
// the confirmation exists for.
const needsOverride =
input.root !== undefined
? []
: items
.filter((item) => item.requires_override_confirmation)
.map((item) => item.source_id);
if (needsOverride.length > 0 && !input.confirmOverrides) {
return {
schema_version: 1,
@@ -154,18 +168,22 @@ export async function connectSourcesWithImport(
request = buildSourceImportRequest({
sourceIds: [...sourceIds],
sourceRoots: Object.fromEntries(
items.map((item) => [item.source_id, item.candidate_root_id]),
items.flatMap((item) =>
input.root === undefined
? [[item.source_id, item.candidate_root_id]]
: [],
),
),
sourceCandidateRevisions: Object.fromEntries(
items.flatMap((item) =>
item.candidate_revision === undefined
item.candidate_revision === undefined || input.root !== undefined
? []
: [[item.source_id, item.candidate_revision]],
),
),
confirmEnvironmentOverrides: needsOverride.length > 0,
fullRescan: false,
onboarding: true,
onboarding: !profile,
enableSync: true,
requestId,
});
@@ -175,7 +193,11 @@ export async function connectSourcesWithImport(
}
throw error;
}
const result = await operations.post<unknown>(request.path, request.body);
const result = await operations.post<unknown>(request.path, {
...request.body,
...(input.userId === undefined ? {} : { user_id: input.userId }),
...(input.root === undefined ? {} : { root: input.root }),
});
if (result.state === "rejected") {
return rejected(requestId, result.code);
}
+6
View File
@@ -107,7 +107,11 @@ export async function readLogReport(
query: LogQuery,
): Promise<{
events: DiagnosticEventV1[];
/** Lines that are not JSON: the only ones `memhub logs` calls damaged. */
invalid_lines: number;
/** Whole JSON lines the reader leaves out, such as an older build's
* records. Nothing is wrong with them, so the text output says nothing. */
unrecognized_lines: number;
failure?: "log_directory_unreadable";
}> {
// `readDiagnosticEvents` treats an unreadable logs path as "no events". That
@@ -119,6 +123,7 @@ export async function readLogReport(
return {
events: [],
invalid_lines: 0,
unrecognized_lines: 0,
failure: "log_directory_unreadable",
};
}
@@ -129,6 +134,7 @@ export async function readLogReport(
return {
events: [],
invalid_lines: 0,
unrecognized_lines: 0,
failure: "log_directory_unreadable",
};
}
+309
View File
@@ -0,0 +1,309 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { run } from "./cli.js";
import { modeEffects, parseSettingsCommand } from "./memory-mode-flow.js";
import { composeStatusReport, renderStatusText } from "./status-report.js";
const transport = vi.hoisted(() => ({
get: vi.fn(),
post: vi.fn(),
put: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
}));
vi.mock("axios", () => ({
default: {
create: vi.fn(() => transport),
get: vi.fn(),
post: vi.fn(),
isAxiosError: () => false,
},
}));
const runtime = vi.hoisted(() => ({
inspectRuntimeStatus: vi.fn(),
discoverRuntimeAttachment: vi.fn(),
}));
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
...runtime,
}));
const RUN_ID = "11111111-2222-4333-8444-555555555555";
function modeState(mode: string, configured = mode) {
return {
mode,
configured_mode: configured,
forced_by_env: false,
modes: ["lite", "pro", "ultra"],
graph_build: { state: "idle", users: {} },
};
}
function receipt(state: string, extra: Record<string, unknown> = {}) {
return {
run_id: RUN_ID,
user_id: "wife",
agent_id: "all-apps",
state,
trigger: "manual",
started_at: "2026-11-01T03:00:00Z",
rounds: state === "running" ? 0 : 2,
tasks_processed: state === "running" ? 0 : 5,
batch_failures: 0,
capped: false,
elapsed_ms: 10,
results: { schemas: ["never printed"] },
...extra,
};
}
beforeEach(() => {
vi.clearAllMocks();
for (const method of Object.values(transport)) method.mockReset();
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
version: "1.2.3",
});
runtime.discoverRuntimeAttachment.mockResolvedValue({
instanceId: "instance-1",
apiToken: "synthetic-runtime-token",
host: "127.0.0.1",
port: 8765,
servicePid: 4242,
health: {
status: "ok",
service: "one-memory-core",
version: "1.2.3",
instance_id: "instance-1",
},
});
transport.get.mockImplementation(async (url: string) =>
url === "/api/v1/settings/memory-mode"
? { status: 200, data: modeState("pro") }
: { status: 404, data: {} },
);
});
describe("memhub settings mode", () => {
it("shows the current mode without changing it", async () => {
const result = await capturedRun(["settings", "mode", "--json"]);
expect(result.code).toBe(0);
expect(JSON.parse(result.stdout).mode.mode).toBe("pro");
expect(transport.put).not.toHaveBeenCalled();
});
it("prints the plan and changes nothing without --yes", async () => {
const result = await capturedRun(["settings", "mode", "lite"]);
expect(result.code).toBe(0);
expect(transport.put).not.toHaveBeenCalled();
expect(result.stdout).toContain("更高模式生成的记忆仍保存着");
expect(result.stdout).toContain("memhub settings mode lite --yes");
const json = JSON.parse(
(await capturedRun(["settings", "mode", "ultra", "--json"])).stdout,
);
expect(json.state).toBe("preview");
expect(json.applied).toBe(false);
expect(json.effects).toEqual(["existing_memories_added_to_graph"]);
});
it("changes the mode with --yes and reports what changed", async () => {
transport.put.mockResolvedValue({
status: 200,
data: {
...modeState("lite"),
previous_mode: "pro",
effects: ["higher_mode_memories_hidden"],
},
});
const result = await capturedRun(["settings", "mode", "lite", "--yes"]);
expect(result.code).toBe(0);
expect(transport.put).toHaveBeenCalledWith(
"/api/v1/settings/memory-mode",
{ mode: "lite" },
undefined,
);
expect(result.stdout).toContain("已切换");
expect(result.stdout).toContain("原始对话:全部保留");
});
it("does nothing for the mode already set, and refuses bad input", async () => {
const same = await capturedRun(["settings", "mode", "pro", "--yes"]);
expect(same.code).toBe(0);
expect(same.stdout).toContain("已经是这个模式");
expect(transport.put).not.toHaveBeenCalled();
for (const bad of [
["settings"],
["settings", "mode", "max"],
["settings", "mode", "--yes"],
["settings", "mode", "lite", "pro"],
["settings", "language", "en"],
]) {
expect((await capturedRun(bad)).code, bad.join(" ")).toBe(2);
}
});
it("plans the same effects Core reports", () => {
expect(modeEffects("lite", "pro")).toEqual([
"earlier_raw_not_extracted",
"lite_search_stops",
]);
expect(modeEffects("ultra", "lite")).toEqual([
"higher_mode_memories_hidden",
"evolve_stopped",
]);
expect(modeEffects("pro", "pro")).toEqual([]);
expect(parseSettingsCommand(["mode", "ultra", "--yes"])).toEqual({
kind: "set",
mode: "ultra",
confirmed: true,
});
});
});
describe("memhub evolve", () => {
it("starts a run for the named user and returns at once", async () => {
transport.post.mockResolvedValue({ status: 202, data: receipt("running") });
const result = await capturedRun([
"evolve",
"--user-id",
"wife",
"--agent-id",
"all-apps",
"--max-batches",
"3",
"--json",
]);
expect(result.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
"/api/v2/evolve",
{ user_id: "wife", agent_id: "all-apps", max_batches: 3 },
undefined,
);
const document = JSON.parse(result.stdout);
expect(document.state).toBe("running");
expect(document.run.results).toBeUndefined();
});
it("follows the run with --wait until it ends", async () => {
transport.post.mockResolvedValue({ status: 202, data: receipt("running") });
const reads = [receipt("running"), receipt("succeeded")];
transport.get.mockImplementation(async (url: string) =>
url === `/api/v2/evolve/${RUN_ID}`
? { status: 200, data: reads.shift() }
: { status: 404, data: {} },
);
const result = await capturedRun(["evolve", "--wait"], {
sleep: async () => undefined,
});
expect(result.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
"/api/v2/evolve",
{},
undefined,
);
expect(result.stdout).toContain("进化完成");
expect(result.stdout).toContain("2 轮");
});
it("names the mode when Core refuses outside ultra", async () => {
transport.post.mockResolvedValue({
status: 409,
data: { detail: "memory_mode_not_ultra" },
});
const result = await capturedRun(["evolve"]);
expect(result.code).toBe(1);
expect(result.stdout).toContain("memhub settings mode ultra --yes");
expect((await capturedRun(["evolve", "--max-batches", "0"])).code).toBe(2);
expect((await capturedRun(["evolve", "--now"])).code).toBe(2);
});
});
describe("status pipeline", () => {
it("reports pipeline.mode and the last evolve from memory-processing", async () => {
const processing = {
pending: 0,
fresh_pending: 0,
backlog_pending: 0,
outside_window_pending: 0,
running: 0,
waiting_for_model: 0,
completed: 3,
failed: 0,
blocked_stages: [],
pipeline: {
mode: "ultra",
configured_mode: "ultra",
forced_by_env: false,
last_evolve_at: "2026-11-01T03:00:05+08:00",
graph_build: { state: "succeeded", users: {} },
evolve: {
mode: "ultra",
schedule: { enabled: true, time: "03:00" },
running: [],
users: {
local: {
last_run_at: "2026-11-01T03:00:05+08:00",
last_result: { state: "succeeded", tasks_processed: 4 },
last_error: null,
},
},
},
},
};
const report = await composeStatusReport({
dataDir: "/tmp/isolated",
runtime: { state: "running", code: "ok", host: "127.0.0.1", port: 1 },
background: null,
reader: (async (path: string) =>
path === "/api/v1/memory-processing"
? { state: "ok", data: processing }
: { state: "unavailable", code: "x" }) as never,
});
expect(report.pipeline?.mode).toBe("ultra");
expect(report.pipeline?.last_evolve_at).toBe("2026-11-01T03:00:05+08:00");
expect(report.pipeline?.evolve_users.local?.state).toBe("succeeded");
const text = renderStatusText(report, "MemHub");
expect(text).toContain("记忆模式:ultra");
expect(text).toContain("每日 03:00 自动进化");
});
});
async function capturedRun(
arguments_: string[],
controls: Parameters<typeof run>[2] = {},
) {
let stdout = "";
let stderr = "";
const stdoutSpy = vi
.spyOn(process.stdout, "write")
.mockImplementation((chunk) => {
stdout += String(chunk);
return true;
});
const stderrSpy = vi
.spyOn(process.stderr, "write")
.mockImplementation((chunk) => {
stderr += String(chunk);
return true;
});
try {
return {
code: await run(
arguments_,
{ ONE_MEMORY_DATA_DIR: "/synthetic/data" },
controls,
),
stdout,
stderr,
};
} finally {
stdoutSpy.mockRestore();
stderrSpy.mockRestore();
}
}
+372
View File
@@ -0,0 +1,372 @@
/**
* `memhub settings mode` and `memhub evolve`.
*
* The mode (lite / pro / ultra) decides what the SDK does with every
* conversation, and switching it migrates nothing: what one mode made is not
* shown by a lower one, and raw kept by lite is not extracted until it is
* imported again. So a change prints its plan first and needs `--yes`, like
* every other command whose effect a person cannot simply take back.
*
* `memhub evolve` starts System 2 (ultra only) and, with `--wait`, follows
* the run until it ends.
*/
import type { CoreReader } from "./core-client.js";
import type { CoreOperations, CorePutOperations } from "./core-operations.js";
import { PRODUCT } from "./version.js";
export const MEMORY_MODES = ["lite", "pro", "ultra"] as const;
export type MemoryMode = (typeof MEMORY_MODES)[number];
export type ModeEffect =
| "earlier_raw_not_extracted"
| "higher_mode_memories_hidden"
| "existing_memories_added_to_graph"
| "evolve_stopped"
| "lite_search_stops"
| "graph_backfill_already_running";
const MODE_SUMMARY: Record<MemoryMode, string> = {
lite: "lite:不调用模型,保留全部原始对话,只在原始对话里检索",
pro: "pro:模型从对话中提取记忆(System 1)",
ultra: "ultra:pro + 每晚自动进化(System 2,使用知识图谱)",
};
const EFFECT_TEXT: Record<ModeEffect, string> = {
earlier_raw_not_extracted:
"lite 下已处理完的对话不会再被提取;只有尚未处理的和之后的新对话会由模型整理成记忆,检索立即改为记忆检索",
higher_mode_memories_hidden:
"更高模式生成的记忆仍保存着,但在这个模式下不会显示",
existing_memories_added_to_graph:
"已有记忆会在后台加入知识图谱(每个用户一次)",
evolve_stopped: "每晚的自动进化(System 2)停止",
lite_search_stops: "lite 只在原始对话里的检索立即停止,改为检索记忆",
graph_backfill_already_running: "知识图谱的补建已在进行中",
};
/** The same rule Core applies (`memory_mode_effects`), for the plan. */
export function modeEffects(
previous: MemoryMode,
next: MemoryMode,
): ModeEffect[] {
if (previous === next) return [];
const rank: Record<MemoryMode, number> = { lite: 0, pro: 1, ultra: 2 };
const effects: ModeEffect[] = [];
if (previous === "lite") {
effects.push("earlier_raw_not_extracted", "lite_search_stops");
}
if (rank[next] < rank[previous]) effects.push("higher_mode_memories_hidden");
if (previous === "ultra") effects.push("evolve_stopped");
if (next === "ultra") effects.push("existing_memories_added_to_graph");
return effects;
}
export type ModeCommand =
| { kind: "show" }
| { kind: "set"; mode: MemoryMode; confirmed: boolean };
/** `settings mode [lite|pro|ultra] [--yes]` after `settings`; null = usage. */
export function parseSettingsCommand(
values: readonly string[],
): ModeCommand | null {
const rest = values.filter((value) => value !== "--json");
if (rest[0] !== "mode") return null;
const tail = rest.slice(1);
const confirmed = tail.includes("--yes");
const positional = tail.filter((value) => value !== "--yes");
if (tail.filter((value) => value === "--yes").length > 1) return null;
if (positional.length === 0) return confirmed ? null : { kind: "show" };
if (positional.length !== 1) return null;
const mode = positional[0] as MemoryMode;
if (!(MEMORY_MODES as readonly string[]).includes(mode)) return null;
return { kind: "set", mode, confirmed };
}
export interface ModeState {
mode: MemoryMode;
configured_mode: MemoryMode;
forced_by_env: boolean;
graph_build?: { state?: string } | null;
previous_mode?: MemoryMode;
effects?: ModeEffect[];
}
function modeState(value: unknown): ModeState | null {
if (!value || typeof value !== "object") return null;
const item = value as Record<string, unknown>;
if (
!(MEMORY_MODES as readonly unknown[]).includes(item.mode) ||
!(MEMORY_MODES as readonly unknown[]).includes(item.configured_mode) ||
typeof item.forced_by_env !== "boolean"
) {
return null;
}
return item as unknown as ModeState;
}
export type ModeDocument = {
schema_version: 1;
product: string;
command: "settings mode";
} & (
| { state: "ok"; mode: ModeState }
| {
state: "preview";
mode: ModeState;
target: MemoryMode;
effects: ModeEffect[];
applied: false;
next_step: string;
}
| { state: "unchanged"; mode: ModeState }
| { state: "committed"; mode: ModeState; effects: ModeEffect[] }
| { state: "rejected" | "unknown" | "unavailable"; code: string }
);
const BASE = {
schema_version: 1 as const,
product: PRODUCT,
command: "settings mode" as const,
};
export async function runModeCommand(
command: ModeCommand,
reader: CoreReader,
operations: CorePutOperations,
): Promise<ModeDocument> {
const read = await reader<unknown>("/api/v1/settings/memory-mode");
if (read.state !== "ok") {
return { ...BASE, state: "unavailable", code: read.code };
}
const current = modeState(read.data);
if (current === null) {
return { ...BASE, state: "unavailable", code: "response_invalid" };
}
if (command.kind === "show") return { ...BASE, state: "ok", mode: current };
if (current.configured_mode === command.mode) {
return { ...BASE, state: "unchanged", mode: current };
}
if (!command.confirmed) {
return {
...BASE,
state: "preview",
mode: current,
target: command.mode,
effects: modeEffects(current.mode, command.mode),
applied: false,
next_step: `${PRODUCT} settings mode ${command.mode} --yes`,
};
}
const result = await operations.put<unknown>("/api/v1/settings/memory-mode", {
mode: command.mode,
});
if (result.state !== "committed") {
return { ...BASE, state: result.state, code: result.code };
}
const changed = modeState(result.data);
if (changed === null) {
return { ...BASE, state: "unknown", code: "response_invalid" };
}
return {
...BASE,
state: "committed",
mode: changed,
effects: changed.effects ?? [],
};
}
export function modeExitCode(document: ModeDocument): number {
return ["ok", "preview", "unchanged", "committed"].includes(document.state)
? 0
: 1;
}
export function renderModeText(document: ModeDocument): string {
if ("code" in document) {
return `记忆模式未能${document.state === "unavailable" ? "读取" : "修改"}:${document.code}\n`;
}
const mode = document.mode;
const lines = [`当前模式:${MODE_SUMMARY[mode.mode]}`];
if (mode.forced_by_env) {
lines.push(
`设置里是 ${mode.configured_mode};环境变量 ONE_MEMORY_GRAPH_STORE 强制使用 ultra。`,
);
}
lines.push("原始对话:全部保留(任何模式都不删除)");
if (document.state === "unchanged") {
lines.push("已经是这个模式,未做改动。");
}
if (document.state === "preview") {
lines.push(`将切换到:${MODE_SUMMARY[document.target]}`);
for (const effect of document.effects)
lines.push(`- ${EFFECT_TEXT[effect]}`);
lines.push(`未执行任何改动。确认后运行:${document.next_step}`);
}
if (document.state === "committed") {
lines.push("已切换。");
for (const effect of document.effects)
lines.push(`- ${EFFECT_TEXT[effect]}`);
}
return `${lines.join("\n")}\n`;
}
export type EvolveCommand = {
agentId?: string;
maxBatches?: number;
wait: boolean;
};
/** `evolve [--agent-id <id>] [--max-batches <n>] [--wait]`; null = usage. */
export function parseEvolveCommand(
values: readonly string[],
): EvolveCommand | null {
const parsed: EvolveCommand = { wait: false };
const rest = values.filter((value) => value !== "--json");
for (let index = 0; index < rest.length; index += 1) {
const flag = rest[index];
if (flag === "--wait" && !parsed.wait) {
parsed.wait = true;
} else if (flag === "--agent-id" && parsed.agentId === undefined) {
const value = rest[index + 1];
if (!value || !/^[A-Za-z0-9._:-]{1,128}$/.test(value)) return null;
parsed.agentId = value;
index += 1;
} else if (flag === "--max-batches" && parsed.maxBatches === undefined) {
const value = Number(rest[index + 1]);
if (!Number.isInteger(value) || value < 1 || value > 1000) return null;
parsed.maxBatches = value;
index += 1;
} else {
return null;
}
}
return parsed;
}
export type EvolveDocument = {
schema_version: 1;
product: string;
command: "evolve";
} & (
| { state: "running" | "succeeded" | "failed"; run: Record<string, unknown> }
| { state: "rejected" | "unknown" | "unavailable"; code: string }
);
const RUN_ID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
function evolveRun(value: unknown): Record<string, unknown> | null {
if (!value || typeof value !== "object") return null;
const run = value as Record<string, unknown>;
if (
typeof run.run_id !== "string" ||
!RUN_ID.test(run.run_id) ||
!["running", "succeeded", "failed"].includes(String(run.state))
) {
return null;
}
return run;
}
export async function runEvolveCommand(
command: EvolveCommand,
options: {
userId?: string;
reader: CoreReader;
operations: CoreOperations;
sleep?: (ms: number) => Promise<void>;
pollMs?: number;
},
): Promise<EvolveDocument> {
const base = {
schema_version: 1 as const,
product: PRODUCT,
command: "evolve" as const,
};
const body: Record<string, unknown> = {};
if (options.userId !== undefined) body.user_id = options.userId;
if (command.agentId !== undefined) body.agent_id = command.agentId;
if (command.maxBatches !== undefined) body.max_batches = command.maxBatches;
const started = await options.operations.post<unknown>(
"/api/v2/evolve",
body,
);
if (started.state !== "committed") {
return { ...base, state: started.state, code: started.code };
}
let run = evolveRun(started.data);
if (run === null)
return { ...base, state: "unknown", code: "response_invalid" };
const sleep =
options.sleep ??
((ms: number) => new Promise((done) => setTimeout(done, ms)));
while (command.wait && run.state === "running") {
await sleep(options.pollMs ?? 2_000);
const read = await options.reader<unknown>(
`/api/v2/evolve/${run.run_id}`,
options.userId === undefined ? {} : { user_id: options.userId },
);
if (read.state !== "ok") {
return { ...base, state: "unavailable", code: read.code };
}
const next = evolveRun(read.data);
if (next === null)
return { ...base, state: "unknown", code: "response_invalid" };
run = next;
}
return {
...base,
state: run.state as "running" | "succeeded" | "failed",
run: projectRun(run),
};
}
/** Counts and codes; SDK result bodies stay in the Core receipt. */
function projectRun(run: Record<string, unknown>): Record<string, unknown> {
const keep = [
"run_id",
"user_id",
"agent_id",
"state",
"trigger",
"started_at",
"finished_at",
"rounds",
"tasks_processed",
"batch_failures",
"cycle_status",
"should_continue",
"continuation_reason",
"remaining_clusters",
"remaining_actionable_clusters",
"remaining_fresh_facts",
"capped",
"elapsed_ms",
"error_code",
"failure_kind",
];
return Object.fromEntries(
keep.filter((key) => key in run).map((key) => [key, run[key]]),
);
}
export function evolveExitCode(document: EvolveDocument): number {
return document.state === "running" || document.state === "succeeded" ? 0 : 1;
}
export function renderEvolveText(document: EvolveDocument): string {
if ("code" in document) {
return document.code === "memory_mode_not_ultra"
? "进化只在 ultra 模式下可用。切换:memhub settings mode ultra --yes\n"
: `进化未能开始:${document.code}\n`;
}
const run = document.run;
if (document.state === "running") {
return `进化已开始(${run.run_id}),在后台运行;加 --wait 等待结果。\n`;
}
if (document.state === "failed") {
return `进化失败(${run.run_id}):${String(run.error_code ?? "unknown")}\n`;
}
return `进化完成(${run.run_id}):${run.rounds} 轮,处理 ${run.tasks_processed} 项${
run.cycle_status ? `,周期 ${run.cycle_status}` : ""
}${run.capped ? ",已到轮数上限,可再次运行继续" : ""}\n`;
}
+37 -3
View File
@@ -14,6 +14,25 @@ import { type CoreReader, RESERVED_MEMORY_SEGMENTS } from "./core-client.js";
import type { CoreOperations } from "./core-operations.js";
import { PRODUCT } from "./version.js";
/** A MemHub user id, the same shape Core accepts. `local` is the default. */
const USER_ID = /^[A-Za-z0-9._-]{1,64}$/;
export const DEFAULT_USER_ID = "local";
export function isUserId(value: string): boolean {
return USER_ID.test(value);
}
/**
* The `user_id` to send, or nothing. The default user is never named on the
* wire, so a request without `--user-id` stays exactly what it was before
* multi-user support and still works against an older Core.
*/
export function userIdParameters(userId?: string): { user_id?: string } {
return userId === undefined || userId === DEFAULT_USER_ID
? {}
: { user_id: userId };
}
/** Output budget: a long memory is shortened visibly, never dropped silently. */
const CONTENT_BUDGET = 240;
@@ -101,10 +120,12 @@ export interface MemoryDetailDocument {
export async function readMemoryPage(
reader: CoreReader,
query: MemoryQuery,
userId?: string,
): Promise<MemoryPageDocument> {
const parameters: Record<string, string | number> = {
offset: query.offset,
limit: query.limit,
...userIdParameters(userId),
};
if (query.status !== undefined) parameters.statuses = query.status;
if (query.source !== undefined) parameters.source = query.source;
@@ -156,8 +177,12 @@ export async function readMemoryDetail(
reader: CoreReader,
memoryId: string,
namespace: MemoryNamespace = {},
userId?: string,
): Promise<MemoryDetailDocument> {
const parameters = namespaceParameters(namespace);
const parameters = {
...namespaceParameters(namespace),
...userIdParameters(userId),
};
const outcome = await reader<unknown>(
`/api/v2/memories/${memoryId}`,
Object.keys(parameters).length > 0 ? parameters : undefined,
@@ -236,6 +261,7 @@ export async function addMemory(
operations: CoreOperations,
text: string,
requestId: string = randomUUID(),
userId?: string,
): Promise<MemoryAddDocument> {
const base = {
schema_version: 2 as const,
@@ -259,7 +285,10 @@ export async function addMemory(
}
throw error;
}
const result = await operations.post<unknown>(request.path, request.body);
const result = await operations.post<unknown>(request.path, {
...request.body,
...userIdParameters(userId),
});
if (result.state !== "committed") {
return { ...base, state: result.state, code: result.code };
}
@@ -273,6 +302,7 @@ export async function addMemory(
export async function readMemoryAddStatus(
reader: CoreReader,
requestId: string,
userId?: string,
): Promise<MemoryAddDocument> {
const base = {
schema_version: 2 as const,
@@ -283,7 +313,11 @@ export async function readMemoryAddStatus(
if (!UUID.test(requestId)) {
return { ...base, state: "invalid", code: "invalid_request_id" };
}
const outcome = await reader<unknown>(memoryAddStatusPath(requestId));
const user = userIdParameters(userId);
const outcome =
user.user_id === undefined
? await reader<unknown>(memoryAddStatusPath(requestId))
: await reader<unknown>(memoryAddStatusPath(requestId), user);
if (outcome.state === "not_found") {
return { ...base, state: "not_found", code: "memory_add_not_found" };
}
@@ -0,0 +1,166 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { run } from "./cli.js";
const transport = vi.hoisted(() => ({
get: vi.fn(),
post: vi.fn(),
put: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
}));
vi.mock("axios", () => ({
default: {
create: vi.fn(() => transport),
get: vi.fn(),
post: vi.fn(),
isAxiosError: () => false,
},
}));
const runtime = vi.hoisted(() => ({
inspectRuntimeStatus: vi.fn(),
discoverRuntimeAttachment: vi.fn(),
}));
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
...runtime,
}));
const VERSIONS = {
user_id: "wife",
request_id: "r-versions",
elapsed_ms: 2,
chains: [
{
memory_id: "l0-head",
content: "The user's name is Fei.",
status: "active",
memory_at: null,
gmt_created: 1_790_000_100,
basic_info: { name: "Fei" },
versions: [
{
memory_id: "l0-head",
content: "The user's name is Fei.",
memory_at: null,
gmt_created: 1_790_000_100,
current: true,
},
{
memory_id: "l0-old",
content: "The user's name is 费天翔.",
memory_at: null,
gmt_created: 1_790_000_000,
current: false,
},
],
},
],
};
beforeEach(() => {
vi.clearAllMocks();
for (const method of Object.values(transport)) method.mockReset();
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
version: "1.2.3",
});
runtime.discoverRuntimeAttachment.mockResolvedValue({
instanceId: "instance-1",
apiToken: "synthetic-runtime-token",
host: "127.0.0.1",
port: 8765,
servicePid: 4242,
health: {
status: "ok",
service: "one-memory-core",
version: "1.2.3",
instance_id: "instance-1",
},
});
transport.get.mockImplementation(async (url: string) =>
url === "/api/v2/user-profile/versions"
? { status: 200, data: VERSIONS }
: { status: 404, data: {} },
);
});
describe("memhub profile --versions", () => {
it("reads the L0 version chain for the named user", async () => {
const result = await capturedRun([
"profile",
"--versions",
"--user-id",
"wife",
"--json",
]);
expect(result.code).toBe(0);
expect(transport.get).toHaveBeenCalledWith(
"/api/v2/user-profile/versions",
{ params: { user_id: "wife" } },
);
const document = JSON.parse(result.stdout);
expect(document.state).toBe("ok");
expect(document.chains[0].versions).toHaveLength(2);
});
it("marks the current version in text and names no user for local", async () => {
const result = await capturedRun(["profile", "--versions"]);
expect(result.code).toBe(0);
const [url, config] = transport.get.mock.calls[0] ?? [];
expect(url).toBe("/api/v2/user-profile/versions");
expect(config?.params?.user_id).toBeUndefined();
expect(result.stdout).toContain("* l0-head");
expect(result.stdout).toContain(" l0-old");
expect(result.stdout).toContain("费天翔");
});
it("refuses an unknown flag and a repeated --versions", async () => {
expect((await capturedRun(["profile", "--version-chain"])).code).toBe(2);
expect(
(await capturedRun(["profile", "--versions", "--versions"])).code,
).toBe(2);
expect(transport.get).not.toHaveBeenCalled();
});
it("reports a malformed answer as invalid", async () => {
transport.get.mockResolvedValue({ status: 200, data: { chains: "x" } });
const result = await capturedRun(["profile", "--versions", "--json"]);
expect(result.code).toBe(1);
expect(JSON.parse(result.stdout).code).toBe("response_invalid");
});
});
async function capturedRun(arguments_: string[]) {
let stdout = "";
let stderr = "";
const stdoutSpy = vi
.spyOn(process.stdout, "write")
.mockImplementation((chunk) => {
stdout += String(chunk);
return true;
});
const stderrSpy = vi
.spyOn(process.stderr, "write")
.mockImplementation((chunk) => {
stderr += String(chunk);
return true;
});
try {
return {
code: await run(arguments_, {
ONE_MEMORY_DATA_DIR: "/synthetic/data",
}),
stdout,
stderr,
};
} finally {
stdoutSpy.mockRestore();
stderrSpy.mockRestore();
}
}
+107 -1
View File
@@ -14,6 +14,11 @@ import {
validateProcessing,
validateSources,
} from "./core-responses.js";
import {
describePipeline,
type PipelineSummary,
pipelineLine,
} from "./status-report.js";
import { PRODUCT } from "./version.js";
type ReadState =
@@ -38,6 +43,21 @@ export interface SourcesDocument {
state: ReadState["state"];
code?: string;
sources: Array<Record<string, unknown>>;
/** Every grant of every user (`GET /api/v1/source-grants`). Absent when
* Core is too old to list them; `sources` is the default user's view. */
grants?: SourceGrant[];
}
/** One source grant. Only Hermes has grants for users other than `local`,
* one per Hermes profile root. */
export interface SourceGrant {
user_id: string;
source_id: string;
root_label: string | null;
root_path: string | null;
authorized: boolean;
revoked_at: string | null;
sync_enabled: boolean;
}
export interface SyncDocument {
@@ -69,6 +89,8 @@ export interface ProgressDocument {
summary_state: ReadState;
summary: Record<string, unknown> | null;
};
/** The SDK mode and last evolve per user; absent on an older Core. */
pipeline?: PipelineSummary | null;
}
export async function readSources(
@@ -86,14 +108,66 @@ export async function readSources(
sources: [],
};
}
const grants = await readSourceGrants(reader);
return {
schema_version: SCHEMA_VERSION,
product: PRODUCT,
state: "ok",
sources: result.data.map(projectSource),
// `/api/v1/sources` is the default user's view of every source.
sources: result.data.map((source) => ({
...projectSource(source),
user_id: "local",
})),
...(grants === null ? {} : { grants }),
};
}
/** The grants of every user, or null when Core cannot list them (an older
* Core answers 404): the source list is then shown as before. */
async function readSourceGrants(
reader: CoreReader,
): Promise<SourceGrant[] | null> {
let outcome: ReadOutcome<unknown>;
try {
outcome = await reader<unknown>("/api/v1/source-grants");
} catch {
return null;
}
if (outcome.state !== "ok") return null;
const value = outcome.data;
if (!value || typeof value !== "object" || Array.isArray(value)) return null;
const items = (value as { grants?: unknown }).grants;
if (!Array.isArray(items)) return null;
const grants: SourceGrant[] = [];
const text = (candidate: unknown): string | null =>
typeof candidate === "string" && candidate.length <= 4096
? candidate
: null;
for (const item of items) {
if (!item || typeof item !== "object" || Array.isArray(item)) return null;
const grant = item as Record<string, unknown>;
if (
typeof grant.user_id !== "string" ||
!/^[A-Za-z0-9._-]{1,64}$/.test(grant.user_id) ||
typeof grant.source_id !== "string" ||
!/^[a-z0-9][a-z0-9-]{0,127}$/.test(grant.source_id) ||
typeof grant.authorized !== "boolean"
) {
return null;
}
grants.push({
user_id: grant.user_id,
source_id: grant.source_id,
root_label: text(grant.root_label),
root_path: text(grant.root_path),
authorized: grant.authorized,
revoked_at: text(grant.revoked_at),
sync_enabled: grant.sync_enabled === true,
});
}
return grants;
}
export async function readSync(reader: CoreReader): Promise<SyncDocument> {
const result = validated(
await reader<unknown>("/api/v1/live-sync"),
@@ -216,6 +290,10 @@ export async function readProgress(
summary:
processing.state === "ok" ? projectProcessing(processing.data) : null,
},
pipeline:
processingRead.state === "ok"
? describePipeline(processingRead.data)
: null,
};
}
@@ -231,6 +309,22 @@ export function renderSourcesText(document: SourcesDocument): string {
);
}
if (document.sources.length === 0) lines.push("(没有来源)");
// The list above is the default user's (`local`). Another user's grant,
// a Hermes profile, is listed with its user so it is not mistaken for it.
const others = (document.grants ?? []).filter(
(grant) => grant.user_id !== "local",
);
if (others.length > 0) {
lines.push("其他用户的连接:");
for (const grant of others) {
const name =
document.sources.find((source) => source.source_id === grant.source_id)
?.display_name ?? grant.source_id;
lines.push(
`- ${name}(用户 ${grant.user_id}):${grant.authorized ? "已连接" : grant.revoked_at ? "已撤销" : "未连接"};处理${grant.sync_enabled ? "已开启" : "已冻结"}${grant.root_path ? `;目录 ${grant.root_path}` : ""}`,
);
}
}
lines.push("");
return lines.join("\n");
}
@@ -270,6 +364,18 @@ export function renderProgressText(document: ProgressDocument): string {
? `记忆整理:待处理 ${document.generation.summary.pending} · 进行中 ${document.generation.summary.running} · 等待模型 ${document.generation.summary.waiting_for_model} · 失败 ${document.generation.summary.failed}`
: "记忆整理:暂时无法读取",
);
if (document.pipeline) {
lines.push(`记忆模式:${pipelineLine(document.pipeline)}`);
for (const [userId, user] of Object.entries(
document.pipeline.evolve_users,
)) {
lines.push(
` 进化 ${userId}:${user.last_run_at ?? "尚未进化"}${
user.last_error ? `(失败:${user.last_error})` : ""
}`,
);
}
}
if (document.imports.output_truncated) {
lines.push("这里只显示汇总;完整记录请使用 --json。");
}
+62 -1
View File
@@ -179,12 +179,72 @@ describe("CLI-07/10 source and synchronization reads", () => {
sync_enabled: true,
sync_completeness: "additions_only",
});
expect(requestedPaths()).toEqual(["/api/v1/sources"]);
// The grants list answers 404 here, as an older Core does: the source
// list is shown as before, every row the default user's.
expect(report.sources[0].user_id).toBe("local");
expect(report.grants).toBeUndefined();
expect(requestedPaths()).toEqual([
"/api/v1/sources",
"/api/v1/source-grants",
]);
expect(requestedPaths().some((path) => path.includes("detect"))).toBe(
false,
);
});
it("shows every user's grant, a Hermes profile with its user", async () => {
const grants = {
schema_version: 1,
grants: [
{
user_id: "local",
source_id: "codex",
root_label: ".codex",
root_path: "/home/synthetic/.codex",
authorized: true,
revoked_at: null,
sync_enabled: true,
},
{
user_id: "wife",
source_id: "hermes",
root_label: "wife",
root_path: "/home/synthetic/.hermes/profiles/wife",
authorized: true,
revoked_at: null,
sync_enabled: true,
},
],
};
transport.get.mockImplementation(async (url: string) =>
url === "/api/v1/source-grants"
? { status: 200, data: grants }
: (RESPONSES.get(url) ?? { status: 404, data: {} }),
);
let streams = captureStreams();
try {
expect(await run(["sources", "list", "--json"], {})).toBe(0);
} finally {
streams.restore();
}
const report = JSON.parse(streams.stdout());
expect(report.grants).toEqual(grants.grants);
expect(report.sources[0]).toMatchObject({
source_id: "codex",
user_id: "local",
});
streams = captureStreams();
try {
expect(await run(["sources", "list"], {})).toBe(0);
} finally {
streams.restore();
}
expect(streams.stdout()).toContain("其他用户的连接:");
expect(streams.stdout()).toContain(
"hermes(用户 wife):已连接;处理已开启;目录 /home/synthetic/.hermes/profiles/wife",
);
});
it("keeps degraded synchronization distinct and prints its error code", async () => {
const streams = captureStreams();
try {
@@ -552,6 +612,7 @@ describe("CLI-19 progress and issue reads", () => {
requestedPaths().every((path) =>
[
"/api/v1/sources",
"/api/v1/source-grants",
"/api/v1/live-sync",
"/api/v1/import-jobs",
"/api/v1/memory-processing",
+112
View File
@@ -57,6 +57,9 @@ export interface StatusDocument {
broker?: BackgroundHealth["broker"];
};
sections: Record<string, Section>;
/** R10: the SDK mode and System 2 state; null when the Runtime is not read
* or is older than the setting. */
pipeline?: PipelineSummary | null;
summary: {
state: "ok" | "partial";
degraded: string[];
@@ -64,6 +67,88 @@ export interface StatusDocument {
};
}
export interface PipelineSummary {
mode: "lite" | "pro" | "ultra";
configured_mode: "lite" | "pro" | "ultra";
forced_by_env: boolean;
last_evolve_at: string | null;
evolve_schedule: { enabled: boolean; time: string } | null;
evolve_running: number;
evolve_users: Record<
string,
{
last_run_at: string | null;
last_error: string | null;
state: string | null;
}
>;
graph_build: string | null;
}
const MODES = new Set(["lite", "pro", "ultra"]);
/** Projection of `pipeline` in `/api/v1/memory-processing`; null if absent or malformed. */
export function describePipeline(value: unknown): PipelineSummary | null {
if (!value || typeof value !== "object") return null;
const pipeline = (value as { pipeline?: unknown }).pipeline;
if (!pipeline || typeof pipeline !== "object") return null;
const item = pipeline as Record<string, unknown>;
if (
!MODES.has(String(item.mode)) ||
!MODES.has(String(item.configured_mode))
) {
return null;
}
const evolve =
item.evolve && typeof item.evolve === "object"
? (item.evolve as Record<string, unknown>)
: {};
const schedule =
evolve.schedule && typeof evolve.schedule === "object"
? (evolve.schedule as Record<string, unknown>)
: null;
const users: PipelineSummary["evolve_users"] = {};
if (evolve.users && typeof evolve.users === "object") {
for (const [userId, raw] of Object.entries(
evolve.users as Record<string, unknown>,
)) {
const user =
raw && typeof raw === "object" ? (raw as Record<string, unknown>) : {};
const result =
user.last_result && typeof user.last_result === "object"
? (user.last_result as Record<string, unknown>)
: {};
users[userId] = {
last_run_at:
typeof user.last_run_at === "string" ? user.last_run_at : null,
last_error:
typeof user.last_error === "string" ? user.last_error : null,
state: typeof result.state === "string" ? result.state : null,
};
}
}
const graph =
item.graph_build && typeof item.graph_build === "object"
? (item.graph_build as Record<string, unknown>)
: {};
return {
mode: item.mode as PipelineSummary["mode"],
configured_mode: item.configured_mode as PipelineSummary["mode"],
forced_by_env: item.forced_by_env === true,
last_evolve_at:
typeof item.last_evolve_at === "string" ? item.last_evolve_at : null,
evolve_schedule:
schedule &&
typeof schedule.enabled === "boolean" &&
typeof schedule.time === "string"
? { enabled: schedule.enabled, time: schedule.time }
: null,
evolve_running: Array.isArray(evolve.running) ? evolve.running.length : 0,
evolve_users: users,
graph_build: typeof graph.state === "string" ? graph.state : null,
};
}
const SECTION_NAMES = [
"sources",
"sync",
@@ -97,6 +182,7 @@ export async function composeStatusReport(options: {
attachFailure?: string;
}): Promise<StatusDocument> {
const sections: Record<string, Section> = {};
let pipeline: PipelineSummary | null = null;
if (options.reader === null) {
const state: SectionState =
options.attachFailure === undefined ? "not_checked" : "unavailable";
@@ -118,6 +204,8 @@ export async function composeStatusReport(options: {
validateProcessing,
describeProcessing,
);
pipeline =
processing.state === "ok" ? describePipeline(processing.data) : null;
sections.model = section(model, validateModelProfiles, describeModel);
sections.memories = section(
memories,
@@ -169,6 +257,7 @@ export async function composeStatusReport(options: {
},
bridge,
sections,
pipeline,
summary: {
state:
degraded.length === 0 && notChecked.length === 0 ? "ok" : "partial",
@@ -205,6 +294,7 @@ export function renderStatusText(
`${SECTION_LABELS[name]}:${sectionLine(report.sections[name])}`,
);
}
if (report.pipeline) lines.push(`记忆模式:${pipelineLine(report.pipeline)}`);
if (report.summary.state === "partial") {
const notes = [
report.summary.degraded.length > 0
@@ -220,6 +310,28 @@ export function renderStatusText(
return lines.join("\n");
}
export function pipelineLine(pipeline: PipelineSummary): string {
const parts: string[] = [pipeline.mode];
if (pipeline.forced_by_env)
parts.push("(ONE_MEMORY_GRAPH_STORE 强制 ultra)");
if (pipeline.mode === "ultra") {
const schedule = pipeline.evolve_schedule;
parts.push(
schedule?.enabled
? `,每日 ${schedule.time} 自动进化`
: ",自动进化已关闭",
);
parts.push(
pipeline.last_evolve_at
? `,上次进化 ${pipeline.last_evolve_at}`
: ",尚未进化",
);
if (pipeline.evolve_running > 0)
parts.push(`,进行中 ${pipeline.evolve_running}`);
}
return parts.join("");
}
function runtimeLine(report: StatusDocument): string {
if (report.runtime.state !== "running") {
return report.runtime.state === "stopped" ? "未运行" : "暂时无法确认";
+405
View File
@@ -0,0 +1,405 @@
import { mkdtemp, writeFile } from "node:fs/promises";
import os, { tmpdir } from "node:os";
import path from "node:path";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { run } from "./cli.js";
const transport = vi.hoisted(() => ({
get: vi.fn(),
post: vi.fn(),
put: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
}));
const direct = vi.hoisted(() => ({ get: vi.fn(), post: vi.fn() }));
vi.mock("axios", () => ({
default: {
create: vi.fn(() => transport),
get: direct.get,
post: direct.post,
isAxiosError: () => false,
},
}));
const runtime = vi.hoisted(() => ({
inspectRuntimeStatus: vi.fn(),
discoverRuntimeAttachment: vi.fn(),
}));
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
...runtime,
}));
const REQUEST = "33333333-3333-4333-8333-333333333333";
const HERMES = {
source_id: "hermes",
display_name: "Hermes",
storage_kinds: ["sqlite"],
available: true,
authorized: true,
detected: null,
root_label: ".hermes",
candidate_path: "~/.hermes",
candidate_kind: "default",
candidate_revision: 0,
candidate_root_id: "hermes-root-identity",
imported_messages: 0,
imported_conversations: 0,
capabilities: {
historical_import: "implemented",
incremental_import: "implemented",
live_sync: "implemented",
recall: "implemented",
installer: "not_implemented",
verification: "fixture_only",
},
sync_completeness: "additions_only",
location_state: "found",
};
const CODEX = {
...HERMES,
source_id: "codex",
display_name: "Codex",
candidate_root_id: "codex-root-identity",
};
function addReceipt() {
return {
evidence_id: "44444444-4444-4444-8444-444444444444",
text: "Prefers tea",
scope: "all-apps",
created_at: "2026-10-27T00:00:00Z",
state: "pending",
task_id: null,
memory_ids: [],
error: null,
};
}
beforeEach(() => {
vi.clearAllMocks();
for (const method of [
...Object.values(transport),
...Object.values(direct),
]) {
method.mockReset();
}
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
version: "1.2.3",
});
runtime.discoverRuntimeAttachment.mockResolvedValue({
instanceId: "instance-1",
apiToken: "synthetic-runtime-token",
host: "127.0.0.1",
port: 8765,
servicePid: 4242,
health: {
status: "ok",
service: "one-memory-core",
version: "1.2.3",
instance_id: "instance-1",
},
});
transport.get.mockImplementation(async (url: string) => {
if (url === "/api/v1/sources") {
return { status: 200, data: [HERMES, CODEX] };
}
if (url === "/api/v2/user-profile") {
return {
status: 200,
data: { request_id: "r-1", profile: null, elapsed_ms: 1 },
};
}
return { status: 404, data: {} };
});
transport.post.mockImplementation(
async (url: string, body: Record<string, unknown>) =>
url === "/api/v1/sources/connect"
? {
status: 200,
data: {
request_id: body.request_id,
accepted_source_ids: ["hermes"],
failed_sources: {},
jobs: [],
},
}
: url === "/api/v2/memories"
? { status: 201, data: addReceipt() }
: { status: 404, data: {} },
);
transport.put.mockResolvedValue({ status: 200, data: {} });
transport.delete.mockResolvedValue({ status: 204, data: "" });
direct.post.mockResolvedValue({
status: 202,
data: {
request_id: REQUEST,
state: "accepted",
created_at: "T1",
updated_at: "T1",
},
});
direct.get.mockResolvedValue({
data: {
request_id: REQUEST,
state: "accepted",
created_at: "T1",
updated_at: "T1",
},
});
});
describe("--user-id on the memory commands", () => {
it("passes the user as a query parameter to memory reads and deletes", async () => {
await capturedRun(["memories", "list", "--user-id", "wife", "--json"]);
expect(transport.get).toHaveBeenCalledWith("/api/v2/memory-library", {
params: { offset: 0, limit: 100, user_id: "wife" },
});
// Global: the flag may come before the subcommand too.
await capturedRun(["memories", "--user-id", "wife", "show", "m-1"]);
expect(transport.get).toHaveBeenCalledWith("/api/v2/memories/m-1", {
params: { user_id: "wife" },
});
await capturedRun(["memories", "add-status", REQUEST, "--user-id", "wife"]);
expect(transport.get).toHaveBeenCalledWith(
`/api/v2/memories/additions/${REQUEST}`,
{ params: { user_id: "wife" } },
);
await capturedRun(["profile", "--user-id", "wife", "--json"]);
expect(transport.get).toHaveBeenCalledWith("/api/v2/user-profile", {
params: { user_id: "wife" },
});
await capturedRun([
"memories",
"delete",
"fact-1",
"--kind",
"fact",
"--yes",
"--user-id",
"wife",
]);
expect(transport.delete).toHaveBeenCalledWith(
"/api/v2/memories/fact-1?kind=fact&user_id=wife",
undefined,
);
await capturedRun([
"memories",
"delete",
"m-1",
"--yes",
"--user-id",
"wife",
]);
expect(transport.delete).toHaveBeenCalledWith(
"/api/v2/memories/m-1?user_id=wife",
undefined,
);
await capturedRun(["write", "status", REQUEST, "--user-id", "wife"]);
expect(direct.get.mock.calls[0][0]).toBe(
`http://127.0.0.1:8765/api/v2/memory-writes/${REQUEST}`,
);
expect(direct.get.mock.calls[0][1].params).toEqual({ user_id: "wife" });
});
it("passes the user in the body of memory writes and searches", async () => {
await capturedRun(["memories", "add", "Prefers tea", "--user-id", "wife"]);
expect(transport.post.mock.calls[0][0]).toBe("/api/v2/memories");
expect(transport.post.mock.calls[0][1]).toMatchObject({
text: "Prefers tea",
scope: "all-apps",
user_id: "wife",
});
const directory = await mkdtemp(path.join(tmpdir(), "om-user-id-"));
const content = path.join(directory, "content.txt");
await writeFile(content, "Prefers green tea\n");
await capturedRun([
"memories",
"update",
"m-1",
"--content-file",
content,
"--user-id",
"wife",
]);
expect(transport.put.mock.calls[0][0]).toBe("/api/v2/memories/m-1");
expect(transport.put.mock.calls[0][1]).toMatchObject({ user_id: "wife" });
await capturedRun(["recall", "tea", "--user-id", "wife", "--json"]);
const search = transport.post.mock.calls.find(
([url]) => url === "/api/v2/context/search",
);
expect(search?.[1]).toMatchObject({ query: "tea", user_id: "wife" });
const messages = path.join(directory, "messages.json");
await writeFile(
messages,
JSON.stringify([{ role: "user", content: "I like tea" }]),
);
await capturedRun([
"write",
"submit",
"--messages-file",
messages,
"--session",
"s-1",
"--user-id",
"wife",
]);
expect(direct.post.mock.calls[0][0]).toBe(
"http://127.0.0.1:8765/api/v2/memory-writes",
);
expect(direct.post.mock.calls[0][1]).toMatchObject({
session_id: "s-1",
user_id: "wife",
});
});
it("names no user by default or for --user-id local", async () => {
for (const extra of [[], ["--user-id", "local"]]) {
transport.get.mockClear();
transport.post.mockClear();
await capturedRun(["memories", "list", ...extra, "--json"]);
expect(transport.get).toHaveBeenCalledWith("/api/v2/memory-library", {
params: { offset: 0, limit: 100 },
});
await capturedRun(["profile", ...extra]);
expect(transport.get).toHaveBeenCalledWith(
"/api/v2/user-profile",
undefined,
);
await capturedRun(["memories", "add", "Prefers tea", ...extra]);
expect(transport.post.mock.calls[0][1]).not.toHaveProperty("user_id");
await capturedRun(["recall", "tea", ...extra, "--json"]);
const search = transport.post.mock.calls.find(
([url]) => url === "/api/v2/context/search",
);
expect(search?.[1]).not.toHaveProperty("user_id");
}
});
it("refuses a malformed user id before contacting Core", async () => {
for (const value of ["bad:id", "", "x".repeat(65), "a b", "--json"]) {
const result = await capturedRun([
"memories",
"list",
"--user-id",
value,
]);
expect(result.code).toBe(2);
}
expect(
(await capturedRun(["profile", "--user-id", "a", "--user-id", "b"])).code,
).toBe(2);
expect((await capturedRun(["recall", "tea", "--user-id"])).code).toBe(2);
expect(transport.get).not.toHaveBeenCalled();
expect(transport.post).not.toHaveBeenCalled();
});
});
describe("sources connect for a Hermes profile", () => {
it("connects hermes for another user from its own root", async () => {
const result = await capturedRun([
"sources",
"connect",
"hermes",
"--user-id",
"wife",
"--root",
"~/.hermes/profiles/wife",
"--json",
]);
expect(result.code).toBe(0);
expect(transport.post).toHaveBeenCalledTimes(1);
const [url, body] = transport.post.mock.calls[0];
expect(url).toBe("/api/v1/sources/connect");
expect(body).toMatchObject({
source_ids: ["hermes"],
// Core reads the custom root's identity itself.
source_roots: {},
source_candidate_revisions: {},
onboarding: false,
user_id: "wife",
root: path.join(os.homedir(), ".hermes/profiles/wife"),
});
});
it("keeps a plain connect unchanged and accepts --import with a profile", async () => {
await capturedRun(["sources", "connect", "hermes", "--import", "--json"]);
const plain = transport.post.mock.calls[0][1];
expect(plain).not.toHaveProperty("user_id");
expect(plain).not.toHaveProperty("root");
expect(plain).toMatchObject({
source_roots: { hermes: "hermes-root-identity" },
onboarding: true,
});
transport.post.mockClear();
const result = await capturedRun([
"sources",
"connect",
"hermes",
"--import",
"--user-id",
"wife",
"--root",
"/profiles/wife",
"--request-id",
REQUEST,
"--json",
]);
expect(result.code).toBe(0);
expect(transport.post.mock.calls[0][1]).toMatchObject({
request_id: REQUEST,
user_id: "wife",
root: "/profiles/wife",
});
});
it("refuses --user-id and --root for any source but hermes", async () => {
for (const argv of [
["sources", "connect", "codex", "--user-id", "wife"],
["sources", "connect", "codex", "--root", "/somewhere", "--import"],
["sources", "connect", "hermes", "codex", "--user-id", "wife"],
["sources", "connect", "--all-detected", "--user-id", "wife"],
["sources", "connect", "hermes", "--user-id", "bad:id"],
]) {
const result = await capturedRun(argv);
expect(result.code).toBe(2);
expect(result.stderr).toContain("hermes");
}
expect(transport.post).not.toHaveBeenCalled();
});
});
async function capturedRun(arguments_: string[]) {
let stdout = "";
let stderr = "";
const stdoutSpy = vi
.spyOn(process.stdout, "write")
.mockImplementation((chunk) => {
stdout += String(chunk);
return true;
});
const stderrSpy = vi
.spyOn(process.stderr, "write")
.mockImplementation((chunk) => {
stderr += String(chunk);
return true;
});
try {
return {
code: await run(arguments_, {
ONE_MEMORY_DATA_DIR: "/synthetic/data",
}),
stdout,
stderr,
};
} finally {
stdoutSpy.mockRestore();
stderrSpy.mockRestore();
}
}
+1 -1
View File
@@ -12,7 +12,7 @@ dependencies = [
"fastapi>=0.141.1",
"filelock>=3.32.4",
# Public SDK version validated against MemHub's permission and recall contracts.
"hy-memory==2.0.8",
"hy-memory==2.0.9",
# chromadb arrives through the HY SDK; locked here so an unrelated upgrade
# cannot move the SDK's store. Built-in embedding consumes numpy directly.
"chromadb==1.5.9",
File diff suppressed because it is too large. Load diff
+3 -2
View File
@@ -584,6 +584,7 @@ def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
}
else:
from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot
from one_memory.sdk_data_plane_migration import (
MemoryAuthority,
SdkDataPlaneMigrationStore,
@@ -595,14 +596,14 @@ def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
authority = SdkDataPlaneMigrationStore(database_path).authority()
if authority is MemoryAuthority.SDK:
profile = read_active_profile(database_path)
profile = read_active_profile(database_path) or builtin_retrieval_snapshot()
leftovers = bool(
legacy_tables_present(database_path)
or abandoned_staging_present(
data_dir=settings.data_dir,
namespace=HY_MEMORY_NAMESPACE,
)
or (profile is not None and legacy_vector_collections(settings.data_dir, profile))
or legacy_vector_collections(settings.data_dir, profile)
)
state = "required" if leftovers else "not_required"
else:
@@ -5,6 +5,9 @@ from typing import Annotated, Literal
from pydantic import BaseModel, ConfigDict, Field, model_validator
from one_memory.sdk_search_options import SdkSearchOptions
from one_memory.user_scope import DEFAULT_USER_ID, UserId
ContextKind = Literal["raw", "memory"]
ContextSearchChannel = Literal["lexical", "semantic"]
ContextSemanticStatus = Literal["not_searched", "completed", "failed", "timed_out"]
@@ -47,17 +50,24 @@ class ContextSearchRequest(BaseModel):
return self
class SdkContextSearchRequest(ContextSearchRequest):
class SdkContextSearchRequest(ContextSearchRequest, SdkSearchOptions):
"""`POST /api/v2/context/search`: the three SDK recall lanes are set apart.
`limit` is the normal lane's top-k and the cap on returned items;
`schema_limit` and `intention_limit` are the SDK's schema (L6) and
proactive intention (L7) lanes. A caller's values are passed through.
The `SdkSearchOptions` fields reach the memory lane's `client.search`;
the raw lane takes the floor and the source, session, agent and time
filters (never wider than the caller's raw authority). Absent, both
lanes make the call they always made.
"""
limit: int = Field(default=10, ge=1, le=20, strict=True)
schema_limit: int = Field(default=3, ge=0, le=20, strict=True)
intention_limit: int = Field(default=1, ge=0, le=20, strict=True)
#: Whose memories to search. The App never sends it and reads `local`.
user_id: UserId = DEFAULT_USER_ID
class ContextMessage(BaseModel):
+100 -25
View File
@@ -24,13 +24,57 @@ from one_memory.hy_runtime_config import HY_MEMORY_READER
from one_memory.hy_sdk_runtime import sdk_relevance_floor
from one_memory.hy_sdk_types import (
HySdkMemoryRecord,
parse_sdk_lane_search,
parse_sdk_raw_search,
parse_sdk_search,
)
from one_memory.imports.storage import EvidenceStore
from one_memory.sdk_search_options import SdkSearchFilters, SearchLane
from one_memory.semantic_deadline import SemanticMemoryHit
from one_memory.user_scope import owns
CALL_EVENT_RESPONSE_BUDGET_MS = 250
#: Pass-through filters the raw lane honours. Scene, layers, reader and the
#: schema/working floors describe memory lanes; the raw lane is always the
#: chat store's L1 raw layer.
_RAW_LANE_FILTERS = ("created_after", "session_ids", "agent_ids", "exclude_sources", "rerank")
def raw_lane_sources(
allowed: Sequence[str] | frozenset[str] | None,
requested: Sequence[str] | None,
) -> list[str] | None:
"""The raw lane's source filter: the caller's, never wider than its authority.
`allowed` empty or None means every source (the call Core always made).
None back means nothing is left to search.
"""
allowed_list = list(allowed or ())
if requested is None:
return allowed_list
chosen = [
source for source in dict.fromkeys(requested) if not allowed_list or source in allowed_list
]
return chosen or None
def raw_lane_overrides(value: SdkSearchFilters) -> dict[str, object]:
"""The caller's raw-lane overrides: its floor and its row filters."""
overrides = value.filter_kwargs()
found = {name: overrides[name] for name in _RAW_LANE_FILTERS if name in overrides}
floor = value.lane_floor("raw")
if floor is not None:
found["min_score"] = floor
return found
def hit_namespace(hit: SemanticMemoryHit) -> dict[str, str]:
"""The `get` namespace of a hit: nothing for chat, kind and variant otherwise."""
if hit.kind in {"experience", "fact"} and hit.variant is not None:
return {"kind": hit.kind, "variant": hit.variant}
return {}
def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
"""SDK-native context: no Evidence/Governance ids or fabricated revisions."""
@@ -66,26 +110,36 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
client = await client_handle.client()
except Exception:
raise HTTPException(503, "sdk_context_unavailable") from None
memory_mode = getattr(request.app.state, "memory_mode", None)
lite = callable(memory_mode) and memory_mode() == "lite"
async def raw_lane() -> tuple[SdkRawContextItem, ...]:
if "raw" not in value.kinds or not authority.raw_allowed:
return ()
sources = raw_lane_sources(authority.raw_sources, value.include_sources)
if sources is None:
return ()
raw = await asyncio.to_thread(
client.search,
value.query,
user_ids=["local"],
agent_ids=[],
include_sources=list(authority.raw_sources or ()),
limit=value.limit,
min_score=0.0,
schema_min_score=0.0,
schema_limit=0,
intention_limit=0,
scene="chat",
reader=HY_MEMORY_READER,
layers=["l1_raw"],
**{
"user_ids": [value.user_id],
"agent_ids": [],
"include_sources": sources,
"limit": value.limit,
"min_score": 0.0,
"schema_min_score": 0.0,
"schema_limit": 0,
"intention_limit": 0,
"scene": "chat",
"reader": HY_MEMORY_READER,
"layers": ["l1_raw"],
**raw_lane_overrides(value),
},
)
hits = tuple(
hit for hit in parse_sdk_raw_search(raw) if value.passes_floor("raw", hit.score)
)
hits = parse_sdk_raw_search(raw)
return tuple(
SdkRawContextItem(
raw_id=hit.raw_id,
@@ -103,27 +157,48 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
)
async def memory_lane() -> tuple[SdkMemoryContextItem, ...]:
if "memory" not in value.kinds or not authority.memory_allowed:
# lite extracts nothing: its search answers raw turns, which the
# raw lane already reads under the raw authority.
if "memory" not in value.kinds or not authority.memory_allowed or lite:
return ()
overrides = value.search_overrides()
raw = await asyncio.to_thread(
client.search,
value.query,
user_ids=["local"],
agent_ids=[],
limit=value.limit,
min_score=sdk_relevance_floor(),
schema_min_score=sdk_relevance_floor(),
schema_limit=value.schema_limit,
intention_limit=value.intention_limit,
scene="normal",
reader=HY_MEMORY_READER,
**{
"user_ids": [value.user_id],
"agent_ids": [],
"limit": value.limit,
"min_score": sdk_relevance_floor(),
"schema_min_score": sdk_relevance_floor(),
"schema_limit": value.schema_limit,
"intention_limit": value.intention_limit,
"scene": "normal",
"reader": HY_MEMORY_READER,
**overrides,
},
)
hits = (
tuple(
hit
for hit in parse_sdk_lane_search(
raw, error_code="hy_memory_search_shape_unknown"
)
if value.passes_floor(cast(SearchLane, hit.lane), hit.score)
)
if overrides
else parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
)
hits = parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
items: list[SdkMemoryContextItem] = []
for hit in hits:
record = await asyncio.to_thread(client.get, hit.memory_id)
record = await asyncio.to_thread(client.get, hit.memory_id, **hit_namespace(hit))
memory = None if record is None else HySdkMemoryRecord.from_sdk(record)
if memory is None or memory.status != "active":
# `get` takes no user: a hit is kept only if it is the caller's.
if (
memory is None
or memory.status != "active"
or not owns(memory.user_id, value.user_id)
):
continue
items.append(
SdkMemoryContextItem(
@@ -71,6 +71,16 @@ def graph_store_enabled() -> bool:
return raw != "none"
def effective_memory_mode(configured: str = "pro") -> str:
"""The SDK mode clients are built with: the setting, unless dev forces ultra.
`ONE_MEMORY_GRAPH_STORE` (any value but empty or `none`) predates the
setting and keeps working for development: it forces ultra whatever the
configured mode. Without it the configured mode (default pro) applies.
"""
return "ultra" if graph_store_enabled() else configured
def inspect_graph_width_mismatch(config: Any) -> GraphWidthMismatch | None:
"""Return a mismatch without renaming or deleting anything."""
db_path = Path(config.graph_store.db_path)
@@ -83,11 +93,12 @@ def inspect_profile_graph_width(
*,
api_key: str | None,
data_dir: Path,
mode: str | None = None,
) -> GraphWidthMismatch | None:
"""Inspect the graph path and width selected by the active profile."""
del api_key
return _inspect_graph_width(
_profile_graph_path(data_dir),
_profile_graph_path(data_dir, mode),
_profile_embedding_width(profile),
)
@@ -98,11 +109,12 @@ def decide_profile_graph_width(
api_key: str | None,
data_dir: Path,
decision: GraphWidthDecision,
mode: str | None = None,
) -> GraphWidthOutcome:
"""Apply the user's decision to the active profile's graph."""
del api_key
config = _GraphAdminConfig(
graph_store=_GraphStoreConfig(str(_profile_graph_path(data_dir))),
graph_store=_GraphStoreConfig(str(_profile_graph_path(data_dir, mode))),
embedder=_EmbeddingWidthConfig(_profile_embedding_width(profile)),
)
return apply_graph_width_decision(
@@ -240,8 +252,9 @@ def _inspect_graph_width(
)
def _profile_graph_path(data_dir: Path) -> Path:
name = "graph" if graph_store_enabled() else "graph-disabled"
def _profile_graph_path(data_dir: Path, mode: str | None = None) -> Path:
ultra = mode == "ultra" if mode is not None else graph_store_enabled()
name = "graph" if ultra else "graph-disabled"
return data_dir / "hy-memory" / HY_MEMORY_NAMESPACE / name
@@ -14,7 +14,7 @@ from pathlib import Path
from typing import Literal, cast
HY_MEMORY_DISTRIBUTION = "hy-memory"
HY_MEMORY_VERSION = "2.0.8"
HY_MEMORY_VERSION = "2.0.9"
HY_MEMORY_NAMESPACE = "governed-v2"
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
@@ -497,7 +497,7 @@ def _ensure_coding_schema(path: Path) -> None:
CodingMemoryStore,
)
# Relies on the pinned SDK's internals (hy-memory==2.0.8 is pinned exactly).
# Relies on the pinned SDK's internals (hy-memory==2.0.9 is pinned exactly).
store = CodingMemoryStore.__new__(CodingMemoryStore)
store._db_path = str(path)
store._conn = None
@@ -31,7 +31,7 @@ from one_memory.embedding_assets import (
)
from one_memory.hy_graph_admin import (
GraphWidthMismatchError,
graph_store_enabled,
effective_memory_mode,
inspect_graph_width_mismatch,
)
from one_memory.hy_sdk_types import HySdkRuntimeError
@@ -170,11 +170,14 @@ class SdkClientHandle:
api_key: str | None,
data_dir: Path,
http_transport_factory: Callable[[], Any] | None = None,
mode: str | None = None,
) -> None:
self._profile = profile
self._api_key = api_key
self._data_dir = data_dir
self._http_transport_factory = http_transport_factory
#: The SDK mode this handle's client is built with, fixed for its life.
self.mode = mode or effective_memory_mode()
self._lease: ClientLease | None = None
self._lock = asyncio.Lock()
@@ -189,6 +192,7 @@ class SdkClientHandle:
api_key=self._api_key,
data_dir=self._data_dir,
http_transport_factory=self._http_transport_factory,
mode=self.mode,
)
return self._lease.client
@@ -217,6 +221,33 @@ def builtin_retrieval_snapshot() -> ModelProfileSnapshot:
)
def retrieval_view(profile: ModelProfileSnapshot) -> ModelProfileSnapshot:
"""The read side of `profile`: its embedding space, and no generation.
Reads must open the collection writes fill (#242), and the collection is
named by the embedding only. Generation stays the built-in placeholder, so
a read never needs the generation model. A remote embedding still needs
the profile's API key, which `build_sdk_config` uses as the embedding key.
"""
provider = profile.embedding_provider_kind or profile.provider_kind
endpoint: str | None = None
if provider is not ProviderKind.LOCAL_ONNX:
endpoint = profile.embedding_base_url
if not endpoint:
raise HySdkRuntimeError("hy_sdk_model_endpoint_missing")
return builtin_retrieval_snapshot().model_copy(
update={
"profile_id": profile.profile_id,
"revision": profile.revision,
"embedding_provider_kind": provider,
"embedding_model": profile.embedding_model,
"embedding_base_url": endpoint,
"embedding_dimensions": profile.embedding_dimensions,
"has_api_key": profile.has_api_key,
}
)
def sdk_relevance_floor() -> float:
raw = os.environ.get("ONE_MEMORY_HY_RELEVANCE_FLOOR", "").strip()
if not raw:
@@ -249,10 +280,8 @@ def build_sdk_config(
embedding_key = None
dimensions = profile.embedding_dimensions or BUILTIN_EMBEDDING_DIMENSIONS
else:
embedding_endpoint = _openai_endpoint(
embedding_provider,
profile.embedding_base_url or profile.base_url,
)
# Never the generation endpoint: the collection name hashes this one.
embedding_endpoint = _openai_endpoint(embedding_provider, profile.embedding_base_url)
embedding_key = "ollama" if embedding_provider is ProviderKind.OLLAMA else api_key
if embedding_provider is ProviderKind.OPENAI_COMPATIBLE and not embedding_key:
raise HySdkRuntimeError("hy_sdk_embedding_api_key_missing")
@@ -327,9 +356,12 @@ def acquire_client(
api_key: str | None,
data_dir: Path,
http_transport_factory: Callable[[], Any] | None = None,
mode: str | None = None,
) -> ClientLease:
"""Construct a public SDK client and return its explicit runtime lease.
`mode` is the SDK pipeline mode (lite / pro / ultra); None keeps the
pre-setting default (ultra when `ONE_MEMORY_GRAPH_STORE` is set, else pro).
Waits while `hold_stores_offline` holds this home's stores.
"""
key = data_dir.resolve()
@@ -343,6 +375,7 @@ def acquire_client(
api_key=api_key,
data_dir=data_dir,
http_transport_factory=http_transport_factory,
mode=mode or effective_memory_mode(),
)
finally:
with _STORE_GATE:
@@ -381,6 +414,7 @@ def _acquire_client(
api_key: str | None,
data_dir: Path,
http_transport_factory: Callable[[], Any] | None,
mode: str = "pro",
) -> ClientLease:
config = build_sdk_config(profile, api_key=api_key, data_dir=data_dir)
config.http_transport_factory = http_transport_factory
@@ -398,7 +432,7 @@ def _acquire_client(
with constructing:
client = HyMemoryClient(
config=config,
mode="ultra" if graph_store_enabled() else "pro",
mode=mode,
runtime=runtime,
embedding_service=embedding_service,
)
@@ -467,7 +501,7 @@ def vector_collection_bases(profile: ModelProfileSnapshot) -> tuple[str, tuple[s
endpoint, dimensions = None, profile.embedding_dimensions or BUILTIN_EMBEDDING_DIMENSIONS
legacy_endpoint = "local-onnx"
else:
endpoint = _openai_endpoint(provider, profile.embedding_base_url or profile.base_url)
endpoint = _openai_endpoint(provider, profile.embedding_base_url)
dimensions = profile.embedding_dimensions
legacy_endpoint = endpoint
legacy_model = profile.embedding_model or "bge-small-zh-v1.5"
@@ -2,6 +2,7 @@
from __future__ import annotations
import re
from collections.abc import Mapping
from datetime import UTC, datetime
from typing import Literal, NamedTuple, cast
@@ -391,6 +392,8 @@ class HySdkRawRecord(BaseModel):
memory_at: int | None = None
gmt_created: int | None = None
custom: dict[str, object] = Field(default_factory=dict)
#: active, or shadow once pro extracted it; None when the SDK gave none.
status: str | None = None
@classmethod
def from_sdk(cls, raw: object) -> HySdkRawRecord:
@@ -417,11 +420,34 @@ class HySdkRawRecord(BaseModel):
raw.get("gmt_created") if isinstance(raw.get("gmt_created"), int) else None
),
custom=dict(custom),
status=str(raw["status"]) if isinstance(raw.get("status"), str) else None,
)
except (KeyError, TypeError, ValueError) as error:
raise HySdkReadError("hy_memory_raw_shape_unknown") from error
_RAW_TURN_LINE = re.compile(r"^\[(user|assistant|tool|system)\]:\s?(.*)$")
def raw_turn_messages(content: str) -> tuple[tuple[str, str], ...]:
"""The `(role, text)` messages of one SDK raw turn.
The SDK stores a turn as `[user]: …\\n[assistant]: …`; a line without a
role prefix continues the message above it. Content that names no role
at all is one message of unknown role (`""`).
"""
messages: list[list[str]] = []
for line in content.splitlines():
match = _RAW_TURN_LINE.match(line)
if match is not None:
messages.append([match.group(1), match.group(2)])
elif messages:
messages[-1][1] += "\n" + line
else:
messages.append(["", line])
return tuple((role, text) for role, text in messages)
class HySdkRawHit(HySdkRawRecord):
score: float = Field(ge=0, le=1)
@@ -516,6 +542,96 @@ class HySdkUserProfile(BaseModel):
)
class HySdkProfileVersion(BaseModel):
"""One version on an L0 basic-info chain."""
model_config = ConfigDict(frozen=True, extra="forbid")
memory_id: str
content: str
#: Conversation time as the SDK renders it on a chain (epoch or text).
memory_at: int | str | None = None
gmt_created: int | None = None
#: The chain head: the version the profile shows now.
current: bool = False
class HySdkProfileChain(BaseModel):
"""One L0 basic-info head and its versions, latest first."""
model_config = ConfigDict(frozen=True, extra="forbid")
memory_id: str
content: str
status: str
memory_at: int | None = None
gmt_created: int | None = None
#: The structured fields this head holds (`basic_info_kv`).
basic_info: dict[str, object] = Field(default_factory=dict)
versions: tuple[HySdkProfileVersion, ...]
class HySdkProfileVersions(BaseModel):
"""`get_user_basic_info`: the long-term profile's L0 version chains."""
model_config = ConfigDict(frozen=True, extra="forbid")
request_id: str
chains: tuple[HySdkProfileChain, ...]
elapsed_ms: float = Field(ge=0)
@classmethod
def from_sdk(cls, raw: object) -> HySdkProfileVersions:
if not isinstance(raw, Mapping):
raise HySdkReadError("hy_memory_profile_versions_shape_unknown")
request_id = raw.get("request_id")
rows = raw.get("memories")
elapsed_ms = raw.get("elapsed_ms")
if (
not isinstance(request_id, str)
or not isinstance(rows, list)
or not isinstance(elapsed_ms, int | float)
or isinstance(elapsed_ms, bool)
):
raise HySdkReadError("hy_memory_profile_versions_shape_unknown")
chains: list[HySdkProfileChain] = []
for row in rows:
if not isinstance(row, Mapping) or not isinstance(row.get("memory_id"), str):
raise HySdkReadError("hy_memory_profile_versions_shape_unknown")
head_id = str(row["memory_id"])
chain = row.get("evolution_chain")
entries = chain if isinstance(chain, list) and chain else [{**row, "node_id": head_id}]
versions = tuple(
HySdkProfileVersion(
memory_id=str(entry.get("node_id") or entry.get("memory_id")),
content=str(entry.get("content") or ""),
memory_at=(
entry.get("memory_at")
if isinstance(entry.get("memory_at"), int | str)
and not isinstance(entry.get("memory_at"), bool)
else None
),
gmt_created=_epoch(entry.get("gmt_created")),
current=str(entry.get("node_id") or entry.get("memory_id")) == head_id,
)
for entry in entries
if isinstance(entry, Mapping) and (entry.get("node_id") or entry.get("memory_id"))
)
basic_info = row.get("basic_info_kv")
chains.append(
HySdkProfileChain(
memory_id=head_id,
content=str(row.get("content") or ""),
status=str(row.get("status") or "active"),
memory_at=_epoch(row.get("memory_at")),
gmt_created=_epoch(row.get("gmt_created")),
basic_info=dict(basic_info) if isinstance(basic_info, Mapping) else {},
versions=versions,
)
)
return cls(request_id=request_id, chains=tuple(chains), elapsed_ms=float(elapsed_ms))
def parse_sdk_search(raw: object, *, error_code: str) -> tuple[SemanticMemoryHit, ...]:
rows = _reply_rows(raw, error_code)
found: list[SemanticMemoryHit] = []
@@ -542,6 +658,120 @@ def parse_sdk_search(raw: object, *, error_code: str) -> tuple[SemanticMemoryHit
return tuple(found)
def parse_sdk_lane_search(raw: object, *, error_code: str) -> tuple[SemanticMemoryHit, ...]:
"""Every scene's search reply as hits that name their lane and namespace.
`parse_sdk_search` reads the chat reply only and forgets which lane a hit
came from. A caller may now pick the scene and a floor per lane, so this
reads all three reply shapes:
- chat: `memories` as `{schema, proactive, normal}` (or one list, normal);
- working/coding: `facts` and `experiences` of that variant;
- all: one fused `memories` list whose rows carry `scene`, `kind` and
their own lane's score (`lane_score`). The fused `score` is a rank
fusion (1/(60+rank)), not a relevance, so the lane score is reported.
Highest score first, first occurrence of an id kept.
"""
if not isinstance(raw, Mapping):
raise HySdkReadError(error_code)
scene = raw.get("scene")
found: list[SemanticMemoryHit] = []
if scene == "all":
rows = raw.get("memories")
if not isinstance(rows, list) or any(not isinstance(row, Mapping) for row in rows):
raise HySdkReadError(error_code)
for row in rows:
hit = _lane_hit(row, scene=str(row.get("scene") or "chat"), score_key="lane_score")
if hit is not None:
found.append(hit)
elif scene in ("productivity", "working", "coding"):
variant = raw.get("variant") if raw.get("variant") in {"working", "coding"} else None
lane = variant or (scene if scene in {"working", "coding"} else "working")
for key, kind in (("facts", "fact"), ("experiences", "experience")):
rows = raw.get(key) or []
if not isinstance(rows, list) or any(not isinstance(row, Mapping) for row in rows):
raise HySdkReadError(error_code)
for row in rows:
hit = _lane_hit({**row, "kind": kind}, scene=lane, score_key="score")
if hit is not None:
found.append(hit)
else:
memories = raw.get("memories")
if isinstance(memories, Mapping):
if set(memories) != {"schema", "proactive", "normal"}:
raise HySdkReadError(error_code)
channels = tuple(memories.items())
elif isinstance(memories, list):
channels = (("normal", memories),)
else:
raise HySdkReadError(error_code)
for lane, rows in channels:
if not isinstance(rows, list) or any(not isinstance(row, Mapping) for row in rows):
raise HySdkReadError(error_code)
for row in rows:
hit = _lane_hit(row, scene="chat", score_key="score", lane=str(lane))
if hit is not None:
found.append(hit)
ordered: list[SemanticMemoryHit] = []
seen: set[tuple[str, str, str | None]] = set()
for hit in sorted(found, key=lambda value: value.score, reverse=True):
identity = (hit.memory_id, hit.kind, hit.variant)
if identity in seen:
continue
seen.add(identity)
ordered.append(hit)
return tuple(ordered)
def _lane_hit(
row: Mapping[str, object],
*,
scene: str,
score_key: str,
lane: str | None = None,
) -> SemanticMemoryHit | None:
kind = str(row.get("kind")) if row.get("kind") in {"experience", "fact"} else "chat"
if kind == "chat":
memory_id = row.get("memory_id") or row.get("id")
variant = None
hit_lane = lane or "normal"
else:
id_key = "experience_id" if kind == "experience" else "fact_id"
memory_id = row.get("memory_id") or row.get(id_key)
variant = scene if scene in {"working", "coding"} else None
if variant is None:
return None
hit_lane = variant
if not isinstance(memory_id, str) or not memory_id:
return None
value = row.get(score_key)
if not isinstance(value, int | float) or isinstance(value, bool):
value = row.get("score")
score = (
max(0.0, min(1.0, float(value)))
if isinstance(value, int | float) and not isinstance(value, bool)
else 0.0
)
if kind != "chat":
return SemanticMemoryHit(
memory_id=memory_id, score=score, lane=hit_lane, kind=kind, variant=variant
)
content = row.get("content")
member_ids, formatting_complete = _formatted_member_ids(row)
return SemanticMemoryHit(
memory_id=memory_id,
score=score,
formatted_content=(
content.strip() if isinstance(content, str) and content.strip() else None
),
member_ids=member_ids,
sdk_formatted=True,
formatting_complete=formatting_complete,
lane=hit_lane,
)
class LibraryHit(NamedTuple):
"""One keyword match in a library search window, with its namespace."""
@@ -31,6 +31,16 @@ class ScanIssueCode(StrEnum):
#: import it. Named for the reason, so a reader can tell a setting from
#: a record the parser could not use.
SUBAGENT_TRACE_DISABLED = "subagent_trace_disabled"
#: A host's own notice wearing the user's role — Hermes posts a finished
#: background process, and its gateway marks the post as not written by a
#: human. Nothing in it is the person's, so it never reaches extraction.
INJECTED_MACHINE_NOTICE = "injected_machine_notice"
#: The host could not transcribe a voice message and left a placeholder
#: naming the audio file. The person did speak, but the placeholder holds
#: only a path, which an extractor would otherwise read as a fact.
VOICE_PLACEHOLDER = "voice_placeholder"
#: A health check such as "Reply with exactly: PONG": no one's words.
PROBE = "probe"
SOURCE_BUSY = "source_busy"
UNSUPPORTED_SCHEMA = "unsupported_schema"
BUDGET_EXCEEDED = "budget_exceeded"
@@ -27,6 +27,7 @@ def discovered_off_loop(
*,
suffixes: frozenset[str],
max_targets: int,
excluded_top_directories: frozenset[str] = frozenset(),
) -> AsyncIterator[SourceFileDiscoveryEvent]:
"""The same walk, for callers that have an event loop to keep free.
@@ -41,6 +42,7 @@ def discovered_off_loop(
grant,
suffixes=suffixes,
max_targets=max_targets,
excluded_top_directories=excluded_top_directories,
),
scope_key=grant.source_id,
)
@@ -51,12 +53,15 @@ def discover_source_files(
*,
suffixes: frozenset[str],
max_targets: int,
excluded_top_directories: frozenset[str] = frozenset(),
) -> Iterator[SourceFileDiscoveryEvent]:
"""Metadata only: one lstat per candidate, no open and no per-file grant lookup.
The grant and root identity are checked once before the walk. Each listed
file keeps the grant's check, which runs again when that file is opened;
the open itself refuses symlinks and any inode other than the one listed.
`excluded_top_directories` are directories directly under the root that
are never entered, such as another user's Hermes profiles.
"""
grant.check_root()
inspected = 0
@@ -64,8 +69,12 @@ def discover_source_files(
grant.root,
followlinks=False,
):
at_root = Path(directory) == grant.root
child_directories[:] = sorted(
child for child in child_directories if not (Path(directory) / child).is_symlink()
child
for child in child_directories
if not (Path(directory) / child).is_symlink()
and not (at_root and child in excluded_top_directories)
)
for filename in sorted(filenames):
if Path(filename).suffix.lower() not in suffixes:
@@ -252,6 +252,18 @@ class SourceGrantStore:
).fetchone()
return self._to_receipt(row) if row else None
def grants(self) -> tuple[tuple[SourceGrantReceipt, str], ...]:
"""Every grant of every user, with its granted root path.
One source may hold several grants, one per `(user_id, source_id)`:
a Hermes root for `local` and a Hermes profile for another user.
"""
with self._connect() as connection:
rows = connection.execute(
"SELECT * FROM source_grants ORDER BY source_id, user_id"
).fetchall()
return tuple((self._to_receipt(row), str(row["requested_path"])) for row in rows)
def require_receipt(self, *, user_id: str, source_id: str) -> SourceGrantReceipt:
receipt = self.receipt(user_id=user_id, source_id=source_id)
if receipt is None:
@@ -28,6 +28,7 @@ from one_memory.imports.adapters import (
)
from one_memory.imports.jobs import ImportJob, ImportJobStore, ImportStatus
from one_memory.imports.supervisor import ImportSupervisor
from one_memory.user_scope import DEFAULT_USER_ID, lane_key, split_lane
#: How long an automatic retry will wait for a wedged worker to notice cancel.
ABANDON_TIMEOUT_SECONDS = 5.0
@@ -64,7 +65,12 @@ class LiveSyncHealth(BaseModel):
class PollingLiveSyncSupervisor:
"""Near-live sync that reuses each adapter's persisted incremental cursor."""
"""Near-live sync that reuses each adapter's persisted incremental cursor.
A "source" here is a lane key (`user_scope.lane_key`): the bare source id
for `local`, `source@user` for another user's grant. Cursors, jobs and
locks are therefore per `(user_id, source_id)`.
"""
def __init__(
self,
@@ -92,8 +98,22 @@ class PollingLiveSyncSupervisor:
}
self._locks = {source_id: asyncio.Lock() for source_id in self.source_ids}
self._task: asyncio.Task[None] | None = None
self._group: asyncio.TaskGroup | None = None
self._observers: set[asyncio.Task[None]] = set()
def ensure_source(self, source_id: str) -> None:
"""Follow a lane granted after start, such as a new Hermes profile."""
if source_id in self._locks:
return
self.source_ids = (*self.source_ids, source_id)
self._health.setdefault(
source_id,
LiveSyncHealth(source_id=source_id, mode=self.mode, state=LiveSyncState.IDLE),
)
self._locks[source_id] = asyncio.Lock()
if self._group is not None:
self._group.create_task(self._poll_source(source_id))
def _set_health(self, source_id: str, value: LiveSyncHealth) -> None:
previous = self._health.get(source_id)
self._health[source_id] = value
@@ -152,7 +172,12 @@ class PollingLiveSyncSupervisor:
self._watched_sources = frozenset(source_ids)
def all_health(self) -> tuple[LiveSyncHealth, ...]:
return tuple(self.health(source_id) for source_id in self.source_ids)
"""`local`'s lanes only: what the App has always been shown."""
return tuple(
self.health(source_id)
for source_id in self.source_ids
if split_lane(source_id)[0] == DEFAULT_USER_ID
)
async def initialize_source(
self, source_id: str, *, baseline_missing_files: bool = True
@@ -164,11 +189,12 @@ class PollingLiveSyncSupervisor:
baselines = (
await adapter.live_baseline() if isinstance(adapter, _LiveBaselineAdapter) else {}
)
user_id, bare_source_id = split_lane(source_id)
await in_the_background(
partial(
self.job_store.ensure_live_cursors,
user_id="local",
source_id=source_id,
user_id=user_id,
source_id=bare_source_id,
baselines=baselines,
baseline_missing_files=baseline_missing_files,
)
@@ -228,7 +254,7 @@ class PollingLiveSyncSupervisor:
(
job
for job in reversed(source_jobs)
if job.source_id == source_id
if lane_key(job.user_id, job.source_id) == source_id
and job.operation == "sync"
and job.status
in {
@@ -277,13 +303,13 @@ class PollingLiveSyncSupervisor:
competing = [
job
for job in source_jobs
if job.source_id == source_id
if lane_key(job.user_id, job.source_id) == source_id
and job.operation != "sync"
and job.status
in {ImportStatus.PENDING, ImportStatus.RUNNING, ImportStatus.INTERRUPTED}
]
bounded_history_incomplete = any(
job.source_id == source_id
lane_key(job.user_id, job.source_id) == source_id
and job.operation == "history"
and job.history_after is not None
and job.status is not ImportStatus.COMPLETED
@@ -379,11 +405,12 @@ class PollingLiveSyncSupervisor:
await close()
async def _live_cursors(self, source_id: str, adapter: object) -> dict[str, SourceCursor]:
user_id, bare_source_id = split_lane(source_id)
cursors = await in_the_background(
partial(
self.job_store.get_source_cursors,
user_id="local",
source_id=source_id,
user_id=user_id,
source_id=bare_source_id,
operation="sync",
)
)
@@ -395,8 +422,8 @@ class PollingLiveSyncSupervisor:
return await in_the_background(
partial(
self.job_store.ensure_live_cursors,
user_id="local",
source_id=source_id,
user_id=user_id,
source_id=bare_source_id,
baselines=baselines,
)
)
@@ -520,9 +547,13 @@ class PollingLiveSyncSupervisor:
async def _run(self) -> None:
# A source owns its cadence; one large catch-up cannot delay every
# other source. The group keeps shutdown responsible for every poller.
async with asyncio.TaskGroup() as group:
for source_id in self.source_ids:
group.create_task(self._poll_source(source_id))
try:
async with asyncio.TaskGroup() as group:
self._group = group
for source_id in self.source_ids:
group.create_task(self._poll_source(source_id))
finally:
self._group = None
async def _poll_source(self, source_id: str) -> None:
while True:
@@ -3,7 +3,7 @@ import logging
import math
import sqlite3
from collections import Counter
from collections.abc import Callable
from collections.abc import AsyncIterator, Callable
from contextlib import suppress
from contextvars import ContextVar
from dataclasses import dataclass, field
@@ -227,14 +227,17 @@ class ImportRunner:
heartbeat = asyncio.create_task(self._keep_alive(job_id=job_id, runner_token=runner_token))
pending = _PendingSkips()
writes = _PendingWrites()
stream = adapter.scan(
ScanOptions(
cursors=cursors,
reuse_listing=(
job.operation == "history"
or (job.operation == "sync" and job.source_id == "workbuddy")
),
)
stream = _owned_by(
adapter.scan(
ScanOptions(
cursors=cursors,
reuse_listing=(
job.operation == "history"
or (job.operation == "sync" and job.source_id == "workbuddy")
),
)
),
job.user_id,
)
slice_cursors = dict(cursors)
next_event: asyncio.Future[ScanEvent] | None = None
@@ -924,3 +927,30 @@ async def _settled_work[T](
with suppress(asyncio.CancelledError, Exception):
task.result()
raise
def _owned_by(stream: AsyncIterator[ScanEvent], user_id: str) -> AsyncIterator[ScanEvent]:
"""Stamp every scanned record with the user whose root is being read.
Adapters know nothing of users and emit `local`. Which user a record
belongs to is decided by the grant the job reads (a Hermes profile's
`state.db` is that profile's user), so the job's user is the record's.
`local` passes the stream through untouched.
"""
if user_id == "local":
return stream
return _restamped(stream, user_id)
async def _restamped(stream: AsyncIterator[ScanEvent], user_id: str) -> AsyncIterator[ScanEvent]:
try:
async for event in stream:
if isinstance(event, ScannedEvidence) and event.evidence.user_id != user_id:
event = event.model_copy(
update={"evidence": event.evidence.model_copy(update={"user_id": user_id})}
)
yield event
finally:
close = getattr(stream, "aclose", None)
if close is not None:
await close()
@@ -1,4 +1,5 @@
import json
import re
import sqlite3
from collections.abc import AsyncIterator, Iterator
from functools import partial
@@ -87,6 +88,7 @@ class HermesSourceAdapter:
self.grant,
suffixes=frozenset({".jsonl", ".db"}),
max_targets=10_000,
excluded_top_directories=frozenset({HERMES_PROFILES_DIRECTORY}),
):
if isinstance(item, AuthorizedSourceFile) and _is_hermes_target(
item.relative_path.as_posix()
@@ -100,6 +102,7 @@ class HermesSourceAdapter:
self.grant,
suffixes=frozenset({".jsonl", ".db"}),
max_targets=options.max_scan_targets or 100_000,
excluded_top_directories=frozenset({HERMES_PROFILES_DIRECTORY}),
):
yield ScanProgressEvent(
phase=ScanPhase.DISCOVER,
@@ -214,7 +217,11 @@ def parse_hermes_rollout_record(
message.get("content"),
allowed_block_types=frozenset({"text", "input_text", "output_text"}),
)
text = strip_injected_memory(text)
text = strip_injected_memory(text or "")
if role == "user":
text, injected = gateway_user_text(text)
if injected is not None:
return ParsedSkip(reason_code=injected, conversation_id=conversation_id)
if not text:
return ParsedSkip(
reason_code=ScanIssueCode.NO_TEXT,
@@ -242,8 +249,17 @@ def parse_hermes_rollout_record(
)
#: Hermes keeps each extra profile under `<root>/profiles/<name>/`, with its
#: own `state.db` and sessions. A profile is another person's history: it is
#: read only through its own grant (its own MemHub user), never as part of the
#: root above it. Hermes' own export skips this directory for the same reason.
HERMES_PROFILES_DIRECTORY = "profiles"
def _is_hermes_target(relative_path: str) -> bool:
path = PurePosixPath(relative_path)
if path.parts and path.parts[0] == HERMES_PROFILES_DIRECTORY:
return False
return path.name == "state.db" or (path.suffix == ".jsonl" and "sessions" in path.parts)
@@ -389,6 +405,15 @@ def _read_state_db(
# `tool` rows after it hold only what came back.
calls = _tool_calls_text(row["tool_calls"]) if role == "assistant" else None
content = strip_injected_memory(str(row["content"] or ""))
if role == "user":
content, injected = gateway_user_text(content)
if injected is not None:
yield ScanSkip(
cursor=cursor,
conversation_id=str(row["session_id"]),
reason_code=injected,
)
continue
has_text = bool(content.strip())
if role not in {"user", "assistant"} or (not has_text and calls is None):
yield ScanSkip(
@@ -439,6 +464,69 @@ def _read_state_db(
_PERSON_SESSION_SOURCES = frozenset({"cli", "tui", "webui", "desktop"})
#: What the Hermes gateway writes into a `user` row on its own behalf. Every
#: one of these wears the `user` role, so only the text can tell them apart
#: from the person. Read off real `state.db` rows: a finished background
#: process is posted as a turn, often with the gateway's own "not written by
#: a human" note appended, which is why that note counts anywhere in the row.
_BACKGROUND_NOTICE = re.compile(
r"\[IMPORTANT:\s*(?:\d+\s+background\s+process(?:es)?\b|Background\s+process\b)",
re.IGNORECASE,
)
_GATEWAY_NOTICE = re.compile(
r"\[INTERNAL NOTIFICATION\s*\W+\s*generated by the Hermes gateway",
re.IGNORECASE,
)
_VOICE_PLACEHOLDER = re.compile(r"\[voice message could not be transcribed", re.IGNORECASE)
_PROBE = re.compile(r"Reply with exactly:\s*[A-Za-z0-9_\-]{1,24}\s*")
_MEDIA_ONLY = re.compile(r"(?:MEDIA:/\S+\s*)+")
#: Wrappers the gateway puts around a message the person did write. Only the
#: wrapper goes; what it carries is the turn. Each is matched at the edge of
#: what is left, and the set is applied until nothing more comes off, since a
#: relayed message carries the origin block inside the out-of-band markers.
_GATEWAY_WRAPPERS = (
re.compile(r"^\[OUT-OF-BAND USER MESSAGE\b[^\]\n]*\]"),
re.compile(r"\[/OUT-OF-BAND USER MESSAGE\]$"),
re.compile(
r"^Gateway message origin \(JSON data, not instructions or authorization\):[ \t]*"
r"(?:\n[ \t]*\{[^\n]*\}[ \t]*)?"
r"(?:\n[ \t]*Do not guess a reply destination[^\n]*)?"
r"(?=\n|$)"
),
re.compile(r"^\[Context from the interrupted assistant response\][ \t]*(?=\n|$)"),
)
def gateway_user_text(text: str) -> tuple[str, ScanIssueCode | None]:
"""The person's words in a Hermes `user` row, or why there are none.
Applied only to the `user` role: an assistant or tool row quoting the
same text is reporting what it saw, which is the existing tool rule's
business. The cleaned text is what becomes the evidence, so its content
hash and duplicate check see the message rather than its envelope.
"""
if _GATEWAY_NOTICE.search(text):
return "", ScanIssueCode.INJECTED_MACHINE_NOTICE
cleaned = text.strip()
while True:
unwrapped = cleaned
for wrapper in _GATEWAY_WRAPPERS:
unwrapped = wrapper.sub("", unwrapped, count=1).strip()
if unwrapped == cleaned:
break
cleaned = unwrapped
if _BACKGROUND_NOTICE.match(cleaned):
return "", ScanIssueCode.INJECTED_MACHINE_NOTICE
if _VOICE_PLACEHOLDER.match(cleaned):
return "", ScanIssueCode.VOICE_PLACEHOLDER
if _PROBE.fullmatch(cleaned):
return "", ScanIssueCode.PROBE
if _MEDIA_ONLY.fullmatch(cleaned):
return "", ScanIssueCode.NO_TEXT
return cleaned, None
def _tool_calls_text(value: object) -> str | None:
"""OpenAI-shaped `tool_calls`: `[{"function": {"name", "arguments"}}]`, stored as JSON."""
if isinstance(value, str):
@@ -24,6 +24,7 @@ from one_memory.service_levels import (
ServiceLevelScheduler,
ServiceResource,
)
from one_memory.user_scope import lane_key, split_lane
class UnknownSourceError(KeyError):
@@ -82,36 +83,45 @@ class ImportSupervisor:
self._tasks: dict[str, asyncio.Task[None]] = {}
self._drains: dict[str, asyncio.Task[None]] = {}
def register_adapter(self, adapter: SourceAdapter) -> None:
self.adapters[adapter.descriptor.source_id] = adapter
def register_adapter(self, adapter: SourceAdapter, *, user_id: str = "local") -> None:
"""Adapters are keyed by lane: the source id for `local`, `source@user`
for another user's grant of the same source."""
self.adapters[lane_key(user_id, adapter.descriptor.source_id)] = adapter
def unregister_adapter(self, source_id: str) -> None:
self.adapters.pop(source_id, None)
@staticmethod
def lane_of(job: ImportJob) -> str:
return lane_key(job.user_id, job.source_id)
def active_job_ids(self) -> frozenset[str]:
return frozenset(job_id for job_id, task in self._tasks.items() if not task.done())
def create_job(
self, source_id: str, *, operation: Literal["history", "sync"] = "history"
) -> ImportJob:
"""`source_id` is a lane key: another user's lane creates that user's job."""
if self.source_enabled is not None and not self.source_enabled(source_id):
raise SourceAccessError(SourceAccessErrorCode.GRANT_REQUIRED)
return self.job_store.create(user_id="local", source_id=source_id, operation=operation)
user_id, bare_source_id = split_lane(source_id)
return self.job_store.create(user_id=user_id, source_id=bare_source_id, operation=operation)
def enqueue(self, job_id: str) -> ImportJob:
if job_id in self._drains:
raise InvalidJobTransitionError("import_cleanup_in_progress")
job = self.job_store.require(job_id)
if job.source_id not in self.adapters:
raise UnknownSourceError(job.source_id)
if self.source_enabled is not None and not self.source_enabled(job.source_id):
lane = self.lane_of(job)
if lane not in self.adapters:
raise UnknownSourceError(lane)
if self.source_enabled is not None and not self.source_enabled(lane):
return job
existing = self._tasks.get(job_id)
if existing is None or existing.done():
self._tasks[job_id] = asyncio.create_task(
self._execute(
job_id,
source_id=job.source_id,
source_id=lane,
operation=job.operation,
)
)
@@ -130,8 +140,10 @@ class ImportSupervisor:
# pending history. Each resumed runner will yield after one bounded slice.
for status in (ImportStatus.INTERRUPTED, ImportStatus.PENDING):
for job in self.job_store.list_jobs(status=status):
if job.source_id in self.adapters:
if self.source_enabled is not None and not self.source_enabled(job.source_id):
if self.lane_of(job) in self.adapters:
if self.source_enabled is not None and not self.source_enabled(
self.lane_of(job)
):
continue
if job.status is not ImportStatus.PENDING:
self.job_store.queue_for_resume(job.job_id)
@@ -227,7 +239,7 @@ class ImportSupervisor:
}:
return
if self.source_enabled is not None and not await in_the_background(
self.source_enabled, job.source_id
self.source_enabled, self.lane_of(job)
):
return
runner = ImportRunner(
@@ -238,7 +250,7 @@ class ImportSupervisor:
sdk_queue=self.sdk_queue,
sdk_authority=self.sdk_authority,
)
adapter = self.adapters[job.source_id]
adapter = self.adapters[self.lane_of(job)]
if self.prepare_adapter is not None:
adapter = await self.prepare_adapter(job, adapter)
@@ -273,7 +285,7 @@ class ImportSupervisor:
and retry_attempt < self.max_history_retries
and (
self.source_enabled is None
or await in_the_background(self.source_enabled, current.source_id)
or await in_the_background(self.source_enabled, self.lane_of(current))
)
):
delay = self.retry_base_seconds * (2**retry_attempt)
@@ -7,9 +7,9 @@ between pages. The matches are kept here under an opaque ``window_id`` that
the client passes back with the next offset.
A window is bound to the filters it was searched with (query, source,
statuses, scene). It expires after ``ttl_seconds`` without being read, and the
store keeps at most ``max_windows`` windows, dropping the least recently read
first.
statuses, scene) and to the user whose library it searched. It expires after
``ttl_seconds`` without being read, and the store keeps at most
``max_windows`` windows, dropping the least recently read first.
Only memory identities are kept, never content: each page is read fresh from
the SDK, so an edit or a delete shows on the next page read.
"""
@@ -28,13 +28,24 @@ from one_memory.hy_sdk_types import LibraryHit
DEFAULT_WINDOW_TTL_SECONDS = 300.0
DEFAULT_MAX_WINDOWS = 32
WindowKey = tuple[str, str | None, tuple[str, ...], str]
#: (query, source, statuses, scene, user, other filters as sorted name/values pairs)
WindowKey = tuple[
str, str | None, tuple[str, ...], str, str, tuple[tuple[str, tuple[str, ...]], ...]
]
def window_key(
query: str, source: str | None, statuses: list[str], scene: str = "all"
query: str,
source: str | None,
statuses: list[str],
scene: str = "all",
user_id: str = "local",
filters: dict[str, list[str]] | None = None,
) -> WindowKey:
return (query, source, tuple(sorted(set(statuses))), scene)
extra = tuple(
sorted((name, tuple(sorted(set(values)))) for name, values in (filters or {}).items())
)
return (query, source, tuple(sorted(set(statuses))), scene, user_id, extra)
@dataclass(frozen=True)
@@ -11,10 +11,11 @@ import asyncio
import sqlite3
from collections.abc import Sequence
from datetime import UTC, datetime
from functools import partial
from typing import Any, Literal, cast
from uuid import UUID
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request
from fastapi import APIRouter, BackgroundTasks, HTTPException, Query, Request
from fastapi.params import Depends
from pydantic import BaseModel, ConfigDict, Field, field_validator
@@ -34,6 +35,7 @@ from one_memory.memory_server.durable_queue import DurableWriteQueue
from one_memory.memory_server.unified_entry import UnifiedWriteIntake
from one_memory.memory_tasks import MemoryTaskStatus, MemoryTaskStore
from one_memory.secrets import SecretStoreUnavailableError
from one_memory.user_scope import DEFAULT_USER_ID, USER_ID_PATTERN, UserId
SOURCE_ID = "one-memory-manual"
MemoryAddState = Literal[
@@ -53,6 +55,8 @@ class MemoryAddInput(BaseModel):
request_id: UUID
text: str = Field(min_length=1, max_length=12_000)
scope: Literal["all-apps"]
#: Whose memory this is. The App never sends it and writes `local`.
user_id: UserId = DEFAULT_USER_ID
@field_validator("text")
@classmethod
@@ -108,7 +112,11 @@ class MemoryAdditionStore:
def save(self, addition: MemoryAddInput) -> EvidenceRecord:
now = datetime.now(UTC)
if addition.user_id != DEFAULT_USER_ID and not self.sdk_authority:
# Legacy authority keeps one person's library only.
raise ValueError("memory_add_user_requires_sdk_authority")
value = EvidenceInput(
user_id=addition.user_id,
source_id=SOURCE_ID,
conversation_id=f"note:{addition.request_id}",
message_id=str(addition.request_id),
@@ -200,7 +208,7 @@ class MemoryAdditionStore:
)
return self.require(value.evidence_id)
def require(self, evidence_id: str) -> EvidenceRecord:
def require(self, evidence_id: str, *, user_id: str = DEFAULT_USER_ID) -> EvidenceRecord:
if self.sdk_authority:
if self.write_queue is None:
raise KeyError(evidence_id)
@@ -219,11 +227,11 @@ class MemoryAdditionStore:
(item for item in payload.input_records if item.evidence_id == evidence_id),
None,
)
if record is None:
if record is None or record.user_id != user_id:
raise KeyError(evidence_id)
return record
record = self.evidence.get(evidence_id)
if record is None or record.user_id != "local" or record.source_id != SOURCE_ID:
if record is None or record.user_id != user_id or record.source_id != SOURCE_ID:
raise KeyError(evidence_id)
metadata = record.origin_meta.get(EvidenceOrigin.MANUAL_IMPORT)
if metadata is None or metadata.raw_meta.get("input_mode") != "direct_memory_note":
@@ -237,8 +245,8 @@ class MemoryAdditionStore:
raise ValueError("memory_add_source_revoked")
return record
def receipt(self, evidence_id: str) -> MemoryAddReceipt:
record = self.require(evidence_id)
def receipt(self, evidence_id: str, *, user_id: str = DEFAULT_USER_ID) -> MemoryAddReceipt:
record = self.require(evidence_id, user_id=user_id)
if self.sdk_authority:
assert self.write_queue is not None
with main_database(self.write_queue.database_path).read() as database:
@@ -300,10 +308,15 @@ class MemoryAdditionStore:
error=error,
)
def receipt_for_request(self, request_id: UUID) -> MemoryAddReceipt:
def receipt_for_request(
self, request_id: UUID, *, user_id: str = DEFAULT_USER_ID
) -> MemoryAddReceipt:
identity = str(request_id)
if self.sdk_authority:
# The Evidence id carries the user, so another user's request id
# names a different note and reads as not found.
probe = EvidenceInput(
user_id=user_id,
source_id=SOURCE_ID,
conversation_id=f"note:{identity}",
message_id=identity,
@@ -316,16 +329,16 @@ class MemoryAdditionStore:
)
},
)
return self.receipt(probe.evidence_id)
return self.receipt(probe.evidence_id, user_id=user_id)
with main_database(self.evidence.database_path).read() as db:
row = db.execute(
"SELECT evidence_id FROM evidence WHERE user_id='local' AND source_id=? "
"SELECT evidence_id FROM evidence WHERE user_id=? AND source_id=? "
"AND conversation_id=? AND message_id=?",
(SOURCE_ID, f"note:{identity}", identity),
(user_id, SOURCE_ID, f"note:{identity}", identity),
).fetchone()
if row is None:
raise KeyError(identity)
return self.receipt(str(row[0]))
return self.receipt(str(row[0]), user_id=user_id)
def memory_add_router(dependencies: Sequence[Depends]) -> APIRouter:
@@ -376,14 +389,20 @@ def memory_add_router(dependencies: Sequence[Depends]) -> APIRouter:
record = await asyncio.to_thread(additions.save, addition)
except ValueError as error:
raise HTTPException(409, str(error)) from error
saved = await asyncio.to_thread(additions.receipt, record.evidence_id)
saved = await asyncio.to_thread(
partial(additions.receipt, record.evidence_id, user_id=addition.user_id)
)
background.add_task(schedule, additions, record)
return saved
@router.get("/additions/{request_id}", response_model=MemoryAddReceipt)
def memory_add_status(request_id: UUID, request: Request) -> MemoryAddReceipt:
def memory_add_status(
request_id: UUID,
request: Request,
user_id: str = Query(default=DEFAULT_USER_ID, pattern=USER_ID_PATTERN),
) -> MemoryAddReceipt:
try:
return store(request).receipt_for_request(request_id)
return store(request).receipt_for_request(request_id, user_id=user_id)
except KeyError as error:
raise HTTPException(404, "memory_add_not_found") from error
except ValueError as error:
@@ -0,0 +1,618 @@
"""System 2 for ultra: evolve runs, their receipts, the nightly schedule, graph backfill.
hy-memory 2.0.9's `client.evolve(user_id, agent_id, max_batches)` is ultra
only, holds no lock, and has no scheduler: MemHub owns all three.
- A run is asynchronous: `start` answers a receipt at once and the run goes
on in the background (evolve makes model calls across the user's whole
library and can take minutes, longer than any HTTP timeout). One run per
user at a time: asking again while one runs returns that run. A run holds
the write coordinator's per-user exclusivity, so it never overlaps that
user's write batch, and loops while the SDK answers `should_continue`, up
to `EVOLVE_MAX_ROUNDS`.
- The schedule runs every user that has memories once a day at the
configured local time, and catches up after the machine slept through it.
- Switching to ultra backfills each user's existing facts into the graph
once (`build_memory_graph`).
What is kept on disk is counts and stable codes only, never memory text:
`<data_dir>/evolve-state.json`.
"""
from __future__ import annotations
import asyncio
import json
import logging
import os
import re
import threading
from collections.abc import Awaitable, Callable, Iterable, Mapping
from datetime import UTC, datetime, timedelta
from pathlib import Path
from tempfile import NamedTemporaryFile
from typing import Any, Literal, Protocol
from uuid import uuid4
from pydantic import BaseModel, ConfigDict, Field
from one_memory.diagnostic_log import diagnostic_event
from one_memory.memory_server.config_state import EvolveSchedule
from one_memory.memory_server.durable_queue import safe_error_code
from one_memory.sdk_write_processor import PRODUCT_AGENT_ID
logger = logging.getLogger(__name__)
#: Rounds one run makes while the SDK says there is more to do.
EVOLVE_MAX_ROUNDS = 20
#: How often the schedule looks at the clock; also bounds catch-up after wake.
SCHEDULE_CHECK_SECONDS = 300.0
#: Receipts kept in memory for `GET /api/v2/evolve/{run_id}`.
_KEPT_RUNS = 64
_STABLE_CODE = re.compile(r"[a-z][a-z0-9_]{0,95}")
_COUNTS = (
"remaining_clusters",
"remaining_actionable_clusters",
"remaining_fresh_facts",
)
RunState = Literal["running", "succeeded", "failed"]
Trigger = Literal["manual", "scheduled"]
class EvolveModeError(RuntimeError):
"""Evolve asked for outside ultra."""
def __init__(self) -> None:
super().__init__("memory_mode_not_ultra")
class EvolveRunReceipt(BaseModel):
"""One evolve run. Counts and codes; `results` only while in memory."""
model_config = ConfigDict(extra="forbid")
run_id: str
user_id: str
agent_id: str
max_batches: int | None = None
trigger: Trigger = "manual"
state: RunState = "running"
started_at: datetime
finished_at: datetime | None = None
rounds: int = 0
tasks_processed: int = 0
batch_failures: int = 0
cycle_status: str | None = None
should_continue: bool | None = None
continuation_reason: str | None = None
remaining_clusters: int | None = None
remaining_actionable_clusters: int | None = None
remaining_fresh_facts: int | None = None
#: True when the round cap stopped a run the SDK would have continued.
capped: bool = False
elapsed_ms: float = 0.0
error_code: str | None = None
failure_kind: str | None = None
#: The last round's SDK `request_id`, for matching SDK logs.
request_id: str | None = None
#: The last round's SDK `results` and `cross_domain_sweeper`. Kept only
#: in this receipt, never written to disk or diagnostics.
results: Any = None
cross_domain_sweeper: Any = None
def summary(self) -> dict[str, object]:
"""What is persisted and shown in status: no SDK result bodies."""
return self.model_dump(mode="json", exclude={"results", "cross_domain_sweeper"})
class UserEvolveState(BaseModel):
model_config = ConfigDict(extra="forbid")
last_run_at: datetime | None = None
last_result: dict[str, object] | None = None
last_error: str | None = None
class GraphBuildUser(BaseModel):
model_config = ConfigDict(extra="forbid")
state: Literal["pending", "running", "succeeded", "failed"] = "pending"
memories_scanned: int | None = None
nodes: int | None = None
error_code: str | None = None
class GraphBuildState(BaseModel):
"""The one-time backfill of existing facts into the graph after → ultra."""
model_config = ConfigDict(extra="forbid")
state: Literal["idle", "running", "succeeded", "failed"] = "idle"
started_at: datetime | None = None
finished_at: datetime | None = None
users: dict[str, GraphBuildUser] = Field(default_factory=dict)
class EvolveStateDocument(BaseModel):
model_config = ConfigDict(extra="forbid")
schema_version: int = Field(default=1, ge=1, le=1)
users: dict[str, UserEvolveState] = Field(default_factory=dict)
#: When the schedule first ran in ultra: the "last run" of a user that
#: never ran, so enabling ultra at noon does not evolve everyone at once.
schedule_anchor: datetime | None = None
graph_build: GraphBuildState = Field(default_factory=GraphBuildState)
class EvolveStateStore:
"""The only writer of `<data_dir>/evolve-state.json`."""
def __init__(self, data_dir: Path) -> None:
self.path = data_dir / "evolve-state.json"
self._lock = threading.RLock()
self._document = self._read()
def document(self) -> EvolveStateDocument:
with self._lock:
return self._document.model_copy(deep=True)
def user(self, user_id: str) -> UserEvolveState:
with self._lock:
return self._document.users.get(user_id, UserEvolveState()).model_copy()
def record_run(self, receipt: EvolveRunReceipt) -> None:
with self._lock:
users = dict(self._document.users)
users[receipt.user_id] = UserEvolveState(
last_run_at=receipt.started_at,
last_result=receipt.summary(),
last_error=receipt.error_code,
)
self._save(self._document.model_copy(update={"users": users}))
def anchor(self, now: datetime) -> datetime:
"""The schedule anchor, set to `now` the first time it is asked."""
with self._lock:
if self._document.schedule_anchor is None:
self._save(self._document.model_copy(update={"schedule_anchor": now}))
assert self._document.schedule_anchor is not None
return self._document.schedule_anchor
def graph_build(self) -> GraphBuildState:
with self._lock:
return self._document.graph_build.model_copy(deep=True)
def set_graph_build(self, state: GraphBuildState) -> None:
with self._lock:
self._save(self._document.model_copy(update={"graph_build": state}))
def _save(self, document: EvolveStateDocument) -> None:
payload = json.dumps(document.model_dump(mode="json"), indent=2, sort_keys=True) + "\n"
self.path.parent.mkdir(parents=True, exist_ok=True)
temporary: Path | None = None
try:
with NamedTemporaryFile(
mode="w",
encoding="utf-8",
dir=self.path.parent,
prefix=".evolve-state.",
suffix=".tmp",
delete=False,
) as stream:
temporary = Path(stream.name)
stream.write(payload)
stream.flush()
os.fsync(stream.fileno())
os.chmod(temporary, 0o600)
os.replace(temporary, self.path)
except OSError:
if temporary is not None:
temporary.unlink(missing_ok=True)
# Status is best-effort: a run must not fail because its record did.
logger.warning("evolve_state_write_failed")
self._document = document
def _read(self) -> EvolveStateDocument:
try:
return EvolveStateDocument.model_validate_json(self.path.read_text(encoding="utf-8"))
except FileNotFoundError:
return EvolveStateDocument()
except (OSError, ValueError):
# A damaged status file only loses history; start a fresh one.
logger.warning("evolve_state_unreadable")
return EvolveStateDocument()
class EvolveCoordinator(Protocol):
"""The write coordinator surface evolve and the graph backfill use."""
def exclusive_for_evolve(self, user_id: str) -> Any: ...
def use_client(self) -> Any: ...
class EvolveService:
"""Start, serialize and record evolve runs."""
def __init__(
self,
*,
coordinator: EvolveCoordinator,
state: EvolveStateStore,
memory_mode: Callable[[], str],
max_rounds: int = EVOLVE_MAX_ROUNDS,
now: Callable[[], datetime] = lambda: datetime.now(UTC),
) -> None:
self.coordinator = coordinator
self.state = state
self.memory_mode = memory_mode
self.max_rounds = max_rounds
self._now = now
self._runs: dict[str, EvolveRunReceipt] = {}
self._tasks: dict[str, asyncio.Task[None]] = {}
self._active: dict[str, str] = {}
def start(
self,
user_id: str,
*,
agent_id: str | None = None,
max_batches: int | None = None,
trigger: Trigger = "manual",
) -> EvolveRunReceipt:
"""Start a run, or return the one already running for this user."""
if self.memory_mode() != "ultra":
raise EvolveModeError()
running = self._active.get(user_id)
if running is not None:
return self._runs[running]
receipt = EvolveRunReceipt(
run_id=str(uuid4()),
user_id=user_id,
agent_id=agent_id or PRODUCT_AGENT_ID,
max_batches=max_batches,
trigger=trigger,
started_at=self._now(),
)
self._remember(receipt)
self._active[user_id] = receipt.run_id
self._tasks[receipt.run_id] = asyncio.create_task(
self._run(receipt.run_id), name="one-memory-evolve"
)
return receipt
def get(self, run_id: str) -> EvolveRunReceipt | None:
return self._runs.get(run_id)
def running(self) -> tuple[EvolveRunReceipt, ...]:
return tuple(self._runs[run_id] for run_id in self._active.values())
async def wait(self, run_id: str) -> EvolveRunReceipt | None:
task = self._tasks.get(run_id)
if task is not None:
await asyncio.shield(task)
return self._runs.get(run_id)
async def shutdown(self) -> None:
tasks = tuple(self._tasks.values())
for task in tasks:
task.cancel()
await asyncio.gather(*tasks, return_exceptions=True)
self._tasks.clear()
def _remember(self, receipt: EvolveRunReceipt) -> None:
self._runs[receipt.run_id] = receipt
while len(self._runs) > _KEPT_RUNS:
oldest = next(iter(self._runs))
if oldest in self._active.values():
break
del self._runs[oldest]
async def _run(self, run_id: str) -> None:
receipt = self._runs[run_id]
diagnostic_event(
level="info",
component="memory.evolve",
event="memory_evolve.started",
outcome="started",
details={"trigger": receipt.trigger, "max_batches": receipt.max_batches},
)
try:
async with (
self.coordinator.exclusive_for_evolve(receipt.user_id),
self.coordinator.use_client() as client,
):
while receipt.rounds < self.max_rounds:
result = await asyncio.to_thread(
client.evolve,
receipt.user_id,
agent_id=receipt.agent_id,
max_batches=receipt.max_batches,
)
receipt = _folded(receipt, result)
self._runs[run_id] = receipt
if receipt.state == "failed" or not receipt.should_continue:
break
else:
receipt = receipt.model_copy(update={"capped": True})
if receipt.state == "running":
receipt = receipt.model_copy(update={"state": "succeeded"})
except asyncio.CancelledError:
receipt = receipt.model_copy(
update={"state": "failed", "error_code": "memory_evolve_cancelled"}
)
raise
except Exception as error: # noqa: BLE001 - recorded as a stable code
code = safe_error_code(error)
receipt = receipt.model_copy(
update={
"state": "failed",
"error_code": _stable(code, "hy_memory_evolve_failed"),
"failure_kind": type(error).__name__,
}
)
finally:
receipt = receipt.model_copy(update={"finished_at": self._now()})
self._runs[run_id] = receipt
self._active.pop(receipt.user_id, None)
self._tasks.pop(run_id, None)
await asyncio.to_thread(self.state.record_run, receipt)
diagnostic_event(
level="info" if receipt.state == "succeeded" else "warning",
component="memory.evolve",
event="memory_evolve.finished",
outcome="succeeded" if receipt.state == "succeeded" else "failed",
details={
"trigger": receipt.trigger,
"rounds": receipt.rounds,
"tasks_processed": receipt.tasks_processed,
"batch_failures": receipt.batch_failures,
"capped": receipt.capped,
"elapsed_ms": round(receipt.elapsed_ms, 1),
"error_code": receipt.error_code,
},
)
def _folded(receipt: EvolveRunReceipt, result: object) -> EvolveRunReceipt:
"""One more evolve round's result added to the run."""
if not isinstance(result, Mapping):
return receipt.model_copy(
update={"state": "failed", "error_code": "hy_memory_evolve_shape_unknown"}
)
failures = result.get("batch_failures")
update: dict[str, object] = {
"rounds": receipt.rounds + 1,
"tasks_processed": receipt.tasks_processed + _int(result.get("tasks_processed")),
"batch_failures": receipt.batch_failures
+ (len(failures) if isinstance(failures, list | tuple) else _int(failures)),
"elapsed_ms": receipt.elapsed_ms + _float(result.get("elapsed_ms")),
"should_continue": result.get("should_continue") is True,
"cycle_status": _text(result.get("cycle_status")),
"continuation_reason": _text(result.get("continuation_reason")),
"request_id": _text(result.get("request_id")),
"results": result.get("results"),
"cross_domain_sweeper": result.get("cross_domain_sweeper"),
}
for name in _COUNTS:
value = result.get(name)
update[name] = value if isinstance(value, int) and not isinstance(value, bool) else None
if result.get("success") is not True:
code = result.get("error_code")
kind = result.get("failure_kind")
update.update(
state="failed",
should_continue=False,
error_code=(
code
if isinstance(code, str) and _STABLE_CODE.fullmatch(code)
else "hy_memory_evolve_rejected"
),
failure_kind=kind if isinstance(kind, str) and _STABLE_CODE.fullmatch(kind) else None,
)
return receipt.model_copy(update=update)
def _stable(code: str, fallback: str) -> str:
return code if _STABLE_CODE.fullmatch(code) else fallback
def _int(value: object) -> int:
return value if isinstance(value, int) and not isinstance(value, bool) else 0
def _float(value: object) -> float:
if isinstance(value, int | float) and not isinstance(value, bool):
return float(value)
return 0.0
def _text(value: object) -> str | None:
return value[:128] if isinstance(value, str) else None
def evolve_due(
now: datetime,
schedule: EvolveSchedule,
last_run_at: datetime | None,
anchor: datetime,
) -> bool:
"""Whether a user's nightly evolve is due at `now` (aware, local time).
Due once the day's slot (`schedule.time`, local) has passed and the last
run (or, for a user who never ran, the anchor) is older than that slot.
A last run more than 24 hours old is due at once as well: a machine asleep
at the slot catches up when it wakes, even before the next day's slot.
"""
if not schedule.enabled:
return False
local = now.astimezone()
hour, minute = (int(part) for part in schedule.time.split(":"))
slot = local.replace(hour=hour, minute=minute, second=0, microsecond=0)
reference = last_run_at or anchor
if local >= slot and reference < slot:
return True
return last_run_at is not None and local - last_run_at > timedelta(hours=24)
class EvolveScheduler:
"""Run evolve nightly for every user that has memories (ultra only)."""
def __init__(
self,
*,
service: EvolveService,
schedule: Callable[[], EvolveSchedule],
users_with_memories: Callable[[], Awaitable[Iterable[str]]],
now: Callable[[], datetime] = lambda: datetime.now().astimezone(),
check_seconds: float = SCHEDULE_CHECK_SECONDS,
) -> None:
self.service = service
self.schedule = schedule
self.users_with_memories = users_with_memories
self._now = now
self.check_seconds = check_seconds
self._task: asyncio.Task[None] | None = None
def start(self) -> None:
if self._task is None or self._task.done():
self._task = asyncio.create_task(self._loop(), name="one-memory-evolve-schedule")
async def tick(self) -> tuple[str, ...]:
"""Run every due user once, one after another; the users run."""
schedule = self.schedule()
if not schedule.enabled or self.service.memory_mode() != "ultra":
return ()
now = self._now()
anchor = await asyncio.to_thread(self.service.state.anchor, now)
ran: list[str] = []
for user_id in await self.users_with_memories():
last = self.service.state.user(user_id).last_run_at
if not evolve_due(now, schedule, last, anchor):
continue
try:
receipt = self.service.start(user_id, trigger="scheduled")
except EvolveModeError:
break
await self.service.wait(receipt.run_id)
ran.append(user_id)
if ran:
diagnostic_event(
level="info",
component="memory.evolve",
event="memory_evolve.schedule_ran",
outcome="succeeded",
details={"users": len(ran)},
)
return tuple(ran)
async def _loop(self) -> None:
while True:
try:
await self.tick()
except asyncio.CancelledError:
raise
except Exception as error: # noqa: BLE001 - the loop must survive
logger.warning("evolve_schedule_tick_failed error_type=%s", type(error).__name__)
await asyncio.sleep(self.check_seconds)
async def shutdown(self) -> None:
if self._task is None:
return
self._task.cancel()
await asyncio.gather(self._task, return_exceptions=True)
self._task = None
class MemoryGraphBackfill:
"""After a switch to ultra, project each user's existing facts into the graph once."""
def __init__(
self,
*,
coordinator: EvolveCoordinator,
state: EvolveStateStore,
now: Callable[[], datetime] = lambda: datetime.now(UTC),
) -> None:
self.coordinator = coordinator
self.state = state
self._now = now
self._task: asyncio.Task[None] | None = None
@property
def running(self) -> bool:
return self._task is not None and not self._task.done()
def start(self, user_ids: Iterable[str]) -> GraphBuildState:
if self.running:
return self.state.graph_build()
users = tuple(dict.fromkeys(user_ids))
state = GraphBuildState(
state="running",
started_at=self._now(),
users={user_id: GraphBuildUser() for user_id in users},
)
self.state.set_graph_build(state)
self._task = asyncio.create_task(self._run(users), name="one-memory-graph-backfill")
return state
async def wait(self) -> None:
if self._task is not None:
await asyncio.shield(self._task)
async def _run(self, users: tuple[str, ...]) -> None:
state = self.state.graph_build()
for user_id in users:
state.users[user_id] = GraphBuildUser(state="running")
await asyncio.to_thread(self.state.set_graph_build, state.model_copy(deep=True))
try:
async with (
self.coordinator.exclusive_for_evolve(user_id),
self.coordinator.use_client() as client,
):
result = await asyncio.to_thread(
client.build_memory_graph,
user_id,
agent_id=None,
limit=10000,
clear_first=False,
)
ok = isinstance(result, Mapping) and result.get("success") is True
state.users[user_id] = GraphBuildUser(
state="succeeded" if ok else "failed",
memories_scanned=_int(result.get("memories_scanned"))
if isinstance(result, Mapping)
else None,
nodes=_int(result.get("nodes")) if isinstance(result, Mapping) else None,
error_code=None if ok else "hy_memory_graph_build_rejected",
)
except asyncio.CancelledError:
raise
except Exception as error: # noqa: BLE001 - recorded as a stable code
code = safe_error_code(error)
state.users[user_id] = GraphBuildUser(
state="failed",
error_code=_stable(code, "hy_memory_graph_build_failed"),
)
failed = any(user.state == "failed" for user in state.users.values())
state.state = "failed" if failed else "succeeded"
state.finished_at = self._now()
await asyncio.to_thread(self.state.set_graph_build, state)
diagnostic_event(
level="warning" if failed else "info",
component="memory.evolve",
event="memory_graph_backfill.finished",
outcome="failed" if failed else "succeeded",
details={
"users": len(state.users),
"failed_users": sum(user.state == "failed" for user in state.users.values()),
},
)
async def shutdown(self) -> None:
if self._task is None:
return
self._task.cancel()
await asyncio.gather(self._task, return_exceptions=True)
self._task = None
@@ -13,6 +13,7 @@ import threading
from collections.abc import Mapping
from pathlib import Path
from tempfile import NamedTemporaryFile
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field, StrictBool, field_validator
@@ -27,6 +28,31 @@ class SourceConfig(BaseModel):
sync_enabled: StrictBool = False
#: The SDK pipeline mode (hy-memory `HyMemoryClient(mode=...)`):
#: lite keeps raw conversations only, with no model call; pro extracts
#: memories (System 1); ultra adds the knowledge graph and evolve (System 2).
MemoryMode = Literal["lite", "pro", "ultra"]
MEMORY_MODES: tuple[MemoryMode, ...] = ("lite", "pro", "ultra")
_SCHEDULE_TIME = re.compile(r"^(?:[01][0-9]|2[0-3]):[0-5][0-9]$")
class EvolveSchedule(BaseModel):
"""When ultra runs evolve for every user: once a day, local time."""
model_config = ConfigDict(frozen=True, extra="forbid", strict=True)
enabled: StrictBool = True
#: `HH:MM`, 24-hour, the machine's local time.
time: str = "03:00"
@field_validator("time")
@classmethod
def valid_time(cls, value: str) -> str:
if _SCHEDULE_TIME.fullmatch(value) is None:
raise ValueError("invalid_evolve_schedule_time")
return value
class ConfigDocument(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid", strict=True)
@@ -39,6 +65,10 @@ class ConfigDocument(BaseModel):
#: needs no re-entry of a key.
sedimentation_enabled: StrictBool = True
extraction_window_preset: ExtractionWindowPreset = ExtractionWindowPreset.THREE_MONTHS
#: The SDK mode every client is built with. `ONE_MEMORY_GRAPH_STORE` (dev)
#: still forces ultra over it; see `hy_graph_admin.effective_memory_mode`.
memory_mode: MemoryMode = "pro"
evolve_schedule: EvolveSchedule = Field(default_factory=EvolveSchedule)
@field_validator("sources")
@classmethod
@@ -61,6 +91,9 @@ class ConfigFileError(RuntimeError):
pass
_WRITTEN_WHEN_SET = frozenset({"memory_mode", "evolve_schedule"})
class ConfigStore:
"""The only writer of ``<data_dir>/config.json`` inside Core."""
@@ -107,6 +140,32 @@ class ConfigStore:
self._document = updated
return updated
def memory_mode(self) -> MemoryMode:
with self._lock:
return self._document.memory_mode
def set_memory_mode(self, mode: MemoryMode) -> ConfigDocument:
if mode not in MEMORY_MODES:
raise ValueError("invalid_memory_mode")
with self._lock:
updated = self._document.model_copy(update={"memory_mode": mode})
self._write(updated)
self._document = updated
return updated
def evolve_schedule(self) -> EvolveSchedule:
with self._lock:
return self._document.evolve_schedule
def set_evolve_schedule(self, schedule: EvolveSchedule) -> ConfigDocument:
if not isinstance(schedule, EvolveSchedule):
raise ValueError("invalid_evolve_schedule")
with self._lock:
updated = self._document.model_copy(update={"evolve_schedule": schedule})
self._write(updated)
self._document = updated
return updated
def migrate_extraction_window_preset(self, preset: ExtractionWindowPreset) -> ConfigDocument:
"""Adopt the legacy SQL value only when config.json has no value."""
if not isinstance(preset, ExtractionWindowPreset):
@@ -152,9 +211,12 @@ class ConfigStore:
raise ConfigFileError("config_invalid") from error
def _write(self, document: ConfigDocument) -> None:
# Settings added after 1.0.5 are written only once a person chose
# them, so an untouched install's file stays what older builds read.
unset = {name for name in _WRITTEN_WHEN_SET if name not in document.model_fields_set}
payload = (
json.dumps(
document.model_dump(mode="json"),
document.model_dump(mode="json", exclude=unset),
ensure_ascii=False,
indent=2,
sort_keys=True,
@@ -75,7 +75,11 @@ def batch_id(*, internal_session_id: str, first_round: int, last_round: int) ->
INTERNAL_SESSION_ID_MAX = 128
def internal_session_id_for(conversation_id: str, source_id: str) -> str:
def internal_session_id_for(
conversation_id: str,
source_id: str,
user_id: str = "local",
) -> str:
"""Normalise a conversation identity to the durable ``internal_session_id``.
This is the single derivation shared by every Evidence-first intake path.
@@ -86,7 +90,17 @@ def internal_session_id_for(conversation_id: str, source_id: str) -> str:
per-session serialisation split). Only a value that the contract would have
rejected before it ever reached the queue is replaced by the versioned
session id, which is deterministic for a given (source, conversation).
Another user's conversation is always derived with the user in it: two
Hermes profiles can hold the same session id, and sharing a session key
would serialise and batch one person's turns behind the other's. `local`
keeps the identity it always had.
"""
if user_id != "local":
return session_id(
harness_id=source_id,
external_session_id=f"user:{user_id}|{conversation_id}",
)
if len(conversation_id) <= INTERNAL_SESSION_ID_MAX:
return conversation_id
derived = session_id(harness_id=source_id, external_session_id=conversation_id)
@@ -54,6 +54,7 @@ from one_memory.memory_tasks import (
)
from one_memory.sdk_migration_log import error_chain_text
from one_memory.service_levels import ServiceLevel, service_level_context
from one_memory.user_scope import DEFAULT_USER_ID, lane_key, other_user_lane_prefix
logger = logging.getLogger(__name__)
@@ -345,6 +346,27 @@ EVENT_TIME_WINDOW = timedelta(minutes=10)
DIRECT_WRITE_SOURCES = frozenset({"one-memory-manual"})
def request_user_id(request: WriteRequest) -> str:
"""Whose write this is: its frozen input's user, `local` for id-only rows."""
users = {record.user_id for record in request.input_records}
if len(users) > 1:
raise ValueError("durable_write_multiple_users")
return next(iter(users), DEFAULT_USER_ID)
def request_source_key(request: WriteRequest) -> str:
"""The FIFO/claim key of a write: its source, per user.
`local` keeps the bare source id it always had. Another user's Hermes
lane is `hermes@user`, so one person's running job never holds the other
person's queue and the claim gate answers for that user's grant.
"""
return lane_key(
request_user_id(request),
request.source_id or request.internal_session_id,
)
@dataclass(frozen=True)
class DurableWriteRecord:
job_id: str
@@ -862,7 +884,7 @@ class DurableWriteQueue:
job_id,
queue_seq,
request.internal_session_id,
request.source_id or request.internal_session_id,
request_source_key(request),
request.batch_identity.batch_id,
key,
payload,
@@ -929,7 +951,9 @@ class DurableWriteQueue:
f"{hashlib.sha256(file_key.encode()).hexdigest()}:"
f"{hashlib.sha256(record.conversation_id.encode()).hexdigest()}"
)
session_key = internal_session_id_for(record.conversation_id, record.source_id)
session_key = internal_session_id_for(
record.conversation_id, record.source_id, record.user_id
)
written = connection.execute(
"SELECT 1 FROM memory_write_jobs AS job, json_each(job.evidence_ids_json) AS ref "
"WHERE job.session_key=? AND job.state!='assembling' AND ref.value=? LIMIT 1",
@@ -998,7 +1022,7 @@ class DurableWriteQueue:
str(uuid4()),
queue_seq,
session_key,
record.source_id,
lane_key(record.user_id, record.source_id),
assembly_key,
assembly_key,
payload_json,
@@ -1034,10 +1058,13 @@ class DurableWriteQueue:
`flush` (history at end of file) completes the last turn and writes a
short batch; without it (live polling) those stay in the row.
"""
# Two users' roots can hold a file with the same relative key (each
# Hermes profile has its own `state.db`): only this user's rows.
assembly_prefix = f"assembly:{user_id}:{source_id}:"
rows = connection.execute(
"SELECT * FROM memory_write_jobs WHERE state='assembling' "
"AND source_key=? ORDER BY queue_seq",
(source_id,),
"AND source_key=? AND substr(idempotency_key,1,?)=? ORDER BY queue_seq",
(lane_key(user_id, source_id), len(assembly_prefix), assembly_prefix),
).fetchall()
finalized: list[str] = []
for row in rows:
@@ -1160,7 +1187,7 @@ class DurableWriteQueue:
Same records, same key: a re-read returns the job already written.
"""
first = records[0]
session_id = internal_session_id_for(first.conversation_id, first.source_id)
session_id = internal_session_id_for(first.conversation_id, first.source_id, first.user_id)
material = ":".join(f"{item.evidence_id}:{item.content_hash}" for item in records)
request_id = f"turn:v2:{hashlib.sha256(material.encode()).hexdigest()}"
existing = connection.execute(
@@ -1205,7 +1232,7 @@ class DurableWriteQueue:
job_id,
queue_seq,
session_id,
first.source_id,
lane_key(first.user_id, first.source_id),
request.batch_identity.batch_id,
request_id,
json.dumps(
@@ -2494,6 +2521,12 @@ async def execute_job(
source_id = request.internal_session_id
if source_ids and source_ids != {source_id}:
raise RuntimeError("durable_write_multiple_sources")
users = {getattr(record, "user_id", DEFAULT_USER_ID) for record in records}
if len(users) > 1:
raise RuntimeError("durable_write_multiple_users")
# The gates answer per user: another user's Hermes job asks about
# that user's grant, not `local`'s.
source_id = lane_key(next(iter(users), DEFAULT_USER_ID), source_id)
async def mark() -> bool:
return await in_the_background(queue.mark_invoked, job_id, owner=owner)
@@ -2530,6 +2563,8 @@ async def execute_job(
in {
"model_content_source_disabled",
"model_sedimentation_disabled",
# Evolve holds this user; the job runs once it is done.
"memory_evolve_in_progress",
}
and not hy_invoked
):
@@ -2799,6 +2834,19 @@ class DurableWriteDrain:
source_clause = f"AND candidate.source_key IN ({placeholders}) "
source_params = tuple(sorted(enabled_sources))
direct_sources = ",".join("?" for _ in DIRECT_WRITE_SOURCES)
# Another user's hand-added memories are `one-memory-manual@user`:
# the same lane ahead of history, still serial per user.
direct_lanes = " OR ".join(
"substr(candidate.source_key,1,?)=?" for _ in DIRECT_WRITE_SOURCES
)
direct_lane_params = tuple(
value
for source in sorted(DIRECT_WRITE_SOURCES)
for value in (
len(other_user_lane_prefix(source)),
other_user_lane_prefix(source),
)
)
# Same eligibility as the window claim below (free lease, no
# running job of its source, nothing earlier in its session), but
# not bound to the window: a note must not wait for history.
@@ -2808,7 +2856,7 @@ class DurableWriteDrain:
"WHERE candidate.state=? "
"AND (candidate.lease_expires_at IS NULL OR candidate.lease_expires_at < ?) "
+ source_clause
+ f"AND candidate.source_key IN ({direct_sources}) "
+ f"AND (candidate.source_key IN ({direct_sources}) OR {direct_lanes}) "
"AND NOT EXISTS ("
"SELECT 1 FROM memory_write_jobs AS held "
"WHERE held.source_key=candidate.source_key AND held.state=? "
@@ -2826,6 +2874,7 @@ class DurableWriteDrain:
now.isoformat(),
*source_params,
*sorted(DIRECT_WRITE_SOURCES),
*direct_lane_params,
RUNNING,
now.isoformat(),
*_FIFO_BLOCKING_STATES,
@@ -19,10 +19,11 @@ does the writing. That separation is D52.
from __future__ import annotations
import hashlib
from datetime import UTC, datetime
from typing import Any, Literal, cast
from fastapi import APIRouter, HTTPException, Request
from fastapi import APIRouter, HTTPException, Query, Request
from pydantic import BaseModel, ConfigDict, Field
from one_memory.imports.models import (
@@ -43,6 +44,24 @@ from one_memory.memory_server.durable_queue import (
)
from one_memory.memory_server.registry import WriteRegistry, WriteState
from one_memory.memory_server.submit import WriteConflictError
from one_memory.user_scope import DEFAULT_USER_ID, USER_ID_PATTERN, UserId
#: Durable keys of another user's writes start with this. A `local` request
#: id never may, so `local` cannot name another user's receipt.
_USER_KEY_PREFIX = "user:"
def durable_request_key(request_id: str, user_id: str) -> str:
"""The queue key of a caller's request id, namespaced per user.
Two users choosing the same request id must not collide: the key is the
queue's idempotency key and the SDK's `request_id`. `local` keeps its own
request id unchanged.
"""
if user_id == DEFAULT_USER_ID:
return request_id
digest = hashlib.sha256(request_id.encode()).hexdigest()
return f"{_USER_KEY_PREFIX}{user_id}:{digest}"
class MemoryWriteReceipt(BaseModel):
@@ -71,6 +90,7 @@ class SdkMemoryWriteInput(BaseModel):
messages: list[OpenAIMessage] = Field(min_length=1, max_length=200)
memory_at: datetime | None = None
priority: Literal["live", "normal"] = "normal"
user_id: UserId = DEFAULT_USER_ID
def _registry(
@@ -124,6 +144,9 @@ def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
) -> MemoryWriteReceipt:
if not any(message.role == "user" for message in payload.messages):
raise HTTPException(422, "messages_require_user_turn")
if payload.user_id == DEFAULT_USER_ID and payload.request_id.startswith(_USER_KEY_PREFIX):
raise HTTPException(422, "request_id_reserved_prefix")
request_key = durable_request_key(payload.request_id, payload.user_id)
observed_at = payload.memory_at or datetime(1970, 1, 1, tzinfo=UTC)
written_by = {
"user": WrittenBy.PERSON,
@@ -133,6 +156,7 @@ def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
records = tuple(
materialize_write_input(
EvidenceInput(
user_id=payload.user_id,
source_id=payload.source_id,
conversation_id=payload.session_id,
message_id=f"{payload.request_id}:{index}",
@@ -157,6 +181,7 @@ def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
internal_session_id = internal_session_id_for(
payload.session_id,
payload.source_id,
payload.user_id,
)
contract = WriteRequest(
internal_session_id=internal_session_id,
@@ -168,7 +193,7 @@ def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
),
evidence_ids=[record.evidence_id for record in records],
input_records=records,
idempotency_key=payload.request_id,
idempotency_key=request_key,
priority=payload.priority,
event_at=payload.memory_at,
time_quality="exact" if payload.memory_at is not None else "approximate",
@@ -178,7 +203,7 @@ def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
except (WriteConflictError, ValueError) as error:
raise HTTPException(409, str(error)) from error
return MemoryWriteReceipt(
request_id=record.request_id,
request_id=payload.request_id,
state=record.state,
created_at=record.created_at,
updated_at=record.updated_at,
@@ -186,16 +211,24 @@ def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
)
@instance.get("/{request_id}", response_model=MemoryWriteReceipt)
def status(request_id: str, request: Request) -> MemoryWriteReceipt:
def status(
request_id: str,
request: Request,
user_id: str = Query(default=DEFAULT_USER_ID, pattern=USER_ID_PATTERN),
) -> MemoryWriteReceipt:
# A receipt belongs to the user that submitted it. Another user's key
# is namespaced, and `local` may not name a namespaced key directly.
if user_id == DEFAULT_USER_ID and request_id.startswith(_USER_KEY_PREFIX):
raise HTTPException(404, "unknown request id")
registry = _registry(request, write=False)
try:
record = registry.get(request_id)
record = registry.get(durable_request_key(request_id, user_id))
except DurableQueueSchemaUnavailableError as error:
raise HTTPException(503, str(error)) from error
if record is None:
raise HTTPException(404, "unknown request id")
return MemoryWriteReceipt(
request_id=record.request_id,
request_id=request_id,
state=record.state,
created_at=record.created_at,
updated_at=record.updated_at,
@@ -271,7 +271,9 @@ def build_request_from_evidence(
# Keep the exact conversation id when it fits the 128-char contract;
# otherwise derive the versioned session id, so a long branch id does
# not reject the whole history import.
internal_session_id=internal_session_id_for(seed.conversation_id, seed.source_id),
internal_session_id=internal_session_id_for(
seed.conversation_id, seed.source_id, seed.user_id
),
first_round=first_round,
last_round=last_round,
evidence_ids=list(ordered),
@@ -28,6 +28,7 @@ from one_memory.memory_server.contracts import (
internal_session_id_for,
)
from one_memory.memory_server.durable_queue import DurableWriteQueue, DurableWriteRecord
from one_memory.user_scope import lane_key
WritePriority = Literal["live", "normal"]
@@ -74,7 +75,7 @@ class UnifiedWriteIntake:
if priority not in {"live", "normal"}:
raise ValueError("priority must be 'live' or 'normal'")
if self.source_writes_permitted is not None and not self.source_writes_permitted(
record.source_id
lane_key(record.user_id, record.source_id)
):
# Freeze is the product's one Harness processing decision. Refuse
# new intake so a frozen source creates no new durable write row;
@@ -106,7 +107,9 @@ class UnifiedWriteIntake:
last_round: int,
priority: WritePriority,
) -> WriteRequest:
session_id = internal_session_id_for(record.conversation_id, record.source_id)
session_id = internal_session_id_for(
record.conversation_id, record.source_id, record.user_id
)
event_at, time_quality = _job_time((record,))
request_key = (
f"entry:{record.evidence_id}"
@@ -145,14 +148,14 @@ class UnifiedWriteIntake:
):
raise ValueError("records must belong to one conversation")
if self.source_writes_permitted is not None and not self.source_writes_permitted(
first.source_id
lane_key(first.user_id, first.source_id)
):
raise SourceWriteDisabledError(first.evidence_id)
material = ":".join(
f"{record.evidence_id}:{record.revision}:{record.content_hash}" for record in records
)
request_id = f"turn:v1:{sha256(material.encode()).hexdigest()}"
session_id = internal_session_id_for(first.conversation_id, first.source_id)
session_id = internal_session_id_for(first.conversation_id, first.source_id, first.user_id)
event_at, time_quality = _job_time(records)
durable = self.queue.submit(
WriteRequest(
@@ -160,7 +160,7 @@ def pending(db: sqlite3.Connection, *, limit: int = 100) -> list[str]:
def _trigger_sql() -> list[tuple[str, str]]:
"""(table it depends on, statement) for each trigger."""
"""(table it is on, statement) for each trigger."""
return [
(
"evidence",
@@ -303,9 +303,61 @@ def install_triggers(db: sqlite3.Connection) -> None:
for table, statement in _trigger_sql():
if table not in present:
continue
if not _body_tables(statement) <= present:
# SQLite resolves a trigger body only when it fires, so a body
# naming a dropped table fails the write it rides on (#233: the
# SDK cutover drops `governed_memory_evidence`, and every grant
# write then failed). Not installed, and an old copy is removed.
cursor.execute(f"DROP TRIGGER IF EXISTS {_trigger_name(statement)}")
continue
cursor.execute(statement)
def _trigger_name(statement: str) -> str:
return statement.split("IF NOT EXISTS", 1)[1].split()[0]
def _body_tables(statement: str) -> set[str]:
"""Tables the trigger body reads or writes besides the one it is on."""
_head, _begin, body = statement.partition("BEGIN")
words = body.replace("(", " ").replace(")", " ").split()
return {
name
for keyword, name in zip(words, words[1:], strict=False)
if keyword.upper() in {"FROM", "JOIN", "INTO"}
}
def drop_stale_triggers(database_path: Path) -> tuple[str, ...]:
"""Drop this module's triggers whose body names a table that is gone.
Startup on SDK authority does not install the cache (it is read only on
legacy authority), but a home migrated by an earlier build still carries
the triggers it installed. One whose body reads a dropped table fails
every write to the table it is on, so it is dropped here. Idempotent.
"""
with main_database(database_path).write() as db, db:
cursor = db.cursor()
cursor.row_factory = sqlite3.Row
present = {
str(row["name"])
for row in cursor.execute("SELECT name FROM sqlite_master WHERE type='table'")
}
installed = {
str(row["name"])
for row in cursor.execute("SELECT name FROM sqlite_master WHERE type='trigger'")
}
dropped = tuple(
name
for _table, statement in _trigger_sql()
if (name := _trigger_name(statement)) in installed
and not _body_tables(statement) <= present
)
for name in dropped:
cursor.execute(f"DROP TRIGGER IF EXISTS {name}")
return dropped
def startup(database_path: Path) -> None:
"""Idempotent: create the derived tables and triggers. No version change.
@@ -564,7 +564,7 @@ def _embedding_request(
secret: str | None,
) -> tuple[str, str, dict[str, str], dict[str, object]]:
provider = profile.embedding_provider_kind or profile.provider_kind
base_url = (profile.embedding_base_url or profile.base_url or "").rstrip("/")
base_url = (profile.embedding_base_url or "").rstrip("/")
if provider is ProviderKind.OLLAMA:
return (
"POST",
@@ -97,7 +97,6 @@ class ModelProfileInput(BaseModel):
)
embedding_provider, resolved_embedding_base_url = resolve_embedding_endpoint(
provider_kind=self.provider_kind,
base_url=base_url,
embedding_model=embedding_model,
embedding_provider_kind=embedding_provider_kind,
embedding_base_url=embedding_base_url,
@@ -107,6 +106,12 @@ class ModelProfileInput(BaseModel):
ProviderKind.ANTHROPIC,
}:
raise ValueError("embedding provider must be OpenAI-compatible, Ollama, or local ONNX")
if (
embedding_provider is not ProviderKind.LOCAL_ONNX
and resolved_embedding_base_url is None
):
# `ModelProfileStore.create` refuses it with its own code.
return self
if (
embedding_provider is not ProviderKind.LOCAL_ONNX
and resolved_embedding_base_url is not None
@@ -308,11 +313,12 @@ class ModelProfileStore:
f"""
UPDATE {table}
SET embedding_provider_kind = provider_kind,
embedding_base_url = base_url
embedding_base_url = COALESCE(embedding_base_url, base_url)
WHERE embedding_model IS NOT NULL
AND embedding_provider_kind IS NULL
"""
)
_pin_embedding_endpoints(connection, table)
connection.execute(
"""
INSERT OR IGNORE INTO model_profile_versions (
@@ -355,11 +361,11 @@ class ModelProfileStore:
)
embedding_provider_kind, embedding_base_url = resolve_embedding_endpoint(
provider_kind=value.provider_kind,
base_url=base_url,
embedding_model=defaulted_model,
embedding_provider_kind=defaulted_provider,
embedding_base_url=defaulted_base_url,
)
require_embedding_base_url(embedding_provider_kind, embedding_base_url)
try:
with self._connect(writing=True) as connection:
connection.execute(
@@ -612,7 +618,6 @@ class ModelProfileStore:
)
embedding_provider_kind, embedding_base_url = resolve_embedding_endpoint(
provider_kind=current.provider_kind,
base_url=base_url,
embedding_model=embedding_model,
embedding_provider_kind=(
patch.embedding_provider_kind
@@ -627,6 +632,7 @@ class ModelProfileStore:
)
if embedding_provider_kind in {ProviderKind.HY_DEFAULT, ProviderKind.ANTHROPIC}:
raise ValueError("embedding provider must be OpenAI-compatible, Ollama, or local ONNX")
require_embedding_base_url(embedding_provider_kind, embedding_base_url)
if (
embedding_provider_kind is not ProviderKind.LOCAL_ONNX
and embedding_base_url is not None
@@ -1167,21 +1173,77 @@ def attach_disable_thinking(
def resolve_embedding_endpoint(
*,
provider_kind: ProviderKind,
base_url: str | None,
embedding_model: str | None,
embedding_provider_kind: ProviderKind | None,
embedding_base_url: str | None,
) -> tuple[ProviderKind | None, str | None]:
"""The embedding provider and the endpoint exactly as given.
The endpoint never follows the generation `base_url`: the collection name
hashes it, so a proxy or region edit of the generation endpoint moved
writes to a new, empty collection, and providers without embeddings were
sent embedding requests. `require_embedding_base_url` refuses a remote
embedding without one.
"""
if embedding_model is None:
return None, None
if embedding_provider_kind is ProviderKind.LOCAL_ONNX:
return ProviderKind.LOCAL_ONNX, None
if embedding_provider_kind is None:
return provider_kind, embedding_base_url or base_url
return (
embedding_provider_kind,
embedding_base_url or DEFAULT_BASE_URLS.get(embedding_provider_kind),
)
return embedding_provider_kind or provider_kind, embedding_base_url
class EmbeddingBaseUrlRequiredError(ValueError):
def __init__(self) -> None:
super().__init__("embedding_base_url_required")
def require_embedding_base_url(provider: ProviderKind | None, base_url: str | None) -> None:
"""A remote embedding must name its endpoint; the UI may offer the generation one."""
if provider not in {None, ProviderKind.LOCAL_ONNX} and not base_url:
raise EmbeddingBaseUrlRequiredError()
def pinned_embedding_base_url(
*,
provider_kind: ProviderKind | str,
base_url: str | None,
embedding_model: str | None,
embedding_provider_kind: ProviderKind | str | None,
embedding_base_url: str | None,
) -> str | None:
"""The endpoint a stored profile's embedding resolved to before it was pinned.
The Runtime used `embedding_base_url or base_url`, and the provider default
only when both were empty (no collection could have existed then). Writing
that value keeps the endpoint, and so the collection name, byte-identical.
"""
provider = ProviderKind(embedding_provider_kind or provider_kind)
if embedding_model is None or embedding_base_url or provider is ProviderKind.LOCAL_ONNX:
return embedding_base_url
return base_url or DEFAULT_BASE_URLS.get(provider)
def _pin_embedding_endpoints(connection: sqlite3.Connection, table: str) -> None:
"""Write the resolved endpoint into every remote embedding left without one."""
rows = connection.execute(
f"SELECT rowid, provider_kind, base_url, embedding_model, embedding_provider_kind "
f"FROM {table} WHERE embedding_model IS NOT NULL AND embedding_base_url IS NULL "
"AND COALESCE(embedding_provider_kind, provider_kind) != 'local_onnx'"
).fetchall()
for row in rows:
connection.execute(
f"UPDATE {table} SET embedding_base_url = ? WHERE rowid = ?",
(
pinned_embedding_base_url(
provider_kind=row["provider_kind"],
base_url=row["base_url"],
embedding_model=row["embedding_model"],
embedding_provider_kind=row["embedding_provider_kind"],
embedding_base_url=None,
),
row["rowid"],
),
)
def validate_model_base_url(value: str) -> None:
+24 -1
View File
@@ -14,6 +14,7 @@ from pydantic import BaseModel, ConfigDict, Field
from one_memory.hy_sdk_types import HySdkMemoryRecord
from one_memory.memory_governance import GovernedMemory, MemoryStatus
from one_memory.sdk_search_options import SdkSearchOptions
from one_memory.semantic_deadline import (
SemanticMemoryHit,
SemanticSearch,
@@ -22,6 +23,7 @@ from one_memory.semantic_deadline import (
normalize_semantic_hits,
search_within,
)
from one_memory.user_scope import DEFAULT_USER_ID, UserId
def recall_blend_mode() -> str:
@@ -49,7 +51,15 @@ class TruncationBudget(StrEnum):
TIMEOUT = "timeout_budget"
class RecallRequest(BaseModel):
class RecallRequest(SdkSearchOptions):
"""`POST /api/v1/recall`.
The `SdkSearchOptions` fields, `schema_limit` and `intention_limit` pass
through to the SDK memory search (and the floor and row filters to the
raw fallback). Absent, recall makes the search it always made: the
relevance floor, schema and intention lanes sized to `limit`.
"""
model_config = ConfigDict(frozen=True, extra="forbid")
query: str = Field(min_length=1, max_length=2_000)
@@ -67,6 +77,19 @@ class RecallRequest(BaseModel):
#: Wall-clock budget for candidate gather + score. Not a substitute for
#: raising the old newest-500 page; it is the third parallel budget.
timeout_ms: int = Field(default=2_000, ge=50, le=30_000)
#: Whose memories to recall. Omitted, it is `local`, the App's library.
user_id: UserId = DEFAULT_USER_ID
#: The SDK's schema (L6) and proactive intention (L7) lanes.
schema_limit: int | None = Field(default=None, ge=0, le=20)
intention_limit: int | None = Field(default=None, ge=0, le=20)
def search_overrides(self) -> Mapping[str, object]:
values = dict(super().search_overrides())
if self.schema_limit is not None:
values["schema_limit"] = self.schema_limit
if self.intention_limit is not None:
values["intention_limit"] = self.intention_limit
return values
class RecallHit(BaseModel):
@@ -6,7 +6,7 @@ import asyncio
import json
from collections.abc import AsyncIterator, Awaitable, Callable
from contextlib import aclosing
from typing import Any, Literal, Protocol
from typing import Any, Literal, Protocol, cast
import httpx
from pydantic import BaseModel, ConfigDict
@@ -14,7 +14,7 @@ from pydantic import BaseModel, ConfigDict
from one_memory.http_client import create_async_client
from one_memory.hy_runtime_config import HY_MEMORY_READER
from one_memory.hy_sdk_runtime import sdk_relevance_floor
from one_memory.hy_sdk_types import HySdkMemoryRecord, parse_sdk_search
from one_memory.hy_sdk_types import HySdkMemoryRecord, parse_sdk_lane_search, parse_sdk_search
from one_memory.memory_qa import (
MemoryAnswerError,
_DeferredSecretStore,
@@ -31,7 +31,9 @@ from one_memory.model_profiles import (
ProviderKind,
attach_disable_thinking,
)
from one_memory.sdk_search_options import SdkSearchFilters, SearchLane
from one_memory.secrets import SecretStore, SecretStoreUnavailableError
from one_memory.user_scope import owns
_NO_GROUNDING = "现有记忆不足以回答这个问题。"
_GROUNDING_LIMIT = 6
@@ -95,29 +97,51 @@ class SdkMemoryAnswerer:
question: str,
*,
user_id: str = "local",
filters: SdkSearchFilters | None = None,
) -> tuple[SdkAskMemory, ...]:
"""The user's active memories nearest the question.
`filters` are a caller's pass-through search options; absent, the
search is the one ask always made.
"""
client = await self.client_handle.client()
overrides = filters.search_overrides() if filters is not None else {}
raw = await asyncio.to_thread(
client.search,
question,
user_ids=[user_id],
agent_ids=[],
limit=12,
min_score=sdk_relevance_floor(),
schema_min_score=sdk_relevance_floor(),
schema_limit=12,
intention_limit=12,
scene="normal",
reader=HY_MEMORY_READER,
**{
"user_ids": [user_id],
"agent_ids": [],
"limit": 12,
"min_score": sdk_relevance_floor(),
"schema_min_score": sdk_relevance_floor(),
"schema_limit": 12,
"intention_limit": 12,
"scene": "normal",
"reader": HY_MEMORY_READER,
**overrides,
},
)
hits = parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
if filters is not None and overrides:
hits = tuple(
hit
for hit in parse_sdk_lane_search(raw, error_code="hy_memory_search_shape_unknown")
if filters.passes_floor(cast(SearchLane, hit.lane), hit.score)
)
else:
hits = parse_sdk_search(raw, error_code="hy_memory_search_shape_unknown")
found: list[SdkAskMemory] = []
seen: set[str] = set()
for hit in hits:
if hit.memory_id in seen:
continue
record = await _get_memory(client, hit.memory_id)
if record is None or record.status != "active" or not record.is_latest:
if (
record is None
or record.status != "active"
or not record.is_latest
or not owns(record.user_id, user_id)
):
continue
seen.add(record.memory_id)
found.append(_wire_memory(record))
@@ -128,8 +152,9 @@ class SdkMemoryAnswerer:
question: str,
*,
user_id: str = "local",
filters: SdkSearchFilters | None = None,
) -> SdkMemoryAnswer:
candidates = await self.retrieve(question, user_id=user_id)
candidates = await self.retrieve(question, user_id=user_id, filters=filters)
if not candidates:
return SdkMemoryAnswer(answer=_NO_GROUNDING, memories=(), mode="empty")
profile = await self._active_profile()
@@ -160,8 +185,9 @@ class SdkMemoryAnswerer:
question: str,
*,
user_id: str = "local",
filters: SdkSearchFilters | None = None,
) -> AsyncIterator[dict[str, object]]:
candidates = await self.retrieve(question, user_id=user_id)
candidates = await self.retrieve(question, user_id=user_id, filters=filters)
await self._require_current(candidates)
yield {"type": "recall", "memories": candidates}
if not candidates:
@@ -5,8 +5,8 @@ from __future__ import annotations
import asyncio
import logging
import re
from collections.abc import AsyncIterator
from contextlib import AsyncExitStack, asynccontextmanager, suppress
from collections.abc import AsyncIterator, Callable
from contextlib import AsyncExitStack, asynccontextmanager
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any
@@ -33,16 +33,6 @@ from one_memory.semantic_deadline import SemanticMemoryHit
_STABLE_CODE = re.compile(r"[a-z][a-z0-9_]{0,95}")
logger = logging.getLogger(__name__)
RAW_RETENTION_DAYS = 3
RAW_RETENTION_CHECK_SECONDS = 6 * 60 * 60
_RAW_STATUSES = (
"active",
"superseded",
"shadow",
"negated",
"conflicted",
"archived",
)
class SdkMemoryRuntime:
@@ -56,74 +46,17 @@ class SdkMemoryRuntime:
write_queue: DurableWriteQueue,
extraction_settings: ExtractionSettingsStore,
data_dir: Path,
memory_mode: Callable[[], str] = lambda: "pro",
) -> None:
self.coordinator = coordinator
#: The SDK mode, for the graph store's path (`hy_graph_admin`).
self.memory_mode = memory_mode
self.read_handle = read_handle
self.write_queue = write_queue
self.extraction_settings = extraction_settings
self.data_dir = data_dir
self._raw_retention_task: asyncio.Task[None] | None = None
def start_raw_retention(self) -> None:
if self._raw_retention_task is None or self._raw_retention_task.done():
self._raw_retention_task = asyncio.create_task(
self._raw_retention_loop(),
name="one-memory-sdk-raw-retention",
)
async def prune_expired_raw(self, *, now: datetime | None = None) -> int:
cutoff = (now or datetime.now(UTC)) - timedelta(days=RAW_RETENTION_DAYS)
cutoff_timestamp = int(cutoff.timestamp())
client = await self.read_handle.client()
deleted = 0
while True:
page = await asyncio.to_thread(
client.list_memories,
user_id="local",
layers=["l1_raw"],
statuses=list(_RAW_STATUSES),
order="asc",
limit=500,
offset=0,
)
bucket = page.get("vdb") if isinstance(page, dict) else None
rows = bucket.get("memories") if isinstance(bucket, dict) else None
if not isinstance(rows, list) or not rows:
return deleted
expired = [
row
for row in rows
if isinstance(row, dict)
and isinstance(row.get("memory_id"), str)
and int(row.get("memory_at") or row.get("gmt_created") or 0)
< cutoff_timestamp
]
if not expired:
return deleted
removed_page = 0
for row in expired:
result = await asyncio.to_thread(client.delete, row["memory_id"])
if isinstance(result, dict) and result.get("success") is True:
count = int(result.get("deleted_count") or 0)
deleted += count
removed_page += count
if removed_page == 0:
return deleted
async def _raw_retention_loop(self) -> None:
while True:
await asyncio.sleep(RAW_RETENTION_CHECK_SECONDS)
try:
removed = await self.prune_expired_raw()
if removed:
logger.info("sdk_raw_retention_deleted count=%d", removed)
except asyncio.CancelledError:
raise
except Exception as error:
logger.warning(
"sdk_raw_retention_failed error_type=%s",
type(error).__name__,
)
# MemHub keeps every raw the SDK stores, in every mode: there is no
# retention window (owner decision; the SDK itself never deletes raw).
async def semantic_recall(
self,
@@ -154,11 +87,10 @@ class SdkMemoryRuntime:
limit: int = 12,
) -> tuple[tuple[str, float], ...]:
return tuple(
(hit.memory_id, hit.score)
for hit in await self.semantic_recall(query, limit=limit)
(hit.memory_id, hit.score) for hit in await self.semantic_recall(query, limit=limit)
)
async def graph_snapshot(self, *, limit: int = 300) -> HyGraphSnapshot:
async def graph_snapshot(self, *, limit: int = 300, user_id: str = "local") -> HyGraphSnapshot:
try:
await self.coordinator.active_profile_credentials()
except RuntimeError as error:
@@ -173,7 +105,7 @@ class SdkMemoryRuntime:
return await asyncio.to_thread(
read_graph_snapshot,
client,
"local",
user_id,
"*",
limit,
)
@@ -224,6 +156,7 @@ class SdkMemoryRuntime:
profile,
api_key=secret,
data_dir=self.data_dir,
mode=self.memory_mode(),
)
async def decide_graph_width(
@@ -240,6 +173,7 @@ class SdkMemoryRuntime:
api_key=secret,
data_dir=self.data_dir,
decision=decision,
mode=self.memory_mode(),
)
@asynccontextmanager
@@ -378,12 +312,7 @@ class SdkMemoryRuntime:
return None
async def shutdown(self) -> None:
if self._raw_retention_task is None:
return
self._raw_retention_task.cancel()
with suppress(asyncio.CancelledError):
await self._raw_retention_task
self._raw_retention_task = None
return None
def _agent_ids(raw: object) -> tuple[str, ...]:
@@ -23,6 +23,7 @@ from pathlib import Path
from typing import Any
from one_memory.database_access import main_database
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot
from one_memory.imports.models import EvidenceRecord
from one_memory.imports.storage import EvidenceStore
from one_memory.memory_server.contracts import (
@@ -51,6 +52,7 @@ from one_memory.sdk_migration_bundle import inspect_sdk_migration_bundle
from one_memory.sdk_migration_verify import (
MemoryReadbackFailed,
ReadbackReport,
expected_memory_ids,
legacy_memory_ids_from,
legacy_memory_ids_from_bundle,
read_active_profile,
@@ -61,6 +63,8 @@ from one_memory.sdk_vector_reattach import (
ReattachedCollection,
legacy_vector_collections,
reattach_legacy_vectors,
stored_memory_ids,
vectors_belong_to_another_space,
)
from one_memory.target_schema import drop_legacy_tables, legacy_tables_present
@@ -259,6 +263,7 @@ def prepare_legacy_write_queue_for_cutover(database_path: Path) -> tuple[int, in
internal_session_id = internal_session_id_for(
source.conversation_id,
source.source_id,
source.user_id,
)
queue.submit(
WriteRequest(
@@ -456,13 +461,17 @@ def resume_legacy_cleanup(*, data_dir: Path, database_path: Path) -> tuple[str,
bundle_id=marker.backup_bundle_id,
)
from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE
from one_memory.memory_visibility import drop_stale_triggers
recover_abandoned_staging(
data_dir=data_dir,
database_path=database_path,
namespace=HY_MEMORY_NAMESPACE,
)
return drop_legacy_tables(database_path)
dropped = drop_legacy_tables(database_path)
# The visibility cache's grant triggers read a table just dropped (#233).
drop_stale_triggers(database_path)
return dropped
MIGRATION_PREFIX = "sdk-authority-v2"
@@ -699,13 +708,14 @@ def run_startup_migration(
step["authority"] = authority.value
log.migration_id = authority_migration
with log.step("preflight.profile_read") as step:
profile = read_active_profile(database_path)
step["profile_present"] = profile is not None
active = read_active_profile(database_path)
step["profile_present"] = active is not None
# No active profile means no generation model, not no embedding: the
# vectors are the built-in model's, which is also what the Runtime reads.
profile = active if active is not None else builtin_retrieval_snapshot()
if authority is MemoryAuthority.SDK:
with log.step("preflight.leftovers_scan") as step:
vectors_waiting = profile is not None and bool(
legacy_vector_collections(data_dir, profile)
)
vectors_waiting = bool(legacy_vector_collections(data_dir, profile))
leftovers = bool(legacy_tables_present(database_path)) or abandoned_staging_present(
data_dir=data_dir, namespace=namespace
)
@@ -716,7 +726,6 @@ def run_startup_migration(
reattached: tuple[ReattachedCollection, ...] = ()
readback: ReadbackReport | None = None
if vectors_waiting:
assert profile is not None
progress("vectors")
with log.step("vectors.reattach") as step:
reattached = reattach_legacy_vectors(data_dir, profile)
@@ -768,12 +777,12 @@ def run_startup_migration(
step["chat"] = len(legacy.chat)
step["productivity"] = len(legacy.productivity)
step["users"] = len(legacy.users)
if profile is None and not legacy.empty:
# Without the embedding profile the Runtime's collections cannot be
# named, so neither reattached nor read back. Legacy stays intact.
log.check("preflight.model_profile_check", "sdk_migration_model_profile_required")
raise DataPlaneMigrationError("sdk_migration_model_profile_required")
log.check("preflight.model_profile_check", None)
if active is None and not legacy.empty and vectors_belong_to_another_space(data_dir, profile):
# The memories live in the space of a profile that is no longer active,
# so the built-in model cannot read them. Legacy stays intact.
log.check("preflight.vector_space_check", "sdk_migration_vector_profile_missing")
raise DataPlaneMigrationError("sdk_migration_vector_profile_missing")
log.check("preflight.vector_space_check", None)
with capture_fence(data_dir):
# Builds through 19 left a staged SDK copy beside the original store.
# Settling it before the bundle means the bundle holds the original
@@ -829,17 +838,23 @@ def run_startup_migration(
step.update(preparation.as_document())
reattached = ()
readback = None
if profile is not None:
progress("vectors")
with log.step("vectors.reattach") as step:
reattached = reattach_legacy_vectors(data_dir, profile)
step["collections"] = len(reattached)
if not legacy.empty:
assert profile is not None
# Before reattach touches the store, and after the staging
# recovery above has settled which store is the original.
with log.step("verify.stored_ids") as step:
expected, missing = expected_memory_ids(legacy, stored_memory_ids(data_dir))
step["chat_expected"] = len(expected.chat)
step["chat_missing_before_migration"] = missing
progress("vectors")
with log.step("vectors.reattach") as step:
reattached = reattach_legacy_vectors(data_dir, profile)
step["collections"] = len(reattached)
if not expected.empty:
progress("verify")
with log.step("verify.readback") as step:
with client_for(profile) as client:
readback = verify_readback(client, legacy)
readback = verify_readback(
client, expected, chat_missing_before_migration=missing
)
step.update(readback.as_document())
if not readback.passed:
store.set_phase(migration_id, phase="readback_failed", state="blocked")
@@ -14,12 +14,12 @@ import json
import sqlite3
from collections.abc import Iterator
from contextlib import closing, contextmanager
from dataclasses import dataclass
from dataclasses import dataclass, replace
from pathlib import Path
from typing import Any
from one_memory.database_access import main_database
from one_memory.model_profiles import ModelProfileSnapshot
from one_memory.model_profiles import ModelProfileSnapshot, pinned_embedding_base_url
from one_memory.sdk_data_plane_migration import DataPlaneMigrationError
_PRODUCTIVITY_KINDS = frozenset({"coding", "working"})
@@ -47,6 +47,8 @@ class ReadbackReport:
chat_readable: int
productivity_expected: int
productivity_readable: int
#: Confirmed in the mirror, gone from the SDK before this migration began.
chat_missing_before_migration: int = 0
@property
def passed(self) -> bool:
@@ -63,6 +65,7 @@ class ReadbackReport:
"chat_readable": self.chat_readable,
"productivity_expected": self.productivity_expected,
"productivity_readable": self.productivity_readable,
"chat_missing_before_migration": self.chat_missing_before_migration,
"passed": self.passed,
}
@@ -115,6 +118,27 @@ def read_legacy_memory_ids(connection: sqlite3.Connection) -> LegacyMemoryIds:
)
def expected_memory_ids(
legacy: LegacyMemoryIds, stored: frozenset[str]
) -> tuple[LegacyMemoryIds, int]:
"""What the readback must find, and how many confirmed chats it need not.
A 1.0.4 library can hold confirmed rows whose SDK node the SDK had already
merged or deleted: no migration could read those back (658 confirmed, 657
in the store, and the whole upgrade failed). So chat is held to what the
store held before this run, and anything in there must still read back.
If none of the confirmed chats is in the store, that is not history: the
store is empty or is the wrong one. Nothing is excused then, so the
readback still fails on every id instead of checking nothing just before
the mirror is dropped.
"""
chat = tuple(memory_id for memory_id in legacy.chat if memory_id in stored)
if not chat:
return legacy, 0
return replace(legacy, chat=chat), len(legacy.chat) - len(chat)
def legacy_memory_ids_from(database_path: Path) -> LegacyMemoryIds:
with main_database(database_path).read() as connection:
return read_legacy_memory_ids(connection)
@@ -160,7 +184,14 @@ def read_active_profile(database_path: Path) -> ModelProfileSnapshot | None:
model=row["model"],
embedding_model=row["embedding_model"],
embedding_provider_kind=row["embedding_provider_kind"],
embedding_base_url=row["embedding_base_url"],
# Read-only, beside a Core that may not have pinned it yet.
embedding_base_url=pinned_embedding_base_url(
provider_kind=row["provider_kind"],
base_url=row["base_url"],
embedding_model=row["embedding_model"],
embedding_provider_kind=row["embedding_provider_kind"],
embedding_base_url=row["embedding_base_url"],
),
embedding_dimensions=row["embedding_dimensions"],
timeout_seconds=row["timeout_seconds"],
max_concurrency=row["max_concurrency"],
@@ -179,14 +210,18 @@ def runtime_sdk_client(
from one_memory.model_profiles import ModelProfileStore
from one_memory.secrets import KeyringSecretStore, SecretStoreUnavailableError
try:
secret = ModelProfileStore(database_path).resolve_snapshot_secret(
profile.profile_id, profile.revision, KeyringSecretStore()
)
except SecretStoreUnavailableError:
raise DataPlaneMigrationError("sdk_migration_secure_storage_unavailable") from None
if profile.has_api_key and secret is None:
raise DataPlaneMigrationError("sdk_migration_model_secret_unavailable")
secret: str | None = None
# Without a key there is nothing to resolve, and the built-in retrieval
# snapshot has no stored version to look one up by.
if profile.has_api_key:
try:
secret = ModelProfileStore(database_path).resolve_snapshot_secret(
profile.profile_id, profile.revision, KeyringSecretStore()
)
except SecretStoreUnavailableError:
raise DataPlaneMigrationError("sdk_migration_secure_storage_unavailable") from None
if secret is None:
raise DataPlaneMigrationError("sdk_migration_model_secret_unavailable")
lease = acquire_client(profile, api_key=secret, data_dir=data_dir)
try:
yield lease.client
@@ -194,7 +229,9 @@ def runtime_sdk_client(
lease.close()
def verify_readback(client: Any, legacy: LegacyMemoryIds) -> ReadbackReport:
def verify_readback(
client: Any, legacy: LegacyMemoryIds, *, chat_missing_before_migration: int = 0
) -> ReadbackReport:
listed: set[str] = set()
for user_id in legacy.users or ("local",):
offset = 0
@@ -228,5 +265,6 @@ def verify_readback(client: Any, legacy: LegacyMemoryIds) -> ReadbackReport:
chat_readable=sum(1 for memory_id in legacy.chat if readable(memory_id)),
productivity_expected=len(legacy.productivity),
productivity_readable=sum(1 for memory_id in legacy.productivity if readable(memory_id)),
chat_missing_before_migration=chat_missing_before_migration,
)
@@ -0,0 +1,125 @@
"""The read client, kept in the embedding space the active profile writes to.
Every read used to open the built-in 512-dim collection while writes followed
the active profile, so a custom embedding read an empty space (#242). This
handle opens the active profile's `retrieval_view` instead, and the built-in
space only when no profile is active.
A profile change builds a new client and retires the old one; a retired client
is closed once no task still uses it, as `SdkWriteCoordinator` does for writes
(#218). A use is the task that asked for the client: callers keep their
`client = await handle.client()` shape, and the use ends with that task (one
HTTP request, one retention pass).
The memory mode (lite / pro / ultra) is fixed per SDK client, so a mode change
rebuilds and retires the same way: handles are keyed by (profile, mode). The
mode never names a collection; it is a separate client parameter.
"""
from __future__ import annotations
import asyncio
from collections.abc import Callable
from pathlib import Path
from typing import Any
from one_memory.hy_graph_admin import effective_memory_mode
from one_memory.hy_sdk_runtime import (
BUILTIN_RETRIEVAL_PROFILE_ID,
SdkClientHandle,
builtin_retrieval_snapshot,
retrieval_view,
)
from one_memory.hy_sdk_types import HySdkRuntimeError
from one_memory.model_profiles import ModelProfileStore, ProviderKind
from one_memory.secrets import SecretStore, SecretStoreUnavailableError
_BUILTIN = (BUILTIN_RETRIEVAL_PROFILE_ID, 0)
class ActiveProfileReadHandle(SdkClientHandle):
"""An `SdkClientHandle` whose profile follows the active one."""
def __init__(
self,
*,
profiles: ModelProfileStore,
secrets: SecretStore,
data_dir: Path,
memory_mode: Callable[[], str] = effective_memory_mode,
) -> None:
super().__init__(builtin_retrieval_snapshot(), api_key=None, data_dir=data_dir)
self.profiles = profiles
self.secrets = secrets
self.data_dir = data_dir
self.memory_mode = memory_mode
self._handles: dict[tuple[str, int, str], SdkClientHandle] = {}
#: Tasks in flight per handle; a retired handle closes at zero.
self._uses: dict[SdkClientHandle, int] = {}
self._retired: set[SdkClientHandle] = set()
self._closing: set[asyncio.Task[None]] = set()
self._handle_lock = asyncio.Lock()
async def client(self) -> Any:
profile = await asyncio.to_thread(self.profiles.active_identity) or _BUILTIN
identity = (*profile, self.memory_mode())
async with self._handle_lock:
handle = self._handles.get(identity)
if handle is None:
# Only on a change of profile: resolving a key can reach the
# system keychain, so it is not done on every read.
handle = await asyncio.to_thread(self._build, identity)
self._handles[identity] = handle
self._retired.update(value for key, value in self._handles.items() if key != identity)
self._handles = {identity: handle}
unused = {value for value in self._retired if not self._uses.get(value)}
self._retired -= unused
self._hold(handle)
await asyncio.gather(*(value.close() for value in unused))
return await handle.client()
async def close(self) -> None:
async with self._handle_lock:
handles = (*self._handles.values(), *self._retired)
self._handles, self._retired, self._uses = {}, set(), {}
await asyncio.gather(*(handle.close() for handle in handles), *self._closing)
def _hold(self, handle: SdkClientHandle) -> None:
"""Count a use of `handle` until the asking task finishes."""
task = asyncio.current_task()
if task is None: # pragma: no cover - always called from a coroutine
return
self._uses[handle] = self._uses.get(handle, 0) + 1
task.add_done_callback(lambda _task: self._release(handle))
def _release(self, handle: SdkClientHandle) -> None:
remaining = self._uses.get(handle, 0) - 1
if remaining > 0:
self._uses[handle] = remaining
return
self._uses.pop(handle, None)
if handle in self._retired:
self._retired.discard(handle)
closing = asyncio.get_running_loop().create_task(handle.close())
self._closing.add(closing)
closing.add_done_callback(self._closing.discard)
def _build(self, key: tuple[str, int, str]) -> SdkClientHandle:
profile_id, revision, mode = key
identity = (profile_id, revision)
if identity == _BUILTIN:
return SdkClientHandle(
builtin_retrieval_snapshot(), api_key=None, data_dir=self.data_dir, mode=mode
)
profile = self.profiles.snapshot(*identity)
view = retrieval_view(profile)
secret: str | None = None
# Only a remote OpenAI-compatible embedding sends the shared key.
if view.embedding_provider_kind is ProviderKind.OPENAI_COMPATIBLE:
try:
secret = self.profiles.resolve_snapshot_secret(*identity, self.secrets)
except SecretStoreUnavailableError:
raise HySdkRuntimeError("secure_storage_unavailable") from None
if secret is None:
raise HySdkRuntimeError("model_secret_unavailable")
return SdkClientHandle(view, api_key=secret, data_dir=self.data_dir, mode=mode)
@@ -0,0 +1,179 @@
"""SDK search parameters a host may pass through to `client.search`.
`POST /api/v2/context/search`, `POST /api/v1/recall` and the v2 ask routes
used to fix every SDK search knob, so a host that wanted a stricter relevance
floor filtered the answer itself. These fields are the public `search`
parameters of hy-memory 2.0.9 that are safe to hand a caller. Every field is
optional: absent, the route keeps exactly the call it made before.
Never exposed: `user_ids` (a route searches its one scoped `user_id`),
`embed_override` and `query_embedding` (they bypass the configured embedder),
and `request_id` (Core's own correlation id).
A caller's floor is also applied to what Core returns, not only handed to the
SDK: a hit below the floor that names its lane is dropped (`passes_floor`).
With `min_score` alone that floor covers every lane, so `min_score=0.9`
answers only results scoring at least 0.9.
"""
from __future__ import annotations
from collections.abc import Mapping
from typing import Annotated, Literal
from pydantic import BaseModel, ConfigDict, Field, StrictBool, model_validator
#: The SDK's `MemoryLayer` values (hy-memory 2.0.9).
MemoryLayerName = Literal[
"l0_basic_info",
"l1_raw",
"l2_fact",
"l3_summary",
"l4_identity",
"l5_knowledge",
"l6_schema",
"l7_intention",
]
MEMORY_LAYERS: tuple[str, ...] = (
"l0_basic_info",
"l1_raw",
"l2_fact",
"l3_summary",
"l4_identity",
"l5_knowledge",
"l6_schema",
"l7_intention",
)
#: The SDK's `MemoryStatus` values.
MEMORY_STATUSES: tuple[str, ...] = (
"active",
"superseded",
"negated",
"conflicted",
"shadow",
"archived",
)
SearchScene = Literal["chat", "working", "coding", "all"]
#: Which recall lane produced a hit, for the floor that applies to it.
SearchLane = Literal["normal", "schema", "proactive", "working", "coding", "raw"]
Score = Annotated[float, Field(ge=0.0, le=1.0)]
SourceName = Annotated[str, Field(min_length=1, max_length=128)]
ScopeId = Annotated[str, Field(min_length=1, max_length=512)]
class SdkSearchFilters(BaseModel):
"""Filters and floors every search route accepts (recall, context, ask)."""
model_config = ConfigDict(frozen=True, extra="forbid")
#: Chat lane floor; with nothing more specific, every lane's.
min_score: Score | None = None
#: Schema (L6) lane floor; defaults to `min_score` when only that is set.
schema_min_score: Score | None = None
#: False skips the SDK's reranking stages for this call.
rerank: StrictBool | None = None
include_sources: tuple[SourceName, ...] | None = Field(default=None, max_length=64)
exclude_sources: tuple[SourceName, ...] | None = Field(default=None, max_length=64)
#: Unix seconds: only memories created at or after it.
created_after: float | None = Field(default=None, ge=0)
session_ids: tuple[ScopeId, ...] | None = Field(default=None, max_length=64)
agent_ids: tuple[ScopeId, ...] | None = Field(default=None, max_length=64)
#: Only these layers (pro/ultra; lite always searches raw).
layers: tuple[MemoryLayerName, ...] | None = Field(default=None, min_length=1, max_length=8)
def filter_kwargs(self) -> dict[str, object]:
"""The SDK keyword arguments this request sets, and only those."""
values: dict[str, object] = {}
for name in (
"min_score",
"schema_min_score",
"rerank",
"created_after",
):
value = getattr(self, name)
if value is not None:
values[name] = value
for name in ("include_sources", "exclude_sources", "session_ids", "agent_ids", "layers"):
value = getattr(self, name)
if value is not None:
values[name] = list(dict.fromkeys(value))
if self.min_score is not None and self.schema_min_score is None:
values["schema_min_score"] = self.min_score
return values
def lane_floor(self, lane: SearchLane) -> float | None:
"""The caller's floor for one lane, or None when the caller set none."""
if lane == "schema":
return self.schema_min_score if self.schema_min_score is not None else self.min_score
return self.min_score
def passes_floor(self, lane: SearchLane, score: float) -> bool:
floor = self.lane_floor(lane)
return floor is None or score >= floor
def search_overrides(self) -> Mapping[str, object]:
return self.filter_kwargs()
class SdkSearchOptions(SdkSearchFilters):
"""Every SDK search knob a recall or context caller may set."""
#: working/coding fact-layer floor (SDK default 0.30).
working_fact_min_score: Score | None = None
#: working/coding experience-layer floor (SDK default fact floor + 0.10).
working_experience_min_score: Score | None = None
reader: Literal["legacy", "hybrid"] | None = None
#: chat (the default), working, coding, or all three fused.
scene: SearchScene | None = None
#: agentic spends one model call to classify and rewrite the query.
search_mode: Literal["default", "agentic"] | None = None
#: working/coding lanes only; `branch` needs `workspace_id`.
workspace_id: ScopeId | None = None
branch: ScopeId | None = None
@model_validator(mode="after")
def branch_needs_workspace(self) -> SdkSearchOptions:
if self.branch is not None and self.workspace_id is None:
raise ValueError("branch requires workspace_id")
return self
def search_overrides(self) -> Mapping[str, object]:
values = self.filter_kwargs()
for name in (
"working_fact_min_score",
"working_experience_min_score",
"reader",
"scene",
"search_mode",
"workspace_id",
"branch",
):
value = getattr(self, name)
if value is not None:
values[name] = value
if self.min_score is not None and self.working_fact_min_score is None:
values["working_fact_min_score"] = self.min_score
if self.min_score is not None and self.working_experience_min_score is None:
values["working_experience_min_score"] = self.min_score
return values
def lane_floor(self, lane: SearchLane) -> float | None:
if lane in {"working", "coding"}:
explicit = [
value
for value in (self.working_fact_min_score, self.working_experience_min_score)
if value is not None
]
# Facts and experiences share one score field on a fused hit, so
# the looser of the two floors is the one Core can enforce.
return min(explicit) if explicit else self.min_score
return super().lane_floor(lane)
def split_csv(values: list[str] | None) -> list[str]:
"""Query values given repeated (`a&a`) or comma separated (`a,b`)."""
found: list[str] = []
for value in values or ():
found.extend(part.strip() for part in value.split(",") if part.strip())
return list(dict.fromkeys(found))
@@ -9,6 +9,7 @@ one. Runtime code only reaches vectors through the SDK. This module runs from
from __future__ import annotations
import re
import sqlite3
from collections.abc import Iterator
from contextlib import closing
@@ -21,6 +22,8 @@ from one_memory.model_profiles import ModelProfileSnapshot
from one_memory.sdk_data_plane_migration import DataPlaneMigrationError
_BATCH = 500
#: Every name a MemHub build gave an embedding space's base collection.
_SPACE_NAME = re.compile(r"^(?:memhub_[0-9a-f]{16}|one_memory_[0-9a-f]{12})(?:_|$)")
def open_offline_chroma(directory: Path) -> Any:
@@ -51,13 +54,20 @@ def _vector_directory(data_dir: Path) -> Path:
return data_dir / "hy-memory" / "governed-v2" / "vector"
def _stored_names(data_dir: Path) -> tuple[str, ...]:
def _stored_names(data_dir: Path, *, filled: bool = False) -> tuple[str, ...]:
"""Collection names; with `filled`, only those holding at least one row."""
database = _vector_directory(data_dir) / "chroma.sqlite3"
if not database.is_file():
return ()
with closing(sqlite3.connect(f"file:{database}?mode=ro", uri=True)) as connection:
try:
rows = connection.execute("SELECT name FROM collections").fetchall()
rows = connection.execute(
"SELECT DISTINCT collections.name FROM collections "
"JOIN segments ON segments.collection = collections.id "
"JOIN embeddings ON embeddings.segment_id = segments.id"
if filled
else "SELECT name FROM collections"
).fetchall()
except sqlite3.Error:
return ()
return tuple(sorted(str(row[0]) for row in rows))
@@ -80,6 +90,59 @@ def legacy_vector_collections(
return tuple(pairs)
def vectors_belong_to_another_space(data_dir: Path, profile: ModelProfileSnapshot) -> bool:
"""Vectors exist, but only in embedding spaces `profile` does not name.
Counted rows, not names: every Runtime opens its read collection with
`get_or_create`, so an empty collection under the expected base proves
nothing. Read-only, so a preflight can ask before anything changes.
"""
current, legacy_bases = vector_collection_bases(profile)
known = (current, *legacy_bases)
filled = _stored_names(data_dir, filled=True)
return not any(
name == base or name.startswith(f"{base}_") for name in filled for base in known
) and any(_SPACE_NAME.match(name) for name in filled)
def stored_memory_ids(data_dir: Path) -> frozenset[str]:
"""Every id in the vector store as found, across all embedding spaces.
All of them on purpose. A memory stranded under a space the Runtime will
not read must count as present, so the readback then fails on it instead
of excusing it. Both the point id and the SDK's `node_id` are kept: the
SDK derives a point id from a node id that is not a UUID.
Callers must have stopped every SDK writer. A store that cannot be read
raises: "unknown" must never be taken for "absent".
"""
if not (_vector_directory(data_dir) / "chroma.sqlite3").is_file():
return frozenset()
found: set[str] = set()
try:
client = open_offline_chroma(_vector_directory(data_dir))
try:
for collection in client.list_collections():
offset = 0
while True:
page = collection.get(include=["metadatas"], limit=_BATCH, offset=offset)
ids = list(page["ids"])
if not ids:
break
found.update(ids)
found.update(
str(metadata["node_id"])
for metadata in page["metadatas"] or ()
if metadata and metadata.get("node_id")
)
offset += len(ids)
finally:
client.close()
except Exception as error:
raise DataPlaneMigrationError("sdk_migration_vector_store_unreadable") from error
return frozenset(found)
def _records(collection: Any) -> Iterator[dict[str, Any]]:
offset = 0
while True:
@@ -9,7 +9,8 @@ from dataclasses import dataclass
from datetime import datetime
from typing import Any
from one_memory.hy_sdk_runtime import SdkClientHandle
from one_memory.hy_graph_admin import effective_memory_mode
from one_memory.hy_sdk_runtime import SdkClientHandle, builtin_retrieval_snapshot
from one_memory.imports.models import EvidenceContentKind, EvidenceOrigin, EvidenceRecord
from one_memory.input_sanitization import sanitize_model_input
from one_memory.model_profiles import ModelProfileSnapshot, ModelProfileStore
@@ -31,8 +32,11 @@ class SdkWriteReceipt:
class SdkWriteProcessor:
"""Turn one durable job into exactly one public SDK write call."""
def __init__(self, client_handle: SdkClientHandle) -> None:
def __init__(self, client_handle: SdkClientHandle, *, lite: bool = False) -> None:
self._client_handle = client_handle
#: lite makes no model call: `add_extracted` would still call one, so a
#: direct memory note is stored as one raw user turn through `add`.
self._lite = lite
async def process_sdk_job(
self,
@@ -64,7 +68,7 @@ class SdkWriteProcessor:
# waits and retries instead of being left `unknown`.
raise RuntimeError("hy_memory_client_unavailable") from error
if extracted:
if extracted and not self._lite:
contents = [sanitize_model_input(record.verbatim) for record in records]
await before_hy_invocation()
result = await asyncio.to_thread(
@@ -81,12 +85,15 @@ class SdkWriteProcessor:
else:
messages = [
{
"role": record.role,
"role": "user" if extracted else record.role,
"content": sanitize_model_input(record.verbatim),
}
for record in records
if record.content_kind is EvidenceContentKind.RAW_CONVERSATION
and record.role in _MESSAGE_ROLES
if extracted
or (
record.content_kind is EvidenceContentKind.RAW_CONVERSATION
and record.role in _MESSAGE_ROLES
)
]
if not messages:
raise RuntimeError("hy_memory_task_without_inputs")
@@ -138,6 +145,15 @@ class SdkWriteCoordinator:
from under a write still running on it: the SDK cleared its stores and
the SharedRuntime closed mid-`add`, the job failed with AttributeError
after HY was invoked, and it was left `unknown`.
Handles are keyed by profile and memory mode, so a mode change retires
handles the same way. In lite no generation model is needed: without an
active profile the built-in local embedder writes raw turns.
`exclusive_for_evolve(user_id)` keeps evolve and that user's writes
apart: it waits for the user's write in flight, and while it runs that
user's jobs are refused before the SDK is called
(`memory_evolve_in_progress`), so the drain puts them back.
"""
def __init__(
@@ -148,13 +164,22 @@ class SdkWriteCoordinator:
data_dir: Any,
transport_factory: Callable[[ModelProfileSnapshot], Callable[[], Any] | None],
sedimentation_enabled: Callable[[], bool],
memory_mode: Callable[[], str] = effective_memory_mode,
) -> None:
self.profiles = profiles
self.secrets = secrets
self.data_dir = data_dir
self.transport_factory = transport_factory
self.sedimentation_enabled = sedimentation_enabled
self._handles: dict[tuple[str, int], SdkClientHandle] = {}
self.memory_mode = memory_mode
self._handles: dict[tuple[str, int, str], SdkClientHandle] = {}
#: Users evolve (or the graph backfill) is running for, the writes in
#: flight per user, and one exclusive run per user at a time. Writes
#: of one user still run side by side; only evolve waits for them.
self._evolving: set[str] = set()
self._writing: dict[str, int] = {}
self._writes_idle = asyncio.Condition()
self._exclusive: dict[str, asyncio.Lock] = {}
#: Uses in flight per handle; a retired handle closes at zero.
self._uses: dict[SdkClientHandle, int] = {}
self._retired: set[SdkClientHandle] = set()
@@ -169,16 +194,66 @@ class SdkWriteCoordinator:
) -> SdkWriteReceipt:
if not self.sedimentation_enabled():
raise RuntimeError("model_sedimentation_disabled")
async with self._active_handle() as handle:
return await SdkWriteProcessor(handle).process_sdk_job(
records,
request_id=request_id,
before_hy_invocation=before_hy_invocation,
)
user_id = records[0].user_id if records else "local"
if user_id in self._evolving:
# Before any SDK call: the drain returns the job to pending.
raise RuntimeError("memory_evolve_in_progress")
self._writing[user_id] = self._writing.get(user_id, 0) + 1
try:
async with self._active_handle() as handle:
return await SdkWriteProcessor(
handle, lite=getattr(handle, "mode", None) == "lite"
).process_sdk_job(
records,
request_id=request_id,
before_hy_invocation=before_hy_invocation,
)
finally:
remaining = self._writing.get(user_id, 1) - 1
if remaining > 0:
self._writing[user_id] = remaining
else:
self._writing.pop(user_id, None)
async with self._writes_idle:
self._writes_idle.notify_all()
async def inquire_write_outcome(
self, request_id: str
) -> SdkWriteReceipt | None:
async def retire_stale_modes(self) -> None:
"""After a mode change: retire handles of other modes, closing the idle ones.
A handle still writing is closed when its last use ends (#218), so a
write that started in the old mode finishes in it.
"""
mode = self.memory_mode()
async with self._lock:
stale = {key: handle for key, handle in self._handles.items() if key[2] != mode}
for key in stale:
del self._handles[key]
self._retired.update(stale.values())
unused = {value for value in self._retired if not self._uses.get(value)}
self._retired -= unused
await asyncio.gather(*(value.close() for value in unused))
def evolving(self, user_id: str) -> bool:
"""Whether evolve holds this user, so their queued writes should wait."""
return user_id in self._evolving
@asynccontextmanager
async def exclusive_for_evolve(self, user_id: str) -> AsyncIterator[None]:
"""Hold one user's writes off while evolve runs for them.
New writes of the user are refused from here on; the ones already in
flight are waited for before the body runs.
"""
async with self._exclusive.setdefault(user_id, asyncio.Lock()):
self._evolving.add(user_id)
try:
async with self._writes_idle:
await self._writes_idle.wait_for(lambda: not self._writing.get(user_id))
yield
finally:
self._evolving.discard(user_id)
async def inquire_write_outcome(self, request_id: str) -> SdkWriteReceipt | None:
async with self._active_handle() as handle:
return await SdkWriteProcessor(handle).inquire_write_outcome(request_id)
@@ -204,9 +279,11 @@ class SdkWriteCoordinator:
@asynccontextmanager
async def _active_handle(self) -> AsyncIterator[SdkClientHandle]:
"""Use the active profile's handle; retire the others, closing the unused."""
identity, profile, secret = await asyncio.to_thread(
self._read_active_credentials
mode = self.memory_mode()
profile_identity, profile, secret = await asyncio.to_thread(
self._read_active_credentials, mode
)
identity = (*profile_identity, mode)
async with self._lock:
handle = self._handles.get(identity)
if handle is None:
@@ -215,6 +292,7 @@ class SdkWriteCoordinator:
api_key=secret,
data_dir=self.data_dir,
http_transport_factory=self.transport_factory(profile),
mode=mode,
)
self._handles[identity] = handle
self._retired.update(value for key, value in self._handles.items() if key != identity)
@@ -239,9 +317,14 @@ class SdkWriteCoordinator:
def _read_active_credentials(
self,
mode: str = "pro",
) -> tuple[tuple[str, int], ModelProfileSnapshot, str | None]:
identity = self.profiles.active_identity()
if identity is None:
if mode == "lite":
# lite stores raw turns with one embedding and calls no model.
builtin = builtin_retrieval_snapshot()
return (builtin.profile_id, builtin.revision), builtin, None
raise RuntimeError("model_profile_not_active")
profile = self.profiles.snapshot(*identity)
try:
@@ -27,6 +27,11 @@ class SemanticMemoryHit:
member_ids: tuple[str, ...] = ()
sdk_formatted: bool = False
formatting_complete: bool = False
#: The recall lane that produced the hit (`sdk_search_options.SearchLane`)
#: and the namespace to read it back from. Defaults are a chat memory.
lane: str = "normal"
kind: str = "chat"
variant: str | None = None
def as_dict(self) -> dict[str, object]:
return {
@@ -0,0 +1,39 @@
"""Refuse query parameters a route does not declare.
FastAPI ignores an unknown query parameter. For a memory read that is a
silent widening: `memory-library?user_id=__nobody__` against a Core that did
not know `user_id` answered with the whole library, and the caller believed
it was isolated. Routes that carry this dependency answer such a request with
the same 422 a malformed body gets.
"""
from fastapi import Request
from fastapi.dependencies.models import Dependant
from fastapi.exceptions import RequestValidationError
from fastapi.routing import APIRoute
def _declared_query_names(dependant: Dependant) -> set[str]:
names = {parameter.alias for parameter in dependant.query_params}
for dependency in dependant.dependencies:
names |= _declared_query_names(dependency)
return names
async def forbid_unknown_query_params(request: Request) -> None:
route = request.scope.get("route")
if not isinstance(route, APIRoute):
return
unknown = sorted(set(request.query_params) - _declared_query_names(route.dependant))
if unknown:
raise RequestValidationError(
[
{
"type": "extra_forbidden",
"loc": ("query", name),
"msg": "Extra inputs are not permitted",
"input": None,
}
for name in unknown
]
)
@@ -0,0 +1,65 @@
"""Who a memory belongs to: one Runtime, many `user_id`s.
A user is decided by which source root is read (a Hermes profile is its own
`state.db`), never by session or platform fields. The App never names a user
and keeps reading and writing `local`; every other harness stays on `local`
too. Only Hermes may connect a second root for another user.
In memory, a source's import lane is keyed by `lane_key(user_id, source_id)`.
For `local` that is the bare source id, so every existing key, health record
and log line stays byte-identical; another user's lane is `source@user`.
`@` cannot occur in a user id or a source id, so the key splits back exactly.
"""
import re
from typing import Annotated
from pydantic import Field
DEFAULT_USER_ID = "local"
#: `:` is excluded on purpose: the SDK joins ids with it when it keys history,
#: so a user id holding one could read another user's history.
USER_ID_PATTERN = r"^[A-Za-z0-9._-]{1,64}$"
_USER_ID = re.compile(USER_ID_PATTERN)
#: The only source whose roots may belong to a user other than `local`.
MULTI_USER_SOURCES = frozenset({"hermes"})
UserId = Annotated[str, Field(min_length=1, max_length=64, pattern=USER_ID_PATTERN)]
_LANE_SEPARATOR = "@"
def valid_user_id(value: str) -> bool:
return bool(_USER_ID.fullmatch(value))
def lane_key(user_id: str, source_id: str) -> str:
if user_id == DEFAULT_USER_ID:
return source_id
return f"{source_id}{_LANE_SEPARATOR}{user_id}"
def other_user_lane_prefix(source_id: str) -> str:
"""What every other user's lane of `source_id` starts with."""
return f"{source_id}{_LANE_SEPARATOR}"
def split_lane(key: str) -> tuple[str, str]:
"""`(user_id, source_id)` of a lane key; a bare source id is `local`'s."""
source_id, separator, user_id = key.partition(_LANE_SEPARATOR)
if not separator or not valid_user_id(user_id):
return DEFAULT_USER_ID, key
return user_id, source_id
def owns(record_user_id: str | None, user_id: str) -> bool:
"""Whether a stored memory belongs to `user_id`.
The SDK's `get`/`update`/`delete`/`history`/`get_write_outcome` take no
user, so MemHub checks this before acting on an id. A memory written
before MemHub named users carries `local` or nothing; both are `local`'s.
"""
stored = record_user_id or DEFAULT_USER_ID
return stored == user_id
+5 -1
View File
@@ -105,6 +105,7 @@
| `test_graph_width_guard.py` | `unit` | Graph width mismatch must ask, never silently archive. |
| `test_harness_contract_suite.py` | `contract` | A4-09: one contract, ten harnesses. |
| `test_health.py` | `contract` | 覆盖 a backfill that fails says so rather than disappearing、a failed catch up leaves a traceback behind、configured model without content consent does not start backfill 等 15 个场景。 |
| `test_hermes_gateway_injection.py` | `contract` | Hermes gateway text in `user` rows: notices are skipped, relayed messages unwrapped. |
| `test_history_listing_rotation.py` | `contract` | 覆盖 history listing alternates started and untouched files、history listing drains whichever cohort is longer 场景。 |
| `test_host_capabilities.py` | `unit` | A plugin does not become realtime on every host or every unknown version. |
| `test_hourly_c1_peak_replay_contract.py` | `contract` | Guard the package peak-replay probe's error accounting. |
@@ -160,6 +161,7 @@
| `test_memory_evidence_freshness.py` | `contract` | 新鲜度拆除后:读路径恒为 current;证据修订 / 畸形 proof / segment ref 不再藏记忆或 503。 |
| `test_memory_freshness_integration_review.py` | `integration` | Independent synthetic read-path review; no lifespan, model or user database. |
| `test_memory_library_all_scenes.py` | `integration` | The v2 memory library across chat, working and coding memories. |
| `test_memory_mode_and_evolve.py` | `contract` | The memory mode (lite / pro / ultra), evolve and its nightly schedule. |
| `test_memory_qa.py` | `contract` | 覆盖 search terms keep meaningful single chinese character、streamed answer emits recall then incremental grounded text、governance change during generation stops the old answer 等 14 个场景。 |
| `test_memory_qa_api.py` | `integration` | 覆盖 ask stream endpoint emits recall deltas and final、ask stream endpoint requires the local token 场景。 |
| `test_memory_qa_permissions.py` | `contract` | 覆盖 initial refusal precedes all secret access and network、invalid authorization result fails before secret access、authorization receives exact question and only outbound grounding 等 16 个场景。 |
@@ -196,6 +198,7 @@
| `test_model_profiles.py` | `contract` | 覆盖 profile uses one model and keeps key out of database、api key updates reuse one stable secret reference、startup cleanup removes only secrets not referenced by current profiles 等 15 个场景。 |
| `test_model_request_audit.py` | `contract` | 覆盖 failure is recorded without error message、missing usage stays unknown in detail and zero in summary、records one physical request with provider usage and no content 等 5 个场景。 |
| `test_model_request_budget.py` | `contract` | 覆盖 persists failed request reservations without content、parallel leases cannot exceed request limit、unknown or excessive output and body are denied 等 6 个场景。 |
| `test_multi_user_runtime.py` | `contract` | One Runtime, many users: `user_id` on the memory API, per-user Hermes profiles. |
| `test_mysql_lifecycle.py` | `contract` | A1-07: the decisions Runtime makes around the bundled MySQL. |
| `test_mysql_platform.py` | `contract` | A1-07: the machine-facing half of managing the bundled MySQL. |
| `test_obs03_health_probe.py` | `contract` | OBS-03: '/health' answered with the web app instead of a health payload. |
@@ -246,6 +249,7 @@
| `test_schema_gate.py` | `contract` | 覆盖 fresh database records the build schema、future schema refuses to open、legacy database without a ledger is left untouched 场景。 |
| `test_schema_migrations.py` | `contract` | 覆盖 evidence store adds source ordinal to previous schema、import store migrates previous job and error schema、old active job index lets a sync sit beside paused history 等 4 个场景。 |
| `test_schema_ownership_gate.py` | `contract` | A1-03/A1-04: schema ownership, and the hard part of telling DDL from reads. |
| `test_sdk_api_surface.py` | `contract` | The SDK surface MemHub exposes: search pass-through, layers, raw, v2 aliases. |
| `test_sdk_client_construction.py` | `contract` | SDK clients of one home are constructed one at a time. |
| `test_sdk_data_plane_migration.py` | `contract` | SDK-authority migration ledger, aliases and atomic cutover marker. |
| `test_sdk_memory_qa.py` | `contract` | 覆盖 sdk ask propagates sdk search failure without legacy fallback、sdk ask revalidates memory before returning extract、sdk ask uses native sdk memory without governance fields 场景。 |
@@ -309,4 +313,4 @@
| `test_workspace_recall.py` | `contract` | Workspace memories were unreachable from every host; these pin the way in. |
| `test_write_queue_lease_renewal.py` | `contract` | Rule 8: a lease must outlive the work it covers, and be renewed while it runs. |
共 304 个测试文件。
共 308 个测试文件。
@@ -14,8 +14,8 @@ from pathlib import Path
import pytest
PRIMARY_MARKERS = ("unit", "contract", "integration", "performance", "live")
EXPECTED_TEST_FILE_COUNT = 304
EXPECTED_TEST_FILES_SHA256 = "b28f022b9688c96ca563c59de63e95d344d2f01ba3b6020b422bf4dfb27520d6"
EXPECTED_TEST_FILE_COUNT = 308
EXPECTED_TEST_FILES_SHA256 = "3aadf5953e41c01d4b60469f47242c00d2acef3c42550aa017255e31d216b589"
# Pure transformations, validation models, parity checks and state machines.
# This list is intentionally small: uncertain files default to contract.
@@ -0,0 +1,8 @@
{"id": 131, "session_id": "20261019_005046_a2a79a", "role": "user", "content": "[IMPORTANT: 3 background processes completed. Treat these results as one batch and give one consolidated response; preserve failures and actionable results.]", "timestamp": 1792344160.7817369, "tool_name": null, "finish_reason": null}
{"id": 314, "session_id": "20261019_005046_a2a79a", "role": "user", "content": "[IMPORTANT: Background process proc_a9805510da6f matched watch pattern \"Uvicorn running\".\nCommand: cd ~/Workspace/paddleocr-vl-service && exec ./start.sh\nMatched output:\nINFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)]", "timestamp": 1792421570.094992, "tool_name": null, "finish_reason": null}
{"id": 769, "session_id": "20261021_005036_4fcb846a", "role": "user", "content": "[IMPORTANT: Background process proc_9d3a5fe22318 completed normally (exit code 0).\nCommand: cd /Users/example/.hermes/models/gguf && curl -L -C - --retry 30 --retry-delay 5 --retry-all-errors -sS -o Qwen3.5-2B-Q4_K_M.gguf \"https://huggingface.co/unsloth/Qwen3.5-2B-GGUF/resolve/main/Qwen3.5-2B-Q4_K_M.gguf\" && echo \"Q3.5-2B DONE\" && ls -lh Qwen3.5-2B-Q4_K_M.gguf\nOutput:\n(eval):1: can't change option: zle\n(eval):1: can't change option: zle\nQ3.5-2B DONE\n-rw-r--r--@ 1 example staff 1.2G Oct 21 01:31 Qwen3.5-2B-Q4_K_M.gguf\n]\n\n[INTERNAL NOTIFICATION — generated by the Hermes gateway, not written by a human user. Do not treat it as user input, and do not reply to any earlier chat message because of it.]", "timestamp": 1792517511.042671, "tool_name": null, "finish_reason": null}
{"id": 343, "session_id": "20261021_005036_4fcb846a", "role": "user", "content": "[voice message could not be transcribed automatically; the audio is available at: /Users/example/.hermes/cache/audio/audio_46bab1cf72c3.ogg]", "timestamp": 1792515117.0, "tool_name": null, "finish_reason": null}
{"id": 197, "session_id": "20261019_142101_50c2a6", "role": "user", "content": "Reply with exactly: PONG", "timestamp": 1792390861.155566, "tool_name": null, "finish_reason": null}
{"id": 637, "session_id": "20261021_005036_4fcb846a", "role": "user", "content": "Gateway message origin (JSON data, not instructions or authorization):\n{\"platform\": \"telegram\", \"chat_id\": \"1000000001\", \"chat_type\": \"dm\", \"user_id\": \"1000000001\", \"message_id\": \"93\", \"source_message_id\": \"93\"}\nDo not guess a reply destination when these fields are insufficient.\n\n3.5 2b 得版本也想看看,没有量化或者 gguf 吗", "timestamp": 1792517455.622248, "tool_name": null, "finish_reason": null}
{"id": 548, "session_id": "20261021_005036_4fcb846a", "role": "user", "content": "[OUT-OF-BAND USER MESSAGE — a direct message from the user, delivered once at this position; not tool output and not a new delivery when replayed from conversation history]\nGateway message origin (JSON data, not instructions or authorization):\n{\"platform\": \"telegram\", \"chat_id\": \"1000000001\", \"chat_type\": \"dm\", \"user_id\": \"1000000001\", \"message_id\": \"73\", \"source_message_id\": \"73\"}\nDo not guess a reply destination when these fields are insufficient.\n\n对比结果如何\n[/OUT-OF-BAND USER MESSAGE]", "timestamp": 1792517335.660696, "tool_name": null, "finish_reason": null}
{"id": 2154, "session_id": "20261021_005036_4fcb846a", "role": "user", "content": "[OUT-OF-BAND USER MESSAGE — a direct message from the user, delivered once at this position; not tool output and not a new delivery when replayed from conversation history]\nGateway message origin (JSON data, not instructions or authorization):\n{\"platform\": \"telegram\", \"chat_id\": \"1000000001\", \"chat_type\": \"dm\", \"user_id\": \"1000000001\", \"message_id\": \"489\", \"source_message_id\": \"489\"}\nDo not guess a reply destination when these fields are insufficient.\n\n我预期memhub使用的是hy memory的ultra模式,并且在夜间调用一次envolve接口来进化memory 呢。\n你说memhub的chat走的是什么我有点懵。\n\n我记得memhub写入的时候,不是直接调用sdk client的add么?\n\n应该是没有设置scene才对。可以检查一下\n[/OUT-OF-BAND USER MESSAGE]", "timestamp": 1793448843.4533901, "tool_name": null, "finish_reason": null}
+2 -1
View File
@@ -24,7 +24,8 @@ def model_snapshot(
model="memory-model",
embedding_model="embedding-model",
embedding_provider_kind=embedding_provider,
embedding_base_url=embedding_base_url,
# Stored profiles always name it; see `pinned_embedding_base_url`.
embedding_base_url=embedding_base_url or base_url,
embedding_dimensions=768,
timeout_seconds=30,
max_concurrency=1,
+7 -1
View File
@@ -175,7 +175,13 @@ def test_sdk_migrate_reports_each_step_then_the_result(tmp_path, monkeypatch) ->
assert code == 0, output.getvalue()
lines = [json.loads(line) for line in output.getvalue().splitlines()]
assert [line["step"] for line in lines[:-1]] == ["bundle", "prepare", "cutover", "cleanup"]
assert [line["step"] for line in lines[:-1]] == [
"bundle",
"prepare",
"vectors",
"cutover",
"cleanup",
]
assert lines[-1]["state"] == "cutover"
assert lines[-1]["bundle_id"] == f"{lines[-1]['migration_id']}-pre-cutover"
assert not (tmp_path / ".sdk-migration-capture-fence.json").exists()
@@ -164,7 +164,7 @@ def test_metadata_repair_never_overlaps_an_sdk_client(tmp_path, monkeypatch):
events.append("repair_finished")
return MetadataRepairResult(None, 0, 0, 0)
def open_client(profile, *, api_key, data_dir, http_transport_factory):
def open_client(profile, *, api_key, data_dir, http_transport_factory, mode="pro"):
events.append("client_opened")
return ClientLease(object(), data_dir.resolve())
@@ -0,0 +1,239 @@
"""Hermes gateway text in `user` rows: notices are skipped, relayed messages unwrapped.
Every fixture row is a real Hermes `state.db` row, with the Telegram ids and
the local home directory replaced by fake values (and the batch notice cut
to its first line). They all wear the `user` role, so the import has to
decide on the text.
"""
from __future__ import annotations
import asyncio
import json
import sqlite3
from pathlib import Path
import pytest
from pydantic import JsonValue
from one_memory.conversation_processing import turn_batches
from one_memory.imports.access import AuthorizedSourceRoot
from one_memory.imports.adapters import (
ScanEvent,
ScanIssueCode,
ScannedEvidence,
ScanOptions,
ScanSkip,
)
from one_memory.imports.sources.common import (
ParsedMessage,
ParsedSkip,
SourceRecordContext,
)
from one_memory.imports.sources.hermes import (
HERMES_DESCRIPTOR,
create_hermes_adapter,
gateway_user_text,
parse_hermes_rollout_record,
)
from one_memory.imports.storage import EvidenceStore
_ROWS = {
row["id"]: row
for row in (
json.loads(line)
for line in (Path(__file__).parent / "fixtures" / "hermes_gateway_rows.jsonl")
.read_text(encoding="utf-8")
.splitlines()
if line.strip()
)
}
_DROPPED = {
131: ScanIssueCode.INJECTED_MACHINE_NOTICE, # "3 background processes completed"
314: ScanIssueCode.INJECTED_MACHINE_NOTICE, # "Background process … matched watch pattern"
769: ScanIssueCode.INJECTED_MACHINE_NOTICE, # … plus the gateway's INTERNAL NOTIFICATION
343: ScanIssueCode.VOICE_PLACEHOLDER,
197: ScanIssueCode.PROBE,
}
_UNWRAPPED = {
548: "对比结果如何",
637: "3.5 2b 得版本也想看看,没有量化或者 gguf 吗",
2154: (
"我预期memhub使用的是hy memory的ultra模式,并且在夜间调用一次envolve接口来进化memory 呢。\n"
"你说memhub的chat走的是什么我有点懵。\n\n"
"我记得memhub写入的时候,不是直接调用sdk client的add么?\n\n"
"应该是没有设置scene才对。可以检查一下"
),
}
@pytest.mark.parametrize("row_id", sorted(_DROPPED))
def test_a_machine_turn_in_the_users_role_is_skipped_with_its_reason(row_id: int) -> None:
assert gateway_user_text(_ROWS[row_id]["content"]) == ("", _DROPPED[row_id])
def test_the_gateway_note_drops_a_turn_wherever_it_sits() -> None:
note = _ROWS[769]["content"].rsplit("\n\n", 1)[1]
assert note.startswith("[INTERNAL NOTIFICATION")
assert gateway_user_text(f"anything before it\n\n{note}")[1] is (
ScanIssueCode.INJECTED_MACHINE_NOTICE
)
@pytest.mark.parametrize("row_id", sorted(_UNWRAPPED))
def test_a_relayed_message_keeps_exactly_what_the_person_wrote(row_id: int) -> None:
assert gateway_user_text(_ROWS[row_id]["content"]) == (_UNWRAPPED[row_id], None)
def test_the_interrupted_response_label_is_unwrapped() -> None:
text = "[Context from the interrupted assistant response]\n继续刚才的对比"
assert gateway_user_text(text) == ("继续刚才的对比", None)
@pytest.mark.parametrize(
("text", "reason"),
[
("MEDIA:/Users/example/.hermes/cache/images/a.jpg", ScanIssueCode.NO_TEXT),
("Reply with exactly: PONG please, and then explain why", None),
("平铺", None),
(" look at [IMPORTANT: Background process] in the log ", None),
],
)
def test_ordinary_text_is_left_as_it_was(text: str, reason: ScanIssueCode | None) -> None:
cleaned, skipped = gateway_user_text(text)
assert skipped is reason
assert cleaned == ("" if reason else text.strip())
def _state_db(root: Path, rows: list[tuple[int, str, str, str]]) -> None:
root.mkdir()
connection = sqlite3.connect(root / "state.db")
connection.execute(
"CREATE TABLE messages (id INTEGER PRIMARY KEY, session_id TEXT, role TEXT,"
" content TEXT, timestamp REAL, tool_name TEXT, active INTEGER)"
)
connection.executemany(
"INSERT INTO messages VALUES (?, ?, ?, ?, 1000, ?, 1)",
[
(row_id, session, role, content, "terminal" if role == "tool" else None)
for row_id, session, role, content in rows
],
)
connection.commit()
connection.close()
def _scan(root: Path, home: Path) -> list[ScanEvent]:
adapter = create_hermes_adapter(
AuthorizedSourceRoot.grant(
source_id=HERMES_DESCRIPTOR.source_id,
root=root,
user_confirmed=True,
home=home,
)
)
async def collect() -> list[ScanEvent]:
return [event async for event in adapter.scan(ScanOptions(cursors={}))]
return asyncio.run(collect())
def test_state_db_import_writes_the_cleaned_turn_and_counts_the_skipped_ones(
tmp_path: Path,
) -> None:
rows = [
(row_id, "session-h", "user", _ROWS[row_id]["content"])
for row_id in [*_DROPPED, *_UNWRAPPED]
]
_state_db(tmp_path / "hermes", rows)
events = _scan(tmp_path / "hermes", tmp_path)
evidence = [event.evidence for event in events if isinstance(event, ScannedEvidence)]
skipped = [event.reason_code for event in events if isinstance(event, ScanSkip)]
assert [item.verbatim for item in evidence] == [_UNWRAPPED[row_id] for row_id in _UNWRAPPED]
assert sorted(skipped) == sorted(_DROPPED.values())
assert "chat_id" not in repr(evidence)
def test_assistant_and_tool_rows_with_the_same_text_are_not_touched(tmp_path: Path) -> None:
"""Only the person's role is cleaned. An assistant that answers "PONG" or
a tool that prints a notice is reporting what it saw; the existing
assistant and tool rules decide what of that is kept."""
probe = _ROWS[197]["content"]
relayed = _ROWS[548]["content"]
_state_db(
tmp_path / "hermes",
[
(1, "session-h", "assistant", probe),
(2, "session-h", "assistant", relayed),
(3, "session-h", "tool", probe),
],
)
evidence = [
event.evidence
for event in _scan(tmp_path / "hermes", tmp_path)
if isinstance(event, ScannedEvidence)
]
assert [item.role for item in evidence] == ["assistant", "assistant", "tool"]
assert evidence[0].verbatim == probe
assert evidence[1].verbatim == relayed
assert probe in evidence[2].verbatim
def test_a_conversation_left_without_its_user_turn_writes_no_batch(tmp_path: Path) -> None:
"""A probe session is the probe and the reply. With the probe skipped,
the turn rule holds the reply back and drops it at the end of the file
rather than send HY a batch with no user message in it."""
_state_db(
tmp_path / "hermes",
[(1, "probe", "user", _ROWS[197]["content"]), (2, "probe", "assistant", "PONG")],
)
store = EvidenceStore(tmp_path / "one-memory.sqlite3")
records = [
store.put(event.evidence).record
for event in _scan(tmp_path / "hermes", tmp_path)
if isinstance(event, ScannedEvidence)
]
assert [record.role for record in records] == ["assistant"]
assert turn_batches(records, flush=True).batches == ()
@pytest.mark.parametrize("role", ["user", "assistant"])
def test_session_transcripts_follow_the_same_rule(role: str) -> None:
context = SourceRecordContext(
file_key="hermes",
relative_path="sessions/session.jsonl",
ordinal="1",
)
def parse(text: str) -> ParsedMessage | ParsedSkip:
record: dict[str, JsonValue] = {
"type": "message",
"role": role,
"content": text,
"sessionId": "s",
}
return parse_hermes_rollout_record(record, context)
notice = parse(_ROWS[314]["content"])
relayed = parse(_ROWS[548]["content"])
if role == "user":
assert notice == ParsedSkip(
reason_code=ScanIssueCode.INJECTED_MACHINE_NOTICE, conversation_id="s"
)
assert isinstance(relayed, ParsedMessage) and relayed.verbatim == "对比结果如何"
else:
assert isinstance(notice, ParsedMessage)
assert notice.verbatim == _ROWS[314]["content"]
assert isinstance(relayed, ParsedMessage)
assert relayed.verbatim == _ROWS[548]["content"]
@@ -102,7 +102,7 @@ def test_memory_add_requires_authentication_explicit_scope_and_nonempty_content(
{"text": " \n"},
{"text": "x" * 12_001},
{"scope": "*"},
{"user_id": "other"},
{"user_id": "not:a-user"},
):
assert (
await client.post("/api/v2/memories", json={**base, **change})
@@ -26,6 +26,7 @@ def snapshot() -> ModelProfileSnapshot:
base_url="http://127.0.0.1:11434",
model="memory-model",
embedding_model="embedding-model",
embedding_base_url="http://127.0.0.1:11434",
embedding_dimensions=768,
timeout_seconds=30,
max_concurrency=1,
@@ -0,0 +1,861 @@
"""The memory mode (lite / pro / ultra), evolve and its nightly schedule.
Item 5: the mode is a saved setting (default pro, `ONE_MEMORY_GRAPH_STORE`
still forces ultra), it decides the mode every SDK client is built with, and
a change rebuilds the handles. lite needs no generation model, never calls
`add_extracted`, and keeps every raw. Item 6 (P1-3 / R7): `POST
/api/v2/evolve` is ultra only, one run per user at a time, never beside that
user's write, and loops while the SDK says `should_continue`; the schedule
runs nightly and catches up after sleep. R10: the status carries
`pipeline.mode` and the last evolve.
"""
from __future__ import annotations
import asyncio
import json
from contextlib import asynccontextmanager
from datetime import UTC, datetime, timedelta
from pathlib import Path
from types import SimpleNamespace
from uuid import uuid4
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory import hy_sdk_runtime, sdk_write_processor
from one_memory.api import _sdk_client_handle, create_app, memory_mode_effects
from one_memory.config import Settings
from one_memory.hy_graph_admin import effective_memory_mode
from one_memory.imports.models import (
EvidenceContentKind,
EvidenceInput,
EvidenceOrigin,
OriginMetadata,
WrittenBy,
materialize_write_input,
)
from one_memory.memory_evolution import (
EvolveModeError,
EvolveScheduler,
EvolveService,
EvolveStateStore,
MemoryGraphBackfill,
evolve_due,
)
from one_memory.memory_server.config_state import ConfigStore, EvolveSchedule
from one_memory.sdk_memory_runtime import SdkMemoryRuntime
from one_memory.secrets import InMemorySecretStore
TOKEN = "memory-mode-test"
# --- the setting and the client mode -----------------------------------------
def test_mode_defaults_to_pro_and_the_env_still_forces_ultra(tmp_path, monkeypatch) -> None:
monkeypatch.delenv("ONE_MEMORY_GRAPH_STORE", raising=False)
store = ConfigStore(tmp_path)
assert store.memory_mode() == "pro"
assert effective_memory_mode(store.memory_mode()) == "pro"
store.set_memory_mode("lite")
assert ConfigStore(tmp_path).memory_mode() == "lite"
assert json.loads((tmp_path / "config.json").read_text())["memory_mode"] == "lite"
monkeypatch.setenv("ONE_MEMORY_GRAPH_STORE", "kuzu")
assert effective_memory_mode("lite") == "ultra"
monkeypatch.setenv("ONE_MEMORY_GRAPH_STORE", "none")
assert effective_memory_mode("lite") == "lite"
with pytest.raises(ValueError):
store.set_memory_mode("max") # type: ignore[arg-type]
def test_an_untouched_install_writes_no_new_settings(tmp_path) -> None:
store = ConfigStore(tmp_path)
store.set_sync("cursor", True)
saved = json.loads((tmp_path / "config.json").read_text())
assert "memory_mode" not in saved and "evolve_schedule" not in saved
def test_acquire_client_builds_the_sdk_client_in_the_given_mode(tmp_path, monkeypatch) -> None:
built: list[str] = []
class Client:
def __init__(self, *, config, mode, runtime, embedding_service):
built.append(mode)
def close(self):
return None
monkeypatch.setattr(hy_sdk_runtime, "HyMemoryClient", Client)
monkeypatch.setattr(hy_sdk_runtime, "_acquire_runtime", lambda config, key: object())
monkeypatch.setattr(hy_sdk_runtime, "_release_runtime", lambda key: None)
monkeypatch.setattr(hy_sdk_runtime, "inspect_graph_width_mismatch", lambda config: None)
monkeypatch.delenv("ONE_MEMORY_GRAPH_STORE", raising=False)
profile = hy_sdk_runtime.builtin_retrieval_snapshot()
for mode in ("lite", "pro", "ultra"):
hy_sdk_runtime.acquire_client(profile, api_key=None, data_dir=tmp_path, mode=mode).close()
hy_sdk_runtime.acquire_client(profile, api_key=None, data_dir=tmp_path).close()
monkeypatch.setenv("ONE_MEMORY_GRAPH_STORE", "kuzu")
hy_sdk_runtime.acquire_client(profile, api_key=None, data_dir=tmp_path).close()
assert built == ["lite", "pro", "ultra", "pro", "ultra"]
def test_the_read_handle_rebuilds_on_a_mode_or_profile_change(monkeypatch, tmp_path) -> None:
"""Reads follow the active profile's embedding (#255) and the memory mode."""
from one_memory import sdk_read_handle
modes = ["pro"]
made: list[_FakeHandle] = []
def handle_for(profile, *, api_key, data_dir, mode=None):
handle = _FakeHandle(mode or "pro", profile.profile_id)
made.append(handle)
return handle
monkeypatch.setattr(sdk_read_handle, "SdkClientHandle", handle_for)
profiles = _Profiles(identity=None)
reads = sdk_read_handle.ActiveProfileReadHandle(
profiles=profiles, # type: ignore[arg-type]
secrets=None, # type: ignore[arg-type]
data_dir=tmp_path,
memory_mode=lambda: modes[0],
)
async def read():
return await reads.client()
async def scenario():
first = await asyncio.create_task(read())
assert (first.mode, first.profile_id) == ("pro", "builtin-retrieval")
modes[0] = "lite"
# A read still in flight on the old client keeps it open.
hold = asyncio.Event()
holding_client = asyncio.Event()
async def holding():
await reads.client()
holding_client.set()
await hold.wait()
modes[0] = "pro"
holder = asyncio.create_task(holding())
await asyncio.wait_for(holding_client.wait(), 5)
modes[0] = "lite"
try:
assert (await asyncio.create_task(read())).mode == "lite"
assert not made[0].closed
finally:
hold.set()
await holder
for _ in range(10):
await asyncio.sleep(0)
assert made[0].closed
assert [item.mode for item in made] == ["pro", "lite"]
await reads.close()
assert made[1].closed
asyncio.run(scenario())
class _FakeHandle:
def __init__(self, mode: str, profile_id: str = "p") -> None:
self.mode = mode
self.profile_id = profile_id
self.closed = False
async def client(self):
return self
async def close(self):
self.closed = True
class _Profiles:
def __init__(self, identity=("profile-a", 1)) -> None:
self.identity = identity
def active_identity(self):
return self.identity
def snapshot(self, profile_id, revision):
return SimpleNamespace(profile_id=profile_id, has_api_key=False)
def resolve_snapshot_secret(self, profile_id, revision, secrets):
return None
def _coordinator(monkeypatch, modes, profiles=None):
opened: list[_FakeHandle] = []
def handle(profile, **kwargs):
value = _FakeHandle(kwargs.get("mode") or "pro", profile.profile_id)
opened.append(value)
return value
monkeypatch.setattr(sdk_write_processor, "SdkClientHandle", handle)
coordinator = sdk_write_processor.SdkWriteCoordinator(
profiles=profiles or _Profiles(), # type: ignore[arg-type]
secrets=None, # type: ignore[arg-type]
data_dir=None,
transport_factory=lambda _profile: None,
sedimentation_enabled=lambda: True,
memory_mode=lambda: modes[0],
)
return coordinator, opened
def test_a_mode_change_retires_idle_write_handles_and_keeps_a_busy_one(monkeypatch) -> None:
modes = ["pro"]
coordinator, opened = _coordinator(monkeypatch, modes)
async def scenario():
async with coordinator.use_client() as client:
assert client.mode == "pro"
modes[0] = "ultra"
await coordinator.retire_stale_modes()
# Still in use: closed only when this use ends (#218).
assert not opened[0].closed
assert opened[0].closed
async with coordinator.use_client() as client:
assert client.mode == "ultra"
modes[0] = "lite"
await coordinator.retire_stale_modes()
assert opened[1].closed
await coordinator.close()
asyncio.run(scenario())
def test_lite_needs_no_generation_model(monkeypatch) -> None:
modes = ["lite"]
coordinator, opened = _coordinator(monkeypatch, modes, _Profiles(identity=None))
async def scenario():
async with coordinator.use_client() as client:
assert client.mode == "lite"
assert client.profile_id == hy_sdk_runtime.BUILTIN_RETRIEVAL_PROFILE_ID
modes[0] = "pro"
with pytest.raises(RuntimeError, match="model_profile_not_active"):
async with coordinator.use_client():
pass
await coordinator.close()
asyncio.run(scenario())
def _record(text: str, *, extracted: bool, user_id: str = "local", role: str = "user"):
record = materialize_write_input(
EvidenceInput(
user_id=user_id,
source_id="one-memory-manual" if extracted else "hermes",
conversation_id="session-1",
message_id=str(uuid4()),
role=role,
written_by=WrittenBy.PERSON,
created_at=datetime(2026, 10, 1, tzinfo=UTC),
verbatim=text,
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
)
if extracted:
record = record.model_copy(update={"content_kind": EvidenceContentKind.MEMORY_CONTENT})
return record
class _WritingSdk:
def __init__(self) -> None:
self.calls: list[tuple[str, object]] = []
def add(self, messages, **kwargs):
self.calls.append(("add", messages))
return {"success": True, "raw_ids": ["raw-1"]}
def add_extracted(self, contents, **kwargs):
self.calls.append(("add_extracted", contents))
return {"success": True, "memory_ids": ["m-1"]}
class _SdkHandle:
def __init__(self, sdk, mode="pro") -> None:
self.sdk = sdk
self.mode = mode
async def client(self):
return self.sdk
async def close(self):
return None
async def _noop() -> None:
return None
def test_a_note_in_lite_is_stored_as_raw_and_never_calls_add_extracted() -> None:
sdk = _WritingSdk()
note = (_record("I prefer green tea", extracted=True),)
async def scenario():
await sdk_write_processor.SdkWriteProcessor(_SdkHandle(sdk), lite=True).process_sdk_job(
note, request_id="r-lite", before_hy_invocation=_noop
)
await sdk_write_processor.SdkWriteProcessor(_SdkHandle(sdk)).process_sdk_job(
note, request_id="r-pro", before_hy_invocation=_noop
)
asyncio.run(scenario())
assert sdk.calls == [
("add", [{"role": "user", "content": "I prefer green tea"}]),
("add_extracted", ["I prefer green tea"]),
]
def test_a_lite_write_runs_through_the_coordinator_without_a_model(monkeypatch) -> None:
sdk = _WritingSdk()
coordinator = sdk_write_processor.SdkWriteCoordinator(
profiles=_Profiles(identity=None), # type: ignore[arg-type]
secrets=None, # type: ignore[arg-type]
data_dir=None,
transport_factory=lambda _profile: None,
sedimentation_enabled=lambda: True,
memory_mode=lambda: "lite",
)
monkeypatch.setattr(
sdk_write_processor, "SdkClientHandle", lambda profile, **kwargs: _SdkHandle(sdk, "lite")
)
turn = (
_record("hello", extracted=False),
_record("hi there", extracted=False, role="assistant"),
)
receipt = asyncio.run(
coordinator.process_sdk_job(turn, request_id="r-1", before_hy_invocation=_noop)
)
assert receipt.raw_ids == ("raw-1",)
assert [name for name, _ in sdk.calls] == ["add"]
def test_no_raw_is_ever_deleted_in_any_mode() -> None:
"""Owner decision: MemHub keeps every raw; there is no retention window."""
from one_memory import sdk_memory_runtime
for name in ("prune_expired_raw", "start_raw_retention", "_raw_retention_loop"):
assert not hasattr(SdkMemoryRuntime, name), name
assert not hasattr(sdk_memory_runtime, "RAW_RETENTION_DAYS")
def test_mode_switch_effects() -> None:
assert memory_mode_effects("lite", "pro") == (
"earlier_raw_not_extracted",
"lite_search_stops",
)
assert memory_mode_effects("ultra", "pro") == ("higher_mode_memories_hidden", "evolve_stopped")
assert memory_mode_effects("pro", "ultra", graph_backfill=True) == (
"existing_memories_added_to_graph",
)
assert memory_mode_effects("pro", "ultra", graph_backfill=False) == (
"graph_backfill_already_running",
)
assert memory_mode_effects("pro", "lite") == ("higher_mode_memories_hidden",)
assert memory_mode_effects("pro", "pro") == ()
# --- evolve ---------------------------------------------------------------------
class _EvolveSdk:
"""`evolve` answers `should_continue` for the first rounds, then stops."""
def __init__(self, continuing: int = 2, *, gate: asyncio.Event | None = None) -> None:
self.calls: list[dict] = []
self.continuing = continuing
self.gate = gate
self.graph_builds: list[str] = []
self.active = 0
self.peak = 0
def evolve(self, user_id, agent_id="default_agent", max_batches=None):
self.active += 1
self.peak = max(self.peak, self.active)
try:
self.calls.append(
{"user_id": user_id, "agent_id": agent_id, "max_batches": max_batches}
)
more = len([call for call in self.calls if call["user_id"] == user_id]) <= (
self.continuing
)
return {
"success": True,
"request_id": f"evolve-{len(self.calls)}",
"tasks_processed": 2,
"results": {"schemas": ["private text"]},
"cross_domain_sweeper": {"merged": 1},
"should_continue": more,
"continuation_reason": "remaining_clusters" if more else None,
"remaining_clusters": 3 if more else 0,
"remaining_actionable_clusters": 1 if more else 0,
"remaining_fresh_facts": 0,
"cycle_status": "partial" if more else "complete",
"batch_failures": [],
"elapsed_ms": 5.0,
}
finally:
self.active -= 1
def build_memory_graph(self, user_id, *, agent_id=None, limit=10000, clear_first=False):
self.graph_builds.append(user_id)
return {"success": True, "memories_scanned": 4, "nodes": 4}
class _Coordinator:
"""The real per-user exclusivity over a stand-in client."""
def __init__(self, sdk, monkeypatch) -> None:
self.inner, _ = _coordinator(monkeypatch, ["ultra"])
self.sdk = sdk
def exclusive_for_evolve(self, user_id):
return self.inner.exclusive_for_evolve(user_id)
@asynccontextmanager
async def use_client(self):
yield self.sdk
def test_evolve_refuses_outside_ultra(tmp_path, monkeypatch) -> None:
service = EvolveService(
coordinator=_Coordinator(_EvolveSdk(), monkeypatch),
state=EvolveStateStore(tmp_path),
memory_mode=lambda: "pro",
)
async def scenario():
with pytest.raises(EvolveModeError):
service.start("local")
asyncio.run(scenario())
def test_evolve_loops_while_should_continue_and_records_counts_only(tmp_path, monkeypatch) -> None:
sdk = _EvolveSdk(continuing=2)
state = EvolveStateStore(tmp_path)
service = EvolveService(
coordinator=_Coordinator(sdk, monkeypatch), state=state, memory_mode=lambda: "ultra"
)
async def scenario():
started = service.start("wife", max_batches=2)
assert started.state == "running"
done = await service.wait(started.run_id)
assert done is not None
return done
done = asyncio.run(scenario())
assert done.state == "succeeded" and done.rounds == 3 and done.tasks_processed == 6
assert done.cycle_status == "complete" and done.should_continue is False
assert {call["agent_id"] for call in sdk.calls} == {"all-apps"}
assert {call["max_batches"] for call in sdk.calls} == {2}
saved = (tmp_path / "evolve-state.json").read_text()
assert "private text" not in saved
assert json.loads(saved)["users"]["wife"]["last_result"]["rounds"] == 3
def test_evolve_stops_at_the_round_cap(tmp_path, monkeypatch) -> None:
sdk = _EvolveSdk(continuing=100)
service = EvolveService(
coordinator=_Coordinator(sdk, monkeypatch),
state=EvolveStateStore(tmp_path),
memory_mode=lambda: "ultra",
max_rounds=4,
)
async def scenario():
return await service.wait(service.start("local").run_id)
done = asyncio.run(scenario())
assert done is not None and done.rounds == 4 and done.capped is True
assert done.state == "succeeded"
def test_one_run_per_user_and_never_beside_that_users_write(tmp_path, monkeypatch) -> None:
sdk = _EvolveSdk(continuing=0)
coordinator = _Coordinator(sdk, monkeypatch)
service = EvolveService(
coordinator=coordinator, state=EvolveStateStore(tmp_path), memory_mode=lambda: "ultra"
)
writing = asyncio.Event()
release = asyncio.Event()
async def fake_write(self, records, *, request_id, before_hy_invocation):
writing.set()
await release.wait()
return sdk_write_processor.SdkWriteReceipt()
monkeypatch.setattr(sdk_write_processor.SdkWriteProcessor, "process_sdk_job", fake_write)
record = (_record("hello", extracted=False, user_id="wife"),)
async def scenario():
inner = coordinator.inner
write = asyncio.create_task(
inner.process_sdk_job(record, request_id="w", before_hy_invocation=_noop)
)
await writing.wait()
first = service.start("wife")
again = service.start("wife")
assert again.run_id == first.run_id
other = service.start("local")
await service.wait(other.run_id)
# The other user is not held; wife's run waits for wife's write.
assert sdk.calls and {call["user_id"] for call in sdk.calls} == {"local"}
assert inner.evolving("wife")
with pytest.raises(RuntimeError, match="memory_evolve_in_progress"):
await inner.process_sdk_job(record, request_id="w2", before_hy_invocation=_noop)
release.set()
await write
done = await service.wait(first.run_id)
assert done is not None and done.state == "succeeded"
assert not inner.evolving("wife")
assert sdk.peak == 1
asyncio.run(scenario())
def test_a_failed_round_records_its_code(tmp_path, monkeypatch) -> None:
class Failing(_EvolveSdk):
def evolve(self, user_id, agent_id="default_agent", max_batches=None):
return {
"success": False,
"error_code": "schema_llm_failed",
"failure_kind": "llm_error",
"error_message": "the provider said something private",
}
state = EvolveStateStore(tmp_path)
service = EvolveService(
coordinator=_Coordinator(Failing(), monkeypatch), state=state, memory_mode=lambda: "ultra"
)
done = asyncio.run(service.wait(_start(service, "local")))
assert done is not None and done.state == "failed"
assert done.error_code == "schema_llm_failed" and done.failure_kind == "llm_error"
assert state.user("local").last_error == "schema_llm_failed"
assert "private" not in (tmp_path / "evolve-state.json").read_text()
def _start(service: EvolveService, user_id: str) -> str:
async def begin() -> str:
return service.start(user_id).run_id
loop = asyncio.new_event_loop()
try:
run_id = loop.run_until_complete(begin())
loop.run_until_complete(service.wait(run_id))
finally:
loop.close()
return run_id
# --- the nightly schedule -------------------------------------------------------
LOCAL = datetime.now().astimezone().tzinfo
def _at(day: int, hour: int, minute: int = 0) -> datetime:
return datetime(2026, 11, day, hour, minute, tzinfo=LOCAL)
def test_evolve_is_due_after_the_slot_once_a_day_and_catches_up_after_sleep() -> None:
schedule = EvolveSchedule()
anchor = _at(1, 12)
# Never ran: not at noon on the day ultra was enabled, then at 03:00 next day.
assert not evolve_due(_at(1, 13), schedule, None, anchor)
assert not evolve_due(_at(2, 2, 59), schedule, None, anchor)
assert evolve_due(_at(2, 3, 0), schedule, None, anchor)
# Ran at 03:00: not again that day, due again the next night.
assert not evolve_due(_at(2, 22), schedule, _at(2, 3, 1), anchor)
assert evolve_due(_at(3, 3, 5), schedule, _at(2, 3, 1), anchor)
# Asleep at 03:00: a run older than the slot is due on wake, the same day.
assert evolve_due(_at(3, 9), schedule, _at(2, 3, 1), anchor)
# Last run more than 24h ago is due even before today's slot.
assert evolve_due(_at(4, 1), schedule, _at(2, 3, 1), anchor)
assert not evolve_due(_at(3, 1), schedule, _at(2, 3, 1), anchor)
# Moved and turned off.
assert evolve_due(_at(2, 23, 30), EvolveSchedule(time="23:30"), _at(2, 3), anchor)
assert not evolve_due(_at(3, 9), EvolveSchedule(enabled=False), _at(1, 3), anchor)
def test_the_scheduler_runs_each_due_user_once_in_ultra_only(tmp_path, monkeypatch) -> None:
sdk = _EvolveSdk(continuing=0)
state = EvolveStateStore(tmp_path)
modes = ["pro"]
clock = [_at(1, 12)]
service = EvolveService(
coordinator=_Coordinator(sdk, monkeypatch),
state=state,
memory_mode=lambda: modes[0],
now=lambda: clock[0],
)
schedule = [EvolveSchedule()]
async def users():
return ("local", "wife")
scheduler = EvolveScheduler(
service=service,
schedule=lambda: schedule[0],
users_with_memories=users,
now=lambda: clock[0],
)
async def scenario():
assert await scheduler.tick() == () # pro: nothing
modes[0] = "ultra"
assert await scheduler.tick() == () # anchored at noon
clock[0] = _at(2, 3, 2)
assert await scheduler.tick() == ("local", "wife")
assert await scheduler.tick() == () # once a night
clock[0] = _at(3, 8) # slept through 03:00
assert await scheduler.tick() == ("local", "wife")
schedule[0] = EvolveSchedule(enabled=False)
clock[0] = _at(5, 8)
assert await scheduler.tick() == ()
asyncio.run(scenario())
assert [call["user_id"] for call in sdk.calls] == ["local", "wife", "local", "wife"]
assert state.user("wife").last_result["trigger"] == "scheduled"
def test_the_graph_backfill_runs_once_per_user(tmp_path, monkeypatch) -> None:
sdk = _EvolveSdk()
state = EvolveStateStore(tmp_path)
backfill = MemoryGraphBackfill(coordinator=_Coordinator(sdk, monkeypatch), state=state)
async def scenario():
backfill.start(["local", "wife", "local"])
await backfill.wait()
asyncio.run(scenario())
assert sdk.graph_builds == ["local", "wife"]
built = state.graph_build()
assert built.state == "succeeded"
assert {user: value.state for user, value in built.users.items()} == {
"local": "succeeded",
"wife": "succeeded",
}
# --- the HTTP surface -----------------------------------------------------------
def application(tmp_path: Path):
settings = Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr(TOKEN),
imports_enabled=False,
automatic_memory_processing=False,
live_sync_enabled=False,
)
return create_app(settings, secret_store=InMemorySecretStore())
def client_for(app) -> AsyncClient:
return AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": f"Bearer {TOKEN}"},
)
class _ReadSdk(_EvolveSdk):
def memory_stats(self, *, user_id, statuses=None):
return {"total": 1, "scenes": {}}
def run_app(tmp_path, monkeypatch, scenario) -> None:
monkeypatch.delenv("ONE_MEMORY_GRAPH_STORE", raising=False)
async def main() -> None:
app = application(tmp_path)
sdk = _ReadSdk(continuing=1)
@asynccontextmanager
async def write_client():
yield sdk
async with app.router.lifespan_context(app):
monkeypatch.setattr(app.state.sdk_write_coordinator, "use_client", write_client)
async with client_for(app) as client:
await scenario(app, sdk, client)
asyncio.run(main())
@pytest.mark.fresh_sdk
def test_mode_api_get_put_and_its_effects(tmp_path, monkeypatch) -> None:
async def scenario(app, sdk, client):
current = (await client.get("/api/v1/settings/memory-mode")).json()
assert current["mode"] == current["configured_mode"] == "pro"
assert current["forced_by_env"] is False
assert app.state.sdk_write_coordinator.memory_mode() == "pro"
lite = await client.put("/api/v1/settings/memory-mode", json={"mode": "lite"})
assert lite.status_code == 200, lite.text
body = lite.json()
assert body["mode"] == "lite" and body["previous_mode"] == "pro"
assert body["effects"] == ["higher_mode_memories_hidden"]
assert app.state.sdk_write_coordinator.memory_mode() == "lite"
assert _sdk_client_handle(app).memory_mode() == "lite"
# lite claims queued work without a generation model.
from one_memory.api import _sedimentation_claim_enabled
assert app.state.model_profile_store.active_identity() is None
app.state.history_import_gate_released = True
assert _sedimentation_claim_enabled(app, "one-memory-manual")
ultra = await client.put("/api/v1/settings/memory-mode", json={"mode": "ultra"})
assert ultra.status_code == 200, ultra.text
assert "existing_memories_added_to_graph" in ultra.json()["effects"]
await app.state.memory_graph_backfill.wait()
assert sdk.graph_builds == ["local"]
reread = (await client.get("/api/v1/settings/memory-mode")).json()
assert reread["graph_build"]["state"] == "succeeded"
for bad in ({"mode": "max"}, {"mode": "pro", "x": 1}):
assert (await client.put("/api/v1/settings/memory-mode", json=bad)).status_code == 422
assert (
await client.get("/api/v1/settings/memory-mode", params={"user_id": "x"})
).status_code == 422
monkeypatch.setenv("ONE_MEMORY_GRAPH_STORE", "kuzu")
await client.put("/api/v1/settings/memory-mode", json={"mode": "pro"})
forced = (await client.get("/api/v1/settings/memory-mode")).json()
assert forced["mode"] == "ultra" and forced["configured_mode"] == "pro"
assert forced["forced_by_env"] is True
run_app(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_evolve_api_is_ultra_only_async_and_scoped_per_user(tmp_path, monkeypatch) -> None:
async def scenario(app, sdk, client):
refused = await client.post("/api/v2/evolve", json={})
assert refused.status_code == 409
assert refused.json()["detail"] == "memory_mode_not_ultra"
await client.put("/api/v1/settings/memory-mode", json={"mode": "ultra"})
await app.state.memory_graph_backfill.wait()
started = await client.post(
"/api/v2/evolve", json={"user_id": "wife", "agent_id": "all-apps", "max_batches": 2}
)
assert started.status_code == 202, started.text
run = started.json()
assert run["state"] == "running" and run["user_id"] == "wife"
await app.state.evolve_service.wait(run["run_id"])
path = f"/api/v2/evolve/{run['run_id']}"
assert (await client.get(path)).status_code == 404
done = (await client.get(path, params={"user_id": "wife"})).json()
assert done["state"] == "succeeded" and done["rounds"] == 2
assert sdk.calls[0] == {"user_id": "wife", "agent_id": "all-apps", "max_batches": 2}
overview = (await client.get("/api/v2/evolve", params={"user_id": "wife"})).json()
assert set(overview["users"]) == {"wife"}
assert overview["schedule"] == {"enabled": True, "time": "03:00"}
processing = (await client.get("/api/v1/memory-processing")).json()
pipeline = processing["pipeline"]
assert pipeline["mode"] == "ultra"
assert pipeline["last_evolve_at"] is not None
assert pipeline["evolve"]["users"]["wife"]["last_result"]["state"] == "succeeded"
for bad in (
{"user_id": "a:b"},
{"max_batches": 0},
{"from_beginning": True},
):
assert (await client.post("/api/v2/evolve", json=bad)).status_code == 422
moved = await client.put(
"/api/v1/settings/evolve-schedule", json={"time": "04:30", "enabled": False}
)
assert moved.json() == {"enabled": False, "time": "04:30"}
assert (
await client.put("/api/v1/settings/evolve-schedule", json={"time": "25:00"})
).status_code == 422
run_app(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_a_job_written_in_lite_stays_done_and_a_pending_job_takes_the_new_mode(
tmp_path, monkeypatch
) -> None:
"""lite to pro reprocesses nothing: the mode is read when a job runs."""
from one_memory.memory_server.durable_queue import DurableWriteDrain, execute_job
modes = ["lite"]
seen: list[str] = []
class Sdk:
def __init__(self, mode):
self.mode = mode
def add(self, messages, **kwargs):
seen.append(self.mode)
return {"success": True, "raw_ids": [f"raw-{len(seen)}"]}
monkeypatch.setattr(
sdk_write_processor,
"SdkClientHandle",
lambda profile, **kwargs: _SdkHandle(Sdk(kwargs["mode"]), kwargs["mode"]),
)
coordinator = sdk_write_processor.SdkWriteCoordinator(
profiles=_Profiles(), # type: ignore[arg-type]
secrets=None, # type: ignore[arg-type]
data_dir=None,
transport_factory=lambda _profile: None,
sedimentation_enabled=lambda: True,
memory_mode=lambda: modes[0],
)
async def never(_job_id):
return None
async def drain_once(queue) -> int:
drain = DurableWriteDrain(queue, execute=never, lease=timedelta(seconds=30))
claimed = drain.claim(limit=8)
for job_id in claimed:
await execute_job(
job_id,
queue=queue,
supervisor=coordinator,
evidence=None, # type: ignore[arg-type]
)
return len(claimed)
async def scenario(app, _sdk, client):
queue = app.state.memory_write_queue
for request_id in ("turn-lite", "turn-pro"):
written = await client.post(
"/api/v2/memory-writes",
json={
"request_id": request_id,
"session_id": f"s-{request_id}",
"source_id": "one-memory-manual",
"messages": [
{"role": "user", "content": request_id},
{"role": "assistant", "content": "ok"},
],
},
)
assert written.status_code == 202, written.text
if request_id == "turn-lite":
assert await drain_once(queue) == 1
modes[0] = "pro"
await coordinator.retire_stale_modes()
assert await drain_once(queue) == 1
assert await drain_once(queue) == 0
assert queue.get("turn-lite").state == queue.get("turn-pro").state == "completed"
assert seen == ["lite", "pro"]
await coordinator.close()
run_app(tmp_path, monkeypatch, scenario)
+4
View File
@@ -66,6 +66,7 @@ def _grounded_setup(tmp_path, *, memory_id: str = "memory-real-data"):
base_url="http://127.0.0.1:11434",
model="test-model",
embedding_model="test-embedding",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -263,6 +264,7 @@ def test_answer_uses_only_grounded_confirmed_memory_citations(tmp_path) -> None:
base_url="http://127.0.0.1:11434",
model="test-model",
embedding_model="test-embedding",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -468,6 +470,7 @@ def test_answer_rejects_model_citation_outside_retrieved_scope(tmp_path) -> None
base_url="http://127.0.0.1:11434",
model="test-model",
embedding_model="test-embedding",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -530,6 +533,7 @@ def _compatible_gateway_setup(tmp_path):
base_url="https://api.example.invalid/standard/v1",
model="deepseek-v4-pro",
embedding_model="text-embedding",
embedding_base_url="https://api.example.invalid/standard/v1",
api_key="probe-key",
),
secrets,
+1 -1
View File
@@ -56,7 +56,7 @@ def test_sdk_ask_v2_stream_is_native(tmp_path, monkeypatch) -> None:
)
class Answerer:
async def answer_stream(self, _question):
async def answer_stream(self, _question, *, user_id="local"):
yield {"type": "recall", "memories": (memory,)}
yield {"type": "delta", "text": "你要求真实数据。"}
yield {
@@ -186,6 +186,7 @@ def _setup(tmp_path: Path, *, model: str = "available") -> _Setup:
base_url="https://memory-qa.example.invalid/v1",
model="synthetic-generation",
embedding_model="synthetic-embedding",
embedding_base_url="https://memory-qa.example.invalid/v1",
api_key=SecretStr(_SECRET),
),
secrets,
@@ -25,6 +25,7 @@ def _stores(tmp_path):
provider_kind=ProviderKind.OLLAMA,
model="hy-bot-phi4",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -83,6 +83,7 @@ def test_task_freezes_active_profile_revision_and_is_idempotent(tmp_path) -> Non
provider_kind=ProviderKind.OLLAMA,
model="qwen",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -240,6 +241,7 @@ def test_task_extraction_checkpoint_is_idempotent_and_recoverable(tmp_path) -> N
provider_kind=ProviderKind.OLLAMA,
model="qwen",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
),
@@ -293,6 +295,7 @@ def test_task_waits_for_model_and_retries_with_same_snapshot(tmp_path) -> None:
provider_kind=ProviderKind.OLLAMA,
model="qwen",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -337,6 +340,7 @@ def test_blocked_stages_name_the_stuck_stage_and_reason(tmp_path) -> None:
provider_kind=ProviderKind.OLLAMA,
model="qwen",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
),
@@ -433,6 +437,7 @@ def test_task_duration_estimate_ignores_time_the_app_was_closed(tmp_path) -> Non
provider_kind=ProviderKind.OLLAMA,
model="qwen",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
),
@@ -537,6 +542,7 @@ def test_running_task_is_recovered_after_worker_restart(tmp_path) -> None:
provider_kind=ProviderKind.OLLAMA,
model="qwen",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
),
@@ -572,6 +578,7 @@ def test_superseded_profile_cancels_only_claimable_tasks(tmp_path) -> None:
provider_kind=ProviderKind.OLLAMA,
model="phi",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
),
@@ -601,6 +608,7 @@ def test_superseded_profile_cancels_only_claimable_tasks(tmp_path) -> None:
provider_kind=ProviderKind.OLLAMA,
model="qwen",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
),
@@ -630,6 +638,7 @@ def test_superseded_ingest_policy_cancels_only_old_conversation_tasks(tmp_path)
provider_kind=ProviderKind.OLLAMA,
model="qwen",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
),
@@ -676,6 +685,7 @@ def test_task_rejects_inactive_profile_and_request_id_reuse(tmp_path) -> None:
provider_kind=ProviderKind.OLLAMA,
model="qwen",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -1700,6 +1710,7 @@ def _profile_and_tasks(tmp_path) -> tuple[MemoryTaskStore, ModelProfile]:
provider_kind=ProviderKind.OLLAMA,
model="qwen",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
),
@@ -2058,6 +2069,7 @@ def test_a_failed_task_below_the_ceiling_is_still_claimable(tmp_path) -> None:
provider_kind=ProviderKind.OLLAMA,
model="qwen",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
),
@@ -55,6 +55,7 @@ def library(tmp_path):
provider_kind=ProviderKind.OLLAMA,
model="hy-bot-phi4",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -611,3 +612,58 @@ def test_granting_a_source_repeatedly_does_not_raise(library):
with main_database_read(database) as db:
assert memory.memory_id in pending(db)
def test_grant_writes_still_work_after_the_cutover_drops_the_link_table(library):
"""#233: the grant triggers read `governed_memory_evidence`.
The SDK cutover drops it, and SQLite resolves a trigger body only when it
fires, so every grant, revoke and connect then failed with "no such
table". A trigger whose body names a missing table is not installed, and
the copy an earlier startup installed is dropped.
"""
database, *_ = library
from one_memory.imports.candidates import RootCandidateKind
from one_memory.imports.grants import SourceGrantStore
from one_memory.memory_visibility import drop_stale_triggers
startup(database)
with sqlite3.connect(database) as db:
db.execute("DROP TABLE governed_memory_evidence")
def grant_triggers() -> set[str]:
with sqlite3.connect(database) as db:
return {
str(row[0])
for row in db.execute(
"SELECT name FROM sqlite_master "
"WHERE type='trigger' AND tbl_name='source_grants'"
)
}
assert grant_triggers() == {
"memory_visibility_dirty_on_grants",
"memory_visibility_dirty_on_grants_update",
}
# The `evidence` triggers read the dropped table too; the cutover drops
# `evidence` as well, which takes them with it.
assert {
"memory_visibility_dirty_on_grants",
"memory_visibility_dirty_on_grants_update",
} <= set(drop_stale_triggers(database))
assert grant_triggers() == set()
assert drop_stale_triggers(database) == ()
# Installing again on this shape skips them instead of bringing them back.
startup(database)
assert grant_triggers() == set()
folder = database.parent / "cursor"
folder.mkdir(exist_ok=True)
store = SourceGrantStore(database, home=database.parent)
store.grant(
user_id="local",
source_id="cursor",
root=folder,
candidate_kind=RootCandidateKind.DEFAULT,
)
assert store.revoke(user_id="local", source_id="cursor").revoked_at is not None
@@ -51,6 +51,7 @@ def library(tmp_path):
provider_kind=ProviderKind.OLLAMA,
model="hy-bot-phi4",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -28,6 +28,7 @@ def test_probe_still_measures_embedding_when_structured_output_fails(tmp_path) -
base_url="http://127.0.0.1:11434",
model="plain-model",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -63,6 +64,7 @@ def test_probe_records_the_real_embedding_width_on_the_profile(tmp_path) -> None
base_url="http://127.0.0.1:11434",
model="qwen3:8b",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -119,6 +121,7 @@ def test_openai_compatible_probe_checks_model_and_structured_output(tmp_path) ->
base_url="https://models.example.com/v1",
model="memory-model",
embedding_model="embedding-model",
embedding_base_url="https://models.example.com/v1",
api_key=SecretStr(API_KEY),
),
secrets,
@@ -260,6 +263,7 @@ def test_probe_reports_missing_model_without_sending_generation(tmp_path) -> Non
provider_kind=ProviderKind.OLLAMA,
model="memory-model",
embedding_model="embedding-model",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -50,6 +50,7 @@ def test_model_profile_api_never_returns_key_and_enforces_revision(tmp_path) ->
"base_url": "https://models.example.com/v1",
"model": "memory-model",
"embedding_model": "embedding-model",
"embedding_base_url": "https://models.example.com/v1",
"api_key": API_KEY,
},
)
@@ -134,6 +135,7 @@ def test_model_profile_probe_route_returns_only_machine_results(tmp_path) -> Non
"base_url": "https://models.example.com/v1",
"model": "memory-model",
"embedding_model": "embedding-model",
"embedding_base_url": "https://models.example.com/v1",
"api_key": API_KEY,
},
)
@@ -244,6 +246,7 @@ def test_activation_survives_a_corpus_already_imported(tmp_path) -> None:
"base_url": "https://models.example.com/v1",
"model": "memory-model",
"embedding_model": "embedding-model",
"embedding_base_url": "https://models.example.com/v1",
"api_key": API_KEY,
},
)
@@ -198,6 +198,7 @@ def test_active_profile_can_be_deleted_without_changing_other_profiles(tmp_path)
provider_kind=ProviderKind.OLLAMA,
model="other",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -208,6 +209,7 @@ def test_active_profile_can_be_deleted_without_changing_other_profiles(tmp_path)
base_url="https://models.example.com/v1",
model="qwen",
embedding_model="nomic-embed-text",
embedding_base_url="https://models.example.com/v1",
api_key=SecretStr(API_KEY),
),
secrets,
@@ -349,6 +351,7 @@ def _probed(store, secrets, *, structured: bool, generates: bool = True):
provider_kind=ProviderKind.OLLAMA,
model="qwen",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -394,3 +397,102 @@ def test_a_model_that_cannot_generate_still_cannot_be_turned_on(tmp_path) -> Non
with pytest.raises(ModelProfileUnavailableError):
store.activate(profile.profile_id, secrets)
def _space(store: ModelProfileStore, profile_id: str) -> str:
"""The embedding space's collection base, as the Runtime names it."""
from one_memory.hy_sdk_runtime import vector_collection_bases
identity = (profile_id, store.require(profile_id, InMemorySecretStore()).revision)
return vector_collection_bases(store.snapshot(*identity))[0]
def test_a_remote_embedding_must_name_its_endpoint(tmp_path) -> None:
"""The endpoint no longer follows the generation `base_url`."""
from one_memory.model_profiles import EmbeddingBaseUrlRequiredError
store = ModelProfileStore(tmp_path / "one-memory.sqlite3")
secrets = InMemorySecretStore()
unnamed = ModelProfileInput(
name="Remote",
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
base_url="https://models.example.com/v1",
model="qwen",
embedding_model="text-embedding-3-small",
api_key=SecretStr(API_KEY),
)
with pytest.raises(EmbeddingBaseUrlRequiredError):
store.create(unnamed, secrets)
assert store.list(secrets) == []
named = store.create(
unnamed.model_copy(update={"embedding_base_url": "https://models.example.com/v1"}),
secrets,
)
with pytest.raises(EmbeddingBaseUrlRequiredError):
store.update(
named.profile_id,
ModelProfilePatch(
expected_revision=named.revision,
embedding_provider_kind=ProviderKind.OLLAMA,
embedding_base_url="",
),
secrets,
)
def test_stored_profiles_get_the_endpoint_they_resolved_to_and_keep_their_space(
tmp_path,
) -> None:
"""An older build stored remote embeddings with a NULL endpoint and resolved
it to the generation `base_url` at runtime. Opening the store writes that
endpoint in once, so the collection, and the memories in it, stay put; a
later edit of the generation endpoint no longer moves them.
"""
from one_memory.hy_sdk_runtime import vector_collection_bases
database = tmp_path / "one-memory.sqlite3"
store = ModelProfileStore(database)
secrets = InMemorySecretStore()
profile = store.create(
ModelProfileInput(
name="Remote",
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
base_url="https://gateway.example.com/v1",
model="qwen",
embedding_model="text-embedding-3-small",
embedding_base_url="https://gateway.example.com/v1",
embedding_dimensions=1536,
api_key=SecretStr(API_KEY),
),
secrets,
)
identity = (profile.profile_id, profile.revision)
# The space the older build's Runtime resolved: `embedding_base_url or base_url`.
before = vector_collection_bases(store.snapshot(*identity))[0]
with sqlite3.connect(database) as db:
# The shape an older build stored.
db.execute("UPDATE model_profiles SET embedding_base_url=NULL")
db.execute("UPDATE model_profile_versions SET embedding_base_url=NULL")
migrated = ModelProfileStore(database)
assert migrated.snapshot(*identity).embedding_base_url == "https://gateway.example.com/v1"
assert _space(migrated, profile.profile_id) == before
# Idempotent: opening again changes nothing.
assert _space(ModelProfileStore(database), profile.profile_id) == before
with sqlite3.connect(database) as db:
assert db.execute(
"SELECT COUNT(*) FROM model_profile_versions WHERE embedding_base_url IS NULL"
).fetchone() == (0,)
edited = migrated.update(
profile.profile_id,
ModelProfilePatch(
expected_revision=profile.revision,
base_url="https://gateway.example.com/eu/v1",
),
secrets,
)
assert edited.base_url == "https://gateway.example.com/eu/v1"
assert edited.embedding_base_url == "https://gateway.example.com/v1"
assert _space(migrated, profile.profile_id) == before
@@ -0,0 +1,858 @@
"""One Runtime, many users: `user_id` on the memory API, per-user Hermes profiles.
A user is the root that is read (a Hermes profile's own `state.db`), never a
session or platform field. The App never names a user and keeps `local`;
every other harness stays on `local`. These tests drive the production app
against a user-aware stand-in for the public SDK client, whose `get`,
`update` and `delete` take no user exactly like hy-memory 2.0.9's.
"""
from __future__ import annotations
import asyncio
import json
import sqlite3
from contextlib import asynccontextmanager
from datetime import UTC, datetime, timedelta
from pathlib import Path
from uuid import uuid4
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import _sdk_recall_target, create_app
from one_memory.config import Settings
from one_memory.imports.access import AuthorizedSourceRoot
from one_memory.imports.adapters import ScannedEvidence, ScanOptions
from one_memory.imports.models import (
EvidenceInput,
EvidenceOrigin,
OriginMetadata,
WrittenBy,
materialize_write_input,
)
from one_memory.imports.sources.hermes import HERMES_DESCRIPTOR, create_hermes_adapter
from one_memory.imports.sources.registry import build_source_catalog
from one_memory.memory_server.contracts import internal_session_id_for
from one_memory.memory_server.durable_queue import (
DurableWriteDrain,
DurableWriteQueue,
execute_job,
)
from one_memory.sdk_write_processor import SdkWriteProcessor
from one_memory.secrets import InMemorySecretStore
from one_memory.user_scope import lane_key, split_lane, valid_user_id
TOKEN = "multi-user-test"
class UserAwareSdk:
"""The public client surface MemHub calls, holding memories per user."""
def __init__(self) -> None:
self.memories: dict[str, dict[str, object]] = {}
self.add_calls: list[dict[str, object]] = []
def _store(self, user_id: str, content: str, *, source: str, session_id: str) -> str:
memory_id = f"memory-{len(self.memories) + 1}"
self.memories[memory_id] = {
"memory_id": memory_id,
"content": content,
"layer": "l2_fact",
"status": "active",
"user_id": user_id,
"agent_id": "all-apps",
"session_id": session_id,
"source": source,
"tags": [],
"gmt_created": 1_790_000_000,
}
return memory_id
def add(self, messages, *, user_id, session_id, source, request_id, **_kwargs):
self.add_calls.append({"user_id": user_id, "request_id": request_id})
text = " ".join(item["content"] for item in messages if item["role"] == "user")
memory_id = self._store(user_id, text, source=source, session_id=session_id)
return {"success": True, "memory_ids": [memory_id]}
def add_extracted(self, contents, *, user_id, session_id, source, request_id, **_kwargs):
self.add_calls.append({"user_id": user_id, "request_id": request_id})
memory_id = self._store(user_id, " ".join(contents), source=source, session_id=session_id)
return {"success": True, "memory_ids": [memory_id]}
def _rows(self, user_ids) -> list[dict[str, object]]:
return [dict(row) for row in self.memories.values() if row["user_id"] in user_ids]
def list_memories(self, *, user_id, scene=None, offset=0, limit=100, **_kwargs):
rows = [] if _kwargs.get("layers") == ["l1_raw"] else self._rows({user_id})
page = rows[offset : offset + limit]
if scene == "all":
return {
"scene": "all",
"memories": [{**row, "scene": "chat", "kind": "chat"} for row in page],
"total": len(rows),
"totals": {"chat": len(rows), "working": 0, "coding": 0},
"errors": {},
"limit": limit,
"offset": offset,
"request_id": "list",
"elapsed_ms": 1.0,
}
return {"vdb": {"memories": page, "total": len(rows), "limit": limit, "offset": offset}}
def memory_stats(self, *, user_id, statuses=None):
return {"total": len(self._rows({user_id})), "request_id": "stats", "elapsed_ms": 1.0}
def search(self, query, *, user_ids, scene=None, **kwargs):
rows = [] if kwargs.get("layers") == ["l1_raw"] else self._rows(set(user_ids))
items = [{**row, "scene": "chat", "kind": "chat", "score": 0.9} for row in rows]
if scene == "all":
return {
"request_id": "search",
"scene": "all",
"memories": items,
"lanes": {"chat": len(items), "working": 0, "coding": 0},
"errors": {},
"elapsed_ms": 1.0,
}
return {"request_id": "search", "scene": scene or "normal", "memories": items}
def get(self, memory_id, kind=None, variant=None):
row = self.memories.get(memory_id)
return None if row is None else {**row, "kind": "chat", "variant": None}
def update(self, memory_id, content, kind=None, variant=None, task=None):
if memory_id not in self.memories:
return {"success": False, "error_message": f"memory_id '{memory_id}' not found"}
self.memories[memory_id]["content"] = content
return {"success": True, "memory_id": memory_id}
def delete(self, memory_id, kind=None, variant=None):
if self.memories.pop(memory_id, None) is None:
return {"success": False, "deleted_count": 0}
return {"success": True, "deleted_count": 1}
def get_user_profile(self, *, user_id):
return {
"request_id": f"profile-{user_id}",
"profile": {
"memory_id": f"l0-{user_id}",
"content": f"name is {user_id}",
"layer": "L0_BASIC_INFO",
"basic_info": {"name": user_id},
},
"elapsed_ms": 1.0,
}
class Handle:
def __init__(self, sdk: UserAwareSdk) -> None:
self.sdk = sdk
async def client(self) -> UserAwareSdk:
return self.sdk
async def close(self) -> None:
return None
class Coordinator:
"""The production write processor over the stand-in client."""
def __init__(self, sdk: UserAwareSdk) -> None:
self.processor = SdkWriteProcessor(Handle(sdk)) # type: ignore[arg-type]
async def process_sdk_job(self, records, *, request_id, before_hy_invocation):
return await self.processor.process_sdk_job(
records, request_id=request_id, before_hy_invocation=before_hy_invocation
)
async def _never(_job_id: str) -> None:
return None
async def drain_all(queue: DurableWriteQueue, sdk: UserAwareSdk) -> None:
"""Claim and run every pending job, ungated, the way the drain would."""
drain = DurableWriteDrain(queue, execute=_never, lease=timedelta(seconds=30))
while claimed := drain.claim(limit=8):
for job_id in claimed:
await execute_job(
job_id,
queue=queue,
supervisor=Coordinator(sdk),
evidence=None, # type: ignore[arg-type]
)
def application(tmp_path: Path, *, home: Path | None = None):
"""The production app; with `home`, imports run over that home's sources."""
settings = Settings(
data_dir=tmp_path / "data",
source_home=home or tmp_path,
api_token=SecretStr(TOKEN),
imports_enabled=home is not None,
automatic_memory_processing=False,
live_sync_enabled=False,
)
return create_app(
settings,
secret_store=InMemorySecretStore(),
source_catalog=(
build_source_catalog(home=home, environment={}, platform="darwin")
if home is not None
else None
),
)
def client_for(app) -> AsyncClient:
return AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": f"Bearer {TOKEN}"},
)
def run_with_sdk(tmp_path, monkeypatch, scenario) -> None:
async def main() -> None:
app = application(tmp_path)
sdk = UserAwareSdk()
@asynccontextmanager
async def write_client():
yield sdk
async with app.router.lifespan_context(app):
app.state.sdk_client_handle = Handle(sdk)
monkeypatch.setattr(app.state.sdk_write_coordinator, "use_client", write_client)
async with client_for(app) as client:
await scenario(app, sdk, client)
asyncio.run(main())
def _messages(text: str) -> list[dict[str, str]]:
return [{"role": "user", "content": text}, {"role": "assistant", "content": "noted"}]
async def _write_both_users(app, sdk, client) -> dict[str, str]:
"""`local` and `wife` each add one memory and submit one conversation."""
receipts = {}
for user, word in (("local", "tea"), ("wife", "coffee")):
request_id = str(uuid4())
body = {"request_id": request_id, "text": f"{user} drinks {word}", "scope": "all-apps"}
if user != "local":
body["user_id"] = user
added = await client.post("/api/v2/memories", json=body)
assert added.status_code == 201, added.text
receipts[f"add:{user}"] = request_id
write = {
"request_id": "same-request-id",
"session_id": "same-session",
"source_id": "one-memory-manual",
"messages": _messages(f"{user} walks the {word} route"),
}
if user != "local":
write["user_id"] = user
written = await client.post("/api/v2/memory-writes", json=write)
assert written.status_code == 202, written.text
assert written.json()["request_id"] == "same-request-id"
await drain_all(app.state.memory_write_queue, sdk)
return receipts
def _contents(page: dict) -> set[str]:
return {memory["content"] for memory in page["memories"]}
@pytest.mark.fresh_sdk
def test_each_user_reads_only_their_own_library_search_profile_and_graph(
tmp_path, monkeypatch
) -> None:
async def scenario(app, sdk, client):
await _write_both_users(app, sdk, client)
assert {call["user_id"] for call in sdk.add_calls} == {"local", "wife"}
# The same caller request id from two users is two SDK requests.
assert len({call["request_id"] for call in sdk.add_calls}) == len(sdk.add_calls) == 4
local = (await client.get("/api/v2/memory-library")).json()
wife = (await client.get("/api/v2/memory-library", params={"user_id": "wife"})).json()
assert _contents(local) == {"local drinks tea", "local walks the tea route"}
assert _contents(wife) == {"wife drinks coffee", "wife walks the coffee route"}
nobody = await client.get("/api/v2/memory-library", params={"user_id": "__nobody__"})
assert nobody.status_code == 200 and nobody.json()["memories"] == []
searched = (
await client.get("/api/v2/memory-library", params={"query": "route", "user_id": "wife"})
).json()
assert _contents(searched) == {"wife walks the coffee route"}
context = await client.post(
"/api/v2/context/search",
json={"query": "drinks", "kinds": ["memory"], "user_id": "wife"},
)
assert context.status_code == 200, context.text
assert {item["content"] for item in context.json()["items"]} == {
"wife drinks coffee",
"wife walks the coffee route",
}
local_context = await client.post(
"/api/v2/context/search", json={"query": "drinks", "kinds": ["memory"]}
)
assert all("wife" not in item["content"] for item in local_context.json()["items"])
profile = (await client.get("/api/v2/user-profile", params={"user_id": "wife"})).json()
assert profile["profile"]["basic_info"] == {"name": "wife"}
assert (await client.get("/api/v2/user-profile")).json()["profile"]["basic_info"] == {
"name": "local"
}
graph = (await client.get("/api/v2/memories/graph", params={"user_id": "wife"})).json()
labels = {node["label"] for node in graph["nodes"] if node["node_type"] == "memory"}
assert labels == {"wife drinks coffee", "wife walks the coffee route"}
summary = (await client.get("/api/v2/home-summary", params={"user_id": "wife"})).json()
assert summary["memory_count"] == 2
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_another_users_memory_id_answers_404_for_read_update_and_delete(
tmp_path, monkeypatch
) -> None:
async def scenario(app, sdk, client):
await _write_both_users(app, sdk, client)
[wife_memory] = [
memory_id
for memory_id, row in sdk.memories.items()
if row["user_id"] == "wife" and row["content"] == "wife drinks coffee"
]
path = f"/api/v2/memories/{wife_memory}"
assert (await client.get(path)).status_code == 404
assert (await client.put(path, json={"content": "stolen"})).status_code == 404
assert (await client.delete(path)).status_code == 204
assert sdk.memories[wife_memory]["content"] == "wife drinks coffee"
own = await client.get(path, params={"user_id": "wife"})
assert own.status_code == 200 and own.json()["user_id"] == "wife"
updated = await client.put(path, json={"content": "wife drinks tea", "user_id": "wife"})
assert updated.status_code == 200
assert (await client.delete(path, params={"user_id": "wife"})).status_code == 204
assert wife_memory not in sdk.memories
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_receipts_belong_to_the_user_that_submitted_them(tmp_path, monkeypatch) -> None:
async def scenario(app, sdk, client):
receipts = await _write_both_users(app, sdk, client)
wife_add = f"/api/v2/memories/additions/{receipts['add:wife']}"
assert (await client.get(wife_add)).status_code == 404
own = await client.get(wife_add, params={"user_id": "wife"})
assert own.status_code == 200 and own.json()["state"] == "ready"
local_add = f"/api/v2/memories/additions/{receipts['add:local']}"
assert (await client.get(local_add, params={"user_id": "wife"})).status_code == 404
# Both users used `same-request-id`; each reads only their own receipt.
local_write = await client.get("/api/v2/memory-writes/same-request-id")
wife_write = await client.get(
"/api/v2/memory-writes/same-request-id", params={"user_id": "wife"}
)
assert local_write.status_code == wife_write.status_code == 200
assert local_write.json()["memory_ids"] != wife_write.json()["memory_ids"]
assert (
await client.get("/api/v2/memory-writes/same-request-id", params={"user_id": "kid"})
).status_code == 404
# `local` cannot name another user's namespaced queue key directly.
with sqlite3.connect(app.state.settings.data_dir / "one-memory.sqlite3") as db:
keys = [row[0] for row in db.execute("SELECT idempotency_key FROM memory_write_jobs")]
wife_key = next(key for key in keys if key.startswith("user:wife:"))
assert (await client.get(f"/api/v2/memory-writes/{wife_key}")).status_code == 404
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_omitting_user_id_keeps_local_and_recall_and_ask_pass_it_through(
tmp_path, monkeypatch
) -> None:
async def scenario(app, sdk, client):
await _write_both_users(app, sdk, client)
seen: list[str] = []
original = sdk.list_memories
def recording(**kwargs):
seen.append(kwargs["user_id"])
return original(**kwargs)
sdk.list_memories = recording # type: ignore[method-assign]
# Built at startup over the real handle; rebuilt here over the stand-in.
app.state.recall_target = _sdk_recall_target(app, "local")
await client.get("/api/v2/memory-library")
await client.get("/api/v2/raw")
await client.get("/api/v2/home-summary")
assert set(seen) == {"local"}
monkeypatch.setattr(
"one_memory.api._recall_model_ready", lambda _app: _true(), raising=True
)
recalled = await client.post("/api/v1/recall", json={"query": "drinks", "user_id": "wife"})
assert recalled.status_code == 200, recalled.text
assert {hit["content"] for hit in recalled.json()["hits"]} == {
"wife drinks coffee",
"wife walks the coffee route",
}
local = await client.post("/api/v1/recall", json={"query": "drinks"})
assert all("wife" not in hit["content"] for hit in local.json()["hits"])
answered = await client.post(
"/api/v2/memories/ask", json={"question": "what does she drink", "user_id": "wife"}
)
assert answered.status_code == 200, answered.text
assert {memory["content"] for memory in answered.json()["memories"]} <= {
"wife drinks coffee",
"wife walks the coffee route",
}
run_with_sdk(tmp_path, monkeypatch, scenario)
async def _true() -> bool:
return True
@pytest.mark.fresh_sdk
@pytest.mark.parametrize(
("method", "path", "kwargs"),
[
("get", "/api/v2/memory-library", {"params": {"nobody": "1"}}),
("get", "/api/v2/home-summary", {"params": {"userid": "wife"}}),
("get", "/api/v2/user-profile", {"params": {"user": "wife"}}),
("get", "/api/v2/raw", {"params": {"scope": "x"}}),
("get", "/api/v2/memories/graph", {"params": {"scene": "all"}}),
("get", "/api/v2/memories/some-id", {"params": {"owner": "wife"}}),
("delete", "/api/v2/memories/some-id", {"params": {"owner": "wife"}}),
("get", "/api/v2/memory-writes/some-id", {"params": {"owner": "wife"}}),
("get", f"/api/v2/memories/additions/{uuid4()}", {"params": {"owner": "wife"}}),
("post", "/api/v2/memories/ask", {"json": {"question": "q", "user": "wife"}}),
("post", "/api/v1/recall", {"json": {"query": "q", "user": "wife"}}),
("post", "/api/v1/recall", {"json": {"query": "q"}, "params": {"user_id": "wife"}}),
("post", "/api/v2/context/search", {"json": {"query": "q", "owner": "wife"}}),
(
"post",
"/api/v2/memories",
{"json": {"request_id": str(uuid4()), "text": "t", "scope": "all-apps", "u": 1}},
),
("put", "/api/v2/memories/some-id", {"json": {"content": "c", "owner": "wife"}}),
],
)
def test_unknown_query_and_body_parameters_answer_422(tmp_path, method, path, kwargs) -> None:
async def scenario():
app = application(tmp_path)
async with app.router.lifespan_context(app), client_for(app) as client:
response = await client.request(method.upper(), path, **kwargs)
assert response.status_code == 422, response.text
assert response.json()["detail"] == "request_validation_failed"
asyncio.run(scenario())
@pytest.mark.fresh_sdk
@pytest.mark.parametrize("bad", ["a:b", "", "x" * 65, "white space", "ü", "a/b"])
def test_a_malformed_user_id_answers_422(tmp_path, bad) -> None:
async def scenario():
app = application(tmp_path)
async with app.router.lifespan_context(app), client_for(app) as client:
responses = [
await client.get("/api/v2/memory-library", params={"user_id": bad}),
await client.get("/api/v2/user-profile", params={"user_id": bad}),
await client.post(
"/api/v2/memories",
json={
"request_id": str(uuid4()),
"text": "t",
"scope": "all-apps",
"user_id": bad,
},
),
await client.post(
"/api/v2/memory-writes",
json={
"request_id": "r",
"session_id": "s",
"messages": _messages("hello"),
"user_id": bad,
},
),
await client.post("/api/v1/recall", json={"query": "q", "user_id": bad}),
]
assert [response.status_code for response in responses] == [422] * len(responses)
asyncio.run(scenario())
def test_user_ids_and_lanes() -> None:
assert valid_user_id("wife") and valid_user_id("A.b_c-9")
assert not valid_user_id("a:b") and not valid_user_id("wife\n")
assert lane_key("local", "hermes") == "hermes"
assert lane_key("wife", "hermes") == "hermes@wife"
assert split_lane("hermes@wife") == ("wife", "hermes")
assert split_lane("hermes") == ("local", "hermes")
# `local` keeps the session identity it always had; another user's never equals it.
assert internal_session_id_for("s-1", "hermes") == "s-1"
assert internal_session_id_for("s-1", "hermes", "wife") != "s-1"
assert internal_session_id_for("s-1", "hermes", "wife") != internal_session_id_for(
"s-1", "hermes", "kid"
)
def _hermes_state_db(root: Path, rows: list[tuple[int, str, str, str]]) -> None:
root.mkdir(parents=True, exist_ok=True)
connection = sqlite3.connect(root / "state.db")
connection.execute(
"CREATE TABLE messages (id INTEGER PRIMARY KEY, session_id TEXT, role TEXT,"
" content TEXT, timestamp REAL, tool_name TEXT, active INTEGER)"
)
connection.executemany(
"INSERT INTO messages VALUES (?, ?, ?, ?, 1790000000, NULL, 1)",
rows,
)
connection.commit()
connection.close()
def _hermes_homes(home: Path) -> tuple[Path, Path]:
hermes = home / ".hermes"
wife = hermes / "profiles" / "wife"
# The same Hermes session id in both profiles: identities must not merge.
_hermes_state_db(
hermes,
[(1, "session-1", "user", "I am the husband"), (2, "session-1", "assistant", "hi")],
)
_hermes_state_db(
wife,
[(1, "session-1", "user", "I am the wife"), (2, "session-1", "assistant", "hello")],
)
return hermes, wife
def _scan_texts(root: Path, home: Path) -> list[str]:
adapter = create_hermes_adapter(
AuthorizedSourceRoot.grant(
source_id=HERMES_DESCRIPTOR.source_id, root=root, user_confirmed=True, home=home
)
)
async def collect() -> list[str]:
return [
event.evidence.verbatim
async for event in adapter.scan(ScanOptions(cursors={}))
if isinstance(event, ScannedEvidence)
]
return asyncio.run(collect())
def test_a_hermes_root_scan_never_reads_its_profiles(tmp_path) -> None:
hermes, wife = _hermes_homes(tmp_path)
(hermes / "profiles" / "wife" / "sessions").mkdir()
(hermes / "profiles" / "wife" / "sessions" / "s.jsonl").write_text(
json.dumps({"type": "message", "role": "user", "content": "wife's transcript"}) + "\n"
)
assert _scan_texts(hermes, tmp_path) == ["I am the husband", "hi"]
assert _scan_texts(wife, tmp_path) == ["I am the wife", "hello", "wife's transcript"]
def _jobs(database: Path) -> list[sqlite3.Row]:
with sqlite3.connect(database) as db:
db.row_factory = sqlite3.Row
return db.execute(
"SELECT session_key,source_key,idempotency_key,state,payload_json "
"FROM memory_write_jobs ORDER BY queue_seq"
).fetchall()
@pytest.mark.fresh_sdk
def test_two_hermes_grants_each_read_their_own_root_into_separate_batches(tmp_path) -> None:
async def scenario():
home = tmp_path / "home"
_hermes, wife = _hermes_homes(home)
app = application(tmp_path, home=home)
async with app.router.lifespan_context(app), client_for(app) as client:
location = (await client.post("/api/v1/sources/hermes/location")).json()
local = await client.post(
"/api/v1/sources/connect",
json={
"source_ids": ["hermes"],
"request_id": str(uuid4()),
"source_roots": {"hermes": location["root_id"]},
"enable_sync": True,
},
)
assert local.status_code == 200, local.text
profile = await client.post(
"/api/v1/sources/connect",
json={
"source_ids": ["hermes"],
"request_id": str(uuid4()),
"user_id": "wife",
"root": str(wife),
},
)
assert profile.status_code == 200, profile.text
[wife_job] = profile.json()["jobs"]
assert wife_job["user_id"] == "wife"
jobs = [*local.json()["jobs"], wife_job]
for job in jobs:
finished = await app.state.import_supervisor.wait(job["job_id"])
assert finished.status == "completed", finished
rows = [row for row in _jobs(app.state.settings.data_dir / "one-memory.sqlite3")]
assert rows, "history import wrote no batches"
by_user: dict[str, list[str]] = {}
for row in rows:
records = json.loads(row["payload_json"])["input_records"]
users = {record["user_id"] for record in records}
assert len(users) == 1, "a batch mixed users"
[user] = users
by_user.setdefault(user, []).extend(record["verbatim"] for record in records)
assert row["source_key"] == lane_key(user, "hermes")
assert row["session_key"] == internal_session_id_for("session-1", "hermes", user)
assert sorted(by_user["local"]) == ["I am the husband", "hi"]
assert sorted(by_user["wife"]) == ["I am the wife", "hello"]
grants = (await client.get("/api/v1/source-grants")).json()["grants"]
assert {(grant["user_id"], grant["source_id"]) for grant in grants} >= {
("local", "hermes"),
("wife", "hermes"),
}
# The App's view stays `local`'s.
listed = (await client.get("/api/v1/import-jobs")).json()
assert {job["user_id"] for job in listed} == {"local"}
wife_jobs = (await client.get("/api/v1/import-jobs", params={"user_id": "wife"})).json()
assert {job["user_id"] for job in wife_jobs} == {"wife"}
assert all(
item["source_id"] != "hermes@wife"
for item in (await client.get("/api/v1/live-sync")).json()
)
revoked = await client.post("/api/v1/sources/hermes/revoke", params={"user_id": "wife"})
assert revoked.status_code == 200 and revoked.json()["revoked_at"] is not None
local_grant = app.state.source_grant_store.receipt(user_id="local", source_id="hermes")
assert local_grant.revoked_at is None
asyncio.run(scenario())
@pytest.mark.fresh_sdk
def test_a_migrated_home_with_the_old_grant_triggers_can_connect_and_revoke(tmp_path) -> None:
"""#233: a home an earlier build migrated kept two `source_grants` triggers
whose body reads the dropped `governed_memory_evidence`. Every grant write
then failed, so connecting and revoking answered 503. Startup drops them.
"""
stale = """
CREATE TRIGGER memory_visibility_dirty_on_grants{suffix}
AFTER {event} ON source_grants
BEGIN
INSERT INTO memory_visibility_dirty (memory_id)
SELECT DISTINCT link.memory_id
FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE e.source_id = NEW.source_id
ON CONFLICT(memory_id) DO NOTHING;
END;"""
async def scenario():
home = tmp_path / "home"
_hermes, wife = _hermes_homes(home)
app = application(tmp_path, home=home)
async with app.router.lifespan_context(app):
database = app.state.settings.data_dir / "one-memory.sqlite3"
with sqlite3.connect(database) as db:
db.execute("CREATE TABLE IF NOT EXISTS memory_visibility_dirty (memory_id TEXT)")
db.execute(stale.format(suffix="", event="INSERT"))
db.execute(stale.format(suffix="_update", event="UPDATE"))
app = application(tmp_path, home=home)
async with app.router.lifespan_context(app), client_for(app) as client:
location = (await client.post("/api/v1/sources/hermes/location")).json()
local = await client.post(
"/api/v1/sources/connect",
json={
"source_ids": ["hermes"],
"request_id": str(uuid4()),
"source_roots": {"hermes": location["root_id"]},
},
)
assert local.status_code == 200, local.text
profile = await client.post(
"/api/v1/sources/connect",
json={
"source_ids": ["hermes"],
"request_id": str(uuid4()),
"user_id": "wife",
"root": str(wife),
},
)
assert profile.status_code == 200, profile.text
for user_id in ("wife", "local"):
revoked = await client.post(
"/api/v1/sources/hermes/revoke", params={"user_id": user_id}
)
assert revoked.status_code == 200, revoked.text
assert revoked.json()["revoked_at"] is not None
with sqlite3.connect(database) as db:
assert (
db.execute(
"SELECT COUNT(*) FROM sqlite_master "
"WHERE type='trigger' AND tbl_name='source_grants'"
).fetchone()[0]
== 0
)
asyncio.run(scenario())
@pytest.mark.fresh_sdk
def test_only_hermes_takes_another_user_or_a_custom_root(tmp_path) -> None:
async def scenario():
home = tmp_path / "home"
(home / ".claude" / "projects").mkdir(parents=True)
app = application(tmp_path, home=home)
async with app.router.lifespan_context(app), client_for(app) as client:
for extra in ({"user_id": "wife"}, {"root": str(home / ".claude" / "projects")}):
response = await client.post(
"/api/v1/sources/connect",
json={
"source_ids": ["claude-code"],
"request_id": str(uuid4()),
"source_roots": {},
**extra,
},
)
assert response.status_code == 422, response.text
assert response.json()["detail"] == "user_id_requires_multi_user_source"
revoke = await client.post(
"/api/v1/sources/claude-code/revoke", params={"user_id": "wife"}
)
assert revoke.status_code == 422
# Hermes too: another user must name their own root.
rootless = await client.post(
"/api/v1/sources/connect",
json={
"source_ids": ["hermes"],
"request_id": str(uuid4()),
"source_roots": {},
"user_id": "wife",
},
)
assert rootless.status_code == 422
assert rootless.json()["detail"] == "source_root_required_for_user"
asyncio.run(scenario())
def _record(user_id: str, message_id: str, role: str, text: str):
return materialize_write_input(
EvidenceInput(
user_id=user_id,
source_id="hermes",
conversation_id="session-1",
message_id=message_id,
role=role,
written_by=WrittenBy.PERSON if role == "user" else WrittenBy.AGENT,
created_at=datetime(2026, 10, 1, tzinfo=UTC),
verbatim=text,
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
)
def test_assembling_rows_and_batches_are_keyed_by_user(tmp_path) -> None:
"""Two users' `state.db` share a relative file key and a session id.
Finishing one user's file must neither take nor drop the other user's
staged turns, and the batches it writes hold one user each.
"""
path = tmp_path / "one-memory.sqlite3"
queue = DurableWriteQueue(path, sdk_authority=True)
file_key = "f" * 64
from one_memory.database_access import main_database
with main_database(path).write(row_factory=sqlite3.Row) as connection, connection:
for user in ("local", "wife"):
for index, role in enumerate(("user", "assistant")):
queue.stage_sdk_input(
_record(user, f"m{index}", role, f"{user} {role}"),
file_key=file_key,
priority="normal",
connection=connection,
)
queue.finalize_sdk_file(
user_id="local", source_id="hermes", file_key=file_key, connection=connection
)
rows = _jobs(path)
pending = [row for row in rows if row["state"] == "pending"]
assembling = [row for row in rows if row["state"] == "assembling"]
assert [
{record["user_id"] for record in json.loads(row["payload_json"])["input_records"]}
for row in pending
] == [{"local"}]
assert [split_lane(row["source_key"]) for row in assembling] == [("wife", "hermes")]
assert pending[0]["session_key"] != assembling[0]["session_key"]
def _observed(response) -> tuple[int, bytes]:
return response.status_code, response.content
@pytest.mark.fresh_sdk
def test_another_users_id_answers_exactly_like_an_id_that_does_not_exist(
tmp_path, monkeypatch
) -> None:
"""Comparing responses must not tell whether another user's id exists."""
async def scenario(app, sdk, client):
receipts = await _write_both_users(app, sdk, client)
[wife_memory] = [
memory_id
for memory_id, row in sdk.memories.items()
if row["user_id"] == "wife" and row["content"] == "wife drinks coffee"
]
before = {key: dict(row) for key, row in sdk.memories.items()}
with sqlite3.connect(app.state.settings.data_dir / "one-memory.sqlite3") as db:
keys = [row[0] for row in db.execute("SELECT idempotency_key FROM memory_write_jobs")]
wife_key = next(key for key in keys if key.startswith("user:wife:"))
pairs = [
("GET", f"/api/v2/memories/{wife_memory}", "/api/v2/memories/no-such-memory", {}),
(
"PUT",
f"/api/v2/memories/{wife_memory}",
"/api/v2/memories/no-such-memory",
{"json": {"content": "stolen"}},
),
("DELETE", f"/api/v2/memories/{wife_memory}", "/api/v2/memories/no-such-memory", {}),
(
"GET",
f"/api/v2/memories/additions/{receipts['add:wife']}",
f"/api/v2/memories/additions/{uuid4()}",
{},
),
("GET", f"/api/v2/memory-writes/{wife_key}", "/api/v2/memory-writes/no-such-id", {}),
]
for method, theirs, missing, kwargs in pairs:
other = await client.request(method, theirs, **kwargs)
absent = await client.request(method, missing, **kwargs)
assert _observed(other) == _observed(absent), (method, theirs)
assert other.status_code == (204 if method == "DELETE" else 404)
assert sdk.memories == before
run_with_sdk(tmp_path, monkeypatch, scenario)
@@ -173,12 +173,14 @@ def test_first_import_gate_waits_for_every_selected_history_job() -> None:
jobs = [
SimpleNamespace(
operation="history",
user_id="local",
source_id="cursor",
created_at=datetime(2026, 10, 18, tzinfo=UTC),
status="completed",
),
SimpleNamespace(
operation="history",
user_id="local",
source_id="codex",
created_at=datetime(2026, 10, 18, tzinfo=UTC),
status="running",
@@ -40,6 +40,7 @@ def library(tmp_path):
provider_kind=ProviderKind.OLLAMA,
model="hy-bot-phi4",
embedding_model="nomic-embed-text",
embedding_base_url="http://127.0.0.1:11434",
),
secrets,
)
@@ -210,7 +210,7 @@ def test_recall_route_falls_back_and_keeps_the_raw_grant(tmp_path, monkeypatch):
async def no_model(_app):
return False
async def raw(_app, query, *, limit, sources):
async def raw(_app, query, *, limit, sources, user_id="local"):
searched.append(sources)
return (_raw("r1", "上周决定用 Postgres", source="cursor"),)
@@ -0,0 +1,744 @@
"""The SDK surface MemHub exposes: search pass-through, layers, raw, v2 aliases.
P2-1 (R8): `context/search`, `recall` and the v2 ask routes pass the public
`client.search` knobs through, validated, and keep a caller's floor on what
they return. P2-2 (R9): the library honours `layers`, `statuses`, `scene` and
the source/session/agent filters, and counts memories per layer. P2-4: raw
lists the SDK's L1 raw turns, active and shadow. P2-3: v2 aliases of the
source and import routes, per user. Profile versions read the L0 chain.
Every route stays scoped to its one `user_id`; the stand-in client records
each call so a test can see exactly what reached the SDK.
"""
from __future__ import annotations
import asyncio
import hashlib
import sqlite3
from contextlib import asynccontextmanager
from pathlib import Path
from uuid import uuid4
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.sources.registry import build_source_catalog
from one_memory.secrets import InMemorySecretStore
TOKEN = "sdk-surface-test"
CHAT_ROWS = {
"local": [
("m-high", "local likes green tea", 0.95, "l2_fact", "hermes"),
("m-mid", "local walks at dawn", 0.6, "l2_fact", "codex"),
],
"wife": [("w-high", "wife likes coffee", 0.92, "l2_fact", "hermes")],
}
RAW_ROWS = {
"local": [
{
"memory_id": "raw-1",
"content": "[user]: 我叫费天翔\n[assistant]: 好的,记住了。",
"status": "shadow",
"session_id": "s-1",
"source": "hermes",
"memory_at": 1_790_000_000,
"gmt_created": 1_790_000_100,
},
{
"memory_id": "raw-2",
"content": "[user]: 你好",
"status": "active",
"session_id": "s-2",
"source": "codex",
"memory_at": 1_790_100_000,
"gmt_created": 1_790_100_100,
},
],
"wife": [
{
"memory_id": "raw-w",
"content": "[user]: I am the wife",
"status": "active",
"session_id": "s-w",
"source": "hermes",
"memory_at": 1_790_000_000,
"gmt_created": 1_790_000_000,
}
],
}
def _chat(user_id: str, memory_id: str, content: str, layer: str, source: str) -> dict:
return {
"memory_id": memory_id,
"content": content,
"layer": layer,
"status": "active",
"user_id": user_id,
"agent_id": "all-apps",
"session_id": "s-1",
"source": source,
"tags": [],
"gmt_created": 1_790_000_000,
}
class RecordingSdk:
"""The public client surface, recording every call's keyword arguments."""
def __init__(self) -> None:
self.calls: list[tuple[str, dict]] = []
def _record(self, name: str, kwargs: dict) -> None:
self.calls.append((name, dict(kwargs)))
def last(self, name: str) -> dict:
return [kwargs for called, kwargs in self.calls if called == name][-1]
def all(self, name: str) -> list[dict]:
return [kwargs for called, kwargs in self.calls if called == name]
# The SDK answers rows below a caller's floor in some lanes (a schema
# floor, a fused rank score), so Core must filter what it returns.
def search(self, query, **kwargs):
self._record("search", {"query": query, **kwargs})
[user_id] = kwargs["user_ids"]
if kwargs.get("layers") == ["l1_raw"]:
hits = [
{**row, "user_id": user_id, "agent_id": "all-apps", "score": score}
for row, score in zip(RAW_ROWS.get(user_id, []), (0.97, 0.3), strict=False)
]
return {"request_id": "s", "memories": hits}
rows = [
{**_chat(user_id, mid, text, layer, source), "score": score}
for mid, text, score, layer, source in CHAT_ROWS.get(user_id, [])
]
scene = kwargs.get("scene")
if scene == "all":
fused = [
{**row, "scene": "chat", "kind": "chat", "lane_score": row["score"], "score": 0.016}
for row in rows
]
return {"request_id": "s", "scene": "all", "memories": fused, "lanes": {}, "errors": {}}
if scene in {"working", "coding"}:
return {
"request_id": "s",
"scene": "productivity",
"variant": scene,
"facts": [
{"fact_id": "f-1", "statement": "use uv", "score": 0.91, "user_id": user_id}
],
"experiences": [],
}
return {
"request_id": "s",
"scene": "normal",
"memories": {"normal": rows, "schema": [], "proactive": []},
}
def get(self, memory_id, kind=None, variant=None):
self._record("get", {"memory_id": memory_id, "kind": kind, "variant": variant})
if kind == "fact":
return {
"kind": "fact",
"variant": variant,
"fact_id": memory_id,
"statement": "use uv",
"status": "active",
"user_id": "local",
}
for user_id, rows in CHAT_ROWS.items():
for mid, text, _score, layer, source in rows:
if mid == memory_id:
return {**_chat(user_id, mid, text, layer, source), "kind": "chat"}
return None
def list_memories(self, **kwargs):
self._record("list_memories", kwargs)
user_id = kwargs["user_id"]
layers = kwargs.get("layers")
if layers == ["l1_raw"]:
statuses = set(kwargs.get("statuses") or ["active"])
rows = [
{**row, "layer": "l1_raw", "user_id": user_id, "agent_id": "all-apps"}
for row in RAW_ROWS.get(user_id, [])
if row["status"] in statuses
and (
not kwargs.get("include_sources") or row["source"] in kwargs["include_sources"]
)
and (not kwargs.get("session_ids") or row["session_id"] in kwargs["session_ids"])
]
return {"vdb": {"memories": rows, "total": len(rows)}}
rows = [_chat(user_id, *row[:2], *row[3:]) for row in CHAT_ROWS.get(user_id, [])]
if layers:
rows = [row for row in rows if row["layer"] in layers]
if "l0_basic_info" in layers:
rows.append(_chat(user_id, "l0-1", "name is Fei", "l0_basic_info", "hermes"))
if kwargs.get("scene") == "all":
return {
"scene": "all",
"memories": [{**row, "scene": "chat", "kind": "chat"} for row in rows],
"total": len(rows),
"totals": {"chat": len(rows), "working": 0, "coding": 0},
"errors": {},
}
return {"vdb": {"memories": rows, "total": len(rows)}}
def memory_stats(self, *, user_id, statuses=None):
return {"total": 0, "request_id": "stats", "elapsed_ms": 1.0}
def get_user_basic_info(self, *, user_id):
self._record("get_user_basic_info", {"user_id": user_id})
return {
"request_id": f"basic-{user_id}",
"elapsed_ms": 1.5,
"memories": [
{
"memory_id": f"{user_id}-l0-head",
"content": "The user's name is Fei.",
"layer": "l0_basic_info",
"status": "active",
"memory_at": 1_790_000_100,
"gmt_created": 1_790_000_100,
"tags": [],
"basic_info_kv": {"name": "Fei"},
"evolution_chain": [
{
"node_id": f"{user_id}-l0-head",
"content": "The user's name is Fei.",
"memory_at": "2026-10-31 10:00",
"gmt_created": 1_790_000_100,
"layer": "l0_basic_info",
},
{
"node_id": f"{user_id}-l0-old",
"content": "The user's name is 费天翔.",
"memory_at": "2026-10-30 10:00",
"gmt_created": 1_790_000_000,
"layer": "l0_basic_info",
},
],
}
],
}
class Handle:
def __init__(self, sdk: RecordingSdk) -> None:
self.sdk = sdk
async def client(self) -> RecordingSdk:
return self.sdk
async def close(self) -> None:
return None
def application(tmp_path: Path, *, home: Path | None = None):
settings = Settings(
data_dir=tmp_path / "data",
source_home=home or tmp_path,
api_token=SecretStr(TOKEN),
imports_enabled=home is not None,
automatic_memory_processing=False,
live_sync_enabled=False,
)
return create_app(
settings,
secret_store=InMemorySecretStore(),
source_catalog=(
build_source_catalog(home=home, environment={}, platform="darwin")
if home is not None
else None
),
)
def client_for(app) -> AsyncClient:
return AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": f"Bearer {TOKEN}"},
)
async def _model_ready(_app) -> bool:
return True
def run_with_sdk(tmp_path, monkeypatch, scenario) -> None:
async def main() -> None:
app = application(tmp_path)
sdk = RecordingSdk()
@asynccontextmanager
async def write_client():
yield sdk
async with app.router.lifespan_context(app):
app.state.sdk_client_handle = Handle(sdk)
monkeypatch.setattr(app.state.sdk_write_coordinator, "use_client", write_client)
monkeypatch.setattr("one_memory.api._recall_model_ready", _model_ready)
async with client_for(app) as client:
await scenario(app, sdk, client)
asyncio.run(main())
PASS_THROUGH = {
"min_score": 0.5,
"schema_min_score": 0.6,
"working_fact_min_score": 0.3,
"working_experience_min_score": 0.45,
"schema_limit": 4,
"intention_limit": 2,
"rerank": False,
"reader": "legacy",
"scene": "chat",
"search_mode": "agentic",
"include_sources": ["hermes", "codex"],
"exclude_sources": ["cursor"],
"created_after": 1_780_000_000,
"session_ids": ["s-1"],
"agent_ids": ["all-apps"],
"workspace_id": "ws-1",
"branch": "main",
"layers": ["l2_fact", "l0_basic_info"],
}
#: How each field reaches `client.search` (lists stay lists).
EXPECTED_SEARCH = {**PASS_THROUGH}
def _memory_search(sdk: RecordingSdk) -> dict:
return [call for call in sdk.all("search") if call.get("layers") != ["l1_raw"]][-1]
@pytest.mark.fresh_sdk
def test_every_context_search_option_reaches_the_sdk_and_absent_ones_keep_the_old_call(
tmp_path, monkeypatch
) -> None:
async def scenario(app, sdk, client):
baseline = await client.post(
"/api/v2/context/search", json={"query": "tea", "kinds": ["memory"]}
)
assert baseline.status_code == 200, baseline.text
before = _memory_search(sdk)
assert before["min_score"] == before["schema_min_score"] == 0.35
assert before["scene"] == "normal" and before["reader"] == "hybrid"
assert before["schema_limit"] == 3 and before["intention_limit"] == 1
for absent in ("rerank", "search_mode", "layers", "session_ids", "created_after"):
assert absent not in before
response = await client.post(
"/api/v2/context/search",
json={"query": "tea", "kinds": ["memory"], **PASS_THROUGH},
)
assert response.status_code == 200, response.text
sent = _memory_search(sdk)
for name, value in EXPECTED_SEARCH.items():
assert sent[name] == value, name
assert sent["user_ids"] == ["local"]
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_every_recall_and_ask_option_reaches_the_sdk(tmp_path, monkeypatch) -> None:
async def scenario(app, sdk, client):
recalled = await client.post(
"/api/v1/recall", json={"query": "tea", "user_id": "wife", **PASS_THROUGH}
)
assert recalled.status_code == 200, recalled.text
sent = _memory_search(sdk)
for name, value in EXPECTED_SEARCH.items():
assert sent[name] == value, name
assert sent["user_ids"] == ["wife"]
ask_options = {
name: value
for name, value in PASS_THROUGH.items()
if name
not in {
"working_fact_min_score",
"working_experience_min_score",
"schema_limit",
"intention_limit",
"reader",
"scene",
"search_mode",
"workspace_id",
"branch",
}
}
asked = await client.post(
"/api/v2/memories/ask", json={"question": "tea?", "user_id": "wife", **ask_options}
)
assert asked.status_code == 200, asked.text
sent = _memory_search(sdk)
for name, value in ask_options.items():
assert sent[name] == value, name
assert sent["user_ids"] == ["wife"] and sent["scene"] == "normal"
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_min_score_is_enforced_on_what_core_returns(tmp_path, monkeypatch) -> None:
"""Acceptance item 9: `min_score=0.9` answers only results scoring ≥ 0.9."""
async def scenario(app, sdk, client):
context = await client.post(
"/api/v2/context/search", json={"query": "tea", "min_score": 0.9}
)
assert context.status_code == 200, context.text
items = context.json()["items"]
assert items and all(item["score"] >= 0.9 for item in items)
assert {item.get("memory_id") or item.get("raw_id") for item in items} == {
"m-high",
"raw-1",
}
raw_call = sdk.last("search") if sdk.last("search").get("layers") else None
assert raw_call is None or raw_call["min_score"] == 0.9
# The fused scene ranks by 1/(60+rank); the lane's own score is the floor's.
fused = await client.post(
"/api/v2/context/search",
json={"query": "tea", "kinds": ["memory"], "scene": "all", "min_score": 0.9},
)
assert [item["memory_id"] for item in fused.json()["items"]] == ["m-high"]
assert fused.json()["items"][0]["score"] == 0.95
recalled = await client.post("/api/v1/recall", json={"query": "tea", "min_score": 0.9})
assert recalled.status_code == 200, recalled.text
assert [hit["memory_id"] for hit in recalled.json()["hits"]] == ["m-high"]
# A working-scene fact is read back from its own namespace.
working = await client.post(
"/api/v2/context/search",
json={"query": "uv", "kinds": ["memory"], "scene": "working", "min_score": 0.9},
)
assert [item["memory_id"] for item in working.json()["items"]] == ["f-1"]
assert sdk.last("get") == {"memory_id": "f-1", "kind": "fact", "variant": "working"}
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
@pytest.mark.parametrize(
"extra",
[
{"min_score": 1.5},
{"min_score": -0.1},
{"schema_min_score": "high"},
{"scene": "normal"},
{"search_mode": "fast"},
{"reader": "vector"},
{"layers": ["l9_nothing"]},
{"layers": []},
{"branch": "main"},
{"rerank": "no"},
{"created_after": -1},
{"user_ids": ["wife"]},
{"embed_override": {"model": "x"}},
{"query_embedding": [0.1]},
{"request_id": "r-1"},
],
)
def test_out_of_range_and_unexposed_search_fields_answer_422(tmp_path, extra) -> None:
async def scenario():
app = application(tmp_path)
async with app.router.lifespan_context(app), client_for(app) as client:
for path, body in (
("/api/v2/context/search", {"query": "q"}),
("/api/v1/recall", {"query": "q"}),
("/api/v2/memories/ask", {"question": "q"}),
):
response = await client.post(path, json={**body, **extra})
assert response.status_code == 422, (path, response.text)
asyncio.run(scenario())
@pytest.mark.fresh_sdk
def test_raw_lane_takes_the_callers_sources_only_inside_its_authority(
tmp_path, monkeypatch
) -> None:
async def scenario(app, sdk, client):
response = await client.post(
"/api/v2/context/search",
json={
"query": "tea",
"kinds": ["raw"],
"include_sources": ["hermes"],
"session_ids": ["s-1"],
"created_after": 1_700_000_000,
},
)
assert response.status_code == 200, response.text
raw_call = sdk.last("search")
assert raw_call["layers"] == ["l1_raw"]
assert raw_call["include_sources"] == ["hermes"]
assert raw_call["session_ids"] == ["s-1"]
assert raw_call["created_after"] == 1_700_000_000
# Memory-lane knobs never reach the raw search.
assert raw_call["scene"] == "chat" and raw_call["schema_limit"] == 0
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_library_honours_layers_statuses_and_filters(tmp_path, monkeypatch) -> None:
"""Acceptance item 10: `layer=l0_basic_info` returns only L0 records."""
async def scenario(app, sdk, client):
page = await client.get("/api/v2/memory-library", params={"layer": "l0_basic_info"})
assert page.status_code == 200, page.text
assert {memory["layer"] for memory in page.json()["memories"]} == {"l0_basic_info"}
sent = sdk.last("list_memories")
assert sent["layers"] == ["l0_basic_info"] and sent["scene"] is None
await client.get(
"/api/v2/memory-library",
params=[
("layers", "l2_fact,l6_schema"),
("layers", "l7_intention"),
("statuses", "active,superseded"),
("session_ids", "s-1"),
("agent_ids", "all-apps"),
("exclude_sources", "cursor"),
("user_id", "wife"),
],
)
sent = sdk.last("list_memories")
assert sent["layers"] == ["l2_fact", "l6_schema", "l7_intention"]
assert sent["statuses"] == ["active", "superseded"]
assert sent["session_ids"] == ["s-1"] and sent["agent_ids"] == ["all-apps"]
assert sent["exclude_sources"] == ["cursor"] and sent["user_id"] == "wife"
# Without a layer the page is what it always was.
await client.get("/api/v2/memory-library")
sent = sdk.last("list_memories")
assert sent["scene"] == "all" and "layers" not in sent
searched = await client.get(
"/api/v2/memory-library", params={"query": "tea", "layers": "l2_fact"}
)
assert searched.status_code == 200, searched.text
assert sdk.last("search")["layers"] == ["l2_fact"]
for params in (
{"layers": "l9_nothing"},
{"statuses": "gone"},
{"layers": "l2_fact", "scene": "working"},
):
refused = await client.get("/api/v2/memory-library", params=params)
assert refused.status_code == 422, (params, refused.text)
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_layer_counts_read_each_layer_for_one_user(tmp_path, monkeypatch) -> None:
async def scenario(app, sdk, client):
counts = await client.get("/api/v2/memory-library/layers", params={"user_id": "wife"})
assert counts.status_code == 200, counts.text
body = counts.json()
assert body["layers"]["l1_raw"] == 1
assert body["layers"]["l2_fact"] == 1
assert body["layers"]["l0_basic_info"] == 1
assert body["total"] == sum(body["layers"].values())
calls = sdk.all("list_memories")
assert {call["user_id"] for call in calls} == {"wife"}
[raw_call] = [call for call in calls if call["layers"] == ["l1_raw"]]
assert raw_call["statuses"] == ["active", "shadow"]
assert (
await client.get("/api/v2/memory-library/layers", params={"layer": "x"})
).status_code == 422
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_raw_lists_active_and_shadow_turns_with_roles_and_hashes(tmp_path, monkeypatch) -> None:
async def scenario(app, sdk, client):
page = await client.get("/api/v2/raw")
assert page.status_code == 200, page.text
body = page.json()
assert body["total"] == 2 and len(body["items"]) == 2
sent = sdk.last("list_memories")
assert sent["layers"] == ["l1_raw"] and sent["statuses"] == ["active", "shadow"]
first = next(item for item in body["items"] if item["raw_id"] == "raw-1")
assert first["status"] == "shadow"
assert [message["role"] for message in first["messages"]] == ["user", "assistant"]
assert first["messages"][0]["content"] == "我叫费天翔"
assert first["role"] is None
expected = hashlib.sha256(first["content"].encode()).hexdigest()
assert first["content_hash"] == f"sha256:{expected}"
assert first["session_id"] == "s-1" and first["source"] == "hermes"
assert first["created"].startswith("2026-")
second = next(item for item in body["items"] if item["raw_id"] == "raw-2")
assert second["role"] == "user"
await client.get(
"/api/v2/raw",
params=[("source", "hermes,codex"), ("session_id", "s-1"), ("order", "asc")],
)
sent = sdk.last("list_memories")
assert sent["include_sources"] == ["hermes", "codex"]
assert sent["session_ids"] == ["s-1"] and sent["order"] == "asc"
windowed = (await client.get("/api/v2/raw", params={"after": 1_790_050_000})).json()
assert windowed["time_filtered"] is True
assert [item["raw_id"] for item in windowed["items"]] == ["raw-2"]
wife = (await client.get("/api/v2/raw", params={"user_id": "wife"})).json()
assert [item["raw_id"] for item in wife["items"]] == ["raw-w"]
assert sdk.last("list_memories")["user_id"] == "wife"
assert (await client.get("/api/v2/raw", params={"statuses": "bogus"})).status_code == 422
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_profile_versions_read_the_l0_chain_for_one_user(tmp_path, monkeypatch) -> None:
async def scenario(app, sdk, client):
response = await client.get("/api/v2/user-profile/versions", params={"user_id": "wife"})
assert response.status_code == 200, response.text
body = response.json()
assert body["user_id"] == "wife"
assert sdk.last("get_user_basic_info") == {"user_id": "wife"}
[chain] = body["chains"]
assert chain["memory_id"] == "wife-l0-head"
assert chain["basic_info"] == {"name": "Fei"}
assert [(v["memory_id"], v["current"]) for v in chain["versions"]] == [
("wife-l0-head", True),
("wife-l0-old", False),
]
local = (await client.get("/api/v2/user-profile/versions")).json()
assert local["user_id"] == "local"
assert sdk.last("get_user_basic_info") == {"user_id": "local"}
refused = await client.get("/api/v2/user-profile/versions", params={"owner": "wife"})
assert refused.status_code == 422
run_with_sdk(tmp_path, monkeypatch, scenario)
@pytest.mark.fresh_sdk
def test_graph_takes_layers_and_session_filters(tmp_path, monkeypatch) -> None:
async def scenario(app, sdk, client):
response = await client.get(
"/api/v2/memories/graph",
params={"layers": "l2_fact", "session_ids": "s-1", "user_id": "wife"},
)
assert response.status_code == 200, response.text
sent = sdk.last("list_memories")
assert sent["layers"] == ["l2_fact"] and sent["scene"] is None
assert sent["session_ids"] == ["s-1"] and sent["user_id"] == "wife"
run_with_sdk(tmp_path, monkeypatch, scenario)
def _hermes_state_db(root: Path, text: str) -> None:
root.mkdir(parents=True, exist_ok=True)
connection = sqlite3.connect(root / "state.db")
connection.execute(
"CREATE TABLE messages (id INTEGER PRIMARY KEY, session_id TEXT, role TEXT,"
" content TEXT, timestamp REAL, tool_name TEXT, active INTEGER)"
)
connection.executemany(
"INSERT INTO messages VALUES (?, ?, ?, ?, 1790000000, NULL, 1)",
[(1, "session-1", "user", text), (2, "session-1", "assistant", "ok")],
)
connection.commit()
connection.close()
@pytest.mark.fresh_sdk
def test_v2_source_and_import_aliases_behave_like_v1_per_user(tmp_path) -> None:
async def scenario():
home = tmp_path / "home"
hermes = home / ".hermes"
wife = hermes / "profiles" / "wife"
_hermes_state_db(hermes, "I am the husband")
_hermes_state_db(wife, "I am the wife")
app = application(tmp_path, home=home)
async with app.router.lifespan_context(app), client_for(app) as client:
location = (await client.post("/api/v1/sources/hermes/location")).json()
local = await client.post(
"/api/v1/sources/connect",
json={
"source_ids": ["hermes"],
"request_id": str(uuid4()),
"source_roots": {"hermes": location["root_id"]},
"enable_sync": True,
},
)
assert local.status_code == 200, local.text
connected = await client.post(
"/api/v1/sources/connect",
json={
"source_ids": ["hermes"],
"request_id": str(uuid4()),
"user_id": "wife",
"root": str(wife),
},
)
assert connected.status_code == 200, connected.text
for job in (*local.json()["jobs"], *connected.json()["jobs"]):
await app.state.import_supervisor.wait(job["job_id"])
v1 = (await client.get("/api/v1/sources")).json()
v2 = (await client.get("/api/v2/sources")).json()
assert v1 == v2
wife_sources = (await client.get("/api/v2/sources", params={"user_id": "wife"})).json()
assert [source["source_id"] for source in wife_sources] == ["hermes"]
assert wife_sources[0]["authorized"] is True
assert (await client.get("/api/v2/sources", params={"u": "x"})).status_code == 422
created = await client.post(
"/api/v2/import-jobs",
json={"source_id": "hermes", "operation": "sync", "user_id": "wife"},
)
assert created.status_code == 200, created.text
wife_job = created.json()
assert wife_job["user_id"] == "wife" and wife_job["source_id"] == "hermes"
await app.state.import_supervisor.wait(wife_job["job_id"])
refused = await client.post(
"/api/v2/import-jobs",
json={"source_id": "claude-code", "user_id": "wife"},
)
assert refused.status_code in {404, 422}
unknown = await client.post(
"/api/v2/import-jobs",
json={"source_id": "hermes", "from_beginning": True},
)
assert unknown.status_code == 422
listed = (await client.get("/api/v2/import-jobs", params={"user_id": "wife"})).json()
assert {job["user_id"] for job in listed} == {"wife"}
assert (await client.get("/api/v2/import-jobs")).json() == (
await client.get("/api/v1/import-jobs")
).json()
path = f"/api/v2/import-jobs/{wife_job['job_id']}"
assert (await client.get(path)).status_code == 404
own = await client.get(path, params={"user_id": "wife"})
assert own.status_code == 200 and own.json()["job_id"] == wife_job["job_id"]
synced = await client.post("/api/v2/sources/hermes/sync", json={"user_id": "wife"})
v1_synced = await client.post("/api/v1/sources/hermes/sync")
assert synced.status_code == v1_synced.status_code, (synced.text, v1_synced.text)
assert synced.status_code == 200
assert synced.json()["user_id"] == "wife"
assert v1_synced.json()["user_id"] == "local"
other = await client.post("/api/v2/sources/claude-code/sync", json={"user_id": "wife"})
assert other.status_code in {404, 422}
assert (
await client.post("/api/v2/sources/hermes/sync", json={"owner": "wife"})
).status_code == 422
asyncio.run(scenario())
@@ -6,7 +6,7 @@ import io
import json
import sqlite3
from contextlib import closing
from datetime import UTC, datetime, timedelta
from datetime import UTC, datetime
from pathlib import Path
import pytest
@@ -30,7 +30,6 @@ from one_memory.sdk_data_plane_migration import (
MemoryAuthority,
SdkDataPlaneMigrationStore,
)
from one_memory.sdk_memory_runtime import SdkMemoryRuntime
from one_memory.sdk_migration_bundle import create_sdk_migration_bundle
from one_memory.sdk_migration_runner import (
PREPARED_PHASE,
@@ -708,6 +707,7 @@ def test_cutover_waits_until_the_new_runtime_reads_every_chat_memory(store, acti
"chat_readable": 1,
"productivity_expected": 0,
"productivity_readable": 0,
"chat_missing_before_migration": 0,
"passed": False,
}
assert store.authority() is MemoryAuthority.LEGACY
@@ -774,18 +774,186 @@ def test_retired_coding_experiences_do_not_block_but_an_empty_store_does(store,
assert (result.readback.productivity_expected, result.readback.productivity_readable) == (2, 1)
def test_memories_without_a_model_profile_keep_legacy_authority(store, monkeypatch):
def test_a_home_without_a_model_profile_migrates_on_the_builtin_embedding(store, monkeypatch):
"""#234: no generation model was ever configured, and the upgrade stopped.
No profile means no LLM, not no embedding: the vectors are the built-in
model's, the same space the Runtime reads with. They are reattached and
read back through it, and the client is opened without resolving a key.
"""
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot, vector_collection_bases
monkeypatch.setattr("one_memory.sdk_migration_runner.read_active_profile", lambda _db: None)
home = store.database_path.parent
builtin = builtin_retrieval_snapshot()
_current, legacy_bases = vector_collection_bases(builtin)
client = _chroma(home)
_fill(client, f"{legacy_bases[0]}_512", ["m1", "m2"])
del client
_confirmed(store, "m1", "m2")
opened: list[str] = []
def open_client(profile):
opened.append(profile.profile_id)
return _reading({"m1", "m2"})(profile)
result = run_startup_migration(
data_dir=home,
runtime_build="21",
sdk_version="2.0.5",
namespace="governed-v2",
open_client=open_client,
)
assert result.state == "cutover"
assert opened == [builtin.profile_id]
assert [item.legacy for item in result.reattached] == [f"{legacy_bases[0]}_512"]
assert result.readback is not None and result.readback.passed
assert store.authority() is MemoryAuthority.SDK
def test_the_builtin_readback_client_resolves_no_secret(tmp_path, monkeypatch):
"""The built-in snapshot has no stored version and no key to look up."""
from one_memory import sdk_migration_verify
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot
leased: list[tuple[str, object]] = []
class Lease:
client = object()
def close(self) -> None:
leased.append(("closed", None))
def acquire(profile, *, api_key, data_dir):
leased.append((profile.profile_id, api_key))
return Lease()
def no_lookup(*_args, **_kwargs):
raise AssertionError("a key-less profile must not touch the secret store")
monkeypatch.setattr("one_memory.hy_sdk_runtime.acquire_client", acquire)
monkeypatch.setattr(
"one_memory.model_profiles.ModelProfileStore.resolve_snapshot_secret", no_lookup
)
with sdk_migration_verify.runtime_sdk_client(
tmp_path / "missing.sqlite3", tmp_path, builtin_retrieval_snapshot()
) as client:
assert client is Lease.client
assert leased == [("builtin-retrieval", None), ("closed", None)]
def test_vectors_left_by_a_deleted_profile_stop_the_upgrade_by_name(store, monkeypatch):
"""No active profile, and the vectors sit in another profile's space.
The built-in model cannot read them, so the old abort was right for this
one shape; it now says what is actually wrong, before anything changes.
"""
from one_memory.hy_sdk_runtime import vector_collection_bases
from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind
monkeypatch.setattr("one_memory.sdk_migration_runner.read_active_profile", lambda _db: None)
deleted = ModelProfileSnapshot(
profile_id="deleted",
revision=3,
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
base_url="https://models.example/v1",
model="chat",
embedding_provider_kind=ProviderKind.OPENAI_COMPATIBLE,
embedding_base_url="https://models.example/v1",
embedding_model="text-embedding-3-large",
embedding_dimensions=1024,
timeout_seconds=60,
max_concurrency=1,
context_window=None,
has_api_key=True,
)
current, _legacy = vector_collection_bases(deleted)
home = store.database_path.parent
client = _chroma(home)
_fill(client, f"{current}_1024", ["m1"])
del client
_confirmed(store, "m1")
with pytest.raises(DataPlaneMigrationError, match="sdk_migration_model_profile_required"):
with pytest.raises(DataPlaneMigrationError, match="sdk_migration_vector_profile_missing"):
run_startup_migration(
data_dir=store.database_path.parent,
data_dir=home,
runtime_build="21",
sdk_version="2.0.5",
namespace="governed-v2",
open_client=_reading({"m1"}),
)
assert store.authority() is MemoryAuthority.LEGACY
assert not (home / "backups").exists()
def test_confirmed_chats_the_sdk_had_already_merged_do_not_fail_the_upgrade(store, active_profile):
"""#229/#240: 658 confirmed chats, 657 in the store, and the upgrade failed.
A 1.0.4 library can hold confirmed rows whose SDK node had been merged or
deleted before the upgrade; no migration can read those back. The check
holds chat to what the store held before this run and counts the rest.
"""
from one_memory.hy_sdk_runtime import vector_collection_bases
home = store.database_path.parent
stored = [f"chat-{index:03d}" for index in range(657)]
current, _legacy = vector_collection_bases(active_profile)
client = _chroma(home)
_fill(client, f"{current}_512", stored)
del client
_confirmed(store, *stored, "chat-merged-before-upgrade")
result = run_startup_migration(
data_dir=home,
runtime_build="21",
sdk_version="2.0.5",
namespace="governed-v2",
open_client=_reading(set(stored)),
)
assert result.state == "cutover"
assert result.readback is not None
assert result.readback.as_document() == {
"chat_expected": 657,
"chat_readable": 657,
"productivity_expected": 0,
"productivity_readable": 0,
"chat_missing_before_migration": 1,
"passed": True,
}
assert store.authority() is MemoryAuthority.SDK
def test_a_chat_in_the_store_before_the_upgrade_must_still_read_back(store, active_profile):
"""Only what was already gone is excused; losing a stored memory still fails."""
from one_memory.hy_sdk_runtime import vector_collection_bases
from one_memory.sdk_migration_verify import MemoryReadbackFailed
home = store.database_path.parent
current, _legacy = vector_collection_bases(active_profile)
client = _chroma(home)
_fill(client, f"{current}_512", ["m1", "m2"])
del client
_confirmed(store, "m1", "m2", "merged")
with pytest.raises(MemoryReadbackFailed) as caught:
run_startup_migration(
data_dir=home,
runtime_build="21",
sdk_version="2.0.5",
namespace="governed-v2",
open_client=_reading({"m1"}),
)
assert caught.value.report.as_document() == {
"chat_expected": 2,
"chat_readable": 1,
"productivity_expected": 0,
"productivity_readable": 0,
"chat_missing_before_migration": 1,
"passed": False,
}
assert store.authority() is MemoryAuthority.LEGACY
def test_a_build_20_home_gets_its_vectors_back_and_is_checked_against_its_backup(
@@ -1072,7 +1240,13 @@ def test_host_capture_is_fenced_for_the_whole_migration(store):
)
assert result.state == "cutover"
assert fenced == {"bundle": True, "prepare": True, "cutover": True, "cleanup": True}
assert fenced == {
"bundle": True,
"prepare": True,
"vectors": True,
"cutover": True,
"cleanup": True,
}
assert not sdk_migration_capture_fenced(home)
@@ -1175,57 +1349,6 @@ def test_raw_http_dto_uses_the_unified_memory_list_shape():
assert page.raw[0].custom == {"role": "user"}
def test_sdk_raw_retention_keeps_only_the_latest_three_days():
now = datetime(2026, 10, 23, tzinfo=UTC)
class Client:
def __init__(self):
self.rows = [
{
"memory_id": "old",
"memory_at": int((now - timedelta(days=4)).timestamp()),
},
{
"memory_id": "recent",
"memory_at": int((now - timedelta(days=2)).timestamp()),
},
]
def list_memories(self, **_kwargs):
return {
"vdb": {
"memories": list(self.rows),
"total": len(self.rows),
"limit": 500,
"offset": 0,
}
}
def delete(self, memory_id):
before = len(self.rows)
self.rows = [row for row in self.rows if row["memory_id"] != memory_id]
return {
"success": True,
"deleted_count": before - len(self.rows),
}
client = Client()
class Handle:
async def client(self):
return client
runtime = SdkMemoryRuntime(
coordinator=None,
read_handle=Handle(),
write_queue=None,
extraction_settings=None,
data_dir=Path("."),
)
assert asyncio.run(runtime.prune_expired_raw(now=now)) == 1
assert [row["memory_id"] for row in client.rows] == ["recent"]
# The UI asks which memory data plane is authoritative and must get the
# answer the API gates use.
@@ -1353,7 +1476,7 @@ def test_the_sdk_memory_map_draws_sdk_memories_without_legacy_tables(tmp_path) -
original = app.state.sdk_client_handle
app.state.sdk_client_handle = Handle()
async def graph_snapshot(*, limit):
async def graph_snapshot(*, limit, user_id="local"):
del limit
return HyGraphSnapshot(
nodes=(HyGraphNode(node_id="t1", content="topic", layer="TOPIC"),),
@@ -91,7 +91,7 @@ def test_bundle_contains_management_sdk_config_and_verified_manifest(tmp_path):
assert manifest["bundle_id"] == "migration-1"
assert manifest["sdk_namespace"] == "governed-v2"
assert manifest["runtime_build"] == "build-13"
assert manifest["snapshot_sdk_version"] == "2.0.8"
assert manifest["snapshot_sdk_version"] == "2.0.9"
paths = {item["path"]: item for item in manifest["files"]}
assert {
"management/one-memory.sqlite3",
@@ -306,7 +306,7 @@ def test_the_runner_names_the_step_and_sub_step_that_failed(tmp_path, monkeypatc
("step.finished", "preflight.profile_read"),
("step.finished", "preflight.resumable_identity"),
("step.finished", "preflight.legacy_ids_scan"),
("step.finished", "preflight.model_profile_check"),
("step.finished", "preflight.vector_space_check"),
("step.finished", "prepare.recover_abandoned_staging"),
("step.finished", "bundle.ensure"),
("step.finished", "bundle.restore_point"),
@@ -463,11 +463,17 @@ def test_a_run_killed_after_its_restore_point_is_rolled_back_on_the_next_start(h
def test_a_preflight_failure_changes_nothing_and_needs_no_restore(home, monkeypatch):
def unreadable(_data_dir, _profile):
raise DataPlaneMigrationError("sdk_migration_vector_profile_missing")
monkeypatch.setattr("one_memory.sdk_migration_runner.read_active_profile", lambda _db: None)
monkeypatch.setattr(
"one_memory.sdk_migration_runner.vectors_belong_to_another_space", unreadable
)
before = _state(home)
report: dict[str, object] = {}
with pytest.raises(DataPlaneMigrationError, match="sdk_migration_model_profile_required"):
with pytest.raises(DataPlaneMigrationError, match="sdk_migration_vector_profile_missing"):
_run(home, report=report)
assert report == {}
@@ -1,5 +1,6 @@
"""Model-free SDK search remains available without an active write profile."""
import asyncio
from pathlib import Path
from types import SimpleNamespace
@@ -95,3 +96,158 @@ async def test_search_without_profile_uses_public_sdk_search() -> None:
assert options["user_ids"] == ["local"]
assert options["agent_ids"] == []
assert options["scene"] == "normal"
def _activate(profiles, secrets, *, dimensions: int, model: str = "chat"):
from pydantic import SecretStr
from one_memory.model_profiles import ModelCapabilities, ModelProfileInput
profile = profiles.create(
ModelProfileInput(
name=f"embedding-{dimensions}",
provider_kind=ProviderKind.OPENAI_COMPATIBLE,
base_url="https://models.example/v1",
model=model,
embedding_provider_kind=ProviderKind.OPENAI_COMPATIBLE,
embedding_base_url="https://models.example/v1",
embedding_model="qwen3-embedding-4b",
embedding_dimensions=dimensions,
api_key=SecretStr("sk-test"),
),
secrets,
)
profiles.record_capabilities(
profile.profile_id,
expected_revision=profile.revision,
capabilities=ModelCapabilities(
endpoint_reachable=True,
model_available=True,
structured_output=True,
embedding_available=True,
),
secrets=secrets,
)
profiles.activate(profile.profile_id, secrets)
return profiles.snapshot(profile.profile_id, profiles.active_identity()[1])
class _Recorder:
"""Stands in for `acquire_client`: records the collection each client opens."""
def __init__(self, data_dir: Path) -> None:
self.data_dir = data_dir
self.opened: list[tuple[str, int | None, str | None]] = []
self.closed: list[str] = []
def __call__(self, profile, *, api_key, data_dir, http_transport_factory=None, mode="pro"):
from one_memory.hy_sdk_runtime import build_sdk_config
config = build_sdk_config(profile, api_key=api_key, data_dir=data_dir)
name = config.vector_store.collection_name
self.opened.append((name, config.vector_store.embedding_dims, api_key))
return SimpleNamespace(
client=SimpleNamespace(collection=name),
close=lambda: self.closed.append(name),
)
@pytest.mark.anyio
async def test_reads_open_the_collection_the_active_profile_writes(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""#242: a 2560-dim profile wrote `memhub_<sha>_2560`, reads opened `_512`."""
from one_memory.hy_sdk_runtime import build_sdk_config
from one_memory.model_profiles import ModelProfileStore
from one_memory.sdk_read_handle import ActiveProfileReadHandle
from one_memory.secrets import InMemorySecretStore
recorder = _Recorder(tmp_path)
monkeypatch.setattr("one_memory.hy_sdk_runtime.acquire_client", recorder)
profiles = ModelProfileStore(tmp_path / "one-memory.sqlite3")
secrets = InMemorySecretStore()
handle = ActiveProfileReadHandle(profiles=profiles, secrets=secrets, data_dir=tmp_path)
builtin = build_sdk_config(builtin_retrieval_snapshot(), api_key=None, data_dir=tmp_path)
assert (await handle.client()).collection == builtin.vector_store.collection_name
written = _activate(profiles, secrets, dimensions=2560)
writes = build_sdk_config(written, api_key="sk-test", data_dir=tmp_path)
async def read() -> str:
return str((await handle.client()).collection)
assert await asyncio.create_task(read()) == writes.vector_store.collection_name
assert recorder.opened[-1] == (writes.vector_store.collection_name, 2560, "sk-test")
assert writes.vector_store.collection_name != builtin.vector_store.collection_name
await handle.close()
@pytest.mark.anyio
async def test_a_profile_change_rebuilds_reads_without_closing_one_in_flight(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
from one_memory.model_profiles import ModelProfileStore
from one_memory.sdk_read_handle import ActiveProfileReadHandle
from one_memory.secrets import InMemorySecretStore
recorder = _Recorder(tmp_path)
monkeypatch.setattr("one_memory.hy_sdk_runtime.acquire_client", recorder)
profiles = ModelProfileStore(tmp_path / "one-memory.sqlite3")
secrets = InMemorySecretStore()
handle = ActiveProfileReadHandle(profiles=profiles, secrets=secrets, data_dir=tmp_path)
_activate(profiles, secrets, dimensions=1024)
holding = asyncio.Event()
finish = asyncio.Event()
async def slow_read() -> str:
client = await handle.client()
holding.set()
await finish.wait()
return str(client.collection)
in_flight = asyncio.create_task(slow_read())
await holding.wait()
[(old, _, _)] = recorder.opened
_activate(profiles, secrets, dimensions=2560, model="chat-2")
async def read() -> str:
return str((await handle.client()).collection)
new = await asyncio.create_task(read())
assert new != old
# The read that started on the old space still has its client.
assert recorder.closed == []
finish.set()
assert await in_flight == old
for _ in range(5):
await asyncio.sleep(0)
assert recorder.closed == [old]
await handle.close()
assert recorder.closed == [old, new]
def test_every_api_read_shares_the_active_profile_handle(tmp_path: Path) -> None:
"""Library, search, recall, graph, profile, raw, ask and context all read
through `_sdk_client_handle`, so it must be the active-profile handle."""
from one_memory.api import _sdk_client_handle
from one_memory.config import Settings
from one_memory.model_profiles import ModelProfileStore
from one_memory.sdk_read_handle import ActiveProfileReadHandle
from one_memory.secrets import InMemorySecretStore
state = SimpleNamespace(
sdk_client_handle=None,
model_profile_store=ModelProfileStore(tmp_path / "one-memory.sqlite3"),
secret_store=InMemorySecretStore(),
settings=Settings(_env_file=None, data_dir=tmp_path),
)
app = SimpleNamespace(state=state)
handle = _sdk_client_handle(app) # type: ignore[arg-type]
assert isinstance(handle, ActiveProfileReadHandle)
assert _sdk_client_handle(app) is handle is state.sdk_client_handle # type: ignore[arg-type]
@@ -57,10 +57,10 @@ def client_for(app, headers=None):
class Answerer:
async def answer(self, question):
async def answer(self, question, *, user_id="local"):
return SdkMemoryAnswer(answer="真实数据。", memories=(MEMORY,), mode="extractive")
async def answer_stream(self, question):
async def answer_stream(self, question, *, user_id="local"):
yield {"type": "recall", "memories": (MEMORY,)}
yield {
"type": "final",
+4 -4
View File
@@ -991,7 +991,7 @@ wheels = [
[[package]]
name = "hy-memory"
version = "2.0.8"
version = "2.0.9"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "chromadb" },
@@ -1008,9 +1008,9 @@ dependencies = [
{ name = "scikit-learn" },
{ name = "tiktoken" },
]
sdist = { url = "https://files.pythonhosted.org/packages/21/15/83ca83b649ead238f186fdb33efdc73615fbba279efbbdca3857c1a86beb/hy_memory-2.0.8.tar.gz", hash = "sha256:0fa0ed1a1160b389a3d7c13a9fb6528b140d7ec23cf49e65f1be5974820a576b", size = 856025, upload-time = "2026-10-28T07:35:47.783Z" }
sdist = { url = "https://files.pythonhosted.org/packages/48/28/e01fd9a92cd090c8236df0dd06dae049c5de66345e5bd9653497ed56125e/hy_memory-2.0.9.tar.gz", hash = "sha256:5f55ea92044ced19a00b4e56c565215adedd3439a1bdb9cfe9ffe42387c8f1a1", size = 858323, upload-time = "2026-11-01T00:29:49.914Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/11/6a/c699cac0ff06bd089fceeff90446130763d8187e649dbdb0313014491a10/hy_memory-2.0.8-py3-none-any.whl", hash = "sha256:68bd789df24e51e64bb231e97a8d24ee89b4aa5f9ba14c3e0824589e9213bf7c", size = 921119, upload-time = "2026-10-28T07:35:45.571Z" },
{ url = "https://files.pythonhosted.org/packages/d5/44/4368555a408a58daf902c81b62184a3c99f12abd45d4f1144bf79f171a6f/hy_memory-2.0.9-py3-none-any.whl", hash = "sha256:95dbea232d2041f9b0ea3d08dd751603c2a5c0e4422b97e74f664e6ca913549e", size = 923250, upload-time = "2026-11-01T00:29:47.814Z" },
]
[[package]]
@@ -1837,7 +1837,7 @@ requires-dist = [
{ name = "fastapi", specifier = ">=0.141.1" },
{ name = "filelock", specifier = ">=3.32.4" },
{ name = "httpx", specifier = ">=0.28.1" },
{ name = "hy-memory", specifier = "==2.0.8" },
{ name = "hy-memory", specifier = "==2.0.9" },
{ name = "keyring", specifier = ">=25.7.0" },
{ name = "numpy", specifier = "==2.5.2" },
{ name = "onnxruntime", specifier = "==1.23.2" },
@@ -133,6 +133,7 @@ const SAFE_STABLE_RUNTIME_ERROR_CODES = new Set([
"sdk_migration_preflight_failed",
"sdk_migration_memory_unreadable",
"sdk_migration_model_profile_required",
"sdk_migration_vector_profile_missing",
"sdk_migration_model_secret_unavailable",
"sdk_migration_secure_storage_unavailable",
"sdk_migration_disk_space_insufficient",
@@ -219,6 +220,8 @@ export const STABLE_RUNTIME_USER_MESSAGES: Record<string, string> = {
"升级前核对发现,新版本还读不到你的全部记忆,所以没有切换数据源,旧数据保持原样可用。请把这个提示反馈给我们;原有记忆数据不会丢失。",
sdk_migration_model_profile_required:
"升级记忆数据格式需要先有可用的模型配置,用来核对新版本能否读回你的记忆。请在旧版本中完成模型配置后重启;你的记忆数据不会受到影响。",
sdk_migration_vector_profile_missing:
"你的记忆是用一个已删除的模型配置生成向量的,当前没有可用的模型配置能读回它们,所以没有升级。请在旧版本中重新添加那个模型配置并启用后重启;你的记忆数据不会受到影响。",
sdk_migration_model_secret_unavailable:
"模型密钥读不到(可能被系统钥匙串拒绝),无法核对新版本能否读回你的记忆。请允许 MemHub 访问钥匙串后重新启动;你的记忆数据不会受到影响。",
sdk_migration_secure_storage_unavailable:
Loaded 100 of 116 files, more files were not shown because too many files have changed in this diff. Show more