Compare commits

...
Author SHA1 Message Date
alvinfeiandClaude Opus 5.5 e540386db1 fix(sdk): gate near-duplicate demotion with the MMR top-k threshold
near_dup_demote compares each candidate with every earlier
non-duplicate by Jaccard, quadratic when candidates differ. Its pool
grows with limit (about max(3*limit, 30) plus keyword hits), so a
limit-200 page could cost ~180k comparisons. It now runs only when the
caller's limit is below MMR_MAX_TOP_K, like MMR; large pages want every
relevant item, similar ones included. The read reports
near_dup_applied next to mmr_applied.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-27 15:34:41 +08:00
feitianxiang 1374ac26b7 Merge pull request #192 from Tencent-Hunyuan/feat/sdk-search-scenes
feat(sdk): rerank switch, size-gated MMR, scene="all" search and list, productivity edit/delete
2026-10-27 15:22:41 +08:00
alvinfeiandClaude Opus 5.5 47dafbb102 feat(sdk): rerank switch, size-gated MMR, scene="all" search and list, productivity edit/delete
search:
- rerank=True (default) keeps today's behavior; rerank=False skips the
  local cross-encoder, graph rerank and the coding facts' BM25 rerank for
  that call (carried on ReadRequest, so concurrent calls are independent).
- MMR now runs only when the caller's limit is below MMR_MAX_TOP_K (20).
  It is O(k^2 * pool) Jaccard and made large result pages slow.
- scene="all" searches chat, working and coding concurrently with one
  shared embedding and fuses them by Reciprocal Rank Fusion (k=60) over
  each lane's own rank, so no lane is buried by its score scale. Items
  carry scene, kind, memory_id, content, fused score, lane score/rank,
  source and times. A failed or disabled lane is reported in "errors".

list_memories:
- scene="coding"|"working" honour include_sources/exclude_sources with
  the filtered total.
- scene="all" lists the three scenes newest first with limit/offset over
  the merged order, per-scene totals and the same unified item fields.

update/delete:
- kind/variant disambiguate like get(); an experience update replaces
  its solution (task= also replaces the task and its search vectors), a
  fact update replaces the statement; delete removes experiences and
  facts with their vectors. Chat calls are unchanged.
- The HTTP server maps an ambiguous id to 409 and a bad namespace to 400
  instead of 500, passes kind/variant through, and strips the query
  string from PUT/DELETE paths, which leaked it into the memory id.

memory_stats now reads productivity times (written as naive UTC) as UTC.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-27 15:13:12 +08:00
feitianxiang 49ad1b3086 Merge pull request #185 from Tencent-Hunyuan/fix/migration-chroma-settings
fix: list and count every memory scene, and migrate without a Chroma clash
2026-10-27 14:35:45 +08:00
alvinfeiandClaude Opus 5.5 3064515b43 Merge main into fix/migration-chroma-settings
Keep both CHANGELOG entries.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-27 14:30:30 +08:00
feitianxiang e5c8cc3819 Merge pull request #190 from Tencent-Hunyuan/feat/mcp-three-tools
feat(memhub): cut the MCP server to recall, write and diagnostic trace
2026-10-27 14:13:53 +08:00
alvinfeiandClaude Opus 5.5 27f64f1b7a chore(memhub): bump build to 31 and teach new-harness skills the three tools
The custom-Harness connection Skill covered pairing and the MCP config
but never told the model what the tools are, and not every Harness
reads the MCP initialize instructions. It now has a "Using MemHub after
connecting" section for MemHub_recall (including raw-mode results),
MemHub_write and MemHub_diagnostic_trace. The command-only Skill (Pi)
says recall can return original-conversation excerpts and that the
host has no write path.

Build number 30 -> 31.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-27 14:07:09 +08:00
kristinhe1111 3c5451db24 Merge pull request #183 from Tencent-Hunyuan/fix/home-recent-update-metadata
fix(memhub): stabilize recent updates and memory details
2026-10-27 13:22:45 +08:00
alvinfeiandClaude Opus 5.5 a08f69a747 feat(memhub): cut the MCP server to recall, write and diagnostic trace
MCP hosts now see three tools: MemHub_recall, MemHub_write and
MemHub_diagnostic_trace. MemHub_ask, MemHub_conversations,
MemHub_context, MemHub_memory_write and MemHub_memory_write_status are
removed, with their broker routes (/ask, /conversations, /memory-writes)
and client functions. Broker /context stays: the WorkBuddy injection
hook, recall-cli --context and the Hermes provider still use it.

MemHub_write saves one hand-added memory exactly like the app's
"添加记忆": broker POST /memory-notes authenticates the installation and
calls Core /api/v1/memory-notes with trusted headers, so the note takes
the direct-write lane ahead of history and is reconciled through
add_extracted. It takes {text, request_id?} and returns the note receipt
with the request_id to retry with.

MemHub_recall falls back to the SDK raw layer when no generation model
is configured, or when the memory search is empty and the library holds
no memory yet. The receipt says so with mode "raw" and fallback_reason;
raw hits are conversation excerpts (content_format "raw", source,
session_id). Raw reads keep the /context raw-lane authority: a host
needs om_conversations and its granted, unrevoked sources, so recall's
own grant never widens to original conversations.

Stored grants holding om_ask or om_conversations still load; new
pairings get om_recall, om_conversations and om_diagnostic_trace.
Skill text (SKILL_VERSION 19), connection prompt, docs and tests follow.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-27 13:16:55 +08:00
kristinheandCursor 099f5ae7c0 test(memhub): update memory search affordance
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-27 13:13:55 +08:00
kristinhe1111 e7625032d8 Merge pull request #187 from Tencent-Hunyuan/fix/memory-filter-clear-query
fix(memhub): 清空搜索框时移除已提交的记忆关键词
2026-10-27 13:11:07 +08:00
kristinhe1111 3aa3b5957d Merge pull request #186 from Tencent-Hunyuan/fix/hy-debug-pipeline-logs
fix(memhub): surface HY pipeline logs in Debug
2026-10-27 12:24:15 +08:00
kristinheandCursor 409ab9a9b4 fix(memhub): clear submitted memory query with input
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-27 12:22:55 +08:00
alvinfeiandClaude Opus 5.5 c2f50dce06 fix: claim hand-added memories first, and never let older memories win
MemHub: "添加记忆" (add a memory) went into the same durable queue as
history, which is claimed through a ten-minute event-time window, oldest
first. A note is stamped "now", so with a backlog it ran only after the
whole history. Jobs from one-memory-manual are now claimed ahead of the
window, still one at a time per source and in session order; history
keeps its strict timeline.

SDK reconcile: memories are not written in the order they happened, so a
new memory can be older than an existing one. Both reconcile prompts now
say only memory_at decides which is newer, that an older memory never
supersedes or overwrites a newer one (it is added as a past state), and
that an unknown time is not newer. The Chinese prompt now lists memory_at
among the new-memory fields, which it had left out. enforce_time_order
makes this hold in code: a SUPERSEDE or UPDATE aimed at a memory with a
later memory_at becomes an ADD.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-27 12:21:01 +08:00
kristinheandCursor 2735d8693a fix(memhub): prioritize memory list space
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-27 12:19:49 +08:00
kristinheandCursor 47bbfb4b27 fix(memhub): simplify memory note copy
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-27 12:11:54 +08:00
kristinheandCursor 3f169c9951 style(memhub): format language selector
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-27 12:01:58 +08:00
kristinheandCursor 24f699a726 fix(memhub): surface HY pipeline logs in Debug
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-27 11:43:35 +08:00
kristinheandCursor cb97aa5822 fix(memhub): show attributed app calls on home
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-27 11:43:25 +08:00
alvinfeiandClaude Opus 5.5 38a4da6849 fix: list and count every memory scene, and migrate without a Chroma clash
SDK: list_memories(scene="chat"|"coding"|"working") lists one kind of
memory. coding and working list the scene's whole store, newest update
first, with workspace_id still narrowing it, and total is the real count
instead of the page size. memory_stats() returns each scene's count and
latest created/updated time plus their total, reading no memory text.
Without a workspace, coding and working memories could not be listed or
counted, so a 45-memory library listed 22.

MemHub migration: reattaching legacy vectors opened Chroma with its own
settings and never closed it, so the readback that followed in the same
process could not open the store ("An instance of Chroma already exists
... with different settings"). Every 1.0.4 home with legacy vectors
failed its first upgrade launch. Offline code now opens Chroma as the SDK
does and closes it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-27 11:41:10 +08:00
kristinheandCursor 48a5d70026 fix(memhub): stabilize language setting layout
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-27 11:20:53 +08:00
kristinheandCursor b21c929728 fix(memhub): stabilize memory detail layout
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-27 11:12:47 +08:00
kristinheandCursor b9b3c587ff fix(memhub): hide recent update internals
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-27 11:01:14 +08:00
feitianxiang 5a5650fc94 Merge pull request #182 from Tencent-Hunyuan/chore/bump-build-30
chore(release): bump MemHub build number to 30
2026-10-27 09:21:00 +08:00
alvinfeiandClaude Opus 5.5 0e149b6c9c chore(release): bump MemHub build number to 30
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-27 09:17:07 +08:00
feitianxiang 0dd2bcd6b1 Merge pull request #181 from Tencent-Hunyuan/fix/core-ready-before-recovery
fix(memhub): make Core ready before it resumes unfinished work
2026-10-27 09:13:25 +08:00
alvinfeiandClaude Opus 5.5 567a8b349d test(desktop): let the attach-race winner exist before the loop gives up
The test wrote the winning Core's receipt and token from inside the
spawn, without awaiting it, while the readiness loop ran with zero
delay. On a slow runner the loop finished first, the re-read found a
receipt without its token, and start() reported a false conflict. The
probe now waits for the winner's files, which is the order the test is
about.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-27 09:00:50 +08:00
alvinfeiandClaude Opus 5.5 d89afb3887 fix(memhub): make Core ready before it resumes unfinished work
A home with pending imports and write jobs kept /health closed while
the lifespan resumed them, so the Runtime hit its 60s readiness budget,
killed a healthy Core with core_start_timeout, and the next Core began
the same recovery again. The write drain also started before readiness
and held the single writer the rest of startup needed.

Readiness is now only "the stores are open and Core answers". After
core.ready a recovery task runs, step by step: the post-cutover
metadata repair, the durable write drain, raw retention (or the legacy
pending-dispatch replay), interrupted imports, live sync, source
reconcile and the legacy catch-up. A failed step is logged and named in
/health without stopping readiness or the steps after it. /health
reports recovery as pending, running, done or failed. Shutdown cancels
recovery first and still releases the content-backfill gate.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-26 23:59:38 +08:00
feitianxiang 0abf87d4b6 Merge pull request #180 from Tencent-Hunyuan/fix/memhub-sdk-only-migration-guard
fix(memhub): serve SDK only and stop on an unfinished upgrade migration
2026-10-26 22:34:53 +08:00
alvinfeiandClaude Opus 5.5 4e5a44b8dc fix(memhub): pass the lint gate
Format cli.test.ts, and mark the map's search form role as intentional:
biome's useSemanticElements also fails main (#179 merge), and a <search>
wrapper would add a box around the flex row .map-search styles.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-26 22:33:12 +08:00
alvinfeiandClaude Opus 5.5 bf36ba765b fix(memhub): serve SDK only and stop on an unfinished upgrade migration
Every 1.0.4 home must pass the offline SDK cutover, but the runtime
service could start Core on a home whose migration failed, so it served
legacy. Callers also asked Core for the memory authority before many
calls and kept v1 data-plane branches that only ever 409 on SDK.

Migration and startup:
- Write every migration step to logs/sdk-migration.log (content-free
  JSON lines) from Desktop, CLI, the runtime service and Core.
- Migrate before the runtime service spawns Core; Core refuses a legacy
  home with sdk_migration_required.
- Desktop shows a dialog that reveals the log and opens a GitHub issue,
  then quits. CLI install/upgrade no longer roll back to a version
  without the gate; start/restart/status/doctor name the log.
- Cycle the service when the migration status is unreadable so no
  pre-upgrade Runtime keeps serving.

SDK only:
- Web, host-bridge and CLI call v2 only; drop the authority query.
- Delete Core's v1 data plane, the Reader and /api/v2/memory-authority.
- Fix MCP memory write on v2, MemHub_conversations, memhub recall and
  status, v2 call events and WorkBuddy delivery reports.
- Remove dead features: matters, boundary peek/ack, memory plan/apply,
  the legacy library and its v1 operations.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-26 22:14:46 +08:00
feitianxiang e9cce69650 Merge pull request #179 from Tencent-Hunyuan/fix/memory-search-button
fix(memhub): search on submit and set SDK recall limits apart
2026-10-26 16:08:51 +08:00
alvinfeiandClaude Opus 5.5 2fed7adf6b fix(memhub): set the SDK context recall lanes apart
POST /api/v2/context/search passed the caller's `limit` as the normal,
schema and proactive intention top-k alike, so limit=5 ran three full
recall lanes including the intention lane the SDK keeps off by default.
The v2 request now takes `limit` (default 10), `schema_limit` (default
3) and `intention_limit` (default 1) and passes each through. The host
broker's default item cap follows v2's default of 10.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-26 15:48:48 +08:00
alvinfeiandClaude Opus 5.5 7308ed9df2 fix(memhub): search the memory list on submit, over 20 results
- The map's memory search no longer queries on every keystroke: typing
  edits the words and the search runs on the 搜索 button or Enter.
  Submitting the words already applied re-reads the page.
- Core asks SDK search for a 20-memory relevance window instead of 500;
  the larger window made a single search slow enough to freeze the page.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-26 15:27:20 +08:00
273 changed files with 10359 additions and 34583 deletions

No files matched your search

+50
View File
@@ -4,6 +4,56 @@ Hy Memory and MemHub are versioned and released independently.
## Unreleased
- SDK: `search(..., rerank=False)` skips every reranking stage for that one
call (the local cross-encoder, the memory-graph rerank, and the BM25 rerank of
working/coding facts), so results come back in retrieval-score order. The
flag travels on the read request, so concurrent searches do not affect each
other. The default (`True`) keeps current behavior; `/api/v1/search` accepts
it too.
- SDK: MMR diversity selection now runs only when the requested `limit` is
below 20 (`MMR_MAX_TOP_K`) and is skipped for larger limits, where its
quadratic token-overlap pass is slow and gains little. Previously it ran for
every limit. `MEMORY_HYBRID_V2_MMR_LAMBDA` still sets the weight.
Near-duplicate demotion follows the same gate: it compares each candidate
with every earlier non-duplicate, which is quadratic in a pool that grows
with `limit`, and large result pages should show similar items anyway.
- SDK: `search(scene="all")` searches chat, working and coding at the same time
and merges them with Reciprocal Rank Fusion (k=60) over each scene's own
ranking, so no scene is crowded out by the other scenes' score scales. Each
result says which scene and kind it came from. A scene that fails or is turned
off is reported in `errors` and does not fail the search. Source filters apply
to all three.
- SDK: `list_memories(scene="all")` lists chat, working and coding memories
together, newest update first, with `limit`/`offset` paging over the merged
order and `total` summing all three. `scene="coding"` and `"working"` now
honor `include_sources` / `exclude_sources`, and `total` counts only the
filtered memories.
- SDK: `update()` and `delete()` now work on working/coding experiences and
facts, not just chat memories. They take the same optional `kind` / `variant`
as `get()`, and an ID found in more than one place raises
`AmbiguousMemoryIdError` instead of guessing. Updating an experience replaces
its solution and keeps its task, unless a new `task=` is also passed; updating
a fact replaces its statement. Deleting removes the stored row and its vectors.
- SDK: `list_memories(scene="chat" | "coding" | "working")` lists one kind of
memory. `coding` and `working` list that scene's whole store, newest update
first (`workspace_id` still narrows it), and `total` is now the real count
rather than the page size. The new `memory_stats()` returns the count and the
latest created/updated time for each scene and their total, without reading
any memory text. Previously coding and working memories could not be listed
or counted without a workspace, so a library of 45 memories listed 22.
- The SDK migration no longer fails its first run on a home with legacy
vectors: reattaching them left a Chroma instance open with its own settings,
and the readback that followed could not open the same store
("An instance of Chroma already exists ... with different settings").
- MemHub "添加记忆" (add a memory) no longer waits behind unprocessed history.
Notes are claimed ahead of the event-time window that keeps history strictly
old to new; they still run one at a time and in order within their session.
- SDK reconcile keeps time order. Both reconcile prompts now state that only
`memory_at` decides which memory is newer, that an older memory must never
supersede or overwrite a newer one, and that an unknown time is not newer;
the Chinese prompt now lists `memory_at` among the new-memory fields. The
reconciler also enforces this in code: a SUPERSEDE or UPDATE aimed at a memory
with a later `memory_at` becomes an ADD, so the newer memory stays current.
- The SDK memory page now reads its list, count, source and time labels and
graph from one page response for one source filter, so rows, totals and the
map no longer disagree. Memories with no recorded source are shown and
+14 -7
View File
@@ -743,6 +743,7 @@ RECONCILE_PROMPT_ZH = """你是一个记忆整合系统(Memory Reconciler)
* content
* owner
* memory_at(发生时间)
* tags
new memories 没有 ID。
@@ -791,6 +792,7 @@ new memories 没有 ID。
* 无损合并
* 保留所有有效信息
* 不要产生重复 memory
* 两者在某个值上不一致时,保留 memory_at 更晚的那个值;绝不用更早的 memory 覆盖更新的值
### 同一次事件的补充 vs 另一次发生
@@ -823,6 +825,7 @@ new memories 没有 ID。
* 新 memory 与旧 memory 在同一维度冲突
* 两者不能同时为真
* 旧 memory 已不再代表当前状态
* 新 memory 的 memory_at 不早于旧 memory 的 memory_at
例如:
@@ -837,6 +840,7 @@ new memories 没有 ID。
* wording 差异
* 同义表达
* 可共存偏好
* 新 memory 的 memory_at 早于旧 memory:它是更早的状态,绝不能取代更新的状态(见「时间规则」)
要求:
@@ -859,7 +863,7 @@ new memories 没有 ID。
### 把多条旧记忆折叠进同一条链
如果**两条或更多**已有记忆描述的是**同一维度**,而新记忆是该维度上的最新变化,
如果**两条或更多**已有记忆描述的是**同一维度**,而新记忆是该维度上的最新变化(它的 memory_at 晚于所有被折叠的记忆),
你可以把它们折叠进同一条演化链。此时用 `memory_ids`(有序列表,**旧→新**)
代替单个 `memory_id`:
@@ -951,13 +955,16 @@ SUPERSEDE 不需要复制旧内容,因为旧节点仍存在。
## 时间规则
`memory_at: null`
每条 memory(新的和已有的)都带 memory_at,表示它发生的时间。记忆**不一定按发生顺序写入**:
历史可能晚导入,导入还没处理完时用户也可能手动添加记忆。所以一条 new memory 可能**比已有记忆更早**。
表示时间未知。
不要编造时间顺序。
不要仅因为“看起来更新”就 supersede。
* 新旧**只看 memory_at**。"New memories" 只表示"正在写入",不表示"更近"。
* 更早的 memory 绝不能取代更新的 memory:不能 SUPERSEDE 一条更新的记忆,也不能用 UPDATE 拿更早的值覆盖更新的值。
* 如果 new memory 比与它冲突的已有记忆更早:保留已有记忆作为当前状态;把 new memory 以"过去状态"ADD
(用过去时,并带上时间,例如"2023 年用户住在上海"),没有新信息就不输出。
* `memory_at: null` 表示时间未知。未知**不算更新**:不要用 memory_at 为 null 的 memory 去 SUPERSEDE 一条有时间的记忆。
* 不要编造时间顺序。
* 不要仅因为“看起来更新”就 supersede。
---
+105 -3
View File
@@ -221,6 +221,8 @@ Rules:
* merge losslessly
* preserve all useful information
* do NOT create duplicates
* if the two disagree on a value, keep the value from the memory with the later
`memory_at`; never let an older memory overwrite a newer value
### Same occurrence vs. another occurrence
@@ -255,6 +257,7 @@ Use ONLY when:
* the new memory contradicts an existing memory
* both cannot be true at the same time
* the old memory would now give the wrong current-state answer
* the new memory's `memory_at` is the same as or LATER than the existing memory's `memory_at`
Examples:
@@ -268,6 +271,8 @@ Do NOT use for:
* compatible preferences
* wording differences
* semantic equivalence
* a new memory whose `memory_at` is EARLIER than the existing memory's — it is an
older state and must never replace a newer one (see "Time order")
Rules:
@@ -289,7 +294,8 @@ Rules:
### Folding several existing memories into one chain
If TWO OR MORE existing memories track the SAME dimension and the new memory is
the latest change on that dimension, you may fold them into a single evolution
the latest change on that dimension — its `memory_at` is later than every folded
memory's `memory_at` — you may fold them into a single evolution
chain. Use `memory_ids` (an ordered list, OLDEST → NEWEST) instead of a single
`memory_id`:
@@ -318,8 +324,7 @@ older memories' text into it; they remain queryable in the chain history.
* Separate dated occurrences stay separate ADDs; use `same_series` only for the
same subject and same concrete activity/event type.
* Keep each memory focused on one coherent topic.
* Do not invent chronology.
* `memory_at: null` means unknown time.
* Follow "Time order" below.
* **NEVER merge or supersede across different subjects.** The subject is whoever
the content is about: "The user ..." vs "John ..." vs "Mary ...". Memories about
different people are independent facts even when the sentence is otherwise
@@ -332,6 +337,27 @@ older memories' text into it; they remain queryable in the chain history.
---
# Time order
Every memory, new or existing, has `memory_at`: when it happened. Memories are
NOT always written in the order they happened — history can be imported late,
and a memory can be added while older history is still being processed. A new
memory can therefore be OLDER than an existing one.
* Decide which memory is newer ONLY by `memory_at`. "New memories" means "being
written now", not "more recent".
* An older memory must never replace a newer one: no SUPERSEDE of a newer
memory, and no UPDATE that overwrites a newer value with an older one.
* If a new memory is older than an existing memory it conflicts with, keep the
existing memory as the current state. ADD the new one as a past state, worded
in the past with its time (for example "In 2023 the user lived in Shanghai"),
or output nothing if it adds no information.
* `memory_at: null` means unknown time. Unknown is NOT newer: never SUPERSEDE a
memory that has a known `memory_at` with one whose `memory_at` is null.
* Do not invent chronology.
---
# Optional: `related`
On any op you MAY add a `related` array recording how the memory being written
@@ -441,6 +467,81 @@ def append_reconcile_structure_prompt(
return prompt + RECONCILE_STRUCTURE_METADATA_PROMPT_EN
def _as_aware(value: Any) -> Optional[datetime]:
"""memory_at as a comparable aware datetime; None when unknown or unreadable."""
if isinstance(value, str) and value:
try:
value = datetime.fromisoformat(value)
except (TypeError, ValueError):
return None
if not isinstance(value, datetime):
return None
# Stored nodes rebuild memory_at from epoch seconds as naive local time
# (MemoryNode.from_dict), so a naive value is local, not UTC.
return value if value.tzinfo is not None else value.astimezone()
def enforce_time_order(
ops: List["ReconcileOp"],
new_memory_at: Any,
existing: Dict[str, Any],
) -> List["ReconcileOp"]:
"""An older memory must never replace a newer one, whatever the model says.
Memories are not always written in the order they happened: history is
imported late, and a person can add a memory while older history is still
being processed. The prompt tells the model to compare `memory_at`; this
makes it hold. A SUPERSEDE or UPDATE aimed at an existing memory whose
`memory_at` is later than the memory being written is turned into an ADD,
so the newer memory stays the current state. Unknown times are not
compared. A content-less UPDATE (the legacy DELETE mapping) is dropped.
"""
written_at = _as_aware(new_memory_at)
if written_at is None:
return ops
kept: List[ReconcileOp] = []
for op in ops:
if op.op not in ("SUPERSEDE", "UPDATE"):
kept.append(op)
continue
targets = list(op.memory_ids) if op.memory_ids else [op.memory_id]
newer = [
target for target in targets
if target and (node := existing.get(target)) is not None
and (target_at := _as_aware(getattr(node, "memory_at", None))) is not None
and target_at > written_at
]
if not newer:
kept.append(op)
continue
logger.warning(
"[reconciler] %s blocked: memory_at %s is older than existing %s; "
"kept the newer memory",
op.op, written_at.isoformat(), ",".join(newer),
)
if not op.content:
continue
kept.append(ReconcileOp(
op="ADD",
content=op.content,
layer=op.layer,
owner=op.owner,
tags=list(op.tags),
tag_ners=list(op.tag_ners),
topic_paths=list(op.topic_paths),
event_facet=dict(op.event_facet),
topic_assignments=list(op.topic_assignments),
structure_references=list(op.structure_references),
topic_paths_provided=op.topic_paths_provided,
event_facet_provided=op.event_facet_provided,
topic_assignments_provided=op.topic_assignments_provided,
speculate=op.speculate,
related=list(op.related),
reason=f"time_order: older than {','.join(newer)}; {op.reason}"[:500],
))
return kept
def build_reconcile_prompt(
*,
new_memories: List[str],
@@ -1399,6 +1500,7 @@ class MemoryReconciler:
_tok_total += response.tokens_used
raw_response = response.content
ops = self._parse_ops(raw_response)
ops = enforce_time_order(ops, current_time, candidate_map)
ops = self._inherit_exact_input_structure(
ops, new_memories_with_meta,
)
+1055 -78
View File
File diff suppressed because it is too large. Load diff
+162 -4
View File
@@ -217,6 +217,21 @@ def _in_clause(column: str, values: Optional[List[str]]) -> Tuple[str, List[Any]
return f"{column} IN ({placeholders})", list(values)
def _source_where(
include_sources: Optional[List[str]], exclude_sources: Optional[List[str]],
) -> Tuple[str, List[Any]]:
"""Caller-source filter as SQL (``1`` when unfiltered); exclusion wins."""
clauses: List[str] = []
params: List[Any] = []
if include_sources:
clauses.append(f"source IN ({','.join('?' for _ in include_sources)})")
params.extend(include_sources)
if exclude_sources:
clauses.append(f"source NOT IN ({','.join('?' for _ in exclude_sources)})")
params.extend(exclude_sources)
return (" AND ".join(clauses) or "1"), params
def _scope_matches(
row_workspace_id: Optional[str],
row_branch: Optional[str],
@@ -603,6 +618,86 @@ class CodingMemoryStore:
logger.info(f"[coding-store] deleted experience_id={experience_id}")
return True
async def edit_experience(
self,
experience_id: str,
*,
user_id: Optional[str] = None,
task: Optional[str] = None,
solution: Optional[str] = None,
search_keys: Optional[List[str]] = None,
) -> Optional[CodingMemory]:
"""调用方手动改一条经验(client.update 用)。返回改后的经验;不存在返回 None。
与 update() 的区别:update 是写入链路的覆盖式重写,带写授权校验(同
agent/session/workspace 才能改);这里是用户对已有条目的直接编辑,只动
给定的字段,其余原样保留,因此不做授权校验。
向量行只嵌 task + search_keys(solution 不进向量),所以只改 solution 时
不重嵌、不碰向量库;改了 task / search_keys 才删旧 key 行重写。
updated_at 总是刷新。
"""
prev = await self.get_by_id(experience_id, user_id=user_id)
if prev is None:
return None
updated = deepcopy(prev)
if task is not None:
updated.task = task
if solution is not None:
updated.solution = solution
if search_keys is not None:
updated.search_keys = list(search_keys)
updated.updated_at = datetime.utcnow()
await _run_in_pool(self._upsert_sql_sync, updated)
if updated.task != prev.task or updated.search_keys != prev.search_keys:
await self._delete_vec_keys_by_count(experience_id, 1 + len(prev.search_keys))
await self._upsert_vec_keys(updated)
logger.info(f"[coding-store] edited experience_id={experience_id}")
return updated
async def edit_fact(
self,
fact_id: str,
*,
statement: str,
user_id: Optional[str] = None,
) -> Optional[WorkingFact]:
"""调用方手动改一条事实的 statement。返回改后的事实;不存在返回 None。
statement 是事实层的检索文本,所以 active 事实会重嵌并覆盖它那一行向量
(node_id 就是 fact_id,upsert 即覆盖)。superseded 事实的向量行早已撤掉,
这里不把它加回去,否则会让过期结论重新占召回名额。
"""
prev = await self.get_fact(fact_id, user_id=user_id)
if prev is None:
return None
updated = deepcopy(prev)
updated.statement = statement
await _run_in_pool(self._upsert_fact_sql_sync, updated)
if updated.status == "active":
await self._upsert_fact_vec(updated)
logger.info(f"[coding-store] edited fact_id={fact_id}")
return updated
async def delete_fact(self, fact_id: str, user_id: Optional[str] = None) -> bool:
"""物理删除一条事实(SQLite 行 + 事实向量行),返回是否真有删到。
别处留下的引用(经验的 source_fact_ids、后继事实的 fact_link)不清:
召回侧本来就按存在性过滤死链(get_many / get_facts 查不到自然掉队)。
"""
prev = await self.get_fact(fact_id, user_id=user_id)
if prev is None:
logger.info(f"[coding-store] delete fact skipped (not found): {fact_id}")
return False
if self._fact_vec is not None:
try:
await self._fact_vec.delete(fact_id)
except Exception as e:
logger.debug(f"[coding-store] delete fact vec {fact_id} ignored: {e}")
await _run_in_pool(self._delete_fact_sql_sync, fact_id, prev.user_id)
logger.info(f"[coding-store] deleted fact_id={fact_id}")
return True
# ------------------------------------------------------------
# 读取
# ------------------------------------------------------------
@@ -687,29 +782,84 @@ class CodingMemoryStore:
branch: Optional[str] = None,
limit: int = 100,
offset: int = 0,
include_sources: Optional[List[str]] = None,
exclude_sources: Optional[List[str]] = None,
) -> List["CodingMemory"]:
"""按 workspace_id(+可选 branch)纯过滤列出该 user 的完整 coding memory。
作用域规则见 _scope_where(与 search 路径共用同一套)。无语义排序,按
updated_at 倒序(最近修改优先)。
updated_at 倒序(最近修改优先)。include_sources / exclude_sources 按
source 列在 SQL 里过滤(排除优先),所以 LIMIT/OFFSET 在过滤之后生效。
"""
return await _run_in_pool(
self._list_by_workspace_sync, user_id, workspace_id, branch, limit, offset
self._list_by_workspace_sync, user_id, workspace_id, branch, limit, offset,
include_sources, exclude_sources,
)
def _list_by_workspace_sync(
self, user_id: str, workspace_id: str, branch: Optional[str],
limit: int, offset: int,
include_sources: Optional[List[str]] = None,
exclude_sources: Optional[List[str]] = None,
) -> List["CodingMemory"]:
scope_sql, scope_params = _scope_where(workspace_id, branch)
source_sql, source_params = _source_where(include_sources, exclude_sources)
with self._lock:
assert self._conn is not None
sql = (f"SELECT * FROM coding_memory_meta WHERE user_id = ? AND {scope_sql} "
f"ORDER BY updated_at DESC LIMIT ? OFFSET ?")
params: List[Any] = [user_id] + scope_params + [limit, offset]
f"AND {source_sql} "
f"ORDER BY updated_at DESC, experience_id ASC LIMIT ? OFFSET ?")
params: List[Any] = ([user_id] + scope_params + source_params
+ [limit, offset])
cur = self._conn.execute(sql, params)
return [_row_to_memory(r) for r in cur.fetchall()]
async def stats(
self,
user_id: str,
workspace_id: Optional[str] = None,
branch: Optional[str] = None,
include_sources: Optional[List[str]] = None,
exclude_sources: Optional[List[str]] = None,
) -> Dict[str, Any]:
"""经验数、最新创建/修改时间、active 事实数。纯聚合,不读正文。
作用域与 list_by_workspace 相同:workspace_id 为空表示该 user 全部;
来源过滤同样与 list_by_workspace 一致,count 因此恒等于分页可翻到的总数。
"""
return await _run_in_pool(
self._stats_sync, user_id, workspace_id, branch,
include_sources, exclude_sources,
)
def _stats_sync(
self, user_id: str, workspace_id: Optional[str], branch: Optional[str],
include_sources: Optional[List[str]] = None,
exclude_sources: Optional[List[str]] = None,
) -> Dict[str, Any]:
scope_sql, scope_params = _scope_where(workspace_id, branch)
source_sql, source_params = _source_where(include_sources, exclude_sources)
with self._lock:
assert self._conn is not None
row = self._conn.execute(
f"SELECT COUNT(*) AS n, MAX(created_at) AS created, "
f"MAX(updated_at) AS updated FROM coding_memory_meta "
f"WHERE user_id = ? AND {scope_sql} AND {source_sql}",
[user_id] + scope_params + source_params,
).fetchone()
facts = self._conn.execute(
f"SELECT COUNT(*) AS n FROM working_fact "
f"WHERE user_id = ? AND status = 'active' AND {scope_sql} "
f"AND {source_sql}",
[user_id] + scope_params + source_params,
).fetchone()
return {
"count": int(row["n"] or 0),
"latest_created_at": _parse_dt(row["created"]) if row["created"] else None,
"latest_updated_at": _parse_dt(row["updated"]) if row["updated"] else None,
"facts": int(facts["n"] or 0),
}
# ------------------------------------------------------------
# 召回
# ------------------------------------------------------------
@@ -1088,6 +1238,14 @@ class CodingMemoryStore:
(experience_id, user_id),
)
def _delete_fact_sql_sync(self, fact_id: str, user_id: str) -> None:
with self._lock:
assert self._conn is not None
self._conn.execute(
"DELETE FROM working_fact WHERE fact_id = ? AND user_id = ?",
(fact_id, user_id),
)
def _get_by_id_sync(
self, experience_id: str, user_id: Optional[str]
) -> Optional[CodingMemory]:
@@ -160,6 +160,28 @@ def near_dup_demote(
return out
# MMR 只在调用方请求的 top-k(search 的 limit)小于这个值时自动启用。
# mmr_select 的贪心是 O(k^2 * pool) 次 Jaccard,k 大时代价陡增,而大 k 本身已
# 足够容纳多样性,收益很小。权重仍由 MEMORY_HYBRID_V2_MMR_LAMBDA 控制。
MMR_MAX_TOP_K = 20
def mmr_applies(top_k: int, lambda_: float) -> bool:
"""本次检索是否走 MMR:λ 在有效区间内,且 top_k < MMR_MAX_TOP_K。"""
return 0.0 < float(lambda_) < 0.999 and int(top_k) < MMR_MAX_TOP_K
def near_dup_applies(top_k: int) -> bool:
"""本次检索是否走近重复降权:与 MMR 同一个 top-k 门槛。
near_dup_demote 让每个候选和此前所有"非重复"候选两两算 Jaccard,候选
互不相似时是 O(pool^2);池子随 limit 放大(约 max(3*limit, 30) + 关键词
命中),limit 达到 MMR_MAX_TOP_K 时代价同样陡增。大 k 的调用方(如记忆
面板翻页)要的是全部相关项,相似项也应照常出现,降权没有收益。
"""
return int(top_k) < MMR_MAX_TOP_K
def mmr_select(
hits: Sequence[Dict[str, Any]],
limit: int,
+5
View File
@@ -401,6 +401,11 @@ class ReadRequest:
intention_limit: int = 0 # Intention 路的 topk,默认 0 关闭
# 重排开关(按次生效)。False 时跳过全部重排阶段:本地 cross-encoder
# (recall.local_rerank_enabled) 与记忆关系图重排 (_apply_graph_rerank),
# 结果按召回分排序。走请求对象而非环境变量,并发请求互不影响。
rerank: bool = True
# 时间过滤
created_after: Optional[float] = None # Unix timestamp (float),只返回 gmt_created >= 此值的记忆
+18 -5
View File
@@ -57,7 +57,9 @@ from ._retrieval.ner import (
memory_matches_time_hints,
)
from ._retrieval.diversity import (
near_dup_applies,
near_dup_demote,
mmr_applies,
mmr_select,
social_boilerplate_demote,
pool_df_demote,
@@ -1279,7 +1281,10 @@ class BaseReadPipeline(ReadPipeline):
SAME_SERIES_TIMELINE_ON = bool(
getattr(self.config.recall, "same_series_timeline_enabled", False)
)
LOCAL_RERANK_ON = bool(
# 按次的重排开关(search(rerank=False)):关掉本地 cross-encoder 与图重排,
# 结果按召回分排序。只读请求对象,不碰共享状态,并发请求互不影响。
RERANK_ON = bool(getattr(request, "rerank", True))
LOCAL_RERANK_ON = RERANK_ON and bool(
getattr(self.config.recall, "local_rerank_enabled", False)
)
LOCAL_RERANK_POOL_FACTOR = max(1, int(
@@ -1431,6 +1436,11 @@ class BaseReadPipeline(ReadPipeline):
MIN_SCORE = _resolve_normal_min_score(request.min_score)
final_limit = request.limit if request.limit > 0 else 10
# MMR 只在调用方的 top-k(final_limit,即 search 的 limit)< MMR_MAX_TOP_K
# 时启用;池子(LOCAL_RERANK 下是 final_limit 的若干倍)不参与判定。
MMR_ON = mmr_applies(final_limit, MMR_LAMBDA)
# 近重复降权同样是候选两两比较,用同一个 top-k 门槛。
NEAR_DUP_ON = near_dup_applies(final_limit)
vdb_sem_limit = max(final_limit * 3, 30)
vdb_kw_limit = max(final_limit * 5, 80)
graph_limit = max(
@@ -2462,7 +2472,7 @@ class BaseReadPipeline(ReadPipeline):
# 只把已选出的 top-N 换个顺序收益很小。
relation_paths: List[Dict[str, Any]] = []
_graph_stats: Dict[str, Any] = {}
if self._expansion_on():
if RERANK_ON and self._expansion_on():
_t_graph = datetime.now()
_pool_before = len(normal_results)
normal_results, relation_paths, _graph_stats = await self._apply_graph_rerank(
@@ -2525,7 +2535,7 @@ class BaseReadPipeline(ReadPipeline):
df_thr=POOL_DF_THR,
penalty=POOL_DF_PENALTY,
)
if NEAR_DUP_THR > 0 and NEAR_DUP_PENALTY > 0:
if NEAR_DUP_ON and NEAR_DUP_THR > 0 and NEAR_DUP_PENALTY > 0:
normal_results = near_dup_demote(
normal_results,
jaccard_thr=NEAR_DUP_THR,
@@ -2538,7 +2548,7 @@ class BaseReadPipeline(ReadPipeline):
))
if LOCAL_RERANK_ON else final_limit
)
if 0.0 < MMR_LAMBDA < 0.999 and len(normal_results) > _pre_rerank_limit:
if MMR_ON and len(normal_results) > _pre_rerank_limit:
normal_results = mmr_select(
normal_results,
_pre_rerank_limit,
@@ -2715,7 +2725,7 @@ class BaseReadPipeline(ReadPipeline):
reverse=True,
)
if (
0.0 < MMR_LAMBDA < 0.999
MMR_ON
and len(multi_seed_pool) > _pre_rerank_limit
):
multi_seed_pool = mmr_select(
@@ -3544,6 +3554,9 @@ class BaseReadPipeline(ReadPipeline):
"episode_capsule_render_mode": EPISODE_CAPSULE_RENDER_MODE,
"same_series_timeline_enabled": SAME_SERIES_TIMELINE_ON,
"local_rerank_enabled": LOCAL_RERANK_ON,
"rerank": RERANK_ON,
"mmr_applied": MMR_ON,
"near_dup_applied": NEAR_DUP_ON,
"local_rerank_model": getattr(
self.config.recall,
"local_rerank_model",
+45 -7
View File
@@ -33,9 +33,28 @@ import threading
from datetime import datetime
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from typing import Any, Dict, Optional
from urllib.parse import parse_qs, urlparse
from .retrieval import AmbiguousMemoryIdError
logger = logging.getLogger("hy_memory.server")
def _namespace_args(values: Dict[str, Any]) -> Dict[str, Any]:
"""kind / variant from a query string (lists) or a JSON body (scalars).
They disambiguate an id that exists in more than one namespace (chat,
experience, fact; working or coding). Absent means "resolve it".
"""
args: Dict[str, Any] = {}
for key in ("kind", "variant"):
value = values.get(key)
if isinstance(value, list):
value = value[0] if value else None
if value:
args[key] = str(value)
return args
# Lazy-init global client
_client = None
_client_lock = None
@@ -148,7 +167,6 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
if path == "/api/v1/metrics":
minutes = 5
if "?" in self.path:
from urllib.parse import parse_qs, urlparse
qs = parse_qs(urlparse(self.path).query)
minutes = int(qs.get("minutes", ["5"])[0])
client = _get_client()
@@ -161,7 +179,8 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
if m:
memory_id = m.group(1)
client = _get_client()
result = client.get(memory_id)
query = parse_qs(urlparse(self.path).query)
result = client.get(memory_id, **_namespace_args(query))
if result is None:
_json_response(self, 404, {"error": "not_found", "memory_id": memory_id})
else:
@@ -170,6 +189,10 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
_json_response(self, 404, {"error": "not_found", "path": path})
except AmbiguousMemoryIdError as e:
_json_response(self, 409, {"error": "ambiguous_memory_id", "detail": str(e)})
except ValueError as e:
_json_response(self, 400, {"error": str(e)})
except Exception as e:
logger.error(f"[server] GET {path} error: {e}", exc_info=True)
_json_response(self, 500, {"error": str(e)})
@@ -213,7 +236,7 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
_json_response(self, 500, {"error": str(e)})
def do_PUT(self):
path = self.path.rstrip("/")
path = self.path.split("?")[0].rstrip("/")
body = _read_json_body(self)
if body is None:
_json_response(self, 400, {"error": "invalid JSON body"})
@@ -228,29 +251,43 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
_json_response(self, 400, {"error": "content is required"})
return
client = _get_client()
result = client.update(memory_id, content)
result = client.update(
memory_id,
content,
**_namespace_args(body),
**({"task": body["task"]} if body.get("task") else {}),
)
_json_response(self, 200, result)
return
_json_response(self, 404, {"error": "not_found", "path": path})
except AmbiguousMemoryIdError as e:
_json_response(self, 409, {"error": "ambiguous_memory_id", "detail": str(e)})
except ValueError as e:
_json_response(self, 400, {"error": str(e)})
except Exception as e:
logger.error(f"[server] PUT {path} error: {e}", exc_info=True)
_json_response(self, 500, {"error": str(e)})
def do_DELETE(self):
path = self.path.rstrip("/")
path = self.path.split("?")[0].rstrip("/")
try:
m = re.match(r"^/api/v1/memories/([^/]+)$", path)
if m:
memory_id = m.group(1)
client = _get_client()
result = client.delete(memory_id)
query = parse_qs(urlparse(self.path).query)
result = client.delete(memory_id, **_namespace_args(query))
_json_response(self, 200, result)
return
_json_response(self, 404, {"error": "not_found", "path": path})
except AmbiguousMemoryIdError as e:
_json_response(self, 409, {"error": "ambiguous_memory_id", "detail": str(e)})
except ValueError as e:
_json_response(self, 400, {"error": str(e)})
except Exception as e:
logger.error(f"[server] DELETE {path} error: {e}", exc_info=True)
_json_response(self, 500, {"error": str(e)})
@@ -455,7 +492,8 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
if key in body:
kwargs[key] = body[key]
for key in ("limit", "min_score", "schema_min_score", "schema_limit", "reader"):
for key in ("limit", "min_score", "schema_min_score", "schema_limit", "reader",
"rerank"):
if key in body:
kwargs[key] = body[key]
+23 -48
View File
@@ -21,18 +21,21 @@ const TOKEN = "probe-token-not-a-real-credential";
const MEMORY = {
memory_id: "probe-1",
user_id: "local",
kind: "chat",
status: "confirmed",
content: "部署脚本要先跑 verify_merge_plan。",
scopes: ["all-apps"],
evidence_ids: ["probe-e1"],
origin: "extracted",
layer: "l2_fact",
status: "active",
is_latest: true,
supersedes: [],
superseded_by: [],
tags: [],
revision: 2,
created_at: "2026-09-19T00:00:00Z",
updated_at: "2026-09-19T00:00:00Z",
deleted_at: null,
owner: null,
source_raw_ids: ["probe-raw-1"],
user_id: "local",
agent_id: "codex",
session_id: "probe-session",
source: "codex",
memory_at: 1758240000,
gmt_created: 1758240000,
};
const ROUTES = new Map([
@@ -124,26 +127,6 @@ const ROUTES = new Map([
},
],
],
[
"/api/v1/memory-tasks",
[
{
task_id: "probe-task",
request_id: "probe-request",
user_id: "local",
kind: "extract",
input_ref: "probe",
status: "waiting_for_model",
profile_id: "probe-profile",
profile_revision: 1,
profile_snapshot: {},
attempts: 0,
last_error_code: "model_missing",
created_at: "2026-09-19T00:00:00Z",
updated_at: "2026-09-19T00:00:01Z",
},
],
],
[
"/api/v1/model-profiles",
[
@@ -159,28 +142,20 @@ const ROUTES = new Map([
},
],
],
["/api/v1/home-summary", { memory_count: 1, latest: null, observed_at: "T" }],
[
"/api/v1/memory-library",
{ total: 1, offset: 0, limit: 100, items: [MEMORY] },
],
[
"/api/v1/memories/probe-1",
"/api/v2/home-summary",
{
memory: MEMORY,
evidence: [
{
evidence_id: "probe-e1",
available: true,
source_id: "codex",
written_by: "user",
created_at: "2026-09-19T00:00:00Z",
verbatim: "部署前先跑 verify_merge_plan,别靠读 diff 猜冲突。",
},
],
history: [{ memory_id: "probe-1", revision: 2, action: "confirm" }],
schema_version: 2,
memory_count: 1,
latest: MEMORY,
observed_at: "2026-09-19T00:00:01Z",
},
],
[
"/api/v2/memory-library",
{ memories: [MEMORY], total: 1, offset: 0, limit: 100 },
],
["/api/v2/memories/probe-1", MEMORY],
]);
const seen = [];
+304 -38
View File
@@ -4,6 +4,7 @@ import path from "node:path";
import {
claimRuntimeServiceForApp,
readRuntimeServiceOwner,
recordSdkMigrationBlock,
resolveRuntimeCliConfig,
} from "@one-memory/runtime-control";
import { describe, expect, it, vi } from "vitest";
@@ -790,7 +791,7 @@ describe("public One Memory CLI", () => {
}
});
it("upgrade failure restarts the restored artifact with the same data identity", async () => {
it("upgrade restart failure keeps the new version and never restarts the old one", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce({
schema_version: 1,
@@ -832,25 +833,27 @@ describe("public One Memory CLI", () => {
command: "upgrade",
code: "restart failed",
});
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
// The CLI manages the objective aliases the package does not own
// (`memhub` + `memhub-cli`); both are rolled back.
expect(aliasMocks.rollbackManagedCliLink).toHaveBeenCalledTimes(2);
expect(runtimeMocks.restartUserService).toHaveBeenCalledTimes(2);
for (const call of runtimeMocks.restartUserService.mock
.calls as unknown[][]) {
expect(call[0]).toMatchObject({
dataDir: "/tmp/custom data",
installRoot: "/tmp/install/current",
});
}
expect(JSON.parse(streams.stderr())).toMatchObject({
rolled_back: false,
});
// Product rule: the previous Runtime may predate the SDK migration gate,
// so it is never restored after the migration was handed the home.
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(aliasMocks.rollbackManagedCliLink).not.toHaveBeenCalled();
expect(runtimeMocks.restartUserService).toHaveBeenCalledOnce();
expect(
(runtimeMocks.restartUserService.mock.calls as unknown[][])[0]?.[0],
).toMatchObject({
dataDir: "/tmp/custom data",
installRoot: "/tmp/install/current",
});
});
// O-22 (2026-09-29 C8 H17): `upgrade --yes` on a manifest-valid but
// crash-looping version reported exit 0 and left `current` on the bad
// version. A restart that merely returns is not proof the new Runtime came
// up, so the CLI now verifies health and rolls back if it never does.
it("rolls back and fails when the upgraded Runtime never becomes healthy", async () => {
it("fails without rolling back when the upgraded Runtime never becomes healthy", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce({
schema_version: 1,
@@ -890,9 +893,9 @@ describe("public One Memory CLI", () => {
command: "upgrade",
code: "upgrade_health_check_failed",
});
// Rolled back to the previous version and restarted it.
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
expect(runtimeMocks.restartUserService).toHaveBeenCalledTimes(2);
// Kept on the new version; the old Runtime is not restarted.
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(runtimeMocks.restartUserService).toHaveBeenCalledOnce();
});
const upgradeArguments = [
@@ -948,7 +951,7 @@ describe("public One Memory CLI", () => {
);
});
it("rolls back to the previous version when the migration does not finish", async () => {
it("stays on the new version, not serving, when the migration does not finish", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
runtimeMocks.runSdkMigration.mockResolvedValueOnce({
@@ -967,11 +970,87 @@ describe("public One Memory CLI", () => {
streams.restore();
}
expect(code!).toBe(1);
expect(JSON.parse(streams.stderr())).toMatchObject({
const failure = JSON.parse(streams.stderr());
expect(failure).toMatchObject({
command: "upgrade",
code: "sdk_migration_disk_space_insufficient",
// The one file support needs, and where to send it.
log_file: "/tmp/custom data/logs/sdk-migration.log",
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
});
expect(failure.message).toContain(
"/tmp/custom data/logs/sdk-migration.log",
);
expect(failure.message).toContain("GitHub");
expect(failure.message).not.toContain("回滚到上一版本");
expect(runtimeMocks.runSdkMigration).toHaveBeenCalledWith(
expect.objectContaining({ caller: "cli-upgrade" }),
);
// No rollback: the old Runtime has no migration gate. The new service is
// (re)started so its gate, not the old Runtime, holds the data dir.
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(aliasMocks.rollbackManagedCliLink).not.toHaveBeenCalled();
expect(runtimeMocks.restartUserService).toHaveBeenCalledOnce();
expect(
(runtimeMocks.restartUserService.mock.calls as unknown[][])[0]?.[0],
).toMatchObject({
installRoot: "/tmp/install/current",
});
});
it("stays on the new version when the migration itself throws", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
runtimeMocks.runSdkMigration.mockRejectedValueOnce(
new Error("sdk_migration_fence_failed"),
);
const streams = captureStreams();
let code: number;
try {
code = await run(
upgradeArguments,
{},
{ waitForRuntimeReady: async () => true },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
expect(JSON.parse(streams.stderr())).toMatchObject({
command: "upgrade",
code: "sdk_migration_fence_failed",
log_file: "/tmp/custom data/logs/sdk-migration.log",
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
});
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
});
it("still rolls back a failure before the migration touched the home", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
aliasMocks.installManagedCliLink.mockRejectedValueOnce(
new Error("cli_link_failed"),
);
const streams = captureStreams();
let code: number;
try {
code = await run(
upgradeArguments,
{},
{ waitForRuntimeReady: async () => true },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
expect(JSON.parse(streams.stderr())).toMatchObject({
command: "upgrade",
code: "cli_link_failed",
});
expect(runtimeMocks.runSdkMigration).not.toHaveBeenCalled();
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
// The previous Runtime was never stopped; nothing is (re)started.
expect(runtimeMocks.restartUserService).not.toHaveBeenCalled();
});
it("keeps the new version when a migrated home is slow to come up", async () => {
@@ -1492,7 +1571,7 @@ describe("public One Memory CLI", () => {
expect(JSON.parse(output.value()).apply).toBe(true);
});
it("rolls back the distribution when service installation fails", async () => {
it("keeps the distribution when service installation fails after the migration ran", async () => {
runtimeMocks.applyLocalInstall.mockResolvedValueOnce({
schema_version: 1,
source: "/tmp/distribution",
@@ -1533,13 +1612,14 @@ describe("public One Memory CLI", () => {
command: "install",
status: "error",
code: "service failed",
rolled_back: false,
});
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
expect(runtimeMocks.rollbackCliLink).toHaveBeenCalledOnce();
expect(aliasMocks.rollbackManagedCliLink).toHaveBeenCalledTimes(2);
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(runtimeMocks.rollbackCliLink).not.toHaveBeenCalled();
expect(aliasMocks.rollbackManagedCliLink).not.toHaveBeenCalled();
});
it("upgrades only an existing managed CLI and rolls back a failed restart", async () => {
it("upgrades only an existing managed CLI and keeps it after a failed restart", async () => {
runtimeMocks.planCliLink.mockResolvedValueOnce({
schema_version: 1,
platform: "linux",
@@ -1579,7 +1659,7 @@ describe("public One Memory CLI", () => {
status: "error",
code: "restart failed",
});
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(runtimeMocks.installCliLink).not.toHaveBeenCalled();
});
@@ -2003,6 +2083,101 @@ describe("lifecycle commands accept --json", () => {
});
});
// A Runtime service that refuses to serve an unmigrated home must be reported
// as that, with the migration log and where to send it, not as a generic
// "not ready within budget".
describe("lifecycle commands report a migration block", () => {
async function blockedHome() {
const dataDir = await mkdtemp(path.join(tmpdir(), "memhub-blocked-"));
await recordSdkMigrationBlock(
dataDir,
"sdk_migration_disk_space_insufficient",
"runtime-service",
);
return dataDir;
}
// darwin: kickstart "restarts" but never becomes ready. Elsewhere: no
// service, so the direct spawn fails. Both must surface the block.
const blockedControls = () => ({
restartManagedRuntime: vi.fn(async () => "restarted" as const),
waitForRuntimeReady: vi.fn(async () => false),
stopRuntime: vi.fn(async () => "stopped" as const),
ensureRuntime: vi.fn(async () => {
throw new Error("Timed out waiting for MemHub Runtime");
}),
});
it("restart --json prints the block code, log file and issues URL", async () => {
const dataDir = await blockedHome();
const output = captureStdout();
let code: number;
try {
code = await run(
["restart", "--json"],
{ ONE_MEMORY_DATA_DIR: dataDir },
blockedControls(),
);
} finally {
output.restore();
await rm(dataDir, { recursive: true, force: true });
}
expect(code!).toBe(1);
const lines = output.value().trim().split("\n");
const document = JSON.parse(lines[lines.length - 1]!);
expect(document).toMatchObject({
command: "restart",
state: "blocked",
code: "sdk_migration_disk_space_insufficient",
log_file: path.join(dataDir, "logs", "sdk-migration.log"),
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
});
expect(output.value()).not.toContain("runtime_not_ready_within_budget");
});
it("restart in text mode names the log file and GitHub issues", async () => {
const dataDir = await blockedHome();
const output = captureStdout();
try {
expect(
await run(
["restart"],
{ ONE_MEMORY_DATA_DIR: dataDir },
blockedControls(),
),
).toBe(1);
} finally {
output.restore();
await rm(dataDir, { recursive: true, force: true });
}
expect(output.value()).toContain("sdk_migration_disk_space_insufficient");
expect(output.value()).toContain(
path.join(dataDir, "logs", "sdk-migration.log"),
);
expect(output.value()).toContain(
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
);
});
it("ui --json reports the block instead of runtime_not_running", async () => {
const dataDir = await blockedHome();
const output = captureStdout();
let code: number;
try {
code = await run(["ui", "--json"], { ONE_MEMORY_DATA_DIR: dataDir });
} finally {
output.restore();
await rm(dataDir, { recursive: true, force: true });
}
expect(code!).not.toBe(0);
expect(JSON.parse(output.value())).toMatchObject({
command: "ui",
state: "blocked",
code: "sdk_migration_disk_space_insufficient",
log_file: path.join(dataDir, "logs", "sdk-migration.log"),
});
});
});
// The objective names `memhub host install <host>` (host as a positional). Both
// that and the `--host <id>` flag form must reach the same command; naming both
// at once is ambiguous and refused rather than guessed.
@@ -2112,7 +2287,7 @@ describe("install gates on Runtime readiness", () => {
});
}
it("fails and rolls back when the installed Runtime never becomes ready", async () => {
it("fails without rolling back when the installed Runtime never becomes ready", async () => {
vi.clearAllMocks();
stubInstallPlan();
const streams = captureStreams();
@@ -2141,10 +2316,10 @@ describe("install gates on Runtime readiness", () => {
}
expect(code!).toBe(1);
expect(streams.stderr()).toContain("install_runtime_not_ready");
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(
lifecycleMocks.rollbackPreparedServiceInstall,
).toHaveBeenCalledOnce();
).not.toHaveBeenCalled();
});
it("takes CLI ownership after a successful install over an App-owned Runtime", async () => {
@@ -2189,7 +2364,7 @@ describe("install gates on Runtime readiness", () => {
}
});
it("restores App ownership before relaunching the old Runtime after install failure", async () => {
it("never hands the home back to the App Runtime after the migration ran", async () => {
vi.clearAllMocks();
stubInstallPlan();
const root = await mkdtemp(path.join(tmpdir(), "memhub-cli-owner-"));
@@ -2199,14 +2374,6 @@ describe("install gates on Runtime readiness", () => {
appBundle: "/Applications/MemHub.app",
appExecutable: "/Applications/MemHub.app/Contents/MacOS/MemHub",
});
lifecycleMocks.rollbackPreparedServiceInstall.mockImplementationOnce(
async (_prepared: unknown, rollback: () => Promise<void>) => {
await rollback();
await expect(readRuntimeServiceOwner(dataDir)).resolves.toMatchObject({
owner: "app",
});
},
);
const streams = captureStreams();
try {
expect(
@@ -2229,15 +2396,114 @@ describe("install gates on Runtime readiness", () => {
{ waitForRuntimeReady: async () => false },
),
).toBe(1);
// The App Runtime may predate the SDK migration gate: CLI keeps it.
await expect(readRuntimeServiceOwner(dataDir)).resolves.toMatchObject({
owner: "app",
owner: "cli",
});
expect(
lifecycleMocks.rollbackPreparedServiceInstall,
).not.toHaveBeenCalled();
} finally {
streams.restore();
await rm(root, { recursive: true, force: true });
}
});
it("keeps the install, not serving, when its migration is deferred", async () => {
vi.clearAllMocks();
stubInstallPlan();
runtimeMocks.runSdkMigration.mockResolvedValueOnce({
state: "deferred",
code: "sdk_migration_memory_unreadable",
});
const waitForRuntimeReady = vi.fn(async () => true);
const streams = captureStreams();
let code: number;
try {
code = await run(
[
"install",
"--source",
"/tmp/dist",
"--home",
"/tmp/home",
"--install-root",
"/tmp/install",
"--bin-dir",
"/tmp/bin",
"--data-dir",
"/tmp/data",
"--yes",
],
{},
{ waitForRuntimeReady },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
const failure = JSON.parse(streams.stderr());
expect(failure).toMatchObject({
command: "install",
code: "sdk_migration_memory_unreadable",
log_file: "/tmp/data/logs/sdk-migration.log",
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
});
expect(failure.message).toContain("/tmp/data/logs/sdk-migration.log");
expect(runtimeMocks.runSdkMigration).toHaveBeenCalledWith(
expect.objectContaining({ caller: "cli-install" }),
);
// The new (gated) service owns the data dir; the previous registration is
// never re-bootstrapped and nothing is rolled back.
expect(lifecycleMocks.applyPreparedServiceInstall).toHaveBeenCalledWith(
expect.objectContaining({ restorePreviousOnFailure: false }),
);
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(
lifecycleMocks.rollbackPreparedServiceInstall,
).not.toHaveBeenCalled();
expect(waitForRuntimeReady).not.toHaveBeenCalled();
});
it("still rolls back an install that failed before its migration ran", async () => {
vi.clearAllMocks();
stubInstallPlan();
runtimeMocks.installCliLink.mockRejectedValueOnce(
new Error("cli_link_failed"),
);
const streams = captureStreams();
let code: number;
try {
code = await run(
[
"install",
"--source",
"/tmp/dist",
"--home",
"/tmp/home",
"--install-root",
"/tmp/install",
"--bin-dir",
"/tmp/bin",
"--data-dir",
"/tmp/data",
"--yes",
],
{},
{ waitForRuntimeReady: async () => true },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
expect(JSON.parse(streams.stderr())).toMatchObject({
command: "install",
code: "cli_link_failed",
});
expect(runtimeMocks.runSdkMigration).not.toHaveBeenCalled();
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
});
it("succeeds when the installed Runtime becomes ready", async () => {
vi.clearAllMocks();
stubInstallPlan();
+281 -266
View File
@@ -32,6 +32,7 @@ import {
processIsAlive,
type ResolvedRuntimeRelease,
RuntimeDiscoveryError,
readSdkMigrationBlock,
removeRuntimeServiceOwner,
resolveRuntimeCliConfig,
resolveRuntimeRelease,
@@ -42,6 +43,7 @@ import {
runSdkMigration,
runtimeInstallLayout,
type SdkMigrationResult,
sdkMigrationLogPath,
stopRuntimeProcess,
} from "@one-memory/runtime-control";
import axios from "axios";
@@ -177,21 +179,10 @@ import {
removeManagedCliLink,
rollbackManagedCliLink,
} from "./managed-cli-link.js";
import {
executeMemoryMutation,
inspectMemoryPlan,
type MemoryFlowDocument,
memoryFlowExitCode,
memoryPlanAcceptsContent,
parseMemoryContent,
planExistingMemoryMutation,
planMemoryAdd,
readMemoryMutationReceipt,
renderMemoryFlowText,
} from "./memory-flows.js";
import {
isMemoryId,
type MemoryQuery,
parseMemoryContent,
readMemoryDetail,
readMemoryPage,
renderMemoryDetailText,
@@ -1197,6 +1188,28 @@ async function runCommand(
: state === "already_running"
? "MemHub Runtime 已在运行。\n"
: "Runtime 服务已注册但未就绪;请运行 memhub logs 查看原因。\n";
// A Runtime that did not come up because its service refuses to serve an
// unmigrated home reports that cause, the migration log and where to send
// it, instead of a generic budget timeout.
const migrationBlocked = async (): Promise<string | null> => {
const blocked = await runtimeMigrationBlock(config.dataDir, command);
if (blocked === null) return null;
return asJson
? `${JSON.stringify({
schema_version: 1,
product: PRODUCT,
command,
state: "blocked",
code: blocked.code,
message: blocked.message,
log_file: blocked.log_file,
issues_url: blocked.issues_url,
})}\n`
: `${blocked.message}\n`;
};
const notReady = async () =>
(await migrationBlocked()) ??
lifecycle("not-ready", "runtime_not_ready_within_budget");
// A launchd-managed Runtime must be driven through launchctl. Spawning a
// second process while the service holds the data directory is exactly what
// produced "Runtime exited before becoming ready".
@@ -1241,16 +1254,12 @@ async function runCommand(
process.stdout.write(lifecycle("restarted", "runtime_verified"));
return EXIT_OK;
}
process.stdout.write(
lifecycle("not-ready", "runtime_not_ready_within_budget"),
);
process.stdout.write(await notReady());
await restoreEnableState();
return EXIT_FAILED;
}
if (restarted === "not-ready") {
process.stdout.write(
lifecycle("not-ready", "runtime_not_ready_within_budget"),
);
process.stdout.write(await notReady());
await restoreEnableState();
return EXIT_FAILED;
}
@@ -1306,16 +1315,12 @@ async function runCommand(
process.stdout.write(lifecycle("started", "runtime_verified"));
return EXIT_OK;
}
process.stdout.write(
lifecycle("not-ready", "runtime_not_ready_within_budget"),
);
process.stdout.write(await notReady());
await restoreEnableState();
return EXIT_FAILED;
}
if (startedManaged === "not-ready") {
process.stdout.write(
lifecycle("not-ready", "runtime_not_ready_within_budget"),
);
process.stdout.write(await notReady());
await restoreEnableState();
return EXIT_FAILED;
}
@@ -1344,6 +1349,11 @@ async function runCommand(
// A spawned Runtime that exits before ready is a failure of this command,
// not a crash: report it as a structured document (JSON) or a plain
// message, with a non-zero exit, instead of letting it escape.
const blocked = await migrationBlocked();
if (blocked !== null) {
process.stdout.write(blocked);
return EXIT_FAILED;
}
const message = error instanceof Error ? error.message : String(error);
process.stdout.write(
asJson
@@ -1540,6 +1550,24 @@ async function runCommand(
return uiExitCode(result);
};
if (attachment === null) {
const migration = await runtimeMigrationBlock(config.dataDir, "ui");
if (migration !== null) {
emit(
asJson
? `${JSON.stringify({
schema_version: 1,
product: PRODUCT,
command: "ui",
state: "blocked",
code: migration.code,
message: migration.message,
log_file: migration.log_file,
issues_url: migration.issues_url,
})}\n`
: `ui 未打开:${migration.code}\n${migration.message}\n`,
);
return uiExitCode({ state: "blocked", code: migration.code });
}
return emitUi(
{ state: "blocked", code: "runtime_not_running" },
`先运行 ${PRODUCT} start 启动 Runtime。`,
@@ -1644,7 +1672,7 @@ async function runCommand(
});
if (runtime.state !== "running") {
const document: ContextSearchDocument = {
schema_version: 1,
schema_version: 2,
product: PRODUCT,
state: "unavailable",
code: runtime.code,
@@ -1668,7 +1696,7 @@ async function runCommand(
timeout_ms: parsed.timeoutMs,
})
: ({
schema_version: 1,
schema_version: 2,
product: PRODUCT,
state: "unavailable",
code: attached.failure ?? "runtime_attach_failed",
@@ -2434,20 +2462,20 @@ async function runCommand(
};
}
if (blocked || !reader) {
if (
parsed.kind === "update" ||
parsed.kind === "delete" ||
parsed.kind === "plan-add" ||
parsed.kind === "plan-existing" ||
parsed.kind === "apply"
) {
const document: MemoryFlowDocument = {
schema_version: 1,
if (parsed.kind === "update" || parsed.kind === "delete") {
const document = {
schema_version: 2,
product: PRODUCT,
state: "unknown",
operation: parsed.kind,
memory_id: parsed.memoryId,
code: blocked?.code ?? "runtime_attach_failed",
};
emitMemoryFlow(document, asJson);
emit(
asJson
? `${JSON.stringify(document)}\n`
: `记忆操作未完成:${document.state}(${document.code})\n`,
);
return runtime.state === "stopped" ? 3 : 1;
}
const document =
@@ -2541,73 +2569,7 @@ async function runCommand(
);
return outcome.state === "committed" ? EXIT_OK : EXIT_FAILED;
}
if (parsed.kind === "plan-add") {
const content = await readMemoryContentFile(parsed.contentFile);
if (content === null) return usage();
const document = planMemoryAdd({
content,
memoryKind: parsed.memoryKind,
scopes: parsed.scopes,
evidenceIds: parsed.evidenceIds,
});
emitMemoryFlow(document, asJson);
return memoryFlowExitCode(document);
}
if (parsed.kind === "plan-existing") {
let content: string | undefined;
if (parsed.contentFile) {
const read = await readMemoryContentFile(parsed.contentFile);
if (read === null) return usage();
content = read;
}
const document = await planExistingMemoryMutation(reader, {
operation: parsed.operation,
memoryId: parsed.memoryId,
...(content === undefined ? {} : { content }),
});
emitMemoryFlow(document, asJson);
return memoryFlowExitCode(document);
}
if (!operations) {
const document: MemoryFlowDocument = {
schema_version: 1,
product: PRODUCT,
state: "unknown",
code: "runtime_attach_failed",
};
emitMemoryFlow(document, asJson);
return EXIT_FAILED;
}
const file = await readBoundedJson(parsed.planFile);
if (file === null) return usage();
const plan = unwrapPlan(file);
const identity = inspectMemoryPlan(plan);
if (!identity) return usage();
let content: string | undefined;
if (parsed.contentFile) {
const read = await readMemoryContentFile(parsed.contentFile);
if (read === null) return usage();
content = read;
}
if (
(identity.needsContent && content === undefined) ||
(!identity.needsContent && content !== undefined) ||
!memoryPlanAcceptsContent(plan, content)
) {
return usage();
}
const attempt = await claimPlanAttempt(parsed.planFile, {
schema_version: 1,
operation_id: identity.operationId,
plan_digest: identity.digest,
});
if (attempt === "invalid") return usage();
const document =
attempt === "prior"
? await readMemoryMutationReceipt(reader, plan)
: await executeMemoryMutation(plan, content, operations, reader);
emitMemoryFlow(document, asJson);
return memoryFlowExitCode(document);
return usage();
}
if (named === "calls" || named === "import-file") {
const parsed = parseUiWriteCommand(named, arguments_.slice(1));
@@ -2817,10 +2779,15 @@ async function runCommand(
if (!ready) throw new Error("runtime_service_activation_unverified");
return plan;
})
.catch((error: unknown) => {
.catch(async (error: unknown) => {
const migration =
error instanceof Error &&
error.message === "runtime_service_activation_unverified"
? await runtimeMigrationBlock(layout.dataDir, "service-install")
: null;
process.stderr.write(
`${JSON.stringify(
serviceErrorDocument("service-install", error),
migration ?? serviceErrorDocument("service-install", error),
null,
2,
)}\n`,
@@ -3155,6 +3122,9 @@ async function runCommand(
let installReceiptWritten = false;
let serviceApplied = false;
let migrated = false;
// `null` until the migration is handed this home; "failed" if it threw.
let migrationState: SdkMigrationResult["state"] | "failed" | null = null;
let deferredCode: string | null = null;
let previousOwner:
| Awaited<ReturnType<typeof claimCliServiceOwnership>>
| undefined;
@@ -3167,6 +3137,7 @@ async function runCommand(
previousOwner = await claimCliServiceOwnership(layout.dataDir);
// `install` over an existing home (reinstall, App-to-CLI takeover) must
// migrate it too, before the new Runtime starts serving it.
migrationState = "failed";
const migration = await (
controls.migrateSdkData ?? migrateSdkDataBeforeStart
)({
@@ -3174,15 +3145,24 @@ async function runCommand(
dataDir: layout.dataDir,
runtimeBuild: String(plan.build_number ?? plan.version),
asJson: true,
caller: "cli-install",
});
migrationState = migration.state;
// A deferred migration keeps this install: the new Runtime service is
// registered and its migration gate keeps Core down until the home is
// migrated. Rolling back would hand the home to the previous (App or CLI)
// Runtime, which has no gate and would serve the legacy data.
if (migration.state === "deferred") {
throw new SdkMigrationDeferred(
migration.code ?? "sdk_migration_incomplete",
);
deferredCode = migration.code ?? "sdk_migration_incomplete";
}
migrated =
migration.state === "cutover" || migration.state === "complete";
await applyPreparedServiceInstall(preparedService);
// From here the previous LaunchAgent is never re-bootstrapped: it may
// be an older Runtime with no migration gate.
await applyPreparedServiceInstall({
...preparedService,
restorePreviousOnFailure: false,
});
serviceApplied = true;
// The receipt exists so a later command can find the data dir without
// flags. An isolated install named its data dir explicitly, and writing
@@ -3198,6 +3178,7 @@ async function runCommand(
});
installReceiptWritten = true;
}
if (deferredCode !== null) throw new SdkMigrationDeferred(deferredCode);
// Registering the service is not the same as a working install: on a host
// where Core cold-start stalls, launchd reports a pid while status is
// still `runtime_service_running`. Gate on real readiness (same predicate
@@ -3233,10 +3214,38 @@ async function runCommand(
);
}
};
if (migrated) {
// An older Runtime refuses a migrated home: keep this install.
// Once the migration has been handed this home nothing rolls back: the
// previous version (App or CLI) may be an older Runtime without the
// migration gate, and restoring it could serve a home that is not on SDK
// authority. This install stays; its service keeps Core down until the
// home is migrated. Only a failure before the migration ran (the
// previous Runtime was never stopped and the service never touched) is
// rolled back below.
if (migrationState !== null) {
if (!serviceApplied && !migrated && deferredCode === null) {
// The migration threw. Hand the data dir to this version's gated
// service anyway so that the previous Runtime does not come back.
await applyPreparedServiceInstall({
...preparedService,
restorePreviousOnFailure: false,
}).catch(() => undefined);
}
process.stderr.write(
`${JSON.stringify(notReadyAfterMigration("install"), null, 2)}\n`,
`${JSON.stringify(
deferredCode !== null
? sdkMigrationFailure("install", deferredCode, layout.dataDir)
: migrated
? notReadyAfterMigration("install")
: migrationState === "failed"
? sdkMigrationFailure(
"install",
sdkMigrationErrorCode(error),
layout.dataDir,
)
: notReadyAfterUpgrade("install", error),
null,
2,
)}\n`,
);
return EXIT_FAILED;
}
@@ -3249,13 +3258,7 @@ async function runCommand(
await rollbackFilesAndOwner();
}
process.stderr.write(
`${JSON.stringify(
error instanceof SdkMigrationDeferred
? sdkMigrationFailure("install", error.code)
: serviceErrorDocument("install", error),
null,
2,
)}\n`,
`${JSON.stringify(serviceErrorDocument("install", error), null, 2)}\n`,
);
return EXIT_FAILED;
}
@@ -3618,9 +3621,17 @@ async function runCommand(
| Awaited<ReturnType<typeof claimCliServiceOwnership>>
| undefined;
let migrated = false;
const rollbackUpgrade = async (
dataDir: string | undefined,
): Promise<void> => {
// `null` until the migration is handed this home; "failed" if it threw.
let migrationState: SdkMigrationResult["state"] | "failed" | null = null;
/**
* Restore the previous version's files and ownership. Only used for a
* failure before the migration ran: the previous Runtime was never stopped
* and the service never restarted, so nothing new starts serving. After
* the migration has been handed the home the upgrade never rolls back:
* the previous Runtime may predate the migration gate and would serve a
* home that is not on SDK authority.
*/
const rollbackUpgrade = async (): Promise<void> => {
for (const receipt of memhubReceipts.reverse()) {
await rollbackManagedCliLink(receipt);
}
@@ -3630,25 +3641,29 @@ async function runCommand(
() => undefined,
);
}
await restartUserService(
runtimeInstallLayout({
platform: plan.platform,
home: parsed.home,
installRoot: canonicalManagedServiceRoot(
plan.platform,
plan.current_link,
),
dataDir,
...hourlyIsolationFromEnvironment(environment),
}),
serviceOptions,
).catch(() => undefined);
};
/**
* After a migration that did not finish, make sure the process holding the
* data dir is the new version's service (gated: it keeps Core down while
* the migration is deferred) and not the previous Runtime. The migration
* fence already stopped it unless the status probe itself failed.
*/
const keepUpgradedRuntimeOffLegacyHome = async (): Promise<void> => {
try {
await restartUserService(layout, serviceOptions);
} catch {
await (controls.stopRuntime ?? stopRuntimeProcess)({
dataDir: parsed.dataDir ?? layout.dataDir,
probeIdentity,
}).catch(() => undefined);
}
};
try {
for (const link of companionLinks) {
memhubReceipts.push(await installManagedCliLink(link));
}
previousOwner = await claimCliServiceOwnership(layout.dataDir);
migrationState = "failed";
const migration = await (
controls.migrateSdkData ?? migrateSdkDataBeforeStart
)({
@@ -3656,16 +3671,22 @@ async function runCommand(
dataDir: parsed.dataDir ?? layout.dataDir,
runtimeBuild: String(plan.build_number ?? plan.version),
asJson,
caller: "cli-upgrade",
});
migrationState = migration.state;
if (migration.state === "deferred") {
await rollbackUpgrade(parsed.dataDir);
// Stay on the new version. Its Runtime service refuses to spawn Core
// while the migration is deferred, so nothing serves the legacy home;
// rolling back would restart the old Runtime, which has no such gate.
await keepUpgradedRuntimeOffLegacyHome();
process.stderr.write(
`${JSON.stringify(sdkMigrationFailure("upgrade", migration.code ?? "sdk_migration_incomplete"), null, 2)}\n`,
`${JSON.stringify(sdkMigrationFailure("upgrade", migration.code ?? "sdk_migration_incomplete", parsed.dataDir ?? layout.dataDir), null, 2)}\n`,
);
return EXIT_FAILED;
}
// An older Runtime refuses a migrated home, so from here a failure
// keeps the new version instead of rolling back to one that cannot start.
// From here a failure keeps the new version: an older Runtime either
// refuses a migrated home or, predating the migration gate, would serve
// one that is not on SDK authority.
migrated =
migration.state === "cutover" || migration.state === "complete";
await restartUserService(layout, serviceOptions);
@@ -3673,8 +3694,8 @@ async function runCommand(
// accepted the command. O-22: the first `upgrade` implementation
// reported success even when the new artifact crash-looped, leaving
// `current` on a version that never came up. Verify the restarted
// Runtime actually becomes healthy; if it does not, roll back to the
// previously working version.
// Runtime actually becomes healthy; if it does not, fail loudly. It is
// not rolled back (product rule: SDK authority or not serving at all).
const dataDir = parsed.dataDir ?? layout.dataDir;
const ready = await (controls.waitForRuntimeReady ?? waitForRuntimeReady)(
dataDir,
@@ -3686,20 +3707,11 @@ async function runCommand(
return EXIT_FAILED;
}
if (!ready) {
await rollbackUpgrade(parsed.dataDir);
// Not rolled back: see `rollbackUpgrade`. If the service is blocked on
// the migration (it re-runs it before spawning Core), say so.
const blocked = await runtimeMigrationBlock(dataDir, "upgrade");
process.stderr.write(
`${JSON.stringify(
{
schema_version: 1,
product: PRODUCT,
command: "upgrade",
status: "error",
code: "upgrade_health_check_failed",
message: "新版本重启后未在限定时间内就绪,已回滚到上一版本。",
},
null,
2,
)}\n`,
`${JSON.stringify(blocked ?? notReadyAfterUpgrade("upgrade"), null, 2)}\n`,
);
return EXIT_FAILED;
}
@@ -3712,9 +3724,32 @@ async function runCommand(
});
}
} catch (error) {
if (!migrated) await rollbackUpgrade(parsed.dataDir);
if (migrationState === null) {
await rollbackUpgrade();
process.stderr.write(
`${JSON.stringify(serviceErrorDocument("upgrade", error), null, 2)}\n`,
);
return EXIT_FAILED;
}
if (migrated) {
// An older Runtime refuses a migrated home: keep the new version.
process.stderr.write(
`${JSON.stringify(notReadyAfterMigration("upgrade"), null, 2)}\n`,
);
return EXIT_FAILED;
}
if (migrationState === "failed") {
// The migration itself threw: its outcome is unknown, so treat the
// home as not migrated and keep the old Runtime off it.
await keepUpgradedRuntimeOffLegacyHome();
process.stderr.write(
`${JSON.stringify(sdkMigrationFailure("upgrade", sdkMigrationErrorCode(error), parsed.dataDir ?? layout.dataDir), null, 2)}\n`,
);
return EXIT_FAILED;
}
// `not_required`: the restart itself failed. Stay on the new version.
process.stderr.write(
`${JSON.stringify(migrated ? notReadyAfterMigration("upgrade") : serviceErrorDocument("upgrade", error), null, 2)}\n`,
`${JSON.stringify(notReadyAfterUpgrade("upgrade", error), null, 2)}\n`,
);
return EXIT_FAILED;
}
@@ -4100,11 +4135,13 @@ async function migrateSdkDataBeforeStart(options: {
dataDir: string;
runtimeBuild: string;
asJson: boolean;
caller?: "cli-install" | "cli-upgrade";
}): Promise<SdkMigrationResult> {
return runSdkMigration({
dataDir: options.dataDir,
core: { command: options.coreCommand },
runtimeBuild: options.runtimeBuild,
...(options.caller === undefined ? {} : { caller: options.caller }),
onProgress: (step) => {
process.stderr.write(
options.asJson
@@ -4133,33 +4170,105 @@ function notReadyAfterMigration(command: "install" | "upgrade") {
};
}
/**
* The new version did not come up and was not rolled back: the previous one
* may predate the SDK migration gate. `error` carries the underlying code
* (a failed service restart), never free text from a child process.
*/
function notReadyAfterUpgrade(command: "install" | "upgrade", error?: unknown) {
return {
schema_version: 1,
product: PRODUCT,
command,
status: "error",
code:
error !== undefined
? serviceErrorDocument(command, error).code
: command === "upgrade"
? "upgrade_health_check_failed"
: "install_runtime_not_ready",
rolled_back: false,
message:
"新版本 Runtime 没有启动或没有在限定时间内就绪。为避免旧版本打开记忆数据,已保留新版本,没有回滚。请运行 memhub restart 重试,或运行 memhub doctor 查看原因;如果仍然失败,请在 GitHub 提交反馈:" +
MIGRATION_ISSUES_URL,
issues_url: MIGRATION_ISSUES_URL,
};
}
const SDK_MIGRATION_FAILURES: Record<string, string> = {
sdk_migration_disk_space_insufficient:
"磁盘空间不足,记忆数据迁移没有完成,已回滚到上一版本。请腾出与记忆数据同等大小的空间后重试;原有记忆数据不会丢失。",
"磁盘空间不足,记忆数据迁移没有完成。请腾出与记忆数据同等大小的空间后运行 memhub restart 重试。",
sdk_migration_memory_unreadable:
"核对发现新版本还读不到你的全部记忆,没有切换数据源,已回滚到上一版本。请把这个错误码反馈给我们;原有记忆数据不会丢失。",
"核对发现新版本还读不到你的全部记忆,没有切换数据源。",
sdk_migration_model_profile_required:
"迁移需要先有可用的模型配置,用来核对新版本能否读回记忆。已回滚到上一版本;请先用 memhub models setup 完成配置后再升级。",
"迁移需要先有可用的模型配置,用来核对新版本能否读回记忆。请先用 memhub models setup 完成配置,再运行 memhub restart 重试。",
};
function sdkMigrationFailure(command: "install" | "upgrade", code: string) {
const MIGRATION_ISSUES_URL =
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new";
/**
* Why MemHub is not serving: the home has not reached SDK authority.
*
* Product rule: after an upgrade the home is on SDK authority or MemHub does
* not serve it at all. Nothing here rolls back to an older Runtime (it has no
* migration gate and would serve the legacy home); the user gets the code, the
* migration log and where to report it.
*/
function sdkMigrationFailure(
command: string,
code: string,
dataDir: string,
context: "upgrade" | "blocked" = "upgrade",
) {
const logFile = sdkMigrationLogPath(dataDir);
const reason =
SDK_MIGRATION_FAILURES[code] ??
"记忆数据迁移没有完成。可以运行 memhub restart 重试。";
const lead =
context === "upgrade"
? `${reason}已保留新版本;在迁移完成前 MemHub 不会启动,也不会用旧版本打开未迁移的数据。`
: `MemHub 没有启动:记忆数据还没有迁移到新版本(${code})。${reason}`;
return {
schema_version: 1,
product: PRODUCT,
command,
status: "error",
code,
message:
SDK_MIGRATION_FAILURES[code] ??
"记忆数据迁移没有完成,已回滚到上一版本。请重试;如果仍然失败,请把这个错误码反馈给我们。原有记忆数据不会丢失。",
message: `${lead}原有记忆数据不会丢失。迁移日志:${logFile}。请在 GitHub 提交反馈并附上这个日志文件:${MIGRATION_ISSUES_URL}`,
log_file: logFile,
issues_url: MIGRATION_ISSUES_URL,
};
}
/** A stable code from a thrown migration error, never its free text. */
function sdkMigrationErrorCode(error: unknown): string {
const message = error instanceof Error ? error.message : "";
return /^[a-z][a-z0-9_]{0,95}$/.test(message)
? message
: "sdk_migration_incomplete";
}
/**
* The Runtime service refuses to spawn Core while this home's migration is
* deferred and leaves a block note; report that instead of a generic
* "not ready within budget".
*/
async function runtimeMigrationBlock(
dataDir: string,
command: string,
): Promise<ReturnType<typeof sdkMigrationFailure> | null> {
const block = await readSdkMigrationBlock(dataDir).catch(() => null);
if (block === null) return null;
return sdkMigrationFailure(command, block.code, dataDir, "blocked");
}
async function waitForRuntimeReady(
dataDir: string,
timeoutMs = 45_000,
): Promise<boolean> {
const deadline = Date.now() + timeoutMs;
const started = Date.now();
const deadline = started + timeoutMs;
while (Date.now() < deadline) {
const runtime = await inspectRuntimeStatus({
dataDir,
@@ -4167,6 +4276,11 @@ async function waitForRuntimeReady(
requireServiceReceipt: true,
}).catch(() => null);
if (runtimeIsReady(runtime)) return true;
// The service deferred its migration during this wait and will not spawn
// Core: stop waiting. An older note may predate a retry still running, so
// only a note written after the wait began ends it early.
const blocked = await readSdkMigrationBlock(dataDir).catch(() => null);
if (blocked !== null && Date.parse(blocked.at) >= started) return false;
await new Promise((resolve) => setTimeout(resolve, 500));
}
return false;
@@ -6086,21 +6200,7 @@ type MemoryCommand =
contentFile: string;
confirmed: boolean;
}
| { kind: "delete"; memoryId: string; confirmed: boolean }
| {
kind: "plan-add";
contentFile: string;
memoryKind: string;
scopes: string[];
evidenceIds: string[];
}
| {
kind: "plan-existing";
operation: "correct" | "delete";
memoryId: string;
contentFile?: string;
}
| { kind: "apply"; planFile: string; contentFile?: string };
| { kind: "delete"; memoryId: string; confirmed: boolean };
async function parseMemoryCommand(
rest: readonly string[],
@@ -6140,31 +6240,6 @@ async function parseMemoryCommand(
? { kind: "delete", memoryId, confirmed }
: null;
}
if (action === "plan" || action === "apply") return null;
if (action === "plan") return parseMemoryPlanCommand(tokens.slice(1));
if (action === "apply") {
let planFile: string | undefined;
let contentFile: string | undefined;
let confirmed = false;
for (let index = 1; index < tokens.length; index += 1) {
const flag = tokens[index];
if (flag === "--confirm") {
if (confirmed) return null;
confirmed = true;
continue;
}
const value = tokens[index + 1];
if (value === undefined || value.startsWith("--")) return null;
if (flag === "--plan-file" && planFile === undefined) planFile = value;
else if (flag === "--content-file" && contentFile === undefined)
contentFile = value;
else return null;
index += 1;
}
return planFile && confirmed
? { kind: "apply", planFile, ...(contentFile ? { contentFile } : {}) }
: null;
}
if (action !== "list") return null;
let status: string | undefined;
let offset = 0;
@@ -6191,73 +6266,11 @@ async function parseMemoryCommand(
return { kind: "list", query: { status, offset, limit } };
}
function parseMemoryPlanCommand(
tokens: readonly string[],
): MemoryCommand | null {
const operation = tokens[0];
const add = operation === "add";
if (!add && !["correct", "delete"].includes(operation)) {
return null;
}
const memoryId = add ? undefined : tokens[1];
if (!add && (!memoryId || !isMemoryId(memoryId))) return null;
let contentFile: string | undefined;
let memoryKind = "chat";
let kindSet = false;
const scopes: string[] = [];
const evidenceIds: string[] = [];
for (let index = add ? 1 : 2; index < tokens.length; index += 1) {
const flag = tokens[index];
const value = tokens[index + 1];
if (value === undefined || value.startsWith("--")) return null;
if (flag === "--content-file" && contentFile === undefined)
contentFile = value;
else if (flag === "--kind" && add && !kindSet) {
memoryKind = value;
kindSet = true;
} else if (flag === "--scope" && add && !scopes.includes(value))
scopes.push(value);
else if (flag === "--evidence" && add && !evidenceIds.includes(value))
evidenceIds.push(value);
else return null;
index += 1;
}
if (add) {
return contentFile && scopes.length > 0 && evidenceIds.length > 0
? {
kind: "plan-add",
contentFile,
memoryKind,
scopes,
evidenceIds,
}
: null;
}
if (
(operation === "correct" && !contentFile) ||
(operation === "delete" && (contentFile !== undefined || scopes.length > 0))
) {
return null;
}
return {
kind: "plan-existing",
operation: operation as "correct" | "delete",
memoryId: memoryId as string,
...(contentFile ? { contentFile } : {}),
};
}
async function readMemoryContentFile(file: string): Promise<string | null> {
const bytes = await readBoundedFile(file, 100_000 * 4 + 2);
return bytes ? parseMemoryContent(bytes) : null;
}
function emitMemoryFlow(document: MemoryFlowDocument, asJson: boolean): void {
emit(
asJson ? `${JSON.stringify(document)}\n` : renderMemoryFlowText(document),
);
}
function emitModelWriteDocument(
document: ModelWriteDocument,
asJson: boolean,
@@ -6430,6 +6443,8 @@ function serviceErrorDocument(
"服务文件不属于当前 MemHub 安装;请保留文件并确认其来源后再处理。",
runtime_service_managed_upgrade_failed:
"新服务启动失败,旧 MemHub 服务已恢复;检查新安装日志后重试。",
runtime_service_replacement_failed:
"新服务启动失败;记忆数据尚未处于新版本可用的状态,所以没有恢复旧服务。请运行 memhub doctor 检查服务状态。",
runtime_service_managed_upgrade_rollback_failed:
"新服务启动失败且旧服务恢复失败;请运行 memhub doctor 检查服务状态。",
runtime_service_file_modified:
+2 -2
View File
@@ -387,7 +387,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
{
flag: "--content-file <path>",
summary:
"从有界普通文件读取新增/纠正正文;计划只保存摘要,执行时再次校验。",
"从有界普通文件读取 update 的新正文;预览只显示摘要,--confirm 时才写入。",
},
DIRECT_CONFIRM_OPTION,
JSON_OPTION,
@@ -485,7 +485,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
{
name: "recall",
summary:
"从当前授权的原文与治理记忆统一检索上下文;本地只读,不调用模型、不触发补索引。",
"从当前授权的原文与已沉淀记忆统一检索上下文;本地只读,不调用模型、不触发补索引。",
usage:
"memhub recall <query> [--kind <raw|memory>] [--source <id>] [--limit <n>] [--character-budget <n>] [--timeout-ms <n>] [--json]\n memhub recall --query-file <path> [...]\n memhub recall --stdin [...]",
options: [
+70 -135
View File
@@ -43,61 +43,35 @@ const ATTACHED = {
};
const RESPONSE = {
schema_version: 1,
schema_version: 2,
request_id: "context-request-1",
state: "grounded",
paths: {
raw: {
status: "ok",
reason: null,
elapsed_ms: 2,
returned_count: 1,
truncated: false,
truncated_by: [],
},
memory: {
status: "ok",
reason: null,
elapsed_ms: 3,
returned_count: 1,
truncated: false,
truncated_by: [],
},
},
items: [
{
kind: "raw",
source_id: "codex",
conversation_id: "conversation-1",
anchor_evidence_id: "evidence-1",
messages: [
{
evidence_id: "evidence-1",
revision: 1,
role: "user",
written_by: "user",
created_at: "2026-09-19T00:00:00Z",
text: "Synthetic raw citation.",
truncated: false,
},
],
raw_id: "raw-1",
content: "Synthetic raw citation.",
source: "codex",
session_id: "session-1",
memory_at: "2026-09-19T00:00:00Z",
score: 0.8,
truncated: false,
},
{
kind: "memory",
memory_id: "memory-1",
revision: 2,
content: "Synthetic governed memory.",
memory_kind: "working",
content: "Synthetic SDK memory.",
layer: "l2_fact",
status: "active",
source: "codex",
updated_at: "2026-09-19T00:00:01Z",
evidence_ids: ["evidence-1"],
score: 0.7,
truncated: false,
},
],
used_characters: 49,
used_characters: 44,
truncated: false,
total_ms: 5,
adoption: "unknown",
};
beforeEach(() => {
@@ -115,42 +89,17 @@ beforeEach(() => {
});
describe("unified context search", () => {
it("accepts an empty tool message in a valid partial context response", async () => {
it("accepts an empty raw item in a valid partial context response", async () => {
transport.post.mockResolvedValue({
status: 200,
data: {
...RESPONSE,
state: "partial",
paths: {
raw: {
...RESPONSE.paths.raw,
status: "degraded",
reason: "lexical_only",
returned_count: 1,
},
memory: {
...RESPONSE.paths.memory,
status: "empty",
returned_count: 0,
},
},
items: [
{
kind: "raw",
source_id: "workbuddy",
conversation_id: "conversation-1",
anchor_evidence_id: "evidence-1",
messages: [
{
evidence_id: "evidence-1",
revision: 1,
role: "tool",
written_by: "tool",
created_at: "2026-10-05T20:48:12Z",
text: "",
truncated: true,
},
],
...RESPONSE.items[0],
source: "workbuddy",
content: "",
truncated: true,
},
],
@@ -165,8 +114,9 @@ describe("unified context search", () => {
streams.restore();
}
expect(JSON.parse(streams.stdout())).toMatchObject({
schema_version: 2,
state: "partial",
items: [{ kind: "raw", messages: [{ text: "" }] }],
items: [{ kind: "raw", content: "" }],
});
});
@@ -178,12 +128,12 @@ describe("unified context search", () => {
streams.restore();
}
const output = streams.stdout();
expect(output).toContain("Synthetic governed memory.");
expect(output).toContain("Synthetic SDK memory.");
expect(output).toContain("Synthetic raw citation.");
expect(output).not.toMatch(/^- raw |memory-1|evidence-1|采用状态/m);
expect(output).not.toMatch(/^- raw |memory-1|raw-1|采用状态/m);
});
it("queries the current read-only context endpoint and preserves citations", async () => {
it("queries the SDK context endpoint directly and preserves item ids", async () => {
const streams = captureStreams();
try {
expect(
@@ -209,7 +159,7 @@ describe("unified context search", () => {
streams.restore();
}
expect(transport.post).toHaveBeenCalledWith(
"/api/v1/context/search",
"/api/v2/context/search",
{
query: "synthetic question",
kinds: ["raw", "memory"],
@@ -222,12 +172,18 @@ describe("unified context search", () => {
);
const output = JSON.parse(streams.stdout());
expect(output).toMatchObject({
schema_version: 2,
state: "grounded",
request_id: "context-request-1",
adoption: "unknown",
});
expect(output.items[0].messages[0].evidence_id).toBe("evidence-1");
expect(output.items[1].evidence_ids).toEqual(["evidence-1"]);
expect(output).not.toHaveProperty("paths");
expect(output.items[0]).toMatchObject({ kind: "raw", raw_id: "raw-1" });
expect(output.items[1]).toMatchObject({
kind: "memory",
memory_id: "memory-1",
layer: "l2_fact",
});
// No authority pre-query: released Core always serves the SDK data plane.
expect(transport.get).not.toHaveBeenCalled();
expect(transport.put).not.toHaveBeenCalled();
expect(transport.patch).not.toHaveBeenCalled();
@@ -242,19 +198,8 @@ describe("unified context search", () => {
status: 200,
data: {
...RESPONSE,
paths: {
...RESPONSE.paths,
raw: {
status: "not_requested",
reason: "not_requested",
elapsed_ms: 0,
returned_count: 0,
truncated: false,
truncated_by: [],
},
},
items: [RESPONSE.items[1]],
used_characters: 26,
used_characters: 21,
},
});
const streams = captureStreams();
@@ -394,29 +339,7 @@ describe("unified context search", () => {
status: 200,
data: {
...RESPONSE,
paths: {
...RESPONSE.paths,
memory: {
...RESPONSE.paths.memory,
status: "empty",
returned_count: 0,
},
},
items: [
{
...(RESPONSE.items[0] as Record<string, unknown>),
messages: [
{
...(
RESPONSE.items[0] as {
messages: Array<Record<string, unknown>>;
}
).messages[0],
text,
},
],
},
],
items: [{ ...RESPONSE.items[0], content: text }],
used_characters: 10,
},
});
@@ -429,7 +352,7 @@ describe("unified context search", () => {
expect(JSON.parse(streams.stdout())).toMatchObject({
state: "grounded",
used_characters: 10,
items: [{ kind: "raw", messages: [{ text }] }],
items: [{ kind: "raw", content: text }],
});
});
@@ -441,29 +364,7 @@ describe("unified context search", () => {
status: 200,
data: {
...RESPONSE,
paths: {
...RESPONSE.paths,
memory: {
...RESPONSE.paths.memory,
status: "empty",
returned_count: 0,
},
},
items: [
{
...(RESPONSE.items[0] as Record<string, unknown>),
messages: [
{
...(
RESPONSE.items[0] as {
messages: Array<Record<string, unknown>>;
}
).messages[0],
text,
},
],
},
],
items: [{ ...RESPONSE.items[0], content: text }],
used_characters: 12,
},
});
@@ -479,6 +380,40 @@ describe("unified context search", () => {
});
});
it.each([
[
"a raw item from a source that was not asked for",
["--source", "codex"],
{ ...RESPONSE.items[0], source: "cursor" },
],
[
"a raw item when only memory was asked for",
["--kind", "memory"],
RESPONSE.items[0],
],
["a score outside 0..1", [], { ...RESPONSE.items[1], score: 1.5 }],
])("rejects %s as an invalid response", async (_label, flags, item) => {
transport.post.mockResolvedValue({
status: 200,
data: {
...RESPONSE,
items: [item],
used_characters: Array.from(String(item.content)).length,
},
});
const streams = captureStreams();
try {
expect(await run(["recall", "query", ...flags, "--json"])).toBe(1);
} finally {
streams.restore();
}
expect(JSON.parse(streams.stdout())).toMatchObject({
schema_version: 2,
state: "invalid",
code: "response_invalid",
});
});
it("reports endpoint denial without exposing the response body", async () => {
transport.post.mockResolvedValue({
status: 403,
+43 -162
View File
@@ -3,31 +3,6 @@ import { PRODUCT } from "./version.js";
export const CONTEXT_QUERY_MAX_CHARACTERS = 2_000;
const CONTEXT_KINDS = ["raw", "memory"] as const;
const PATH_STATES = [
"ok",
"empty",
"not_requested",
"not_authorized",
"unavailable",
"degraded",
] as const;
const PATH_REASONS = [
"not_requested",
"permission_required",
"source_revoked",
"read_unavailable",
"timeout_budget",
"evidence_changed",
"memory_changed",
"lexical_only",
] as const;
const LIMIT_REASONS = [
"count_budget",
"character_budget",
"timeout_budget",
"metadata_budget",
] as const;
export interface ContextSearchInput {
query: string;
kinds: Array<(typeof CONTEXT_KINDS)[number]>;
@@ -37,8 +12,9 @@ export interface ContextSearchInput {
timeout_ms: number;
}
/** The CLI's JSON document mirrors Core's SDK-native context schema v2. */
export interface ContextSearchDocument {
schema_version: 1;
schema_version: 2;
product: string;
state:
| "grounded"
@@ -50,12 +26,10 @@ export interface ContextSearchDocument {
| "invalid";
code?: string;
request_id?: string;
paths?: Record<string, unknown>;
items: Array<Record<string, unknown>>;
used_characters?: number;
truncated?: boolean;
total_ms?: number;
adoption?: "unknown";
}
export async function searchContext(
@@ -67,7 +41,7 @@ export async function searchContext(
const validated = validateContextResponse(outcome.data, input);
return (
validated ?? {
schema_version: 1,
schema_version: 2,
product: PRODUCT,
state: "invalid",
code: "response_invalid",
@@ -131,13 +105,7 @@ export function renderContextText(document: ContextSearchDocument): string {
: "查到以下相关内容:",
];
for (const item of document.items) {
if (item.kind === "memory") {
lines.push(`- ${String(item.content)}`);
} else {
for (const message of item.messages as Array<Record<string, unknown>>) {
lines.push(`- ${String(message.text)}`);
}
}
lines.push(`- ${String(item.content)}`);
}
if (document.truncated) lines.push("结果已按请求预算截断。");
return `${lines.join("\n")}\n`;
@@ -149,84 +117,37 @@ function validateContextResponse(
): ContextSearchDocument | null {
const response = record(value);
if (
response?.schema_version !== 1 ||
response?.schema_version !== 2 ||
!oneOf(response.state, ["grounded", "empty", "partial", "unavailable"]) ||
!boundedText(response.request_id, 64) ||
!record(response.paths) ||
!Array.isArray(response.items) ||
response.items.length > request.limit ||
!integer(response.used_characters, 0, request.character_budget) ||
typeof response.truncated !== "boolean" ||
!number(response.total_ms, 0) ||
response.adoption !== "unknown"
!number(response.total_ms, 0)
) {
return null;
}
const paths = response.paths as Record<string, unknown>;
const rawPath = validatePath(paths.raw);
const memoryPath = validatePath(paths.memory);
if (!rawPath || !memoryPath) return null;
const requested = new Set(request.kinds);
if (
(!requested.has("raw") && rawPath.status !== "not_requested") ||
(!requested.has("memory") && memoryPath.status !== "not_requested")
) {
return null;
}
const items: Array<Record<string, unknown>> = [];
let characters = 0;
for (const itemValue of response.items) {
const item = validateItem(itemValue, request);
if (!item || !requested.has(item.kind as "raw" | "memory")) return null;
items.push(item);
characters +=
item.kind === "memory"
? characterCount(item.content)
: (item.messages as Array<Record<string, unknown>>).reduce(
(sum, message) => sum + characterCount(message.text),
0,
);
characters += characterCount(item.content);
}
if (characters !== response.used_characters) return null;
if (response.state === "empty" && items.length !== 0) return null;
return {
schema_version: 1,
schema_version: 2,
product: PRODUCT,
state: response.state,
request_id: response.request_id as string,
paths: { raw: rawPath, memory: memoryPath },
items,
used_characters: response.used_characters as number,
truncated: response.truncated,
total_ms: response.total_ms as number,
adoption: "unknown",
};
}
function validatePath(value: unknown): Record<string, unknown> | null {
const path = record(value);
if (
!path ||
!oneOf(path.status, PATH_STATES) ||
!number(path.elapsed_ms, 0) ||
!integer(path.returned_count, 0, 20) ||
typeof path.truncated !== "boolean" ||
!enumList(path.truncated_by, LIMIT_REASONS) ||
!(
path.reason === null ||
path.reason === undefined ||
oneOf(path.reason, PATH_REASONS)
)
) {
return null;
}
return {
status: path.status,
reason: path.reason ?? null,
elapsed_ms: path.elapsed_ms,
returned_count: path.returned_count,
truncated: path.truncated,
truncated_by: path.truncated_by,
};
}
@@ -235,88 +156,65 @@ function validateItem(
request: ContextSearchInput,
): Record<string, unknown> | null {
const item = record(value);
if (!item || !oneOf(item.kind, CONTEXT_KINDS)) return null;
if (
!item ||
!oneOf(item.kind, CONTEXT_KINDS) ||
boundedText(item.content, 20_000, true) === null ||
!number(item.score, 0) ||
(item.score as number) > 1 ||
typeof item.truncated !== "boolean"
) {
return null;
}
if (item.kind === "memory") {
if (
!boundedText(item.memory_id, 512) ||
!integer(item.revision, 1) ||
boundedText(item.content, 20_000, true) === null ||
boundedText(item.memory_kind, 64, true) === null ||
!boundedText(item.updated_at, 64) ||
!textList(item.evidence_ids, 128, 512) ||
typeof item.truncated !== "boolean"
boundedText(item.layer, 64, true) === null ||
boundedText(item.status, 32, true) === null ||
!optionalText(item.source, 128) ||
!optionalText(item.updated_at, 64)
) {
return null;
}
return {
kind: "memory",
memory_id: item.memory_id,
revision: item.revision,
content: item.content,
memory_kind: item.memory_kind,
updated_at: item.updated_at,
evidence_ids: item.evidence_ids,
layer: item.layer,
status: item.status,
source: item.source ?? null,
updated_at: item.updated_at ?? null,
score: item.score,
truncated: item.truncated,
};
}
if (
!boundedText(item.source_id, 128) ||
!boundedText(item.raw_id, 512) ||
!boundedText(item.source, 128) ||
(request.raw_source_ids !== undefined &&
!request.raw_source_ids.includes(item.source_id as string)) ||
!boundedText(item.conversation_id, 512) ||
!boundedText(item.anchor_evidence_id, 512) ||
!Array.isArray(item.messages) ||
item.messages.length < 1 ||
item.messages.length > 3 ||
typeof item.truncated !== "boolean"
!request.raw_source_ids.includes(item.source as string)) ||
!boundedText(item.session_id, 512) ||
!optionalText(item.memory_at, 64)
) {
return null;
}
const messages = item.messages.map(validateMessage);
if (messages.some((message) => message === null)) return null;
return {
kind: "raw",
source_id: item.source_id,
conversation_id: item.conversation_id,
anchor_evidence_id: item.anchor_evidence_id,
messages,
raw_id: item.raw_id,
content: item.content,
source: item.source,
session_id: item.session_id,
memory_at: item.memory_at ?? null,
score: item.score,
truncated: item.truncated,
};
}
function validateMessage(value: unknown): Record<string, unknown> | null {
const message = record(value);
if (
!message ||
!boundedText(message.evidence_id, 512) ||
!integer(message.revision, 1) ||
boundedText(message.role, 32, true) === null ||
boundedText(message.written_by, 32, true) === null ||
!(
message.created_at === null ||
boundedText(message.created_at, 64) !== null
) ||
boundedText(message.text, 20_000, true) === null ||
typeof message.truncated !== "boolean"
) {
return null;
}
return {
evidence_id: message.evidence_id,
revision: message.revision,
role: message.role,
written_by: message.written_by,
created_at: message.created_at,
text: message.text,
truncated: message.truncated,
};
}
function failed(
outcome: Exclude<ReadOutcome<unknown>, { state: "ok" }>,
): ContextSearchDocument {
return {
schema_version: 1,
schema_version: 2,
product: PRODUCT,
state: outcome.state,
code: outcome.code,
@@ -374,27 +272,10 @@ function oneOf<const T extends readonly string[]>(
return typeof value === "string" && values.includes(value as T[number]);
}
function enumList(
value: unknown,
values: readonly string[],
): value is string[] {
function optionalText(value: unknown, maximum: number): boolean {
return (
Array.isArray(value) &&
value.length <= 4 &&
new Set(value).size === value.length &&
value.every((item) => oneOf(item, values))
);
}
function textList(
value: unknown,
maximumItems: number,
maximumText: number,
): value is string[] {
return (
Array.isArray(value) &&
value.length <= maximumItems &&
new Set(value).size === value.length &&
value.every((item) => boundedText(item, maximumText) !== null)
value === null ||
value === undefined ||
boundedText(value, maximum) !== null
);
}
+4 -11
View File
@@ -8,14 +8,12 @@ import axios from "axios";
*/
const READ_ONLY_PATHS = [
"/api/v1/call-events",
"/api/v1/home-summary",
"/api/v1/import-jobs",
"/api/v1/live-sync",
"/api/v1/memory-library",
"/api/v1/memory-processing",
"/api/v1/memory-tasks",
"/api/v1/model-profiles",
"/api/v1/sources",
"/api/v2/home-summary",
"/api/v2/memory-library",
"/api/v2/user-profile",
] as const;
@@ -67,13 +65,10 @@ export function createUiSessionRequest(attachment: {
};
}
const MEMORY_DETAIL = /^\/api\/v1\/memories\/[A-Za-z0-9][\w.-]{0,127}$/;
const IMPORT_JOB =
/^\/api\/v1\/import-jobs\/[A-Za-z0-9][\w.-]{0,127}(?:\/issues)?$/;
const OPERATION_RECEIPT =
/^\/api\/v1\/(?:source-connections|model-profile-creations|model-probe-operations)\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const MEMORY_MUTATION_RECEIPT =
/^\/api\/v1\/memory-mutations\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
/**
* Core registers these static routes before `/{memory_id}`, so a memory whose
* id happens to be one of them would reach a different handler: `graph`
@@ -173,7 +168,7 @@ export function createCoreContextQuery(attachment: {
let response: { status: number; data: unknown };
try {
response = await client.post(
"/api/v1/context/search",
"/api/v2/context/search",
body,
options?.signal ? { signal: options.signal } : undefined,
);
@@ -197,10 +192,8 @@ export function isReadOnlyPath(path: string): boolean {
if ((READ_ONLY_PATHS as readonly string[]).includes(path)) return true;
if (IMPORT_JOB.test(path)) return true;
if (OPERATION_RECEIPT.test(path)) return true;
if (MEMORY_MUTATION_RECEIPT.test(path)) return true;
if (SDK_MEMORY_DETAIL.test(path)) return true;
if (!MEMORY_DETAIL.test(path)) return false;
const segment = path.slice("/api/v1/memories/".length);
if (!SDK_MEMORY_DETAIL.test(path)) return false;
const segment = path.slice("/api/v2/memories/".length);
return !(RESERVED_MEMORY_SEGMENTS as readonly string[]).includes(segment);
}
+24 -212
View File
@@ -1,18 +1,20 @@
import type {
GovernedMemory,
HomeSummary,
ImportIssue,
ImportJob,
LiveSyncHealth,
MemoryDetail,
MemoryEvidenceExcerpt,
MemoryLibraryPage,
MemoryProcessingSummary,
ModelProfile,
SourceResponse,
} from "@one-memory/contracts";
/** The only memory states a response may carry, and the only ones a filter may ask for. */
export const MEMORY_STATUSES = ["confirmed", "deleted", "superseded"] as const;
/** SDK memory states `memhub memories list --status` may ask for. */
export const MEMORY_STATUSES = [
"active",
"superseded",
"shadow",
"negated",
"conflicted",
"archived",
] as const;
/**
* HTTP 200 is not a contract. Everything here re-checks the shape and the
@@ -23,7 +25,6 @@ export const MEMORY_STATUSES = ["confirmed", "deleted", "superseded"] as const;
const ID_MAX = 128;
const LABEL_MAX = 256;
const TEXT_MAX = 100_000;
const LIST_MAX = 500;
const LIVE_SYNC_STATES = [
"idle",
@@ -39,14 +40,6 @@ const PROVIDER_KINDS = [
"local_onnx",
] as const;
const CREDENTIAL_STATES = ["available", "missing", "unavailable"] as const;
const MEMORY_KINDS = [
"chat",
"working",
"coding",
"schema",
"intention",
] as const;
const MEMORY_ORIGINS = ["extracted", "answered", "agent", "manual"] as const;
const IMPORT_STATUSES = [
"pending",
"running",
@@ -57,16 +50,6 @@ const IMPORT_STATUSES = [
"cancelled",
] as const;
const SCAN_PHASES = ["discover", "read", "emit", "done"] as const;
const MEMORY_TASK_KINDS = ["extract", "system2", "ultra", "sweeper"] as const;
const MEMORY_TASK_STATUSES = [
"pending",
"running",
"waiting_for_model",
"completed",
"failed",
"cancelled",
"deferred_outside_window",
] as const;
const CAPABILITY_STATES = ["not_implemented", "implemented"] as const;
const VERIFICATION_STATES = ["fixture_only", "real_replayed"] as const;
const SYNC_COMPLETENESS = [
@@ -87,16 +70,6 @@ const LOCATION_STATES = [
] as const;
const LIVE_SYNC_MODES = ["polling", "watch_and_polling"] as const;
export interface ValidatedMemoryTask {
task_id: string;
kind: (typeof MEMORY_TASK_KINDS)[number];
status: (typeof MEMORY_TASK_STATUSES)[number];
attempts: number;
last_error_code: string | null;
created_at: string;
updated_at: string;
}
export interface SourceConnectionCandidate {
source_id: string;
display_name: string;
@@ -352,45 +325,6 @@ export function validateImportIssues(
});
}
export function validateMemoryTasks(
value: unknown,
): ValidatedMemoryTask[] | null {
return list(value, (item) => {
const task = record(item);
if (!task) return null;
const taskId = text(task.task_id, ID_MAX);
if (
task.last_error_code !== null &&
text(task.last_error_code, LABEL_MAX) === null
) {
return null;
}
const error = task.last_error_code as string | null;
const createdAt = text(task.created_at, LABEL_MAX);
const updatedAt = text(task.updated_at, LABEL_MAX);
const attempts = count(task.attempts);
if (
taskId === null ||
createdAt === null ||
updatedAt === null ||
attempts === null ||
!oneOf(task.kind, MEMORY_TASK_KINDS) ||
!oneOf(task.status, MEMORY_TASK_STATUSES)
) {
return null;
}
return {
task_id: taskId,
kind: task.kind,
status: task.status,
attempts,
last_error_code: error,
created_at: createdAt,
updated_at: updatedAt,
};
});
}
export function validateProcessing(
value: unknown,
): MemoryProcessingSummary | null {
@@ -444,150 +378,32 @@ export function validateModelProfiles(value: unknown): ModelProfile[] | null {
});
}
export function validateHomeSummary(value: unknown): HomeSummary | null {
/** `GET /api/v2/home-summary`: the SDK-authority home card. */
export interface SdkHomeSummary {
schema_version: 2;
memory_count: number;
latest: { memory_id: string } | null;
observed_at: string;
}
export function validateHomeSummary(value: unknown): SdkHomeSummary | null {
const summary = record(value);
if (
!summary ||
summary.schema_version !== 2 ||
count(summary.memory_count) === null ||
text(summary.observed_at, LABEL_MAX) === null
) {
return null;
}
return summary as unknown as HomeSummary;
}
/** The page must be the page that was asked for, not merely a page. */
export function validateMemoryPage(
value: unknown,
request: { offset: number; limit: number },
): MemoryLibraryPage | null {
const page = record(value);
if (!page) return null;
const total = count(page.total);
if (
total === null ||
page.offset !== request.offset ||
page.limit !== request.limit
) {
return null;
}
const items = list(page.items, validateGovernedMemory);
if (items === null || items.length > request.limit) return null;
if (items.length > 0 && total < request.offset + items.length) return null;
return { items, total, offset: page.offset, limit: page.limit };
}
export function validateMemoryDetail(
value: unknown,
memoryId: string,
): MemoryDetail | null {
const detail = record(value);
if (!detail) return null;
const memory = validateGovernedMemory(detail.memory);
if (memory === null || memory.memory_id !== memoryId) return null;
const cited = new Set(memory.evidence_ids);
const returned = new Set<string>();
const evidence = list(detail.evidence ?? [], (item) => {
const excerpt = record(item);
if (!excerpt) return null;
const evidenceId = text(excerpt.evidence_id, ID_MAX);
// Core builds this list from the memory's own citations; anything else is
// another record's excerpt and must not be shown under this memory.
if (
evidenceId === null ||
!cited.has(evidenceId) ||
returned.has(evidenceId) ||
typeof excerpt.available !== "boolean"
) {
const latest = summary.latest;
if (latest !== null) {
const memory = record(latest);
if (!memory || typeof memory.memory_id !== "string" || !memory.memory_id) {
return null;
}
returned.add(evidenceId);
const available = excerpt.available === true;
const verbatim = available ? text(excerpt.verbatim ?? "", TEXT_MAX) : "";
if (verbatim === null) return null;
return {
evidence_id: evidenceId,
available,
source_id: optional(excerpt.source_id, ID_MAX),
written_by: optional(excerpt.written_by, LABEL_MAX),
created_at: optional(excerpt.created_at, LABEL_MAX),
// An excerpt the user cannot read has no text, whatever the server sent.
verbatim,
} as unknown as MemoryEvidenceExcerpt;
});
if (evidence === null || evidence.length !== memory.evidence_ids.length) {
return null;
}
const history = list(detail.history ?? [], (item) => {
const version = record(item);
if (
!version ||
typeof version.revision !== "number" ||
!Number.isSafeInteger(version.revision) ||
version.revision < 1 ||
text(version.action, LABEL_MAX) === null ||
!oneOf(version.status, MEMORY_STATUSES) ||
text(version.content, TEXT_MAX) === null ||
identifiers(version.scopes) === null ||
text(version.created_at, LABEL_MAX) === null
) {
return null;
}
return {
revision: version.revision,
action: version.action,
status: version.status,
content: version.content,
scopes: identifiers(version.scopes) as string[],
created_at: version.created_at,
};
});
if (history === null) return null;
return {
memory,
evidence,
history,
index_sync: null,
} as unknown as MemoryDetail;
}
export function validateGovernedMemory(value: unknown): GovernedMemory | null {
const memory = record(value);
if (
!memory ||
!text(memory.memory_id, ID_MAX) ||
!text(memory.user_id, LABEL_MAX) ||
!text(memory.content, TEXT_MAX) ||
!oneOf(memory.kind, MEMORY_KINDS) ||
!oneOf(memory.origin, MEMORY_ORIGINS) ||
typeof memory.revision !== "number" ||
!Number.isSafeInteger(memory.revision) ||
memory.revision < 1 ||
!oneOf(memory.status, MEMORY_STATUSES) ||
!text(memory.created_at, LABEL_MAX) ||
!text(memory.updated_at, LABEL_MAX)
) {
return null;
}
const scopes = identifiers(memory.scopes);
const evidenceIds = identifiers(memory.evidence_ids);
const tags = identifiers(memory.tags ?? []);
if (
scopes === null ||
scopes.length < 1 ||
evidenceIds === null ||
evidenceIds.length < 1 ||
tags === null ||
tags.length > 32
) {
return null;
}
return {
...memory,
scopes,
evidence_ids: evidenceIds,
tags,
} as unknown as GovernedMemory;
return summary as unknown as SdkHomeSummary;
}
function identifiers(value: unknown): string[] | null {
@@ -618,10 +434,6 @@ function text(value: unknown, max: number): string | null {
return typeof value === "string" && value.length <= max ? value : null;
}
function optional(value: unknown, max: number): string | undefined {
return typeof value === "string" && value.length <= max ? value : undefined;
}
function count(value: unknown): number | null {
return typeof value === "number" && Number.isSafeInteger(value) && value >= 0
? value
@@ -134,11 +134,6 @@ function progress(): ProgressDocument {
generation: {
summary_state: { state: "ok" },
summary: { waiting_for_model: 1, failed: 0 },
tasks_state: { state: "ok" },
tasks: [{ task_id: "t1", status: "waiting_for_model" }],
returned: 1,
validated: 1,
output_truncated: false,
},
};
}
+21 -12
View File
@@ -65,6 +65,9 @@ export function renderDoctorText(document: DoctorDocument): string {
model: "模型",
pending_operations: "待处理操作",
};
const migration = document.checks.find(
(check) => check.id === "core" && check.details?.log_file,
);
return [
`MemHub 检查:${doctorState(document.state)}`,
...document.checks.map(
@@ -73,7 +76,13 @@ export function renderDoctorText(document: DoctorDocument): string {
),
...(document.state === "healthy"
? []
: ["下一步:打开 MemHub 的“帮助与诊断”查看并处理异常。"]),
: migration
? [
`升级迁移没有完成(错误码:${migration.code}),MemHub 不会启动。`,
`下一步:把迁移日志附在 GitHub 反馈里发给我们:${migration.details?.log_file}`,
`反馈地址:${migration.details?.issues_url}`,
]
: ["下一步:打开 MemHub 的“帮助与诊断”查看并处理异常。"]),
"",
].join("\n");
}
@@ -107,6 +116,14 @@ function coreCheck(status: StatusDocument): DoctorCheck {
id: "core",
state: status.runtime.state === "stopped" ? "warn" : "fail",
code: status.runtime.code,
...(status.runtime.log_file
? {
details: {
log_file: status.runtime.log_file,
issues_url: status.runtime.issues_url ?? "",
},
}
: {}),
};
}
@@ -200,8 +217,7 @@ function modelCheck(status: StatusDocument): DoctorCheck {
function pendingCheck(progress: ProgressDocument | null): DoctorCheck {
if (
progress?.imports.state !== "ok" ||
progress.generation.summary_state.state !== "ok" ||
progress.generation.tasks_state.state !== "ok"
progress.generation.summary_state.state !== "ok"
) {
return {
id: "pending_operations",
@@ -212,27 +228,20 @@ function pendingCheck(progress: ProgressDocument | null): DoctorCheck {
const jobAttention = progress.imports.jobs.filter((item) =>
["paused", "interrupted", "failed"].includes(text(item.status) ?? ""),
).length;
const taskAttention = progress.generation.tasks.filter((item) =>
["waiting_for_model", "failed"].includes(text(item.status) ?? ""),
).length;
const summary = progress.generation.summary;
const waiting = summary ? number(summary.waiting_for_model) : 0;
const failed = summary ? number(summary.failed) : 0;
const attention = jobAttention + taskAttention + waiting + failed;
const attention = jobAttention + waiting + failed;
return {
id: "pending_operations",
state: attention > 0 ? "warn" : "pass",
code: attention > 0 ? "operations_need_attention" : "pending_operations_ok",
details: {
import_jobs_validated: progress.imports.validated,
generation_tasks_validated: progress.generation.validated,
jobs_need_attention: jobAttention,
tasks_need_attention: taskAttention,
waiting_for_model: waiting,
failed,
output_truncated:
progress.imports.output_truncated ||
progress.generation.output_truncated,
output_truncated: progress.imports.output_truncated,
},
};
}
-700
View File
@@ -1,700 +0,0 @@
import { createHash, randomUUID, timingSafeEqual } from "node:crypto";
import type { GovernedMemory } from "@one-memory/contracts";
import {
buildMemoryActionRequest,
buildMemoryCorrectRequest,
buildMemoryCreateRequest,
parseMemoryReceipt,
} from "@one-memory/operations";
import type { CoreReader } from "./core-client.js";
import type { CoreOperations } from "./core-operations.js";
import { validateMemoryDetail } from "./core-responses.js";
import { isMemoryId } from "./memory-report.js";
import { PRODUCT } from "./version.js";
const MEMORY_KINDS = [
"chat",
"working",
"coding",
"schema",
"intention",
] as const;
const OPERATIONS = ["add", "correct", "delete"] as const;
const MEMORY_ORIGINS = ["extracted", "answered", "agent", "manual"] as const;
const MUTATION_RECEIPT_KEYS = new Set([
"schema_version",
"request_id",
"state",
"operation",
"memory_id",
"result_revision",
"result_status",
"memory_state",
"migrated_from",
]);
const CONTENT_MAX_CHARACTERS = 100_000;
const ID = /^[A-Za-z0-9][\w.-]{0,127}$/;
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
interface PlanBody {
schema_version: 1;
kind: "memory_mutation";
operation: (typeof OPERATIONS)[number];
request_id: string;
memory_id?: string;
expected_revision?: number;
expected_status?: string;
expected_origin?: string;
memory_kind?: (typeof MEMORY_KINDS)[number];
scopes?: string[];
evidence_ids?: string[];
content_sha256?: string;
content_characters?: number;
current?: {
status: string;
origin: string;
revision: number;
scopes: string[];
evidence: Array<{
evidence_id: string;
available: boolean;
source_id?: string;
written_by?: string;
}>;
};
}
export type MemoryMutationPlan = PlanBody & { digest: string };
export type MemoryFlowDocument =
| {
schema_version: 1;
product: string;
state: "preview";
plan: MemoryMutationPlan;
}
| {
schema_version: 1;
product: string;
state: "committed" | "recovered";
operation_id: string;
receipt: Record<string, unknown>;
}
| {
schema_version: 1;
product: string;
state: "rejected" | "unknown" | "invalid";
operation_id?: string;
code: string;
};
export function parseMemoryContent(bytes: Buffer): string | null {
if (bytes.byteLength > CONTENT_MAX_CHARACTERS * 4 + 2) return null;
let decoded: string;
try {
decoded = new TextDecoder("utf-8", { fatal: true }).decode(bytes);
} catch {
return null;
}
const content = decoded.replace(/\r?\n$/, "");
return content.trim().length > 0 &&
[...content].length <= CONTENT_MAX_CHARACTERS &&
![...content].some((character) => {
const code = character.charCodeAt(0);
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
})
? content
: null;
}
export function planMemoryAdd(input: {
content: string;
memoryKind: string;
scopes: string[];
evidenceIds: string[];
}): MemoryFlowDocument {
if (
!validContent(input.content) ||
!oneOf(input.memoryKind, MEMORY_KINDS) ||
!validScopes(input.scopes) ||
!validIds(input.evidenceIds, 500)
) {
return invalid("memory_add_input_invalid");
}
return preview(
withDigest({
schema_version: 1,
kind: "memory_mutation",
operation: "add",
request_id: randomUUID(),
memory_kind: input.memoryKind,
scopes: input.scopes,
evidence_ids: input.evidenceIds,
...contentIdentity(input.content),
}),
);
}
export async function planExistingMemoryMutation(
reader: CoreReader,
input: {
operation: Exclude<(typeof OPERATIONS)[number], "add">;
memoryId: string;
content?: string;
},
): Promise<MemoryFlowDocument> {
if (
!isMemoryId(input.memoryId) ||
(input.operation === "correct" && input.content === undefined) ||
(input.content !== undefined && !validContent(input.content))
) {
return invalid("memory_plan_input_invalid");
}
const outcome = await reader<unknown>(`/api/v1/memories/${input.memoryId}`);
if (outcome.state !== "ok") {
return invalid(`memory_${outcome.state}:${outcome.code}`);
}
const detail = validateMemoryDetail(outcome.data, input.memoryId);
if (!detail) return invalid("memory_response_invalid");
const memory = detail.memory;
if (memory.status !== "confirmed") {
return invalid("memory_operation_not_allowed");
}
const body: PlanBody = {
schema_version: 1,
kind: "memory_mutation",
operation: input.operation,
request_id: randomUUID(),
memory_id: memory.memory_id,
expected_revision: memory.revision,
expected_status: memory.status,
expected_origin: memory.origin,
...(input.content ? contentIdentity(input.content) : {}),
current: {
status: memory.status,
origin: memory.origin,
revision: memory.revision,
scopes: [...memory.scopes],
evidence: detail.evidence.map((item) => ({
evidence_id: item.evidence_id,
available: item.available,
...(item.source_id ? { source_id: item.source_id } : {}),
...(item.written_by ? { written_by: item.written_by } : {}),
})),
},
};
return preview(withDigest(body));
}
export function inspectMemoryPlan(
value: unknown,
): { operationId: string; digest: string; needsContent: boolean } | null {
const plan = validatePlan(value);
return plan
? {
operationId: plan.request_id,
digest: plan.digest,
needsContent: plan.operation === "add" || plan.operation === "correct",
}
: null;
}
export function memoryPlanAcceptsContent(
value: unknown,
content: string | undefined,
): boolean {
const plan = validatePlan(value);
if (!plan) return false;
if (plan.operation !== "add" && plan.operation !== "correct") {
return content === undefined;
}
if (content === undefined) return false;
const identity = contentIdentity(content);
return (
identity.content_sha256 === plan.content_sha256 &&
identity.content_characters === plan.content_characters
);
}
export async function executeMemoryMutation(
value: unknown,
content: string | undefined,
operations: CoreOperations,
reader?: CoreReader,
): Promise<MemoryFlowDocument> {
const plan = validatePlan(value);
if (!plan) return invalid("memory_plan_invalid");
if (!memoryPlanAcceptsContent(plan, content)) {
return invalid("memory_content_mismatch");
}
const { path, body } = operationRequest(plan, content);
const outcome = await operations.post<unknown>(path, body);
if (outcome.state === "rejected") {
return {
schema_version: 1,
product: PRODUCT,
state: "rejected",
operation_id: plan.request_id,
code: outcome.code,
};
}
if (outcome.state === "unknown") {
return reader
? readMemoryMutationReceipt(reader, plan)
: pending(plan.request_id, outcome.code);
}
const memory = validateWriteResponse(outcome.data, plan);
if (!memory || !matchesReceipt(memory, plan, content)) {
return reader
? readMemoryMutationReceipt(reader, plan)
: pending(plan.request_id, "memory_receipt_invalid");
}
return {
schema_version: 1,
product: PRODUCT,
state: "committed",
operation_id: plan.request_id,
receipt: {
memory_id: memory.memory_id,
status: memory.status,
origin: memory.origin,
revision: memory.revision,
scopes: memory.scopes,
evidence_ids: memory.evidence_ids,
content_sha256: contentIdentity(memory.content).content_sha256,
content_characters: [...memory.content].length,
created_at: memory.created_at,
updated_at: memory.updated_at,
},
};
}
/**
* Validate the write response through the shared `parseMemoryReceipt`, which
* owns the GovernedMemory shape and the revision/status binding Core must
* satisfy. The CLI keeps only its own plan binding (`matchesReceipt`, which
* also checks `origin`, `kind` and the echoed content), so the two surfaces
* judge a write response identically and no shape rule is duplicated here.
*
* `add` creates the first revision and expects `confirmed`; the shared parser
* derives that from `expectedRevision: 0`. Every other operation advances the
* revision the caller acted against and expects `confirmed` (correct/scope)
* or `deleted` (delete).
*/
function validateWriteResponse(
value: unknown,
plan: MemoryMutationPlan,
): GovernedMemory | null {
const expectedStatus = plan.operation === "delete" ? "deleted" : "confirmed";
// For `add` there is no id or prior revision to bind yet, so the parser's
// own receipt is the only source for both; the plan binding that follows is
// what proves the response belongs to this request.
const memoryId =
plan.operation === "add"
? String((value as { memory_id?: unknown } | null)?.memory_id ?? "")
: (plan.memory_id as string);
if (memoryId === "") return null;
return parseMemoryReceipt(value, {
memoryId,
expectedRevision:
plan.operation === "add" ? 0 : (plan.expected_revision as number),
expectedStatus,
});
}
export async function readMemoryMutationReceipt(
reader: CoreReader,
value: unknown,
): Promise<MemoryFlowDocument> {
const plan = validatePlan(value);
if (!plan) return invalid("memory_plan_invalid");
const outcome = await reader<unknown>(
`/api/v1/memory-mutations/${encodeURIComponent(plan.request_id)}`,
);
if (outcome.state !== "ok") {
return pending(plan.request_id, `memory_mutation_receipt_${outcome.code}`);
}
const receipt = validateMutationReceipt(outcome.data, plan);
if (receipt === "not_found") {
return pending(
plan.request_id,
"memory_mutation_receipt_not_found_after_attempt",
);
}
if (!receipt) {
return pending(plan.request_id, "memory_mutation_receipt_invalid");
}
return {
schema_version: 1,
product: PRODUCT,
state: "recovered",
operation_id: plan.request_id,
receipt,
};
}
export function memoryFlowExitCode(document: MemoryFlowDocument): number {
return ["preview", "committed", "recovered"].includes(document.state) ? 0 : 1;
}
export function renderMemoryFlowText(document: MemoryFlowDocument): string {
if (document.state === "preview") {
const plan = document.plan;
const lines = [
`记忆操作预览:${plan.operation}`,
`request_id:${plan.request_id}`,
];
if (plan.memory_id) lines.push(`memory_id:${plan.memory_id}`);
if (plan.expected_revision) {
lines.push(
`当前版本:${plan.expected_revision} 状态:${plan.expected_status} 来源方式:${plan.expected_origin}`,
);
}
if (plan.scopes) lines.push(`使用范围:${plan.scopes.join(",")}`);
if (plan.current) {
for (const evidence of plan.current.evidence) {
lines.push(
`证据 ${evidence.evidence_id}:source=${evidence.source_id ?? "unknown"} author=${evidence.written_by ?? "unknown"} available=${evidence.available}`,
);
}
}
if (plan.content_characters !== undefined) {
lines.push(
`正文:${plan.content_characters} 字符,sha256=${plan.content_sha256}`,
);
}
lines.push("尚未写入;保存 JSON 计划后用 memories apply --confirm 执行。");
return `${lines.join("\n")}\n`;
}
if (document.state === "committed" || document.state === "recovered") {
return `记忆操作${document.state === "recovered" ? "已从治理回执恢复" : "已提交"}:${document.operation_id}\n${JSON.stringify(document.receipt)}\n`;
}
return `记忆操作未完成:${document.state}(${"code" in document ? document.code : "memory_operation_state_invalid"})${document.operation_id ? ` request_id=${document.operation_id}` : ""}\n`;
}
function operationRequest(
plan: MemoryMutationPlan,
content: string | undefined,
): { path: string; body: Record<string, unknown> } {
if (plan.operation === "add") {
// The shared create builder is the Web console's manual-add body, which
// always creates `chat` (the Web has no kind picker). The CLI exposes
// `--kind`, so it builds through the same builder and then sets the kind,
// keeping every other field (user_id/scope/evidence/origin) shared.
const request = buildMemoryCreateRequest(
{
content: content as string,
scopes: plan.scopes ?? [],
evidence_ids: plan.evidence_ids ?? [],
origin: "manual",
},
plan.request_id,
);
const draft = request.body.draft as Record<string, unknown>;
draft.kind = plan.memory_kind;
return { path: request.path, body: request.body };
}
if (plan.operation === "correct") {
return buildMemoryCorrectRequest(
plan.memory_id as string,
plan.expected_revision as number,
content as string,
undefined,
plan.request_id,
);
}
return buildMemoryActionRequest(
plan.memory_id as string,
plan.operation,
plan.expected_revision as number,
plan.request_id,
);
}
function matchesReceipt(
memory: GovernedMemory,
plan: MemoryMutationPlan,
content: string | undefined,
): boolean {
if (plan.operation === "add") {
return (
memory.status === "confirmed" &&
memory.origin === "manual" &&
memory.revision === 1 &&
memory.kind === plan.memory_kind &&
same(memory.scopes, plan.scopes ?? []) &&
same(memory.evidence_ids, plan.evidence_ids ?? []) &&
memory.content === content
);
}
if (
memory.memory_id !== plan.memory_id ||
memory.revision !== (plan.expected_revision as number) + 1 ||
memory.origin !== plan.expected_origin
) {
return false;
}
if (plan.operation === "delete" && memory.status !== "deleted") return false;
if (plan.operation === "correct" && memory.status !== plan.expected_status) {
return false;
}
if (plan.operation === "correct" && memory.content !== content) return false;
const expectedScopes =
plan.scopes ?? (plan.current?.scopes as string[] | undefined);
return expectedScopes ? same(memory.scopes, expectedScopes) : true;
}
function validateMutationReceipt(
value: unknown,
plan: MemoryMutationPlan,
): Record<string, unknown> | "not_found" | null {
const receipt = record(value);
if (
!receipt ||
Object.keys(receipt).length !== MUTATION_RECEIPT_KEYS.size ||
Object.keys(receipt).some((key) => !MUTATION_RECEIPT_KEYS.has(key)) ||
receipt.schema_version !== 2 ||
receipt.request_id !== plan.request_id
) {
return null;
}
if (receipt.state === "not_found") {
return [
receipt.operation,
receipt.memory_id,
receipt.result_revision,
receipt.result_status,
receipt.memory_state,
receipt.migrated_from,
].every((item) => item === null)
? "not_found"
: null;
}
const expectedOperation =
plan.operation === "add" ? "create" : plan.operation;
const expectedRevision =
plan.operation === "add" ? 1 : (plan.expected_revision as number) + 1;
const expectedStatus = plan.operation === "delete" ? "deleted" : "confirmed";
// A migrated receipt records an operation from before v2; no plan made
// today can have produced it.
if (
receipt.state !== "committed" ||
receipt.migrated_from !== null ||
receipt.operation !== expectedOperation ||
typeof receipt.memory_id !== "string" ||
!isMemoryId(receipt.memory_id) ||
(plan.memory_id !== undefined && receipt.memory_id !== plan.memory_id) ||
receipt.result_revision !== expectedRevision ||
receipt.result_status !== expectedStatus ||
!["present", "deleted"].includes(String(receipt.memory_state))
) {
return null;
}
return {
schema_version: 2,
request_id: plan.request_id,
state: "committed",
operation: expectedOperation,
memory_id: receipt.memory_id,
result_revision: expectedRevision,
result_status: expectedStatus,
memory_state: receipt.memory_state,
proof_scope: "governance_transaction_only",
downstream_sync_verified: false,
payload_fingerprint_verified: false,
};
}
function validatePlan(value: unknown): MemoryMutationPlan | null {
const plan = record(value);
if (
plan?.schema_version !== 1 ||
plan.kind !== "memory_mutation" ||
!oneOf(plan.operation, OPERATIONS) ||
typeof plan.request_id !== "string" ||
!UUID.test(plan.request_id) ||
typeof plan.digest !== "string"
) {
return null;
}
const body = { ...plan };
delete body.digest;
const expected = withDigest(body as unknown as PlanBody);
if (!safeEqual(expected.digest, plan.digest)) return null;
if (plan.operation === "add") {
if (
!oneOf(plan.memory_kind, MEMORY_KINDS) ||
!validScopes(plan.scopes) ||
!validIds(plan.evidence_ids, 500) ||
!contentFields(plan)
) {
return null;
}
} else if (
typeof plan.memory_id !== "string" ||
!isMemoryId(plan.memory_id) ||
!Number.isSafeInteger(plan.expected_revision) ||
(plan.expected_revision as number) < 1 ||
plan.expected_status !== "confirmed" ||
!oneOf(plan.expected_origin, MEMORY_ORIGINS) ||
!validCurrent(plan) ||
(plan.operation === "correct" && !contentFields(plan))
) {
return null;
}
return plan as unknown as MemoryMutationPlan;
}
function contentFields(plan: Record<string, unknown>): boolean {
return (
typeof plan.content_sha256 === "string" &&
/^[0-9a-f]{64}$/.test(plan.content_sha256) &&
Number.isSafeInteger(plan.content_characters) &&
(plan.content_characters as number) >= 1 &&
(plan.content_characters as number) <= CONTENT_MAX_CHARACTERS
);
}
function validCurrent(plan: Record<string, unknown>): boolean {
const current = record(plan.current);
if (
!current ||
current.status !== plan.expected_status ||
current.origin !== plan.expected_origin ||
current.revision !== plan.expected_revision ||
!validScopes(current.scopes) ||
!Array.isArray(current.evidence) ||
current.evidence.length < 1 ||
current.evidence.length > 500
) {
return false;
}
return current.evidence.every((value) => {
const evidence = record(value);
return (
evidence !== null &&
typeof evidence.evidence_id === "string" &&
ID.test(evidence.evidence_id) &&
typeof evidence.available === "boolean" &&
(evidence.source_id === undefined ||
(typeof evidence.source_id === "string" &&
evidence.source_id.length <= 128)) &&
(evidence.written_by === undefined ||
(typeof evidence.written_by === "string" &&
evidence.written_by.length <= 256))
);
});
}
function contentIdentity(content: string): {
content_sha256: string;
content_characters: number;
} {
return {
content_sha256: createHash("sha256").update(content, "utf8").digest("hex"),
content_characters: [...content].length,
};
}
function validContent(content: string): boolean {
return (
content.trim().length > 0 &&
[...content].length <= CONTENT_MAX_CHARACTERS &&
![...content].some((character) => {
const code = character.charCodeAt(0);
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
})
);
}
function validScopes(value: unknown): value is string[] {
if (
!Array.isArray(value) ||
value.length < 1 ||
value.length > 128 ||
new Set(value).size !== value.length ||
value.some(
(scope) =>
typeof scope !== "string" ||
scope.trim() !== scope ||
scope.length < 1 ||
scope.length > 512 ||
scope === "*",
)
) {
return false;
}
return !value.includes("private") || value.length === 1;
}
function validIds(value: unknown, maximum: number): value is string[] {
return (
Array.isArray(value) &&
value.length >= 1 &&
value.length <= maximum &&
new Set(value).size === value.length &&
value.every((item) => typeof item === "string" && ID.test(item))
);
}
function withDigest(body: PlanBody): MemoryMutationPlan {
return {
...body,
digest: createHash("sha256")
.update(JSON.stringify(body), "utf8")
.digest("hex"),
};
}
function preview(plan: MemoryMutationPlan): MemoryFlowDocument {
return {
schema_version: 1,
product: PRODUCT,
state: "preview",
plan,
};
}
function invalid(code: string): MemoryFlowDocument {
return {
schema_version: 1,
product: PRODUCT,
state: "invalid",
code,
};
}
function pending(operationId: string, code: string): MemoryFlowDocument {
return {
schema_version: 1,
product: PRODUCT,
state: "unknown",
operation_id: operationId,
code,
};
}
function same(left: readonly string[], right: readonly string[]): boolean {
return (
left.length === right.length &&
left.every((value, index) => value === right[index])
);
}
function safeEqual(left: string, right: string): boolean {
if (left.length !== right.length) return false;
return timingSafeEqual(Buffer.from(left), Buffer.from(right));
}
function oneOf<const T extends readonly string[]>(
value: unknown,
values: T,
): value is T[number] {
return typeof value === "string" && values.includes(value as T[number]);
}
function record(value: unknown): Record<string, unknown> | null {
return value !== null && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: null;
}
@@ -1,211 +0,0 @@
import { readFileSync } from "node:fs";
import { describe, expect, it } from "vitest";
import type { CoreReader } from "./core-client.js";
import {
planExistingMemoryMutation,
planMemoryAdd,
readMemoryMutationReceipt,
} from "./memory-flows.js";
// Written by a real Core and checked against it on the Core side, so these are
// the receipts the CLI meets in use rather than ones shaped to fit the parser.
const FIXTURE = JSON.parse(
readFileSync(
new URL(
"../../../packages/contracts/fixtures/memory-mutation-receipts.v2.json",
import.meta.url,
),
"utf8",
),
) as { receipts: Record<"add" | "correct" | "delete", Receipt> };
interface Receipt {
schema_version: number;
request_id: string;
memory_id: string;
result_revision: number;
result_status: string;
operation: string;
}
function reader(data: unknown): CoreReader {
return async <T>() => ({ state: "ok", data: data as T });
}
function detail(sample: Receipt) {
const revision = sample.result_revision - 1;
return {
memory: {
memory_id: sample.memory_id,
user_id: "local",
kind: "chat",
status: "confirmed",
content: "Synthetic memory.",
scopes: ["all-apps"],
evidence_ids: ["evidence-1"],
origin: "manual",
tags: [],
revision,
created_at: "2026-09-19T00:00:00Z",
updated_at: "2026-09-19T00:00:01Z",
deleted_at: null,
},
evidence: [
{
evidence_id: "evidence-1",
available: true,
source_id: "fixture",
written_by: "person",
created_at: "2026-09-19T00:00:00Z",
verbatim: "Synthetic source text.",
},
],
history: [
{
revision,
action: "create",
status: "confirmed",
content: "Synthetic memory.",
scopes: ["all-apps"],
created_at: "2026-09-19T00:00:01Z",
},
],
index_sync: null,
};
}
describe("memory mutation receipt v2 from Core", () => {
it("recovers an add from the receipt Core returns for a create", async () => {
const sample = FIXTURE.receipts.add;
expect(sample.schema_version).toBe(2);
const preview = planMemoryAdd({
content: "Synthetic memory.",
memoryKind: "chat",
scopes: ["all-apps"],
evidenceIds: ["evidence-1"],
});
if (preview.state !== "preview") throw new Error("preview expected");
// The request id is the one binding a receipt to its plan; the plan picks it.
const result = await readMemoryMutationReceipt(
reader({ ...sample, request_id: preview.plan.request_id }),
preview.plan,
);
expect(result).toMatchObject({
state: "recovered",
receipt: {
schema_version: 2,
operation: "create",
memory_id: sample.memory_id,
result_revision: 1,
result_status: "confirmed",
memory_state: "present",
},
});
});
it.each<["correct" | "delete", { content?: string }]>([
["correct", { content: "Corrected synthetic memory." }],
["delete", {}],
])(
"recovers %s from the receipt Core returns for it",
async (operation, extra) => {
const sample = FIXTURE.receipts[operation];
const preview = await planExistingMemoryMutation(reader(detail(sample)), {
operation,
memoryId: sample.memory_id,
...extra,
});
if (preview.state !== "preview") {
throw new Error(`preview expected: ${JSON.stringify(preview)}`);
}
const result = await readMemoryMutationReceipt(
reader({ ...sample, request_id: preview.plan.request_id }),
preview.plan,
);
expect(result).toMatchObject({
state: "recovered",
receipt: {
schema_version: 2,
operation: sample.operation,
memory_id: sample.memory_id,
result_revision: sample.result_revision,
result_status: sample.result_status,
},
});
},
);
it("does not take a v1 receipt for a v2 one", async () => {
const sample = FIXTURE.receipts.delete;
const preview = await planExistingMemoryMutation(reader(detail(sample)), {
operation: "delete",
memoryId: sample.memory_id,
});
if (preview.state !== "preview") throw new Error("preview expected");
const { migrated_from: _dropped, ...v1 } = sample as Receipt & {
migrated_from: null;
};
// As v1 was, and with v2's fields but v1's number: either alone is refused.
for (const receipt of [
{ ...v1, schema_version: 1 },
{ ...sample, schema_version: 1 },
]) {
const result = await readMemoryMutationReceipt(
reader({ ...receipt, request_id: preview.plan.request_id }),
preview.plan,
);
expect(result).toMatchObject({
state: "unknown",
code: "memory_mutation_receipt_invalid",
});
}
});
it("does not take a not-found receipt that names a migrated operation", async () => {
const sample = FIXTURE.receipts.delete;
const preview = await planExistingMemoryMutation(reader(detail(sample)), {
operation: "delete",
memoryId: sample.memory_id,
});
if (preview.state !== "preview") throw new Error("preview expected");
const notFound = {
schema_version: 2,
request_id: preview.plan.request_id,
state: "not_found",
operation: null,
memory_id: null,
result_revision: null,
result_status: null,
memory_state: null,
};
expect(
await readMemoryMutationReceipt(
reader({ ...notFound, migrated_from: null }),
preview.plan,
),
).toMatchObject({
code: "memory_mutation_receipt_not_found_after_attempt",
});
expect(
await readMemoryMutationReceipt(
reader({ ...notFound, migrated_from: "reject" }),
preview.plan,
),
).toMatchObject({ code: "memory_mutation_receipt_invalid" });
});
it("plans nothing on a memory that is not in effect", async () => {
const sample = FIXTURE.receipts.delete;
const superseded = detail(sample);
superseded.memory.status = "superseded";
superseded.history[0].status = "superseded";
const result = await planExistingMemoryMutation(reader(superseded), {
operation: "delete",
memoryId: sample.memory_id,
});
expect(result).toMatchObject({
state: "invalid",
code: "memory_operation_not_allowed",
});
});
});
+23
View File
@@ -15,6 +15,29 @@ export function isMemoryId(value: string): boolean {
);
}
const CONTENT_MAX_CHARACTERS = 100_000;
/** A memory body read from `--content-file`: UTF-8, non-blank, bounded, and
* free of control characters other than tab and newline. */
export function parseMemoryContent(bytes: Buffer): string | null {
if (bytes.byteLength > CONTENT_MAX_CHARACTERS * 4 + 2) return null;
let decoded: string;
try {
decoded = new TextDecoder("utf-8", { fatal: true }).decode(bytes);
} catch {
return null;
}
const content = decoded.replace(/\r?\n$/, "");
return content.trim().length > 0 &&
[...content].length <= CONTENT_MAX_CHARACTERS &&
![...content].some((character) => {
const code = character.charCodeAt(0);
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
})
? content
: null;
}
export interface MemoryQuery {
query?: string;
status?: string;
+39 -16
View File
@@ -184,8 +184,16 @@ const CORE = new Map<string, { status: number; data: unknown }>([
},
],
[
"/api/v1/home-summary",
{ status: 200, data: { memory_count: 42, latest: null, observed_at: "T" } },
"/api/v2/home-summary",
{
status: 200,
data: {
schema_version: 2,
memory_count: 42,
latest: null,
observed_at: "T",
},
},
],
]);
@@ -194,9 +202,7 @@ const READ_ONLY_PATHS = [
"/api/v1/live-sync",
"/api/v1/memory-processing",
"/api/v1/model-profiles",
"/api/v1/home-summary",
"/api/v1/memory-library",
"/api/v1/memories/",
"/api/v2/home-summary",
"/api/v2/memory-library",
"/api/v2/memories/",
"/api/v2/user-profile",
@@ -816,7 +822,7 @@ describe("CLI-21 memory reads", () => {
expect(report.history).toEqual([]);
});
// Core registers /api/v1/memories/graph before /{memory_id}, and that static
// Core registers /api/v2/memories/graph before /{memory_id}, and that static
// handler initializes the HY bridge. A memory id that collides with a
// reserved segment must never become a request.
it("refuses a memory id that collides with a static Core route", async () => {
@@ -830,18 +836,15 @@ describe("CLI-21 memory reads", () => {
expect(streams.stdout()).toBe("");
}
expect(transport.get).not.toHaveBeenCalled();
expect(isReadOnlyPath("/api/v1/memories/graph")).toBe(false);
expect(isReadOnlyPath("/api/v1/memories/ask")).toBe(false);
expect(isReadOnlyPath("/api/v1/memories/m1")).toBe(true);
expect(isReadOnlyPath("/api/v2/memory-library")).toBe(true);
expect(isReadOnlyPath("/api/v2/memories/graph")).toBe(false);
expect(isReadOnlyPath("/api/v2/memories/ask")).toBe(false);
expect(isReadOnlyPath("/api/v2/memories/m1")).toBe(true);
expect(isReadOnlyPath("/api/v2/memory-library")).toBe(true);
expect(isReadOnlyPath("/api/v2/home-summary")).toBe(true);
// The retired v1 data plane is no longer a CLI read.
expect(isReadOnlyPath("/api/v1/memories/m1")).toBe(false);
expect(isReadOnlyPath("/api/v1/home-summary")).toBe(false);
expect(isReadOnlyPath("/api/v1/model-content-permission")).toBe(false);
expect(
isReadOnlyPath(
"/api/v1/memory-mutations/123e4567-e89b-12d3-a456-426614174000",
),
).toBe(true);
expect(isReadOnlyPath("/api/v1/memory-mutations/../sources")).toBe(false);
});
it("refuses a memory id that would escape the memories path", async () => {
@@ -861,6 +864,26 @@ describe("CLI-21 memory reads", () => {
// was cast to the expected type, so a wrong record, an unavailable excerpt, an
// oversized page and a non-string model field all reached stdout as success.
describe("malformed success responses", () => {
it("treats a v1-shaped home summary as invalid, not as a memory count", async () => {
transport.get.mockImplementation(async (url: string) =>
url === "/api/v2/home-summary"
? {
status: 200,
data: { memory_count: 7, latest: null, observed_at: "T" },
}
: (CORE.get(url) ?? { status: 404, data: { detail: "not_found" } }),
);
const streams = captureStreams();
try {
await run(["status", "--json"], {});
} finally {
streams.restore();
}
expect(JSON.parse(streams.stdout()).sections.memories).toMatchObject({
state: "invalid",
});
});
it("refuses a detail that is not the memory that was asked for", async () => {
transport.get.mockResolvedValue({
status: 200,
@@ -315,6 +315,40 @@ describe("macOS service lifecycle", () => {
expect(deps.verifyRegistrationSnapshot).toHaveBeenCalledTimes(2);
});
it("does not re-bootstrap a previous registration that may serve a legacy home", async () => {
const f = fixture();
const oldPath = "/Users/example/Library/LaunchAgents/old-runtime.plist";
const runCommand = vi
.fn()
.mockResolvedValueOnce(undefined)
.mockRejectedValueOnce(new Error("bootstrap failed"));
const deps = dependencies({
inspectRegistration: vi.fn(async () => ({
kind: "managed" as const,
plistPath: oldPath,
content: "old managed plist",
})),
runCommand,
});
const prepared = await prepareServiceInstall(f.layout, f.options, deps);
await expect(
applyPreparedServiceInstall(
{ ...prepared, restorePreviousOnFailure: false },
deps,
),
).rejects.toThrow("runtime_service_replacement_failed");
// bootout + the failed bootstrap only; the old plist is not bootstrapped.
expect(runCommand).toHaveBeenCalledTimes(2);
expect(runCommand).not.toHaveBeenCalledWith([
"launchctl",
"bootstrap",
"gui/501",
oldPath,
]);
});
it("removes the replacement plist and restores the previous registration", async () => {
const f = fixture();
const oldPath = "/Users/example/Library/LaunchAgents/old-runtime.plist";
+16 -1
View File
@@ -72,6 +72,13 @@ export type PreparedServiceInstall = {
mode: "direct" | "bootstrap-existing" | "restart" | "replace";
previous?: ManagedSnapshot;
previouslyEnabled: boolean;
/**
* Re-bootstrap the previous (possibly older) LaunchAgent when the new one
* fails to bootstrap. `install` turns this off once the home has been handed
* to this version's SDK migration: an older Runtime has no migration gate and
* would serve a home that is not on SDK authority.
*/
restorePreviousOnFailure?: boolean;
};
const defaultDependencies: ServiceLifecycleDependencies = {
@@ -191,7 +198,8 @@ export async function applyPreparedServiceInstall(
if (!previous) throw new Error("runtime_service_upgrade_state_invalid");
await verifyPreparedRegistration(prepared, dependencies);
const domain = launchdDomain(options.uid);
let replacementFailure: "upgrade" | "rollback" | null = null;
let replacementFailure: "upgrade" | "rollback" | "not-restored" | null =
null;
const replaceCommand: ServiceCommandRunner = async (command) => {
if (command[0] !== "launchctl" || command[1] !== "bootstrap") {
await dependencies.runCommand(command);
@@ -208,6 +216,10 @@ export async function applyPreparedServiceInstall(
await dependencies.runCommand(command);
} catch {
replacementFailure = "upgrade";
if (prepared.restorePreviousOnFailure === false) {
replacementFailure = "not-restored";
throw new Error("runtime_service_replacement_failed");
}
try {
await dependencies.verifyRegistrationSnapshot(previous);
await dependencies.runCommand([
@@ -234,6 +246,9 @@ export async function applyPreparedServiceInstall(
if (replacementFailure === "upgrade") {
throw new Error("runtime_service_managed_upgrade_failed");
}
if (replacementFailure === "not-restored") {
throw new Error("runtime_service_replacement_failed");
}
throw error;
}
} catch (error) {
+4 -35
View File
@@ -7,12 +7,10 @@ import type {
} from "@one-memory/contracts";
import type { CoreReader, ReadOutcome } from "./core-client.js";
import {
type ValidatedMemoryTask,
validateImportIssues,
validateImportJob,
validateImportJobs,
validateLiveSync,
validateMemoryTasks,
validateProcessing,
validateSources,
} from "./core-responses.js";
@@ -70,11 +68,6 @@ export interface ProgressDocument {
generation: {
summary_state: ReadState;
summary: Record<string, unknown> | null;
tasks_state: ReadState;
tasks: Array<Record<string, unknown>>;
returned: number;
validated: number;
output_truncated: boolean;
};
}
@@ -134,12 +127,13 @@ export async function readProgress(
const importsPath = options.jobId
? `/api/v1/import-jobs/${options.jobId}`
: "/api/v1/import-jobs";
const [importsRead, sourcesRead, processingRead, tasksRead, issuesRead] =
// Generation progress comes from `memory-processing` alone: under SDK
// authority the per-task list (`/api/v1/memory-tasks`) is retired and empty.
const [importsRead, sourcesRead, processingRead, issuesRead] =
await Promise.all([
reader<unknown>(importsPath),
reader<unknown>("/api/v1/sources"),
reader<unknown>("/api/v1/memory-processing"),
reader<unknown>("/api/v1/memory-tasks"),
options.jobId && options.issues
? reader<unknown>(`/api/v1/import-jobs/${options.jobId}/issues`)
: Promise.resolve(null),
@@ -153,7 +147,6 @@ export async function readProgress(
: validated(importsRead, validateImportJobs);
const processing = validated(processingRead, validateProcessing);
const sources = validated(sourcesRead, validateSources);
const tasks = validated(tasksRead, validateMemoryTasks);
const issues =
options.jobId && options.issues && issuesRead
? validated(issuesRead, (value) =>
@@ -162,25 +155,19 @@ export async function readProgress(
: null;
const jobs = imports.state === "ok" ? imports.data : [];
const taskItems = tasks.state === "ok" ? tasks.data : [];
const issueItems = issues?.state === "ok" ? issues.data : [];
const importState = stateOnly(imports);
const summaryState = stateOnly(processing);
const tasksState = stateOnly(tasks);
const hasReadFailure =
importState.state !== "ok" ||
sources.state !== "ok" ||
summaryState.state !== "ok" ||
tasksState.state !== "ok" ||
(issues !== null && issues.state !== "ok");
const hasProblem =
jobs.some((job) =>
["paused", "interrupted", "failed"].includes(job.status),
) ||
taskItems.some((task) =>
["waiting_for_model", "failed"].includes(task.status),
) ||
(processing.state === "ok" &&
(processing.data.failed > 0 ||
processing.data.waiting_for_model > 0 ||
@@ -188,7 +175,6 @@ export async function readProgress(
issueItems.length > 0;
const hasActive =
jobs.some((job) => ["pending", "running"].includes(job.status)) ||
taskItems.some((task) => ["pending", "running"].includes(task.status)) ||
(processing.state === "ok" &&
(processing.data.pending > 0 || processing.data.running > 0));
@@ -229,11 +215,6 @@ export async function readProgress(
summary_state: summaryState,
summary:
processing.state === "ok" ? projectProcessing(processing.data) : null,
tasks_state: tasksState,
tasks: taskItems.slice(0, options.limit).map(projectTask),
returned: Math.min(taskItems.length, options.limit),
validated: taskItems.length,
output_truncated: taskItems.length > options.limit,
},
};
}
@@ -289,10 +270,7 @@ export function renderProgressText(document: ProgressDocument): string {
? `记忆整理:待处理 ${document.generation.summary.pending} · 进行中 ${document.generation.summary.running} · 等待模型 ${document.generation.summary.waiting_for_model} · 失败 ${document.generation.summary.failed}`
: "记忆整理:暂时无法读取",
);
if (
document.imports.output_truncated ||
document.generation.output_truncated
) {
if (document.imports.output_truncated) {
lines.push("这里只显示汇总;完整记录请使用 --json。");
}
lines.push("");
@@ -353,11 +331,6 @@ export function unreadProgress(
generation: {
summary_state: failed,
summary: null,
tasks_state: failed,
tasks: [],
returned: 0,
validated: 0,
output_truncated: false,
},
};
}
@@ -465,7 +438,3 @@ function projectProcessing(
})),
};
}
function projectTask(task: ValidatedMemoryTask): Record<string, unknown> {
return { ...task };
}
+6 -48
View File
@@ -129,18 +129,6 @@ const RESPONSES = new Map<string, { status: number; data: unknown }>([
},
},
],
[
"/api/v1/memory-tasks",
{
status: 200,
data: [
memoryTask("task-pending", "pending"),
memoryTask("task-running", "running"),
memoryTask("task-model", "waiting_for_model", "model_missing"),
memoryTask("task-failed", "failed", "provider_unavailable"),
],
},
],
[
"/api/v1/import-jobs/job-running",
{ status: 200, data: importJob("job-running", "running") },
@@ -282,9 +270,9 @@ describe("CLI-19 progress and issue reads", () => {
imported_messages: 20,
totals_state: { state: "ok" },
});
expect(
report.generation.tasks.map((task: { status: string }) => task.status),
).toEqual(["pending", "running", "waiting_for_model", "failed"]);
// The retired per-task list is no longer read or reported.
expect(report.generation).not.toHaveProperty("tasks");
expect(requestedPaths()).not.toContain("/api/v1/memory-tasks");
expect(streams.stdout()).not.toMatch(/percent|percentage|%/i);
});
@@ -374,7 +362,7 @@ describe("CLI-19 progress and issue reads", () => {
expect(streams.stdout()).not.toContain("SYNTHETIC_FOREIGN_ISSUE");
});
it("bounds each task list and emits an explicit truncation receipt", async () => {
it("bounds the import job list and emits an explicit truncation receipt", async () => {
const streams = captureStreams();
try {
expect(await run(["progress", "--limit", "1", "--json"], {})).toBe(1);
@@ -388,12 +376,6 @@ describe("CLI-19 progress and issue reads", () => {
validated: 3,
output_truncated: true,
});
expect(report.generation.tasks).toHaveLength(1);
expect(report.generation).toMatchObject({
returned: 1,
validated: 4,
output_truncated: true,
});
});
it("streams bounded progress snapshots as serial NDJSON", async () => {
@@ -428,7 +410,6 @@ describe("CLI-19 progress and issue reads", () => {
"/api/v1/import-jobs",
"/api/v1/sources",
"/api/v1/memory-processing",
"/api/v1/memory-tasks",
]) {
expect(requestedPaths().filter((item) => item === path)).toHaveLength(2);
}
@@ -500,7 +481,7 @@ describe("CLI-19 progress and issue reads", () => {
it("preserves denied, unavailable and malformed sections in one partial report", async () => {
transport.get.mockImplementation(async (url: string) => {
if (url === "/api/v1/import-jobs") return { status: 503, data: {} };
if (url === "/api/v1/memory-tasks") return { status: 403, data: {} };
if (url === "/api/v1/sources") return { status: 403, data: {} };
if (url === "/api/v1/memory-processing") {
return { status: 200, data: { pending: { secret: "nested" } } };
}
@@ -521,7 +502,7 @@ describe("CLI-19 progress and issue reads", () => {
state: "invalid",
code: "response_invalid",
});
expect(report.generation.tasks_state).toEqual({
expect(report.imports.totals_state).toEqual({
state: "denied",
code: "http_403",
});
@@ -574,7 +555,6 @@ describe("CLI-19 progress and issue reads", () => {
"/api/v1/live-sync",
"/api/v1/import-jobs",
"/api/v1/memory-processing",
"/api/v1/memory-tasks",
].some((allowed) => path === allowed || path.startsWith(`${allowed}/`)),
),
).toBe(true);
@@ -634,28 +614,6 @@ function importJob(
};
}
function memoryTask(
taskId: string,
status: string,
error: string | null = null,
) {
return {
task_id: taskId,
request_id: `request-${taskId}`,
user_id: "local",
kind: "extract",
input_ref: "synthetic",
status,
profile_id: "profile-1",
profile_revision: 1,
profile_snapshot: {},
attempts: status === "failed" ? 2 : 0,
last_error_code: error,
created_at: "T1",
updated_at: "T2",
};
}
function requestedPaths(): string[] {
return (transport.get.mock.calls as [string][]).map(([path]) => path);
}
+43 -1
View File
@@ -1,6 +1,11 @@
import { describe, expect, it } from "vitest";
import { renderStatusText, type StatusDocument } from "./status-report.js";
import { composeDoctorReport, renderDoctorText } from "./doctor-report.js";
import {
composeStatusReport,
renderStatusText,
type StatusDocument,
} from "./status-report.js";
describe("status default output", () => {
it("formats blocked task objects recursively instead of printing object coercions", () => {
@@ -35,3 +40,40 @@ describe("status default output", () => {
);
});
});
describe("status and doctor on an unfinished upgrade migration", () => {
it("names the migration log and the issues page", async () => {
const report = await composeStatusReport({
dataDir: "/tmp/isolated/.one-memory",
runtime: { state: "stopped", code: "sdk_migration_required" },
background: null,
reader: null,
});
expect(report.runtime.log_file).toBe(
"/tmp/isolated/.one-memory/logs/sdk-migration.log",
);
expect(report.runtime.issues_url).toBe(
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
);
const text = renderStatusText(report, "MemHub");
expect(text).toContain("sdk_migration_required");
expect(text).toContain("/tmp/isolated/.one-memory/logs/sdk-migration.log");
const doctor = renderDoctorText(composeDoctorReport(report, null));
expect(doctor).toContain(
"/tmp/isolated/.one-memory/logs/sdk-migration.log",
);
expect(doctor).not.toContain("帮助与诊断");
});
it("adds no log path for an ordinary stopped runtime", async () => {
const report = await composeStatusReport({
dataDir: "/tmp/isolated/.one-memory",
runtime: { state: "stopped", code: "runtime_not_running" },
background: null,
reader: null,
});
expect(report.runtime.log_file).toBeUndefined();
});
});
+23 -3
View File
@@ -1,5 +1,4 @@
import type {
HomeSummary,
LiveSyncHealth,
MemoryProcessingSummary,
ModelProfile,
@@ -9,8 +8,10 @@ import type {
BackgroundHealth,
RuntimeStatus,
} from "@one-memory/runtime-control";
import { sdkMigrationLogPath } from "@one-memory/runtime-control";
import type { CoreReader, ReadOutcome } from "./core-client.js";
import {
type SdkHomeSummary,
validateHomeSummary,
validateLiveSync,
validateModelProfiles,
@@ -19,6 +20,9 @@ import {
} from "./core-responses.js";
import { PRODUCT } from "./version.js";
const MIGRATION_ISSUES_URL =
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new";
export type SectionState =
| "ok"
| "denied"
@@ -42,6 +46,9 @@ export interface StatusDocument {
host?: string;
port?: number;
version?: string;
/** Set only when an unfinished upgrade migration is why nothing serves. */
log_file?: string;
issues_url?: string;
};
bridge: {
state: BackgroundHealth["state"] | "not_checked";
@@ -102,7 +109,7 @@ export async function composeStatusReport(options: {
reader<unknown>("/api/v1/live-sync"),
reader<unknown>("/api/v1/memory-processing"),
reader<unknown>("/api/v1/model-profiles"),
reader<unknown>("/api/v1/home-summary"),
reader<unknown>("/api/v2/home-summary"),
]);
sections.sources = section(sources, validateSources, describeSources);
sections.sync = section(sync, validateLiveSync, describeSync);
@@ -153,6 +160,12 @@ export async function composeStatusReport(options: {
host: options.runtime.host,
port: options.runtime.port,
version: options.runtime.version,
...(options.runtime.code.startsWith("sdk_migration_")
? {
log_file: sdkMigrationLogPath(options.dataDir),
issues_url: MIGRATION_ISSUES_URL,
}
: {}),
},
bridge,
sections,
@@ -177,6 +190,13 @@ export function renderStatusText(
): string {
const lines = [
`${heading}:${runtimeLine(report)}`,
...(report.runtime.log_file
? [
`升级迁移没有完成(错误码:${report.runtime.code}),MemHub 不会启动。`,
`请把迁移日志附在 GitHub 反馈里发给我们:${report.runtime.log_file}`,
`反馈地址:${report.runtime.issues_url}`,
]
: []),
`资料空间:${report.data_dir}`,
`应用连接:${BRIDGE_LABELS[report.bridge.state]}`,
];
@@ -347,7 +367,7 @@ function describeModel(profiles: ModelProfile[]): Record<string, unknown> {
};
}
function describeMemories(summary: HomeSummary): Record<string, unknown> {
function describeMemories(summary: SdkHomeSummary): Record<string, unknown> {
return {
memory_count: summary.memory_count,
};
File diff suppressed because it is too large. Load diff
@@ -15,7 +15,7 @@ from one_memory.call_event_store import (
CallTool,
HostId,
)
from one_memory.context_models import ContextSearchResponse, MemoryContextItem, RawContextItem
from one_memory.context_models import SdkContextSearchResponse
from one_memory.external_memory_access import ExternalMemoryAccess
from one_memory.recall import RecallReceipt
@@ -41,75 +41,27 @@ def _host_id(access: ExternalMemoryAccess | None) -> HostId | None:
return cast(HostId, host) if host in HOSTS else None
def context_call_event(
result: ContextSearchResponse,
def sdk_context_call_event(
result: SdkContextSearchResponse,
access: ExternalMemoryAccess | None,
*,
memory_searched: bool,
timed_out: bool,
) -> CallEvent:
references: list[CallReference] = []
seen: set[tuple[str, str, int]] = set()
for item in result.items:
if isinstance(item, RawContextItem):
candidates = [
CallReference(
kind="raw",
id=message.evidence_id,
revision=message.revision,
source_id=item.source_id,
)
for message in item.messages
]
elif isinstance(item, MemoryContextItem):
candidates = [
CallReference(
kind="memory", id=item.memory_id, revision=item.revision, source_id=None
)
]
else:
continue
for reference in candidates:
identity = (reference.kind, reference.id, reference.revision)
if identity not in seen:
seen.add(identity)
references.append(reference)
timed_out = any(
path.reason == "timeout_budget" for path in (result.paths.raw, result.paths.memory)
# SDK items carry no Evidence/Governance revision, and the reference
# resolver reads the legacy store, so the event records counts only.
raw_count = sum(item.kind == "raw" for item in result.items)
memory_count = sum(item.kind == "memory" for item in result.items)
state: CallState = "timed_out" if timed_out else result.state
return _metadata_event(
tool="om_recall" if memory_searched else "om_conversations",
state=state,
elapsed_ms=result.total_ms,
access=access,
raw_count=raw_count,
memory_count=memory_count,
event_id=UUID(result.request_id),
)
state: CallState = (
"partial" if timed_out and references else "timed_out" if timed_out else result.state
)
return CallEvent.model_validate(
{
"event_id": UUID(result.request_id),
"occurred_at": datetime.now(UTC),
"host_id": _host_id(access),
"installation_id": UUID(access.installation_id) if access else None,
"tool": _context_tool(result),
"state": state,
"stage": "core_result",
"delivery": "unknown",
"adoption": "unknown",
"host_session_id": None,
"injection_elapsed_ms": None,
"elapsed_ms": result.total_ms,
"raw_count": sum(isinstance(item, RawContextItem) for item in result.items),
"memory_count": sum(isinstance(item, MemoryContextItem) for item in result.items),
"references": tuple(references),
}
)
def record_context_result(
store: CallEventStore,
result: ContextSearchResponse,
access: ExternalMemoryAccess | None,
) -> None:
store.record(context_call_event(result, access))
def _context_tool(result: ContextSearchResponse) -> CallTool:
if result.paths.memory.status not in {"not_requested", "not_authorized"}:
return "om_recall"
return "om_conversations"
def _metadata_event(
@@ -142,26 +94,6 @@ def _metadata_event(
)
def conversation_call_event(
result: object,
access: ExternalMemoryAccess | None,
elapsed_ms: float,
) -> CallEvent:
hits = tuple(getattr(result, "hits", ()) or ())
conversations = {
conversation_id
for hit in hits
if (conversation_id := getattr(hit, "conversation_id", None))
}
return _metadata_event(
tool="om_conversations",
state="grounded" if hits else "empty",
elapsed_ms=elapsed_ms,
access=access,
raw_count=min(len(conversations), 20),
)
def ask_call_event(
result: object,
access: ExternalMemoryAccess | None,
@@ -255,8 +187,9 @@ def record_legacy_result(
host_session_id=None,
injection_elapsed_ms=None,
elapsed_ms=elapsed_ms,
raw_count=0,
memory_count=len(receipt.hits),
# A raw-fallback recall returned conversation excerpts, not memories.
raw_count=len(receipt.hits) if receipt.mode == "raw" else 0,
memory_count=0 if receipt.mode == "raw" else len(receipt.hits),
references=(),
)
)
+253 -84
View File
@@ -1,6 +1,5 @@
import os
import signal
import socket
import sys
# A handed-off capture ends itself: the hook that gave it the prompt has
@@ -29,10 +28,13 @@ def _end_by(seconds: str | None) -> None:
if __name__ == "__main__" and sys.argv[1:2] == ["capture"]:
_end_by(os.environ.get("ONE_MEMORY_CAPTURE_DEADLINE_S"))
import contextlib # noqa: E402
import errno # noqa: E402
import json # noqa: E402
import re # noqa: E402
import time # noqa: E402
from collections.abc import Iterator, Sequence # noqa: E402
from pathlib import Path # noqa: E402
from typing import TextIO # noqa: E402
from one_memory.config import load_settings # noqa: E402
@@ -68,6 +70,9 @@ def migration_failure_code(
return "sdk_migration_disk_space_insufficient"
for item in chain:
code = getattr(item, "code", "")
# `sdk_data_plane_write_job_input_missing:<job_id>`: the name is the
# code; the id is for the log, not for the dialog.
code = code.split(":", 1)[0] if isinstance(code, str) else code
if not isinstance(code, str) or not _SAFE_MIGRATION_CODE.match(code):
continue
if code.startswith(("sdk_data_plane_", "sdk_migration_")):
@@ -103,12 +108,8 @@ def run(arguments: Sequence[str], *, output: TextIO | None = None) -> int:
if not arguments or list(arguments) in (
["serve"],
["serve", "--maintenance"],
["serve", "--reader"],
):
serve(
maintenance=list(arguments) == ["serve", "--maintenance"],
reader=list(arguments) == ["serve", "--reader"],
)
serve(maintenance=list(arguments) == ["serve", "--maintenance"])
return 0
command, *options = arguments
if command == "restore":
@@ -117,29 +118,17 @@ def run(arguments: Sequence[str], *, output: TextIO | None = None) -> int:
return sdk_migrate(options, stream)
if command == "sdk-migration-status":
return sdk_migration_status(options, stream)
if command == "boundary" and not options:
from one_memory.workbuddy_boundary import write_hook
return write_hook(stream, sys.stdin)
if command == "boundary" and options == ["--peek"]:
from one_memory.workbuddy_boundary import write_peek
return write_peek(stream, sys.stdin)
if command == "boundary" and len(options) == 2 and options[0] == "--ack":
from one_memory.workbuddy_boundary import run_confirm
return run_confirm(options[1])
if command == "capture" and len(options) == 1:
from one_memory.live_capture import capture_stdio
return capture_stdio(options[0], sys.stdin, stream)
if command not in {"status", "doctor"} or options not in ([], ["--json"]):
stream.write(
"Usage: one-memory [serve [--maintenance|--reader]|status [--json]|doctor [--json]"
"Usage: one-memory [serve [--maintenance]|status [--json]|doctor [--json]"
"|restore --source <managed-backup> --yes [--json]"
"|sdk-migrate --runtime-build <build> --yes [--json]"
"|sdk-migration-status [--json]"
"|boundary [--peek|--ack TOKEN]|capture SOURCE]\n"
"|capture SOURCE]\n"
)
return 2
@@ -312,7 +301,22 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
stream.write("one-memory: sdk-migrate 需要 --runtime-build <build>\n")
return 2
from one_memory.sdk_migration_log import authority_snapshot, write_migration_log
settings = load_settings()
database_path = settings.data_dir / "one-memory.sqlite3"
started = time.monotonic()
write_migration_log(
settings.data_dir,
writer="core-sdk-migrate",
event="migrate.invoked",
outcome="started",
details={
"runtime_build": runtime_build,
"confirmed": confirmed,
**authority_snapshot(database_path),
},
)
status = inspect_runtime(settings.data_dir)
if status.state != "stopped":
document = {
@@ -321,6 +325,14 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
"code": "sdk_migration_runtime_running",
"runtime_state": status.state,
}
write_migration_log(
settings.data_dir,
writer="core-sdk-migrate",
event="migrate.blocked",
outcome="blocked",
code="sdk_migration_runtime_running",
details={"runtime_state": status.state},
)
_write_restore(stream, as_json, document, "Runtime 仍在运行;不能迁移。")
return 4
if not confirmed:
@@ -328,7 +340,13 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
_write_restore(stream, as_json, document, "预览通过;加 --yes 后执行迁移。")
return 4
from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE, HY_MEMORY_VERSION
from one_memory.hy_runtime_config import (
HY_MEMORY_NAMESPACE,
HY_MEMORY_VERSION,
configure_hy_runtime_environment,
)
configure_hy_runtime_environment(settings.data_dir)
from one_memory.sdk_migration_runner import run_startup_migration
step_names = {
@@ -340,23 +358,43 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
"cleanup": "正在删除旧表……",
}
# Everything before the first progress step (authority read, profile
# read, resumable identity, legacy id scan, model-profile check) is the
# preflight. A failure is logged with the phase it happened in.
phase = {"name": "preflight", "started": time.monotonic()}
def progress(step: str) -> None:
now = time.monotonic()
write_migration_log(
settings.data_dir,
writer="core-sdk-migrate",
event="phase.finished",
outcome="succeeded",
details={"phase": phase["name"], "duration_ms": (now - phase["started"]) * 1000},
)
phase["name"], phase["started"] = step, now
write_migration_log(
settings.data_dir,
writer="core-sdk-migrate",
event="phase.started",
outcome="started",
details={"phase": step},
)
if as_json:
stream.write(
json.dumps({"schema_version": 1, "state": "running", "step": step}) + "\n"
)
stream.write(json.dumps({"schema_version": 1, "state": "running", "step": step}) + "\n")
else:
stream.write(f"{step_names.get(step, step)}\n")
stream.flush()
try:
result = run_startup_migration(
data_dir=settings.data_dir,
runtime_build=runtime_build,
sdk_version=HY_MEMORY_VERSION,
namespace=HY_MEMORY_NAMESPACE,
progress=progress,
)
with _migration_lock(settings.data_dir):
result = run_startup_migration(
data_dir=settings.data_dir,
runtime_build=runtime_build,
sdk_version=HY_MEMORY_VERSION,
namespace=HY_MEMORY_NAMESPACE,
progress=progress,
)
except Exception as error:
from one_memory.sdk_migration_verify import MemoryReadbackFailed
@@ -364,19 +402,143 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
failed: dict[str, object] = {"schema_version": 1, "state": "failed", "code": code}
if isinstance(error, MemoryReadbackFailed):
failed["readback"] = error.report.as_document()
write_migration_log(
settings.data_dir,
writer="core-sdk-migrate",
event="migrate.failed",
outcome="failed",
code=code,
error=error,
details={
"phase": phase["name"],
"phase_duration_ms": (time.monotonic() - phase["started"]) * 1000,
"duration_ms": (time.monotonic() - started) * 1000,
# A generic fallback code means the cause is only in `error`.
"code_is_fallback": code == "sdk_migration_preflight_failed"
and not _names_code(error, code),
**({"readback": failed["readback"]} if "readback" in failed else {}),
**authority_snapshot(database_path),
},
)
_write_restore(stream, as_json, failed, f"迁移未完成:{code}")
return 1
document = result.as_document()
write_migration_log(
settings.data_dir,
writer="core-sdk-migrate",
event="migrate.finished",
outcome="succeeded",
migration_id=result.migration_id,
details={
"state": result.state,
"last_phase": phase["name"],
"duration_ms": (time.monotonic() - started) * 1000,
"result": document,
**authority_snapshot(database_path),
},
)
messages = {
"not_required": "不需要迁移。",
"complete": "迁移已完成;遗留的旧表或向量已处理。",
"cutover": "迁移完成:记忆数据已以 SDK 为准。",
}
_write_restore(stream, as_json, result.as_document(), messages[result.state])
_write_restore(stream, as_json, document, messages[result.state])
return 0
@contextlib.contextmanager
def _migration_lock(data_dir: Path) -> Iterator[None]:
"""One `sdk-migrate` per home at a time.
Desktop, CLI install/upgrade and the Runtime service each migrate before
serving, and a login-started service can be migrating when the person
opens the app. The second waits here and then finds the work done (or
resumes it); two runs never interleave.
"""
from one_memory.sdk_migration_log import write_migration_log
data_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
descriptor = os.open(data_dir / "sdk-migration.lock", os.O_RDWR | os.O_CREAT, 0o600)
try:
if os.name == "nt":
import msvcrt
def acquire(blocking: bool) -> None:
while True:
try:
os.lseek(descriptor, 0, os.SEEK_SET)
msvcrt.locking(descriptor, msvcrt.LK_NBLCK, 1)
return
except OSError:
if not blocking:
raise
time.sleep(0.5)
def release() -> None:
os.lseek(descriptor, 0, os.SEEK_SET)
msvcrt.locking(descriptor, msvcrt.LK_UNLCK, 1)
else:
import fcntl
def acquire(blocking: bool) -> None:
fcntl.flock(descriptor, fcntl.LOCK_EX | (0 if blocking else fcntl.LOCK_NB))
def release() -> None:
fcntl.flock(descriptor, fcntl.LOCK_UN)
try:
acquire(False)
except OSError:
write_migration_log(
data_dir,
writer="core-sdk-migrate",
event="migrate.waiting_for_lock",
outcome="started",
)
waited = time.monotonic()
acquire(True)
write_migration_log(
data_dir,
writer="core-sdk-migrate",
event="migrate.lock_acquired",
outcome="succeeded",
details={"waited_ms": (time.monotonic() - waited) * 1000},
)
try:
yield
finally:
release()
finally:
os.close(descriptor)
def _names_code(error: BaseException, code: str) -> bool:
return any(
str(getattr(item, "code", "")).split(":", 1)[0] == code for item in _exception_chain(error)
)
def sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
"""Read-only probe: does this home need `sdk-migrate`?"""
try:
return _sdk_migration_status(options, stream)
except Exception as error:
from one_memory.sdk_migration_log import write_migration_log
# The orchestrator only sees a non-zero exit and reports
# `sdk_migration_status_unavailable`; the cause is recorded here.
write_migration_log(
load_settings().data_dir,
writer="core-sdk-migration-status",
event="status.failed",
outcome="failed",
code="sdk_migration_status_unavailable",
error=error,
)
raise
def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
if list(options) not in ([], ["--json"]):
stream.write("one-memory: sdk-migration-status 只接受 --json\n")
return 2
@@ -409,15 +571,21 @@ def sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
data_dir=settings.data_dir,
namespace=HY_MEMORY_NAMESPACE,
)
or (
profile is not None
and legacy_vector_collections(settings.data_dir, profile)
)
or (profile is not None and legacy_vector_collections(settings.data_dir, profile))
)
state = "required" if leftovers else "not_required"
else:
state = "required"
document = {"schema_version": 1, "authority": authority.value, "state": state}
from one_memory.sdk_migration_log import write_migration_log
write_migration_log(
settings.data_dir,
writer="core-sdk-migration-status",
event="status.probed",
outcome="succeeded",
details={"authority": document["authority"], "state": document["state"]},
)
if as_json:
stream.write(f"{json.dumps(document, ensure_ascii=False)}\n")
else:
@@ -437,30 +605,58 @@ def _status_text(command: str, status: RuntimeStatus) -> str:
return f"MemHub Runtime 状态不可验证:{status.code}。"
def _reserve_reader_port(host: str, port: int) -> int:
"""Resolve the port the Reader will listen on, including the ephemeral case.
#: Exit status of a serving Core refused because the home is not on SDK
#: authority (`EX_CONFIG`). The Runtime maps it to `sdk_migration_required`.
SDK_MIGRATION_REQUIRED_EXIT = 78
The supervisor asks for an ephemeral port (`ONE_MEMORY_READER_PORT=0`) and
then reads the real one back from stderr. Letting uvicorn bind port 0
itself would give a port nobody can report, so claim a free port here by
binding and immediately releasing it, then serve on that exact port.
def _refuse_unmigrated_home(data_dir: Path) -> None:
"""Exit before binding when an existing home has not reached SDK authority.
A brand-new home has no database yet and is created on SDK authority by
startup. Anything else that is not SDK is an upgrade that did not finish,
and is never served: the migration must run first.
"""
if port != 0:
return port
probe = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
from one_memory.sdk_data_plane_migration import MemoryAuthority, SdkDataPlaneMigrationStore
from one_memory.sdk_migration_log import authority_snapshot, write_migration_log
database_path = data_dir / "one-memory.sqlite3"
if not database_path.is_file():
return
try:
probe.bind((host, 0))
return int(probe.getsockname()[1])
finally:
probe.close()
authority = SdkDataPlaneMigrationStore(database_path).authority()
except Exception as error: # noqa: BLE001 - an unreadable home is not served either
write_migration_log(
data_dir,
writer="core-startup",
event="core.refused_legacy",
outcome="refused",
code="sdk_migration_required",
error=error,
)
print("sdk_migration_required", file=sys.stderr, flush=True)
raise SystemExit(SDK_MIGRATION_REQUIRED_EXIT) from None
if authority is MemoryAuthority.SDK:
return
write_migration_log(
data_dir,
writer="core-startup",
event="core.refused_legacy",
outcome="refused",
code="sdk_migration_required",
details={"stage": "serve_entry", **authority_snapshot(database_path)},
)
print("sdk_migration_required", file=sys.stderr, flush=True)
raise SystemExit(SDK_MIGRATION_REQUIRED_EXIT)
def serve(*, maintenance: bool = False, reader: bool = False) -> None:
def serve(*, maintenance: bool = False) -> None:
import uvicorn
from one_memory.diagnostic_log import configure_diagnostics, diagnostic_event
from one_memory.hy_runtime_config import (
apply_diagnostic_log_level,
configure_hy_runtime_environment,
diagnostic_log_level,
)
from one_memory.log_file import start_logging_to_file
@@ -468,40 +664,13 @@ def serve(*, maintenance: bool = False, reader: bool = False) -> None:
from one_memory.runtime import CoreRuntime, parent_pid
settings = load_settings()
if reader:
settings = settings.model_copy(
update={
"port": int(os.environ.get("ONE_MEMORY_READER_PORT", "0") or "0"),
"api_token": settings.api_token,
}
)
from one_memory.reader_app import ReaderSettings, create_reader_app
core_port = int(os.environ.get("ONE_MEMORY_CORE_PORT", "8765"))
# The supervisor learns the Reader's port by watching stderr for
# `reader_port:<n>`. With `ONE_MEMORY_READER_PORT=0` uvicorn would bind
# an OS-chosen port that nothing reports back, so claim the port up
# front and serve on that exact one.
port = _reserve_reader_port(settings.host, settings.port)
reader_settings = ReaderSettings(
data_dir=settings.data_dir,
host=settings.host,
port=port,
api_token=settings.api_token,
core_port=core_port,
core_api_token=os.environ.get("ONE_MEMORY_CORE_API_TOKEN", ""),
)
application = create_reader_app(reader_settings)
print(f"reader_port:{port}", file=sys.stderr, flush=True)
uvicorn.run(
application,
host=settings.host,
port=port,
reload=False,
access_log=False,
)
return
if not maintenance:
_refuse_unmigrated_home(settings.data_dir)
# HY resolves some trace paths at import time and its pipeline path when
# SharedRuntime starts. Bind both to the MemHub root before importing the
# application graph; otherwise HY falls back to ~/.hy_memory while the
# desktop Debug panel watches the managed data directory.
configure_hy_runtime_environment(settings.data_dir)
from one_memory.api import create_app
start_logging_to_file(settings.data_dir)
@@ -47,6 +47,19 @@ class ContextSearchRequest(BaseModel):
return self
class SdkContextSearchRequest(ContextSearchRequest):
"""`POST /api/v2/context/search`: the three SDK recall lanes are set apart.
`limit` is the normal lane's top-k and the cap on returned items;
`schema_limit` and `intention_limit` are the SDK's schema (L6) and
proactive intention (L7) lanes. A caller's values are passed through.
"""
limit: int = Field(default=10, ge=1, le=20, strict=True)
schema_limit: int = Field(default=3, ge=0, le=20, strict=True)
intention_limit: int = Field(default=1, ge=0, le=20, strict=True)
class ContextMessage(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
+35 -409
View File
@@ -1,7 +1,6 @@
"""Authenticated composite read route, without generation or reindex side effects."""
"""Authenticated SDK context search route, without generation or reindex side effects."""
import asyncio
import sqlite3
import time
from collections.abc import Sequence
from datetime import UTC, datetime
@@ -11,36 +10,15 @@ from uuid import uuid4
from fastapi import APIRouter, HTTPException, Request, Response
from fastapi.params import Depends
from one_memory.background_work import (
in_interactive_read,
in_read_pool,
memory_read_pool,
raw_read_pool,
)
from one_memory.call_event_recorder import context_call_event
from one_memory.call_event_recorder import sdk_context_call_event
from one_memory.context_access import resolve_context_authority
from one_memory.context_memory import (
merge_memory_candidates,
read_memory_context,
revalidate_memory_context,
)
from one_memory.context_models import (
ContextItem,
ContextKind,
ContextLaneResult,
ContextSearchRequest,
ContextSearchResponse,
ContextSemanticStatus,
MemoryContextItem,
RawContextItem,
SdkContextItem,
SdkContextSearchRequest,
SdkContextSearchResponse,
SdkMemoryContextItem,
SdkRawContextItem,
)
from one_memory.context_raw import read_raw_context, revalidate_raw_context
from one_memory.context_search import search_context
from one_memory.diagnostic_log import diagnostic_event, diagnostic_exception
from one_memory.external_memory_access import external_access
from one_memory.hy_runtime_config import HY_MEMORY_READER
from one_memory.hy_sdk_runtime import sdk_relevance_floor
@@ -50,387 +28,8 @@ from one_memory.hy_sdk_types import (
parse_sdk_search,
)
from one_memory.imports.storage import EvidenceStore
from one_memory.semantic_deadline import hy_wait_budget, search_within
CALL_EVENT_RESPONSE_BUDGET_MS = 250
RESPONSE_SERIALIZATION_RESERVE_MS = 100
INTERACTIVE_TOTAL_BUDGET_MS = 900
RAW_LANE_BUDGET_SECONDS = 0.3
MEMORY_LANE_BUDGET_SECONDS = 0.75
MEMORY_FINALIZE_RESERVE_SECONDS = 0.1
def context_router(dependencies: Sequence[Depends]) -> APIRouter:
router = APIRouter(prefix="/api/v1/context", dependencies=dependencies)
@router.post("/search", response_model=ContextSearchResponse)
async def search(
value: ContextSearchRequest, request: Request, response: Response
) -> ContextSearchResponse:
response.headers["Cache-Control"] = "no-store"
started = time.monotonic()
search_budget_ms = min(
INTERACTIVE_TOTAL_BUDGET_MS,
max(50, value.timeout_ms - RESPONSE_SERIALIZATION_RESERVE_MS),
)
deadline = started + search_budget_ms / 1000
access = external_access(request.headers)
try:
database = cast(EvidenceStore, request.app.state.evidence_store).database_path
authority = await asyncio.wait_for(
in_interactive_read(
resolve_context_authority, value, access, database, deadline=deadline
),
timeout=max(0, deadline - time.monotonic()),
)
except HTTPException:
raise
except Exception:
# Authority could not be established; do not try an unbounded local lane.
raise HTTPException(503, "context_authority_unavailable") from None
lane_observations: dict[ContextKind, ContextLaneResult] = {}
lane_failures: dict[ContextKind, tuple[str, str]] = {}
stage_timings: dict[str, float] = {}
async def raw(end: float) -> ContextLaneResult:
lane_end = min(end, time.monotonic() + RAW_LANE_BUDGET_SECONDS)
read = asyncio.create_task(
in_read_pool(
raw_read_pool(),
read_raw_context,
database,
value,
source_ids=authority.raw_sources,
deadline=lane_end,
)
)
done, _ = await asyncio.wait({read}, timeout=max(0, lane_end - time.monotonic()))
if read not in done:
read.cancel()
read.add_done_callback(
lambda task: task.exception() if not task.cancelled() else None
)
result = ContextLaneResult(
status="degraded",
reason="timeout_budget",
searched_channels=("lexical", "semantic"),
timed_out=True,
truncated_by=("timeout_budget",),
)
else:
result = read.result()
lane_observations["raw"] = result
return result
async def memory(end: float) -> ContextLaneResult:
lane_end = min(end, time.monotonic() + MEMORY_LANE_BUDGET_SECONDS)
target = getattr(request.app.state, "recall_target", None)
lexical = asyncio.create_task(
in_read_pool(
memory_read_pool(),
read_memory_context,
database,
value,
scopes=authority.memory_scopes,
deadline=lane_end,
semantic_ids=(),
semantic_outcome="absent",
)
)
semantic_hits, outcome = await search_within(
getattr(target, "semantic_search", None),
value.query,
hy_wait_budget(lane_end - time.monotonic()),
)
done, _ = await asyncio.wait({lexical}, timeout=max(0, lane_end - time.monotonic()))
if lexical not in done:
lexical.cancel()
lexical.add_done_callback(
lambda task: task.exception() if not task.cancelled() else None
)
result = ContextLaneResult(
status="unavailable",
reason="timeout_budget",
searched_channels=("lexical", "semantic"),
semantic_status=cast(
ContextSemanticStatus,
{
"absent": "not_searched",
"answered": "completed",
"failed": "failed",
"timed_out": "timed_out",
}[outcome],
),
timed_out=True,
truncated_by=("timeout_budget",),
)
lane_observations["memory"] = result
return result
try:
lexical_result = lexical.result()
except (sqlite3.Error, OSError, ValueError) as error:
diagnostic_event(
level="warn",
component="context.search",
event="context.lexical_memory_failed",
outcome="degraded",
error_code="lexical_memory_read_failed",
**diagnostic_exception(error),
)
lexical_result = ContextLaneResult(
status="unavailable",
reason="read_unavailable",
searched_channels=("lexical",),
failure_site="memory_lexical",
error_class=type(error).__name__,
)
if outcome == "absent":
result = lexical_result
elif outcome in {"failed", "timed_out"}:
truncated_by = list(lexical_result.truncated_by)
has_lexical_memory = bool(lexical_result.items)
if (
outcome == "timed_out"
and not has_lexical_memory
and "timeout_budget" not in truncated_by
):
truncated_by.append("timeout_budget")
result = lexical_result.model_copy(
update={
"status": "degraded",
"reason": "lexical_only" if has_lexical_memory else "timeout_budget",
"truncated_by": tuple(truncated_by),
"searched_channels": ("lexical", "semantic"),
"semantic_status": "timed_out" if outcome == "timed_out" else "failed",
"timed_out": outcome == "timed_out" and not has_lexical_memory,
}
)
else:
semantic_request = value.model_copy(update={"query": ""})
enriched = asyncio.create_task(
in_interactive_read(
read_memory_context,
database,
semantic_request,
scopes=authority.memory_scopes,
deadline=lane_end,
semantic_hits=semantic_hits,
semantic_outcome=outcome,
)
)
done, _ = await asyncio.wait(
{enriched},
timeout=max(
0,
lane_end - MEMORY_FINALIZE_RESERVE_SECONDS - time.monotonic(),
),
)
if enriched not in done:
enriched.cancel()
enriched.add_done_callback(
lambda task: task.exception() if not task.cancelled() else None
)
result = lexical_result.model_copy(
update={
"status": "degraded",
"reason": "lexical_only",
"searched_channels": ("lexical", "semantic"),
"semantic_status": "completed",
"timed_out": False,
}
)
else:
try:
semantic_result = enriched.result()
except (sqlite3.Error, OSError, ValueError) as error:
diagnostic_event(
level="warn",
component="context.search",
event="context.semantic_enrichment_failed",
outcome="degraded",
error_code="semantic_enrichment_read_failed",
**diagnostic_exception(error),
)
result = lexical_result.model_copy(
update={
"status": "degraded",
"reason": "lexical_only",
"searched_channels": ("lexical", "semantic"),
"semantic_status": "completed",
"timed_out": False,
}
)
lane_observations["memory"] = result
return result
if semantic_result.status == "unavailable" and lexical_result.items:
result = lexical_result.model_copy(
update={
"status": "degraded",
"reason": "lexical_only",
"searched_channels": ("lexical", "semantic"),
"semantic_status": "completed",
"timed_out": False,
}
)
lane_observations["memory"] = result
return result
semantic_items = tuple(
item
for item in semantic_result.items
if isinstance(item, MemoryContextItem)
)
lexical_items = tuple(
item for item in lexical_result.items if isinstance(item, MemoryContextItem)
)
combined_count = len(
{item.memory_id for item in (*semantic_items, *lexical_items)}
)
merged = merge_memory_candidates(
lexical_items,
semantic_items,
limit=value.limit + 1,
)
truncated_by = list(semantic_result.truncated_by)
if combined_count > len(merged) and "count_budget" not in truncated_by:
truncated_by.append("count_budget")
result = semantic_result.model_copy(
update={
"items": tuple(merged),
# An answered semantic search may legitimately
# return no IDs. Lexical can still have a grounded
# hit; retaining semantic's `empty` status while
# attaching that hit violates the lane contract
# and makes the aggregator reject the whole lane.
"status": "ok" if merged else semantic_result.status,
"truncated_by": tuple(truncated_by),
}
)
lane_observations["memory"] = result
return result
async def finalizer(
kind: ContextKind, items: tuple[ContextItem, ...], end: float
) -> tuple[ContextItem, ...]:
if kind == "raw":
return await in_interactive_read(
revalidate_raw_context,
database,
tuple(item for item in items if isinstance(item, RawContextItem)),
source_ids=authority.raw_sources,
deadline=end,
)
return await in_interactive_read(
revalidate_memory_context,
database,
tuple(item for item in items if isinstance(item, MemoryContextItem)),
scopes=authority.memory_scopes,
deadline=end,
)
result = await search_context(
value,
raw_reader=raw if authority.raw_allowed else None,
memory_reader=memory if authority.memory_allowed else None,
finalizer=finalizer,
deadline=deadline,
timing_observer=stage_timings.update,
failure_observer=lambda kind, site, error_class: lane_failures.__setitem__(
kind, (site, error_class)
),
)
final = result.model_copy(update={"total_ms": (time.monotonic() - started) * 1000})
serialization_started = time.monotonic()
final.model_dump_json()
stage_timings["serialization_ms"] = (time.monotonic() - serialization_started) * 1000
for phase, duration in stage_timings.items():
diagnostic_event(
level="info",
component="context.search",
event="context_stage.completed",
outcome="succeeded",
request_id=final.request_id,
duration_ms=max(0, round(duration)),
details={"phase": phase},
)
for kind in ("raw", "memory"):
path = getattr(final.paths, kind)
observed = lane_observations.get(kind)
failure = lane_failures.get(kind)
lexical_hits = (
sum(
1
for item in observed.items
if isinstance(item, MemoryContextItem) and "lexical" in item.matched_by
)
if observed is not None and kind == "memory"
else 0
)
semantic_hits = (
sum(
1
for item in observed.items
if isinstance(item, MemoryContextItem)
and "governed_semantic" in item.matched_by
)
if observed is not None and kind == "memory"
else 0
)
timed_out = path.reason == "timeout_budget" or (
observed.timed_out if observed is not None else False
)
diagnostic_failure_site = failure[0] if failure is not None else None
diagnostic_error_class = failure[1] if failure is not None else None
if observed is not None:
diagnostic_failure_site = diagnostic_failure_site or observed.failure_site
diagnostic_error_class = diagnostic_error_class or observed.error_class
diagnostic_event(
level="warn" if path.status in {"degraded", "unavailable"} else "info",
component="context.search",
event="context_lane.completed",
outcome=path.status,
request_id=final.request_id,
duration_ms=round(path.elapsed_ms),
details={
"operation_kind": kind,
"mode": ",".join(observed.searched_channels) if observed else "none",
"status": path.status,
"index_complete": observed.index_complete if observed else None,
**(
{"lexical_hits": lexical_hits, "semantic_hits": semantic_hits}
if kind == "memory"
else {}
),
"reason": path.reason,
"truncated": path.truncated,
"timed_out": timed_out,
**(
{
"failure_site": diagnostic_failure_site,
"error_class": diagnostic_error_class,
}
if diagnostic_failure_site is not None
and diagnostic_error_class is not None
else {}
),
},
)
store = getattr(request.app.state, "call_event_store", None)
if store is not None and value.query != "__om_probe__":
event = context_call_event(final, access)
try:
await asyncio.to_thread(
store.record,
event,
busy_timeout_ms=CALL_EVENT_RESPONSE_BUDGET_MS,
)
response.headers["X-One-Memory-Call-Recorded"] = "true"
except Exception:
response.headers["X-One-Memory-Call-Recorded"] = "false"
request.app.state.call_event_recording_error = True
return final
return router
def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
@@ -439,7 +38,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
@router.post("/search", response_model=SdkContextSearchResponse)
async def search(
value: ContextSearchRequest,
value: SdkContextSearchRequest,
request: Request,
response: Response,
) -> SdkContextSearchResponse:
@@ -514,8 +113,8 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
limit=value.limit,
min_score=sdk_relevance_floor(),
schema_min_score=sdk_relevance_floor(),
schema_limit=value.limit,
intention_limit=value.limit,
schema_limit=value.schema_limit,
intention_limit=value.intention_limit,
scene="normal",
reader=HY_MEMORY_READER,
)
@@ -549,6 +148,29 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
)
return tuple(items)
async def record(final: SdkContextSearchResponse, *, timed_out: bool) -> None:
# Same boundary as v2 ask: only a host's call is history. The
# renderer's own searches are the person reading their library.
store = getattr(request.app.state, "call_event_store", None)
if store is None or access is None or value.query == "__om_probe__":
return
event = sdk_context_call_event(
final,
access,
memory_searched=authority.memory_allowed,
timed_out=timed_out,
)
try:
await asyncio.to_thread(
store.record,
event,
busy_timeout_ms=CALL_EVENT_RESPONSE_BUDGET_MS,
)
response.headers["X-One-Memory-Call-Recorded"] = "true"
except Exception:
response.headers["X-One-Memory-Call-Recorded"] = "false"
request.app.state.call_event_recording_error = True
lanes = (raw_lane(), memory_lane())
try:
raw_items, memory_items = await asyncio.wait_for(
@@ -556,7 +178,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
timeout=value.timeout_ms / 1000,
)
except TimeoutError:
return SdkContextSearchResponse(
final = SdkContextSearchResponse(
request_id=str(uuid4()),
state="unavailable",
items=(),
@@ -564,6 +186,8 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
truncated=True,
total_ms=(time.monotonic() - started) * 1000,
)
await record(final, timed_out=True)
return final
partial = isinstance(raw_items, Exception) or isinstance(memory_items, Exception)
raw_candidates = (
() if isinstance(raw_items, Exception) else cast(tuple[SdkContextItem, ...], raw_items)
@@ -595,7 +219,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
truncated = True
selected.append(item)
used += len(item.content)
return SdkContextSearchResponse(
final = SdkContextSearchResponse(
request_id=str(uuid4()),
state=("partial" if partial else "grounded" if selected else "empty"),
items=tuple(selected),
@@ -603,5 +227,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
truncated=truncated,
total_ms=(time.monotonic() - started) * 1000,
)
await record(final, timed_out=False)
return final
return router
@@ -1,464 +0,0 @@
"""Bounded two-lane orchestration; readers and finalizers own trusted read access."""
import asyncio
import time
from collections.abc import Awaitable, Callable, Mapping
from dataclasses import dataclass, field
from uuid import uuid4
from pydantic import TypeAdapter
from one_memory.context_models import (
ContextItem,
ContextKind,
ContextLaneResult,
ContextLimit,
ContextPath,
ContextPaths,
ContextSearchRequest,
ContextSearchResponse,
MemoryContextItem,
RawContextItem,
)
Reader = Callable[[float], Awaitable[ContextLaneResult]]
Finalizer = Callable[
[ContextKind, tuple[ContextItem, ...], float], Awaitable[tuple[ContextItem, ...]]
]
TimingObserver = Callable[[Mapping[str, float]], None]
FailureObserver = Callable[[ContextKind, str, str], None]
_KINDS: tuple[ContextKind, ...] = ("raw", "memory")
_ITEMS = TypeAdapter(tuple[ContextItem, ...])
@dataclass
class _Lane:
result: ContextLaneResult
elapsed_ms: float = 0
limits: list[ContextLimit] = field(default_factory=list)
def _identity(item: ContextItem) -> tuple[str, ...]:
if isinstance(item, RawContextItem):
return (item.kind, item.source_id, item.conversation_id, item.anchor_evidence_id)
return (item.kind, item.memory_id)
def _validate_items(kind: ContextKind, items: tuple[ContextItem, ...]) -> None:
if len(items) > 21 or any(item.kind != kind for item in items):
raise ValueError("invalid lane items")
for item in items:
if isinstance(item, RawContextItem) and item.anchor_evidence_id not in {
message.evidence_id for message in item.messages
}:
raise ValueError("missing raw anchor")
async def _read(reader: Reader, kind: ContextKind, deadline: float) -> ContextLaneResult:
value = await reader(deadline)
# Revalidate even BaseModel instances: model_construct/model_copy bypass validation.
result = ContextLaneResult.model_validate(value.model_dump())
_validate_items(kind, result.items)
if result.status not in ("ok", "empty", "degraded", "unavailable"):
raise ValueError("reader cannot decide requested authority")
if result.status in ("empty", "unavailable") and result.items:
raise ValueError("non-result lane has items")
return result
async def _finalize(
finalizer: Finalizer, kind: ContextKind, items: tuple[ContextItem, ...], deadline: float
) -> tuple[ContextItem, ...]:
value = await finalizer(kind, items, deadline)
if not isinstance(value, tuple):
raise ValueError("invalid finalization result")
checked = _ITEMS.validate_python([item.model_dump() for item in value])
_validate_items(kind, checked)
if not {_identity(item) for item in checked} <= {_identity(item) for item in items}:
raise ValueError("finalizer introduced an unselected identity")
return checked
async def _collect(
jobs: dict[ContextKind, Awaitable[object]], deadline: float
) -> tuple[
dict[ContextKind, object],
dict[ContextKind, BaseException],
set[ContextKind],
dict[ContextKind, float],
]:
"""Cancel and await every owned task on deadline, failure or caller cancellation."""
durations: dict[ContextKind, float] = {}
async def measured(kind: ContextKind, job: Awaitable[object]) -> object:
started = time.monotonic()
try:
value = await job
if time.monotonic() >= deadline:
raise TimeoutError
return value
finally:
durations[kind] = (time.monotonic() - started) * 1000
tasks = {kind: asyncio.create_task(measured(kind, job)) for kind, job in jobs.items()}
values: dict[ContextKind, object] = {}
failed: dict[ContextKind, BaseException] = {}
timed_out: set[ContextKind] = set()
try:
if not tasks:
return values, failed, timed_out, durations
done, pending = await asyncio.wait(
tasks.values(), timeout=max(0, deadline - time.monotonic())
)
for kind, task in tasks.items():
if task in pending:
timed_out.add(kind)
elif task in done:
if not task.cancelled() and isinstance(task.exception(), TimeoutError):
timed_out.add(kind)
elif task.cancelled() or task.exception() is not None:
failed[kind] = task.exception() or asyncio.CancelledError()
else:
values[kind] = task.result()
return values, failed, timed_out, durations
finally:
for task in tasks.values():
if not task.done():
task.cancel()
if tasks:
# Cooperative readers finish immediately and still get their
# normal cleanup. A bridge that suppresses cancellation must not
# hold a host response past its hard deadline while an underlying
# SDK thread winds down.
cleanup: asyncio.Future[list[object]] = asyncio.gather(
*tasks.values(), return_exceptions=True
)
current = asyncio.current_task()
if current is not None and current.cancelling():
await cleanup
else:
try:
await asyncio.wait_for(asyncio.shield(cleanup), timeout=0.05)
except TimeoutError:
cleanup.add_done_callback(_consume_cleanup)
def _consume_cleanup(task: asyncio.Future[list[object]]) -> None:
if not task.cancelled():
task.exception()
def _deduplicate(
items: tuple[ContextItem, ...], limits: list[ContextLimit]
) -> tuple[ContextItem, ...]:
unique: dict[tuple[str, ...], ContextItem] = {}
for item in items:
key = _identity(item)
previous = unique.get(key)
if isinstance(previous, MemoryContextItem) and isinstance(item, MemoryContextItem):
if item.revision != previous.revision:
# Never graft older evidence onto a newer memory version.
unique[key] = item if item.revision > previous.revision else previous
continue
evidence = tuple(dict.fromkeys((*previous.evidence_ids, *item.evidence_ids)))
if len(evidence) > 128:
limits.append("metadata_budget")
unique[key] = previous.model_copy(
update={
"evidence_ids": evidence[:128],
"truncated": previous.truncated or item.truncated or len(evidence) > 128,
}
)
elif isinstance(previous, RawContextItem) and isinstance(item, RawContextItem):
messages = {message.evidence_id: message for message in previous.messages}
for message in item.messages:
if (
message.evidence_id not in messages
or message.revision > messages[message.evidence_id].revision
):
messages[message.evidence_id] = message
if len(messages) > 3:
limits.append("metadata_budget")
# The first window already contains the anchor; dict order preserves it.
unique[key] = previous.model_copy(
update={
"messages": tuple(messages.values())[:3],
"truncated": previous.truncated or item.truncated or len(messages) > 3,
}
)
else:
unique[key] = item
# Reserve each raw anchor for its own window before deduplicating context messages.
anchors = {
(item.source_id, item.conversation_id, item.anchor_evidence_id)
for item in unique.values()
if isinstance(item, RawContextItem)
}
seen: set[tuple[str, ...]] = set()
result: list[ContextItem] = []
for item in unique.values():
if isinstance(item, MemoryContextItem):
result.append(
item.model_copy(update={"evidence_ids": tuple(dict.fromkeys(item.evidence_ids))})
)
continue
retained_messages = []
for message in item.messages:
key = (item.source_id, item.conversation_id, message.evidence_id)
if key in seen or (key in anchors and message.evidence_id != item.anchor_evidence_id):
continue
seen.add(key)
retained_messages.append(message)
result.append(item.model_copy(update={"messages": tuple(retained_messages)}))
return tuple(result)
def _characters(item: ContextItem) -> int:
if isinstance(item, MemoryContextItem):
return len(item.content)
return sum(len(message.text) for message in item.messages)
def _shares(lengths: list[int], budget: int) -> list[int]:
"""Water-fill allocations so a large first item cannot consume every character."""
allocations = [0] * len(lengths)
pending = list(range(len(lengths)))
while pending and budget:
share, remainder = divmod(budget, len(pending))
next_pending = []
for position, index in enumerate(pending):
added = min(lengths[index] - allocations[index], share + (position < remainder))
allocations[index] += added
budget -= added
if allocations[index] < lengths[index]:
next_pending.append(index)
pending = next_pending
return allocations
def _clip(item: ContextItem, budget: int) -> ContextItem:
if _characters(item) <= budget:
return item
if isinstance(item, MemoryContextItem):
return item.model_copy(update={"content": item.content[:budget], "truncated": True})
# The first raw message is the lexical anchor. Give it the first share of
# the budget, then water-fill the adjacent context; otherwise a long
# neighbor can consume the whole window and hide the answer.
anchor = item.messages[0]
anchor_budget = min(len(anchor.text), budget)
remaining = budget - anchor_budget
adjacent = list(item.messages[1:])
shares = [anchor_budget, *_shares([len(message.text) for message in adjacent], remaining)]
return item.model_copy(
update={
"truncated": True,
"messages": tuple(
message.model_copy(
update={
"text": message.text[:share],
"truncated": message.truncated or len(message.text) > share,
}
)
for message, share in zip(item.messages, shares, strict=True)
),
}
)
async def search_context(
request: ContextSearchRequest,
*,
raw_reader: Reader | None,
memory_reader: Reader | None,
finalizer: Finalizer,
deadline: float | None = None,
timing_observer: TimingObserver | None = None,
failure_observer: FailureObserver | None = None,
) -> ContextSearchResponse:
request = ContextSearchRequest.model_validate(request.model_dump())
started = time.monotonic()
deadline = min(
deadline if deadline is not None else float("inf"), started + request.timeout_ms / 1000
)
lanes: dict[ContextKind, _Lane] = {}
jobs: dict[ContextKind, Awaitable[object]] = {}
readers: dict[ContextKind, Reader | None] = {"raw": raw_reader, "memory": memory_reader}
for kind, reader in readers.items():
if kind not in request.kinds:
lanes[kind] = _Lane(ContextLaneResult(status="not_requested", reason="not_requested"))
elif reader is None:
lanes[kind] = _Lane(
ContextLaneResult(status="not_authorized", reason="permission_required")
)
elif time.monotonic() >= deadline:
lanes[kind] = _Lane(
ContextLaneResult(status="degraded", reason="timeout_budget", timed_out=True),
limits=["timeout_budget"],
)
else:
jobs[kind] = _read(reader, kind, deadline)
values, failed, timed_out, read_ms = await _collect(jobs, deadline)
for kind in jobs:
result = values.get(kind)
if kind in timed_out:
result = ContextLaneResult(
status="degraded",
reason="timeout_budget",
truncated_by=("timeout_budget",),
timed_out=True,
)
elif kind in failed or not isinstance(result, ContextLaneResult):
error = failed.get(kind)
if failure_observer is not None:
failure_observer(
kind,
"collect",
type(error).__name__ if error is not None else "InvalidLaneResult",
)
result = ContextLaneResult(
status="unavailable",
reason="read_unavailable",
failure_site="collect",
error_class=type(error).__name__ if error is not None else "InvalidLaneResult",
)
lanes[kind] = _Lane(result, read_ms.get(kind, 0), list(dict.fromkeys(result.truncated_by)))
final_jobs: dict[ContextKind, Awaitable[object]] = {}
for kind in jobs:
lane = lanes[kind]
if lane.result.status not in ("ok", "empty", "degraded"):
continue
if time.monotonic() >= deadline:
lane.result = ContextLaneResult(
status="degraded", reason="timeout_budget", timed_out=True
)
lane.limits.append("timeout_budget")
else:
final_jobs[kind] = _finalize(finalizer, kind, lane.result.items, deadline)
final_values, final_failed, final_timed_out, revalidation_ms = await _collect(
final_jobs, deadline
)
for kind in final_jobs:
lane = lanes[kind]
lane.elapsed_ms += revalidation_ms.get(kind, 0)
if kind in final_timed_out:
lane.result = ContextLaneResult(
status="unavailable", reason="timeout_budget", timed_out=True
)
lane.limits.append("timeout_budget")
elif kind in final_failed:
error = final_failed[kind]
if failure_observer is not None:
failure_observer(kind, "finalize", type(error).__name__)
lane.result = ContextLaneResult(
status="unavailable",
reason="read_unavailable",
failure_site="finalize",
error_class=type(error).__name__,
)
else:
final_items = _ITEMS.validate_python(final_values[kind])
changed = {_identity(item): item for item in lane.result.items} != {
_identity(item): item for item in final_items
}
lane.result = lane.result.model_copy(
update={
"items": final_items,
"status": "degraded" if changed else lane.result.status,
"reason": ("evidence_changed" if kind == "raw" else "memory_changed")
if changed
else lane.result.reason,
}
)
# Nothing previously verified may escape once the common release deadline expired.
if time.monotonic() >= deadline:
for kind in jobs:
if lanes[kind].result.status in ("ok", "empty", "degraded"):
lanes[kind].result = ContextLaneResult(
status="degraded", reason="timeout_budget", timed_out=True
)
lanes[kind].limits.append("timeout_budget")
fusion_started = time.monotonic()
queues = {kind: _deduplicate(lanes[kind].result.items, lanes[kind].limits) for kind in _KINDS}
interleaved = [
queues[kind][index]
for index in range(max(map(len, queues.values()), default=0))
for kind in _KINDS
if index < len(queues[kind])
]
selected = interleaved[: request.limit]
for item in interleaved[request.limit :]:
lanes[item.kind].limits.append("count_budget")
shares = _shares([_characters(item) for item in selected], request.character_budget)
items = tuple(_clip(item, share) for item, share in zip(selected, shares, strict=True))
for before, after in zip(selected, items, strict=True):
if _characters(before) != _characters(after):
lanes[after.kind].limits.append("character_budget")
if time.monotonic() >= deadline:
items = ()
for kind in jobs:
if lanes[kind].result.status in ("ok", "empty", "degraded"):
lanes[kind].result = ContextLaneResult(
status="degraded", reason="timeout_budget", timed_out=True
)
lanes[kind].limits.append("timeout_budget")
lanes[kind].elapsed_ms = (time.monotonic() - started) * 1000
paths: dict[ContextKind, ContextPath] = {}
for kind in _KINDS:
lane = lanes[kind]
emitted = [item for item in items if item.kind == kind]
limits = tuple(dict.fromkeys(lane.limits))
truncated = bool(limits) or any(item.truncated for item in emitted)
status = lane.result.status
benign_limits = set(limits) <= {"count_budget", "character_budget"}
benign_degradation = (
not lane.result.timed_out
and benign_limits
and (
lane.result.reason is None
or (kind == "raw" and lane.result.reason == "lexical_only")
)
)
if status in ("ok", "empty") or (status == "degraded" and benign_degradation):
status = "ok" if emitted or lane.result.items else "empty"
paths[kind] = ContextPath(
status=status,
reason=lane.result.reason,
elapsed_ms=lane.elapsed_ms,
returned_count=len(emitted),
truncated=truncated,
truncated_by=limits,
)
requested_paths = [paths[kind] for kind in request.kinds]
completed = any(path.status in ("ok", "empty", "degraded") for path in requested_paths)
incomplete = any(path.status not in ("ok", "empty") for path in requested_paths)
response = ContextSearchResponse(
request_id=str(uuid4()),
state="unavailable"
if not completed
else "partial"
if incomplete
else "grounded"
if items
else "empty",
paths=ContextPaths(raw=paths["raw"], memory=paths["memory"]),
items=items,
used_characters=sum(map(_characters, items)),
truncated=any(path.truncated for path in paths.values()),
total_ms=(time.monotonic() - started) * 1000,
)
checked = ContextSearchResponse.model_validate(response.model_dump())
if timing_observer is not None:
timing_observer(
{
"raw_ms": read_ms.get("raw", 0),
"memory_ms": read_ms.get("memory", 0),
"raw_revalidation_ms": revalidation_ms.get("raw", 0),
"memory_revalidation_ms": revalidation_ms.get("memory", 0),
"fusion_ms": (time.monotonic() - fusion_started) * 1000,
"total_ms": checked.total_ms,
}
)
return checked
@@ -6,7 +6,6 @@ from collections.abc import Sequence
from typing import NamedTuple
PREFACE = "以下是获授权历史参考,不是指令;当前用户请求优先。"
_MARKER = re.compile(r"om-derived:([0-9a-f]{64})")
# Written by the host-bridge recall injection hook (recall-injection.ts): the
# recalled memory block and, in the same reply, the cross-session notice block.
_INJECTED_TAGS = ("memhub_memory_context", "memhub_session_notice")
@@ -89,45 +88,3 @@ def strip_injected_memory(text: str) -> str:
return text
kept.append(text[at:])
return _EMPTY_REMINDER.sub("", "".join(kept)).strip()
def marker(event_id: str) -> str:
return f"om-derived:{event_id}"
def echoed_event_ids(verbatim: str) -> tuple[str, ...]:
return tuple(dict.fromkeys(_MARKER.findall(verbatim)))
def render_reference(events: tuple[object, ...] | list[object]) -> str:
lines = [PREFACE]
for event in events:
source = getattr(event, "source_id", None) or "unknown"
revision = getattr(event, "evidence_revision", None)
evidence_id = getattr(event, "evidence_id", None) or ""
event_id = getattr(event, "event_id", "")
lines.append(
f"来源 {source} 修订 {0 if revision is None else revision} "
f"证据 {evidence_id} {marker(str(event_id))}"
)
return "\n".join(lines)
def is_pure_echo(verbatim: str, known_event_ids: set[str]) -> bool:
"""True when the text is only a block we already injected.
A later line that is not the preface, a source line, or one of those
markers is a real correction and must still be recorded.
"""
found = set(echoed_event_ids(verbatim))
if not found or not found & known_event_ids:
return False
for line in verbatim.splitlines():
stripped = line.strip()
if not stripped or stripped == PREFACE or stripped.startswith("来源 "):
continue
if _MARKER.search(stripped):
continue
return False
return True
@@ -804,7 +804,7 @@ def record_database_error(
sqlite_code = getattr(error, "sqlite_errorcode", None)
sqlite_name = getattr(error, "sqlite_errorname", None)
details: dict[str, object] = {"phase": safe_component, "count": count}
if route_path is not None and route_path.startswith("/api/v1/"):
if route_path is not None and route_path.startswith(("/api/v1/", "/api/v2/")):
details["route_path"] = route_path
if isinstance(sqlite_code, int):
details["sqlite_errorcode"] = sqlite_code
@@ -23,13 +23,9 @@ _ENVELOPE_HEADERS = frozenset(
)
_BROKER_ROUTES = {
("POST", "/api/v1/recall"): "om_recall",
("POST", "/api/v1/memories/ask"): "om_ask",
("POST", "/api/v2/memories/ask"): "om_ask",
("POST", "/api/v2/memories/ask/stream"): "om_ask",
("GET", "/api/v1/conversations/search"): "om_conversations",
("POST", "/api/v1/diagnostics/trace"): "om_diagnostic_trace",
("POST", "/api/v1/boundary/peek"): "om_recall",
("POST", "/api/v1/boundary/ack"): "om_recall",
}
_CONCRETE_CALL_EVENT_DELIVERY = re.compile(
r"/api/v1/call-events/[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-"
@@ -128,11 +124,9 @@ def require_external_endpoint(request: Request) -> None:
access = external_access(request.headers)
if access is None:
return
if request.method == "POST" and request.url.path in {
"/api/v1/context/search",
"/api/v2/context/search",
}:
# The route resolves each requested lane against its original grant.
if request.method == "POST" and request.url.path == "/api/v2/context/search":
# om_conversations reaches Core only here, as the raw lane; the route
# resolves each requested lane against its original grant.
# Passing this envelope gate does not authorize either lane by itself.
if access.tools.isdisjoint({"om_recall", "om_conversations"}):
raise HTTPException(403, "external_recall_scope_denied")
@@ -146,7 +140,3 @@ def require_external_endpoint(request: Request) -> None:
raise HTTPException(403, "external_endpoint_denied")
if tool not in access.tools:
raise HTTPException(403, "external_recall_scope_denied")
if tool == "om_conversations":
access.require(tool, frozenset({"all-apps"}))
if not any(scope.startswith("source:") for scope in access.scopes):
raise HTTPException(403, "external_source_permission_required")
@@ -1,127 +0,0 @@
"""Bounded homepage projection without loading every memory into the renderer."""
import sqlite3
import time
from datetime import UTC, datetime
from pathlib import Path
from pydantic import BaseModel, ConfigDict
from one_memory.database_access import main_database
from one_memory.memory_evidence_freshness import register_memory_freshness
class HomeLatestMemory(BaseModel):
model_config = ConfigDict(frozen=True)
memory_id: str
content: str
truncated: bool
revision: int
action: str
updated_at: datetime
source_ids: tuple[str, ...]
class HomeSummary(BaseModel):
model_config = ConfigDict(frozen=True)
memory_count: int
latest: HomeLatestMemory | None
observed_at: datetime
#: 2026-10-12: the read hit its deadline and the numbers below are not a
#: snapshot of reality. Callers should keep their previous summary on
#: screen instead of treating count=0 as "no memories".
stale: bool = False
class HomeSummaryUnavailable(Exception):
"""The homepage projection could not be read inside its own deadline.
Raised instead of letting SQLite's interrupt escape as a bare
`OperationalError`, so the route can answer with an empty projection
rather than a 500.
"""
def read_home_summary(database_path: Path, *, user_id: str = "local") -> HomeSummary:
# The exclusion mirrors SourceReadAccess: one revoked active citation hides
# the whole memory. Permission filtering precedes both count and LIMIT.
# `+link.active` keeps SQLite on the (memory_id, evidence_id) primary key.
# Without the unary plus it picks the (active, ...) index and walks every
# active link once per memory: 2.9 s instead of 0.07 s on a 1,810-memory
# library, enough to time the homepage out whenever the machine is busy.
readable = """
m.user_id = ? AND m.status = 'confirmed' AND m.deleted_at IS NULL
AND NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
JOIN source_grants g ON g.user_id=e.user_id AND g.source_id=e.source_id
WHERE link.memory_id=m.memory_id AND +link.active=1
AND g.revoked_at IS NOT NULL
)
AND NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
LEFT JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.memory_id=m.memory_id AND +link.active=1
AND (e.evidence_id IS NULL OR e.user_id != m.user_id)
)
"""
# The total counts with the cheap permission filters only: the per-row
# freshness UDF scales with the whole library and can burn the whole read
# budget on its own (D22). Product accepted a small drift here; the rows
# that are actually displayed are still checked row by row below.
with main_database(database_path).read() as db:
deadline = time.monotonic() + 5
db.set_progress_handler(lambda: int(time.monotonic() >= deadline), 1000)
db.row_factory = sqlite3.Row
try:
db.execute("BEGIN")
count = db.execute(
f"SELECT COUNT(*) FROM governed_memories m WHERE {readable}", (user_id,)
).fetchone()[0]
readable += " AND " + register_memory_freshness(db)
row = db.execute(
f"""SELECT m.memory_id, substr(m.content,1,320) AS content,
length(m.content)>320 AS truncated, m.revision,
COALESCE(v.action, 'unknown') AS action, m.updated_at
FROM governed_memories m
LEFT JOIN governed_memory_versions v
ON v.memory_id=m.memory_id AND v.revision=m.revision
WHERE {readable}
ORDER BY m.updated_at DESC, m.memory_id LIMIT 1""",
(user_id,),
).fetchone()
latest = None
if row is not None:
sources = db.execute(
"""SELECT DISTINCT e.source_id FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id=link.evidence_id
WHERE link.memory_id=? AND +link.active=1 ORDER BY e.source_id""",
(row["memory_id"],),
).fetchall()
latest = HomeLatestMemory(
memory_id=row["memory_id"],
content=row["content"],
truncated=bool(row["truncated"]),
revision=row["revision"],
action=row["action"],
updated_at=row["updated_at"],
source_ids=tuple(source[0] for source in sources),
)
except sqlite3.Error as error:
# The progress handler above is what interrupts this read, so a
# slow homepage surfaces as SQLITE_INTERRUPT (code 9) and used to
# escape as a 500. 2026-10-12: answer with a `stale` projection
# instead of raising: the caller keeps its last good summary on
# screen (the UI poll retries within 10s), and the homepage never
# zeroes out or errors just because the machine is busy importing.
if getattr(error, "sqlite_errorcode", None) != sqlite3.SQLITE_INTERRUPT:
raise
return HomeSummary(
memory_count=0,
latest=None,
observed_at=datetime.now(UTC),
stale=True,
)
return HomeSummary(memory_count=count, latest=latest, observed_at=datetime.now(UTC))
@@ -326,13 +326,9 @@ def _repair_vector(
directory = data_dir / "hy-memory" / HY_MEMORY_NAMESPACE / "vector"
if not (directory / "chroma.sqlite3").is_file():
return 0
import chromadb
from chromadb.config import Settings
from one_memory.sdk_vector_reattach import open_offline_chroma
client = chromadb.PersistentClient(
path=str(directory),
settings=Settings(anonymized_telemetry=False),
)
client = open_offline_chroma(directory)
# The SDK client is not open yet (startup has not created its handle), so
# this is the only writer; release the store before the SDK takes it.
try:
@@ -79,9 +79,9 @@ SDK_MIGRATION_CAPTURE_FENCE = ".sdk-migration-capture-fence.json"
def database_path() -> Path:
from one_memory.workbuddy_boundary import database_path as boundary_database
from one_memory.config import load_settings
return boundary_database()
return load_settings().data_dir / "one-memory.sqlite3"
def capture_stdio(source_id: str, stdin: TextIO, output: TextIO) -> int:
@@ -1,342 +0,0 @@
"""Read-only local library projection over the existing governed memory store."""
import json
import os
import sqlite3
import time
from pathlib import Path
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field
from one_memory.database_access import main_database
from one_memory.legacy_lifecycle_types import HyLifecycle
from one_memory.memory_evidence_freshness import register_memory_freshness
from one_memory.memory_governance import (
GovernedMemory,
MemoryGovernanceStore,
MemoryStatus,
lifecycle_predicate,
)
#: Read budget for one library page. 2026-10-12: raised from 5s (which starved
#: under a concurrent history import and surfaced as 503) to 15s, still well
#: under the 30s client timeout; past it the read degrades to a `stale` page.
READ_BUDGET_SECONDS = 15
class MemorySourceTime(BaseModel):
"""When the conversation a memory came from happened (D96).
`basis` is `conversation` when the cited messages carry their own time,
`import` when only the import time is known; the UI words them apart.
"""
model_config = ConfigDict(frozen=True, extra="forbid")
at: str
basis: Literal["conversation", "import"]
class MemoryLibraryPage(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
items: list[GovernedMemory]
total: int = Field(ge=0)
offset: int = Field(ge=0)
limit: int = Field(ge=1, le=500)
source_ids_by_memory: dict[str, list[str]] = Field(default_factory=dict)
source_time_by_memory: dict[str, MemorySourceTime] = Field(default_factory=dict)
#: True when the read budget ran out mid-page (typically during a history
#: import): the counts and rows above are a partial snapshot, not an
#: authoritative view. Clients should show "importing" instead of an error.
stale: bool = False
def _sort_by_conversation() -> bool:
"""D99: the list reads as a timeline of the conversations, newest first.
A memory generated today from a three-week-old dialogue belongs next to
that week, not at the top. ONE_MEMORY_LIBRARY_SORT=updated brings back the
old "newest generated first" order without a code change.
"""
return os.environ.get("ONE_MEMORY_LIBRARY_SORT", "").strip().lower() != "updated"
def _origin_join() -> str:
"""The newest cited message per memory, as one joined aggregate.
A correlated subquery would re-scan the link table for every candidate row;
this is a single pass. Memories with no readable citation keep their old
position through COALESCE.
"""
if not _sort_by_conversation():
return ""
return """LEFT JOIN (
SELECT link.memory_id AS memory_id,
MAX(COALESCE(e.created_at, e.imported_at)) AS origin_at
FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.active = 1
GROUP BY link.memory_id
) origin ON origin.memory_id = m.memory_id"""
def _order_by() -> str:
if not _sort_by_conversation():
return "m.updated_at DESC, m.memory_id ASC"
# Conversations that share a timestamp keep the generated order between
# themselves; ties would otherwise fall back to the id and look arbitrary.
return "COALESCE(origin.origin_at, m.updated_at) DESC, m.updated_at DESC, m.memory_id ASC"
class _LibraryQuery(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
user_id: str = Field(min_length=1, max_length=256)
query: str | None = None
status: MemoryStatus | None = None
active_only: bool = False
source: str | None = Field(default=None, min_length=1)
offset: int = Field(default=0, ge=0, le=2**63 - 1, strict=True)
limit: int = Field(default=100, ge=1, le=500, strict=True)
def read_conversation_memories(
database_path: Path,
*,
user_id: str,
source_id: str,
conversation_id: str,
workspace: str | None,
limit: int = 100,
) -> list[GovernedMemory]:
"""Current memories backed by any active Evidence in one conversation."""
with main_database(database_path).read() as db:
db.row_factory = sqlite3.Row
freshness = register_memory_freshness(db)
rows = db.execute(
f"""
SELECT DISTINCT m.* FROM governed_memories m
JOIN governed_memory_evidence link ON link.memory_id=m.memory_id AND link.active=1
JOIN evidence e ON e.evidence_id=link.evidence_id
WHERE m.user_id=? AND m.status='confirmed' AND m.deleted_at IS NULL
AND e.user_id=? AND e.source_id=? AND e.conversation_id=?
AND e.workspace IS ? AND {freshness}
ORDER BY m.updated_at DESC, m.memory_id
LIMIT ?
""",
(user_id, user_id, source_id, conversation_id, workspace, limit),
).fetchall()
return MemoryGovernanceStore.memories_from_rows(db, rows)
def read_memory_library(
database_path: Path,
*,
user_id: str = "local",
query: str | None = None,
status: MemoryStatus | None = None,
active_only: bool = False,
source: str | None = None,
offset: int = 0,
limit: int = 100,
lifecycle: HyLifecycle | None = None,
) -> MemoryLibraryPage:
"""Read one page and its exact readable total from the same SQLite snapshot.
This is the local UI policy, not an external agent authorization entry point.
Missing grants do not revoke already stored content; explicit revocations do.
Read/parse failures propagate rather than masquerading as an empty library.
Which memories are current is HY's `lifecycle` when given, as in
`MemoryGovernanceStore.list`.
"""
request = _LibraryQuery(
user_id=user_id,
query=query,
status=status,
active_only=active_only,
source=source,
offset=offset,
limit=limit,
)
if request.active_only and request.status is not None:
raise ValueError("active_only_conflicts_with_status")
clauses = ["m.user_id = ?"]
values: list[str | int] = [request.user_id]
selected, selected_values = lifecycle_predicate(
lifecycle, MemoryStatus.CONFIRMED if request.active_only else request.status, alias="m"
)
if selected is not None:
clauses.append(selected)
values.extend(selected_values)
if request.status != MemoryStatus.DELETED:
clauses.append("m.deleted_at IS NULL")
# SourceReadAccess hides the whole memory if any active citation is revoked.
# Do this before COUNT/LIMIT; a readable citation cannot rescue a mixed memory.
clauses.append("""NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
JOIN source_grants g ON g.user_id = e.user_id AND g.source_id = e.source_id
WHERE link.memory_id = m.memory_id AND +link.active = 1
AND g.revoked_at IS NOT NULL
)""")
# Governance normally guarantees ownership and referential integrity. Fail
# closed for broken active links instead of exposing another user's citation.
clauses.append("""NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
LEFT JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.memory_id = m.memory_id AND +link.active = 1
AND (e.evidence_id IS NULL OR e.user_id != m.user_id)
)""")
if request.query:
clauses.append("m.content LIKE ? ESCAPE '\\'")
escaped = request.query.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
values.append(f"%{escaped}%")
if request.source is not None:
# Start from the compact active-link index, then resolve each Evidence
# identity by primary key. The old correlated EXISTS let SQLite scan
# every Evidence row for every candidate memory on large libraries.
clauses.append("""m.memory_id IN (
SELECT link.memory_id FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.active = 1 AND e.user_id = ? AND e.source_id = ?
)""")
values.extend((request.user_id, request.source))
predicate = " AND ".join(clauses)
def _interrupted(error: Exception) -> bool:
return isinstance(error, sqlite3.OperationalError) and "interrupted" in str(error).lower()
with main_database(database_path).read() as db:
# 2026-10-12: 5s starved the page whenever a history import saturated
# SQLite, and the caller turned the deadline abort into a 503, so the
# memory page read as broken for the whole import. 15s stays well under
# the 30s client timeout; past it we return a `stale` page instead of
# an error — an incomplete list is honest, an error is not.
deadline = time.monotonic() + READ_BUDGET_SECONDS
db.set_progress_handler(lambda: int(time.monotonic() >= deadline), 1000)
db.row_factory = sqlite3.Row
db.execute("BEGIN")
# The total counts with the cheap permission filters only: the per-row
# freshness UDF scales with the whole library and can burn the whole
# read budget on its own (D22). Product accepted a small drift here;
# the rows that are actually displayed are still checked row by row.
try:
total = db.execute(
f"SELECT COUNT(*) FROM governed_memories m WHERE {predicate}", values
).fetchone()[0]
except sqlite3.OperationalError as error:
if not _interrupted(error):
raise
return MemoryLibraryPage(
items=[],
total=0,
offset=request.offset,
limit=request.limit,
stale=True,
)
predicate += " AND " + register_memory_freshness(db)
try:
if request.query:
current = db.execute(
f"""SELECT m.* FROM governed_memories m {_origin_join()}
WHERE {predicate}
ORDER BY {_order_by()}""",
values,
).fetchall()
rows = current[request.offset : request.offset + request.limit]
else:
rows = db.execute(
f"""SELECT m.* FROM governed_memories m {_origin_join()}
WHERE {predicate}
ORDER BY {_order_by()} LIMIT ? OFFSET ?""",
[*values, request.limit, request.offset],
).fetchall()
except sqlite3.OperationalError as error:
if not _interrupted(error):
raise
# The count already succeeded: report it, flag the page stale.
return MemoryLibraryPage(
items=[],
total=total,
offset=request.offset,
limit=request.limit,
stale=True,
)
# Static read helper only: constructing the Store would initialize schema.
items = MemoryGovernanceStore.memories_from_rows(db, rows, lifecycle=lifecycle)
# Only the authorized page participates; source filters select memories,
# not a subset of their provenance. Keep this in the count/page snapshot.
source_ids_by_memory: dict[str, list[str]] = {item.memory_id: [] for item in items}
# D96: the list shows when the cited conversation happened, not when
# the memory was generated (which is "today" for a fresh import). Read
# in the same query as the sources, so the page costs no extra round
# trip and only this page's IDs are looked at, never the whole library.
conversation_at: dict[str, str] = {}
imported_at: dict[str, str] = {}
if items:
try:
for link in db.execute(
# D104: the conversation time is when the person spoke, not
# when the turn ended. A memory's evidence covers the whole
# round (question, tool calls, reply), so MAX() lands on the
# reply and reads as a later, contradictory time next to the
# cited source. Prefer the earliest person-side evidence; only
# fall back to the earliest of any kind when the round has no
# person-authored row at all.
"""SELECT link.memory_id, e.source_id,
COALESCE(
MIN(
CASE WHEN e.role = 'user'
AND e.role = 'user'
THEN e.created_at END
),
MIN(e.created_at)
) AS conversation_at,
MAX(e.imported_at) AS imported_at
FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.active = 1 AND e.user_id = ?
AND link.memory_id IN (SELECT value FROM json_each(?))
GROUP BY link.memory_id, e.source_id
ORDER BY link.memory_id, e.source_id""",
(request.user_id, json.dumps(list(source_ids_by_memory))),
):
memory_id = link["memory_id"]
source_ids_by_memory[memory_id].append(link["source_id"])
for seen, value in (
(conversation_at, link["conversation_at"]),
(imported_at, link["imported_at"]),
):
if value and str(value) > seen.get(memory_id, ""):
seen[memory_id] = str(value)
except sqlite3.OperationalError as error:
if not _interrupted(error):
raise
# Page rows are already read; ship them without source maps
# rather than failing the whole request at the last step.
return MemoryLibraryPage(
items=items,
total=total,
offset=request.offset,
limit=request.limit,
source_ids_by_memory=source_ids_by_memory,
stale=True,
)
source_time_by_memory = {
memory_id: MemorySourceTime(at=conversation_at[memory_id], basis="conversation")
if memory_id in conversation_at
else MemorySourceTime(at=imported_at[memory_id], basis="import")
for memory_id in source_ids_by_memory
if memory_id in conversation_at or memory_id in imported_at
}
return MemoryLibraryPage(
items=items,
total=total,
offset=request.offset,
limit=request.limit,
source_ids_by_memory=source_ids_by_memory,
source_time_by_memory=source_time_by_memory,
)
@@ -1,180 +0,0 @@
"""Bounded, permission-aware receipt projection; never initialize a write store."""
import sqlite3
from pathlib import Path
from typing import Literal
from pydantic import BaseModel, ConfigDict
from one_memory.database_access import main_database
#: `legacy` is a confirmation recorded before memories stopped waiting for
#: one; it changed nothing that still exists, and `migrated_from` says so.
Operation = Literal["create", "correct", "set_scopes", "undo", "delete", "hy_reconcile", "legacy"]
ResultStatus = Literal["confirmed", "deleted"]
MigratedFrom = Literal["confirm", "reject"]
OPERATIONS = {"create", "correct", "set_scopes", "undo", "delete", "hy_reconcile", "legacy"}
STATUSES = {"confirmed", "deleted"}
#: Which operation each pre-v2 one now reads as.
MIGRATED_OPERATIONS = {"confirm": "legacy", "reject": "delete"}
#: A memory may move on to these after an earlier receipt was written; the
#: receipt still reports what that request committed, not the current status.
CURRENT_STATUSES = STATUSES | {"superseded"}
class MemoryMutationReceiptForbidden(RuntimeError):
def __init__(self) -> None:
super().__init__("memory_mutation_receipt_forbidden")
class MemoryMutationReceiptUnavailable(RuntimeError):
def __init__(self) -> None:
super().__init__("memory_mutation_receipt_unavailable")
class MemoryMutationReceipt(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
schema_version: Literal[2] = 2
request_id: str
state: Literal["not_found", "committed"]
operation: Operation | None = None
memory_id: str | None = None
result_revision: int | None = None
result_status: ResultStatus | None = None
memory_state: Literal["present", "deleted"] | None = None
#: The operation as it was stored before v2, when it has no v2 name of its own.
migrated_from: MigratedFrom | None = None
def _identity(value: str, maximum: int) -> bool:
return isinstance(value, str) and 0 < len(value) <= maximum
def _read_snapshot(
database_path: Path, request_id: str, user_id: str
) -> tuple[MemoryMutationReceipt, tuple[object, ...]]:
with main_database(database_path).read(row_factory=sqlite3.Row) as db:
db.execute("PRAGMA query_only=ON")
db.execute("BEGIN")
rows = db.execute(
"SELECT user_id,request_id,memory_id,operation,result_revision "
"FROM governed_memory_mutations WHERE user_id=? AND request_id=? LIMIT 2",
(user_id, request_id),
).fetchall()
if not rows:
return MemoryMutationReceipt(request_id=request_id, state="not_found"), ()
if len(rows) != 1:
raise MemoryMutationReceiptUnavailable()
mutation = rows[0]
memory_id, operation, revision = (
mutation["memory_id"],
mutation["operation"],
mutation["result_revision"],
)
if (
not _identity(memory_id, 512)
or operation not in OPERATIONS
or type(revision) is not int
or not 1 <= revision <= 2**53 - 1
):
raise MemoryMutationReceiptUnavailable()
current = db.execute(
"SELECT user_id,revision,status,deleted_at FROM governed_memories WHERE memory_id=?",
(memory_id,),
).fetchone()
if (
current is None
or current["user_id"] != user_id
or type(current["revision"]) is not int
or current["revision"] < revision
or current["status"] not in CURRENT_STATUSES
or (current["status"] == "deleted") != (current["deleted_at"] is not None)
):
raise MemoryMutationReceiptUnavailable()
# Versions have no historical evidence snapshot. Check every retained link,
# not only active ones, before disclosing even operation/ID metadata.
links = db.execute(
"SELECT l.evidence_id,l.active,e.user_id,e.source_id,e.revision,"
"g.granted_at,g.revoked_at FROM governed_memory_evidence l "
"LEFT JOIN evidence e ON e.evidence_id=l.evidence_id "
"LEFT JOIN source_grants g ON g.user_id=e.user_id AND g.source_id=e.source_id "
"WHERE l.memory_id=? ORDER BY l.evidence_id LIMIT 10001",
(memory_id,),
).fetchall()
if not links or len(links) > 10000:
raise MemoryMutationReceiptUnavailable()
for link in links:
if (
link["user_id"] != user_id
or link["source_id"] is None
or link["active"] not in (0, 1)
):
raise MemoryMutationReceiptUnavailable()
if link["revoked_at"] is not None:
raise MemoryMutationReceiptForbidden()
versions = db.execute(
"SELECT memory_id,revision,action,status,migrated_from FROM governed_memory_versions "
"WHERE memory_id=? AND revision=? LIMIT 2",
(memory_id, revision),
).fetchall()
if len(versions) != 1:
raise MemoryMutationReceiptUnavailable()
version = versions[0]
if version["action"] != operation or version["status"] not in STATUSES:
raise MemoryMutationReceiptUnavailable()
migrated_from = version["migrated_from"]
if migrated_from is not None and MIGRATED_OPERATIONS.get(migrated_from) != operation:
raise MemoryMutationReceiptUnavailable()
if (operation == "legacy") != (migrated_from == "confirm"):
raise MemoryMutationReceiptUnavailable()
required_status = {
"create": "confirmed",
"legacy": "confirmed",
"delete": "deleted",
}
if operation in required_status and version["status"] != required_status[operation]:
raise MemoryMutationReceiptUnavailable()
if (operation == "create") != (revision == 1):
raise MemoryMutationReceiptUnavailable()
if (
operation in {"correct", "set_scopes", "hy_reconcile", "undo"}
and version["status"] != "confirmed"
):
raise MemoryMutationReceiptUnavailable()
receipt = MemoryMutationReceipt(
request_id=request_id,
state="committed",
operation=operation,
memory_id=memory_id,
result_revision=revision,
result_status=version["status"],
memory_state="deleted" if current["status"] == "deleted" else "present",
migrated_from=migrated_from,
)
return receipt, (
tuple(mutation),
tuple(current),
tuple(version),
tuple(tuple(link) for link in links),
)
def read_memory_mutation_receipt(
database_path: Path, *, request_id: str, user_id: str = "local"
) -> MemoryMutationReceipt:
"""History only, never current authorization, HY cleanup or safe-retry proof."""
if not _identity(request_id, 256):
raise ValueError("invalid_memory_mutation_request_id")
if not _identity(user_id, 256):
raise ValueError("invalid_memory_mutation_user_id")
try:
first = _read_snapshot(database_path, request_id, user_id)
current = _read_snapshot(database_path, request_id, user_id)
if first != current:
raise MemoryMutationReceiptUnavailable()
return current[0]
except MemoryMutationReceiptForbidden:
raise
except (sqlite3.Error, ValueError, TypeError, KeyError, IndexError, OSError):
raise MemoryMutationReceiptUnavailable() from None
@@ -67,20 +67,6 @@ class MemoryNoteReceipt(BaseModel):
error: str | None = None
class SavedMemoryNoteEvidence(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
available: Literal[True] = True
evidence_id: str
source_id: str
revision: int
imported_at: datetime
role: str
written_by: WrittenBy
created_at: datetime | None
verbatim: str
class LegacyMemoryNoteReader:
"""Read pre-cutover note receipts without retaining any write runtime."""
@@ -441,48 +427,6 @@ def memory_note_router(dependencies: Sequence[Depends]) -> APIRouter:
except ValueError as error:
raise HTTPException(409, str(error)) from error
@router.get("/{evidence_id}/evidence", response_model=SavedMemoryNoteEvidence)
def evidence(evidence_id: str, request: Request) -> SavedMemoryNoteEvidence:
try:
notes = store(request)
if notes.sdk_authority:
raise HTTPException(409, "memory_note_evidence_retired")
with notes.evidence._connect() as connection:
record = notes.evidence._get_with_connection(connection, evidence_id)
metadata = record.origin_meta.get(EvidenceOrigin.MANUAL_IMPORT) if record else None
if (
record is None
or record.user_id != "local"
or record.source_id != SOURCE_ID
or metadata is None
or metadata.raw_meta.get("input_mode") != "direct_memory_note"
):
raise KeyError(evidence_id)
grant = connection.execute(
"SELECT revoked_at FROM source_grants WHERE user_id='local' AND source_id=?",
(SOURCE_ID,),
).fetchone()
if grant is None or grant[0] is not None:
raise ValueError("memory_note_source_revoked")
result = SavedMemoryNoteEvidence(
evidence_id=record.evidence_id,
source_id=record.source_id,
revision=record.revision,
imported_at=record.imported_at,
role=record.role,
written_by=record.written_by,
created_at=record.created_at,
verbatim=record.verbatim,
)
# The returned body and reference share one snapshot. Recheck authority
# after that read so a revoked source is not released from a stale view.
notes.require(evidence_id)
return result
except KeyError as error:
raise HTTPException(404, "memory_note_not_found") from error
except ValueError as error:
raise HTTPException(409, str(error)) from error
@router.post("/{evidence_id}/process", response_model=MemoryNoteReceipt)
async def process(evidence_id: str, request: Request) -> MemoryNoteReceipt:
notes = store(request)
@@ -316,6 +316,12 @@ LEGACY_MEMORY_WRITE_JOB_COLUMNS = (
_FIFO_BLOCKING_STATES = (PENDING, RUNNING, WAITING_FOR_PERMISSION, WAITING_FOR_MODEL)
EVENT_TIME_WINDOW = timedelta(minutes=10)
#: Sources whose jobs are a person adding a memory by hand ("添加记忆"), not
#: history. History is digested strictly old to new through the event-time
#: window; a note is stamped "now", so inside that window it waited behind the
#: whole unprocessed history. These jobs are claimed ahead of the window,
#: still one at a time per source and in order within their session.
DIRECT_WRITE_SOURCES = frozenset({"one-memory-manual"})
@dataclass(frozen=True)
@@ -2601,6 +2607,39 @@ class DurableWriteDrain:
placeholders = ",".join("?" for _ in enabled_sources)
source_clause = f"AND candidate.source_key IN ({placeholders}) "
source_params = tuple(sorted(enabled_sources))
direct_sources = ",".join("?" for _ in DIRECT_WRITE_SOURCES)
# Same eligibility as the window claim below (free lease, no
# running job of its source, nothing earlier in its session), but
# not bound to the window: a note must not wait for history.
direct_rows = connection.execute(
"SELECT candidate.job_id,candidate.source_key "
"FROM memory_write_jobs AS candidate "
"WHERE candidate.state=? "
"AND (candidate.lease_expires_at IS NULL OR candidate.lease_expires_at < ?) "
+ source_clause
+ f"AND candidate.source_key IN ({direct_sources}) "
"AND NOT EXISTS ("
"SELECT 1 FROM memory_write_jobs AS held "
"WHERE held.source_key=candidate.source_key AND held.state=? "
"AND held.lease_owner IS NOT NULL AND held.lease_expires_at>=?) "
"AND NOT EXISTS ("
"SELECT 1 FROM memory_write_jobs AS earlier "
"WHERE earlier.session_key=candidate.session_key "
f"AND earlier.state IN ({blocking}) "
"AND (earlier.event_at<candidate.event_at OR "
"(earlier.event_at=candidate.event_at "
"AND earlier.queue_seq<candidate.queue_seq))) "
"ORDER BY candidate.queue_seq",
(
PENDING,
now.isoformat(),
*source_params,
*sorted(DIRECT_WRITE_SOURCES),
RUNNING,
now.isoformat(),
*_FIFO_BLOCKING_STATES,
),
).fetchall()
window = connection.execute(
"SELECT window_anchor,window_end FROM memory_write_schedule_state WHERE singleton=1"
).fetchone()
@@ -2654,6 +2693,7 @@ class DurableWriteDrain:
if active is None:
connection.execute("DELETE FROM memory_write_schedule_state WHERE singleton=1")
window = None
anchor: Any = None
if window is None:
anchor = connection.execute(
"SELECT candidate.event_at "
@@ -2682,8 +2722,10 @@ class DurableWriteDrain:
*_FIFO_BLOCKING_STATES,
),
).fetchone()
if anchor is None:
if anchor is None and not direct_rows:
return ()
has_window = window is not None or anchor is not None
if window is None and anchor is not None:
anchor_at = datetime.fromisoformat(str(anchor[0]))
if anchor_at.tzinfo is None:
anchor_at = anchor_at.replace(tzinfo=UTC)
@@ -2695,39 +2737,41 @@ class DurableWriteDrain:
"VALUES (1,?,?,?)",
(anchor_text, end_text, now.isoformat()),
)
rows = connection.execute(
"SELECT candidate.job_id,candidate.source_key "
"FROM memory_write_jobs AS candidate "
"WHERE candidate.state=? "
"AND (candidate.lease_expires_at IS NULL OR candidate.lease_expires_at < ?) "
+ source_clause
+ "AND candidate.event_at>=? AND candidate.event_at<? "
"AND NOT EXISTS ("
"SELECT 1 FROM memory_write_jobs AS held "
"WHERE held.source_key=candidate.source_key AND held.state=? "
"AND held.lease_owner IS NOT NULL AND held.lease_expires_at>=?) "
"AND NOT EXISTS ("
"SELECT 1 FROM memory_write_jobs AS earlier "
"WHERE earlier.session_key=candidate.session_key "
f"AND earlier.state IN ({blocking}) "
"AND (earlier.event_at<candidate.event_at OR "
"(earlier.event_at=candidate.event_at "
"AND earlier.queue_seq<candidate.queue_seq))) "
"ORDER BY candidate.event_at,candidate.queue_seq",
(
PENDING,
now.isoformat(),
*source_params,
anchor_text,
end_text,
RUNNING,
now.isoformat(),
*_FIFO_BLOCKING_STATES,
),
).fetchall()
rows: list[Any] = []
if has_window:
rows = connection.execute(
"SELECT candidate.job_id,candidate.source_key "
"FROM memory_write_jobs AS candidate "
"WHERE candidate.state=? "
"AND (candidate.lease_expires_at IS NULL OR candidate.lease_expires_at < ?) "
+ source_clause
+ "AND candidate.event_at>=? AND candidate.event_at<? "
"AND NOT EXISTS ("
"SELECT 1 FROM memory_write_jobs AS held "
"WHERE held.source_key=candidate.source_key AND held.state=? "
"AND held.lease_owner IS NOT NULL AND held.lease_expires_at>=?) "
"AND NOT EXISTS ("
"SELECT 1 FROM memory_write_jobs AS earlier "
"WHERE earlier.session_key=candidate.session_key "
f"AND earlier.state IN ({blocking}) "
"AND (earlier.event_at<candidate.event_at OR "
"(earlier.event_at=candidate.event_at "
"AND earlier.queue_seq<candidate.queue_seq))) "
"ORDER BY candidate.event_at,candidate.queue_seq",
(
PENDING,
now.isoformat(),
*source_params,
anchor_text,
end_text,
RUNNING,
now.isoformat(),
*_FIFO_BLOCKING_STATES,
),
).fetchall()
selected: list[str] = []
selected_sources: set[str] = set()
for row in rows:
for row in (*direct_rows, *rows):
source_key = str(row["source_key"])
if source_key in selected_sources:
continue
@@ -1,7 +1,7 @@
"""A3-01: the HTTP entry to the unified write path.
`POST /api/v1/memory-writes` accepts a write and returns a request id;
`GET /api/v1/memory-writes/{id}` reports its outcome.
`POST /api/v2/memory-writes` accepts OpenAI-style messages and returns a
request id; `GET /api/v2/memory-writes/{id}` reports its outcome.
Two rules shape this endpoint, both consequences of A0-03's measurement that HY
has no idempotency key:
@@ -45,17 +45,6 @@ from one_memory.memory_server.registry import WriteRegistry, WriteState
from one_memory.memory_server.submit import WriteConflictError
class MemoryWriteInput(BaseModel):
model_config = ConfigDict(extra="forbid")
internal_session_id: str = Field(min_length=1, max_length=128)
first_round: int = Field(ge=1)
last_round: int = Field(ge=1)
evidence_ids: list[str] = Field(min_length=1)
priority: Literal["live", "normal"] = "normal"
idempotency_key: str | None = Field(default=None, max_length=256)
class MemoryWriteReceipt(BaseModel):
request_id: str
state: WriteState
@@ -121,67 +110,6 @@ def _registry(
return cast(WriteRegistry | DurableWriteRegistry, registry)
def router(dependencies: list[Any] | None = None) -> APIRouter:
instance = APIRouter(
prefix="/api/v1/memory-writes",
tags=["memory-writes"],
dependencies=dependencies or [],
)
@instance.post("", response_model=MemoryWriteReceipt, status_code=202)
def submit(payload: MemoryWriteInput, request: Request) -> MemoryWriteReceipt:
registry = _registry(request)
batch = BatchIdentity(
internal_session_id=payload.internal_session_id,
first_round=payload.first_round,
last_round=payload.last_round,
)
request_contract = WriteRequest(
internal_session_id=payload.internal_session_id,
batch_identity=batch,
evidence_ids=payload.evidence_ids,
idempotency_key=payload.idempotency_key or batch.batch_id,
priority=payload.priority,
)
try:
record, enqueued = registry.submit(request_contract)
except WriteConflictError as error:
raise HTTPException(409, str(error)) from error
except LookupError as error:
# Evidence-first: the write points at Evidence the business
# database does not hold. Rejecting it here is a caller error, not
# a server fault, and it must not create a queue row.
raise HTTPException(422, str(error)) from error
return MemoryWriteReceipt(
request_id=record.request_id,
state=record.state,
created_at=record.created_at,
updated_at=record.updated_at,
detail="" if enqueued else "already accepted; not enqueued again",
)
@instance.get("/{request_id}", response_model=MemoryWriteReceipt)
def status(request_id: str, request: Request) -> MemoryWriteReceipt:
registry = _registry(request, write=False)
try:
record = registry.get(request_id)
except DurableQueueSchemaUnavailableError as error:
raise HTTPException(503, str(error)) from error
if record is None:
raise HTTPException(404, "unknown request id")
return MemoryWriteReceipt(
request_id=record.request_id,
state=record.state,
created_at=record.created_at,
updated_at=record.updated_at,
detail=record.detail,
raw_ids=record.raw_ids,
memory_ids=record.memory_ids,
)
return instance
def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
instance = APIRouter(
prefix="/api/v2/memory-writes",
@@ -1,623 +0,0 @@
"""Read-only process: serves the same read routes as Core without write access.
Opens the database in `mode=ro` with `PRAGMA query_only=ON`; every store
operation here goes through `main_database().read()`, which already enforces
this. No HY SDK, no Chroma, no ONNX runtime, no keychain: the imports that
would pull them in are simply not reached, and the routes below never call
into a write path.
Reads that need HY (semantic search, graph overlay) are delegated to the
Core's internal endpoints; the Core is the only process that owns the
vector stores. Side effects (call events, feedback) are queued and sent
asynchronously, never blocking the read response.
"""
from __future__ import annotations
import asyncio
import builtins
import contextlib
import logging
import os
import sqlite3
import time
from collections.abc import AsyncIterator, Collection, Mapping, Sequence
from contextlib import asynccontextmanager
from datetime import UTC, datetime
from pathlib import Path
from typing import Any, cast
from fastapi import Depends, FastAPI, HTTPException, Query, Request
from pydantic import BaseModel, ConfigDict, SecretStr
from one_memory.database_access import main_database
from one_memory.home_summary import (
HomeSummary,
HomeSummaryUnavailable,
read_home_summary,
)
from one_memory.memory_governance import GovernedMemory, MemoryStatus
from one_memory.memory_library import MemoryLibraryPage, read_memory_library
from one_memory.recall import RecallRequest, _KeywordCandidates
from one_memory.semantic_deadline import SemanticMemoryHit
from one_memory.source_read_access import SourceReadAccess
logger = logging.getLogger(__name__)
MAX_READER_RSS_BYTES = 250 * 1024 * 1024
_PROBE_QUERY = "__om_probe__"
def _check_imports() -> None:
"""Fail fast if any forbidden module is already loaded."""
forbidden = ("hy_memory", "chromadb", "onnxruntime", "keyring")
import sys
loaded = [name for name in forbidden if name in sys.modules]
if loaded:
raise ImportError(f"reader_process_loaded_forbidden_modules: {', '.join(loaded)}")
def _memory_readable_sql() -> str:
return """
m.user_id = ? AND m.status = 'confirmed' AND m.deleted_at IS NULL
AND NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
JOIN source_grants g ON g.user_id=e.user_id AND g.source_id=e.source_id
WHERE link.memory_id=m.memory_id AND +link.active=1
AND g.revoked_at IS NOT NULL
)
AND NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
LEFT JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.memory_id=m.memory_id AND +link.active=1
AND (e.evidence_id IS NULL OR e.user_id != m.user_id)
)
"""
class ReaderSettings(BaseModel):
model_config = ConfigDict(frozen=True)
data_dir: Path
host: str = "127.0.0.1"
port: int = 0 # 0 = ephemeral
api_token: SecretStr
core_port: int
core_api_token: str
class SideEffectForwarder:
"""Queues read-path side effects and forwards them to Core.
Reads in the Reader must stay read-only: the database is opened with
`query_only`, so a call event cannot be written here. Instead each read
that would have recorded one enqueues it, and a background task posts it
to Core's internal endpoint. The response never waits for it; a Core that
is unavailable drops it and counts the drop.
"""
def __init__(self, settings: ReaderSettings) -> None:
self.settings = settings
self.queue: asyncio.Queue[dict[str, object]] = asyncio.Queue(maxsize=256)
self.dropped = 0
self.task: asyncio.Task[None] | None = None
def enqueue(self, event: dict[str, object]) -> None:
try:
self.queue.put_nowait(event)
except asyncio.QueueFull:
self.dropped += 1
def start(self) -> None:
if self.task is None or self.task.done():
self.task = asyncio.ensure_future(self._drain())
def stop(self) -> None:
if self.task is not None and not self.task.done():
self.task.cancel()
async def _drain(self) -> None:
import httpx
while True:
event = await self.queue.get()
try:
async with httpx.AsyncClient(
headers={
"Authorization": f"Bearer {self.settings.core_api_token}",
"Content-Type": "application/json",
},
timeout=5.0,
proxy=None,
) as client:
await client.post(
f"http://127.0.0.1:{self.settings.core_port}/internal/call-event",
json=event,
)
except Exception:
# A dropped side effect is never a failed read: recall already
# answered. Counted so a persistently unavailable Core is
# visible rather than silent.
self.dropped += 1
async def _core_get(
settings: ReaderSettings,
path: str,
timeout_s: float = 5.0,
params: Mapping[
str, str | int | float | bool | None | Sequence[str | int | float | bool | None]
]
| None = None,
) -> Any:
"""Call the Core's internal read endpoint for data only Core can serve.
`params` is passed to httpx rather than interpolated into the path: a
query containing `&`, `#` or a space would otherwise be cut short or
read as a fragment, silently searching for the wrong thing.
"""
import httpx
url = f"http://127.0.0.1:{settings.core_port}{path}"
async with httpx.AsyncClient(
headers={"Authorization": f"Bearer {settings.core_api_token}"},
timeout=timeout_s,
proxy=None,
) as client:
response = await client.get(url, params=params)
response.raise_for_status()
return response.json()
class _ReaderGovernance:
"""Minimal read interface over the governance store for recall."""
def __init__(self, database_path: Path) -> None:
self.database_path = database_path
def list(
self,
*,
user_id: str,
status: MemoryStatus | None = None,
query: str | None = None,
limit: int = 100,
offset: int = 0,
) -> list[GovernedMemory]:
"""One page of memories, read through the Core's own read function.
`MemoryGovernanceStore.list` cannot be used directly: constructing the
store runs schema DDL on the write connection, and the Reader opens
with `query_only`. `read_memory_library` is the same read expressed as
a static SQL read (revoked sources excluded, per-row freshness checked
for the rows it returns, cheap total for the rest), so the Reader and
the Core cannot drift on what "readable" means.
"""
page = read_memory_library(
self.database_path,
user_id=user_id,
query=query,
status=status,
offset=offset,
limit=limit,
)
return list(page.items)
def find_confirmed_matching_terms(
self,
*,
user_id: str,
terms: Collection[str],
limit: int = 200,
scopes: Collection[str] | None = None,
) -> tuple[builtins.list[GovernedMemory], bool]:
if not 1 <= limit <= 2_000:
raise ValueError("invalid candidate limit")
cleaned = tuple(dict.fromkeys(term.strip() for term in terms if term.strip()))
if not cleaned or scopes is not None and not scopes:
return [], False
clauses = ["m.user_id=?", "m.status='confirmed'", "m.deleted_at IS NULL"]
values: builtins.list[object] = [user_id]
matches = []
for term in cleaned[:32]:
matches.append("m.content LIKE ? ESCAPE '\\'")
escaped = term.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
values.append(f"%{escaped}%")
clauses.append(f"({' OR '.join(matches)})")
if scopes is not None:
clauses.append(
"EXISTS (SELECT 1 FROM json_each(m.scopes_json) "
f"WHERE json_each.value IN ({','.join('?' for _ in scopes)}))"
)
values.extend(scopes)
values.append(limit + 1)
with main_database(self.database_path).read(row_factory=sqlite3.Row) as db:
rows = db.execute(
f"SELECT m.* FROM governed_memories m WHERE {' AND '.join(clauses)} "
"ORDER BY m.updated_at DESC,m.memory_id LIMIT ?",
values,
).fetchall()
items = self._memories_from_rows(db, rows[:limit])
return items, len(rows) > limit or len(cleaned) > 32
def get(self, memory_id: str, *, user_id: str) -> GovernedMemory | None:
memory = _get_memory_direct(self.database_path, memory_id, user_id)
return memory if isinstance(memory, GovernedMemory) else None
def sources_behind_memories(
self,
*,
user_id: str,
memory_ids: Collection[str],
) -> dict[str, frozenset[str]]:
if not memory_ids:
return {}
identifiers = tuple(memory_ids)
placeholders = ",".join("?" for _ in identifiers)
with main_database(self.database_path).read(row_factory=sqlite3.Row) as db:
rows = db.execute(
"SELECT link.memory_id,e.source_id FROM governed_memory_evidence link "
"JOIN evidence e ON e.evidence_id=link.evidence_id "
f"WHERE link.active=1 AND e.user_id=? AND link.memory_id IN ({placeholders})",
(user_id, *identifiers),
).fetchall()
found: dict[str, set[str]] = {}
for row in rows:
found.setdefault(str(row["memory_id"]), set()).add(str(row["source_id"]))
return {key: frozenset(values) for key, values in found.items()}
@staticmethod
def _memories_from_rows(
db: sqlite3.Connection, rows: Sequence[sqlite3.Row]
) -> builtins.list[GovernedMemory]:
from one_memory.memory_governance import MemoryGovernanceStore
return MemoryGovernanceStore.memories_from_rows(db, builtins.list(rows))
def _conversation_hit(row: Any) -> dict[str, Any]:
"""One stored row as a search hit.
The author travels with the row. A client that is not told who wrote a row
has to invent an author, and the invention is always "the user", which
files an assistant turn under the person who was talking to it.
"""
return {
"evidence_id": row["evidence_id"],
"conversation_id": row["conversation_id"],
"source_id": row["source_id"],
"workspace": row["workspace"],
"project": row["project"],
"created_at": row["created_at"],
"excerpt": row["verbatim"][:200] if row["verbatim"] else "",
# Both columns are NOT NULL on `evidence`, so a stored row always
# knows; a summary of a whole conversation has no single author and
# never reaches here.
"role": row["role"],
"written_by": row["written_by"],
}
def create_reader_app(settings: ReaderSettings) -> FastAPI:
"""A read-only FastAPI application for the Reader process."""
_check_imports()
database_path = settings.data_dir / "one-memory.sqlite3"
if not database_path.exists():
raise FileNotFoundError(f"reader_database_missing: {database_path}")
# Validate schema version without writing.
from one_memory.memory_server.migrations import SCHEMA_TABLE
from one_memory.schema_gate import BUILD_SCHEMA_VERSION
with main_database(database_path).read() as db:
tables = {
str(row[0]) for row in db.execute("SELECT name FROM sqlite_master WHERE type='table'")
}
if SCHEMA_TABLE in tables:
row = db.execute(f"SELECT MAX(version) FROM {SCHEMA_TABLE}").fetchone()
found = int(row[0]) if row and row[0] is not None else 0
if found > BUILD_SCHEMA_VERSION:
raise RuntimeError(f"reader_schema_too_new: {found} > {BUILD_SCHEMA_VERSION}")
elif not tables:
raise RuntimeError("reader_database_empty")
access = SourceReadAccess(database_path)
forwarder = SideEffectForwarder(settings)
governance = _ReaderGovernance(database_path)
@asynccontextmanager
async def lifespan(_: FastAPI) -> AsyncIterator[None]:
monitor = asyncio.get_running_loop().create_task(_memory_monitor())
forwarder.start()
yield
monitor.cancel()
forwarder.stop()
async def _memory_monitor() -> None:
import resource
import sys
while True:
await asyncio.sleep(30)
usage = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
rss_bytes = usage if sys.platform == "darwin" else usage * 1024
if rss_bytes > MAX_READER_RSS_BYTES:
logger.error(
"reader_memory_limit_exceeded: %d bytes > %d",
rss_bytes,
MAX_READER_RSS_BYTES,
)
os._exit(1)
app = FastAPI(title="MemHub Reader", lifespan=lifespan)
app.state.settings = settings
app.state.database_path = database_path
expected_token = settings.api_token.get_secret_value()
def require_auth(request: Request) -> None:
import hmac
origin = request.headers.get("origin")
if origin is not None and origin not in (
"one-memory://app",
"http://localhost:5173",
):
raise HTTPException(403, "origin_not_allowed")
authorization = request.headers.get("authorization", "")
scheme, _, token = authorization.partition(" ")
if not (scheme.lower() == "bearer" and hmac.compare_digest(token, expected_token)):
raise HTTPException(401, "invalid_local_token")
protected = [Depends(require_auth)]
@app.get("/api/v1/home-summary", response_model=HomeSummary, dependencies=protected)
def home_summary() -> HomeSummary:
try:
return read_home_summary(database_path)
except HomeSummaryUnavailable:
# Belt and braces: read_home_summary answers stale itself, but if
# that ever changes, never let the homepage 500.
return HomeSummary(
memory_count=0, latest=None, observed_at=datetime.now(UTC), stale=True
)
@app.get("/api/v1/memory-library", response_model=MemoryLibraryPage, dependencies=protected)
def memory_library(
query: str | None = Query(default=None, max_length=512),
status: str | None = None,
active_only: bool = False,
source: str | None = Query(default=None, min_length=1),
offset: int = Query(default=0, ge=0),
limit: int = Query(default=100, ge=1, le=500),
) -> MemoryLibraryPage:
from one_memory.memory_governance import MemoryStatus as MS
ms = MS(status) if status else None
return read_memory_library(
database_path,
user_id="local",
query=query,
status=ms,
active_only=active_only,
source=source,
offset=offset,
limit=limit,
)
@app.get("/api/v1/memories", dependencies=protected)
def list_memories(
status: str | None = None,
query: str | None = None,
limit: int = Query(default=100, ge=1, le=500),
offset: int = Query(default=0, ge=0),
) -> list[dict[str, Any]]:
from one_memory.memory_governance import MemoryStatus as MS
ms = MS(status) if status else None
memories = governance.list(
user_id="local", status=ms, query=query, limit=limit, offset=offset
)
visible = access.memories(memories)
return [m.model_dump(mode="json") for m in visible]
@app.get("/api/v1/memories/graph", dependencies=protected)
async def memory_graph(
limit: int = Query(default=160, ge=1, le=300),
) -> dict[str, Any]:
memories = governance.list(user_id="local", status=MemoryStatus.CONFIRMED, limit=500)
projected = access.memories(memories[:limit])
nodes: dict[str, dict[str, Any]] = {}
edges: list[dict[str, Any]] = []
source_ids: set[str] = set()
with main_database(database_path).read(row_factory=sqlite3.Row) as db:
for memory in projected:
node_id = f"memory:{memory.memory_id}"
nodes[node_id] = {
"node_id": node_id,
"node_type": "memory",
"label": memory.content,
"memory_id": memory.memory_id,
"kind": memory.kind.value
if hasattr(memory.kind, "value")
else str(memory.kind),
"status": memory.status.value
if hasattr(memory.status, "value")
else str(memory.status),
}
evidence_ids = list(memory.evidence_ids)
if evidence_ids:
placeholders = ",".join("?" for _ in evidence_ids)
sql = (
f"SELECT DISTINCT source_id FROM evidence"
f" WHERE evidence_id IN ({placeholders})"
)
for row in db.execute(
sql,
evidence_ids,
):
sid = str(row["source_id"])
source_ids.add(sid)
for sid in sorted(source_ids):
source_node_id = f"source:{sid}"
nodes[source_node_id] = {
"node_id": source_node_id,
"node_type": "source",
"label": sid,
"source_id": sid,
}
for memory in projected:
if any(_source_for_evidence(db, eid) == sid for eid in memory.evidence_ids):
memory_node_id = f"memory:{memory.memory_id}"
edges.append(
{
"edge_id": f"{source_node_id}->{memory_node_id}",
"source_node_id": source_node_id,
"target_node_id": memory_node_id,
}
)
# Fetch HY graph overlay from Core (not available in Reader).
hy_overlay = None
with contextlib.suppress(Exception):
hy_overlay = await _core_get(
settings,
"/internal/graph-snapshot",
timeout_s=5.0,
params={"limit": limit},
)
if hy_overlay and not hy_overlay.get("consulted", False):
hy_overlay = None
return {
"limit": limit,
"total": len(projected),
"nodes": tuple(nodes.values()),
"edges": edges,
**({"hy_overlay": hy_overlay} if hy_overlay else {}),
}
@app.get("/api/v1/memories/{memory_id}", dependencies=protected)
def get_memory(memory_id: str) -> dict[str, Any]:
memory = _get_memory_direct(database_path, memory_id, "local")
if memory is None:
raise HTTPException(404, "memory_not_found")
if not access.memories([memory]):
raise HTTPException(403, "memory_source_revoked")
return cast(dict[str, Any], memory.model_dump(mode="json"))
@app.post("/api/v1/recall", response_model=None, dependencies=protected)
async def recall(request: RecallRequest) -> dict[str, Any]:
from one_memory.recall import GovernedRecallTarget
started = time.monotonic()
target = GovernedRecallTarget(
governance=governance,
semantic_search=None,
resolve_workspace=None,
filter_memories=access.memories,
)
if request.query == _PROBE_QUERY:
return target.recall(request, user_id="local").model_dump(mode="json")
deadline = time.monotonic() + request.timeout_ms / 1_000
semantic_ids: tuple[str, ...] = ()
try:
remaining = max(0.1, deadline - time.monotonic())
result = await asyncio.wait_for(
_core_get(
settings,
"/internal/semantic-ids",
timeout_s=remaining,
params={"query": request.query, "limit": request.limit},
),
timeout=remaining,
)
semantic_ids = tuple(str(value) for value in result.get("ids", []) if value)
except Exception:
pass
readable = target._readable_scopes(request)
# Keyword candidates from read-only SQL.
keywords = target._keyword_candidates(
request, user_id="local", readable=readable, deadline=deadline
)
receipt = target._rank(
request,
user_id="local",
readable=readable,
semantic_hits={
memory_id: SemanticMemoryHit(
memory_id,
1.0 - index / max(1, len(semantic_ids)),
)
for index, memory_id in enumerate(semantic_ids)
},
keywords=keywords
if isinstance(keywords, _KeywordCandidates)
else _KeywordCandidates((), False, False),
deadline=deadline,
timed_out=False,
)
forwarder.enqueue(
{
"tool": "om_recall",
"query": request.query,
"scope": request.scope,
"state": receipt.state.value,
"memory_count": len(receipt.hits),
"elapsed_ms": (time.monotonic() - started) * 1000,
}
)
return receipt.model_dump(mode="json")
@app.get("/api/v1/conversations/search", dependencies=protected)
def search_conversations(
query: str = Query(min_length=1, max_length=512),
limit: int = Query(default=20, ge=1, le=100),
) -> dict[str, Any]:
if not database_path.exists():
raise HTTPException(503, "conversation_search_unavailable")
with main_database(database_path).read(row_factory=sqlite3.Row) as db:
try:
rows = db.execute(
"""SELECT * FROM evidence
WHERE user_id = 'local' AND verbatim LIKE ? ESCAPE '\\'
ORDER BY imported_at DESC LIMIT ?""",
(f"%{query}%", limit),
).fetchall()
except Exception:
raise HTTPException(503, "conversation_search_unavailable") from None
hits = [_conversation_hit(row) for row in rows]
return {
"hits": hits,
"empty_reason": None if hits else "no_matches",
"imported_messages": len(hits),
}
@app.get("/health")
def health() -> dict[str, str]:
return {"status": "ok", "service": "one-memory-reader"}
return app
def _get_memory_direct(database_path: Path, memory_id: str, user_id: str) -> Any:
from one_memory.memory_governance import _memory_from_row, _memory_links
with main_database(database_path).read(row_factory=sqlite3.Row) as db:
row = db.execute(
"SELECT * FROM governed_memories WHERE memory_id = ? AND user_id = ?",
(memory_id, user_id),
).fetchone()
if row is None:
return None
links = _memory_links(db, [memory_id])
return _memory_from_row(row, *(tuple(vals) for vals in links[memory_id]))
def _source_for_evidence(db: sqlite3.Connection, evidence_id: str) -> str | None:
row = db.execute(
"SELECT source_id FROM evidence WHERE evidence_id = ?", (evidence_id,)
).fetchone()
return str(row["source_id"]) if row else None
+10 -1
View File
@@ -81,7 +81,11 @@ class RecallHit(BaseModel):
score: float = Field(ge=0, le=1)
#: HY already rendered memory time/evolution/capsule context into content.
#: Hosts must not append their own governance timestamp in this case.
content_format: Literal["plain", "hy-sdk"] = "plain"
#: "raw" is an excerpt of an imported conversation, not a memory.
content_format: Literal["plain", "hy-sdk", "raw"] = "plain"
#: Raw excerpts only: the app and conversation the excerpt came from.
source: str | None = None
session_id: str | None = None
class RecallReceipt(BaseModel):
@@ -103,6 +107,11 @@ class RecallReceipt(BaseModel):
#: whether the answer is that nothing was found or that reading was not
#: allowed.
excluded_sources: tuple[str, ...] = ()
#: "raw" when recall searched the imported conversations instead of
#: memories, because no generation model is configured or no memory
#: exists yet (`fallback_reason`). Raw hits are excerpts, not memories.
mode: Literal["memory", "raw"] = "memory"
fallback_reason: Literal["model_unavailable", "no_memories"] | None = None
class RecallTarget(Protocol):
@@ -0,0 +1,149 @@
"""Recall that degrades to the original conversations instead of answering empty.
MemHub_recall searches the memories the SDK distilled from conversations. That
needs a configured generation model to have produced any, and a library that
holds some. Without either, the memory search can only come back empty, so
recall searches the SDK's raw layer (the imported conversations themselves)
instead: plain retrieval over what was imported, the same lane
`/api/v2/context/search` reads, under the same raw-source authority.
"""
from __future__ import annotations
import asyncio
import time
from collections.abc import Awaitable, Callable
from datetime import UTC, datetime
from typing import Literal
from one_memory.hy_sdk_types import HySdkRawHit
from one_memory.recall import (
PROBE_QUERY,
RecallHit,
RecallReceipt,
RecallRequest,
RecallState,
TruncationBudget,
)
FallbackReason = Literal["model_unavailable", "no_memories"]
#: Raw sources this caller may read, or None for every source (a trusted local
#: caller). An empty set means the caller may read no raw conversation.
RawSources = frozenset[str] | None
async def recall_with_fallback(
request: RecallRequest,
*,
model_ready: Callable[[], Awaitable[bool]],
recall_memories: Callable[[RecallRequest], Awaitable[RecallReceipt]],
has_memories: Callable[[], Awaitable[bool]],
raw_sources: Callable[[], Awaitable[RawSources]],
search_raw: Callable[[str, int, RawSources], Awaitable[tuple[HySdkRawHit, ...]]],
) -> RecallReceipt:
"""Memory recall when a model and memories exist, otherwise the raw layer.
The library check runs only after an empty memory search: with memories
present and nothing relevant, the answer stays an honest empty memory
result rather than a different kind of hit.
"""
if request.query == PROBE_QUERY:
return await recall_memories(request)
started = time.monotonic()
if await model_ready():
receipt = await recall_memories(request)
if receipt.hits or await has_memories():
return receipt
reason: FallbackReason = "no_memories"
else:
reason = "model_unavailable"
remaining = request.timeout_ms / 1_000 - (time.monotonic() - started)
return await _raw_recall(
request,
reason=reason,
remaining_seconds=remaining,
raw_sources=raw_sources,
search_raw=search_raw,
)
async def _raw_recall(
request: RecallRequest,
*,
reason: FallbackReason,
remaining_seconds: float,
raw_sources: Callable[[], Awaitable[RawSources]],
search_raw: Callable[[str, int, RawSources], Awaitable[tuple[HySdkRawHit, ...]]],
) -> RecallReceipt:
def receipt(
hits: tuple[RecallHit, ...] = (),
*,
used: int = 0,
truncated_by: TruncationBudget | None = None,
unexamined: int = 0,
) -> RecallReceipt:
return RecallReceipt(
state=RecallState.GROUNDED if hits else RecallState.EMPTY,
query=request.query,
scope=request.scope,
hits=hits,
used_characters=used,
truncated=truncated_by is not None,
truncated_by=truncated_by,
unexamined_count=unexamined,
mode="raw",
fallback_reason=reason,
)
sources = await raw_sources()
if sources is not None and not sources:
# The caller may read no original conversation: an empty answer, not
# a wider grant than recall had.
return receipt()
if remaining_seconds <= 0:
return receipt(truncated_by=TruncationBudget.TIMEOUT)
try:
found = await asyncio.wait_for(
search_raw(request.query, request.limit, sources),
timeout=remaining_seconds,
)
except TimeoutError:
return receipt(truncated_by=TruncationBudget.TIMEOUT)
hits: list[RecallHit] = []
used = 0
truncated_by: TruncationBudget | None = None
for index, hit in enumerate(found):
if sources is not None and hit.source not in sources:
continue
if len(hits) >= request.limit:
truncated_by = TruncationBudget.COUNT
return receipt(
tuple(hits),
used=used,
truncated_by=truncated_by,
unexamined=len(found) - index,
)
if used + len(hit.content) > request.character_budget:
truncated_by = TruncationBudget.CHARACTER
continue
hits.append(
RecallHit(
memory_id=hit.raw_id,
content=hit.content,
kind="raw",
scopes=(),
evidence_ids=(),
updated_at=(
datetime.fromtimestamp(hit.memory_at, tz=UTC)
if hit.memory_at is not None
else datetime(1970, 1, 1, tzinfo=UTC)
),
score=hit.score,
content_format="raw",
source=hit.source,
session_id=hit.session_id,
)
)
used += len(hit.content)
return receipt(tuple(hits), used=used, truncated_by=truncated_by)
@@ -41,6 +41,20 @@ class MigrationProgress:
phase: str
def _ledger_log(database_path: Path, event: str, migration_id: str, **details: object) -> None:
"""Every ledger transition goes to `sdk-migration.log`; never raises."""
from one_memory.sdk_migration_log import write_migration_log
write_migration_log(
database_path.parent,
writer="core-sdk-migrate",
event=event,
outcome="succeeded",
migration_id=migration_id,
details=details,
)
class SdkDataPlaneMigrationStore:
def __init__(self, database_path: Path) -> None:
self.database_path = database_path
@@ -151,7 +165,16 @@ class SdkDataPlaneMigrationStore:
elif backup_bundle_id and existing["backup_bundle_id"] != backup_bundle_id:
raise DataPlaneMigrationError("sdk_data_plane_migration_bundle_conflict")
connection.execute("COMMIT")
return self.progress(migration_id)
progress = self.progress(migration_id)
_ledger_log(
self.database_path,
"ledger.started",
migration_id,
created=existing is None,
state=progress.state,
phase=progress.phase,
)
return progress
def set_phase(self, migration_id: str, *, phase: str, state: str = "running") -> None:
with main_database(self.database_path).write(autocommit=True) as connection:
@@ -164,6 +187,7 @@ class SdkDataPlaneMigrationStore:
if cursor.rowcount != 1:
raise DataPlaneMigrationError("sdk_data_plane_migration_not_mutable")
connection.execute("COMMIT")
_ledger_log(self.database_path, "ledger.phase_set", migration_id, phase=phase, state=state)
def progress(self, migration_id: str) -> MigrationProgress:
with main_database(self.database_path).read(row_factory=sqlite3.Row) as connection:
@@ -257,6 +281,7 @@ class SdkDataPlaneMigrationStore:
(migration_id, now),
)
connection.execute("COMMIT")
_ledger_log(self.database_path, "ledger.authority_committed", migration_id, authority="sdk")
marker = self.cutover()
if marker is None:
raise DataPlaneMigrationError("sdk_data_plane_cutover_missing")
@@ -0,0 +1,243 @@
"""The one log support asks for when an upgrade did not reach SDK authority.
`<data_dir>/logs/sdk-migration.log` is JSON Lines, shared with the TypeScript
orchestrator (`packages/runtime-control/src/sdk-migration-log.ts`), which writes
the same line shape. A person attaches this single file to a report, so every
line says who wrote it, which build, which migration and what happened.
It must never carry user content: no memory or conversation text, no provider
response, no exception message (a message can quote either). Errors are logged
as their type, a sanitised code and traceback frames as `file:line:function`.
Detail values are numbers, booleans and short identifier-shaped strings only;
anything else is dropped rather than escaped.
Writing is synchronous and flushed: a migration is rare and the line that
explains a failure is worth more than the milliseconds. A write failure is
swallowed, because logging must never be the reason a migration or a startup
fails.
"""
from __future__ import annotations
import json
import os
import re
import traceback
from collections.abc import Iterator, Mapping
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
LOG_NAME = "sdk-migration.log"
#: Rotated to `sdk-migration.log.1` beyond this; one generation is kept.
MAX_BYTES = 4 * 1024 * 1024
_SAFE_TEXT = re.compile(r"^[A-Za-z0-9_.:/@+\-]{0,160}$")
_CODE = re.compile(r"^[a-z][a-z0-9]*(?:[._-][a-z0-9]+){0,15}$")
_SUBJECT = re.compile(r"^[A-Za-z0-9_.\-]{1,96}$")
_MAX_ITEMS = 32
_MAX_DEPTH = 4
_MAX_FRAMES = 24
_FRAME = re.compile(r"^[A-Za-z0-9_.\-]{1,96}:\d{1,7}:<?[A-Za-z0-9_]{1,96}>?$")
def log_path(data_dir: Path) -> Path:
return data_dir / "logs" / LOG_NAME
def _safe(value: object, depth: int = 0) -> Any:
if value is None or isinstance(value, bool):
return value
if isinstance(value, int):
return value
if isinstance(value, float):
return round(value, 3) if value == value and abs(value) != float("inf") else None
if isinstance(value, str):
return value if _SAFE_TEXT.fullmatch(value) else "<redacted>"
if depth >= _MAX_DEPTH:
return "<nested>"
if isinstance(value, Mapping):
result: dict[str, Any] = {}
for index, (key, item) in enumerate(value.items()):
if index >= _MAX_ITEMS:
result["<truncated>"] = len(value) - _MAX_ITEMS
break
if isinstance(key, str) and _SAFE_TEXT.fullmatch(key):
result[key] = _safe(item, depth + 1)
return result
if isinstance(value, (list, tuple, set, frozenset)):
items = list(value)
safe = [_safe(item, depth + 1) for item in items[:_MAX_ITEMS]]
if len(items) > _MAX_ITEMS:
safe.append(f"<+{len(items) - _MAX_ITEMS}>")
return safe
return "<unsupported>"
def _chain(error: BaseException) -> Iterator[BaseException]:
seen: set[int] = set()
current: BaseException | None = error
while current is not None and id(current) not in seen:
seen.add(id(current))
yield current
current = current.__cause__ or current.__context__
def describe_error(error: BaseException) -> dict[str, object]:
"""Type, code and frames of an exception chain; never its message."""
chain: list[dict[str, object]] = []
for item in _chain(error):
name = type(item).__name__
entry: dict[str, object] = {"type": name if _SAFE_TEXT.fullmatch(name) else "<unknown>"}
code = getattr(item, "code", None)
if isinstance(code, str):
# `<code>:<opaque id>` names the row that failed, e.g. a write job.
name, _, subject = code.partition(":")
if _CODE.fullmatch(name):
entry["code"] = name
if subject and _SUBJECT.fullmatch(subject):
entry["subject"] = subject
message = str(item)
# A message that is itself a stable code (the codebase's convention
# for domain errors) is safe to keep; anything else is content.
if _CODE.fullmatch(message):
entry["code"] = entry.get("code") or message
errno = getattr(item, "errno", None)
if isinstance(errno, int):
entry["errno"] = errno
if isinstance(item, OSError) and item.filename is not None:
# A path names a file, not its contents, but home directories can
# name a person: keep only the file name.
entry["file"] = Path(str(item.filename)).name
frames = traceback.extract_tb(item.__traceback__)[-_MAX_FRAMES:]
# Code locations, not content: built here from file names, line
# numbers and function names (`<module>`, `<lambda>` included).
entry["frames"] = [
location
for frame in frames
if _FRAME.fullmatch(
location := f"{Path(frame.filename).name}:{frame.lineno}:{frame.name}"
)
]
chain.append(entry)
if len(chain) >= 6:
break
return {"chain": chain}
def _build() -> dict[str, object]:
from one_memory import __version__
build: dict[str, object] = {"core_version": __version__}
for key, name in (
("ONE_MEMORY_VERSION", "app_version"),
("ONE_MEMORY_BUILD_NUMBER", "build_number"),
("ONE_MEMORY_COMMIT", "build_commit"),
("ONE_MEMORY_MIGRATION_CALLER", "caller"),
("ONE_MEMORY_MIGRATION_RUNTIME_BUILD", "runtime_build"),
):
value = os.environ.get(key, "").strip()
if value:
build[name] = value
return _safe(build)
def write_migration_log(
data_dir: Path,
*,
writer: str,
event: str,
outcome: str,
migration_id: str | None = None,
details: Mapping[str, object] | None = None,
error: BaseException | None = None,
code: str | None = None,
) -> None:
"""Append one line. Never raises."""
try:
line: dict[str, object] = {
"schema_version": 1,
"ts": datetime.now(UTC).isoformat(),
"pid": os.getpid(),
"writer": writer,
"event": event,
"outcome": outcome,
**_build(),
}
if migration_id is not None:
line["migration_id"] = migration_id
if code is not None:
line["code"] = code
if details:
line["details"] = dict(details)
safe_line = _safe(line)
if not isinstance(safe_line, dict):
return
if error is not None:
# Already content-free by construction; `_safe` would redact the
# `<module>` frames.
safe_line["error"] = describe_error(error)
destination = log_path(data_dir)
destination.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
try:
if destination.stat().st_size > MAX_BYTES:
os.replace(destination, destination.with_name(f"{LOG_NAME}.1"))
except FileNotFoundError:
pass
payload = (json.dumps(safe_line, ensure_ascii=False) + "\n").encode("utf-8")
descriptor = os.open(destination, os.O_WRONLY | os.O_APPEND | os.O_CREAT, 0o600)
try:
os.write(descriptor, payload)
os.fsync(descriptor)
finally:
os.close(descriptor)
except Exception: # noqa: BLE001 - logging must never break a migration
return
def migration_rows(database_path: Path) -> list[dict[str, object]]:
"""The recorded migration attempts, newest first; ids, states and phases only."""
import sqlite3
try:
connection = sqlite3.connect(f"file:{database_path}?mode=ro", uri=True, timeout=1)
except sqlite3.Error:
return []
try:
rows = connection.execute(
"SELECT migration_id,state,phase,created_at,updated_at "
"FROM sdk_data_plane_migrations ORDER BY created_at DESC LIMIT 8"
).fetchall()
except sqlite3.Error:
return []
finally:
connection.close()
return [
{
"migration_id": row[0],
"state": row[1],
"phase": row[2],
# Older builds stored `YYYY-MM-DD HH:MM:SS`; keep it identifier-shaped.
"created_at": str(row[3]).replace(" ", "T"),
"updated_at": str(row[4]).replace(" ", "T"),
}
for row in rows
]
def authority_snapshot(database_path: Path) -> dict[str, object]:
"""Authority and recorded attempts, read without changing the database."""
if not database_path.is_file():
return {"database": "absent"}
snapshot: dict[str, object] = {"database": "present"}
try:
from one_memory.sdk_data_plane_migration import SdkDataPlaneMigrationStore
authority, authority_migration = SdkDataPlaneMigrationStore(database_path).authority_state()
snapshot["authority"] = authority.value
snapshot["authority_migration_id"] = authority_migration
except Exception as error: # noqa: BLE001 - a snapshot must not fail its caller
snapshot["authority"] = "unreadable"
snapshot["authority_error"] = type(error).__name__
snapshot["migrations"] = migration_rows(database_path)
return snapshot
@@ -14,6 +14,7 @@ import json
import os
import shutil
import sqlite3
import time
from collections.abc import Callable, Iterator
from contextlib import AbstractContextManager, contextmanager
from dataclasses import dataclass
@@ -63,6 +64,71 @@ from one_memory.target_schema import drop_legacy_tables, legacy_tables_present
PREPARED_PHASE = "prepared"
class _StepLog:
"""Each runner step, its duration, its counts and how it ended.
The CLI only sees the exception that escapes `run_startup_migration` and
names it with a generic code when it is not one of ours; these lines say
which step raised it. Content-free: only names, counts and codes.
"""
def __init__(self, data_dir: Path) -> None:
self.data_dir = data_dir
self.migration_id: str | None = None
def check(self, step: str, code: str | None) -> None:
"""One preflight check that is a decision, not a call: its result code."""
from one_memory.sdk_migration_log import write_migration_log
write_migration_log(
self.data_dir,
writer="core-sdk-migrate",
event="step.failed" if code else "step.finished",
outcome="failed" if code else "succeeded",
migration_id=self.migration_id,
code=code,
details={"step": step, "result": code or "passed"},
)
@contextmanager
def step(self, name: str) -> Iterator[dict[str, object]]:
from one_memory.sdk_migration_log import write_migration_log
started = time.monotonic()
details: dict[str, object] = {}
try:
yield details
except BaseException as error:
code = getattr(error, "code", None)
write_migration_log(
self.data_dir,
writer="core-sdk-migrate",
event="step.failed",
outcome="failed",
migration_id=self.migration_id,
code=code if isinstance(code, str) else None,
error=error,
details={
"step": name,
"duration_ms": (time.monotonic() - started) * 1000,
**details,
},
)
raise
write_migration_log(
self.data_dir,
writer="core-sdk-migrate",
event="step.finished",
outcome="succeeded",
migration_id=self.migration_id,
details={
"step": name,
"duration_ms": (time.monotonic() - started) * 1000,
**details,
},
)
@dataclass(frozen=True)
class CutoverPreparation:
"""What the cutover did to legacy work, for the receipt and the dialog."""
@@ -83,8 +149,7 @@ class CutoverPreparation:
def _tables(database: sqlite3.Connection) -> set[str]:
return {
str(row[0])
for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
str(row[0]) for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
}
@@ -128,9 +193,7 @@ def prepare_legacy_write_queue_for_cutover(database_path: Path) -> tuple[int, in
if "evidence" not in tables:
return (0, 0)
jobs = (
tuple(
database.execute("SELECT job_id,state,payload_json FROM memory_write_jobs")
)
tuple(database.execute("SELECT job_id,state,payload_json FROM memory_write_jobs"))
if "memory_write_jobs" in tables
else ()
)
@@ -281,15 +344,27 @@ def abandoned_legacy_work(database_path: Path) -> dict[str, int]:
return abandoned
def prepare_legacy_work_for_cutover(database_path: Path) -> CutoverPreparation:
"""Make the surviving queue self-contained. Safe to repeat."""
def prepare_legacy_work_for_cutover(
database_path: Path, trace: dict[str, object] | None = None
) -> CutoverPreparation:
"""Make the surviving queue self-contained. Safe to repeat.
`trace` (the migration log's step details) learns which sub-step is
running, so a failure names it.
"""
trace = {} if trace is None else trace
trace["sub_step"] = "write_queue"
frozen, moved = prepare_legacy_write_queue_for_cutover(database_path)
trace["sub_step"] = "pending_dispatches"
dispatches = move_pending_dispatches(database_path)
trace["sub_step"] = "abandoned_work"
abandoned = abandoned_legacy_work(database_path)
trace.pop("sub_step", None)
return CutoverPreparation(
frozen_jobs=frozen,
moved_tasks=moved,
moved_dispatches=dispatches,
abandoned=abandoned_legacy_work(database_path),
abandoned=abandoned,
)
@@ -478,14 +553,25 @@ def run_startup_migration(
return open_client(profile)
return runtime_sdk_client(database_path, data_dir, profile)
log = _StepLog(data_dir)
store = SdkDataPlaneMigrationStore(database_path)
authority, authority_migration = store.authority_state()
profile = read_active_profile(database_path)
with log.step("preflight.authority_read") as step:
authority, authority_migration = store.authority_state()
step["authority"] = authority.value
log.migration_id = authority_migration
with log.step("preflight.profile_read") as step:
profile = read_active_profile(database_path)
step["profile_present"] = profile is not None
if authority is MemoryAuthority.SDK:
vectors_waiting = profile is not None and bool(legacy_vector_collections(data_dir, profile))
leftovers = bool(legacy_tables_present(database_path)) or abandoned_staging_present(
data_dir=data_dir, namespace=namespace
)
with log.step("preflight.leftovers_scan") as step:
vectors_waiting = profile is not None and bool(
legacy_vector_collections(data_dir, profile)
)
leftovers = bool(legacy_tables_present(database_path)) or abandoned_staging_present(
data_dir=data_dir, namespace=namespace
)
step["vectors_waiting"] = vectors_waiting
step["leftovers"] = leftovers
if not vectors_waiting and not leftovers:
return StartupMigrationResult(state="not_required")
reattached: tuple[ReattachedCollection, ...] = ()
@@ -493,20 +579,28 @@ def run_startup_migration(
if vectors_waiting:
assert profile is not None
progress("vectors")
reattached = reattach_legacy_vectors(data_dir, profile)
legacy = legacy_memory_ids_from_bundle(data_dir, authority_migration)
with log.step("vectors.reattach") as step:
reattached = reattach_legacy_vectors(data_dir, profile)
step["collections"] = len(reattached)
with log.step("verify.legacy_ids_from_bundle") as step:
legacy = legacy_memory_ids_from_bundle(data_dir, authority_migration)
step["bundle_ids_present"] = legacy is not None
if legacy is not None and not legacy.empty:
# Authority already moved, so nothing is left to protect by
# refusing; record what the repaired home reads back.
progress("verify")
with client_for(profile) as client:
with log.step("verify.readback") as step, client_for(profile) as client:
readback = verify_readback(client, legacy)
step.update(readback.as_document())
dropped: tuple[str, ...] = ()
if leftovers:
progress("cleanup")
dropped = resume_legacy_cleanup(data_dir=data_dir, database_path=database_path)
with log.step("cleanup.drop_legacy_tables") as step:
dropped = resume_legacy_cleanup(data_dir=data_dir, database_path=database_path)
step["dropped_tables"] = len(dropped)
if dropped:
_vacuum(database_path)
with log.step("cleanup.vacuum"):
_vacuum(database_path)
return StartupMigrationResult(
state="complete",
migration_id=authority_migration,
@@ -520,63 +614,89 @@ def run_startup_migration(
from one_memory.memory_server.migration_runner import SqliteMigrationStore, migrate
from one_memory.memory_server.migrations import MIGRATIONS
migration_id, bundle_id = _resumable_identity(database_path)
legacy = legacy_memory_ids_from(database_path)
with log.step("preflight.resumable_identity") as step:
migration_id, bundle_id = _resumable_identity(database_path)
step["resumed"] = migration_id != f"{MIGRATION_PREFIX}-{database_generation(database_path)}"
log.migration_id = migration_id
with log.step("preflight.legacy_ids_scan") as step:
legacy = legacy_memory_ids_from(database_path)
step["chat"] = len(legacy.chat)
step["productivity"] = len(legacy.productivity)
step["users"] = len(legacy.users)
if profile is None and not legacy.empty:
# Without the embedding profile the Runtime's collections cannot be
# named, so neither reattached nor read back. Legacy stays intact.
log.check("preflight.model_profile_check", "sdk_migration_model_profile_required")
raise DataPlaneMigrationError("sdk_migration_model_profile_required")
log.check("preflight.model_profile_check", None)
with capture_fence(data_dir):
progress("bundle")
manifest = _ensure_bundle(
data_dir=data_dir,
bundle_id=bundle_id,
runtime_build=runtime_build,
)
with log.step("bundle.ensure") as step:
manifest = _ensure_bundle(
data_dir=data_dir,
bundle_id=bundle_id,
runtime_build=runtime_build,
)
step["bundle_id"] = bundle_id
progress("prepare")
asyncio.run(migrate(SqliteMigrationStore(database_path), MIGRATIONS))
close_main_databases()
recover_abandoned_staging(
data_dir=data_dir,
database_path=database_path,
namespace=namespace,
)
store.start(
migration_id=migration_id,
source_runtime_build=str(manifest.get("runtime_build") or ""),
target_sdk_version=sdk_version,
source_schema_version=MIGRATIONS[-1].version,
namespace_fingerprint=hashlib.sha256(namespace.encode()).hexdigest(),
backend_fingerprint=hashlib.sha256(b"chroma:kuzu:sqlite").hexdigest(),
backup_bundle_id=bundle_id,
)
preparation = prepare_legacy_work_for_cutover(database_path)
with log.step("prepare.schema_migrate") as step:
asyncio.run(migrate(SqliteMigrationStore(database_path), MIGRATIONS))
close_main_databases()
step["schema_version"] = MIGRATIONS[-1].version
with log.step("prepare.recover_abandoned_staging"):
recover_abandoned_staging(
data_dir=data_dir,
database_path=database_path,
namespace=namespace,
)
with log.step("prepare.ledger_start"):
store.start(
migration_id=migration_id,
source_runtime_build=str(manifest.get("runtime_build") or ""),
target_sdk_version=sdk_version,
source_schema_version=MIGRATIONS[-1].version,
namespace_fingerprint=hashlib.sha256(namespace.encode()).hexdigest(),
backend_fingerprint=hashlib.sha256(b"chroma:kuzu:sqlite").hexdigest(),
backup_bundle_id=bundle_id,
)
with log.step("prepare.legacy_work") as step:
preparation = prepare_legacy_work_for_cutover(database_path, step)
step.update(preparation.as_document())
reattached = ()
readback = None
if profile is not None:
progress("vectors")
reattached = reattach_legacy_vectors(data_dir, profile)
with log.step("vectors.reattach") as step:
reattached = reattach_legacy_vectors(data_dir, profile)
step["collections"] = len(reattached)
if not legacy.empty:
assert profile is not None
progress("verify")
with client_for(profile) as client:
readback = verify_readback(client, legacy)
if not readback.passed:
store.set_phase(migration_id, phase="readback_failed", state="blocked")
raise MemoryReadbackFailed(readback)
store.set_phase(migration_id, phase=PREPARED_PHASE, state="ready")
with log.step("verify.readback") as step:
with client_for(profile) as client:
readback = verify_readback(client, legacy)
step.update(readback.as_document())
if not readback.passed:
store.set_phase(migration_id, phase="readback_failed", state="blocked")
raise MemoryReadbackFailed(readback)
with log.step("prepare.mark_prepared"):
store.set_phase(migration_id, phase=PREPARED_PHASE, state="ready")
progress("cutover")
commit_sdk_cutover(
data_dir=data_dir,
database_path=database_path,
migration_id=migration_id,
bundle_id=bundle_id,
sdk_version=sdk_version,
)
with log.step("cutover.commit"):
commit_sdk_cutover(
data_dir=data_dir,
database_path=database_path,
migration_id=migration_id,
bundle_id=bundle_id,
sdk_version=sdk_version,
)
progress("cleanup")
dropped = resume_legacy_cleanup(data_dir=data_dir, database_path=database_path)
with log.step("cleanup.drop_legacy_tables") as step:
dropped = resume_legacy_cleanup(data_dir=data_dir, database_path=database_path)
step["dropped_tables"] = len(dropped)
if dropped:
_vacuum(database_path)
with log.step("cleanup.vacuum"):
_vacuum(database_path)
return StartupMigrationResult(
state="cutover",
migration_id=migration_id,
@@ -23,6 +23,21 @@ from one_memory.sdk_data_plane_migration import DataPlaneMigrationError
_BATCH = 500
def open_offline_chroma(directory: Path) -> Any:
"""A Chroma client on the SDK's vector store, configured as the SDK opens it.
Chroma refuses a second client on a directory whose shared instance was
created with different settings, so offline code must not pass its own.
Telemetry is turned off the way the Runtime turns it off, by environment.
"""
import os
import chromadb
os.environ["ANONYMIZED_TELEMETRY"] = "False"
return chromadb.PersistentClient(path=str(directory))
@dataclass(frozen=True)
class ReattachedCollection:
legacy: str
@@ -97,45 +112,46 @@ def reattach_legacy_vectors(
pairs = legacy_vector_collections(data_dir, profile)
if not pairs:
return ()
import chromadb
from chromadb.config import Settings
client = chromadb.PersistentClient(
path=str(_vector_directory(data_dir)),
settings=Settings(anonymized_telemetry=False),
)
existing = set(_stored_names(data_dir))
done: list[ReattachedCollection] = []
for legacy_name, current_name in pairs:
legacy = client.get_collection(legacy_name)
legacy_count = legacy.count()
merged = 0
if current_name in existing:
current = client.get_collection(current_name)
legacy_ids = set(_ids(legacy))
current_ids: set[str] = set()
for batch in _records(current):
legacy.upsert(**batch)
current_ids.update(batch["ids"])
merged = len(current_ids)
if legacy.count() != len(legacy_ids | current_ids):
raise DataPlaneMigrationError("sdk_migration_vector_merge_incomplete")
client.delete_collection(current_name)
legacy.modify(name=current_name)
final = client.get_collection(current_name).count()
if final < legacy_count:
raise DataPlaneMigrationError("sdk_migration_vector_reattach_incomplete")
existing.discard(legacy_name)
existing.add(current_name)
done.append(
ReattachedCollection(
legacy=legacy_name,
current=current_name,
legacy_count=legacy_count,
merged_count=merged,
final_count=final,
client = open_offline_chroma(_vector_directory(data_dir))
# Closed, not left to the process: Chroma keeps one shared instance per
# directory, and the readback that follows opens the same directory
# through the SDK. A leaked instance failed that open on every 1.0.4 home
# with legacy vectors, so the first launch after upgrade always failed.
try:
existing = set(_stored_names(data_dir))
done: list[ReattachedCollection] = []
for legacy_name, current_name in pairs:
legacy = client.get_collection(legacy_name)
legacy_count = legacy.count()
merged = 0
if current_name in existing:
current = client.get_collection(current_name)
legacy_ids = set(_ids(legacy))
current_ids: set[str] = set()
for batch in _records(current):
legacy.upsert(**batch)
current_ids.update(batch["ids"])
merged = len(current_ids)
if legacy.count() != len(legacy_ids | current_ids):
raise DataPlaneMigrationError("sdk_migration_vector_merge_incomplete")
client.delete_collection(current_name)
legacy.modify(name=current_name)
final = client.get_collection(current_name).count()
if final < legacy_count:
raise DataPlaneMigrationError("sdk_migration_vector_reattach_incomplete")
existing.discard(legacy_name)
existing.add(current_name)
done.append(
ReattachedCollection(
legacy=legacy_name,
current=current_name,
legacy_count=legacy_count,
merged_count=merged,
final_count=final,
)
)
)
finally:
client.close()
return tuple(done)
@@ -1,518 +0,0 @@
"""Per-session incremental read. A pull or an ack is not adoption."""
import json
import secrets
import sqlite3
from collections.abc import Iterator
from contextlib import contextmanager
from dataclasses import dataclass
from pathlib import Path
#: Schema owned by `memory_server.migrations` (migration 11). No local copy.
from one_memory.change_outbox import evidence_event_id
from one_memory.database_access import main_database
from one_memory.derived_echo import echoed_event_ids, is_pure_echo
from one_memory.memory_server.migrations import session_inbox_ddl
from one_memory.session_registry import SessionRegistry
_SCHEMA = session_inbox_ddl("sqlite")
class CursorRejected(Exception):
pass
@dataclass(frozen=True)
class InboxEvent:
event_id: str
evidence_id: str | None
evidence_revision: int | None
source_id: str | None
@dataclass(frozen=True)
class InboxPage:
batch_id: str
cursor: str
state: str
adoption: str
events: tuple[InboxEvent, ...]
@dataclass(frozen=True)
class InboxPosition:
"""Where a cursor would stand once the events handed over are confirmed."""
event_id: str | None
permission_epoch: int
handed_event_ids: frozenset[str] = frozenset()
receipt_id: str | None = None
@dataclass(frozen=True)
class InboxPeek:
events: tuple[InboxEvent, ...]
position: InboxPosition
@dataclass(frozen=True)
class InboxAck:
batch_id: str
state: str
adoption: str
@dataclass(frozen=True)
class InboxSignal:
signal_id: str
class SessionInbox:
def __init__(self, database_path: Path) -> None:
self._path = database_path
self._registry = SessionRegistry(database_path)
def initialize(self) -> None:
with main_database(self._path).write() as db, db:
db.executescript(_SCHEMA)
def subscribe(
self,
session_key: str,
sources: tuple[str, ...],
permission_epoch: int,
) -> None:
self._registry.require(session_key)
with main_database(self._path).write() as db, db:
db.execute(
"""
INSERT INTO session_subscriptions (
session_key, granted_sources, permission_epoch
) VALUES (?, ?, ?)
ON CONFLICT(session_key) DO UPDATE SET
granted_sources = excluded.granted_sources,
permission_epoch = excluded.permission_epoch
""",
(session_key, json.dumps(list(sources)), permission_epoch),
)
def pull(self, session_key: str, cursor: str | None, limit: int) -> InboxPage:
self._registry.require(session_key)
with self._snapshot() as db:
start_after, epoch = self._cursor_position(session_key, cursor, db)
granted = self._granted(session_key, db)
events = self._without_host_echo(self._events(db), db)
current = self._epoch(session_key, db)
selected, found = self._scan(granted, start_after, limit, events)
state = "delivered"
# A cursor whose event is gone, or whose permission epoch no longer
# matches, must not look like "nothing new". Rebuild from the start.
if cursor is not None and (not found or epoch != current):
selected, _found = self._scan(granted, None, limit, events)
state = "resync-required"
epoch = current
if selected:
scanned_id = selected[-1].event_id
elif state == "resync-required":
scanned_id = None
else:
scanned_id = start_after
return self._store_page(session_key, scanned_id, epoch, state, selected)
def peek(self, session_key: str, limit: int) -> InboxPeek:
"""What the next pull from the latest cursor would return, without moving it."""
self._registry.require(session_key)
# Grants, epoch and events are read in one snapshot: a revoke landing
# between them would pair the old grants with the new epoch.
with self._snapshot() as db:
cursor = self._latest_cursor(session_key, db)
start_after, epoch = self._cursor_position(session_key, cursor, db)
granted = self._granted(session_key, db)
events = self._without_host_echo(self._events(db), db)
current = self._epoch(session_key, db)
selected, found = self._scan(granted, start_after, limit, events)
if cursor is not None and (not found or epoch != current):
selected, _found = self._scan(granted, None, limit, events)
start_after = None
end = selected[-1].event_id if selected else start_after
receipt_id = None
if selected:
receipt_id = secrets.token_hex(16)
receipt = self._new_receipt(
session_key,
position_event_id=str(end),
permission_epoch=current,
handed_event_ids=tuple(event.event_id for event in selected),
)
with main_database(self._path).write() as db, db:
db.execute(
"""
INSERT INTO session_snapshots (
snapshot_token, session_key, frozen_ids
)
VALUES (?, ?, ?)
""",
(receipt_id, session_key, receipt),
)
return InboxPeek(
tuple(selected),
InboxPosition(
end,
current,
frozenset(event.event_id for event in selected),
receipt_id,
),
)
@staticmethod
def _new_receipt(
session_key: str,
*,
position_event_id: str,
permission_epoch: int,
handed_event_ids: tuple[str, ...],
) -> str:
return json.dumps(
{
"kind": "peek",
"session_key": session_key,
"event_id": position_event_id,
"permission_epoch": permission_epoch,
"handed_event_ids": list(handed_event_ids),
},
separators=(",", ":"),
)
def advance_to(self, session_key: str, position: InboxPosition) -> bool | None:
"""Move the latest cursor forward to a peeked position, never back.
Two rounds may peek overlapping pages and confirm in either order. The
cursor only ever moves to the furthest position actually handed over.
"""
self._registry.require(session_key)
if position.event_id is None:
return False
if position.receipt_id is None:
raise CursorRejected("missing_peek_receipt")
# IMMEDIATE takes the file's write lock at once: the confirming process
# is not the one that peeked, so the in-process writer queue is not enough.
with main_database(self._path).write() as db, db:
db.execute("BEGIN IMMEDIATE")
events = self._without_host_echo(self._events(db), db)
order = [event.event_id for event in events]
by_id = {event.event_id: event for event in events}
receipt = self._peek_receipt(session_key, position, db)
row = db.execute(
"SELECT permission_epoch FROM session_subscriptions WHERE session_key = ?",
(session_key,),
).fetchone()
epoch = int(row[0]) if row is not None else 0
handed = receipt
if any(event_id not in order for event_id in handed):
raise CursorRejected("handed_event_missing")
granted = self._granted(session_key, db)
target = by_id.get(position.event_id)
if target is None or target.source_id not in granted:
raise CursorRejected("target_not_selectable")
if any(by_id[event_id].source_id not in granted for event_id in handed):
raise CursorRejected("handed_event_not_selectable")
if position.permission_epoch != epoch:
return None
latest_token = self._latest_cursor(session_key, db)
cursor_position = -1
if latest_token is not None:
start_after, _cursor_epoch = self._cursor_position(session_key, latest_token, db)
if start_after in order:
cursor_position = order.index(str(start_after))
target_position = order.index(position.event_id)
if cursor_position >= target_position:
return None
required = [
event.event_id
for event in events[cursor_position + 1 : target_position + 1]
if event.source_id in granted
]
handed_after_cursor = [
event_id for event_id in handed if order.index(event_id) > cursor_position
]
if handed_after_cursor != required:
raise CursorRejected("incomplete_handed_range")
latest = db.execute(
"""
SELECT scanned_event_id, permission_epoch FROM session_inbox_cursors
WHERE session_key = ? ORDER BY rowid DESC LIMIT 1
""",
(session_key,),
).fetchone()
if (
latest is not None
and int(latest[1]) == epoch
and latest[0] in order
and order.index(latest[0]) > order.index(position.event_id)
):
return None
db.execute(
"""
INSERT INTO session_inbox_cursors (
cursor_token, session_key, scanned_event_id, permission_epoch
) VALUES (?, ?, ?, ?)
""",
(secrets.token_hex(32), session_key, position.event_id, epoch),
)
return True
@staticmethod
def _peek_receipt(
session_key: str, position: InboxPosition, db: sqlite3.Connection
) -> tuple[str, ...]:
row = db.execute(
"""
SELECT frozen_ids FROM session_snapshots
WHERE snapshot_token = ? AND session_key = ?
""",
(position.receipt_id, session_key),
).fetchone()
if row is None:
raise CursorRejected("invalid_peek_receipt")
try:
receipt = json.loads(str(row[0]))
except json.JSONDecodeError as error:
raise CursorRejected("invalid_peek_receipt") from error
if not isinstance(receipt, dict) or receipt.get("kind") != "peek":
raise CursorRejected("invalid_peek_receipt")
if receipt.get("session_key") != session_key:
raise CursorRejected("peek_receipt_session_mismatch")
if receipt.get("event_id") != position.event_id:
raise CursorRejected("peek_receipt_target_mismatch")
if receipt.get("permission_epoch") != position.permission_epoch:
raise CursorRejected("peek_receipt_epoch_mismatch")
handed = receipt.get("handed_event_ids")
if not isinstance(handed, list) or any(not isinstance(item, str) for item in handed):
raise CursorRejected("invalid_peek_receipt")
loaded = tuple(handed)
if frozenset(loaded) != position.handed_event_ids:
raise CursorRejected("peek_receipt_handed_mismatch")
return loaded
def latest_cursor(self, session_key: str) -> str | None:
self._registry.require(session_key)
with main_database(self._path).read() as db:
return self._latest_cursor(session_key, db)
@staticmethod
def _latest_cursor(session_key: str, db: sqlite3.Connection) -> str | None:
row = db.execute(
"""
SELECT cursor_token FROM session_inbox_cursors
WHERE session_key = ? ORDER BY rowid DESC LIMIT 1
""",
(session_key,),
).fetchone()
return None if row is None else str(row[0])
@contextmanager
def _snapshot(self) -> Iterator[sqlite3.Connection]:
"""One read transaction; under WAL every read in it sees the same state."""
# A pooled reader does not open a transaction for a SELECT on its own;
# the transaction is rolled back when the connection is handed back.
with main_database(self._path).read() as db:
db.execute("BEGIN")
yield db
def ack(self, session_key: str, batch_id: str) -> InboxAck:
self._registry.require(session_key)
with main_database(self._path).write() as db, db:
row = db.execute(
"SELECT session_key FROM session_inbox_batches WHERE batch_id = ?",
(batch_id,),
).fetchone()
if row is None or row[0] != session_key:
raise CursorRejected(batch_id)
db.execute(
"""
UPDATE session_inbox_batches SET state = 'acknowledged'
WHERE batch_id = ?
""",
(batch_id,),
)
return InboxAck(batch_id, "acknowledged", "unknown")
def begin_snapshot(self, session_key: str) -> str:
self._registry.require(session_key)
frozen = json.dumps([event.event_id for event in self._events()])
token = secrets.token_hex(16)
with main_database(self._path).write() as db, db:
db.execute(
"""
INSERT INTO session_snapshots (snapshot_token, session_key, frozen_ids)
VALUES (?, ?, ?)
""",
(token, session_key, frozen),
)
return token
def finish_snapshot(self, session_key: str, token: str) -> InboxPage:
self._registry.require(session_key)
with main_database(self._path).read() as db:
db.row_factory = sqlite3.Row
row = db.execute(
"SELECT session_key FROM session_snapshots WHERE snapshot_token = ?",
(token,),
).fetchone()
if row is None or row["session_key"] != session_key:
raise CursorRejected(token)
# Events committed before finish belong in this page. The cursor stops
# at the last event actually returned, so a late insert is not skipped.
return self.pull(session_key, cursor=None, limit=128)
def signal(self, session_key: str) -> InboxSignal:
self._registry.require(session_key)
return InboxSignal(secrets.token_hex(16))
def _cursor_position(
self, session_key: str, cursor: str | None, db: sqlite3.Connection
) -> tuple[str | None, int]:
if cursor is None:
return None, self._epoch(session_key, db)
row = db.execute(
"""
SELECT session_key, scanned_event_id, permission_epoch
FROM session_inbox_cursors WHERE cursor_token = ?
""",
(cursor,),
).fetchone()
if row is None or row[0] != session_key:
raise CursorRejected(cursor)
return row[1], int(row[2])
@staticmethod
def _scan(
granted: set[str],
start_after: str | None,
limit: int,
events: tuple[InboxEvent, ...],
) -> tuple[list[InboxEvent], bool]:
found = start_after is None
selected: list[InboxEvent] = []
for event in events:
if not found:
if event.event_id == start_after:
found = True
continue
if event.source_id not in granted:
continue
if len(selected) >= limit:
break
selected.append(event)
return selected, found
def _store_page(
self,
session_key: str,
scanned_id: str | None,
epoch: int,
state: str,
selected: list[InboxEvent],
) -> InboxPage:
token = secrets.token_hex(32)
batch_id = secrets.token_hex(16)
with main_database(self._path).write() as db, db:
db.execute(
"""
INSERT INTO session_inbox_cursors (
cursor_token, session_key, scanned_event_id, permission_epoch
) VALUES (?, ?, ?, ?)
""",
(token, session_key, scanned_id, epoch),
)
db.execute(
"""
INSERT INTO session_inbox_batches (batch_id, session_key, state)
VALUES (?, ?, ?)
""",
(batch_id, session_key, state),
)
return InboxPage(batch_id, token, state, "unknown", tuple(selected))
@staticmethod
def _epoch(session_key: str, db: sqlite3.Connection) -> int:
row = db.execute(
"SELECT permission_epoch FROM session_subscriptions WHERE session_key = ?",
(session_key,),
).fetchone()
return int(row[0]) if row is not None else 0
@staticmethod
def _granted(session_key: str, db: sqlite3.Connection) -> set[str]:
row = db.execute(
"SELECT granted_sources FROM session_subscriptions WHERE session_key = ?",
(session_key,),
).fetchone()
if row is None:
return set()
loaded = json.loads(row[0])
return set(loaded) if isinstance(loaded, list) else set()
def _events(self, db: sqlite3.Connection | None = None) -> tuple[InboxEvent, ...]:
if db is None:
with main_database(self._path).read() as own:
return self._events(own)
events: dict[str, tuple[str, InboxEvent]] = {}
for evidence_id, revision, source_id, created_at in db.execute(
"SELECT evidence_id, revision, source_id, updated_at FROM evidence"
):
event_id = evidence_event_id(str(evidence_id), int(revision))
events[event_id] = (
str(created_at),
InboxEvent(event_id, str(evidence_id), int(revision), str(source_id)),
)
for (
event_id,
evidence_id,
revision,
source_id,
created_at,
) in db.execute(
"""
SELECT o.event_id, o.evidence_id, o.evidence_revision, e.source_id, o.created_at
FROM change_outbox o
LEFT JOIN evidence e ON e.evidence_id = o.evidence_id
"""
):
events.setdefault(
str(event_id),
(str(created_at), InboxEvent(str(event_id), evidence_id, revision, source_id)),
)
return tuple(
event
for _occurred_at, event in sorted(
events.values(), key=lambda item: (item[0], item[1].event_id)
)
)
@staticmethod
def _without_host_echo(
events: tuple[InboxEvent, ...], db: sqlite3.Connection
) -> tuple[InboxEvent, ...]:
"""The host can save our notice back; that is not a new decision."""
known = {event.event_id for event in events}
delivered: set[str] = set()
kept: list[InboxEvent] = []
for event in events:
if event.evidence_id is None:
kept.append(event)
continue
row = db.execute(
"SELECT verbatim FROM evidence WHERE evidence_id = ?",
(event.evidence_id,),
).fetchone()
verbatim = "" if row is None else str(row[0])
if is_pure_echo(verbatim, known):
delivered.update(echoed_event_ids(verbatim))
continue
kept.append(event)
return tuple(event for event in kept if event.event_id not in delivered)
@@ -1,140 +0,0 @@
"""Online means an unexpired lease, not a session file that is still on disk."""
import sqlite3
from collections.abc import Iterator
from contextlib import contextmanager
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any
from one_memory.database_access import main_database
#: Schema owned by `memory_server.migrations` (migration 5). No local copy,
#: so the migration cannot drift from what this module actually creates.
from one_memory.memory_server.migrations import session_leases_ddl
from one_memory.memory_server.sql_compat import (
execute,
execute_script,
is_mysql,
row_to_dict,
)
from one_memory.memory_server.store_binding import store_for
from one_memory.session_registry import SessionRegistry
#: Both renderings, chosen at use time (same reason as every migrated module:
#: the sqlite rendering carries SQLite types and index forms MySQL rejects).
SCHEMA_BY_ENGINE = {False: session_leases_ddl("sqlite"), True: session_leases_ddl("mysql")}
def schema_for(mysql: bool) -> str:
return SCHEMA_BY_ENGINE[mysql]
#: The heartbeat upsert, per engine.
#:
#: `ON CONFLICT ... DO UPDATE` (SQLite) vs `ON DUPLICATE KEY UPDATE` (MySQL).
#: Wrong either way and a session's lease silently keeps its old expiry, which
#: reads as "session went offline" rather than as a database error.
BEAT_SQLITE = """
INSERT INTO session_leases (session_key, pid, generation, expires_at)
VALUES (?, ?, ?, ?)
ON CONFLICT(session_key) DO UPDATE SET
pid = excluded.pid,
generation = excluded.generation,
expires_at = excluded.expires_at
"""
BEAT_MYSQL = """
INSERT INTO session_leases (session_key, pid, generation, expires_at)
VALUES (%s, %s, %s, %s)
ON DUPLICATE KEY UPDATE
pid = VALUES(pid),
generation = VALUES(generation),
expires_at = VALUES(expires_at)
"""
SELECT_SQLITE = "SELECT generation, expires_at FROM session_leases WHERE session_key = ?"
SELECT_MYSQL = "SELECT generation, expires_at FROM session_leases WHERE session_key = %s"
class SessionLease:
def __init__(self, database_path: Path) -> None:
self._path = database_path
self._store: Any = store_for(database_path)
self._registry = SessionRegistry(database_path)
def initialize(self) -> None:
mysql = is_mysql(self._store)
with self._connect(writing=True) as db:
execute_script(db, mysql, schema_for(mysql))
def beat(
self,
session_key: str,
*,
pid: int,
generation: int,
now: datetime,
ttl_seconds: int,
) -> None:
self._registry.require(session_key)
expires = (now + timedelta(seconds=ttl_seconds)).isoformat()
mysql = is_mysql(self._store)
with self._connect(writing=True) as db:
execute(
db,
mysql,
BEAT_MYSQL if mysql else BEAT_SQLITE,
(session_key, pid, generation, expires),
)
def online(
self,
session_key: str,
*,
now: datetime,
generation: int,
source_file: Path | None = None,
) -> bool:
del source_file
self._registry.require(session_key)
mysql = is_mysql(self._store)
with self._connect() as db:
row = execute(
db, mysql, SELECT_MYSQL if mysql else SELECT_SQLITE, (session_key,)
).fetchone()
if row is None:
return False
# Named access only works on `sqlite3.Row`; MySQL gives a tuple, so the
# row is normalised to a mapping on both engines first.
values = row_to_dict(row, ("generation", "expires_at"), mysql)
if int(values["generation"]) != generation:
return False
return now < _as_datetime(values["expires_at"], now)
@contextmanager
def _connect(self, *, writing: bool = False) -> Iterator[Any]:
"""A checkout on whichever engine is in force."""
if is_mysql(self._store):
checkout = self._store.write() if writing else self._store.read()
with checkout as connection:
yield connection
return
database = main_database(self._path)
checkout = database.write() if writing else database.read(row_factory=sqlite3.Row)
with checkout as connection, connection:
yield connection
def _as_datetime(value: object, reference: datetime) -> datetime:
"""Parse the stored expiry, matching the caller's awareness.
SQLite returns the stored ISO string so it parses aware. MySQL `DATETIME`
returns a naive `datetime`, and comparing that with an aware `now` raises
`TypeError` — a crash on a perfectly ordinary read. The stored value is the
same instant, so the zone is taken from the caller's clock rather than
assumed, which keeps the comparison meaningful either way.
"""
if isinstance(value, datetime):
if value.tzinfo is None and reference.tzinfo is not None:
return value.replace(tzinfo=reference.tzinfo)
return value
return datetime.fromisoformat(str(value))
@@ -1,137 +0,0 @@
"""Trusted host sessions. A directory is a relevance hint, not an identity."""
import hashlib
import sqlite3
from dataclasses import dataclass
from pathlib import Path
from one_memory.call_event_store import HostId
from one_memory.database_access import main_database
#: Schema owned by `memory_server.migrations` (migration 6). No local copy,
#: so the migration cannot drift from what this module actually creates.
from one_memory.memory_server.migrations import trusted_sessions_ddl
_SCHEMA = trusted_sessions_ddl("sqlite")
class SessionIdentityError(Exception):
pass
class AmbiguousWorkspace(SessionIdentityError):
pass
class ForgedSession(SessionIdentityError):
pass
@dataclass(frozen=True)
class TrustedSession:
session_key: str
installation_id: str
host_id: HostId
native_session_id: str
runtime_instance: str
workspace: str | None
class SessionRegistry:
def __init__(self, database_path: Path) -> None:
self._path = database_path
def initialize(self) -> None:
with main_database(self._path).write() as db, db:
db.execute(_SCHEMA)
def open(
self,
*,
installation_id: str,
host_id: HostId,
native_session_id: str,
runtime_instance: str,
workspace: str | None,
) -> TrustedSession:
if not installation_id or not native_session_id or not runtime_instance:
raise SessionIdentityError("incomplete_session")
key = _identity_key(installation_id, host_id, native_session_id, runtime_instance)
with main_database(self._path).write() as db, db:
row = db.execute(
"SELECT session_key FROM trusted_sessions WHERE session_key = ?",
(key,),
).fetchone()
if row is None:
db.execute(
"""
INSERT INTO trusted_sessions (
session_key, installation_id, host_id, native_session_id,
runtime_instance, workspace
) VALUES (?, ?, ?, ?, ?, ?)
""",
(key, installation_id, host_id, native_session_id, runtime_instance, workspace),
)
return self.require(key)
def require(self, session_key: str) -> TrustedSession:
with main_database(self._path).read() as db:
db.row_factory = sqlite3.Row
row = db.execute(
"SELECT * FROM trusted_sessions WHERE session_key = ?",
(session_key,),
).fetchone()
if row is None:
raise ForgedSession(session_key)
return _session(row)
def in_workspace(self, workspace: str) -> tuple[TrustedSession, ...]:
with main_database(self._path).read() as db:
db.row_factory = sqlite3.Row
rows = db.execute(
"SELECT * FROM trusted_sessions WHERE workspace = ? ORDER BY session_key",
(workspace,),
).fetchall()
return tuple(_session(row) for row in rows)
def matching_native(
self, host_id: HostId, native_session_id: str
) -> tuple[TrustedSession, ...]:
with main_database(self._path).read() as db:
db.row_factory = sqlite3.Row
rows = db.execute(
"""
SELECT * FROM trusted_sessions
WHERE host_id = ? AND native_session_id = ?
ORDER BY session_key
""",
(host_id, native_session_id),
).fetchall()
return tuple(_session(row) for row in rows)
def sole_session(self, workspace: str) -> TrustedSession:
found = self.in_workspace(workspace)
if len(found) == 1:
return found[0]
raise AmbiguousWorkspace(workspace)
def _identity_key(
installation_id: str,
host_id: str,
native_session_id: str,
runtime_instance: str,
) -> str:
material = "\n".join((installation_id, host_id, native_session_id, runtime_instance))
return hashlib.sha256(material.encode()).hexdigest()
def _session(row: sqlite3.Row) -> TrustedSession:
return TrustedSession(
session_key=row["session_key"],
installation_id=row["installation_id"],
host_id=row["host_id"],
native_session_id=row["native_session_id"],
runtime_instance=row["runtime_instance"],
workspace=row["workspace"],
)
@@ -1,180 +0,0 @@
"""WorkBuddy SessionStart and UserPromptSubmit. No idle turn is started.
The notice stream is the current Evidence revision projection in
`SessionInbox`; `change_outbox` is retained there only as a legacy event source.
"""
import base64
import binascii
import json
import sqlite3
from pathlib import Path
from typing import TextIO
from one_memory.derived_echo import render_reference
from one_memory.session_inbox import CursorRejected, InboxPosition, SessionInbox
from one_memory.session_registry import SessionIdentityError, SessionRegistry
BOUNDARY_EVENTS = frozenset({"SessionStart", "UserPromptSubmit"})
def database_path() -> Path:
from one_memory.config import load_settings
return load_settings().data_dir / "one-memory.sqlite3"
PAGE_LIMIT = 8
def _boundary_session(payload: object, database: Path) -> tuple[str, str] | None:
"""The hook event and the one trusted session it belongs to.
Identity is the payload's own session id. A parent id is not a substitute.
More than one matching trusted session is ambiguous, so nothing is injected.
"""
if not isinstance(payload, dict):
return None
event = payload.get("hook_event_name") or payload.get("hookEventName")
if event not in BOUNDARY_EVENTS:
return None
native = payload.get("session_id") or payload.get("sessionId")
if not isinstance(native, str) or not native or not database.is_file():
return None
matches = SessionRegistry(database).matching_native("workbuddy", native)
if len(matches) != 1:
return None
return str(event), matches[0].session_key
def hook_output(payload: object, database: Path) -> dict[str, object]:
"""The JSON WorkBuddy reads on the next session or prompt boundary.
Reading moves the cursor, so a notice lost before it reaches the host is
not sent again. Kept for installs whose hook predates `--peek`.
"""
silent: dict[str, object] = {"continue": True}
found = _boundary_session(payload, database)
if found is None:
return silent
event, session_key = found
inbox = SessionInbox(database)
page = inbox.pull(session_key, inbox.latest_cursor(session_key), PAGE_LIMIT)
if not page.events:
return silent
return {
"continue": True,
"hookSpecificOutput": {
"hookEventName": event,
"additionalContext": render_reference(page.events),
},
}
def peek_output(payload: object, database: Path) -> dict[str, object]:
"""The same notice without moving the cursor, plus the token that confirms it.
Until `confirm` receives the token the notice is sent again next round.
"""
found = _boundary_session(payload, database)
if found is None:
return {}
event, session_key = found
peeked = SessionInbox(database).peek(session_key, PAGE_LIMIT)
if not peeked.events:
return {}
return {
"hookSpecificOutput": {
"hookEventName": event,
"additionalContext": render_reference(peeked.events),
},
"one_memory_ack": _token(session_key, peeked.position),
}
def confirm(token: str, database: Path) -> bool | None:
"""Move the session's cursor up to what the token's peek handed over.
Raises ValueError for a token this module did not issue.
"""
session_key, position = _untoken(token)
return SessionInbox(database).advance_to(session_key, position)
def _token(session_key: str, position: InboxPosition) -> str:
body = json.dumps(
{
"s": session_key,
"e": position.event_id,
"p": position.permission_epoch,
"h": sorted(position.handed_event_ids),
"r": position.receipt_id,
},
separators=(",", ":"),
)
return base64.urlsafe_b64encode(body.encode()).decode().rstrip("=")
def _untoken(token: str) -> tuple[str, InboxPosition]:
try:
raw = base64.urlsafe_b64decode(token + "=" * (-len(token) % 4))
body = json.loads(raw)
except (binascii.Error, ValueError) as error:
raise ValueError("not a boundary token") from error
if not isinstance(body, dict):
raise ValueError("not a boundary token")
session_key, event_id, epoch, handed, receipt_id = (
body.get("s"),
body.get("e"),
body.get("p"),
body.get("h", []),
body.get("r"),
)
if not isinstance(session_key, str) or not isinstance(event_id, str):
raise ValueError("not a boundary token")
if not isinstance(epoch, int) or isinstance(epoch, bool):
raise ValueError("not a boundary token")
if not isinstance(handed, list) or any(not isinstance(item, str) for item in handed):
raise ValueError("not a boundary token")
if not isinstance(receipt_id, str) or not receipt_id:
raise ValueError("not a boundary token")
return session_key, InboxPosition(event_id, epoch, frozenset(handed), receipt_id)
def _payload(stdin: TextIO) -> object:
try:
return json.loads(stdin.read() or "{}")
except json.JSONDecodeError:
return {}
def write_hook(stream: TextIO, stdin: TextIO) -> int:
"""Print the reply only when it carries a notice.
WorkBuddy reads stdout without `hookSpecificOutput` as context text, so a
bare `{"continue": true}` would reach the model. Empty stdout also continues.
"""
body = hook_output(_payload(stdin), database_path())
if "hookSpecificOutput" in body:
stream.write(f"{json.dumps(body, ensure_ascii=False)}\n")
return 0
def write_peek(stream: TextIO, stdin: TextIO) -> int:
body = peek_output(_payload(stdin), database_path())
if body:
stream.write(f"{json.dumps(body, ensure_ascii=False)}\n")
return 0
def run_confirm(token: str) -> int:
try:
confirm(token, database_path())
except (ValueError, CursorRejected, SessionIdentityError, sqlite3.Error):
return 1
return 0
+5 -15
View File
@@ -63,17 +63,15 @@
| `test_context_memory.py` | `contract` | 覆盖 permissions precede candidate limit and do not leak counts、scope user status and active source isolation、denied scope does not open database 等 60 个场景。 |
| `test_context_raw.py` | `contract` | 覆盖 empty authority and unrequested path do not open database、permissions precede candidate limit and no model needed、explicit source filter never broadens authority 等 37 个场景。 |
| `test_context_raw_scale_benchmark.py` | `contract` | 覆盖 object scale probe has exact positive control and plan、raw scale probe uses fts and has a nonzero positive control 场景。 |
| `test_context_search.py` | `contract` | 覆盖 authorization and requested paths never invoke other lane、empty partial zero unavailable and safe failures、parallel common deadline and finalizers wait for both reads 等 24 个场景。 |
| `test_context_search_both_paths.py` | `integration` | The objective names 'recall' and requires BOTH paths: raw and memory. |
| `test_context_v2_contract_parity.py` | `unit` | Mechanical parity checks for the shared v2 coverage receipt contract. |
| `test_contract_parity.py` | `contract` | Python 的枚举和 TypeScript 里那份手写的联合类型必须一致。 |
| `test_conversation_search.py` | `integration` | Searching imported conversations before any generation model is configured. |
| `test_conversation_seed_ids.py` | `contract` | 覆盖 seed ids read metadata only without hydrating、seed eligibility keeps person and memory content only、latest activity includes nonseed turns and import time fallback 等 13 个场景。 |
| `test_conversation_turn_grouping.py` | `contract` | A proposal and the turn that adopts it must reach HY as one turn. |
| `test_core_ready_before_recovery.py` | `integration` | Readiness never waits for picking up work an earlier run left unfinished. |
| `test_core_test_reporting.py` | `contract` | 覆盖 report writes sanitized machine readable views、file and marker aggregates are stable、classification suggestions never guess unit contract or integration 等 6 个场景。 |
| `test_credential_responsiveness.py` | `integration` | 覆盖 credentials do not block the event loop、disabled automatic processing does not digest onboarding、answering reads credentials off loop 等 5 个场景。 |
| `test_cursor_default_root_windows.py` | `contract` | R-29 剩余:Cursor 默认根路径不能写死成 macOS。 |
| `test_d93_same_process_load.py` | `integration` | D93 test 2: read latency while pure-Python work holds the GIL. |
| `test_daemon_contract.py` | `unit` | A5-01/A5-03: process ownership, and the three grants kept apart. |
| `test_daemon_scheduler.py` | `contract` | A5-02/A5-04: the daemon's scan scheduler and composed health. |
| `test_data_safety.py` | `contract` | A5-05: backup usability/rotation, crash recovery, log retention. |
@@ -108,7 +106,6 @@
| `test_harness_contract_suite.py` | `contract` | A4-09: one contract, ten harnesses. |
| `test_health.py` | `contract` | 覆盖 a backfill that fails says so rather than disappearing、a failed catch up leaves a traceback behind、configured model without content consent does not start backfill 等 15 个场景。 |
| `test_history_listing_rotation.py` | `contract` | 覆盖 history listing alternates started and untouched files、history listing drains whichever cohort is longer 场景。 |
| `test_home_summary.py` | `integration` | 覆盖 summary counts beyond page limit and excludes ineligible rows、summary revocation excludes whole mixed memory not only citation、latest is bounded and other users revocations do not hide local data 等 6 个场景。 |
| `test_host_capabilities.py` | `unit` | A plugin does not become realtime on every host or every unknown version. |
| `test_hourly_c1_peak_replay_contract.py` | `contract` | Guard the package peak-replay probe's error accounting. |
| `test_hourly_c1_recall_stage_probe_contract.py` | `contract` | Guard the packaged recall stage probe's command and error semantics. |
@@ -159,8 +156,6 @@
| `test_memory_content_import.py` | `integration` | 覆盖 existing memory preview keeps unknown author and exact original、invalid existing memory records never leave a partial upload、memory content requires explicit mode and rejects vendor guessing 等 5 个场景。 |
| `test_memory_evidence_freshness.py` | `contract` | 新鲜度拆除后:读路径恒为 current;证据修订 / 畸形 proof / segment ref 不再藏记忆或 503。 |
| `test_memory_freshness_integration_review.py` | `integration` | Independent synthetic read-path review; no lifespan, model or user database. |
| `test_memory_governance_api.py` | `integration` | 覆盖 memory governance api preserves evidence and reports cleanup、memory validation does not echo content 场景。 |
| `test_memory_graph_api.py` | `integration` | 覆盖 graph labels where every edge came from、a created memory is on the map with its source links、drawing the map does not stop the app answering 等 12 个场景。 |
| `test_memory_notes.py` | `integration` | 覆盖 empty library can save a note without a matter or configured model、note requires authentication explicit scope and nonempty content、saved note evidence returns raw and consent reference from same record 等 6 个场景。 |
| `test_memory_qa.py` | `contract` | 覆盖 search terms keep meaningful single chinese character、streamed answer emits recall then incremental grounded text、governance change during generation stops the old answer 等 14 个场景。 |
| `test_memory_qa_api.py` | `integration` | 覆盖 ask stream endpoint emits recall deltas and final、ask stream endpoint requires the local token 场景。 |
@@ -213,19 +208,16 @@
| `test_probe_payment_required.py` | `contract` | 覆盖 http 402 without a useful body is still understood as unpaid、a wrong address is still a missing model not an unpaid one 场景。 |
| `test_processing_requests.py` | `contract` | 覆盖 concurrent claim only one snapshot wins、concurrent create same grant has one id and profile、concurrent different profile never reuses another scope 等 14 个场景。 |
| `test_quality_acceptance.py` | `contract` | 覆盖 acceptance pack is deterministic and scores personal boundary、acceptance score fails missing personal memory 场景。 |
| `test_raw_search_without_model.py` | `integration` | Search imported conversations with no generative model configured at all. |
| `test_raw_vector_scale_benchmark.py` | `contract` | 覆盖 vector scale probe uses real 512 dimension filtered search 场景。 |
| `test_read_path_freshness_batch.py` | `contract` | The row-by-row read check must not scale its query count with the page. |
| `test_reader_app.py` | `integration` | The Reader process stays read-only and never blocks a read on its side effects. |
| `test_reader_conversation_hit.py` | `contract` | A search hit carries who wrote the row it matched. |
| `test_real_load_metrics.py` | `contract` | Run the real access layer under a production-like combined load. |
| `test_recall.py` | `contract` | 覆盖 recall is confirmed only grounded and scope bound、wildcard scope means every scope just like it does in qa、recall returns empty instead of unrelated fallback 等 10 个场景。 |
| `test_recall_during_import_probe_contract.py` | `contract` | Guard the source-level recall-during-import probe's sampling semantics. |
| `test_recall_excluded_sources.py` | `contract` | D94: an empty result names the sources this host may not read. |
| `test_recall_fusion_live.py` | `integration` | RCL-04 against a real HY instance instead of a stand-in. |
| `test_recall_library_scale.py` | `contract` | 覆盖 recall reaches a memory that is not among the newest 500、semantic recall reaches a memory outside the newest 500、recall still works just below the boundary 等 5 个场景。 |
| `test_recall_log_probe_exclusion.py` | `contract` | A-21:R-10 一旦引入召回日志,「探针不污染」必须自动变红而不是静默失效。 |
| `test_recall_probe.py` | `contract` | The self-check query, and what it must not touch on its way through. |
| `test_recall_raw_fallback.py` | `integration` | MemHub_recall degrades to the imported conversations instead of answering empty. |
| `test_recall_revoked_candidate_crowding.py` | `contract` | Revoked-source rows must not crowd a readable recall hit out of the page. |
| `test_recall_scope_candidate_budget.py` | `contract` | 覆盖 other scopes cannot consume the recall candidate budget、internal wildcard still reports real candidate truncation、workspace resolution does not remove shared results 等 9 个场景。 |
| `test_recall_scope_control.py` | `integration` | 覆盖 external scope cannot override shared or host workspace boundary、scope only mutation is versioned without changing status or reindexing 场景。 |
@@ -254,6 +246,7 @@
| `test_sdk_data_plane_migration.py` | `contract` | SDK-authority migration ledger, aliases and atomic cutover marker. |
| `test_sdk_memory_qa.py` | `contract` | 覆盖 sdk ask propagates sdk search failure without legacy fallback、sdk ask revalidates memory before returning extract、sdk ask uses native sdk memory without governance fields 场景。 |
| `test_sdk_migration_bundle.py` | `contract` | SDK-authority cutover requires a verified management + SDK data-plane bundle. |
| `test_sdk_migration_log.py` | `integration` | `sdk-migration.log` explains an unfinished upgrade, and a legacy home is never served. |
| `test_sdk_verbatim_fallback.py` | `contract` | D98: the SDK's verbatim fallback must stay off. |
| `test_search_without_profile.py` | `contract` | What 'semantic_search' should do once the profile that filled the store is gone. |
| `test_secondary_source_adapters.py` | `contract` | 覆盖 pi and qwenwork keep tool and sidechain content out of the persons words、qwen code reads only the transcript beside its bookkeeping、qwen native parts preserve visible text not reasoning or tools 等 22 个场景。 |
@@ -263,9 +256,6 @@
| `test_sedimentation_switch.py` | `contract` | Stopping model-driven sedimentation without losing the configuration. |
| `test_service_level_pressure_acceptance.py` | `contract` | 覆盖 workbuddy recall remains available during 100k import 场景。 |
| `test_service_levels.py` | `contract` | 覆盖 background never consumes the realtime reservation、realtime is next for the same source after background yields、import history gets one slice after nine realtime admissions 等 8 个场景。 |
| `test_session_inbox.py` | `contract` | A session inbox is not a shared mailbox and a pull is not adoption. |
| `test_session_lease.py` | `contract` | A session file on disk is not proof that the session is online. |
| `test_session_registry.py` | `contract` | Trusted session identity is not a workspace path. |
| `test_snapshot_adapter.py` | `contract` | Selection-only binding reopens checksum-verified frozen copies. |
| `test_snapshot_pool.py` | `contract` | A whole-directory snapshot must not hold a thread everything else is waiting for. |
| `test_soak_probe_effectiveness.py` | `integration` | The soak probe must observe the load it reports on. |
@@ -305,13 +295,13 @@
| `test_ui_session_api.py` | `integration` | 覆盖 ui session exchange sets httponly strict cookie、ui session requires bearer for exchange、ui session reports missing web assets 场景。 |
| `test_unanswered_turn.py` | `contract` | BUG-06: which conversations end on a question nobody answered. |
| `test_unified_write_entry_wiring.py` | `integration` | The composition root routes every Evidence-first entry through one queue. |
| `test_v2_authority_and_call_events.py` | `integration` | v2 is the SDK data plane: one authority gate, and host calls are history. |
| `test_vendor_sqlite_audit.py` | `contract` | A1-08: the audit that says which SQLite is MemHub's and which is a vendor's. |
| `test_visibility_background.py` | `integration` | The visibility background task: when it runs, and that it says so. |
| `test_whole_turn_hy_input.py` | `contract` | D74: a turn reaches HY whole; only what HY will read counts toward the ceiling. |
| `test_windows_process_control.py` | `contract` | R-29:Windows 上 'os.kill' 不是发信号,是无条件杀进程。 |
| `test_workbuddy_boundary.py` | `contract` | WorkBuddy only injects on SessionStart and UserPromptSubmit. |
| `test_workbuddy_parsing.py` | `contract` | 覆盖 automation preserves query without claiming fresh human authorship、automation context attributes do not depend on order or quote style、quoted full automation envelope is not scheduled provenance 等 17 个场景。 |
| `test_workspace_recall.py` | `contract` | Workspace memories were unreachable from every host; these pin the way in. |
| `test_write_queue_lease_renewal.py` | `contract` | Rule 8: a lease must outlive the work it covers, and be renewed while it runs. |
共 309 个测试文件。
共 299 个测试文件。
@@ -14,8 +14,8 @@ from pathlib import Path
import pytest
PRIMARY_MARKERS = ("unit", "contract", "integration", "performance", "live")
EXPECTED_TEST_FILE_COUNT = 309
EXPECTED_TEST_FILES_SHA256 = "ef6165c249971f5a2ab71a0c49f6d859c8c0f3612e3f749fba0acce0eab513ea"
EXPECTED_TEST_FILE_COUNT = 299
EXPECTED_TEST_FILES_SHA256 = "7b6afc4f9da9d5ba2829639f38d964f89c78bb71d9cadae3dba9b4aad91e3a2c"
# Pure transformations, validation models, parity checks and state machines.
# This list is intentionally small: uncertain files default to contract.
@@ -51,23 +51,22 @@ INTEGRATION_FILES = frozenset(
{
"test_auth.py",
"test_backup_endpoint.py",
"test_recall_raw_fallback.py",
"test_core_ready_before_recovery.py",
"test_backup_restore_api.py",
"test_c1_import_via_api_path.py",
"test_capture_deadline.py",
"test_chat_exports.py",
"test_cli_lifecycle.py",
"test_connection_lifetime.py",
"test_context_search_both_paths.py",
"test_conversation_search.py",
"test_credential_responsiveness.py",
"test_d93_same_process_load.py",
"test_diagnostic_trace.py",
"test_diagnostic_trace_api.py",
"test_external_installation_permissions.py",
"test_fault_matrix.py",
"test_file_import_operations.py",
"test_graph_width_endpoint.py",
"test_home_summary.py",
"test_import_api.py",
"test_live_capture.py",
"test_live_watch_grants.py",
@@ -78,8 +77,6 @@ INTEGRATION_FILES = frozenset(
"test_measure_c1_loop_blocking_contract.py",
"test_memory_content_import.py",
"test_memory_freshness_integration_review.py",
"test_memory_governance_api.py",
"test_memory_graph_api.py",
"test_memory_notes.py",
"test_memory_qa_api.py",
"test_memory_server_durable_queue.py",
@@ -94,13 +91,11 @@ INTEGRATION_FILES = frozenset(
"test_onboarding_profile.py",
"test_parent_watchdog.py",
"test_policy_api_wiring.py",
"test_raw_search_without_model.py",
"test_reader_app.py",
"test_recall_fusion_live.py",
"test_recall_scope_control.py",
"test_retrieval_lifespan.py",
"test_runtime.py",
"test_secret_store_error_code_api.py",
"test_sdk_migration_log.py",
"test_secret_store_error_codes.py",
"test_secure_storage.py",
"test_soak_probe_effectiveness.py",
@@ -114,6 +109,7 @@ INTEGRATION_FILES = frozenset(
"test_source_sync_permissions.py",
"test_ui_session_api.py",
"test_unified_write_entry_wiring.py",
"test_v2_authority_and_call_events.py",
"test_visibility_background.py",
}
)
@@ -1,194 +0,0 @@
"""D93 test 3: inventory every database write a Reader-bound read route makes.
Plan `读写隔离方案-20261009.html` section 5 (B) and section 8 row 3. The Reader
opens the library `mode=ro` + `query_only`, so any write a read route makes
today will fail there. Instead of a manual audit, run the existing route tests
with this plugin loaded; it records, per route, every INSERT / UPDATE / DELETE /
DDL that happens while that route is being served, with the one_memory frames
that issued it. Recording only, nothing is blocked, so one run lists them all.
bash /private/tmp/wb-d76-pytest.sh -p tests.d93_side_effects_plugin \
tests/test_home_summary.py tests/test_memory_library_api.py ...
The report goes to $ONE_MEMORY_D93_SIDE_EFFECTS (default
`d93-side-effects.json` in the pytest root) and a summary is printed at the end.
Writes outside a request (fixture setup, background tasks) are not counted.
"""
from __future__ import annotations
import contextlib
import contextvars
import json
import os
import sqlite3
import traceback
from collections import defaultdict
from pathlib import Path
#: The routes the Reader takes over (plan section 5, B, reader_app.py row).
READER_ROUTES = {
("GET", "/api/v1/home-summary"),
("GET", "/api/v1/memory-library"),
("GET", "/api/v1/memories"),
("GET", "/api/v1/memories/{memory_id}"),
("GET", "/api/v1/memories/graph"),
("POST", "/api/v1/context/search"),
("GET", "/api/v1/context/search"),
("POST", "/api/v1/recall"),
("GET", "/api/v1/conversations/search"),
("POST", "/api/v1/conversations/search"),
("GET", "/api/v1/health"),
("GET", "/health"),
}
_WRITES = {
sqlite3.SQLITE_INSERT: "INSERT",
sqlite3.SQLITE_UPDATE: "UPDATE",
sqlite3.SQLITE_DELETE: "DELETE",
sqlite3.SQLITE_CREATE_TABLE: "CREATE TABLE",
sqlite3.SQLITE_CREATE_INDEX: "CREATE INDEX",
sqlite3.SQLITE_CREATE_TRIGGER: "CREATE TRIGGER",
sqlite3.SQLITE_DROP_TABLE: "DROP TABLE",
sqlite3.SQLITE_ALTER_TABLE: "ALTER TABLE",
}
_current: contextvars.ContextVar[tuple[str, str] | None] = contextvars.ContextVar(
"d93_route", default=None
)
_served: dict[tuple[str, str], dict[int, int]] = defaultdict(lambda: defaultdict(int))
_found: dict[tuple, int] = defaultdict(int)
def _frames() -> tuple[str, ...]:
"""Last product frames; a leading `[test]` marks writes a test's stand-in made."""
kept = []
from_test = False
for frame in traceback.extract_stack()[:-2]:
name = frame.filename.replace(os.sep, "/")
if "/apps/core/tests/" in name and not name.endswith("d93_side_effects_plugin.py"):
from_test = True
if "/one_memory/" in name and "/tests/" not in name:
kept.append(f"{name.split('/one_memory/', 1)[1]}:{frame.lineno} {frame.name}")
return (("[test]",) if from_test else ()) + tuple(kept[-4:])
#: Self-check: record every route, so a known write route (POST /api/v1/memories)
#: proves the recorder sees writes at all.
_ALL = os.environ.get("ONE_MEMORY_D93_ALL_ROUTES") == "1"
def _authorizer(action, arg1, arg2, db_name, _trigger):
route = _current.get()
if route is not None and (_ALL or route in READER_ROUTES) and action in _WRITES:
table = arg1 if action != sqlite3.SQLITE_ALTER_TABLE else arg2
if db_name != "temp" and not str(table).startswith("sqlite_"):
_found[(route, _WRITES[action], str(table), _frames())] += 1
return sqlite3.SQLITE_OK
_real_connect = sqlite3.connect
def _connect(*args, **kwargs):
connection = _real_connect(*args, **kwargs)
with contextlib.suppress(Exception):
connection.set_authorizer(_authorizer)
return connection
class _RouteTag:
def __init__(self, app):
self.app = app
async def __call__(self, scope, receive, send):
if scope.get("type") != "http":
return await self.app(scope, receive, send)
status = {"code": 0}
async def sending(message):
if message.get("type") == "http.response.start":
status["code"] = message.get("status", 0)
await send(message)
holder: dict[str, tuple[str, str]] = {}
token = _current.set(None)
# The route template is only known after routing; resolve it up front
# from the app's routes so writes made during the handler carry it.
for route in getattr(self._root(scope), "routes", ()):
match = getattr(route, "matches", None)
if match is None:
continue
kind, _ = match(scope)
if kind.name == "FULL":
holder["route"] = (scope["method"], getattr(route, "path", scope["path"]))
break
key = holder.get("route", (scope["method"], scope["path"]))
_current.set(key)
try:
await self.app(scope, receive, sending)
finally:
_current.reset(token)
if key in READER_ROUTES:
_served[key][status["code"]] += 1
def _root(self, scope):
return scope.get("app")
def pytest_configure(config):
sqlite3.connect = _connect
# Pooled connections are handed back through `_reset`, which clears any
# authorizer; put ours back so the next checkout is still recorded.
from one_memory import database_access
real_reset = database_access._reset
def _reset(connection, *, reading):
clean = real_reset(connection, reading=reading)
with contextlib.suppress(Exception):
connection.set_authorizer(_authorizer)
return clean
database_access._reset = _reset
from one_memory import api
original = api.create_app
def create_app(*args, **kwargs):
application = original(*args, **kwargs)
application.add_middleware(_RouteTag)
return application
api.create_app = create_app
def pytest_terminal_summary(terminalreporter):
report = {
"served": {
f"{method} {path}": dict(codes) for (method, path), codes in sorted(_served.items())
},
"writes": [
{
"route": f"{route[0]} {route[1]}",
"op": op,
"table": table,
"frames": list(frames),
"count": count,
}
for (route, op, table, frames), count in sorted(_found.items(), key=str)
],
}
target = Path(
os.environ.get("ONE_MEMORY_D93_SIDE_EFFECTS", str(Path.cwd() / "d93-side-effects.json"))
)
target.write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
write = terminalreporter.write_line
write(f"D93 side effects: {len(report['writes'])} write sites -> {target}")
for name, codes in report["served"].items():
write(f" served {name}: {codes}")
for item in report["writes"]:
write(f" {item['route']}: {item['op']} {item['table']} x{item['count']}")
for frame in item["frames"]:
write(f" {frame}")
-34
View File
@@ -55,37 +55,3 @@ def test_browser_origin_is_checked_in_addition_to_token() -> None:
)
== 200
)
async def internal_request(
path: str,
*,
token: str | None,
method: str = "GET",
) -> int:
settings = Settings(api_token=SecretStr("test-token"))
app = create_app(settings)
headers = {}
if token:
headers["Authorization"] = f"Bearer {token}"
async with AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
) as client:
response = await client.request(method, path, headers=headers, json={})
return response.status_code
def test_reader_internal_endpoints_require_local_token() -> None:
for path in (
"/internal/semantic-ids?query=anything",
"/internal/graph-snapshot",
):
assert asyncio.run(internal_request(path, token=None)) == 401
assert asyncio.run(internal_request(path, token="test-token")) == 200
assert asyncio.run(internal_request("/internal/call-event", token=None, method="POST")) == 401
assert (
asyncio.run(internal_request("/internal/call-event", token="test-token", method="POST"))
== 200
)
@@ -1,6 +1,7 @@
import asyncio
import sqlite3
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
@@ -44,6 +45,8 @@ def _database(path, *, value: str) -> None:
asyncio.run(SqliteMigrationStore(path).record(Migration(34, "synthetic", lambda _: None)))
# A serving Core refuses a legacy home outright, so this one starts on SDK.
@pytest.mark.fresh_sdk
def test_restore_endpoint_refuses_when_runtime_owns_the_core(tmp_path) -> None:
monkeypatch_app = create_app(_settings(tmp_path), manage_runtime=True)
@@ -1,104 +1,10 @@
from datetime import UTC, datetime
from one_memory.api import ConversationSearchHit, ConversationSearchResponse
from one_memory.call_event_recorder import (
ask_call_event,
context_call_event,
conversation_call_event,
)
from one_memory.context_models import ContextPath, ContextPaths, ContextSearchResponse
from one_memory.external_memory_access import ExternalMemoryAccess
from one_memory.call_event_recorder import ask_call_event
from one_memory.memory_governance import GovernedMemory, MemoryKind, MemoryOrigin, MemoryStatus
from one_memory.memory_qa import MemoryAnswer
def _paths(memory: str, raw: str) -> ContextPaths:
lane = dict(elapsed_ms=1.0, returned_count=0, truncated=False)
return ContextPaths(
raw=ContextPath(status=raw, **lane),
memory=ContextPath(status=memory, **lane),
)
def _context(memory: str, raw: str) -> ContextSearchResponse:
return ContextSearchResponse(
request_id="11111111-1111-4111-8111-111111111111",
state="partial",
paths=_paths(memory, raw),
items=(),
used_characters=0,
truncated=False,
total_ms=3.5,
)
def test_denied_memory_lane_is_attributed_to_conversations():
event = context_call_event(_context("not_authorized", "ok"), None)
assert event.tool == "om_conversations"
def test_requested_memory_lane_stays_recall():
event = context_call_event(_context("empty", "not_authorized"), None)
assert event.tool == "om_recall"
def test_zero_result_timeout_is_recorded_separately_from_unavailable() -> None:
lane = dict(elapsed_ms=750.0, returned_count=0)
result = ContextSearchResponse(
request_id="11111111-1111-4111-8111-111111111111",
state="partial",
paths=ContextPaths(
raw=ContextPath(
status="degraded",
reason="timeout_budget",
truncated=True,
truncated_by=("timeout_budget",),
**lane,
),
memory=ContextPath(status="empty", truncated=False, **lane),
),
items=(),
used_characters=0,
truncated=True,
total_ms=1500,
)
assert context_call_event(result, None).state == "timed_out"
def test_conversation_search_records_counts_without_query_or_excerpt():
result = ConversationSearchResponse(
hits=(
ConversationSearchHit(
evidence_id="e1",
conversation_id="c1",
source_id="cursor",
excerpt="SECRET_EXCERPT",
),
ConversationSearchHit(
evidence_id="e2",
conversation_id="c1",
source_id="cursor",
excerpt="SECRET_EXCERPT_2",
),
)
)
access = ExternalMemoryAccess(
installation_id="00000000-0000-4000-8000-000000000001",
revision=1,
scopes=frozenset({"all-apps", "source:cursor"}),
tools=frozenset({"om_conversations"}),
host_id="codex",
)
event = conversation_call_event(result, access, 12.0)
encoded = event.model_dump_json()
assert event.tool == "om_conversations"
assert event.state == "grounded"
assert event.raw_count == 1 and event.memory_count == 0 and not event.references
assert event.host_id == "codex"
assert "SECRET" not in encoded
def test_ask_records_memory_refs_without_answer():
memory = GovernedMemory(
memory_id="m1",
@@ -8,7 +8,7 @@ from one_memory.external_memory_access import require_external_endpoint
def request(
tools, scopes=("all-apps", "source:cursor"), method="POST", path="/api/v1/context/search"
tools, scopes=("all-apps", "source:cursor"), method="POST", path="/api/v2/context/search"
):
headers = {
"x-one-memory-read-context": "external",
@@ -44,9 +44,13 @@ def test_other_capabilities_do_not_enable_composite_read(tools):
@pytest.mark.parametrize(
"method,path",
[
("GET", "/api/v1/context/search"),
("GET", "/api/v2/context/search"),
("POST", "/api/v1/context/search"),
("POST", "/api/v1/memory-library"),
("GET", "/api/v1/conversations/search"),
("GET", "/api/v1/conversations/evidence-id"),
("POST", "/api/v1/memories/ask"),
("POST", "/api/v1/boundary/peek"),
],
)
def test_composite_read_does_not_open_other_routes(method, path):
@@ -59,15 +63,3 @@ def test_composite_read_does_not_open_other_routes(method, path):
def test_legacy_recall_still_requires_its_own_tool():
with pytest.raises(HTTPException):
require_external_endpoint(request(["om_conversations"], path="/api/v1/recall"))
def test_legacy_raw_route_still_requires_raw_source_authority():
with pytest.raises(HTTPException):
require_external_endpoint(
request(
["om_conversations"],
scopes=("all-apps",),
method="GET",
path="/api/v1/conversations/search",
)
)
@@ -1,547 +0,0 @@
import asyncio
import time
from datetime import UTC, datetime
import pytest
from pydantic import ValidationError
from one_memory.context_models import (
ContextLaneResult,
ContextMessage,
ContextSearchRequest,
ContextSearchResponse,
MemoryContextItem,
RawContextItem,
)
from one_memory.context_search import search_context
def memory(identity="m", text="memory", evidence=("e",)):
return MemoryContextItem(
memory_id=identity,
revision=1,
content=text,
memory_kind="chat",
updated_at=datetime.now(UTC),
evidence_ids=evidence,
)
def raw(identity="e", text="raw"):
return RawContextItem(
source_id="cursor",
conversation_id="c",
anchor_evidence_id=identity,
messages=(
ContextMessage(
evidence_id=identity,
revision=2,
role="user",
written_by="person",
created_at=None,
text=text,
),
),
)
def reader(*items, status="ok", reason=None, truncated_by=()):
async def read(deadline):
assert deadline > time.monotonic()
return ContextLaneResult(
items=items, status=status, reason=reason, truncated_by=truncated_by
)
return read
async def finalize(kind, items, deadline):
assert deadline > time.monotonic()
assert all(item.kind == kind for item in items)
return items
def run(request=None, raw_reader=None, memory_reader=None, finalizer=finalize):
return asyncio.run(
search_context(
request or ContextSearchRequest(query="q"),
raw_reader=raw_reader,
memory_reader=memory_reader,
finalizer=finalizer,
)
)
def test_authorization_and_requested_paths_never_invoke_other_lane():
async def forbidden(deadline):
pytest.fail("unrequested lane invoked")
result = run(
ContextSearchRequest(query="q", kinds=("memory",)),
raw_reader=forbidden,
memory_reader=reader(memory()),
)
assert result.paths.raw.status == "not_requested"
assert result.state == "grounded"
denied = run(memory_reader=reader(status="empty"))
assert denied.paths.raw.status == "not_authorized"
assert denied.paths.raw.reason == "permission_required"
assert denied.state == "partial" and denied.items == ()
def test_empty_partial_zero_unavailable_and_safe_failures():
assert (
run(raw_reader=reader(status="empty"), memory_reader=reader(status="empty")).state
== "empty"
)
async def fail(deadline):
raise RuntimeError("PRIVATE_SECRET_SQL_PATH")
result = run(raw_reader=fail, memory_reader=fail)
assert result.state == "unavailable"
assert "PRIVATE" not in result.model_dump_json()
assert result.paths.raw.reason == "read_unavailable"
partial = run(raw_reader=fail, memory_reader=reader(status="empty"))
assert partial.state == "partial" and partial.used_characters == 0
lexical_only = run(
raw_reader=reader(status="degraded", reason="lexical_only"),
memory_reader=reader(status="empty"),
)
assert lexical_only.state == "empty"
assert lexical_only.paths.raw.status == "empty"
def test_parallel_common_deadline_and_finalizers_wait_for_both_reads():
async def scenario():
entered = set()
deadlines = []
finished = set()
async def read(kind, deadline):
entered.add(kind)
deadlines.append(deadline)
while len(entered) != 2:
await asyncio.sleep(0)
await asyncio.sleep(0.005 if kind == "raw" else 0.015)
finished.add(kind)
return ContextLaneResult(status="ok", items=(raw() if kind == "raw" else memory(),))
async def final(kind, items, deadline):
assert finished == {"raw", "memory"}
deadlines.append(deadline)
return () if kind == "raw" else items
result = await search_context(
ContextSearchRequest(query="q"),
raw_reader=lambda d: read("raw", d),
memory_reader=lambda d: read("memory", d),
finalizer=final,
)
assert len(set(deadlines)) == 1
assert [i.kind for i in result.items] == ["memory"]
assert result.paths.raw.reason == "evidence_changed"
assert result.state == "partial"
asyncio.run(scenario())
def test_stage_timings_separate_lanes_revalidation_fusion_and_total():
async def scenario():
observed = []
async def delayed(item, delay):
await asyncio.sleep(delay)
return ContextLaneResult(status="ok", items=(item,))
async def delayed_final(kind, items, deadline):
await asyncio.sleep(0.004 if kind == "raw" else 0.008)
return items
result = await search_context(
ContextSearchRequest(query="q"),
raw_reader=lambda _deadline: delayed(raw(), 0.006),
memory_reader=lambda _deadline: delayed(memory(), 0.012),
finalizer=delayed_final,
timing_observer=observed.append,
)
assert result.state == "grounded" and len(observed) == 1
timing = observed[0]
assert set(timing) == {
"raw_ms",
"memory_ms",
"raw_revalidation_ms",
"memory_revalidation_ms",
"fusion_ms",
"total_ms",
}
assert 4 <= timing["raw_ms"] < timing["memory_ms"]
assert 2 <= timing["raw_revalidation_ms"] < timing["memory_revalidation_ms"]
assert timing["fusion_ms"] >= 0
assert timing["total_ms"] >= timing["memory_ms"] + timing["memory_revalidation_ms"]
asyncio.run(scenario())
def test_finalizer_failure_never_releases_old_items():
async def fail(kind, items, deadline):
raise ValueError("PRIVATE")
result = run(raw_reader=reader(raw()), memory_reader=reader(memory()), finalizer=fail)
assert result.items == () and result.state == "unavailable"
assert "PRIVATE" not in result.model_dump_json()
def test_shared_count_text_fairness_and_anchor_metadata():
result = run(
ContextSearchRequest(query="q", character_budget=200, limit=2),
raw_reader=reader(raw(text="R" * 1000), raw("e2")),
memory_reader=reader(memory(text="M" * 1000), memory("m2")),
)
assert [i.kind for i in result.items] == ["raw", "memory"]
assert result.used_characters == 200
assert len(result.items[0].messages[0].text) == 100
assert len(result.items[1].content) == 100
assert result.items[0].anchor_evidence_id == "e"
assert result.items[0].messages[0].revision == 2
assert result.items[0].messages[0].written_by == "person"
for path in (result.paths.raw, result.paths.memory):
assert path.returned_count == 1
assert set(path.truncated_by) == {"count_budget", "character_budget"}
assert all(i.truncated for i in result.items)
assert result.state == "grounded" and result.adoption == "unknown"
assert result.paths.raw.status == "ok"
assert result.paths.memory.status == "ok"
ContextSearchResponse.model_validate_json(result.model_dump_json())
def test_dedup_preserves_cross_kind_evidence_relations():
item = raw()
result = run(
raw_reader=reader(item, item),
memory_reader=reader(memory(), memory(), memory("other", evidence=("e", "e"))),
)
assert len(result.items) == 3
assert result.paths.raw.returned_count == 1
assert result.paths.memory.returned_count == 2
assert result.items[-1].evidence_ids == ("e",)
def test_timeout_cancels_tasks_and_late_results_are_not_released():
async def scenario():
cancelled = []
async def slow(deadline):
try:
await asyncio.sleep(10)
except asyncio.CancelledError:
cancelled.append(True)
return ContextLaneResult(status="ok", items=(raw(text="late"),))
result = await search_context(
ContextSearchRequest(query="q", timeout_ms=50),
raw_reader=slow,
memory_reader=reader(memory()),
finalizer=finalize,
)
assert cancelled == [True]
assert result.items == ()
assert result.paths.raw.reason == "timeout_budget"
assert result.paths.memory.reason == "timeout_budget"
assert not [t for t in asyncio.all_tasks() if t is not asyncio.current_task()]
asyncio.run(scenario())
def test_caller_cancellation_propagates_and_cleans_readers():
async def scenario():
started = asyncio.Event()
exits = []
async def slow(deadline):
started.set()
try:
await asyncio.sleep(10)
finally:
exits.append(1)
task = asyncio.create_task(
search_context(
ContextSearchRequest(query="q"),
raw_reader=slow,
memory_reader=slow,
finalizer=finalize,
)
)
await started.wait()
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
assert len(exits) == 2
assert not [t for t in asyncio.all_tasks() if t is not asyncio.current_task()]
asyncio.run(scenario())
def test_invalid_lane_shape_and_wrong_kind_fail_closed():
async def invalid(deadline):
return {"status": "ok", "items": [], "private_diagnostic": "secret"}
assert run(raw_reader=invalid, memory_reader=reader(memory())).paths.raw.status == "unavailable"
assert (
run(raw_reader=reader(memory()), memory_reader=reader(memory())).paths.raw.status
== "unavailable"
)
with pytest.raises(ValidationError):
ContextSearchRequest(query="q", limit=21)
with pytest.raises(ValidationError):
ContextSearchRequest(query="q", character_budget=20001)
def test_expired_entry_deadline_never_reads_or_releases_content():
async def forbidden(*args):
pytest.fail("expired deadline must not invoke work")
result = asyncio.run(
search_context(
ContextSearchRequest(query="q"),
raw_reader=forbidden,
memory_reader=forbidden,
finalizer=forbidden,
deadline=time.monotonic() - 1,
)
)
assert result.state == "partial" and result.items == ()
assert result.paths.raw.reason == result.paths.memory.reason == "timeout_budget"
def test_entry_deadline_is_not_reset_and_request_timeout_caps_it():
async def scenario():
observed = []
async def read(deadline):
observed.append(deadline)
return ContextLaneResult(status="ok", items=(memory(),))
entry = time.monotonic() + 0.1
await search_context(
ContextSearchRequest(query="q"),
raw_reader=None,
memory_reader=read,
finalizer=finalize,
deadline=entry,
)
assert observed == [entry]
started = time.monotonic()
await search_context(
ContextSearchRequest(query="q", timeout_ms=50),
raw_reader=None,
memory_reader=read,
finalizer=finalize,
deadline=started + 10,
)
assert observed[-1] < started + 0.1
asyncio.run(scenario())
def test_finalizer_timeout_cannot_release_previously_read_items():
async def scenario():
cancelled = []
async def slow(kind, items, deadline):
try:
await asyncio.sleep(10)
except asyncio.CancelledError:
cancelled.append(kind)
return items
result = await search_context(
ContextSearchRequest(query="q", timeout_ms=50),
raw_reader=reader(raw()),
memory_reader=reader(memory()),
finalizer=slow,
)
assert set(cancelled) == {"raw", "memory"}
assert result.items == () and result.state == "unavailable"
assert result.paths.memory.truncated_by == ("timeout_budget",)
assert not [t for t in asyncio.all_tasks() if t is not asyncio.current_task()]
asyncio.run(scenario())
def test_overlapping_raw_windows_keep_each_anchor_and_cross_kind_reference():
first = raw("e1")
second = raw("e2")
first = first.model_copy(update={"messages": first.messages + second.messages})
result = run(
raw_reader=reader(first, second), memory_reader=reader(memory(evidence=("e1", "e2")))
)
windows = [item for item in result.items if item.kind == "raw"]
assert [item.anchor_evidence_id for item in windows] == ["e1", "e2"]
assert [message.evidence_id for item in windows for message in item.messages] == ["e1", "e2"]
assert next(item for item in result.items if item.kind == "memory").evidence_ids == ("e1", "e2")
def test_twenty_item_and_twenty_thousand_character_caps():
result = run(
ContextSearchRequest(query="q", limit=20, character_budget=20000),
raw_reader=reader(*(raw(f"e{i}", "r" * 20000) for i in range(21))),
memory_reader=reader(*(memory(f"m{i}", "m" * 20000) for i in range(21))),
)
assert len(result.items) == 20 and result.used_characters == 20000
assert result.paths.raw.returned_count == result.paths.memory.returned_count == 10
assert (
sum(
len(i.content) if i.kind == "memory" else sum(len(m.text) for m in i.messages)
for i in result.items
)
== 20000
)
def test_caller_cancel_during_finalization_cleans_all_tasks():
async def scenario():
started = asyncio.Event()
finished = []
async def final(kind, items, deadline):
started.set()
try:
await asyncio.sleep(10)
finally:
finished.append(kind)
task = asyncio.create_task(
search_context(
ContextSearchRequest(query="q"),
raw_reader=reader(raw()),
memory_reader=reader(memory()),
finalizer=final,
)
)
await started.wait()
task.cancel()
with pytest.raises(asyncio.CancelledError):
await task
assert set(finished) == {"raw", "memory"}
assert not [t for t in asyncio.all_tasks() if t is not asyncio.current_task()]
asyncio.run(scenario())
def test_invalid_finalizer_output_and_constructed_models_fail_closed():
async def wrong(kind, items, deadline):
return (memory("unselected"),)
assert run(memory_reader=reader(memory()), finalizer=wrong).items == ()
invalid = memory().model_copy(update={"content": "x" * 20001})
assert run(memory_reader=reader(invalid)).state == "unavailable"
with pytest.raises(ValidationError):
run(ContextSearchRequest.model_construct(query="q", limit=100))
def test_duplicate_memory_identity_keeps_combined_evidence_relations():
result = run(memory_reader=reader(memory(evidence=("a",)), memory(evidence=("b",))))
assert result.items[0].evidence_ids == ("a", "b")
def test_one_slow_finalizer_exhausts_release_deadline_for_both_lanes():
async def final(kind, items, deadline):
if kind == "raw":
await asyncio.sleep(10)
return items
result = run(
ContextSearchRequest(query="q", timeout_ms=50),
raw_reader=reader(raw()),
memory_reader=reader(memory()),
finalizer=final,
)
assert result.items == ()
assert result.state == "partial"
def test_multiple_raw_messages_clipped_without_losing_anchor_or_authorship():
item = raw("anchor", "A" * 20000)
item = item.model_copy(
update={
"messages": (
raw("before", "B" * 20000).messages[0],
item.messages[0],
raw("after", "C" * 20000).messages[0],
)
}
)
result = run(
ContextSearchRequest(query="q", character_budget=200),
raw_reader=reader(item),
memory_reader=reader(memory(text="M" * 20000)),
)
emitted = result.items[0]
assert emitted.anchor_evidence_id == "anchor"
assert [m.evidence_id for m in emitted.messages] == ["before", "anchor", "after"]
assert all(
m.truncated and m.written_by == "person" and m.revision == 2 for m in emitted.messages
)
assert result.used_characters == 200
def test_metadata_union_stays_bounded_and_reports_truncation():
result = run(
memory_reader=reader(
memory(evidence=tuple(f"a{i}" for i in range(128))), memory(evidence=("extra",))
)
)
assert len(result.items[0].evidence_ids) == 128
assert result.items[0].truncated
assert "metadata_budget" in result.paths.memory.truncated_by
def test_finalizer_removed_context_message_reports_partial_not_complete():
item = raw()
item = item.model_copy(update={"messages": item.messages + raw("other").messages})
async def final(kind, items, deadline):
return tuple(i.model_copy(update={"messages": i.messages[:1]}) for i in items)
result = run(
ContextSearchRequest(query="q", kinds=("raw",)), raw_reader=reader(item), finalizer=final
)
assert result.state == "partial"
assert result.paths.raw.reason == "evidence_changed"
def test_noncooperative_synchronous_reader_return_after_deadline_is_discarded():
async def late(deadline):
time.sleep(0.06)
return ContextLaneResult(status="ok", items=(raw(),))
result = run(ContextSearchRequest(query="q", kinds=("raw",), timeout_ms=50), raw_reader=late)
assert result.items == () and result.paths.raw.reason == "timeout_budget"
def test_timeout_does_not_wait_for_a_reader_that_suppresses_cancellation():
async def scenario():
release = asyncio.Event()
async def stuck(_deadline):
try:
await asyncio.Event().wait()
except asyncio.CancelledError:
await release.wait()
return ContextLaneResult(status="empty")
started = time.monotonic()
result = await search_context(
ContextSearchRequest(query="q", kinds=("memory",), timeout_ms=50),
raw_reader=None,
memory_reader=stuck,
finalizer=finalize,
)
assert time.monotonic() - started < 0.2
assert result.paths.memory.reason == "timeout_budget"
release.set()
await asyncio.sleep(0)
asyncio.run(scenario())
@@ -1,161 +0,0 @@
"""The objective names `recall` and requires BOTH paths: raw and memory.
Verified against a real Core with seeded evidence and one governed memory:
asking for `raw` returns the stored verbatim and leaves `memory` as
`not_requested`; asking for `memory` returns the governed memory and leaves
`raw` as `not_requested`. That the unrequested side is marked, rather than
silently empty, is what proves the filter is actually applied.
"""
from __future__ import annotations
import sqlite3
import uuid
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
def _seed(data_dir) -> tuple[str, str]:
"""One evidence row and one governed memory; return their verbatim/content."""
verbatim = "帮我总结一下 SQLite 的写锁调优方法"
memory = "用户偏好:数据库调优时优先看写入批处理和读池容量,不要一上来就换引擎。"
connection = sqlite3.connect(data_dir / "one-memory.sqlite3")
try:
cursor = connection.cursor()
columns = [row[1] for row in cursor.execute("PRAGMA table_info(evidence)")]
row: dict[str, object] = {}
for column in columns:
if column == "evidence_id" or column == "idempotency_key":
row[column] = str(uuid.uuid4())
elif column in ("user_id", "source_id"):
row[column] = "local" if column == "user_id" else "codex"
elif column in ("conversation_id", "message_id"):
row[column] = str(uuid.uuid4())
elif column == "role":
row[column] = "user"
elif column == "written_by":
row[column] = "person"
elif column == "verbatim":
row[column] = verbatim
elif column in ("created_at", "updated_at", "imported_at", "recorded_at"):
row[column] = "2026-09-26T00:00:00Z"
elif column == "revision" or column == "source_ordinal":
row[column] = 1
elif column.endswith("_hash") or column == "fingerprint":
row[column] = str(uuid.uuid4())[:32]
names = ",".join(row)
markers = ",".join(["?"] * len(row))
cursor.execute(f"INSERT INTO evidence ({names}) VALUES ({markers})", list(row.values()))
memory_columns = [r[1] for r in cursor.execute("PRAGMA table_info(governed_memories)")]
memory_row: dict[str, object] = {}
for column in memory_columns:
if column == "memory_id":
memory_row[column] = str(uuid.uuid4())
elif column == "user_id":
memory_row[column] = "local"
elif column == "kind":
memory_row[column] = "working"
elif column == "status":
memory_row[column] = "confirmed"
elif column == "content":
memory_row[column] = memory
elif column in ("scopes_json", "tags_json"):
memory_row[column] = "[]"
elif column == "origin":
memory_row[column] = "manual"
elif column == "revision":
memory_row[column] = 1
elif column in ("created_at", "updated_at"):
memory_row[column] = "2026-09-26T00:00:00Z"
names = ",".join(memory_row)
markers = ",".join(["?"] * len(memory_row))
cursor.execute(
f"INSERT INTO governed_memories ({names}) VALUES ({markers})",
list(memory_row.values()),
)
connection.commit()
finally:
connection.close()
return verbatim, memory
def _client(tmp_path):
data_dir = tmp_path / "data"
data_dir.mkdir(parents=True, exist_ok=True)
app = create_app(
Settings(
_env_file=None,
data_dir=data_dir,
source_home=tmp_path,
api_token=SecretStr("test-token"),
)
)
return (
app,
data_dir,
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer test-token",
"Origin": "http://127.0.0.1:5173",
},
),
)
@pytest.mark.anyio
async def test_raw_path_returns_verbatim_and_leaves_memory_unrequested(tmp_path) -> None:
app, data_dir, client = _client(tmp_path)
async with app.router.lifespan_context(app), client:
# Seed inside the lifespan: the schema is created on startup, so seeding
# before it yields no columns and an empty INSERT.
verbatim, _ = _seed(data_dir)
response = await client.post(
"/api/v1/context/search",
json={
"query": verbatim,
"kinds": ["raw"],
"limit": 5,
"character_budget": 2000,
"timeout_ms": 5000,
},
)
assert response.status_code == 200, response.text
document = response.json()
assert document["paths"]["raw"]["status"] == "ok"
assert document["paths"]["memory"]["status"] == "not_requested", "memory must not run"
texts = [message["text"] for item in document["items"] for message in item.get("messages", [])]
assert verbatim in texts, "raw path must return the stored verbatim"
@pytest.mark.anyio
async def test_memory_path_returns_governed_memory_and_leaves_raw_unrequested(tmp_path) -> None:
app, data_dir, client = _client(tmp_path)
async with app.router.lifespan_context(app), client:
_, memory = _seed(data_dir)
response = await client.post(
"/api/v1/context/search",
json={
"query": "数据库调优偏好",
"kinds": ["memory"],
"limit": 5,
"character_budget": 2000,
"timeout_ms": 5000,
},
)
assert response.status_code == 200, response.text
document = response.json()
assert document["paths"]["memory"]["status"] == "ok"
assert document["paths"]["raw"]["status"] == "not_requested", "raw must not run"
contents = [item["content"] for item in document["items"]]
assert memory in contents, "memory path must return the governed memory"
@@ -10,11 +10,7 @@ from contextlib import contextmanager
from datetime import UTC, datetime
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.models import (
EvidenceInput,
EvidenceOrigin,
@@ -22,7 +18,6 @@ from one_memory.imports.models import (
WrittenBy,
)
from one_memory.imports.storage import ConversationSearchUnavailableError, EvidenceStore
from one_memory.memory_governance import MemoryDraft, MemoryKind
def message(
@@ -479,268 +474,3 @@ def test_a_missing_index_is_raised_rather_than_answered_as_no_matches(tmp_path)
with pytest.raises(ConversationSearchUnavailableError):
store.search_conversations("抽取终于", user_id="local", limit=10)
def _client(tmp_path):
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
return app, AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
)
@pytest.mark.anyio
async def test_recent_list_ignores_a_missing_search_index(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
app.state.evidence_store.put(
message(verbatim="刚从 Cursor 同步进来的原话", message_id="live")
)
with sqlite3.connect(app.state.evidence_store.database_path) as connection:
connection.execute("DROP TABLE IF EXISTS evidence_search")
response = await client.get("/api/v1/conversations", params={"limit": 5})
assert response.status_code == 200
body = response.json()
assert body["total"] == 1
assert body["items"][0]["excerpt"] == "刚从 Cursor 同步进来的原话"
assert body["items"][0]["truncated"] is False
assert body["items"][0]["source_id"] == "cursor"
@pytest.mark.anyio
async def test_search_answers_with_no_model_profile_configured(tmp_path) -> None:
"""The whole point of C-02: no generation model, and still an answer."""
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
app.state.evidence_store.put(message(verbatim="昨天晚上重新打了一个包,抽取终于跑通了"))
assert app.state.model_profile_store.active(app.state.secret_store) is None
response = await client.get("/api/v1/conversations/search", params={"query": "抽取终于"})
assert response.status_code == 200
body = response.json()
assert body["empty_reason"] is None
assert len(body["hits"]) == 1
assert "抽取终于跑通了" in body["hits"][0]["excerpt"]
@pytest.mark.anyio
async def test_search_groups_hits_and_keeps_title_separate_from_match(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
app.state.evidence_store.put(message(verbatim="发布准备", message_id="title"))
app.state.evidence_store.put(
message(
verbatim="第一次提到公证",
message_id="match-1",
created_at=datetime(2026, 9, 3, 1, 18, tzinfo=UTC),
)
)
app.state.evidence_store.put(
message(
verbatim="第二次提到公证",
message_id="match-2",
created_at=datetime(2026, 9, 3, 1, 19, tzinfo=UTC),
)
)
response = await client.get("/api/v1/conversations/search", params={"query": "公证"})
assert response.status_code == 200
assert len(response.json()["hits"]) == 1
assert response.json()["hits"][0]["title"] == "发布准备"
assert response.json()["hits"][0]["excerpt"] == "第二次提到公证"
assert response.json()["hits"][0]["turn_count"] == 3
assert response.json()["hits"][0]["turn_offset"] == 0
@pytest.mark.anyio
async def test_nothing_imported_is_told_apart_from_nothing_matching(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
empty = await client.get("/api/v1/conversations/search", params={"query": "抽取终于"})
assert empty.status_code == 200
assert empty.json()["empty_reason"] == "nothing_imported"
app.state.evidence_store.put(message(verbatim="周末想去爬山"))
unmatched = await client.get("/api/v1/conversations/search", params={"query": "量子纠缠"})
assert unmatched.status_code == 200
assert unmatched.json()["empty_reason"] == "no_matches"
@pytest.mark.anyio
async def test_a_broken_index_is_reported_rather_than_shown_as_no_matches(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
app.state.evidence_store.put(message(verbatim="抽取终于跑通了"))
with sqlite3.connect(app.state.evidence_store.database_path) as connection:
connection.execute("DROP TABLE IF EXISTS evidence_search")
response = await client.get("/api/v1/conversations/search", params={"query": "抽取终于"})
assert response.status_code == 503
assert response.json()["detail"] == "conversation_search_unavailable"
@pytest.mark.anyio
async def test_a_hit_opens_the_conversation_it_came_from(tmp_path) -> None:
"""Unverifiable facts read like invented ones, so the original has to open."""
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
app.state.evidence_store.put(
message(
verbatim="抽取终于跑通了",
message_id="m-1",
created_at=datetime(2026, 9, 3, 1, 17, tzinfo=UTC),
)
)
app.state.evidence_store.put(
message(
verbatim="那就先把包发出去",
message_id="m-2",
created_at=datetime(2026, 9, 3, 1, 19, tzinfo=UTC),
)
)
found = await client.get("/api/v1/conversations/search", params={"query": "抽取终于"})
evidence_id = found.json()["hits"][0]["evidence_id"]
transcript = await client.get(f"/api/v1/conversations/{evidence_id}")
assert transcript.status_code == 200
assert [item["verbatim"] for item in transcript.json()["messages"]] == [
"抽取终于跑通了",
"那就先把包发出去",
]
@pytest.mark.anyio
async def test_assistant_only_conversation_opens_in_time_order(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
first = app.state.evidence_store.put(
message(
verbatim="第一条助手回复",
message_id="assistant-only-1",
created_at=datetime(2026, 9, 3, 1, 17, tzinfo=UTC),
).model_copy(update={"role": "assistant", "written_by": "agent"})
).record
app.state.evidence_store.put(
message(
verbatim="第二条助手回复",
message_id="assistant-only-2",
created_at=datetime(2026, 9, 3, 1, 18, tzinfo=UTC),
).model_copy(update={"role": "assistant", "written_by": "agent"})
)
transcript = await client.get(f"/api/v1/conversations/{first.evidence_id}")
assert transcript.status_code == 200
assert transcript.json()["total_turns"] == 1
assert [item["verbatim"] for item in transcript.json()["messages"]] == [
"第一条助手回复",
"第二条助手回复",
]
@pytest.mark.anyio
async def test_conversation_browser_and_turn_pages_are_bounded(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
first_id = ""
for turn in range(25):
stored = app.state.evidence_store.put(
message(
verbatim=f"用户问题 {turn}",
message_id=f"user-{turn:02d}",
created_at=datetime(2026, 9, 3, 1, turn, tzinfo=UTC),
)
)
if turn == 0:
first_id = stored.record.evidence_id
assistant = message(
verbatim=f"助手回答 {turn}",
message_id=f"assistant-{turn:02d}",
created_at=datetime(2026, 9, 3, 1, turn, 30, tzinfo=UTC),
)
app.state.evidence_store.put(
assistant.model_copy(update={"role": "assistant", "written_by": "agent"})
)
app.state.evidence_store.put(
message(
verbatim="Codex 的同名会话",
source_id="codex",
conversation_id="conversation-1",
message_id="codex-1",
)
)
browser = await client.get("/api/v1/conversation-browser")
recent = await client.get(f"/api/v1/conversations/{first_id}", params={"limit": 20})
older = await client.get(
f"/api/v1/conversations/{first_id}", params={"offset": 20, "limit": 20}
)
assert browser.status_code == 200
assert browser.json()["total"] == 2
assert {
(item["source_id"], item["conversation_id"]) for item in browser.json()["items"]
} == {("cursor", "conversation-1"), ("codex", "conversation-1")}
assert recent.json()["total_turns"] == 25
assert recent.json()["has_more"] is True
assert len(recent.json()["messages"]) == 40
assert older.json()["has_more"] is False
assert len(older.json()["messages"]) == 10
@pytest.mark.anyio
async def test_conversation_memories_returns_only_linked_current_memories(tmp_path) -> None:
app, transport = _client(tmp_path)
async with app.router.lifespan_context(app), transport as client:
linked = app.state.evidence_store.put(
message(verbatim="发布前确认签名", message_id="linked")
).record
other = app.state.evidence_store.put(
message(
verbatim="另一段会话",
conversation_id="other-conversation",
message_id="other",
)
).record
app.state.person_writes.create(
MemoryDraft(
user_id="local",
kind=MemoryKind.WORKING,
content="发布前需要确认签名。",
scopes=("all-apps",),
evidence_ids=(linked.evidence_id,),
),
request_id="linked-memory",
)
app.state.person_writes.create(
MemoryDraft(
user_id="local",
kind=MemoryKind.WORKING,
content="不属于这段对话。",
scopes=("all-apps",),
evidence_ids=(other.evidence_id,),
),
request_id="other-memory",
)
response = await client.get(f"/api/v1/conversations/{linked.evidence_id}/memories")
assert response.status_code == 200
assert [item["content"] for item in response.json()["items"]] == ["发布前需要确认签名。"]
@@ -0,0 +1,144 @@
"""Readiness never waits for picking up work an earlier run left unfinished.
A 1.0.4 home with pending imports and write jobs used to resume all of it
before the lifespan yielded, so /health did not answer within the Runtime's
60s budget and the Runtime killed a healthy Core, which then began the same
recovery again. Recovery now runs after readiness.
"""
import asyncio
import threading
import time
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory import hy_sdk_migration
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.hy_sdk_migration import MetadataRepairResult
from one_memory.secrets import InMemorySecretStore
pytestmark = pytest.mark.fresh_sdk
def application(tmp_path):
return create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("ready-test"),
imports_enabled=False,
automatic_memory_processing=False,
live_sync_enabled=False,
),
secret_store=InMemorySecretStore(),
)
def client_for(app):
return AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer ready-test"},
)
async def health(client):
response = await client.get("/api/v1/health")
assert response.status_code == 200
return response.json()
async def wait_for_recovery(client, *, seconds=10.0):
deadline = time.monotonic() + seconds
while time.monotonic() < deadline:
body = await health(client)
if body["recovery"] in {"done", "failed"}:
return body
await asyncio.sleep(0.01)
raise AssertionError("recovery did not finish")
def test_core_is_ready_while_recovery_is_still_running(tmp_path, monkeypatch):
release = threading.Event()
entered = threading.Event()
def slow_repair(**_):
entered.set()
release.wait(10)
return MetadataRepairResult(None, 0, 0, 0)
monkeypatch.setattr(hy_sdk_migration, "repair_sdk_metadata_from_bundle", slow_repair)
async def scenario():
app = application(tmp_path)
try:
started = time.monotonic()
async with app.router.lifespan_context(app), client_for(app) as client:
# Startup finished although the first recovery step cannot.
assert time.monotonic() - started < 5
assert await asyncio.to_thread(entered.wait, 5)
body = await health(client)
assert body["status"] == "ok"
assert body["recovery"] == "running"
# Nothing after the blocked step has been started yet: the
# write drain in particular would hold the single writer.
assert app.state.memory_write_drain_task is None
# Reads answer during recovery.
assert (await client.get("/api/v1/sources")).status_code == 200
release.set()
body = await wait_for_recovery(client)
assert body["recovery"] == "done"
assert body["recovery_failed_steps"] == []
assert app.state.memory_write_drain_task is not None
finally:
release.set()
asyncio.run(scenario())
def test_a_failed_recovery_step_does_not_stop_readiness_or_later_steps(tmp_path, monkeypatch):
def broken_repair(**_):
raise OSError("bundle unreadable")
monkeypatch.setattr(hy_sdk_migration, "repair_sdk_metadata_from_bundle", broken_repair)
async def scenario():
app = application(tmp_path)
async with app.router.lifespan_context(app), client_for(app) as client:
body = await wait_for_recovery(client)
assert body["status"] == "ok"
assert body["recovery"] == "failed"
assert body["recovery_failed_steps"] == ["sdk_metadata_repair"]
# The steps after the failed one still ran.
assert app.state.memory_write_drain_task is not None
asyncio.run(scenario())
def test_shutdown_during_recovery_stops_it_without_starting_more(tmp_path, monkeypatch):
release = threading.Event()
entered = threading.Event()
def slow_repair(**_):
entered.set()
release.wait(10)
return MetadataRepairResult(None, 0, 0, 0)
monkeypatch.setattr(hy_sdk_migration, "repair_sdk_metadata_from_bundle", slow_repair)
async def scenario():
app = application(tmp_path)
try:
async with app.router.lifespan_context(app):
assert await asyncio.to_thread(entered.wait, 5)
recovery = app.state.recovery_task
# Shutdown cancelled recovery before it could start the drain.
assert recovery.done()
assert app.state.memory_write_drain_task is None
finally:
release.set()
asyncio.run(scenario())
@@ -8,7 +8,6 @@ from test_onboarding_profile import _app, _client
from one_memory.memory_qa import MemoryAnswerer
from one_memory.model_probe import ModelProber
from one_memory.secrets import SecretStoreUnavailableError
def test_disabled_automatic_processing_does_not_digest_onboarding(tmp_path):
@@ -82,21 +81,3 @@ def test_model_probe_credential_lookup_is_off_loop(tmp_path, monkeypatch):
transport=httpx.MockTransport(lambda request: httpx.Response(503)),
)
asyncio.run(prober.probe(profile.profile_id))
def test_unavailable_credentials_keep_governed_graph_readable(tmp_path, monkeypatch):
async def scenario():
app = _app(tmp_path)
async with app.router.lifespan_context(app), _client(app) as client:
def denied(*args):
raise SecretStoreUnavailableError("private-system-error")
monkeypatch.setattr(app.state.model_profile_store, "active", denied)
response = await client.get("/api/v1/memories/graph")
assert response.status_code == 200
assert response.json()["hy_graph_available"] is False
assert "private-system-error" not in response.text
assert (await client.get("/api/v1/health")).status_code == 200
asyncio.run(scenario())
@@ -1,314 +0,0 @@
"""D93 test 2: read latency while pure-Python work holds the GIL.
Plan `读写隔离方案-20261009.html` section 8 row 2, fixing the D92 experiment
as a test. Three pure-Python busy threads stand in for Core's background work
(scan, extraction bookkeeping, recompute). Meanwhile the home summary, the
memory library page and the map are requested in a loop; every non-200 is an
error and latencies give p95.
Two targets:
* `core` (default): the routes are served by `api.create_app` in this same
process, next to the busy threads. This is today's architecture and gives
the baseline. It is expected to be slow; it is measured, not gated, unless
`ONE_MEMORY_D93_GATE=1`.
* `reader`: B's Reader is started as a separate process
(`one-memory serve --reader`) on the same library, the busy threads stay in
this process. This is the claim the plan makes: work in Core no longer
touches reads. Gated: 0 errors, home and library p95 below 1 second.
Knobs: ONE_MEMORY_D93_LOAD_SECONDS (gate 600, default 20),
ONE_MEMORY_D93_MEMORIES (default 600), ONE_MEMORY_D93_TARGET,
ONE_MEMORY_D93_REPORT (JSON path for the numbers).
Recall ("at least one hit on a known question") needs the HY supervisor and is
not exercised here; it stays with acceptance item 7b.
"""
from __future__ import annotations
import asyncio
import json
import os
import socket
import sqlite3
import subprocess
import sys
import threading
import time
from pathlib import Path
import httpx
import pytest
from pydantic import SecretStr
from one_memory import api
from one_memory.config import Settings
from one_memory.imports.grants import GRANT_SCHEMA
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
from one_memory.imports.storage import EvidenceStore
from one_memory.memory_governance import MemoryDraft, MemoryGovernanceStore, MemoryKind
from one_memory.memory_tasks import MemoryTaskOutput, MemoryTaskStore, evidence_ref
from tests.person_door import as_person
SECONDS = float(os.environ.get("ONE_MEMORY_D93_LOAD_SECONDS", "20"))
MEMORIES = int(os.environ.get("ONE_MEMORY_D93_MEMORIES", "600"))
TARGET = os.environ.get("ONE_MEMORY_D93_TARGET", "core")
GATE = os.environ.get("ONE_MEMORY_D93_GATE", "1" if TARGET == "reader" else "") == "1"
TOKEN = "d93-load-token"
AUTH = {"Authorization": f"Bearer {TOKEN}"}
ROUTES = (
("home", "/api/v1/home-summary", {}),
("library", "/api/v1/memory-library", {"limit": 100, "active_only": "true"}),
("map", "/api/v1/memories/graph", {"limit": 160}),
)
def _build_library(database: Path) -> None:
now = "2026-10-09T00:00:00+00:00"
with sqlite3.connect(database) as setup:
setup.executescript(GRANT_SCHEMA)
for source in ("cursor", "codex"):
setup.execute(
"INSERT INTO source_grants "
"(user_id,source_id,root_path,requested_path,root_identity,"
"candidate_kind,granted_at,revoked_at,revision) "
"VALUES ('local',?,?,?,'fixture','default',?,NULL,1)",
(source, f"/fixture/{source}", f"/fixture/{source}", now),
)
evidence = EvidenceStore(database)
tasks = MemoryTaskStore(database)
person = as_person(MemoryGovernanceStore(database))
for index in range(MEMORIES):
record = evidence.put(
EvidenceInput(
source_id=("cursor", "codex")[index % 2],
conversation_id=f"conversation-{index // 10}",
message_id=f"m-{index}",
correlation_id=f"m-{index}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"第 {index} 条对话原文,讨论读写隔离和记忆页的加载速度。" * 4,
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
memory = person.create(
MemoryDraft(
user_id="local",
kind=MemoryKind.CHAT,
content=f"记忆 {index}:读写隔离要让记忆页在后台忙时也能打开。",
scopes=("all-apps",),
evidence_ids=(record.evidence_id,),
),
request_id=f"create-{index}",
)
tasks.save_output(
MemoryTaskOutput(
task_id=f"task-{index}",
memory_id=memory.memory_id,
memory_revision=memory.revision,
hy_memory_id=f"hy-{index}",
owner="user",
layer="L2_FACT",
engine_version="fixture",
support_refs=(evidence_ref(record),),
)
)
class _Busy:
"""Three pure-Python threads: no I/O, no sleep, they only want the GIL."""
def __init__(self, count: int = 3) -> None:
self.stop = threading.Event()
self.loops = [0] * count
self.threads = [
threading.Thread(target=self._spin, args=(index,), daemon=True)
for index in range(count)
]
def _spin(self, index: int) -> None:
while not self.stop.is_set():
total = 0
for value in range(20_000):
total += value * value % 7
self.loops[index] += 1
def __enter__(self):
for thread in self.threads:
thread.start()
return self
def __exit__(self, *_):
self.stop.set()
for thread in self.threads:
thread.join(timeout=5)
_SPIN = """
import threading
def spin():
while True:
total = 0
for value in range(20_000):
total += value * value % 7
for _ in range(3):
threading.Thread(target=spin, daemon=True).start()
threading.Event().wait()
"""
def _p95(values: list[float]) -> float | None:
if not values:
return None
ordered = sorted(values)
return ordered[min(len(ordered) - 1, int(round(0.95 * (len(ordered) - 1))))]
async def _drive(client: httpx.AsyncClient, deadline: float) -> dict:
stats = {name: {"ok": 0, "errors": {}, "latency": []} for name, _, _ in ROUTES}
while time.monotonic() < deadline:
for name, url, params in ROUTES:
started = time.monotonic()
try:
response = await client.get(url, params=params, headers=AUTH, timeout=30)
code = str(response.status_code)
except httpx.HTTPError as error:
code = type(error).__name__
elapsed = time.monotonic() - started
entry = stats[name]
entry["latency"].append(elapsed)
if code == "200":
entry["ok"] += 1
else:
entry["errors"][code] = entry["errors"].get(code, 0) + 1
return {
name: {
"requests": len(entry["latency"]),
"ok": entry["ok"],
"errors": entry["errors"],
"p95_seconds": _p95(entry["latency"]),
"max_seconds": max(entry["latency"], default=None),
}
for name, entry in stats.items()
}
def _core_app(data_dir: Path, root: Path):
"""Core as the app runs it (lifespan included), background processing off."""
return api.create_app(
Settings(
data_dir=data_dir,
source_home=root / "home",
api_token=SecretStr(TOKEN),
automatic_memory_processing=False,
live_sync_enabled=False,
),
source_catalog={},
)
def _free_port() -> int:
with socket.socket() as probe:
probe.bind(("127.0.0.1", 0))
return probe.getsockname()[1]
def _start_reader(data_dir: Path) -> tuple[subprocess.Popen, str]:
pytest.importorskip("one_memory.reader_app", reason="B (Reader) not in this build")
port = _free_port()
env = {
**os.environ,
"ONE_MEMORY_DATA_DIR": str(data_dir),
"ONE_MEMORY_READER_PORT": str(port),
"ONE_MEMORY_API_TOKEN": TOKEN,
"ONE_MEMORY_CORE_PORT": str(_free_port()), # nothing there: Core-only parts degrade
}
process = subprocess.Popen(
[sys.executable, "-m", "one_memory.cli", "serve", "--reader"],
env=env,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
)
base = f"http://127.0.0.1:{port}"
for _ in range(100):
try:
if httpx.get(f"{base}/health", timeout=0.5).status_code < 500:
return process, base
except httpx.HTTPError:
pass
if process.poll() is not None:
raise AssertionError(f"Reader exited: {process.stderr.read()[-2000:]!r}")
time.sleep(0.1)
process.kill()
raise AssertionError("Reader did not answer /health within 10 seconds")
def test_reads_stay_up_while_python_threads_hold_the_gil(tmp_path):
data_dir = tmp_path / "data"
data_dir.mkdir()
database = data_dir / "one-memory.sqlite3"
_build_library(database)
async def run(client):
# Warm once outside the measurement so lazy imports do not count.
for _, url, params in ROUTES:
await client.get(url, params=params, headers=AUTH, timeout=30)
idle = await _drive(client, time.monotonic() + min(5.0, SECONDS))
if TARGET == "reader":
# The busy threads belong to Core, not to this client: put them in
# a process of their own so the client does not queue for the GIL.
spinner = subprocess.Popen([sys.executable, "-c", _SPIN])
try:
loaded = await _drive(client, time.monotonic() + SECONDS)
finally:
spinner.terminate()
spinner.wait(timeout=10)
return idle, loaded, [1, 1, 1]
with _Busy() as busy:
loaded = await _drive(client, time.monotonic() + SECONDS)
return idle, loaded, busy.loops
reader = None
try:
if TARGET == "reader":
reader, base = _start_reader(data_dir)
transport = httpx.AsyncHTTPTransport()
application = None
else:
base = "http://test"
application = _core_app(data_dir, tmp_path)
transport = httpx.ASGITransport(app=application)
async def main():
async with httpx.AsyncClient(transport=transport, base_url=base) as client:
if application is None:
return await run(client)
async with application.router.lifespan_context(application):
return await run(client)
idle, loaded, loops = asyncio.run(main())
finally:
if reader is not None:
reader.terminate()
reader.wait(timeout=10)
report = {
"target": TARGET,
"memories": MEMORIES,
"seconds": SECONDS,
"busy_loops": loops,
"idle": idle,
"loaded": loaded,
}
path = os.environ.get("ONE_MEMORY_D93_REPORT")
if path:
Path(path).write_text(json.dumps(report, ensure_ascii=False, indent=2), encoding="utf-8")
print(json.dumps(report, ensure_ascii=False))
assert all(count > 0 for count in loops), "busy threads never ran"
assert all(entry["requests"] > 0 for entry in loaded.values())
if GATE:
for name, entry in loaded.items():
assert entry["errors"] == {}, f"{name}: {entry['errors']}"
for name in ("home", "library"):
assert loaded[name]["p95_seconds"] < 1.0, f"{name} p95 {loaded[name]['p95_seconds']}"
@@ -1,86 +1,5 @@
"""A host that saves our injection back must not mint a new user decision."""
import sqlite3
from contextlib import closing
from uuid import uuid4
from one_memory.change_outbox import ChangeOutbox, evidence_event_id
from one_memory.derived_echo import render_reference
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
from one_memory.imports.storage import EvidenceStore
from one_memory.session_inbox import SessionInbox
from one_memory.session_registry import SessionRegistry
def _evidence(message_id: str, verbatim: str) -> EvidenceInput:
return EvidenceInput(
source_id="workbuddy",
conversation_id="session-b",
message_id=message_id,
role="assistant",
written_by=WrittenBy.PERSON,
verbatim=verbatim,
origin_meta={EvidenceOrigin.FILE: OriginMetadata(raw_meta={"root": "proj"})},
)
def test_a_restatement_is_not_broadcast_and_a_correction_still_is(tmp_path) -> None:
path = tmp_path / "one-memory.sqlite3"
store = EvidenceStore(path)
registry = SessionRegistry(path)
registry.initialize()
inbox = SessionInbox(path)
inbox.initialize()
opened = registry.open(
installation_id=str(uuid4()),
host_id="workbuddy",
native_session_id="session-b",
runtime_instance="runtime-1",
workspace="/work/same-project",
)
inbox.subscribe(opened.session_key, ("workbuddy",), permission_epoch=1)
with closing(sqlite3.connect(path)) as db:
db.row_factory = sqlite3.Row
db.execute("BEGIN IMMEDIATE")
decision = store.put(_evidence("decision", "周五前交一版"), connection=db)
ChangeOutbox(path).record(
db,
event_id=evidence_event_id(decision.record.evidence_id, decision.record.revision),
cause="import",
evidence_id=decision.record.evidence_id,
evidence_revision=decision.record.revision,
permission_epoch=0,
)
db.commit()
page = inbox.pull(opened.session_key, cursor=None, limit=8)
injected = render_reference(page.events)
with closing(sqlite3.connect(path)) as db:
db.row_factory = sqlite3.Row
db.execute("BEGIN IMMEDIATE")
store.put(_evidence("echo", injected), connection=db)
db.commit()
with closing(sqlite3.connect(path)) as db:
assert db.execute("SELECT COUNT(*) FROM change_outbox").fetchone()[0] == 1
echoed = inbox.pull(opened.session_key, cursor=page.cursor, limit=8)
assert echoed.events == ()
with closing(sqlite3.connect(path)) as db:
db.row_factory = sqlite3.Row
db.execute("BEGIN IMMEDIATE")
correction = store.put(
_evidence("correction", injected + "\n不对,改成下周三"), connection=db
)
ChangeOutbox(path).record(
db,
event_id=evidence_event_id(correction.record.evidence_id, correction.record.revision),
cause="import",
evidence_id=correction.record.evidence_id,
evidence_revision=correction.record.revision,
permission_epoch=0,
)
db.commit()
corrected = inbox.pull(opened.session_key, cursor=page.cursor, limit=8)
assert len(corrected.events) == 1
def test_stripping_removes_the_notice_block_as_well_as_the_memory_block() -> None:
from one_memory.derived_echo import strip_injected_memory
@@ -110,11 +110,24 @@ def test_log_level_document_migrates_the_old_boolean_shape(tmp_path: Path) -> No
assert json.loads(target.read_text()) == {"schema_version": 2, "level": "error"}
def test_configure_environment_passes_debug_content_settings_to_sdk(tmp_path: Path) -> None:
def test_configure_environment_passes_debug_content_settings_to_sdk(
tmp_path: Path, monkeypatch
) -> None:
for name in (
"MEMORY_DATA_DIR",
"MEMORY_LOG_DIR",
"MEMORY_LOG_LEVEL",
"MEMORY_PIPELINE_TRACE_ENABLED",
"MEMORY_LOG_LLM_CONTENT",
):
monkeypatch.delenv(name, raising=False)
set_diagnostic_log_level(tmp_path, "debug")
configure_hy_runtime_environment(tmp_path)
environment = __import__("os").environ
memory_root = tmp_path / "hy-memory" / "governed-v2"
assert environment["MEMORY_DATA_DIR"] == str(memory_root)
assert environment["MEMORY_LOG_DIR"] == str(memory_root / "logs")
assert environment["MEMORY_LOG_LEVEL"] == "DEBUG"
assert environment["MEMORY_PIPELINE_TRACE_ENABLED"] == "true"
assert environment["MEMORY_LOG_LLM_CONTENT"] == "true"
@@ -561,11 +561,11 @@ def test_database_error_records_safe_route_and_sqlite_code(tmp_path, monkeypatch
error.sqlite_errorcode = sqlite3.SQLITE_BUSY
error.sqlite_errorname = "SQLITE_BUSY"
try:
record_database_error("core.api", error, route_path="/api/v1/context/search")
record_database_error("core.api", error, route_path="/api/v2/context/search")
finally:
reset_diagnostics()
payload = json.loads(receipt.path.read_text().splitlines()[-1])
assert payload["details"]["route_path"] == "/api/v1/context/search"
assert payload["details"]["route_path"] == "/api/v2/context/search"
assert payload["details"]["sqlite_errorcode"] == sqlite3.SQLITE_BUSY
assert payload["details"]["sqlite_errorname"] == "SQLITE_BUSY"
@@ -7,7 +7,6 @@ from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
def headers(*, scopes=("all-apps",), tools=("om_recall",)):
@@ -51,12 +50,14 @@ def test_external_marker_is_not_an_installation_grant(tmp_path):
)
assert approved.status_code == 200
denied_tool = await client.post(
"/api/v1/memories/ask", json={"question": "history"}, headers=headers()
"/api/v2/memories/ask", json={"question": "history"}, headers=headers()
)
assert denied_tool.status_code == 403
no_raw = await client.get(
"/api/v1/conversations/search",
params={"query": "history"},
# om_conversations reaches Core only as the v2 raw lane, and only
# with a source grant: all-apps alone opens no raw source.
no_raw = await client.post(
"/api/v2/context/search",
json={"query": "history", "kinds": ["raw"]},
headers=headers(tools=("om_conversations",)),
)
assert no_raw.status_code == 403
@@ -99,48 +100,3 @@ def test_external_diagnostic_trace_uses_its_own_read_only_tool(tmp_path):
assert denied.status_code == 403
asyncio.run(scenario())
def test_raw_source_grants_filter_before_limit_and_private_body_loading(tmp_path):
async def scenario():
app = create_app(
Settings(
data_dir=tmp_path,
source_home=tmp_path,
api_token=SecretStr("test"),
automatic_memory_processing=False,
live_sync_enabled=False,
),
source_catalog={},
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer test"},
) as client,
):
for source in ("allowed", "denied"):
app.state.evidence_store.put(
EvidenceInput(
source_id=source,
conversation_id="one",
message_id="one",
role="user",
written_by=WrittenBy.PERSON,
verbatim="needle " + source,
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
)
result = await client.get(
"/api/v1/conversations/search",
params={"query": "needle", "limit": 1},
headers=headers(scopes=("all-apps", "source:allowed"), tools=("om_conversations",)),
)
assert result.status_code == 200
assert result.json()["imported_messages"] == 1
assert [hit["source_id"] for hit in result.json()["hits"]] == ["allowed"]
assert "needle denied" not in result.text
asyncio.run(scenario())
+7
View File
@@ -45,6 +45,9 @@ def test_health_reports_the_single_core() -> None:
"memory_backfill_error": None,
"content_candidate_backfill": "idle",
"content_candidate_backfill_failure": None,
# No lifespan ran, so recovery has not begun.
"recovery": "pending",
"recovery_failed_steps": [],
}
@@ -210,6 +213,8 @@ def test_lifespan_drains_committed_pending_dispatch_through_enqueue(tmp_path, mo
async def scenario() -> None:
async with app.router.lifespan_context(app):
# Replayed by the recovery that runs after readiness.
await app.state.recovery_task
assert record.evidence_id in seen
assert evidence.take_all_stable_records() == ()
@@ -434,6 +439,7 @@ def test_a_backfill_that_fails_says_so_rather_than_disappearing(tmp_path, monkey
app.router.lifespan_context(app),
AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client,
):
await app.state.recovery_task
await app.state.memory_backfill
response = await client.get("/api/v1/health")
@@ -497,6 +503,7 @@ def test_a_failed_catch_up_leaves_a_traceback_behind(tmp_path, monkeypatch, capl
app.router.lifespan_context(app),
AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client,
):
await app.state.recovery_task
with suppress(BaseException):
await app.state.memory_backfill
return (await client.get("/api/v1/health")).json()
-164
View File
@@ -1,164 +0,0 @@
import sqlite3
from one_memory.home_summary import read_home_summary
def database(tmp_path):
path = tmp_path / "summary.sqlite3"
with sqlite3.connect(path) as db:
db.executescript("""
CREATE TABLE governed_memories(memory_id TEXT, user_id TEXT, status TEXT,
deleted_at TEXT, content TEXT, revision INTEGER, updated_at TEXT);
CREATE TABLE governed_memory_evidence(memory_id TEXT, evidence_id TEXT, active INTEGER);
CREATE TABLE governed_memory_versions(memory_id TEXT, revision INTEGER, action TEXT);
CREATE TABLE evidence(evidence_id TEXT, user_id TEXT, source_id TEXT);
CREATE TABLE source_grants(user_id TEXT, source_id TEXT, revoked_at TEXT);
""")
db.execute("INSERT INTO evidence VALUES ('e1','local','cursor')")
db.execute("INSERT INTO evidence VALUES ('e2','local','workbuddy')")
return path
def put(
path,
key,
*,
status="confirmed",
user="local",
content="Memory",
time="2026-09-19T01:00:00+00:00",
deleted=None,
):
with sqlite3.connect(path) as db:
db.execute(
"INSERT INTO governed_memories VALUES (?,?,?,?,?,?,?)",
(key, user, status, deleted, content, 2, time),
)
db.execute("INSERT INTO governed_memory_evidence VALUES (?, 'e1', 1)", (key,))
db.execute("INSERT INTO governed_memory_versions VALUES (?, 2, 'correct')", (key,))
def test_summary_counts_beyond_page_limit_and_excludes_ineligible_rows(tmp_path):
path = database(tmp_path)
for i in range(503):
put(path, str(i))
put(path, "superseded", status="superseded")
put(path, "deleted", deleted="2026-09-19T00:00:00+00:00")
put(path, "other", user="other")
result = read_home_summary(path)
assert result.memory_count == 503
assert result.latest is not None
assert result.latest.source_ids == ("cursor",)
assert result.latest.revision == 2
assert result.latest.action == "correct"
def test_summary_revocation_excludes_whole_mixed_memory_not_only_citation(tmp_path):
path = database(tmp_path)
put(path, "mixed", time="2026-09-19T03:00:00+00:00")
put(path, "retained")
with sqlite3.connect(path) as db:
db.execute("INSERT INTO governed_memory_evidence VALUES ('mixed','e2',1)")
db.execute("INSERT INTO source_grants VALUES ('local','workbuddy','revoked')")
result = read_home_summary(path)
assert result.memory_count == 1
assert result.latest.memory_id == "retained"
with sqlite3.connect(path) as db:
db.execute("UPDATE governed_memory_evidence SET active=0 WHERE evidence_id='e2'")
assert read_home_summary(path).memory_count == 2
def test_latest_is_bounded_and_other_users_revocations_do_not_hide_local_data(tmp_path):
path = database(tmp_path)
put(path, "first")
put(path, "latest", time="2026-09-19T02:00:00+00:00", content="x" * 2000)
with sqlite3.connect(path) as db:
db.execute("INSERT INTO source_grants VALUES ('other','cursor','revoked')")
result = read_home_summary(path)
assert result.memory_count == 2
assert result.latest.memory_id == "latest"
assert result.latest.truncated is True
assert len(result.latest.content) == 320
def test_empty_summary_is_distinct_from_failed_read(tmp_path):
path = database(tmp_path)
result = read_home_summary(path)
assert result.memory_count == 0
assert result.latest is None
def test_a_read_past_its_deadline_returns_stale_projection(tmp_path):
"""A timed-out homepage read must not escape as SQLITE_INTERRUPT.
Real trigger: `/api/v1/home-summary` returned a bare OperationalError,
sqlite_errorcode 9, because the progress handler installed to bound the
read is itself what interrupts it and nothing caught the result.
2026-10-12: instead of raising HomeSummaryUnavailable (which still 500ed
on builds before fab1f6b6), the read answers a `stale` projection so the
UI keeps its last good summary on screen.
"""
import time as time_module
import one_memory.home_summary as home_summary
path = database(tmp_path)
# Enough rows that the progress handler actually gets to tick.
for index in range(20000):
put(path, str(index))
# Expire the deadline up front: the progress handler installed to bound
# the read then interrupts on its first tick, which is exactly the
# slow-machine path.
real = time_module.monotonic
home_summary.time = type("ExpiredClock", (), {"monotonic": staticmethod(lambda: real() + 100)})
try:
result = read_home_summary(path)
except Exception as error: # noqa: BLE001 - nothing may escape
raise AssertionError(
f"a bare {type(error).__name__} "
f"(sqlite_errorcode={getattr(error, 'sqlite_errorcode', '-')}) escaped"
) from error
finally:
home_summary.time = time_module
assert result.stale is True
assert result.memory_count == 0
assert result.latest is None
def test_home_summary_api_is_authenticated_and_has_typed_empty_state(tmp_path):
import asyncio
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
async def scenario():
home = tmp_path / "home"
home.mkdir()
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=home,
api_token=SecretStr("synthetic-token"),
automatic_memory_processing=False,
live_sync_enabled=False,
),
source_catalog={},
)
async with (
app.router.lifespan_context(app),
AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client,
):
denied = await client.get("/api/v1/home-summary")
assert denied.status_code in (401, 403)
result = await client.get(
"/api/v1/home-summary", headers={"Authorization": "Bearer synthetic-token"}
)
assert result.status_code == 200
assert result.json()["memory_count"] == 0
assert result.json()["latest"] is None
assert result.json()["observed_at"]
asyncio.run(scenario())
+4 -9
View File
@@ -97,15 +97,10 @@ def test_importing_and_searching_leaves_no_original_query_or_token_behind(tmp_pa
started.raise_for_status()
job = await app.state.import_supervisor.wait(started.json()["job_id"])
assert job.status == "completed" and job.stored == 1
found = await client.post("/api/v1/context/search", json={"query": SECRET_QUERY})
found.raise_for_status()
# The turn is reachable, so there was something to leak.
assert any(
SECRET_TURN in message["text"]
for item in found.json()["items"]
if item["kind"] == "raw"
for message in item["messages"]
)
# The search is logged as a request whatever the SDK answers; the
# query it carried must not be part of that record.
searched = await client.post("/api/v2/context/search", json={"query": SECRET_QUERY})
assert searched.status_code in {200, 503}
try:
asyncio.run(scenario())
@@ -79,6 +79,11 @@ ALLOWED = {
"normal WAL-aware read-only view",
calls=2,
),
("sdk_migration_log.py", "migration_rows"): Allowed(
"a read-only snapshot of migration attempts for the migration log, taken "
"before the access layer exists (Core refusing a legacy home, sdk-migrate)",
opened_with="?mode=ro",
),
("library_upgrade.py", "_backup_problem"): Allowed(
"the finished upgrade backup, checked read-only before migration relies on it",
opened_with="?mode=ro&immutable=1",
@@ -120,10 +120,10 @@ def test_manual_import_reports_segment_then_follows_normal_authorize_and_dedup(t
assert connected.status_code == 200 and connected.json()["failed_sources"] == {}
job = await app.state.import_supervisor.wait(connected.json()["jobs"][0]["job_id"])
assert job.stored == 2 and job.errors == 0
found = (
await client.get("/api/v1/conversations/search", params={"query": "保留来源"})
).json()
assert found["hits"][0]["created_at"] is None
found = app.state.evidence_store.search_conversations(
"保留来源", user_id="local", limit=5
)
assert found[0].created_at is None
again = await client.post(
"/api/v1/import-jobs", json={"source_id": receipt["source_id"]}
)
@@ -6,7 +6,7 @@ from pathlib import Path
import pytest
from one_memory import context_memory, home_summary, memory_library, source_read_access
from one_memory import context_memory, source_read_access
from one_memory.imports.grants import GRANT_SCHEMA
from one_memory.imports.storage import EvidenceStore
from one_memory.memory_evidence_freshness import register_memory_freshness
@@ -18,15 +18,10 @@ def _source(module) -> str:
def _predicates():
home = re.search(r'readable = """(.*?)"""', _source(home_summary), re.S).group(1)
library = _source(memory_library)
library_clauses = re.findall(r'clauses\.append\("""(NOT EXISTS.*?)"""\)', library, re.S)
access = re.search(
r"_MEMORY_READABLE = \"\"\"(.*?)\"\"\"", _source(source_read_access), re.S
).group(1)
return {
"home": (home, ("local",)),
"library": ("m.user_id = ? AND " + " AND ".join(library_clauses), ("local",)),
"context": (context_memory._READABLE, ()),
"source_read_access": (access, ()),
}
@@ -45,7 +40,7 @@ def db(tmp_path):
conn.close()
@pytest.mark.parametrize("name", ["home", "library", "context", "source_read_access"])
@pytest.mark.parametrize("name", ["context", "source_read_access"])
def test_citation_subqueries_use_the_memory_primary_key(db, name):
predicate, args = _predicates()[name]
statement = f"SELECT COUNT(*) FROM governed_memories m WHERE {predicate}"
@@ -134,10 +134,10 @@ def test_memory_content_normal_api_import_preserves_kind_and_dedup_revoke(tmp_pa
if s["source_id"] == receipt["source_id"]
)
assert source["sync_completeness"] == "one_time_snapshot"
found = (
await client.get("/api/v1/conversations/search", params={"query": "原始聊天"})
).json()
evidence = app.state.evidence_store.get(found["hits"][0]["evidence_id"])
found = app.state.evidence_store.search_conversations(
"原始聊天", user_id="local", limit=5
)
evidence = app.state.evidence_store.get(found[0].evidence_id)
assert evidence.content_kind is EvidenceContentKind.MEMORY_CONTENT
assert evidence.written_by is WrittenBy.UNKNOWN and evidence.created_at is None
assert evidence.verbatim == value["memories"][1]["content"]
@@ -145,7 +145,7 @@ def test_memory_content_normal_api_import_preserves_kind_and_dedup_revoke(tmp_pa
"/api/v1/import-jobs", json={"source_id": receipt["source_id"]}
)
assert (await app.state.import_supervisor.wait(again.json()["job_id"])).stored == 0
assert (await client.get("/api/v1/memories")).json() == []
assert app.state.memory_governance_store.list(user_id="local") == []
await client.post(f"/api/v1/sources/{receipt['source_id']}/revoke")
assert (
await client.get(f"/api/v1/sources/{receipt['source_id']}/import-receipt")
@@ -1,6 +1,5 @@
"""Independent synthetic read-path review; no lifespan, model or user database."""
import asyncio
import json
import sqlite3
import time
@@ -8,19 +7,15 @@ from contextlib import closing
from types import SimpleNamespace
import pytest
from httpx import ASGITransport, AsyncClient
from one_memory import api, source_read_access
from one_memory.config import Settings
from one_memory import source_read_access
from one_memory.context_memory import read_memory_context
from one_memory.context_models import ContextSearchRequest
from one_memory.home_summary import read_home_summary
from one_memory.imports.grants import GRANT_SCHEMA
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
from one_memory.imports.storage import EvidenceStore
from one_memory.memory_evidence_freshness import register_memory_freshness
from one_memory.memory_governance import MemoryDraft, MemoryGovernanceStore, MemoryKind
from one_memory.memory_library import read_memory_library
from one_memory.memory_tasks import MemoryTaskOutput, MemoryTaskStore, evidence_ref
from one_memory.source_read_access import SourceReadAccess
from tests.person_door import as_person
@@ -89,8 +84,6 @@ def context(case, *, deadline=None):
def visible(case, memory=None):
return (
read_memory_library(case.path).total,
read_home_summary(case.path).memory_count,
len(context(case).items),
len(SourceReadAccess(case.path).memories([memory or case.memory])),
)
@@ -104,9 +97,9 @@ def revise(case):
def test_revised_evidence_remains_readable_after_freshness_retirement(case):
"""Evidence revision no longer hides library / context / source reads."""
assert visible(case) == (1, 1, 1, 1)
assert visible(case) == (1, 1)
revise(case)
assert visible(case) == (1, 1, 1, 1)
assert visible(case) == (1, 1)
assert case.governance.get(case.memory.memory_id, user_id="local") == case.memory
assert case.tasks.outputs(case.output.task_id) == (case.output,)
@@ -117,7 +110,7 @@ def test_revised_evidence_remains_readable_after_freshness_retirement(case):
def test_scope_only_revision_keeps_memory_readable_after_evidence_change(case, revise_first):
if revise_first:
revise(case)
assert visible(case) == (1, 1, 1, 1)
assert visible(case) == (1, 1)
changed = as_person(case.governance).set_scopes(
case.memory.memory_id,
user_id="local",
@@ -127,9 +120,9 @@ def test_scope_only_revision_keeps_memory_readable_after_evidence_change(case, r
)
assert changed.content == case.memory.content
if not revise_first:
assert visible(case, changed) == (1, 1, 1, 1)
assert visible(case, changed) == (1, 1)
revise(case)
assert visible(case, changed) == (1, 1, 1, 1)
assert visible(case, changed) == (1, 1)
def test_explicit_correct_remains_readable_but_never_overrides_revocation(case):
@@ -141,7 +134,7 @@ def test_explicit_correct_remains_readable_but_never_overrides_revocation(case):
expected_revision=case.memory.revision,
content="alpha beta explicit correction",
)
assert visible(case, corrected) == (1, 1, 1, 1)
assert visible(case, corrected) == (1, 1)
with closing(sqlite3.connect(case.path)) as db, db:
db.execute(
"""INSERT INTO source_grants
@@ -150,14 +143,14 @@ def test_explicit_correct_remains_readable_but_never_overrides_revocation(case):
VALUES ('local','cursor','/fixture','/fixture','fixture','default',?,?,1)""",
("2026-09-20T00:00:00Z", "2026-09-20T00:00:01Z"),
)
assert visible(case, corrected) == (0, 0, 0, 0)
assert visible(case, corrected) == (0, 0)
def test_missing_proof_is_legacy_compatibility_not_a_new_authorization(case):
with closing(sqlite3.connect(case.path)) as db, db:
db.execute("DELETE FROM memory_task_outputs")
revise(case)
assert visible(case) == (1, 1, 1, 1)
assert visible(case) == (1, 1)
result = read_memory_context(
case.path,
ContextSearchRequest(query="alpha beta"),
@@ -188,7 +181,6 @@ def test_revocation_before_freshness_snapshot_cannot_release_memory(case, monkey
monkeypatch.setattr(source_read_access, "register_memory_freshness", revoke_then_register)
returned = SourceReadAccess(case.path).memories([case.memory])
assert changes == [True]
assert read_memory_library(case.path).total == 0
assert returned == []
@@ -219,8 +211,7 @@ def _oversize(case, bad):
def test_material_too_large_to_verify_reads_as_current(case, bad):
"""D76: a budget overrun is not stale and must not fail the whole read."""
_oversize(case, bad)
assert read_home_summary(case.path).memory_count == 1
read_memory_library(case.path)
assert len(SourceReadAccess(case.path).memories([case.memory])) == 1
assert context(case).status != "unavailable"
@@ -231,40 +222,12 @@ def test_bad_proof_material_no_longer_makes_reads_unavailable(case, bad):
"UPDATE memory_task_outputs SET body=?",
('{"memory_revision":2,"memory_revision":2}',),
)
assert visible(case) == (1, 1, 1, 1)
assert visible(case) == (1, 1)
result = context(case)
assert result.status != "unavailable"
assert len(result.items) == 1
def test_route_serves_memories_whose_proof_is_too_large(case, tmp_path):
app = api.create_app(
Settings(
data_dir=tmp_path / "unused",
api_token="fixture-token",
automatic_memory_processing=False,
live_sync_enabled=False,
),
source_catalog={},
)
app.state.evidence_store = SimpleNamespace(database_path=case.path)
with closing(sqlite3.connect(case.path)) as db, db:
db.execute("UPDATE memory_task_outputs SET body=?", (" " * 65537,))
async def run():
async with AsyncClient(
transport=ASGITransport(app=app, raise_app_exceptions=False),
base_url="http://fixture",
headers={"Authorization": "Bearer fixture-token"},
) as client:
library = await client.get("/api/v1/memory-library")
home = await client.get("/api/v1/home-summary")
assert library.status_code == 200
assert home.status_code == 200
asyncio.run(run())
def test_expired_context_budget_never_reads_proofs(case):
result = context(case, deadline=time.monotonic() - 1)
assert result.status == "unavailable" and result.reason == "timeout_budget"
@@ -285,7 +248,7 @@ def test_indexed_unrelated_proofs_keep_small_read_bounded(case):
).fetchall()
assert any("memory_task_outputs_memory" in row[3] for row in plan)
started = time.monotonic()
assert visible(case) == (1, 1, 1, 1)
assert visible(case) == (1, 1)
elapsed = time.monotonic() - started
print(f"indexed 4000 unrelated proofs, four reads: {elapsed:.6f}s")
assert elapsed < 2
@@ -1,151 +0,0 @@
import asyncio
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.models import (
EvidenceInput,
EvidenceOrigin,
OriginMetadata,
WrittenBy,
)
def test_memory_governance_api_preserves_evidence_and_reports_cleanup(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation",
message_id="message",
correlation_id="message",
role="user",
written_by=WrittenBy.PERSON,
verbatim="Original private words; token=local-secret-value",
origin_meta={
EvidenceOrigin.FILE: OriginMetadata(
raw_meta={"absolute_path": "/must/not/leave/api"}
)
},
)
).record
created = await client.post(
"/api/v1/memories",
json={
"request_id": "create-1",
"draft": {
"user_id": "local",
"kind": "chat",
"content": "Initial memory",
"scopes": ["all-apps"],
"evidence_ids": [evidence.evidence_id],
},
},
)
assert created.status_code == 200
memory_id = created.json()["memory_id"]
assert created.json()["status"] == "confirmed"
recalled = await client.post(
"/api/v1/recall",
json={"query": "Initial memory", "scope": "all-apps"},
)
assert recalled.status_code == 200
assert recalled.json()["state"] == "grounded"
assert recalled.json()["hits"][0]["memory_id"] == memory_id
corrected = await client.post(
f"/api/v1/memories/{memory_id}/correct",
json={
"request_id": "correct-1",
"expected_revision": 1,
"content": "Corrected memory",
"scopes": ["cursor"],
},
)
assert corrected.status_code == 200
detail = await client.get(f"/api/v1/memories/{memory_id}")
assert evidence.verbatim == "Original private words; token=local-secret-value"
assert detail.json()["evidence"][0]["verbatim"] == (
"Original private words; token=<redacted-secret>"
)
assert "absolute_path" not in detail.text
assert detail.json()["history"][0]["action"] == "correct"
graph = await client.get("/api/v1/memories/graph")
assert graph.status_code == 200
assert graph.json()["projected_memories"] == 1
assert {node["node_type"] for node in graph.json()["nodes"]} == {"memory", "source"}
assert graph.json()["edges"][0]["relation"] == "evidence_source"
deleted = await client.post(
f"/api/v1/memories/{memory_id}/actions/delete",
json={"request_id": "delete-1", "expected_revision": 2},
)
assert deleted.json()["status"] == "deleted"
cleanup = await client.get(f"/api/v1/memories/{memory_id}/deletion")
assert cleanup.json()["status"] == "failed"
assert cleanup.json()["last_error_code"] == "model_profile_not_active"
asyncio.run(scenario())
def test_memory_validation_does_not_echo_content(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
response = await client.post(
"/api/v1/memories",
json={
"request_id": "create-1",
"draft": {
"user_id": "local",
"kind": "chat",
"content": "secret body",
"scopes": [],
"evidence_ids": [],
},
},
)
assert response.status_code == 422
assert "secret body" not in response.text
assert "input" not in response.text
asyncio.run(scenario())
@@ -1,895 +0,0 @@
import asyncio
import time
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory import api as api_module
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.candidates import RootCandidateKind
from one_memory.imports.models import (
EvidenceInput,
EvidenceOrigin,
OriginMetadata,
WrittenBy,
)
def _confirm(app, client, content: str, evidence_id: str, index: int):
async def run() -> None:
created = await client.post(
"/api/v1/memories",
json={
"request_id": f"create-{index}",
"draft": {
"user_id": "local",
"kind": "chat",
"content": content,
"scopes": ["all-apps"],
"evidence_ids": [evidence_id],
},
},
)
assert created.status_code == 200
return run()
def test_graph_labels_where_every_edge_came_from(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index, (message_id, content) in enumerate(
(
("message-1", "用户要求看板只用真实数据。"),
("message-2", "用户希望问答给出引用。"),
)
):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation-a",
message_id=message_id,
correlation_id=message_id,
role="user",
written_by=WrittenBy.PERSON,
verbatim=content,
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(app, client, content, evidence.evidence_id, index)
other = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation-b",
message_id="message-3",
correlation_id="message-3",
role="user",
written_by=WrittenBy.PERSON,
verbatim="用户在另一次对话里提到偏好。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(app, client, "用户在另一次对话里提到偏好。", other.evidence_id, 2)
graph = await client.get("/api/v1/memories/graph")
assert graph.status_code == 200
body = graph.json()
assert body["projected_memories"] == 3
origins = {edge["origin"] for edge in body["edges"]}
assert origins == {"provenance", "same_conversation"}
# Only the two memories from one conversation are linked to each
# other; the memory from another conversation is not.
same = [edge for edge in body["edges"] if edge["origin"] == "same_conversation"]
assert len(same) == 1
# HY was never reachable in this test, and the response says so
# instead of letting the map imply the relations are all there is.
assert body["hy_graph_available"] is False
asyncio.run(scenario())
def test_a_created_memory_is_on_the_map_with_its_source_links(tmp_path):
async def scenario():
app = create_app(
Settings(
data_dir=tmp_path / "data", source_home=tmp_path, api_token=SecretStr("local-token")
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer local-token"},
) as client,
):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="candidate",
message_id="c1",
role="assistant",
written_by=WrittenBy.AGENT,
verbatim="Agent建议蓝色按钮。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
created = await client.post(
"/api/v1/memories",
json={
"request_id": "candidate-create",
"draft": {
"user_id": "local",
"kind": "chat",
"content": "Agent建议蓝色按钮。",
"origin": "agent",
"scopes": ["all-apps"],
"evidence_ids": [evidence.evidence_id],
},
},
)
assert created.status_code == 200
identifier = created.json()["memory_id"]
graph = (await client.get("/api/v1/memories/graph")).json()
assert graph["projected_memories"] == 1
assert any(
node["memory_id"] == identifier and node["status"] == "confirmed"
for node in graph["nodes"]
)
assert any(edge["origin"] == "provenance" for edge in graph["edges"])
asyncio.run(scenario())
def test_drawing_the_map_does_not_stop_the_app_answering(tmp_path) -> None:
"""The map is what the memory view opens with, and it used to block the
process while it was read.
Every store call in the read is synchronous and the evidence is fetched an
id at a time, so a full map is hundreds of blocking calls. Made on the
loop, they were made instead of answering anyone: the watchdog caught this
endpoint holding the loop for 2.1s, and /health — which does no work at
all — was measured at 17.9s on the real library while a map was drawn.
"""
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index in range(4):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation-a",
message_id=f"message-{index}",
correlation_id=f"message-{index}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"用户说了第 {index} 件事。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
said = f"用户说了第 {index} 件事。"
await _confirm(app, client, said, evidence.evidence_id, index)
# Each evidence read takes a moment, the way a contended one does.
inner = app.state.evidence_store
delay = 0.1
class _Sluggish:
def __getattr__(self, name: str):
attribute = getattr(inner, name)
if not callable(attribute):
return attribute
def slowly(*args, **kwargs):
time.sleep(delay)
return attribute(*args, **kwargs)
return slowly
app.state.evidence_store = _Sluggish()
longest = 0.0
answered = 0
async def keep_asking() -> None:
# The gap between turns, not the duration of one request: a
# task cannot time a stall it was not running for, and timing
# its own request measures nothing while the loop is blocked.
nonlocal longest, answered
last = time.perf_counter()
while True:
reply = await client.get("/api/v1/health")
assert reply.status_code == 200
now = time.perf_counter()
longest = max(longest, now - last)
last = now
answered += 1
await asyncio.sleep(0.005)
asking = asyncio.create_task(keep_asking())
graph = await client.get("/api/v1/memories/graph")
asking.cancel()
assert graph.status_code == 200
# The map really was read, so this cannot pass by drawing nothing.
assert graph.json()["projected_memories"] == 4
# Read on the loop, not one of these got an answer until the map
# was finished, which is the whole defect.
assert answered > 1, "画图期间没有任何请求被应答:循环被堵住了"
# Several tenth-of-a-second reads happen while the map is drawn;
# on the loop none of these could have been answered until it
# finished.
assert longest < delay * 3, f"画图期间循环被堵住 {longest:.3f}s"
asyncio.run(scenario())
def _confirm_tagged(app, client, content: str, evidence_id: str, index: int, tags):
async def run() -> None:
created = await client.post(
"/api/v1/memories",
json={
"request_id": f"create-tagged-{index}",
"draft": {
"user_id": "local",
"kind": "chat",
"content": content,
"scopes": ["all-apps"],
"evidence_ids": [evidence_id],
"tags": list(tags),
},
},
)
assert created.status_code == 200, created.text
return run()
def test_a_shared_tag_becomes_a_topic_on_the_map(tmp_path) -> None:
"""标签要在图上长成一个点,并把带它的记忆连起来。
图上原先只有三种点:记忆、来源 App、以及 HY 自己 TOPIC 层的归纳。我们从 HY
接回来的 tags 只在左栏当文字展示,没有进图 —— 于是一张记忆还很少的库打开来
满屏都是来源 App,看起来像是按 App 聚类,而不是按事项聚类。
只连「被两条以上记忆共用」的标签:只出现一次的标签连出去的是一根挂着单个
记忆的线,那不是聚类,只是把同一条记忆画了两遍。
"""
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
plans = (
("m-1", "用户要求前端统一走 axios 封装。", ("axios", "frontend")),
("m-2", "用户要求测试先行。", ("axios", "testing")),
("m-3", "用户在做记忆产品。", ("memory-product",)),
)
for index, (message_id, content, tags) in enumerate(plans):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id=f"conversation-{index}",
message_id=message_id,
correlation_id=message_id,
role="user",
written_by=WrittenBy.PERSON,
verbatim=content,
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm_tagged(app, client, content, evidence.evidence_id, index, tags)
body = (await client.get("/api/v1/memories/graph")).json()
topics = {
node["label"]: node["node_id"]
for node in body["nodes"]
if node["node_type"] == "topic"
}
# axios 被两条记忆共用,所以它是一个主题。
assert "axios" in topics
# frontend / testing / memory-product 各只出现一次 —— 不成聚类。
assert set(topics) == {"axios"}
tagged = [edge for edge in body["edges"] if edge["origin"] == "shared_tag"]
assert len(tagged) == 2
assert {edge["source_node_id"] for edge in tagged} == {topics["axios"]}
asyncio.run(scenario())
def test_a_tag_used_once_does_not_become_a_topic(tmp_path) -> None:
"""正向对照:全库只有一条带标签的记忆时,图上不许出现主题点。
没有这条,上一条可以被「每个标签都建一个点」骗过 —— 那样 axios 照样在,
只是同时多出三个各挂一条记忆的孤点。
"""
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation-solo",
message_id="m-solo",
correlation_id="m-solo",
role="user",
written_by=WrittenBy.PERSON,
verbatim="用户要求前端统一走 axios 封装。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm_tagged(
app, client, "用户要求前端统一走 axios 封装。", evidence.evidence_id, 9, ("axios",)
)
body = (await client.get("/api/v1/memories/graph")).json()
assert [node for node in body["nodes"] if node["node_type"] == "topic"] == []
assert [edge for edge in body["edges"] if edge["origin"] == "shared_tag"] == []
asyncio.run(scenario())
def test_the_map_only_verifies_the_memories_it_draws(tmp_path) -> None:
"""A full library must not be verified row by row just to count it.
The map reads 500 memories but draws at most `limit`. Verifying all 500
meant one freshness check plus one citation check per memory, and on a
1,226-memory library it passed the 5 s interrupt, so /memories/graph
answered 503 every time while /memories, which verifies one page, stayed
at 0.1 s.
"""
calls: list[int] = []
def counting(memories):
calls.append(len(memories))
return list(memories)
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index in range(12):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id=f"conversation-{index}",
message_id=f"m-{index}",
correlation_id=f"m-{index}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"第 {index} 条真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(
app, client, f"第 {index} 条真实对话内容。", evidence.evidence_id, index
)
app.state.source_read_access.memories = counting
body = (await client.get("/api/v1/memories/graph?limit=5")).json()
assert body["total_memories"] == 12
assert body["projected_memories"] == 5
# The count is not row-verified, and neither is the tail: only the
# five memories the map draws are checked.
assert calls == [5]
asyncio.run(scenario())
def test_the_map_resolves_the_source_catalog_once(tmp_path) -> None:
"""The catalog is resolved once per request, not once per drawn source.
`_source_catalog` re-reads and re-resolves every registered source. The
map called it inside the per-memory loop, once per distinct source: on a
160-memory library that is over a thousand resolutions, and it is where
the map's five seconds went.
"""
from one_memory.imports.candidates import SourceCandidateStore
calls: list[int] = []
original = SourceCandidateStore.resolve_catalog
def counting(self, catalog, *, user_id="local"):
calls.append(1)
return original(self, catalog, user_id=user_id)
SourceCandidateStore.resolve_catalog = counting
try:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index in range(6):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id=f"conversation-{index}",
message_id=f"m-{index}",
correlation_id=f"m-{index}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"第 {index} 条真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(
app, client, f"第 {index} 条真实对话内容。", evidence.evidence_id, index
)
calls.clear()
body = (await client.get("/api/v1/memories/graph")).json()
assert body["projected_memories"] == 6
# Six memories share one source. One resolution, not six.
assert len(calls) == 1
asyncio.run(scenario())
finally:
SourceCandidateStore.resolve_catalog = original
def test_the_map_count_leaves_a_revoked_source_out(tmp_path) -> None:
"""The count is the cheap one, but not so cheap it counts revoked work.
D22 settled that a total skips the per-row freshness check. It still has
to leave out memories a revoked source supplied, or the number the map
reports includes memories it refuses to draw.
"""
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index, source_id in enumerate(("cursor", "claude-code")):
folder = tmp_path / source_id
folder.mkdir(exist_ok=True)
app.state.source_grant_store.grant(
user_id="local",
source_id=source_id,
root=folder,
candidate_kind=RootCandidateKind.DEFAULT,
)
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id=source_id,
conversation_id=f"conversation-{source_id}",
message_id=f"m-{source_id}",
correlation_id=f"m-{source_id}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"{source_id} 的真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(
app, client, f"{source_id} 的真实对话内容。", evidence.evidence_id, index
)
before = (await client.get("/api/v1/memories/graph")).json()
assert before["total_memories"] == 2
app.state.source_grant_store.revoke(user_id="local", source_id="claude-code")
after = (await client.get("/api/v1/memories/graph")).json()
assert after["total_memories"] == 1
assert after["projected_memories"] == 1
asyncio.run(scenario())
def test_the_map_reads_its_evidence_in_one_pass(tmp_path) -> None:
"""D89: one query for the map's evidence, not one per citation.
Each drawn memory looked up every one of its citations separately: 1,274
round trips for a 160-memory map, each taking a turn through the pool.
That is what put the map over its five second budget on a library being
written to at the same time — the acceptance run saw 4.2-14.8 s and a 503
on three calls out of ten.
"""
from one_memory.imports.storage import EvidenceStore
calls: list[int] = []
original = EvidenceStore.get
def counting(self, evidence_id):
calls.append(1)
return original(self, evidence_id)
EvidenceStore.get = counting
try:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index in range(6):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id=f"conversation-{index % 2}",
message_id=f"m-{index}",
correlation_id=f"m-{index}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"第 {index} 条真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(
app, client, f"第 {index} 条真实对话内容。", evidence.evidence_id, index
)
calls.clear()
body = (await client.get("/api/v1/memories/graph")).json()
assert body["projected_memories"] == 6
# Six memories with one citation each, read together.
assert len(calls) <= 1
asyncio.run(scenario())
finally:
EvidenceStore.get = original
def test_a_timed_out_map_falls_back_to_the_last_good_one(tmp_path) -> None:
"""D91: a slow map must not answer 503 while a good one is known.
While a library is being written to, the map read can cross its five
second interrupt. The memory view then has nothing to show, even though
the very same map was drawn successfully a moment ago and the library has
not changed since.
"""
import sqlite3
calls: list[int] = []
real = api_module._governed_graph_part
def counting(app, limit):
calls.append(limit)
if len(calls) > 1:
raise sqlite3.OperationalError("interrupted")
return real(app, limit)
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for index in range(3):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id=f"conversation-{index}",
message_id=f"m-{index}",
correlation_id=f"m-{index}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"第 {index} 条真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(
app, client, f"第 {index} 条真实对话内容。", evidence.evidence_id, index
)
api_module._governed_graph_part = counting
try:
first = await client.get("/api/v1/memories/graph")
assert first.status_code == 200
assert first.json()["projected_memories"] == 3
# The next read crosses the interrupt. Nothing has changed
# since the one that succeeded, so the known map is the better
# answer than a 503.
second = await client.get("/api/v1/memories/graph")
finally:
api_module._governed_graph_part = real
assert second.status_code == 200
assert second.json()["projected_memories"] == 3
asyncio.run(scenario())
def test_a_changed_library_is_drawn_again(tmp_path) -> None:
"""The cache is keyed on the library, so a new memory redraws the map."""
calls: list[int] = []
real = api_module._governed_graph_part
def counting(app, limit):
calls.append(limit)
return real(app, limit)
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation-0",
message_id="m-0",
correlation_id="m-0",
role="user",
written_by=WrittenBy.PERSON,
verbatim="第 0 条真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(app, client, "第 0 条真实对话内容。", evidence.evidence_id, 0)
api_module._governed_graph_part = counting
try:
assert (await client.get("/api/v1/memories/graph")).status_code == 200
# Same library: served from the last good map.
assert (await client.get("/api/v1/memories/graph")).status_code == 200
assert len(calls) == 1
second = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation-1",
message_id="m-1",
correlation_id="m-1",
role="user",
written_by=WrittenBy.PERSON,
verbatim="第 1 条真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(app, client, "第 1 条真实对话内容。", second.evidence_id, 1)
body = await client.get("/api/v1/memories/graph")
finally:
api_module._governed_graph_part = real
assert body.status_code == 200
assert body.json()["projected_memories"] == 2
assert len(calls) == 2
asyncio.run(scenario())
def test_withdrawing_a_second_source_redraws_the_map(tmp_path) -> None:
"""D95: the cache must notice a second revocation, not just the first.
Keying on whether *any* source is revoked keeps serving the map drawn
before the second withdrawal, so that source's memories stay on a map the
reader is no longer allowed to see them on.
"""
from one_memory.imports.candidates import RootCandidateKind
calls: list[int] = []
real = api_module._governed_graph_part
def counting(app, limit):
calls.append(limit)
return real(app, limit)
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
for source_id in ("cursor", "claude-code"):
folder = tmp_path / source_id
folder.mkdir(exist_ok=True)
app.state.source_grant_store.grant(
user_id="local",
source_id=source_id,
root=folder,
candidate_kind=RootCandidateKind.DEFAULT,
)
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id=source_id,
conversation_id=f"conversation-{source_id}",
message_id=f"m-{source_id}",
correlation_id=f"m-{source_id}",
role="user",
written_by=WrittenBy.PERSON,
verbatim=f"{source_id} 的真实对话内容。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
await _confirm(
app, client, f"{source_id} 的真实对话内容。", evidence.evidence_id, source_id
)
api_module._governed_graph_part = counting
try:
assert (await client.get("/api/v1/memories/graph")).status_code == 200
app.state.source_grant_store.revoke(user_id="local", source_id="cursor")
assert (await client.get("/api/v1/memories/graph")).status_code == 200
assert len(calls) == 2
# A second withdrawal with one already revoked. The library
# itself has not changed, so only the grant fingerprint can
# tell this apart from the previous call.
app.state.source_grant_store.revoke(user_id="local", source_id="claude-code")
body = await client.get("/api/v1/memories/graph")
finally:
api_module._governed_graph_part = real
assert body.status_code == 200
assert len(calls) == 3
asyncio.run(scenario())
+52 -35
View File
@@ -78,9 +78,6 @@ def test_empty_library_can_save_a_note_without_a_matter_or_configured_model(tmp_
assert (
app.state.memory_note_store.require(saved["evidence_id"]).verbatim == note["text"]
)
retired = await client.get(f"/api/v1/memory-notes/{saved['evidence_id']}/evidence")
assert retired.status_code == 409
assert retired.json()["detail"] == "memory_note_evidence_retired"
assert legacy_tables_present(app.state.settings.data_dir / "one-memory.sqlite3") == ()
restarted = application(tmp_path)
async with restarted.router.lifespan_context(restarted), client_for(restarted) as client:
@@ -118,37 +115,6 @@ def test_note_requires_authentication_explicit_scope_and_nonempty_content(tmp_pa
asyncio.run(scenario())
def test_saved_note_evidence_returns_raw_and_consent_reference_from_same_record(tmp_path):
async def scenario():
app = legacy_application(tmp_path)
async with app.router.lifespan_context(app), client_for(app) as client:
text = "Original spacing\n预算<5,不能改为>5。"
saved = (
await client.post(
"/api/v1/memory-notes",
json={
"request_id": str(uuid4()),
"text": text,
"scope": "all-apps",
},
)
).json()
endpoint = f"/api/v1/memory-notes/{saved['evidence_id']}/evidence"
response = await client.get(endpoint)
assert response.status_code == 200, response.text
value = response.json()
assert value["verbatim"] == text
assert value["evidence_id"] == saved["evidence_id"] and value["revision"] == 1
assert value["role"] == "user" and value["written_by"] == "person"
assert value["available"] is True and value["source_id"] == "one-memory-manual"
assert app.state.memory_task_store.list() == []
assert (await client.get("/api/v1/memory-notes/missing/evidence")).status_code == 404
client.headers.pop("Authorization")
assert (await client.get(endpoint)).status_code == 401
asyncio.run(scenario())
def test_note_and_source_grant_are_atomic(tmp_path):
from one_memory.memory_notes import MemoryNoteInput
@@ -409,7 +375,7 @@ def test_sdk_memory_library_uses_sdk_search_and_skips_empty_rows_without_mutatio
assert [item["memory_id"] for item in body["memories"]] == ["release"]
assert search_calls[0][0] == "release"
assert search_calls[0][1]["include_sources"] == ["cursor"]
assert search_calls[0][1]["limit"] == 500
assert search_calls[0][1]["limit"] == 20
assert search_calls[0][1]["intention_limit"] == 0
assert search_calls[0][1]["min_score"] == 0.0
assert list_calls == []
@@ -440,3 +406,54 @@ def test_sdk_memory_library_uses_sdk_search_and_skips_empty_rows_without_mutatio
assert "source:default" not in node_ids
asyncio.run(scenario())
@pytest.mark.fresh_sdk
def test_sdk_context_search_sets_each_recall_lane_apart(tmp_path):
"""Normal, schema and intention lanes default to 10 / 3 / 1 and pass through."""
async def scenario():
app = application(tmp_path)
calls = []
class Client:
def search(self, query, **kwargs):
calls.append(kwargs)
return {"memories": {"schema": [], "proactive": [], "normal": []}}
def get(self, memory_id):
return None
sdk = Client()
class Handle:
async def client(self):
return sdk
async def close(self):
return None
async with app.router.lifespan_context(app):
app.state.sdk_client_handle = Handle()
async with client_for(app) as client:
defaulted = await client.post(
"/api/v2/context/search",
json={"query": "release", "kinds": ["memory"]},
)
chosen = await client.post(
"/api/v2/context/search",
json={
"query": "release",
"kinds": ["memory"],
"limit": 5,
"schema_limit": 0,
"intention_limit": 2,
},
)
assert defaulted.status_code == 200
assert chosen.status_code == 200
assert [
(call["limit"], call["schema_limit"], call["intention_limit"]) for call in calls
] == [(10, 3, 1), (5, 0, 2)]
asyncio.run(scenario())
+2 -124
View File
@@ -7,35 +7,9 @@ from pydantic import SecretStr
from one_memory import api
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.models import (
EvidenceInput,
EvidenceOrigin,
OriginMetadata,
WrittenBy,
)
from one_memory.sdk_data_plane_migration import MemoryAuthority
from one_memory.sdk_memory_qa import SdkAskMemory
class _StubAnswerer:
def __init__(self, events: tuple[dict, ...]) -> None:
self.events = events
self.calls: list[tuple[str, str]] = []
self.last_workspace_path: str | None = None
async def answer_stream(
self,
question: str,
*,
scope: str = "all-apps",
workspace_path: str | None = None,
):
self.calls.append((question, scope))
self.last_workspace_path = workspace_path
for event in self.events:
yield event
def _sse_events(text: str) -> list[dict]:
return [
json.loads(frame.strip().removeprefix("data:").strip())
@@ -44,96 +18,6 @@ def _sse_events(text: str) -> list[dict]:
]
def test_ask_stream_endpoint_emits_recall_deltas_and_final(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
evidence = app.state.evidence_store.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation",
message_id="message",
correlation_id="message",
role="user",
written_by=WrittenBy.PERSON,
verbatim="我要求看板只用真实数据。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
created = await client.post(
"/api/v1/memories",
json={
"request_id": "create-1",
"draft": {
"user_id": "local",
"kind": "chat",
"content": "用户要求看板只用真实数据。",
"scopes": ["all-apps"],
"evidence_ids": [evidence.evidence_id],
},
},
)
memory = app.state.memory_governance_store.get(
created.json()["memory_id"],
user_id="local",
)
stub = _StubAnswerer(
(
{"type": "recall", "memories": (memory,)},
{"type": "delta", "text": "你要求"},
{"type": "delta", "text": "真实数据。"},
{
"type": "final",
"answer": "你要求真实数据。",
"memories": (memory,),
"mode": "generated",
"retrieval": {"semantic_state": "failed"},
},
)
)
app.state.memory_answerer = stub
response = await client.post(
"/api/v1/memories/ask/stream",
json={"question": "我对数据有什么要求?", "scope": "all-apps"},
)
assert response.status_code == 200
assert response.headers["content-type"].startswith("text/event-stream")
events = _sse_events(response.text)
assert [event["type"] for event in events] == [
"recall",
"delta",
"delta",
"final",
]
assert events[0]["memories"][0]["memory_id"] == memory.memory_id
assert (
"".join(event["text"] for event in events if event["type"] == "delta")
== events[-1]["answer"]
)
assert events[-1]["retrieval"] == {"semantic_state": "failed"}
assert stub.calls == [("我对数据有什么要求?", "all-apps")]
asyncio.run(scenario())
def test_ask_stream_endpoint_requires_the_local_token(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
@@ -152,7 +36,7 @@ def test_ask_stream_endpoint_requires_the_local_token(tmp_path) -> None:
) as client,
):
response = await client.post(
"/api/v1/memories/ask/stream",
"/api/v2/memories/ask/stream",
json={"question": "我是谁?", "scope": "all-apps"},
)
@@ -161,7 +45,7 @@ def test_ask_stream_endpoint_requires_the_local_token(tmp_path) -> None:
asyncio.run(scenario())
def test_sdk_ask_v2_stream_is_native_and_v1_fails_after_cutover(tmp_path, monkeypatch) -> None:
def test_sdk_ask_v2_stream_is_native(tmp_path, monkeypatch) -> None:
memory = SdkAskMemory(
memory_id="memory-1",
content="用户要求真实数据。",
@@ -204,12 +88,6 @@ def test_sdk_ask_v2_stream_is_native_and_v1_fails_after_cutover(tmp_path, monkey
},
) as client,
):
app.state.memory_authority = MemoryAuthority.SDK
legacy = await client.post(
"/api/v1/memories/ask/stream",
json={"question": "要求是什么?"},
)
assert legacy.status_code == 409
response = await client.post(
"/api/v2/memories/ask/stream",
json={"question": "要求是什么?"},
@@ -5330,3 +5330,70 @@ def test_equal_event_time_uses_queue_sequence_inside_one_session(tmp_path) -> No
owner="equal-owner",
)
assert drain.claim(limit=10) == (second.job_id,)
def test_a_hand_added_memory_is_claimed_ahead_of_unprocessed_history(tmp_path) -> None:
"""A hand-added memory ("添加记忆") waited behind every unprocessed history job.
History is claimed through a ten-minute event-time window, oldest first. A
note is stamped "now", so with a backlog it sat at the far end of the
timeline and ran only after the whole history was digested.
"""
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
old = datetime(2025, 1, 1, tzinfo=UTC)
history = [
queue.submit(
_request(
key=f"history-{index}",
evidence=(f"h{index}",),
source="cursor",
session=f"cursor-{index}",
event_at=old + timedelta(hours=index),
)
)
for index in range(3)
]
note = queue.submit(
_request(
key="note",
evidence=("n1",),
source="one-memory-manual",
session="note:1",
event_at=datetime.now(UTC),
)
)
drain = DurableWriteDrain(queue, execute=_never_execute)
assert drain.claim(limit=1) == (note.job_id,)
# History keeps its own order through the window.
assert drain.claim(limit=1) == (history[0].job_id,)
def test_hand_added_memories_still_run_one_at_a_time_in_order(tmp_path) -> None:
queue = DurableWriteQueue(tmp_path / "one-memory.sqlite3")
now = datetime.now(UTC)
first = queue.submit(
_request(
key="note-1",
evidence=("n1",),
source="one-memory-manual",
session="note:1",
event_at=now,
)
)
second = queue.submit(
_request(
key="note-2",
evidence=("n2",),
source="one-memory-manual",
session="note:2",
event_at=now + timedelta(seconds=1),
)
)
drain = DurableWriteDrain(queue, execute=_never_execute)
assert drain.claim(limit=5) == (first.job_id,)
# Its source is busy: the next note waits for the running one, not for history.
assert drain.claim(limit=5) == ()
assert queue.get("note-2").state == PENDING
assert second.job_id
@@ -742,49 +742,6 @@ def test_change_outbox_ddl_mysql_and_sqlite_agree_on_columns() -> None:
assert "event_id" in columns(mysql)
@pytest.mark.parametrize(
("getter", "table"),
[
("session_leases_ddl", "session_leases"),
("trusted_sessions_ddl", "trusted_sessions"),
],
)
def test_migrated_ddl_has_one_source(getter: str, table: str) -> None:
"""Each migrated module must use the migration's definition, not a copy.
Modules that ran on one engine held a single `_SCHEMA` constant. Migrated
ones hold both renderings and select at use time, so the check reads
whichever the module actually uses — the point is still that the text comes
from the migration and is not retyped inline.
"""
import importlib
migrations = importlib.import_module("one_memory.memory_server.migrations")
render = getattr(migrations, getter)
module_name = {
"session_leases_ddl": "one_memory.session_lease",
"trusted_sessions_ddl": "one_memory.session_registry",
}[getter]
module = importlib.import_module(module_name)
if hasattr(module, "schema_for"):
sqlite_text = module.schema_for(False)
mysql_text = module.schema_for(True)
else:
sqlite_text = module._SCHEMA
mysql_text = None
assert sqlite_text == render("sqlite"), "module copies the DDL instead of using the migration"
assert table in sqlite_text
assert "ENGINE=InnoDB" not in sqlite_text
# A migrated module must also be able to render MySQL, or it will send
# SQLite DDL to MySQL when the engine switches.
if mysql_text is not None:
assert mysql_text == render("mysql")
assert table in mysql_text
def test_import_jobs_ddl_is_split_before_execution() -> None:
"""MySQL takes one DDL per call, so a script must be split.
+1 -127
View File
@@ -10,11 +10,7 @@ import time
from contextlib import closing
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.context_memory import read_memory_context
from one_memory.context_models import ContextSearchRequest
from one_memory.imports.grants import SourceGrantStore
@@ -30,8 +26,6 @@ from one_memory.memory_governance import (
MemoryMutationRequestConflictError,
MemoryStatus,
)
from one_memory.memory_library import read_memory_library
from one_memory.memory_mutation_receipts import read_memory_mutation_receipt
from one_memory.recall import GovernedRecallTarget, RecallRequest, RecallState
from tests.person_door import as_person
@@ -267,7 +261,6 @@ def test_a_superseded_memory_leaves_every_current_read_path(tmp_path) -> None:
)
return {
"listed": {m.memory_id for m in store.list(user_id="local")},
"library": {m.memory_id for m in read_memory_library(database).items},
"recall": {hit.memory_id for hit in recall.hits},
"semantic": {hit.memory_id for hit in semantic.hits},
"matching": {
@@ -286,7 +279,7 @@ def test_a_superseded_memory_leaves_every_current_read_path(tmp_path) -> None:
after = current_ids()
assert all(old.memory_id not in ids for ids in after.values()), after
assert all(new.memory_id in after[path] for path in ("listed", "library", "recall")), after
assert all(new.memory_id in after[path] for path in ("listed", "recall")), after
assert GovernedRecallTarget(store).recall(RecallRequest(query="权限评审")).state is (
RecallState.GROUNDED
)
@@ -295,9 +288,6 @@ def test_a_superseded_memory_leaves_every_current_read_path(tmp_path) -> None:
assert [m.memory_id for m in store.list(user_id="local", status=MemoryStatus.SUPERSEDED)] == [
old.memory_id
]
library = read_memory_library(database, status=MemoryStatus.SUPERSEDED)
assert [m.memory_id for m in library.items] == [old.memory_id]
assert library.items[0].superseded_by == (new.memory_id,)
async def _async(value):
@@ -413,122 +403,6 @@ def test_the_link_is_stored_additively_without_rewriting_existing_rows(tmp_path)
).fetchone()
def test_an_earlier_receipt_stays_readable_after_its_memory_is_superseded(tmp_path) -> None:
database, store, old, new = _pair(tmp_path)
old = as_person(store).set_scopes(
old.memory_id,
user_id="local",
request_id="scope-old",
expected_revision=old.revision,
scopes=("all-apps", "cursor"),
)
before = read_memory_mutation_receipt(database, request_id="scope-old")
assert before.state == "committed" and before.result_status == "confirmed"
_supersede(store, old, new.memory_id)
after = read_memory_mutation_receipt(database, request_id="scope-old")
assert after == before
assert set(type(after).model_fields) == {
"schema_version",
"request_id",
"state",
"operation",
"memory_id",
"result_revision",
"result_status",
"memory_state",
"migrated_from",
}
def test_the_api_shows_the_chain_refuses_edits_and_still_deletes_through_hy(tmp_path) -> None:
async def scenario() -> None:
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("local-token"),
)
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={
"Authorization": "Bearer local-token",
"Origin": "http://127.0.0.1:5173",
},
) as client,
):
store = app.state.memory_governance_store
evidence_id = _evidence(app.state.evidence_store.database_path)
door = app.state.person_writes
old = _confirmed(door, evidence_id, OLD, "old")
new = _confirmed(door, evidence_id, NEW, "new")
listed = await client.get("/api/v1/memories")
assert {m["memory_id"] for m in listed.json()} == {"old", "new"}
replaced = _supersede(store, old, new.memory_id)
listed = await client.get("/api/v1/memories")
assert [m["memory_id"] for m in listed.json()] == ["new"]
library = await client.get("/api/v1/memory-library")
assert [m["memory_id"] for m in library.json()["items"]] == ["new"]
only_old = await client.get("/api/v1/memory-library", params={"status": "superseded"})
assert [m["memory_id"] for m in only_old.json()["items"]] == ["old"]
detail = (await client.get("/api/v1/memories/old")).json()["memory"]
assert detail["status"] == "superseded"
assert detail["superseded_by"] == ["new"]
assert detail["supersedes"] == []
head = (await client.get("/api/v1/memories/new")).json()["memory"]
assert head["supersedes"] == ["old"] and head["superseded_by"] == []
recalled = await client.post(
"/api/v1/recall", json={"query": "权限评审", "scope": "all-apps"}
)
assert [hit["memory_id"] for hit in recalled.json()["hits"]] == ["new"]
for action in ("undo",):
refused = await client.post(
f"/api/v1/memories/old/actions/{action}",
json={"request_id": f"{action}-x", "expected_revision": replaced.revision},
)
assert (refused.status_code, refused.json()["detail"]) == (
409,
"memory_superseded",
)
refused = await client.post(
"/api/v1/memories/old/correct",
json={
"request_id": "correct-x",
"expected_revision": replaced.revision,
"content": "改一下",
},
)
assert (refused.status_code, refused.json()["detail"]) == (409, "memory_superseded")
cascaded: list[str] = []
async def finish_deletion(memory_id: str) -> None:
cascaded.append(memory_id)
app.state.sdk_memory_runtime.finish_deletion = finish_deletion
deleted = await client.post(
"/api/v1/memories/old/actions/delete",
json={"request_id": "delete-x", "expected_revision": replaced.revision},
)
assert deleted.status_code == 200 and deleted.json()["status"] == "deleted"
assert cascaded == ["old"]
cleanup = await client.get("/api/v1/memories/old/deletion")
assert cleanup.json()["status"] == "completed"
spec = app.openapi()["components"]["schemas"]
assert "superseded" in spec["MemoryStatus"]["enum"]
assert {"supersedes", "superseded_by"} <= set(spec["GovernedMemory"]["properties"])
asyncio.run(scenario())
def test_supersession_integrity_reports_a_cycle_and_stays_quiet_when_healthy(tmp_path) -> None:
"""The read-only audit must flag a real cycle and not cry wolf otherwise.
@@ -2,310 +2,123 @@
Mounted on a minimal app rather than the production one, so the test proves the
router's own contract (status codes, replay, conflict) instead of depending on
the whole server's auth and state wiring.
the whole server's auth and state wiring. Core serves only the SDK write door,
`/api/v2/memory-writes`.
"""
from __future__ import annotations
import sqlite3
from datetime import UTC, datetime, timedelta
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
from one_memory.memory_server.durable_queue import DurableWriteRegistry
from one_memory.memory_server.registry import WriteRegistry
from one_memory.memory_server.routes import router
from one_memory.memory_server.contracts import BatchIdentity, WriteRequest
from one_memory.memory_server.durable_queue import DurableWriteDrain, DurableWriteRegistry
from one_memory.memory_server.routes import sdk_router
@pytest.fixture()
def client() -> TestClient:
app = FastAPI()
app.state.memory_write_registry = WriteRegistry()
app.include_router(router())
return TestClient(app)
ROUTE = "/api/v2/memory-writes"
def _payload(**overrides: object) -> dict:
body = {
"internal_session_id": "sess-1",
"first_round": 1,
"last_round": 5,
"evidence_ids": ["e1"],
body: dict[str, object] = {
"request_id": "request-1",
"session_id": "session-1",
"messages": [
{"role": "user", "content": "remember the deploy window"},
{"role": "assistant", "content": "noted"},
],
}
body.update(overrides)
return body
def test_submit_returns_202_with_a_request_id(client: TestClient) -> None:
response = client.post("/api/v1/memory-writes", json=_payload())
assert response.status_code == 202
assert response.json()["request_id"]
assert response.json()["state"] == "accepted"
def test_a_replay_is_not_a_second_write(client: TestClient) -> None:
"""Duplicating a memory is worse than admitting it already exists."""
first = client.post("/api/v1/memory-writes", json=_payload())
second = client.post("/api/v1/memory-writes", json=_payload())
assert second.status_code == 202
assert second.json()["request_id"] == first.json()["request_id"]
assert "already accepted" in second.json()["detail"]
def test_same_key_different_payload_is_409(client: TestClient) -> None:
"""Overwriting would discard the earlier intent."""
client.post(
"/api/v1/memory-writes",
json=_payload(idempotency_key="fixed", evidence_ids=["e1"]),
)
conflict = client.post(
"/api/v1/memory-writes",
json=_payload(idempotency_key="fixed", evidence_ids=["different"]),
)
assert conflict.status_code == 409
def test_status_can_be_queried_after_submit(client: TestClient) -> None:
created = client.post("/api/v1/memory-writes", json=_payload())
request_id = created.json()["request_id"]
status = client.get(f"/api/v1/memory-writes/{request_id}")
assert status.status_code == 200
assert status.json()["request_id"] == request_id
def test_status_of_an_unknown_id_is_404(client: TestClient) -> None:
"""'We never accepted this' must not read as 'accepted and failed'."""
assert client.get("/api/v1/memory-writes/absent").status_code == 404
def test_missing_evidence_is_rejected(client: TestClient) -> None:
response = client.post("/api/v1/memory-writes", json=_payload(evidence_ids=[]))
assert response.status_code == 422
def test_unknown_fields_are_rejected(client: TestClient) -> None:
"""extra=forbid: a typo'd field must not be silently ignored."""
response = client.post("/api/v1/memory-writes", json=_payload(evidences=["e1"]))
assert response.status_code == 422
def test_no_registry_means_unavailable_not_a_crash(client: TestClient) -> None:
app = FastAPI()
app.include_router(router())
bare = TestClient(app)
assert bare.post("/api/v1/memory-writes", json=_payload()).status_code == 503
def test_durable_queue_can_back_the_same_route_contract(tmp_path) -> None:
"""The route shape must survive a process restart when backed by SQLite.
This is the missing D52 bridge: the HTTP contract already exists, and the
durable queue already records acceptance. The adapter must make the same
POST/GET contract durable without changing the route or inventing a second
idempotency rule.
"""
path = tmp_path / "one-memory.sqlite3"
app = FastAPI()
app.state.memory_write_registry = DurableWriteRegistry(path)
app.include_router(router())
client = TestClient(app)
created = client.post("/api/v1/memory-writes", json=_payload())
assert created.status_code == 202
request_id = created.json()["request_id"]
assert created.json()["state"] == "accepted"
reopened = FastAPI()
reopened.state.memory_write_registry = DurableWriteRegistry(path)
reopened.include_router(router())
status = TestClient(reopened).get(f"/api/v1/memory-writes/{request_id}")
assert status.status_code == 200
assert status.json()["request_id"] == request_id
assert status.json()["state"] == "accepted"
replay = TestClient(reopened).post("/api/v1/memory-writes", json=_payload())
assert replay.status_code == 202
assert replay.json()["request_id"] == request_id
assert "already accepted" in replay.json()["detail"]
def test_completed_write_status_returns_sdk_raw_and_memory_ids(tmp_path) -> None:
path = tmp_path / "one-memory.sqlite3"
app = FastAPI()
registry = DurableWriteRegistry(path)
app.state.memory_write_registry = registry
app.include_router(router())
client = TestClient(app)
created = client.post("/api/v1/memory-writes", json=_payload())
request_id = created.json()["request_id"]
with sqlite3.connect(path) as database:
job_id = str(
database.execute(
"SELECT job_id FROM memory_write_jobs WHERE idempotency_key=?",
(request_id,),
).fetchone()[0]
)
registry.queue.settle(
job_id,
state="completed",
sdk_raw_ids=("raw-1",),
sdk_memory_ids=("memory-1",),
)
status = client.get(f"/api/v1/memory-writes/{request_id}")
assert status.status_code == 200
assert status.json()["raw_ids"] == ["raw-1"]
assert status.json()["memory_ids"] == ["memory-1"]
def test_route_selects_durable_registry_when_a_path_is_configured(tmp_path) -> None:
"""C0 only has to hand the route a database path.
The route keeps its legacy in-memory fallback for isolated tests, but when
the production app provides `memory_write_queue_path`, the first request
must create the durable adapter and write a real row.
"""
path = tmp_path / "one-memory.sqlite3"
def _durable_client(path) -> tuple[FastAPI, TestClient]:
app = FastAPI()
app.state.memory_write_queue_path = path
app.include_router(router())
client = TestClient(app)
app.state.memory_authority = "sdk"
app.include_router(sdk_router())
return app, TestClient(app)
created = client.post("/api/v1/memory-writes", json=_payload())
assert created.status_code == 202
def test_submit_returns_202_and_replay_is_not_a_second_write(tmp_path) -> None:
path = tmp_path / "one-memory.sqlite3"
app, client = _durable_client(path)
first = client.post(ROUTE, json=_payload())
second = client.post(ROUTE, json=_payload())
assert first.status_code == 202
assert second.status_code == 202
assert first.json()["request_id"] == second.json()["request_id"]
assert "already accepted" in second.json()["detail"]
assert isinstance(app.state.memory_write_registry, DurableWriteRegistry)
with sqlite3.connect(path) as db:
assert db.execute("SELECT count(*) FROM memory_write_jobs").fetchone()[0] == 1
def test_durable_route_reports_unknown_without_collapsing_to_failed(tmp_path) -> None:
"""The HTTP state vocabulary must preserve the no-blind-retry outcome."""
path = tmp_path / "one-memory.sqlite3"
app = FastAPI()
registry = DurableWriteRegistry(path)
app.state.memory_write_registry = registry
app.include_router(router())
client = TestClient(app)
def test_a_write_without_a_user_turn_is_rejected(tmp_path) -> None:
_app, client = _durable_client(tmp_path / "one-memory.sqlite3")
created = client.post("/api/v1/memory-writes", json=_payload())
request_id = created.json()["request_id"]
assert registry.set_state(request_id, "unknown", "outcome not proven")
response = client.get(f"/api/v1/memory-writes/{request_id}")
response = client.post(
ROUTE,
json=_payload(messages=[{"role": "assistant", "content": "only me"}]),
)
assert response.status_code == 200
assert response.json()["state"] == "unknown"
assert response.status_code == 422
assert response.json()["detail"] == "messages_require_user_turn"
def test_unknown_fields_are_rejected(tmp_path) -> None:
_app, client = _durable_client(tmp_path / "one-memory.sqlite3")
assert client.post(ROUTE, json=_payload(evidence_ids=["e1"])).status_code == 422
def test_status_of_an_unknown_id_is_404(tmp_path) -> None:
_app, client = _durable_client(tmp_path / "one-memory.sqlite3")
client.post(ROUTE, json=_payload())
assert client.get(f"{ROUTE}/absent").status_code == 404
def test_durable_route_maps_a_dead_letter_to_a_valid_failed_state(tmp_path) -> None:
"""`dead` is a queue concept; the HTTP contract has no such state.
Retries exhausted must still return a terminal receipt the response model
accepts, without hiding that it was a dead letter rather than a normal
failure.
"""
app = FastAPI()
"""`dead` is a queue concept; the HTTP contract has no such state."""
database_path = tmp_path / "one-memory.sqlite3"
app.state.memory_write_queue_path = database_path
app.include_router(router())
client = TestClient(app)
created = client.post("/api/v1/memory-writes", json=_payload())
request_id = created.json()["request_id"]
_app, client = _durable_client(database_path)
request_id = client.post(ROUTE, json=_payload()).json()["request_id"]
registry = DurableWriteRegistry(database_path)
registry.queue.settle( # noqa: SLF001 - test needs the durable job id
registry = DurableWriteRegistry(database_path, sdk_authority=True)
registry.queue.settle(
registry.queue.get(request_id).job_id,
state="dead",
error_code="attempts exhausted",
)
response = client.get(f"/api/v1/memory-writes/{request_id}")
response = client.get(f"{ROUTE}/{request_id}")
assert response.status_code == 200
assert response.json()["state"] == "failed"
assert "dead letter" in response.json()["detail"].lower()
def test_durable_configuration_overrides_an_in_memory_registry(tmp_path) -> None:
"""`api.py` always installs an in-memory registry; durable must still win.
The production composition root sets `memory_write_registry` eagerly, so a
durable router that preferred that object would silently accept writes
into memory and lose them on restart. Setting the durable queue path must
take precedence.
"""
path = tmp_path / "one-memory.sqlite3"
app = FastAPI()
app.state.memory_write_registry = WriteRegistry()
app.state.memory_write_queue_path = path
app.include_router(router())
client = TestClient(app)
created = client.post("/api/v1/memory-writes", json=_payload())
assert created.status_code == 202
request_id = created.json()["request_id"]
# The receipt is durable: it survives a brand-new registry built from the
# same file, which the in-memory registry could not do.
assert DurableWriteRegistry(path).get(request_id) is not None
with sqlite3.connect(path) as db:
assert db.execute("SELECT count(*) FROM memory_write_jobs").fetchone()[0] == 1
def test_route_passes_the_evidence_store_into_the_durable_registry(tmp_path, monkeypatch) -> None:
"""A configured Evidence store must enforce Evidence-first at the route.
Without this wiring the queue accepts a write pointing at evidence the
business database does not hold, and the failure only surfaces later as a
`failed` drain job. When the composition root provides the store, the POST
must be refused up front and nothing may be enqueued.
"""
from one_memory.imports.storage import EvidenceStore
path = tmp_path / "one-memory.sqlite3"
app = FastAPI()
app.state.memory_write_queue_path = path
app.state.memory_write_evidence_store = EvidenceStore(path)
app.include_router(router())
client = TestClient(app)
response = client.post(
"/api/v1/memory-writes",
json=_payload(evidence_ids=["does-not-exist"]),
)
assert response.status_code >= 400
with sqlite3.connect(path) as db:
assert db.execute("SELECT count(*) FROM memory_write_jobs").fetchone()[0] == 0
def test_http_get_reports_a_claimed_running_job_as_accepted(tmp_path) -> None:
"""A live drain turns a submitted row `running`; GET must not 500."""
from datetime import timedelta
from one_memory.memory_server.durable_queue import DurableWriteDrain
async def never_execute(_job_id: str) -> None:
return None
path = tmp_path / "one-memory.sqlite3"
app = FastAPI()
registry = DurableWriteRegistry(path)
app.state.memory_write_registry = registry
app.include_router(router())
client = TestClient(app)
created = client.post("/api/v1/memory-writes", json=_payload())
request_id = created.json()["request_id"]
app, client = _durable_client(path)
request_id = client.post(ROUTE, json=_payload()).json()["request_id"]
DurableWriteDrain(
registry.queue,
app.state.memory_write_registry.queue,
execute=never_execute,
owner="owner-a",
lease=timedelta(seconds=30),
).claim(limit=1)
response = client.get(f"/api/v1/memory-writes/{request_id}")
response = client.get(f"{ROUTE}/{request_id}")
assert response.status_code == 200
assert response.json()["state"] == "accepted"
@@ -319,10 +132,6 @@ def test_read_get_does_not_reclaim_an_expired_lease(tmp_path) -> None:
row to `unknown`/`pending` — a read that mutates the queue and can collide
with the drain. The read path must open without initializing.
"""
from datetime import UTC, datetime, timedelta
from one_memory.memory_server.contracts import BatchIdentity, WriteRequest
path = tmp_path / "one-memory.sqlite3"
queue = DurableWriteRegistry(path).queue
created = queue.submit(
@@ -342,10 +151,7 @@ def test_read_get_does_not_reclaim_an_expired_lease(tmp_path) -> None:
((datetime.now(UTC) - timedelta(hours=1)).isoformat(), created.job_id),
)
app = FastAPI()
app.state.memory_write_queue_path = path
app.include_router(router())
client = TestClient(app)
_app, client = _durable_client(path)
def row():
with sqlite3.connect(path) as db:
@@ -358,7 +164,7 @@ def test_read_get_does_not_reclaim_an_expired_lease(tmp_path) -> None:
)
before = row()
response = client.get(f"/api/v1/memory-writes/{created.request_id}")
response = client.get(f"{ROUTE}/{created.request_id}")
after = row()
assert response.status_code == 200
@@ -373,11 +179,6 @@ def test_read_get_on_an_unmigrated_database_is_503_not_500(tmp_path) -> None:
with sqlite3.connect(path) as db:
db.execute("CREATE TABLE unrelated (id INTEGER)")
app = FastAPI()
app.state.memory_write_queue_path = path
app.include_router(router())
client = TestClient(app)
_app, client = _durable_client(path)
response = client.get("/api/v1/memory-writes/does-not-exist")
assert response.status_code == 503
assert client.get(f"{ROUTE}/does-not-exist").status_code == 503
@@ -1,18 +1,24 @@
import inspect
from types import SimpleNamespace
from one_memory import api, cli, log_file, runtime
from one_memory import api, cli, hy_runtime_config, log_file, runtime
def test_production_serve_passes_only_supported_app_options(monkeypatch, tmp_path):
import uvicorn
serve_source = inspect.getsource(cli.serve)
assert serve_source.index("configure_hy_runtime_environment(settings.data_dir)") < (
serve_source.index("from one_memory.api import create_app")
)
settings = SimpleNamespace(data_dir=tmp_path, host="127.0.0.1", port=12345)
captured = {}
startup_order = []
marker = object()
create_app_signature = inspect.signature(api.create_app)
def factory(*args, **kwargs):
startup_order.append("create_app")
create_app_signature.bind(*args, **kwargs)
captured.update(kwargs)
return marker
@@ -27,12 +33,18 @@ def test_production_serve_passes_only_supported_app_options(monkeypatch, tmp_pat
}
monkeypatch.setattr(cli, "load_settings", lambda: settings)
monkeypatch.setattr(
hy_runtime_config,
"configure_hy_runtime_environment",
lambda data_dir: startup_order.append(f"configure_hy:{data_dir}"),
)
monkeypatch.setattr(log_file, "start_logging_to_file", lambda _: None)
monkeypatch.setattr(runtime, "parent_pid", lambda: None)
monkeypatch.setattr(api, "create_app", factory)
monkeypatch.setattr(uvicorn, "run", run)
cli.serve()
assert captured == {"manage_runtime": True}
assert startup_order == [f"configure_hy:{tmp_path}", "create_app"]
def test_maintenance_serve_does_not_own_the_runtime(monkeypatch, tmp_path):
@@ -59,6 +71,11 @@ def test_maintenance_serve_does_not_own_the_runtime(monkeypatch, tmp_path):
}
monkeypatch.setattr(cli, "load_settings", lambda: settings)
monkeypatch.setattr(
hy_runtime_config,
"configure_hy_runtime_environment",
lambda _data_dir: None,
)
monkeypatch.setattr(log_file, "start_logging_to_file", lambda _: None)
monkeypatch.setattr(runtime, "parent_pid", lambda: None)
monkeypatch.setattr(api, "create_app", factory)
@@ -1,125 +0,0 @@
"""Search imported conversations with no generative model configured at all.
This answers the question a user would ask: I imported my history and I
never set up a model, can One Memory find things I said? This goes through the
product's own entry points only: authorize the source, run an import job,
search `/api/v1/context/search`.
"""
import asyncio
import json
from datetime import UTC, datetime
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.sources.registry import build_source_catalog
from one_memory.secrets import InMemorySecretStore
from tests.source_consent import authorized_request, thaw_source
TOKEN = "raw-search-no-model"
AUTH = {"Authorization": f"Bearer {TOKEN}"}
LICENCE = "我们最后决定 One Memory 用 MIT 协议发布,不再考虑 GPL。"
BACKUP = "备份先写到本地磁盘,再同步到对象存储。"
SYNONYM = "开源软件的授权条款最后怎么挑的"
pytestmark = [pytest.mark.integration, pytest.mark.serial]
def transcript(root, name, *texts):
lines = [
json.dumps(
{
"type": "response_item",
"timestamp": datetime(2026, 9, 1, tzinfo=UTC).isoformat(),
"payload": {
"type": "message",
"role": "user",
"content": [{"type": "text", "text": text}],
},
}
)
for text in texts
]
(root / f"rollout-{name}.jsonl").write_text("\n".join(lines) + "\n")
def application(tmp_path):
home = tmp_path / "home"
root = home / ".codex/sessions"
root.mkdir(parents=True)
transcript(root, "licence", LICENCE)
transcript(root, "backup", BACKUP)
data = tmp_path / "data"
data.mkdir()
return create_app(
Settings(
_env_file=None,
data_dir=data,
source_home=home,
api_token=SecretStr(TOKEN),
automatic_memory_processing=False,
live_sync_enabled=False,
),
source_catalog={"codex": build_source_catalog(home=home, environment={})["codex"]},
secret_store=InMemorySecretStore(),
)
async def imported(app, client):
assert (await client.get("/api/v1/model-profiles")).json() == [], "no model may be configured"
authorized = await client.post(
"/api/v1/sources/codex/authorize",
json=await authorized_request(client, "codex"),
)
authorized.raise_for_status()
await thaw_source(client, "codex", authorized.json())
started = await client.post("/api/v1/import-jobs", json={"source_id": "codex"})
started.raise_for_status()
job = await app.state.import_supervisor.wait(started.json()["job_id"])
assert job.status == "completed" and job.stored == 2
async def search(client, query, *, attempts=1):
for remaining in range(attempts, 0, -1):
response = await client.post("/api/v1/context/search", json={"query": query}, headers=AUTH)
response.raise_for_status()
body = response.json()
if body["items"] or remaining == 1:
return body
await asyncio.sleep(0.5)
raise AssertionError("unreachable")
def texts(body):
return [
message["text"]
for item in body["items"]
if item["kind"] == "raw"
for message in item["messages"]
]
def test_shared_words_are_found_without_any_model_or_index(tmp_path):
async def scenario():
app = application(tmp_path)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app), base_url="http://test", headers=AUTH
) as client,
):
await imported(app, client)
body = await search(client, "MIT 协议")
assert body["state"] == "grounded"
assert any(LICENCE in text for text in texts(body))
missed = await search(client, SYNONYM)
# Different words are not a lexical match, and there is no private
# vector index to pretend otherwise.
assert not texts(missed)
asyncio.run(scenario())
@@ -102,7 +102,7 @@ def test_the_read_does_not_scale_its_query_count_with_the_page(library):
the citation subquery 0.08 s. The UDF runs several queries per memory —
proofs, versions, active citations, evidence sizes — so the count grows
with the page, and the cost grows again when the library is being written
to. That is what put both `/api/v1/memories` and the map over their five
to. That is what put both the legacy memory list and the map over their five
second interrupt.
The per-memory lookups can be read once for the whole page. Quadrupling
-255
View File
@@ -1,255 +0,0 @@
"""The Reader process stays read-only and never blocks a read on its side effects."""
import asyncio
from sqlite3 import OperationalError
from pydantic import SecretStr
from one_memory.imports.models import (
EvidenceInput,
EvidenceOrigin,
OriginMetadata,
WrittenBy,
)
from one_memory.imports.storage import EvidenceStore
from one_memory.memory_governance import MemoryDraft, MemoryGovernanceStore, MemoryKind
from one_memory.reader_app import ReaderSettings, SideEffectForwarder, _ReaderGovernance
from one_memory.recall import GovernedRecallTarget, RecallRequest, RecallState
from tests.person_door import as_person
def _settings(tmp_path) -> ReaderSettings:
return ReaderSettings(
data_dir=tmp_path,
api_token=SecretStr("reader-token"),
core_port=8765,
core_api_token="core-token",
)
def test_side_effects_are_queued_not_written(tmp_path) -> None:
settings = _settings(tmp_path)
forwarder = SideEffectForwarder(settings)
for index in range(5):
forwarder.enqueue({"tool": "om_recall", "index": index})
# The queue is in memory only: nothing was written to the data directory.
assert not any(tmp_path.iterdir())
assert forwarder.queue.qsize() == 5
def test_a_full_queue_drops_instead_of_blocking(tmp_path) -> None:
settings = _settings(tmp_path)
forwarder = SideEffectForwarder(settings)
for index in range(300):
forwarder.enqueue({"index": index})
assert forwarder.queue.qsize() == 256
assert forwarder.dropped == 44
def test_an_unreachable_core_counts_the_drop(tmp_path) -> None:
settings = _settings(tmp_path).model_copy(update={"core_port": 1})
async def scenario() -> None:
forwarder = SideEffectForwarder(settings)
forwarder.enqueue({"tool": "om_recall"})
task = asyncio.ensure_future(forwarder._drain())
# The read answered already; the forward never waits on it.
deadline = asyncio.get_running_loop().time() + 2.0
while forwarder.dropped < 1 and asyncio.get_running_loop().time() < deadline:
await asyncio.sleep(0.01)
task.cancel()
await asyncio.gather(task, return_exceptions=True)
assert forwarder.dropped >= 1
asyncio.run(scenario())
def test_the_reader_connection_refuses_writes(tmp_path) -> None:
"""query_only is what keeps a read from becoming a write.
Without it, a store that opens DDL would silently take the write lock in a
process whose whole reason for existing is to stay out of the writer's way.
"""
from one_memory.database_access import main_database
database_path = tmp_path / "one-memory.sqlite3"
with main_database(database_path).write() as connection:
connection.execute("CREATE TABLE t (a INTEGER)")
with main_database(database_path).read() as connection:
try:
connection.execute("INSERT INTO t VALUES (1)")
except OperationalError as error:
assert "readonly" in str(error).lower() or "read-only" in str(error).lower()
else:
raise AssertionError("reader connection accepted a write")
def test_reader_governance_supports_non_probe_recall(tmp_path) -> None:
database = tmp_path / "one-memory.sqlite3"
evidence = (
EvidenceStore(database)
.put(
EvidenceInput(
source_id="cursor",
conversation_id="conversation",
message_id="message",
role="user",
written_by=WrittenBy.PERSON,
verbatim="用户偏好本地优先架构。",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
)
.record
)
governance = MemoryGovernanceStore(database)
memory = as_person(governance).create(
MemoryDraft(
user_id="local",
kind=MemoryKind.CHAT,
content="用户偏好本地优先架构",
scopes=("all-apps",),
evidence_ids=(evidence.evidence_id,),
),
request_id="reader-recall",
)
result = GovernedRecallTarget(_ReaderGovernance(database)).recall(
RecallRequest(query="本地优先架构")
)
assert result.state is RecallState.GROUNDED
assert [hit.memory_id for hit in result.hits] == [memory.memory_id]
def test_importing_the_reader_never_loads_the_write_side() -> None:
"""The Reader must not load keyring, HY, Chroma or ONNX.
Checked in a child process because `sys.modules` in the test process is
already polluted by whatever else the suite imported; only a fresh
interpreter can say what importing `reader_app` drags in. This is the
check the Reader's own startup guard performs, so a regression here is
also a Reader that exits immediately on launch.
"""
import subprocess
import sys
from pathlib import Path
source_root = Path(__file__).resolve().parents[1] / "src"
script = (
"import sys\n"
"import one_memory.reader_app\n"
"loaded = sorted(\n"
" name.split('.')[0]\n"
" for name in sys.modules\n"
" if name.split('.')[0] in ('keyring', 'chromadb', 'onnxruntime', 'hy_memory')\n"
")\n"
"sys.stdout.write(repr(loaded))\n"
)
result = subprocess.run(
[sys.executable, "-c", script],
capture_output=True,
text=True,
env={"PYTHONPATH": str(source_root), "PATH": "/usr/bin:/bin"},
timeout=120,
)
assert result.returncode == 0, result.stderr
assert result.stdout.strip() == "[]", result.stdout
def test_reader_reports_the_port_it_serves_on(tmp_path) -> None:
"""D103: the supervisor can only learn the Reader port from stderr.
It asks for an ephemeral port (`ONE_MEMORY_READER_PORT=0`), so the Reader
has to claim a real port first, print `reader_port:<n>` and then serve on
that same port. Letting uvicorn pick its own port, or printing a different
one, leaves the supervisor waiting out its 10s timeout and then connecting
to a port nobody listens on.
Run in a child process for the same reason as the import guard above:
`create_reader_app` refuses to start when the suite has already imported
`hy_memory`, so only a fresh interpreter exercises this path.
"""
import json
import subprocess
import sys
from pathlib import Path
from one_memory.database_access import main_database
source_root = Path(__file__).resolve().parents[1] / "src"
script = """
import io
import json
import re
import sys
from pathlib import Path
import uvicorn
from pydantic import SecretStr
from one_memory import cli
from one_memory.config import Settings
served_on = []
served_with = []
def fake_run(application, *, host=None, port=None, **_kwargs):
served_on.append(port)
served_with.append(_kwargs)
uvicorn.run = fake_run
cli.load_settings = lambda: Settings(
_env_file=None,
data_dir=Path(sys.argv[1]),
api_token=SecretStr("token"),
port=0,
)
stderr = io.StringIO()
previous = sys.stderr
sys.stderr = stderr
try:
cli.serve(reader=True)
finally:
sys.stderr = previous
printed = stderr.getvalue()
match = re.search(r"reader_port:([0-9]+)", printed)
json.dump(
{
"printed": printed,
"served_on": served_on,
"served_with": served_with,
"port": int(match.group(1)) if match else None,
},
sys.stdout,
)
"""
database = tmp_path / "one-memory.sqlite3"
with main_database(database).write() as connection:
connection.execute("CREATE TABLE t (a INTEGER)")
result = subprocess.run(
[sys.executable, "-c", script, str(tmp_path)],
capture_output=True,
text=True,
env={
"PYTHONPATH": str(source_root),
"PATH": "/usr/bin:/bin",
"HOME": str(tmp_path),
"ONE_MEMORY_DATA_DIR": str(tmp_path),
"ONE_MEMORY_READER_PORT": "0",
"ONE_MEMORY_CORE_PORT": "8765",
"ONE_MEMORY_CORE_API_TOKEN": "core-token",
},
timeout=120,
)
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert payload["port"], payload["printed"]
# Port 0 means "any free port", so the reported value must be a real one.
assert payload["port"] > 0
assert payload["served_on"] == [payload["port"]]
assert payload["served_with"] == [{"reload": False, "access_log": False}]
@@ -1,41 +0,0 @@
"""A search hit carries who wrote the row it matched.
The Reader used to send back everything about a matched row except its author,
and the one client that lists hits as dialogue filled the gap in by hand, as
"the user". These pin the author to the row so the gap cannot reopen.
"""
from one_memory.reader_app import _conversation_hit
def _row(**overrides: object) -> dict[str, object]:
row: dict[str, object] = {
"evidence_id": "evidence-1",
"conversation_id": "conversation-1",
"source_id": "codex",
"workspace": "/Users/k/one-memory",
"project": "one-memory",
"created_at": "2026-10-04T10:00:00Z",
"verbatim": "红眼航班要靠过道",
"role": "assistant",
"written_by": "agent",
}
row.update(overrides)
return row
def test_an_assistant_row_stays_attributed_to_the_assistant() -> None:
hit = _conversation_hit(_row())
assert hit["role"] == "assistant"
assert hit["written_by"] == "agent"
def test_a_row_the_person_wrote_is_not_relabelled_on_the_way_out() -> None:
hit = _conversation_hit(_row(role="user", written_by="person"))
assert hit["role"] == "user"
assert hit["written_by"] == "person"
def test_a_long_row_is_excerpted_and_an_empty_one_is_not_a_crash() -> None:
assert _conversation_hit(_row(verbatim="航" * 400))["excerpt"] == "航" * 200
assert _conversation_hit(_row(verbatim=None))["excerpt"] == ""
@@ -1,196 +0,0 @@
"""RCL-04 against a real HY instance instead of a stand-in.
Enable it with a configured, redacted model-profile database:
ONE_MEMORY_MODEL_TEST_DB=/path/to/profile.sqlite3 \
uv run --directory apps/core pytest -q tests/test_recall_fusion_live.py
The fusion tests elsewhere drive a fake client, so they prove the aggregator
merges two lanes but not that the semantic lane is really HY. This one puts a
memory in through the product's own import path, lets real HY store it, and
asks two questions of the same live instance:
a natural question -- overlaps the stored wording on one word, so only
HY's semantics can reach the answer
the stored wording -- which the lexical lane can answer on its own
The second is not decoration. The natural question comes back with an empty
raw lane, and empty alone cannot tell an unauthorized lane from an unwired one
from a merely strict one. Asking both in the same run settles which.
Seeding goes through `add_extracted` on purpose. This test isolates semantic
storage and retrieval from generative extraction quality, which has separate
deterministic and live acceptance coverage.
"""
import asyncio
import os
import runpy
from pathlib import Path
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.recall import _terms
from one_memory.secrets import KeyringSecretStore
from tests.source_consent import connection_request
PROFILE_DB = os.environ.get("ONE_MEMORY_MODEL_TEST_DB")
pytestmark = [
pytest.mark.live,
pytest.mark.serial,
pytest.mark.skipif(not PROFILE_DB, reason="explicit configured provider required"),
]
STORED = "项目周会固定避开周五,星期五不安排项目周会。"
# Shares no term with the stored wording at all, so the lexical half of the
# memory lane cannot reach it and a hit can only have come from HY. The test
# checks that disjointness rather than trusting this comment.
QUESTION = "团队碰头会应该躲开一周里的哪一天"
# The same fact in the stored wording: the control for the lexical lane.
QUOTED = "项目周会固定避开周五"
def test_both_lanes_answer_through_one_real_hy_instance(tmp_path, monkeypatch):
from hy_memory.client import HyMemoryClient
# This run is fast because `add_extracted` needs no generation, which makes
# "real HY" easy to assert and impossible to see. Counting the calls on the
# real client is the difference between the two.
stored: list[list[str]] = []
searched: list[str] = []
real_add_extracted = HyMemoryClient.add_extracted
real_search = HyMemoryClient.search
def counted_add_extracted(self, contents, **kwargs):
stored.append(list(contents))
return real_add_extracted(self, contents, **kwargs)
def counted_search(self, query=None, **kwargs):
searched.append(query)
return real_search(self, query, **kwargs)
def forbidden(*args, **kwargs):
pytest.fail("Already-extracted content must not be replayed as raw chat")
monkeypatch.setattr(HyMemoryClient, "add_extracted", counted_add_extracted)
monkeypatch.setattr(HyMemoryClient, "search", counted_search)
monkeypatch.setattr(HyMemoryClient, "add", forbidden)
async def scenario():
helpers = runpy.run_path(
str(Path(__file__).resolve().parents[3] / "scripts/run_matter_first_import.py")
)
directory = tmp_path / "data"
directory.mkdir(parents=True)
helpers["copy_model_configuration"](Path(PROFILE_DB), directory / "one-memory.sqlite3")
app = create_app(
Settings(
data_dir=directory,
source_home=tmp_path / "home",
api_token=SecretStr("live-fusion"),
live_sync_enabled=False,
),
secret_store=KeyringSecretStore(),
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer live-fusion"},
) as client,
):
upload = await client.post(
"/api/v1/import-files?filename=weekly.txt&input_kind=memory_content",
content=STORED.encode(),
)
assert upload.status_code == 201, upload.text
receipt = upload.json()
(
await client.post(
f"/api/v1/import-files/{receipt['upload_id']}/commit",
json={"confirmed": True},
)
).raise_for_status()
linked = await client.post(
"/api/v1/sources/connect",
json=await connection_request(
client,
[receipt["source_id"]],
enable_sync=True,
),
)
linked.raise_for_status()
job = await app.state.import_supervisor.wait(linked.json()["jobs"][0]["job_id"])
assert job.stored == 1 and job.errors == 0, job
deadline = asyncio.get_running_loop().time() + 300
while True:
counts = app.state.sdk_memory_runtime.status_counts()
if not any(
counts.get(state, 0)
for state in ("pending", "running", "waiting_for_model")
):
break
if asyncio.get_running_loop().time() >= deadline:
raise AssertionError(f"SDK write queue did not drain: {counts}")
await asyncio.sleep(0.05)
# Imported content is not shared with every app by default. Sharing
# it here is what the memory lane is later asked to find.
memories = (await client.get("/api/v1/memories")).json()
assert memories, "real HY stored nothing to search for"
for item in memories:
(
await client.post(
f"/api/v1/memories/{item['memory_id']}/correct",
json={
"request_id": f"share-{item['memory_id']}",
"expected_revision": item["revision"],
"content": item["content"],
"scopes": ["all-apps"],
},
)
).raise_for_status()
async def search(query: str) -> dict:
response = await client.post(
"/api/v1/context/search",
json={"query": query, "kinds": ["raw", "memory"]},
)
assert response.status_code == 200, response.text
return response.json()
return await search(QUESTION), await search(QUOTED)
asked, quoted = asyncio.run(scenario())
assert stored == [[STORED]], stored
# Both questions reached the real client, so the memory lane below is HY
# answering and not a cache, a fixture or a lexical result mislabelled.
assert searched == [QUESTION, QUOTED], searched
# The memory lane matches terms as well as taking HY's ids, so a shared
# word would let it answer without HY and make the next assertion empty.
assert not (set(_terms(STORED)) & set(_terms(QUESTION))), "the question is not disjoint"
memory = asked["paths"]["memory"]
assert memory["status"] == "ok" and memory["returned_count"] > 0, asked["paths"]
remembered = [item for item in asked["items"] if item["kind"] == "memory"]
assert any("周五" in item["content"] or "星期五" in item["content"] for item in remembered), (
remembered
)
# Same instance, same authorization, wording the lexical lane can match.
# Without this the empty raw lane above would be unreadable.
assert quoted["paths"]["raw"]["status"] == "ok", quoted["paths"]
assert quoted["paths"]["raw"]["returned_count"] > 0, quoted["paths"]
assert {item["kind"] for item in quoted["items"]} == {"raw", "memory"}, quoted["items"]
# So the finding is about reach, not plumbing: the lexical lane wants most
# of the query's words present and drops a question overlapping on one.
# The synthetic benchmark measured the same shape (lexical-only hit 0.429);
# this is that result on a memory the product actually stored.
assert asked["paths"]["raw"]["status"] == "empty", asked["paths"]
@@ -0,0 +1,261 @@
"""MemHub_recall degrades to the imported conversations instead of answering empty.
Without a usable generation model no memory can be distilled, and with no
memory in the library a memory search can only come back empty. In both cases
recall searches the SDK raw layer and says so with `mode: "raw"`. Original
conversations stay under their own grant: recall's grant does not widen it.
"""
import asyncio
import json
from datetime import UTC, datetime
import pytest
from httpx import ASGITransport, AsyncClient
from pydantic import SecretStr
from one_memory import api
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.hy_sdk_types import HySdkRawHit
from one_memory.recall import RecallHit, RecallReceipt, RecallRequest, RecallState
from one_memory.recall_fallback import recall_with_fallback
from one_memory.secrets import InMemorySecretStore
AT = int(datetime(2026, 9, 1, tzinfo=UTC).timestamp())
def _raw(raw_id, content, *, source="cursor", score=0.8):
return HySdkRawHit(
raw_id=raw_id,
content=content,
user_id="local",
agent_id="a",
session_id=f"s-{raw_id}",
source=source,
memory_at=AT,
score=score,
)
def _memory_receipt(request, *hits):
return RecallReceipt(
state=RecallState.GROUNDED if hits else RecallState.EMPTY,
query=request.query,
scope=request.scope,
hits=hits,
used_characters=sum(len(hit.content) for hit in hits),
truncated=False,
)
def _memory_hit():
return RecallHit(
memory_id="m1",
content="The user prefers tea.",
kind="l2_fact",
scopes=(),
evidence_ids=(),
updated_at=datetime(2026, 9, 1, tzinfo=UTC),
score=0.9,
content_format="hy-sdk",
)
def _fakes(*, ready, memories, has_memories, raw, sources=None):
calls = {"memory": 0, "raw": 0, "library": 0, "sources": None}
async def model_ready():
return ready
async def recall_memories(request):
calls["memory"] += 1
return _memory_receipt(request, *memories)
async def library():
calls["library"] += 1
return has_memories
async def raw_sources():
return sources
async def search_raw(query, limit, allowed):
calls["raw"] += 1
calls["sources"] = allowed
return tuple(raw)
return calls, {
"model_ready": model_ready,
"recall_memories": recall_memories,
"has_memories": library,
"raw_sources": raw_sources,
"search_raw": search_raw,
}
def run(request, fakes):
return asyncio.run(recall_with_fallback(request, **fakes))
def test_with_a_model_and_memories_recall_searches_memories_only():
calls, fakes = _fakes(ready=True, memories=[_memory_hit()], has_memories=True, raw=[])
receipt = run(RecallRequest(query="tea"), fakes)
assert receipt.mode == "memory" and receipt.fallback_reason is None
assert [hit.memory_id for hit in receipt.hits] == ["m1"]
assert calls["raw"] == 0 and calls["library"] == 0
def test_without_a_model_recall_searches_the_raw_layer():
calls, fakes = _fakes(
ready=False, memories=[], has_memories=True, raw=[_raw("r1", "我喜欢喝茶")]
)
receipt = run(RecallRequest(query="tea"), fakes)
assert calls["memory"] == 0
assert receipt.mode == "raw" and receipt.fallback_reason == "model_unavailable"
assert receipt.state == RecallState.GROUNDED
hit = receipt.hits[0]
assert (hit.memory_id, hit.kind, hit.content_format) == ("r1", "raw", "raw")
assert (hit.source, hit.session_id) == ("cursor", "s-r1")
assert hit.updated_at == datetime(2026, 9, 1, tzinfo=UTC)
def test_with_no_memories_yet_recall_searches_the_raw_layer():
calls, fakes = _fakes(ready=True, memories=[], has_memories=False, raw=[_raw("r1", "x")])
receipt = run(RecallRequest(query="tea"), fakes)
assert calls["memory"] == 1 and calls["library"] == 1
assert receipt.mode == "raw" and receipt.fallback_reason == "no_memories"
def test_with_memories_but_none_relevant_the_answer_stays_an_empty_memory_result():
calls, fakes = _fakes(ready=True, memories=[], has_memories=True, raw=[_raw("r1", "x")])
receipt = run(RecallRequest(query="tea"), fakes)
assert receipt.mode == "memory" and receipt.hits == ()
assert calls["raw"] == 0
def test_raw_fallback_respects_the_callers_raw_sources():
calls, fakes = _fakes(
ready=False,
memories=[],
has_memories=False,
raw=[_raw("r1", "a", source="cursor"), _raw("r2", "b", source="codex")],
sources=frozenset({"cursor"}),
)
receipt = run(RecallRequest(query="q"), fakes)
assert calls["sources"] == frozenset({"cursor"})
assert [hit.memory_id for hit in receipt.hits] == ["r1"]
def test_a_caller_with_no_raw_grant_gets_nothing_rather_than_a_wider_read():
calls, fakes = _fakes(
ready=False,
memories=[],
has_memories=False,
raw=[_raw("r1", "a")],
sources=frozenset(),
)
receipt = run(RecallRequest(query="q"), fakes)
assert calls["raw"] == 0
assert receipt.mode == "raw" and receipt.hits == () and receipt.state == RecallState.EMPTY
def test_raw_fallback_keeps_the_count_and_character_budgets():
_, fakes = _fakes(
ready=False,
memories=[],
has_memories=False,
raw=[_raw(f"r{index}", "x" * 150) for index in range(5)],
)
by_count = run(RecallRequest(query="q", limit=2), fakes)
by_chars = run(RecallRequest(query="q", limit=10, character_budget=300), fakes)
assert len(by_count.hits) == 2 and by_count.truncated_by.value == "count_budget"
assert len(by_chars.hits) == 2 and by_chars.truncated_by.value == "character_budget"
assert by_chars.used_characters == 300
def test_the_probe_never_falls_back():
calls, fakes = _fakes(ready=False, memories=[], has_memories=False, raw=[_raw("r1", "x")])
receipt = run(RecallRequest(query="__om_probe__"), fakes)
assert calls["memory"] == 1 and calls["raw"] == 0
assert receipt.mode == "memory"
EXTERNAL = {
"X-One-Memory-Read-Context": "external",
"X-One-Memory-Installation-Id": "24361b44-5a15-4f87-ad09-05571dc281c3",
"X-One-Memory-Installation-Revision": "1",
}
@pytest.mark.fresh_sdk
def test_recall_route_falls_back_and_keeps_the_raw_grant(tmp_path, monkeypatch):
searched: list[object] = []
async def no_model(_app):
return False
async def raw(_app, query, *, limit, sources):
searched.append(sources)
return (_raw("r1", "上周决定用 Postgres", source="cursor"),)
monkeypatch.setattr(api, "_recall_model_ready", no_model)
monkeypatch.setattr(api, "_search_sdk_raw", raw)
async def scenario():
app = create_app(
Settings(
data_dir=tmp_path / "data",
source_home=tmp_path,
api_token=SecretStr("recall-test"),
imports_enabled=False,
automatic_memory_processing=False,
live_sync_enabled=False,
),
secret_store=InMemorySecretStore(),
)
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer recall-test"},
) as client,
):
local = await client.post("/api/v1/recall", json={"query": "数据库"})
recall_only = await client.post(
"/api/v1/recall",
json={"query": "数据库"},
headers={
**EXTERNAL,
"X-One-Memory-Allowed-Tools": json.dumps(["om_recall"]),
"X-One-Memory-Allowed-Scopes": json.dumps(["all-apps"]),
},
)
return local.json(), recall_only.json()
local, recall_only = asyncio.run(scenario())
assert local["mode"] == "raw" and local["fallback_reason"] == "model_unavailable"
assert local["hits"][0]["content_format"] == "raw"
assert local["hits"][0]["source"] == "cursor"
# A trusted local caller reads every source.
assert searched[0] is None
# A host granted recall alone may not read original conversations.
assert recall_only["mode"] == "raw" and recall_only["hits"] == []
assert len(searched) == 1
@@ -8,12 +8,16 @@ from pydantic import SecretStr
from one_memory.api import create_app
from one_memory.config import Settings
from one_memory.imports.access import opaque_workspace_id
from one_memory.imports.models import EvidenceInput, EvidenceOrigin, OriginMetadata, WrittenBy
from one_memory.memory_governance import MemoryDraft, MemoryKind
from one_memory.secrets import InMemorySecretStore
def test_external_scope_cannot_override_shared_or_host_workspace_boundary(tmp_path):
def test_external_scope_cannot_override_shared_or_host_workspace_boundary(tmp_path, monkeypatch):
# Memory recall, not the raw fallback a home without a model would take.
async def model_ready(_app):
return True
monkeypatch.setattr("one_memory.api._recall_model_ready", model_ready)
async def scenario():
app = create_app(
Settings(
@@ -32,47 +36,22 @@ def test_external_scope_cannot_override_shared_or_host_workspace_boundary(tmp_pa
headers={"Authorization": "Bearer scope-test"},
) as client,
):
source = app.state.evidence_store.put(
EvidenceInput(
source_id="manual",
conversation_id="scope",
message_id="1",
role="user",
written_by=WrittenBy.PERSON,
verbatim="Project meeting avoids Friday",
origin_meta={EvidenceOrigin.FILE: OriginMetadata()},
)
).record
scopes = ["private", "all-apps", opaque_workspace_id("claude-code", "/test/project")]
ids = []
for scope in scopes:
item = app.state.person_writes.create(
MemoryDraft(
user_id="local",
kind=MemoryKind.CHAT,
content="Project meeting avoids Friday",
scopes=(scope,),
evidence_ids=(source.evidence_id,),
),
request_id=f"create-{scope}",
)
ids.append(item.memory_id)
workspace = opaque_workspace_id("claude-code", "/test/project")
semantic = AsyncMock(return_value=())
app.state.recall_target.semantic_search = semantic
app.state.memory_answerer.semantic_search = semantic
external = {
"X-One-Memory-Read-Context": "external",
"X-One-Memory-Installation-Id": "24361b44-5a15-4f87-ad09-05571dc281c3",
"X-One-Memory-Installation-Revision": "1",
"X-One-Memory-Allowed-Tools": json.dumps(["om_recall", "om_ask"]),
"X-One-Memory-Allowed-Scopes": json.dumps(["all-apps", scopes[2]]),
"X-One-Memory-Allowed-Scopes": json.dumps(["all-apps", workspace]),
}
for endpoint, key in [
("/api/v1/recall", "query"),
("/api/v1/memories/ask", "question"),
("/api/v1/memories/ask/stream", "question"),
("/api/v2/memories/ask", "question"),
("/api/v2/memories/ask/stream", "question"),
]:
for scope in ["*", "private", scopes[2], "unrelated-scope"]:
for scope in ["*", "private", workspace, "unrelated-scope"]:
response = await client.post(
endpoint, json={key: "Project meeting", "scope": scope}, headers=external
)
@@ -81,28 +60,8 @@ def test_external_scope_cannot_override_shared_or_host_workspace_boundary(tmp_pa
normal = await client.post(
"/api/v1/recall", json={"query": "Project meeting"}, headers=external
)
assert [hit["memory_id"] for hit in normal.json()["hits"]] == [ids[1]]
workspace = await client.post(
"/api/v1/recall",
json={
"query": "Project meeting",
"scope": scopes[2],
"workspace_path": "/test/project",
},
headers=external,
)
assert [hit["memory_id"] for hit in workspace.json()["hits"]] == [ids[2]]
denied_expansion = await client.post(
"/api/v1/recall",
json={"query": "Project meeting", "workspace_path": "/test/project"},
headers={**external, "X-One-Memory-Allowed-Scopes": '["all-apps"]'},
)
assert denied_expansion.status_code == 200
assert [hit["memory_id"] for hit in denied_expansion.json()["hits"]] == [ids[1]]
internal = await client.post(
"/api/v1/recall", json={"query": "Project meeting", "scope": "*"}
)
assert {hit["memory_id"] for hit in internal.json()["hits"]} == set(ids)
assert normal.status_code == 200
semantic.assert_awaited()
asyncio.run(scenario())
@@ -16,7 +16,6 @@ from fastapi import FastAPI
from starlette.requests import Request
from one_memory.api import _require_external_recall_scope
from one_memory.workspace_resolution import resolve_workspace_scopes
class _Access:
@@ -61,73 +60,3 @@ def test_workspace_lookup_for_an_external_recall_runs_off_the_loop(monkeypatch)
)
asyncio.run(scenario())
def test_the_ask_path_also_reads_the_workspace_lookup_off_the_loop(monkeypatch) -> None:
"""Same read, other door: /ask reaches it through MemoryAnswerer.retrieve.
That file is not ours to edit, so the read is done in `_request_answerer`
— where both ask paths meet — and the answerer is handed a callable that
only resolves against the list already fetched. Two things have to hold:
the read is off the loop, and the scopes are exactly what the inline
version would have returned. The second is the one that matters: this
callable decides which memories a host may be answered from.
"""
known = [_Known("/repo", "p")]
seen: list[int] = []
class Store:
def known_workspaces(self, *, user_id: str) -> list[Any]:
seen.append(threading.get_ident())
return known
class Access:
scopes = frozenset({"all-apps"})
def require(self, tool: str, scopes: frozenset[str]) -> None:
return None
import one_memory.api as api
monkeypatch.setattr(api, "external_access", lambda headers: Access())
monkeypatch.setattr(api, "_evidence_store", lambda app: Store())
answerer = _BareAnswerer()
answerer.resolve_workspace = lambda path: ("replaced-by-the-fix",)
monkeypatch.setattr(api, "_memory_answerer", lambda app: answerer)
async def scenario() -> None:
loop_thread = threading.get_ident()
request = Request(
{
"type": "http",
"method": "POST",
"path": "/api/v1/memories/ask",
"headers": [],
"query_string": b"",
}
)
resolved = await api._request_answerer(FastAPI(), request, workspace_path="/repo")
assert seen, "the workspace lookup was never called"
assert seen[0] != loop_thread, (
"known_workspaces ran on the event loop thread; it must run in a worker"
)
# The boundary must be unchanged, not merely non-empty.
assert resolved.resolve_workspace("/repo") == resolve_workspace_scopes("/repo", known)
asyncio.run(scenario())
class _Known:
def __init__(self, workspace: str, project: str) -> None:
self.source_id = "cursor"
self.workspace = workspace
self.project = project
class _BareAnswerer:
"""Just enough of MemoryAnswerer for the per-request copy to work."""
filter_memories = staticmethod(tuple)
resolve_workspace = None
@@ -119,14 +119,10 @@ def test_post_cutover_metadata_repair_restores_source_and_original_times(store):
),
)
import chromadb
from chromadb.config import Settings as ChromaSettings
from one_memory.sdk_vector_reattach import open_offline_chroma
vector = data_dir / "hy-memory" / "governed-v2" / "vector"
collection = chromadb.PersistentClient(
path=str(vector),
settings=ChromaSettings(anonymized_telemetry=False),
).get_or_create_collection("memory")
collection = open_offline_chroma(vector).get_or_create_collection("memory")
collection.add(
ids=["sdk-1"],
documents=["remember"],
@@ -230,13 +226,10 @@ def test_metadata_repair_reaches_memories_the_cutover_backfilled_under_legacy_id
"INSERT INTO evidence VALUES ('e7','workbuddy','2026-09-01T00:00:00+00:00',NULL)"
)
import chromadb
from chromadb.config import Settings as ChromaSettings
from one_memory.sdk_vector_reattach import open_offline_chroma
vector = data_dir / "hy-memory" / "governed-v2" / "vector"
client = chromadb.PersistentClient(
path=str(vector), settings=ChromaSettings(anonymized_telemetry=False)
)
client = open_offline_chroma(vector)
collection = client.get_or_create_collection("memory")
imported = int(datetime(2026, 10, 20, tzinfo=UTC).timestamp())
collection.add(
@@ -256,9 +249,7 @@ def test_metadata_repair_reaches_memories_the_cutover_backfilled_under_legacy_id
now=datetime(2026, 10, 25, tzinfo=UTC),
)
client = chromadb.PersistentClient(
path=str(vector), settings=ChromaSettings(anonymized_telemetry=False)
)
client = open_offline_chroma(vector)
rows = client.get_collection("memory").get(ids=["legacy-7", "unlinked"], include=["metadatas"])
client.close()
metadata = dict(zip(rows["ids"], rows["metadatas"], strict=True))
@@ -546,13 +537,9 @@ def test_finished_jobs_without_input_do_not_stop_the_migration(store):
def _chroma(home: Path):
import chromadb
from chromadb.config import Settings as ChromaSettings
from one_memory.sdk_vector_reattach import open_offline_chroma
return chromadb.PersistentClient(
path=str(home / "hy-memory" / "governed-v2" / "vector"),
settings=ChromaSettings(anonymized_telemetry=False),
)
return open_offline_chroma(home / "hy-memory" / "governed-v2" / "vector")
def _fill(client, name: str, ids: list[str], *, text: str = "old") -> None:
@@ -599,6 +586,38 @@ def test_legacy_vectors_are_renamed_and_newer_writes_are_kept(tmp_path) -> None:
assert reattach_legacy_vectors(tmp_path, profile) == ()
def test_the_readback_can_open_the_store_right_after_reattach(tmp_path, monkeypatch) -> None:
"""The 1.0.4 upgrade failed its first launch at `verify.readback`.
Reattach opened Chroma with `anonymized_telemetry=False` and never closed
it. The SDK then opened the same directory in the same process with
default settings, and Chroma refused: "An instance of Chroma already
exists ... with different settings". The second launch only passed because
nothing was left to reattach.
"""
import chromadb
from one_memory.hy_sdk_runtime import builtin_retrieval_snapshot, vector_collection_bases
from one_memory.sdk_vector_reattach import reattach_legacy_vectors
profile = builtin_retrieval_snapshot()
current, legacy = vector_collection_bases(profile)
seeded = _chroma(tmp_path)
_fill(seeded, f"{legacy[0]}_4", ["a", "b"])
seeded.close()
# The migration process has not configured the SDK environment yet.
monkeypatch.delenv("ANONYMIZED_TELEMETRY", raising=False)
assert len(reattach_legacy_vectors(tmp_path, profile)) == 1
# What the SDK does next, in the same process: no settings of its own.
sdk = chromadb.PersistentClient(path=str(tmp_path / "hy-memory" / "governed-v2" / "vector"))
try:
assert sdk.get_collection(f"{current}_4").count() == 2
finally:
sdk.close()
def _confirmed(store, *memory_ids: str, kind: str = "chat") -> None:
with main_database(store.database_path).write() as database, database:
for memory_id in memory_ids:
@@ -1070,18 +1089,14 @@ def _app_settings(tmp_path) -> Settings:
async def _authority(tmp_path) -> tuple[dict[str, object], str]:
"""What a starting Core reads as authority, and the gate it then holds."""
app = create_app(_app_settings(tmp_path), secret_store=InMemorySecretStore())
async with (
app.router.lifespan_context(app),
AsyncClient(
transport=ASGITransport(app=app),
base_url="http://test",
headers={"Authorization": "Bearer authority-test"},
) as client,
):
response = await client.get("/api/v2/memory-authority")
assert response.status_code == 200
return response.json(), app.state.memory_authority.value
async with app.router.lifespan_context(app):
authority, migration_id = app.state.sdk_data_plane_migrations.authority_state()
return (
{"authority": authority.value, "migration_id": migration_id},
app.state.memory_authority.value,
)
@pytest.mark.integration
@@ -1196,7 +1211,6 @@ def test_the_sdk_memory_map_draws_sdk_memories_without_legacy_tables(tmp_path) -
)
app.state.sdk_memory_runtime.graph_snapshot = graph_snapshot
assert (await client.get("/api/v1/memories/graph")).status_code == 409
response = await client.get(
"/api/v2/memories/graph",
params={"source": "cursor", "offset": 100, "limit": 100},
Loaded 100 of 273 files, more files were not shown because too many files have changed in this diff. Show more