Compare commits

..
Author SHA1 Message Date
1d0fc66e0d refactor(memhub): remove private Hy Memory bridge
Co-authored-by: kristinhe <kristinhe@tencent.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-10-22 19:53:51 +08:00
447 changed files with 55428 additions and 30202 deletions

No files matched your search

+61 -167
View File
@@ -2,156 +2,6 @@
Hy Memory and MemHub are versioned and released independently.
## Unreleased
- The SDK migration no longer fails its first run on a home with legacy
vectors: reattaching them left a Chroma instance open with its own settings,
and the readback that followed could not open the same store
("An instance of Chroma already exists ... with different settings").
- MemHub "添加记忆" (add a memory) no longer waits behind unprocessed history.
Notes are claimed ahead of the event-time window that keeps history strictly
old to new; they still run one at a time and in order within their session.
- The SDK memory page now reads its list, count, source and time labels and
graph from one page response for one source filter, so rows, totals and the
map no longer disagree. Memories with no recorded source are shown and
filterable as "unidentified source"; library search keeps only memories
that contain the words within the SDK's relevance window.
- Metadata repair v2 follows `governed_memory_evidence` across every
pre-cutover bundle, which restores the source of memories the cutover
backfilled without one, and moves impossible future conversation times back
to the write time.
- Reused the existing three-column memory map for SDK-authority list/detail
views and removed the retired "original evidence" surface. SDK-native status,
source, conversation time and creation time now render without fabricated
revisions; source and text filters are applied by the paged Core endpoint.
- SDK memory search now calls the SDK search contract instead of listing the
entire library in Core. Corrupt empty records are skipped and diagnosed
without mutating data or failing a page, and post-cutover metadata repair
restores source and original times from the verified migration bundle.
- Kept Cursor's WAL change token separate from the file mtime so a hash can no
longer appear as a conversation date centuries in the future.
- A trusted packaged Runtime now repairs a dirty same-build installation by
staging a clean replacement, and `runtime_service_not_ready` is rendered as
user-facing startup guidance instead of an internal code.
- Fixed legacy-to-SDK startup migration to adopt native Coding Experience and
Working Fact IDs instead of importing them as Chat layers. Raw migration now
rebuilds bounded user/assistant turns from the latest three days, excludes
tools, and enforces the same rolling retention after cutover.
- Paused host capture writes with a live Desktop migration fence; Cursor
History catch-up resumes from its existing checkpoint after cutover.
- Removed MemHub's private `hy_memory_bridge` implementation. MemHub now calls
public `HyMemoryClient` methods at runtime; prompt/reconcile/response
monkey-patches and private VDB/Coding runtime access were deleted. Kuzu Graph
projection/admin and stopped-writer App migration are the documented product
exceptions.
- Added a MemHub-managed native Hermes MemoryProvider with automatic prefetch,
persistent per-session memory-ID deduplication, authorized Broker reads, and
recall-context stripping to prevent memory recursion.
- Added an extensible MemHub interface-localization layer with Simplified
Chinese, English, and Spanish catalogs shared by Web and Electron-native
surfaces. Interface language remains independent from memory content and
model processing language.
- Fixed governed-memory revisions and repeated evidence links failing when the
visibility dirty-marker trigger encounters the same memory twice. Startup
now replaces the affected legacy trigger without changing schema versions
or governed and visibility data.
- Kept post-write authority verification failures available for read-only
reconciliation while preventing automatic or generic confirmed retries from
invoking the model again. Automatic reconciliation skips those sticky rows
before applying its batch limit, and linked durable jobs exclusively own
retry scheduling for their memory tasks.
- Removed MemHub's task-scoped HyPermission store wrappers and global
model-content consent switch. The trusted embedded Hy Memory SDK now uses
its native local stores directly.
- Existing installations ignore the retired config flag, readmit previously
parked pre-invocation work, and drop only the obsolete
`hy_node_permissions` proof table. Evidence, governed memories, model
profiles, SDK stores, and source/host grants are preserved.
## Hy Memory 2.0.6 — 2026-10-27
- Published
[`hy-memory==2.0.6`](https://pypi.org/project/hy-memory/2.0.6/).
- `search(..., rerank=False)` skips every reranking stage for that one
call (the local cross-encoder, the memory-graph rerank, and the BM25 rerank of
working/coding facts), so results come back in retrieval-score order. The
flag travels on the read request, so concurrent searches do not affect each
other. The default (`True`) keeps current behavior; `/api/v1/search` accepts
it too.
- MMR diversity selection now runs only when the requested `limit` is
below 20 (`MMR_MAX_TOP_K`) and is skipped for larger limits, where its
quadratic token-overlap pass is slow and gains little. Previously it ran for
every limit. `MEMORY_HYBRID_V2_MMR_LAMBDA` still sets the weight.
Near-duplicate demotion follows the same gate: it compares each candidate
with every earlier non-duplicate, which is quadratic in a pool that grows
with `limit`, and large result pages should show similar items anyway.
- `search(scene="all")` searches chat, working and coding at the same time
and merges them with Reciprocal Rank Fusion (k=60) over each scene's own
ranking, so no scene is crowded out by the other scenes' score scales. Each
result says which scene and kind it came from. A scene that fails or is turned
off is reported in `errors` and does not fail the search. Source filters apply
to all three.
- `list_memories(scene="all")` lists chat, working and coding memories
together, newest update first, with `limit`/`offset` paging over the merged
order and `total` summing all three. `scene="coding"` and `"working"` now
honor `include_sources` / `exclude_sources`, and `total` counts only the
filtered memories.
- `update()` and `delete()` now work on working/coding experiences and
facts, not just chat memories. They take the same optional `kind` / `variant`
as `get()`, and an ID found in more than one place raises
`AmbiguousMemoryIdError` instead of guessing. Updating an experience replaces
its solution and keeps its task, unless a new `task=` is also passed; updating
a fact replaces its statement. Deleting removes the stored row and its vectors.
- `list_memories(scene="chat" | "coding" | "working")` lists one kind of
memory. `coding` and `working` list that scene's whole store, newest update
first (`workspace_id` still narrows it), and `total` is now the real count
rather than the page size. The new `memory_stats()` returns the count and the
latest created/updated time for each scene and their total, without reading
any memory text. Previously coding and working memories could not be listed
or counted without a workspace, so a library of 45 memories listed 22.
- Reconcile keeps time order. Both reconcile prompts now state that only
`memory_at` decides which memory is newer, that an older memory must never
supersede or overwrite a newer one, and that an unknown time is not newer;
the Chinese prompt now lists `memory_at` among the new-memory fields. The
reconciler also enforces this in code: a SUPERSEDE or UPDATE aimed at a memory
with a later `memory_at` becomes an ADD, so the newer memory stays current.
- The HTTP server returns 409 `ambiguous_memory_id` and 400 for a bad
`kind`/`variant` instead of 500, passes `kind`/`variant` (and `task` on PUT)
to get/update/delete, and no longer reads a PUT/DELETE query string as part
of the memory id.
- `memory_stats()` reads working/coding times, which are stored as naive UTC,
as UTC rather than local time.
## Hy Memory 2.0.5 — 2026-10-22
- Published
[`hy-memory==2.0.5`](https://pypi.org/project/hy-memory/2.0.5/).
- Extended `get()` across Chat, Coding Experience, Working Fact, and raw IDs;
the returned `layer`, `kind`, and `variant` identify the actual record.
- Added explicit raw retrieval through the existing `search()` and
`list_memories()` contracts, including user, agent, session, and source
filters.
- Added public embedding-service injection and corrected extraction language
detection for structured OpenAI-style messages.
- Added lifecycle status filters, durable idempotent write outcomes, and
quiesced local data-plane snapshot/restore.
- Kept legacy migration mechanics out of the public Client API; separate
migration-only raw/import methods from development previews were removed.
- Physical delete now reports partial VDB/Graph cleanup as failure.
- Coding search no longer injects the version-evolution diff block by default;
opt in with `MEMORY_CODING_CHAIN_BLOCK_ENABLED=true`.
- Serialized SDK SQLite cache/history connection lifecycle and tightened
Coding reconciliation to its agent/session/workspace authority.
- Clarified persistent caller context through `meta`, while `metadata` remains
legacy processing configuration.
## Hy Memory 2.0.5.dev4 — 2026-10-21
- Development preview used for SDK-authority migration validation; not the
final `2.0.5` release.
- Published as
[`hy-memory==2.0.5.dev4`](https://pypi.org/project/hy-memory/2.0.5.dev4/).
## Hy Memory 2.0.5.dev3 — 2026-10-21
- Development preview; not the final `2.0.5` release.
@@ -178,12 +28,6 @@ Hy Memory and MemHub are versioned and released independently.
APIs, offline local data-plane snapshot/restore, and fail-closed physical
delete receipts.
## Hy Memory 2.0.4 — 2026-10-17
- Published
[`hy-memory==2.0.4`](https://pypi.org/project/hy-memory/2.0.4/) as a
maintenance release.
## Hy Memory 2.0.3 — 2026-10-17
- Added an optional runtime HTTP transport factory so embedded products can
@@ -194,6 +38,66 @@ Hy Memory and MemHub are versioned and released independently.
unchanged memory extraction, reconciliation, storage, and Coding/Working
behavior.
## Unreleased
- Removed MemHub's private `hy_memory_bridge` implementation. MemHub now calls
public `HyMemoryClient` methods through explicit runtime, write, migration,
and read modules; prompt/reconcile/response monkey-patches and private
VDB/Coding access were deleted. Kuzu Graph projection/admin remains the one
documented product exception.
- Extended public SDK `get()` to resolve Chat, Coding Experience, and Working
Fact IDs, including owner/session fields and explicit ambiguity handling;
added constructor-time public embedding-service injection.
- Added a MemHub-managed native Hermes MemoryProvider with automatic prefetch,
persistent per-session memory-ID deduplication, authorized Broker reads, and
recall-context stripping to prevent memory recursion.
- Added an extensible MemHub interface-localization layer with Simplified
Chinese, English, and Spanish catalogs shared by Web and Electron-native
surfaces. Interface language remains independent from memory content and
model processing language.
- `search(..., layers=["l1_raw"])` now explicitly searches SDK raw records in
pro/ultra mode and keeps existing user/agent/session/source filters; default
search continues to hide raw.
- Added `import_memory_exact` for idempotent migration of an existing semantic
memory with its canonical ID, lifecycle and supersession links, without
extraction or reconciliation.
- Added SDK-authoritative lifecycle listing with explicit status filters,
idempotent durable write outcomes, exact raw import/get/list APIs, and
quiesced local data-plane snapshot/restore contracts for the MemHub
authority migration.
- Physical delete now reports incomplete VDB/Graph cleanup as a partial failure
instead of claiming success after only one store was changed.
- Coding search no longer injects the version-evolution diff block
(`chain_block`) by default. Enable with
`MEMORY_CODING_CHAIN_BLOCK_ENABLED=true` or
`CodingConfig.chain_block_enabled=True` (write-side observe unchanged).
- Fixed governed-memory revisions and repeated evidence links failing when the
visibility dirty-marker trigger encounters the same memory twice. Startup
now replaces the affected legacy trigger without changing schema versions
or governed and visibility data.
- Serialized each SDK SQLite store's shared connection lifecycle so concurrent
history/cache operations cannot race with `close()` and crash the process.
- Scoped Coding reconciliation targets to the current HY authority,
conversation, and workspace, preventing near-duplicate sessions from
mutating an experience or working fact owned by another evidence context.
- Kept post-write authority verification failures available for read-only
reconciliation while preventing automatic or generic confirmed retries from
invoking the model again. Automatic reconciliation skips those sticky rows
before applying its batch limit, and linked durable jobs exclusively own
retry scheduling for their memory tasks.
- Clarified that `add(metadata=...)` carries legacy SDK processing options,
while persistent caller business context belongs in `meta`.
- Added an opt-in `get(..., include_raw=True)` path for reading `L1_RAW`
memories, including memories written in lite mode. The default remains
hidden for backward compatibility.
- Removed MemHub's task-scoped HyPermission store wrappers and global
model-content consent switch. The trusted embedded Hy Memory SDK now uses
its native local stores directly.
- Existing installations ignore the retired config flag, readmit previously
parked pre-invocation work, and drop only the obsolete
`hy_node_permissions` proof table. Evidence, governed memories, model
profiles, SDK stores, and source/host grants are preserved.
## Hy Memory 2.0.2 — 2026-10-17
- Published
@@ -208,17 +112,7 @@ Hy Memory and MemHub are versioned and released independently.
Because PyPI release metadata is immutable, the next SDK patch release must
carry the updated license metadata; `2.0.1` retains its published metadata.
## MemHub 1.0.5 — Unreleased
- Updated the source version to `1.0.5` (build 14).
- Pinned the public SDK to the stable `hy-memory==2.0.5` release.
## MemHub 1.0.4 — 2026-10-18
- Updated the source version to `1.0.4` (build 13).
- Published the `v1.0.4` source and release metadata.
## MemHub 1.0.3 — 2026-10-17
## MemHub 1.0.3 — Unreleased
- Updated the source version to `1.0.3` (build 7).
- Pinned the public SDK to `hy-memory==2.0.3`.
+6 -7
View File
@@ -273,8 +273,8 @@ catalog 也可以在没有基础 L0 时形成只读的合成画像。
```python
memory = client.get("memory_id")
# raw ID 返回 L1_RAW,fact ID 返回对应事实层。
raw_memory = client.get("raw_id")
# lite 模式写入的是 L1_RAW;按 ID 读取时必须显式选择返回原文。
raw_memory = client.get("memory_id", include_raw=True)
result = client.update("memory_id", "更新后的内容")
result = client.delete("memory_id")
result = client.delete_all(user_id="user_001")
@@ -284,7 +284,7 @@ page = client.list_memories(
limit=100,
offset=0,
order="desc",
layers=["l1_raw"], # 显式列出 raw;不传仍沿用默认语义层列表
layers=None,
with_evolution=False,
)
@@ -294,10 +294,9 @@ result = client.clone_user("user_001", "user_002", agent_id=None)
对应异步方法为 `async_get()`、`async_update()`、`async_delete()`、
`async_delete_all()`、`async_list_memories()`、`async_clone_user()`。
`get()` / `async_get()` 是统一的 ID 精确读取:返回结果的 `layer`、`kind` 和
`variant` 表明实际类型。授权由调用 SDK 前的产品边界完成,不由 SDK 根据 layer
猜测。`list_memories()` 默认仍不包含 `L1_RAW`;显式传
`layers=["l1_raw"]` 时按同一列表合同返回 raw。
`get()` / `async_get()` 默认继续隐藏 `L1_RAW`,因此不会改变现有调用方的原文访问
边界。只有显式传入 `include_raw=True` 才返回该层;这也是 lite 模式按 ID 读取
写入内容及其 `meta` 的方式。MemHub 的 HTTP 读取接口不开放此参数。
普通 `list_memories()` 返回 `vdb` 桶,并在 Graph 有结果时附带 `graph`;传入
`workspace_id` 时返回生产力记忆的 `scene="productivity"` 和 `experiences`。
+7 -14
View File
@@ -743,7 +743,6 @@ RECONCILE_PROMPT_ZH = """你是一个记忆整合系统(Memory Reconciler)
* content
* owner
* memory_at(发生时间)
* tags
new memories 没有 ID。
@@ -792,7 +791,6 @@ new memories 没有 ID。
* 无损合并
* 保留所有有效信息
* 不要产生重复 memory
* 两者在某个值上不一致时,保留 memory_at 更晚的那个值;绝不用更早的 memory 覆盖更新的值
### 同一次事件的补充 vs 另一次发生
@@ -825,7 +823,6 @@ new memories 没有 ID。
* 新 memory 与旧 memory 在同一维度冲突
* 两者不能同时为真
* 旧 memory 已不再代表当前状态
* 新 memory 的 memory_at 不早于旧 memory 的 memory_at
例如:
@@ -840,7 +837,6 @@ new memories 没有 ID。
* wording 差异
* 同义表达
* 可共存偏好
* 新 memory 的 memory_at 早于旧 memory:它是更早的状态,绝不能取代更新的状态(见「时间规则」)
要求:
@@ -863,7 +859,7 @@ new memories 没有 ID。
### 把多条旧记忆折叠进同一条链
如果**两条或更多**已有记忆描述的是**同一维度**,而新记忆是该维度上的最新变化(它的 memory_at 晚于所有被折叠的记忆),
如果**两条或更多**已有记忆描述的是**同一维度**,而新记忆是该维度上的最新变化,
你可以把它们折叠进同一条演化链。此时用 `memory_ids`(有序列表,**旧→新**)
代替单个 `memory_id`:
@@ -955,16 +951,13 @@ SUPERSEDE 不需要复制旧内容,因为旧节点仍存在。
## 时间规则
每条 memory(新的和已有的)都带 memory_at,表示它发生的时间。记忆**不一定按发生顺序写入**:
历史可能晚导入,导入还没处理完时用户也可能手动添加记忆。所以一条 new memory 可能**比已有记忆更早**。
`memory_at: null`
* 新旧**只看 memory_at**。"New memories" 只表示"正在写入",不表示"更近"。
* 更早的 memory 绝不能取代更新的 memory:不能 SUPERSEDE 一条更新的记忆,也不能用 UPDATE 拿更早的值覆盖更新的值。
* 如果 new memory 比与它冲突的已有记忆更早:保留已有记忆作为当前状态;把 new memory 以"过去状态"ADD
(用过去时,并带上时间,例如"2023 年用户住在上海"),没有新信息就不输出。
* `memory_at: null` 表示时间未知。未知**不算更新**:不要用 memory_at 为 null 的 memory 去 SUPERSEDE 一条有时间的记忆。
* 不要编造时间顺序。
* 不要仅因为“看起来更新”就 supersede。
表示时间未知。
不要编造时间顺序。
不要仅因为“看起来更新”就 supersede。
---
+3 -105
View File
@@ -221,8 +221,6 @@ Rules:
* merge losslessly
* preserve all useful information
* do NOT create duplicates
* if the two disagree on a value, keep the value from the memory with the later
`memory_at`; never let an older memory overwrite a newer value
### Same occurrence vs. another occurrence
@@ -257,7 +255,6 @@ Use ONLY when:
* the new memory contradicts an existing memory
* both cannot be true at the same time
* the old memory would now give the wrong current-state answer
* the new memory's `memory_at` is the same as or LATER than the existing memory's `memory_at`
Examples:
@@ -271,8 +268,6 @@ Do NOT use for:
* compatible preferences
* wording differences
* semantic equivalence
* a new memory whose `memory_at` is EARLIER than the existing memory's — it is an
older state and must never replace a newer one (see "Time order")
Rules:
@@ -294,8 +289,7 @@ Rules:
### Folding several existing memories into one chain
If TWO OR MORE existing memories track the SAME dimension and the new memory is
the latest change on that dimension — its `memory_at` is later than every folded
memory's `memory_at` — you may fold them into a single evolution
the latest change on that dimension, you may fold them into a single evolution
chain. Use `memory_ids` (an ordered list, OLDEST → NEWEST) instead of a single
`memory_id`:
@@ -324,7 +318,8 @@ older memories' text into it; they remain queryable in the chain history.
* Separate dated occurrences stay separate ADDs; use `same_series` only for the
same subject and same concrete activity/event type.
* Keep each memory focused on one coherent topic.
* Follow "Time order" below.
* Do not invent chronology.
* `memory_at: null` means unknown time.
* **NEVER merge or supersede across different subjects.** The subject is whoever
the content is about: "The user ..." vs "John ..." vs "Mary ...". Memories about
different people are independent facts even when the sentence is otherwise
@@ -337,27 +332,6 @@ older memories' text into it; they remain queryable in the chain history.
---
# Time order
Every memory, new or existing, has `memory_at`: when it happened. Memories are
NOT always written in the order they happened — history can be imported late,
and a memory can be added while older history is still being processed. A new
memory can therefore be OLDER than an existing one.
* Decide which memory is newer ONLY by `memory_at`. "New memories" means "being
written now", not "more recent".
* An older memory must never replace a newer one: no SUPERSEDE of a newer
memory, and no UPDATE that overwrites a newer value with an older one.
* If a new memory is older than an existing memory it conflicts with, keep the
existing memory as the current state. ADD the new one as a past state, worded
in the past with its time (for example "In 2023 the user lived in Shanghai"),
or output nothing if it adds no information.
* `memory_at: null` means unknown time. Unknown is NOT newer: never SUPERSEDE a
memory that has a known `memory_at` with one whose `memory_at` is null.
* Do not invent chronology.
---
# Optional: `related`
On any op you MAY add a `related` array recording how the memory being written
@@ -467,81 +441,6 @@ def append_reconcile_structure_prompt(
return prompt + RECONCILE_STRUCTURE_METADATA_PROMPT_EN
def _as_aware(value: Any) -> Optional[datetime]:
"""memory_at as a comparable aware datetime; None when unknown or unreadable."""
if isinstance(value, str) and value:
try:
value = datetime.fromisoformat(value)
except (TypeError, ValueError):
return None
if not isinstance(value, datetime):
return None
# Stored nodes rebuild memory_at from epoch seconds as naive local time
# (MemoryNode.from_dict), so a naive value is local, not UTC.
return value if value.tzinfo is not None else value.astimezone()
def enforce_time_order(
ops: List["ReconcileOp"],
new_memory_at: Any,
existing: Dict[str, Any],
) -> List["ReconcileOp"]:
"""An older memory must never replace a newer one, whatever the model says.
Memories are not always written in the order they happened: history is
imported late, and a person can add a memory while older history is still
being processed. The prompt tells the model to compare `memory_at`; this
makes it hold. A SUPERSEDE or UPDATE aimed at an existing memory whose
`memory_at` is later than the memory being written is turned into an ADD,
so the newer memory stays the current state. Unknown times are not
compared. A content-less UPDATE (the legacy DELETE mapping) is dropped.
"""
written_at = _as_aware(new_memory_at)
if written_at is None:
return ops
kept: List[ReconcileOp] = []
for op in ops:
if op.op not in ("SUPERSEDE", "UPDATE"):
kept.append(op)
continue
targets = list(op.memory_ids) if op.memory_ids else [op.memory_id]
newer = [
target for target in targets
if target and (node := existing.get(target)) is not None
and (target_at := _as_aware(getattr(node, "memory_at", None))) is not None
and target_at > written_at
]
if not newer:
kept.append(op)
continue
logger.warning(
"[reconciler] %s blocked: memory_at %s is older than existing %s; "
"kept the newer memory",
op.op, written_at.isoformat(), ",".join(newer),
)
if not op.content:
continue
kept.append(ReconcileOp(
op="ADD",
content=op.content,
layer=op.layer,
owner=op.owner,
tags=list(op.tags),
tag_ners=list(op.tag_ners),
topic_paths=list(op.topic_paths),
event_facet=dict(op.event_facet),
topic_assignments=list(op.topic_assignments),
structure_references=list(op.structure_references),
topic_paths_provided=op.topic_paths_provided,
event_facet_provided=op.event_facet_provided,
topic_assignments_provided=op.topic_assignments_provided,
speculate=op.speculate,
related=list(op.related),
reason=f"time_order: older than {','.join(newer)}; {op.reason}"[:500],
))
return kept
def build_reconcile_prompt(
*,
new_memories: List[str],
@@ -1500,7 +1399,6 @@ class MemoryReconciler:
_tok_total += response.tokens_used
raw_response = response.content
ops = self._parse_ops(raw_response)
ops = enforce_time_order(ops, current_time, candidate_map)
ops = self._inherit_exact_input_structure(
ops, new_memories_with_meta,
)
+518 -1118
View File
File diff suppressed because it is too large. Load diff
+4 -162
View File
@@ -217,21 +217,6 @@ def _in_clause(column: str, values: Optional[List[str]]) -> Tuple[str, List[Any]
return f"{column} IN ({placeholders})", list(values)
def _source_where(
include_sources: Optional[List[str]], exclude_sources: Optional[List[str]],
) -> Tuple[str, List[Any]]:
"""Caller-source filter as SQL (``1`` when unfiltered); exclusion wins."""
clauses: List[str] = []
params: List[Any] = []
if include_sources:
clauses.append(f"source IN ({','.join('?' for _ in include_sources)})")
params.extend(include_sources)
if exclude_sources:
clauses.append(f"source NOT IN ({','.join('?' for _ in exclude_sources)})")
params.extend(exclude_sources)
return (" AND ".join(clauses) or "1"), params
def _scope_matches(
row_workspace_id: Optional[str],
row_branch: Optional[str],
@@ -618,86 +603,6 @@ class CodingMemoryStore:
logger.info(f"[coding-store] deleted experience_id={experience_id}")
return True
async def edit_experience(
self,
experience_id: str,
*,
user_id: Optional[str] = None,
task: Optional[str] = None,
solution: Optional[str] = None,
search_keys: Optional[List[str]] = None,
) -> Optional[CodingMemory]:
"""调用方手动改一条经验(client.update 用)。返回改后的经验;不存在返回 None。
与 update() 的区别:update 是写入链路的覆盖式重写,带写授权校验(同
agent/session/workspace 才能改);这里是用户对已有条目的直接编辑,只动
给定的字段,其余原样保留,因此不做授权校验。
向量行只嵌 task + search_keys(solution 不进向量),所以只改 solution 时
不重嵌、不碰向量库;改了 task / search_keys 才删旧 key 行重写。
updated_at 总是刷新。
"""
prev = await self.get_by_id(experience_id, user_id=user_id)
if prev is None:
return None
updated = deepcopy(prev)
if task is not None:
updated.task = task
if solution is not None:
updated.solution = solution
if search_keys is not None:
updated.search_keys = list(search_keys)
updated.updated_at = datetime.utcnow()
await _run_in_pool(self._upsert_sql_sync, updated)
if updated.task != prev.task or updated.search_keys != prev.search_keys:
await self._delete_vec_keys_by_count(experience_id, 1 + len(prev.search_keys))
await self._upsert_vec_keys(updated)
logger.info(f"[coding-store] edited experience_id={experience_id}")
return updated
async def edit_fact(
self,
fact_id: str,
*,
statement: str,
user_id: Optional[str] = None,
) -> Optional[WorkingFact]:
"""调用方手动改一条事实的 statement。返回改后的事实;不存在返回 None。
statement 是事实层的检索文本,所以 active 事实会重嵌并覆盖它那一行向量
(node_id 就是 fact_id,upsert 即覆盖)。superseded 事实的向量行早已撤掉,
这里不把它加回去,否则会让过期结论重新占召回名额。
"""
prev = await self.get_fact(fact_id, user_id=user_id)
if prev is None:
return None
updated = deepcopy(prev)
updated.statement = statement
await _run_in_pool(self._upsert_fact_sql_sync, updated)
if updated.status == "active":
await self._upsert_fact_vec(updated)
logger.info(f"[coding-store] edited fact_id={fact_id}")
return updated
async def delete_fact(self, fact_id: str, user_id: Optional[str] = None) -> bool:
"""物理删除一条事实(SQLite 行 + 事实向量行),返回是否真有删到。
别处留下的引用(经验的 source_fact_ids、后继事实的 fact_link)不清:
召回侧本来就按存在性过滤死链(get_many / get_facts 查不到自然掉队)。
"""
prev = await self.get_fact(fact_id, user_id=user_id)
if prev is None:
logger.info(f"[coding-store] delete fact skipped (not found): {fact_id}")
return False
if self._fact_vec is not None:
try:
await self._fact_vec.delete(fact_id)
except Exception as e:
logger.debug(f"[coding-store] delete fact vec {fact_id} ignored: {e}")
await _run_in_pool(self._delete_fact_sql_sync, fact_id, prev.user_id)
logger.info(f"[coding-store] deleted fact_id={fact_id}")
return True
# ------------------------------------------------------------
# 读取
# ------------------------------------------------------------
@@ -782,84 +687,29 @@ class CodingMemoryStore:
branch: Optional[str] = None,
limit: int = 100,
offset: int = 0,
include_sources: Optional[List[str]] = None,
exclude_sources: Optional[List[str]] = None,
) -> List["CodingMemory"]:
"""按 workspace_id(+可选 branch)纯过滤列出该 user 的完整 coding memory。
作用域规则见 _scope_where(与 search 路径共用同一套)。无语义排序,按
updated_at 倒序(最近修改优先)。include_sources / exclude_sources 按
source 列在 SQL 里过滤(排除优先),所以 LIMIT/OFFSET 在过滤之后生效。
updated_at 倒序(最近修改优先)。
"""
return await _run_in_pool(
self._list_by_workspace_sync, user_id, workspace_id, branch, limit, offset,
include_sources, exclude_sources,
self._list_by_workspace_sync, user_id, workspace_id, branch, limit, offset
)
def _list_by_workspace_sync(
self, user_id: str, workspace_id: str, branch: Optional[str],
limit: int, offset: int,
include_sources: Optional[List[str]] = None,
exclude_sources: Optional[List[str]] = None,
) -> List["CodingMemory"]:
scope_sql, scope_params = _scope_where(workspace_id, branch)
source_sql, source_params = _source_where(include_sources, exclude_sources)
with self._lock:
assert self._conn is not None
sql = (f"SELECT * FROM coding_memory_meta WHERE user_id = ? AND {scope_sql} "
f"AND {source_sql} "
f"ORDER BY updated_at DESC, experience_id ASC LIMIT ? OFFSET ?")
params: List[Any] = ([user_id] + scope_params + source_params
+ [limit, offset])
f"ORDER BY updated_at DESC LIMIT ? OFFSET ?")
params: List[Any] = [user_id] + scope_params + [limit, offset]
cur = self._conn.execute(sql, params)
return [_row_to_memory(r) for r in cur.fetchall()]
async def stats(
self,
user_id: str,
workspace_id: Optional[str] = None,
branch: Optional[str] = None,
include_sources: Optional[List[str]] = None,
exclude_sources: Optional[List[str]] = None,
) -> Dict[str, Any]:
"""经验数、最新创建/修改时间、active 事实数。纯聚合,不读正文。
作用域与 list_by_workspace 相同:workspace_id 为空表示该 user 全部;
来源过滤同样与 list_by_workspace 一致,count 因此恒等于分页可翻到的总数。
"""
return await _run_in_pool(
self._stats_sync, user_id, workspace_id, branch,
include_sources, exclude_sources,
)
def _stats_sync(
self, user_id: str, workspace_id: Optional[str], branch: Optional[str],
include_sources: Optional[List[str]] = None,
exclude_sources: Optional[List[str]] = None,
) -> Dict[str, Any]:
scope_sql, scope_params = _scope_where(workspace_id, branch)
source_sql, source_params = _source_where(include_sources, exclude_sources)
with self._lock:
assert self._conn is not None
row = self._conn.execute(
f"SELECT COUNT(*) AS n, MAX(created_at) AS created, "
f"MAX(updated_at) AS updated FROM coding_memory_meta "
f"WHERE user_id = ? AND {scope_sql} AND {source_sql}",
[user_id] + scope_params + source_params,
).fetchone()
facts = self._conn.execute(
f"SELECT COUNT(*) AS n FROM working_fact "
f"WHERE user_id = ? AND status = 'active' AND {scope_sql} "
f"AND {source_sql}",
[user_id] + scope_params + source_params,
).fetchone()
return {
"count": int(row["n"] or 0),
"latest_created_at": _parse_dt(row["created"]) if row["created"] else None,
"latest_updated_at": _parse_dt(row["updated"]) if row["updated"] else None,
"facts": int(facts["n"] or 0),
}
# ------------------------------------------------------------
# 召回
# ------------------------------------------------------------
@@ -1238,14 +1088,6 @@ class CodingMemoryStore:
(experience_id, user_id),
)
def _delete_fact_sql_sync(self, fact_id: str, user_id: str) -> None:
with self._lock:
assert self._conn is not None
self._conn.execute(
"DELETE FROM working_fact WHERE fact_id = ? AND user_id = ?",
(fact_id, user_id),
)
def _get_by_id_sync(
self, experience_id: str, user_id: Optional[str]
) -> Optional[CodingMemory]:
@@ -160,28 +160,6 @@ def near_dup_demote(
return out
# MMR 只在调用方请求的 top-k(search 的 limit)小于这个值时自动启用。
# mmr_select 的贪心是 O(k^2 * pool) 次 Jaccard,k 大时代价陡增,而大 k 本身已
# 足够容纳多样性,收益很小。权重仍由 MEMORY_HYBRID_V2_MMR_LAMBDA 控制。
MMR_MAX_TOP_K = 20
def mmr_applies(top_k: int, lambda_: float) -> bool:
"""本次检索是否走 MMR:λ 在有效区间内,且 top_k < MMR_MAX_TOP_K。"""
return 0.0 < float(lambda_) < 0.999 and int(top_k) < MMR_MAX_TOP_K
def near_dup_applies(top_k: int) -> bool:
"""本次检索是否走近重复降权:与 MMR 同一个 top-k 门槛。
near_dup_demote 让每个候选和此前所有"非重复"候选两两算 Jaccard,候选
互不相似时是 O(pool^2);池子随 limit 放大(约 max(3*limit, 30) + 关键词
命中),limit 达到 MMR_MAX_TOP_K 时代价同样陡增。大 k 的调用方(如记忆
面板翻页)要的是全部相关项,相似项也应照常出现,降权没有收益。
"""
return int(top_k) < MMR_MAX_TOP_K
def mmr_select(
hits: Sequence[Dict[str, Any]],
limit: int,
-5
View File
@@ -401,11 +401,6 @@ class ReadRequest:
intention_limit: int = 0 # Intention 路的 topk,默认 0 关闭
# 重排开关(按次生效)。False 时跳过全部重排阶段:本地 cross-encoder
# (recall.local_rerank_enabled) 与记忆关系图重排 (_apply_graph_rerank),
# 结果按召回分排序。走请求对象而非环境变量,并发请求互不影响。
rerank: bool = True
# 时间过滤
created_after: Optional[float] = None # Unix timestamp (float),只返回 gmt_created >= 此值的记忆
+5 -18
View File
@@ -57,9 +57,7 @@ from ._retrieval.ner import (
memory_matches_time_hints,
)
from ._retrieval.diversity import (
near_dup_applies,
near_dup_demote,
mmr_applies,
mmr_select,
social_boilerplate_demote,
pool_df_demote,
@@ -1281,10 +1279,7 @@ class BaseReadPipeline(ReadPipeline):
SAME_SERIES_TIMELINE_ON = bool(
getattr(self.config.recall, "same_series_timeline_enabled", False)
)
# 按次的重排开关(search(rerank=False)):关掉本地 cross-encoder 与图重排,
# 结果按召回分排序。只读请求对象,不碰共享状态,并发请求互不影响。
RERANK_ON = bool(getattr(request, "rerank", True))
LOCAL_RERANK_ON = RERANK_ON and bool(
LOCAL_RERANK_ON = bool(
getattr(self.config.recall, "local_rerank_enabled", False)
)
LOCAL_RERANK_POOL_FACTOR = max(1, int(
@@ -1436,11 +1431,6 @@ class BaseReadPipeline(ReadPipeline):
MIN_SCORE = _resolve_normal_min_score(request.min_score)
final_limit = request.limit if request.limit > 0 else 10
# MMR 只在调用方的 top-k(final_limit,即 search 的 limit)< MMR_MAX_TOP_K
# 时启用;池子(LOCAL_RERANK 下是 final_limit 的若干倍)不参与判定。
MMR_ON = mmr_applies(final_limit, MMR_LAMBDA)
# 近重复降权同样是候选两两比较,用同一个 top-k 门槛。
NEAR_DUP_ON = near_dup_applies(final_limit)
vdb_sem_limit = max(final_limit * 3, 30)
vdb_kw_limit = max(final_limit * 5, 80)
graph_limit = max(
@@ -2472,7 +2462,7 @@ class BaseReadPipeline(ReadPipeline):
# 只把已选出的 top-N 换个顺序收益很小。
relation_paths: List[Dict[str, Any]] = []
_graph_stats: Dict[str, Any] = {}
if RERANK_ON and self._expansion_on():
if self._expansion_on():
_t_graph = datetime.now()
_pool_before = len(normal_results)
normal_results, relation_paths, _graph_stats = await self._apply_graph_rerank(
@@ -2535,7 +2525,7 @@ class BaseReadPipeline(ReadPipeline):
df_thr=POOL_DF_THR,
penalty=POOL_DF_PENALTY,
)
if NEAR_DUP_ON and NEAR_DUP_THR > 0 and NEAR_DUP_PENALTY > 0:
if NEAR_DUP_THR > 0 and NEAR_DUP_PENALTY > 0:
normal_results = near_dup_demote(
normal_results,
jaccard_thr=NEAR_DUP_THR,
@@ -2548,7 +2538,7 @@ class BaseReadPipeline(ReadPipeline):
))
if LOCAL_RERANK_ON else final_limit
)
if MMR_ON and len(normal_results) > _pre_rerank_limit:
if 0.0 < MMR_LAMBDA < 0.999 and len(normal_results) > _pre_rerank_limit:
normal_results = mmr_select(
normal_results,
_pre_rerank_limit,
@@ -2725,7 +2715,7 @@ class BaseReadPipeline(ReadPipeline):
reverse=True,
)
if (
MMR_ON
0.0 < MMR_LAMBDA < 0.999
and len(multi_seed_pool) > _pre_rerank_limit
):
multi_seed_pool = mmr_select(
@@ -3554,9 +3544,6 @@ class BaseReadPipeline(ReadPipeline):
"episode_capsule_render_mode": EPISODE_CAPSULE_RENDER_MODE,
"same_series_timeline_enabled": SAME_SERIES_TIMELINE_ON,
"local_rerank_enabled": LOCAL_RERANK_ON,
"rerank": RERANK_ON,
"mmr_applied": MMR_ON,
"near_dup_applied": NEAR_DUP_ON,
"local_rerank_model": getattr(
self.config.recall,
"local_rerank_model",
+7 -45
View File
@@ -33,28 +33,9 @@ import threading
from datetime import datetime
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from typing import Any, Dict, Optional
from urllib.parse import parse_qs, urlparse
from .retrieval import AmbiguousMemoryIdError
logger = logging.getLogger("hy_memory.server")
def _namespace_args(values: Dict[str, Any]) -> Dict[str, Any]:
"""kind / variant from a query string (lists) or a JSON body (scalars).
They disambiguate an id that exists in more than one namespace (chat,
experience, fact; working or coding). Absent means "resolve it".
"""
args: Dict[str, Any] = {}
for key in ("kind", "variant"):
value = values.get(key)
if isinstance(value, list):
value = value[0] if value else None
if value:
args[key] = str(value)
return args
# Lazy-init global client
_client = None
_client_lock = None
@@ -167,6 +148,7 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
if path == "/api/v1/metrics":
minutes = 5
if "?" in self.path:
from urllib.parse import parse_qs, urlparse
qs = parse_qs(urlparse(self.path).query)
minutes = int(qs.get("minutes", ["5"])[0])
client = _get_client()
@@ -179,8 +161,7 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
if m:
memory_id = m.group(1)
client = _get_client()
query = parse_qs(urlparse(self.path).query)
result = client.get(memory_id, **_namespace_args(query))
result = client.get(memory_id)
if result is None:
_json_response(self, 404, {"error": "not_found", "memory_id": memory_id})
else:
@@ -189,10 +170,6 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
_json_response(self, 404, {"error": "not_found", "path": path})
except AmbiguousMemoryIdError as e:
_json_response(self, 409, {"error": "ambiguous_memory_id", "detail": str(e)})
except ValueError as e:
_json_response(self, 400, {"error": str(e)})
except Exception as e:
logger.error(f"[server] GET {path} error: {e}", exc_info=True)
_json_response(self, 500, {"error": str(e)})
@@ -236,7 +213,7 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
_json_response(self, 500, {"error": str(e)})
def do_PUT(self):
path = self.path.split("?")[0].rstrip("/")
path = self.path.rstrip("/")
body = _read_json_body(self)
if body is None:
_json_response(self, 400, {"error": "invalid JSON body"})
@@ -251,43 +228,29 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
_json_response(self, 400, {"error": "content is required"})
return
client = _get_client()
result = client.update(
memory_id,
content,
**_namespace_args(body),
**({"task": body["task"]} if body.get("task") else {}),
)
result = client.update(memory_id, content)
_json_response(self, 200, result)
return
_json_response(self, 404, {"error": "not_found", "path": path})
except AmbiguousMemoryIdError as e:
_json_response(self, 409, {"error": "ambiguous_memory_id", "detail": str(e)})
except ValueError as e:
_json_response(self, 400, {"error": str(e)})
except Exception as e:
logger.error(f"[server] PUT {path} error: {e}", exc_info=True)
_json_response(self, 500, {"error": str(e)})
def do_DELETE(self):
path = self.path.split("?")[0].rstrip("/")
path = self.path.rstrip("/")
try:
m = re.match(r"^/api/v1/memories/([^/]+)$", path)
if m:
memory_id = m.group(1)
client = _get_client()
query = parse_qs(urlparse(self.path).query)
result = client.delete(memory_id, **_namespace_args(query))
result = client.delete(memory_id)
_json_response(self, 200, result)
return
_json_response(self, 404, {"error": "not_found", "path": path})
except AmbiguousMemoryIdError as e:
_json_response(self, 409, {"error": "ambiguous_memory_id", "detail": str(e)})
except ValueError as e:
_json_response(self, 400, {"error": str(e)})
except Exception as e:
logger.error(f"[server] DELETE {path} error: {e}", exc_info=True)
_json_response(self, 500, {"error": str(e)})
@@ -492,8 +455,7 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
if key in body:
kwargs[key] = body[key]
for key in ("limit", "min_score", "schema_min_score", "schema_limit", "reader",
"rerank"):
for key in ("limit", "min_score", "schema_min_score", "schema_limit", "reader"):
if key in body:
kwargs[key] = body[key]
+1 -21
View File
@@ -9,14 +9,8 @@
## Architecture Principles
1. Memory 读写与存储 backend **只经 SDK**(config / env)。
2. MemHub 运行时对 memory 的原子操作 **只经 SDK 公共 Client**;缺正常业务
接口先补 SDK。Runtime writers 停止后的 App 启动迁移可直接操作底层存储,
但不得把迁移专用方法加入公共 Client。
2. MemHub 对 memory 的原子操作 **只经 SDK**;缺接口先补 SDK。
3. MemHub **不存 memory 实体**;**可以存 `memory_id`** 做组织/回执。原文进入 **SDK raw**。
4. SDK raw 只保留滚动三天;一条 raw 对应一个完整 turn,只含真人 user 与
assistant,tool 不持久化。提取后的高层 memory 不受该窗口影响。
5. `source_raw_ids` 是 SDK 内部溯源关系,不是产品 Evidence 合同;记忆详情不提供
“原始依据”或按 raw ID 展开原文的入口。
生产入口通过共享 Runtime 租约取得公开 `HyMemoryClient`;MemHub 只在调用前执行
模型配置、密钥和 harness/model 门禁,不维护第二套 memory 正文权威。
@@ -172,20 +166,6 @@ result = client.add_extracted(
该入口仍走 SDK reconcile/持久化;MemHub 只做管理面门禁与可选
`memory_id` 记录。
应用的「添加记忆」、MCP `MemHub_write`(经 broker `POST /memories` 以本机可信身份转发)
共用同一个本机 HTTP 合同,直接新增一条记忆:
```text
POST /api/v2/memories {request_id, text, scope: "all-apps"}
GET /api/v2/memories/additions/{request_id} # 该次新增的状态
```
两者返回同一回执(`evidence_id`、`text`、`scope`、`created_at`、`state`、`task_id`、
`memory_ids`、`error`)。同一 `request_id` 重放返回原回执;内容不同返回 409
`memory_add_conflict`,未知 `request_id` 返回 404 `memory_add_not_found`。新增走持久
写入队列的优先通道,最终调用 `add_extracted` + reconcile。带外部读取信封的请求
一律拒绝(403 `external_endpoint_denied`)。
## Search:搜索记忆
```python
+7 -8
View File
@@ -18,13 +18,13 @@ If you run into a problem, please open an Issue with the version, platform,repro
## 发行与平台合同
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.6`。MemHub 源码版本为 `1.0.5`
(build 14),GitHub 仓库为
源码当前固定使用公开 PyPI 开发预览 `hy-memory==2.0.5.dev3`。MemHub 源码版本为 `1.0.4`
(build 13),GitHub 仓库为
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
`memhub-cli`。当前代码架构见
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)。
`v1.0.5` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
`v1.0.4` 的发行目标是 Linux x64/arm64 Runtime 与 macOS arm64/x64 App 和 Runtime。
Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命周期验收;macOS
制品由发行流水线完成 Developer ID 签名、公证和验收后手动上传。Windows 不支持。
只有同时出现在 GitHub Release 与 `release-manifest.json` 中、且通过公开校验的制品才属于
@@ -45,7 +45,7 @@ Linux 制品由 GitHub Actions 在原生架构 runner 上构建并完成生命
## 仓库结构
- `apps/core`:Python / FastAPI Kernel,负责授权、模型、队列、SDK Runtime 边界与 API。
- `apps/core`:Python / FastAPI Kernel,负责授权、模型、队列、SDK bridge 与 API。
- `apps/cli`:安装、控制和诊断共享 Runtime 的 `one-memory` CLI。
- `apps/web`:React / TypeScript 管理界面。
- `apps/desktop`:Electron 桌面壳与图形安装入口。
@@ -109,16 +109,15 @@ Node、Host Bridge 或模型。用户显式执行 `memhub install` 后,CLI 才
Python Core 和运行所需的 Python 依赖位于 MemHub 受管 Runtime,不写入系统 Python。
MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime 版本目录中。
`npm uninstall` 只移除 npm CLI;`memhub uninstall` 移除服务、受管 Runtime 和 CLI 链接,
默认保留 Memory。只有独立的 `memhub delete-data` 在 Runtime 停止、显式加 `--yes` 并用
`--confirm <数据目录>` 再写一遍同一个目录后才会删除数据(不加参数只预览要删除的目录)。`memhub stop` 停止受管服务时也会关闭
登录自动启动,再次 `memhub start` 会恢复。
默认保留 Memory。只有独立的 `memhub delete-data` 在 Runtime 停止且二次确认精确路径后
才会删除数据。
当前平台能力和安装边界以源码、CI 与发布说明为准。未完成真机验收的平台不会仅凭跨平台
测试宣称可用。
## Hy Memory
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.6`。SDK 配置、权限边界及 search
SDK 依赖固定为公开 PyPI 开发预览 `hy-memory==2.0.5.dev3`。SDK 配置、权限边界及 search
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
+3 -3
View File
@@ -3,9 +3,9 @@
"dataLicense": "CC0-1.0",
"SPDXID": "SPDXRef-DOCUMENT",
"name": "one-memory",
"documentNamespace": "https://one-memory.local/spdx/2026-10-22T13:18:16Z",
"documentNamespace": "https://one-memory.local/spdx/2026-10-22T08:09:31Z",
"creationInfo": {
"created": "2026-10-22T13:18:16Z",
"created": "2026-10-22T08:09:31Z",
"creators": [
"Tool: one-memory-generate-sbom"
]
@@ -2352,7 +2352,7 @@
{
"SPDXID": "SPDXRef-Package-hy-memory",
"name": "hy-memory",
"versionInfo": "2.0.6",
"versionInfo": "2.0.5.dev3",
"downloadLocation": "NOASSERTION",
"filesAnalyzed": false,
"licenseConcluded": "Apache-2.0",
+2 -2
View File
@@ -2,14 +2,14 @@
MemHub itself is licensed under Apache-2.0 (see `LICENSE`).
It bundles the runtime dependencies listed below.
Generated from the installed dependency tree on 2026-10-22T13:18:16Z
Generated from the installed dependency tree on 2026-10-22T08:09:31Z
by `python3 scripts/generate_sbom.py`. Do not edit by hand.
Per-package license texts are not reproduced here yet; the license
column is filled in as part of release review, and the SBOM records
`NOASSERTION` until then rather than guessing.
## hy-memory 2.0.6
## hy-memory 2.0.5.dev3
License: Apache-2.0
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@one-memory/cli",
"version": "1.0.5",
"version": "1.0.4",
"description": "MemHub bootstrap CLI for installing and managing the local-first shared memory runtime",
"license": "Apache-2.0",
"repository": {
+49 -24
View File
@@ -21,21 +21,18 @@ const TOKEN = "probe-token-not-a-real-credential";
const MEMORY = {
memory_id: "probe-1",
content: "部署脚本要先跑 verify_merge_plan。",
layer: "l2_fact",
status: "active",
is_latest: true,
supersedes: [],
superseded_by: [],
tags: [],
owner: null,
source_raw_ids: ["probe-raw-1"],
user_id: "local",
agent_id: "codex",
session_id: "probe-session",
source: "codex",
memory_at: 1758240000,
gmt_created: 1758240000,
kind: "chat",
status: "confirmed",
content: "部署脚本要先跑 verify_merge_plan。",
scopes: ["all-apps"],
evidence_ids: ["probe-e1"],
origin: "extracted",
tags: [],
revision: 2,
created_at: "2026-09-19T00:00:00Z",
updated_at: "2026-09-19T00:00:00Z",
deleted_at: null,
};
const ROUTES = new Map([
@@ -127,6 +124,26 @@ const ROUTES = new Map([
},
],
],
[
"/api/v1/memory-tasks",
[
{
task_id: "probe-task",
request_id: "probe-request",
user_id: "local",
kind: "extract",
input_ref: "probe",
status: "waiting_for_model",
profile_id: "probe-profile",
profile_revision: 1,
profile_snapshot: {},
attempts: 0,
last_error_code: "model_missing",
created_at: "2026-09-19T00:00:00Z",
updated_at: "2026-09-19T00:00:01Z",
},
],
],
[
"/api/v1/model-profiles",
[
@@ -142,20 +159,28 @@ const ROUTES = new Map([
},
],
],
["/api/v1/home-summary", { memory_count: 1, latest: null, observed_at: "T" }],
[
"/api/v2/home-summary",
"/api/v1/memory-library",
{ total: 1, offset: 0, limit: 100, items: [MEMORY] },
],
[
"/api/v1/memories/probe-1",
{
schema_version: 2,
memory_count: 1,
latest: MEMORY,
observed_at: "2026-09-19T00:00:01Z",
memory: MEMORY,
evidence: [
{
evidence_id: "probe-e1",
available: true,
source_id: "codex",
written_by: "user",
created_at: "2026-09-19T00:00:00Z",
verbatim: "部署前先跑 verify_merge_plan,别靠读 diff 猜冲突。",
},
],
history: [{ memory_id: "probe-1", revision: 2, action: "confirm" }],
},
],
[
"/api/v2/memory-library",
{ memories: [MEMORY], total: 1, offset: 0, limit: 100 },
],
["/api/v2/memories/probe-1", MEMORY],
]);
const seen = [];
@@ -30,7 +30,9 @@ const REQUIRED_COMMAND_SURFACE = [
"sync",
"memories",
"profile",
"graph-width",
"calls",
"notes",
"graph",
"models",
"host",
"doctor",
@@ -48,17 +50,19 @@ const REQUIRED_COMMAND_SURFACE = [
"uninstall",
"delete-data",
"recall",
"setup",
"scan",
"progress",
"jobs",
"scan-roots",
"path",
"adapter",
"write",
"task",
"serve",
];
const REQUIRED_MARKERS = [
"backup_restore_confirmation_mismatch",
"backup_restore_confirmation_required",
"backup_restore_runtime_state_unknown",
"backup_restore_maintenance_identity_invalid",
"runtime_service_activation_unverified",
@@ -75,18 +79,16 @@ const REQUIRED_MARKERS = [
* exact literal the catalog usage string contains.
*/
const REQUIRED_SUBCOMMANDS = [
{ command: "memories", marker: "memhub memories add" },
{ command: "memories", marker: "memhub memories add-status" },
{ command: "calls", marker: "memhub calls clear" },
{ command: "notes", marker: "memhub notes add" },
{ command: "notes", marker: "memhub notes process" },
{ command: "memories", marker: "memhub memories update" },
{ command: "memories", marker: "memhub memories delete" },
{ command: "write", marker: "memhub write submit" },
{ command: "write", marker: "memhub write status" },
{ command: "graph-width", marker: "memhub graph-width --decision" },
{ command: "models", marker: "memhub models add" },
{ command: "models", marker: "memhub models test" },
{ command: "graph", marker: "memhub graph width" },
{ command: "models", marker: "memhub models edit" },
{ command: "models", marker: "memhub models activate" },
{ command: "sources", marker: "memhub sources connect" },
{ command: "jobs", marker: "memhub jobs start" },
{ command: "jobs", marker: "memhub jobs <pause|resume|cancel>" },
];
@@ -98,30 +100,57 @@ const REQUIRED_SUBCOMMANDS = [
* counterexamples on the packaged Runtime") had no machine-checkable probe.
* These run without `--lifecycle`, so they are safe against any package.
*/
const PROFILE_ID = "44444444-4444-4444-8444-444444444444";
const STOPPED_DESTRUCTIVE_PROBES = [
{ name: "memories_add", args: ["memories", "add", "synthetic memory"] },
{
name: "memories_delete_apply",
args: ["memories", "delete", "m-1", "--yes"],
name: "calls_clear_preview",
args: ["calls", "clear", "--event", "11111111-1111-4111-8111-111111111111"],
},
{
name: "calls_clear_apply",
args: [
"calls",
"clear",
"--event",
"11111111-1111-4111-8111-111111111111",
"--confirm",
],
},
{
name: "calls_clear_all_apply",
args: ["calls", "clear", "--all", "--confirm"],
},
{
name: "notes_process_apply",
args: [
"notes",
"process",
"33333333-3333-4333-8333-333333333333",
"--confirm",
],
},
{
name: "graph_width_apply",
args: ["graph-width", "--decision", "keep_existing"],
args: ["graph", "width", "--decision", "keep_existing", "--confirm"],
},
{
name: "graph_width_rebuild_apply",
args: ["graph-width", "--decision", "rebuild_with_new", "--yes"],
},
{ name: "models_activate_apply", args: ["models", "activate", PROFILE_ID] },
{ name: "models_test", args: ["models", "test", PROFILE_ID] },
{
name: "models_remove_apply",
args: ["models", "remove", PROFILE_ID, "--yes"],
name: "models_activate_apply",
args: [
"models",
"activate",
"44444444-4444-4444-8444-444444444444",
"--confirm",
],
},
{
name: "sources_connect_import",
args: ["sources", "connect", "codex", "--import"],
name: "models_edit_apply",
args: [
"models",
"edit",
"44444444-4444-4444-8444-444444444444",
"--config-file",
"/nonexistent/patch.json",
"--confirm",
],
},
];
@@ -119,7 +119,6 @@ describe("A19 batch execution", () => {
readerFor([]),
operations,
planned.notFound,
{ confirmOverrides: false },
);
// Exactly one request for two sources — the batch endpoint, per contract.
expect(post).toHaveBeenCalledTimes(1);
@@ -166,7 +165,6 @@ describe("A19 batch execution", () => {
readerFor([]),
operations,
planned.notFound,
{ confirmOverrides: false },
);
expect(document.state).toBe("partial");
const byId = Object.fromEntries(
@@ -208,7 +206,6 @@ describe("A19 batch execution", () => {
readerFor([]),
operations,
[],
{ confirmOverrides: false },
);
expect(document.sources.every((row) => row.status === "failed")).toBe(true);
expect(
@@ -252,7 +249,6 @@ describe("A19 batch execution", () => {
readerFor([]),
operations,
[],
{ confirmOverrides: false },
);
const busy = document.sources.find((row) => row.source_id === "cursor");
expect(document.state).toBe("partial");
@@ -292,7 +288,6 @@ describe("A19 batch execution", () => {
readerFor([]),
operations,
planned.notFound,
{ confirmOverrides: false },
);
const notFoundRow = document.sources.find(
(row) => row.source_id === "cursor",
@@ -301,82 +296,3 @@ describe("A19 batch execution", () => {
expect(document.not_found_source_ids).toContain("cursor");
});
});
describe("environment-override sources need --confirm-overrides", () => {
it("leaves them out and names them unless confirmed", async () => {
const planned = await planBatchImport(
readerFor([
found("codex"),
found("dsh", {
candidate_kind: "environment_override",
requires_override_confirmation: true,
}),
]),
);
if (planned.state !== "preview") throw new Error("expected preview");
const post = vi.fn(
async (_path: string, body: Record<string, unknown>) => ({
state: "committed" as const,
data: {
request_id: body.request_id,
accepted_source_ids: body.source_ids,
failed_sources: {},
jobs: [],
},
}),
);
const operations = { post } as unknown as CoreOperations;
const skipped = await executeBatchImport(
planned.candidates,
readerFor([]),
operations,
planned.notFound,
{ confirmOverrides: false },
);
expect(post.mock.calls[0]![1]).toMatchObject({
source_ids: ["codex"],
confirm_environment_overrides: false,
});
expect(skipped.override_required_source_ids).toEqual(["dsh"]);
expect(renderBatchScanText(skipped)).toContain("--confirm-overrides");
const confirmed = await executeBatchImport(
planned.candidates,
readerFor([]),
operations,
planned.notFound,
{ confirmOverrides: true },
);
expect(post.mock.calls[1]![1]).toMatchObject({
source_ids: ["codex", "dsh"],
confirm_environment_overrides: true,
});
expect(confirmed.override_required_source_ids).toBeUndefined();
});
it("refuses without a request when every detected source needs confirmation", async () => {
const planned = await planBatchImport(
readerFor([
found("dsh", {
candidate_kind: "environment_override",
requires_override_confirmation: true,
}),
]),
);
if (planned.state !== "preview") throw new Error("expected preview");
const post = vi.fn();
const document = await executeBatchImport(
planned.candidates,
readerFor([]),
{ post } as unknown as CoreOperations,
planned.notFound,
{ confirmOverrides: false },
);
expect(post).not.toHaveBeenCalled();
expect(document).toMatchObject({
state: "rejected",
code: "environment_override_confirmation_required",
});
expect(batchScanExitCode(document)).toBe(1);
});
});
+9 -47
View File
@@ -1,5 +1,4 @@
/** A19: one-shot connect/import of every *detected* source
* (`sources connect --all-detected --import`).
/** A19: one-shot scan/import of every *detected* source.
*
* The engine already takes many sources in a single `POST /api/v1/sources/connect`
* (`source_ids` is a 1..32 tuple) and answers per source. The CLI used to reject
@@ -38,16 +37,13 @@ export interface BatchSourceRow {
export interface BatchScanDocument {
schema_version: 1;
product: string;
command: "sources connect";
command: "scan" | "init";
state: "committed" | "partial" | "rejected" | "unknown" | "no_sources";
request_id: string | null;
accepted_source_ids: string[];
sources: BatchSourceRow[];
/** `location_state != found` rows are listed so the user can see what was skipped. */
not_found_source_ids: string[];
/** Detected sources whose location comes from an environment override and
* that were left out because `--confirm-overrides` was not given. */
override_required_source_ids?: string[];
code?: string;
}
@@ -122,39 +118,12 @@ export async function planBatchImport(reader: CoreReader): Promise<
/** Run the batch connect once and fold the per-source receipt into rows. */
export async function executeBatchImport(
detected: readonly SourceConnectionCandidate[],
candidates: readonly SourceConnectionCandidate[],
reader: CoreReader,
operations: CoreOperations,
notFound: readonly string[],
options: { confirmOverrides: boolean },
): Promise<BatchScanDocument> {
const requestId = randomUUID();
// An environment-override location is a choice the user has to confirm:
// without `--confirm-overrides` those sources are left out and named.
const skipped = options.confirmOverrides
? []
: detected
.filter((candidate) => candidate.requires_override_confirmation)
.map((candidate) => candidate.source_id);
const overrideFields =
skipped.length > 0 ? { override_required_source_ids: skipped } : {};
const candidates = detected.filter(
(candidate) => !skipped.includes(candidate.source_id),
);
if (candidates.length === 0) {
return {
schema_version: 1,
product: PRODUCT,
command: "sources connect",
state: "rejected",
request_id: null,
accepted_source_ids: [],
sources: [],
not_found_source_ids: [...notFound],
...overrideFields,
code: "environment_override_confirmation_required",
};
}
// Same shared builder as the single-source import and the Web console, so a
// batch connect cannot drift from either.
const request = buildSourceImportRequest({
@@ -172,9 +141,9 @@ export async function executeBatchImport(
: [[candidate.source_id, candidate.candidate_revision]],
),
),
confirmEnvironmentOverrides:
options.confirmOverrides &&
candidates.some((candidate) => candidate.requires_override_confirmation),
confirmEnvironmentOverrides: candidates.some(
(candidate) => candidate.requires_override_confirmation,
),
fullRescan: false,
onboarding: true,
enableSync: true,
@@ -185,7 +154,7 @@ export async function executeBatchImport(
return {
schema_version: 1,
product: PRODUCT,
command: "sources connect",
command: "scan",
state: result.state,
request_id: requestId,
accepted_source_ids: [],
@@ -199,7 +168,6 @@ export async function executeBatchImport(
failure_reason: result.code,
})),
not_found_source_ids: [...notFound],
...overrideFields,
code: result.code,
};
}
@@ -286,7 +254,7 @@ export async function executeBatchImport(
return {
schema_version: 1,
product: PRODUCT,
command: "sources connect",
command: "scan",
state:
failedCount === 0
? "committed"
@@ -302,7 +270,6 @@ export async function executeBatchImport(
accepted_source_ids: accepted,
sources: rows,
not_found_source_ids: [...notFound],
...overrideFields,
};
}
@@ -334,10 +301,5 @@ export function renderBatchScanText(document: BatchScanDocument): string {
: document.state === "partial"
? `部分完成:${document.accepted_source_ids.length} 个成功,其余见上方`
: `未完成:${document.code ?? "unknown"}`;
const overrides = document.override_required_source_ids ?? [];
const hint =
overrides.length > 0
? `\n未接入(目录来自环境变量覆盖,需确认):${overrides.join("、")}\n确认目录无误后加 --confirm-overrides 重新运行。`
: "";
return `${summary}\n${lines.join("\n")}${hint}\n`;
return `${summary}\n${lines.join("\n")}\n`;
}
@@ -1,371 +0,0 @@
// Contract fixes from the CLI review: exit codes, where JSON goes, the global
// `--data-dir`, and the write idempotency key.
import { mkdtemp, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { run } from "./cli.js";
import { COMMANDS } from "./command-catalog.js";
const transport = vi.hoisted(() => ({
get: vi.fn(),
post: vi.fn(),
put: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
request: vi.fn(),
}));
vi.mock("axios", () => ({
default: {
create: vi.fn(() => transport),
get: transport.get,
post: transport.post,
request: transport.request,
isAxiosError: () => false,
},
}));
const runtime = vi.hoisted(() => ({
inspectRuntimeStatus: vi.fn(),
discoverRuntimeAttachment: vi.fn(),
inspectBackgroundHealth: vi.fn(),
}));
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
...runtime,
}));
const ATTACHED = {
instanceId: "instance-1",
apiToken: "synthetic-runtime-token",
host: "127.0.0.1",
port: 8765,
servicePid: 4242,
health: {
status: "ok",
service: "one-memory-core",
version: "1.2.3",
instance_id: "instance-1",
},
};
beforeEach(() => {
vi.clearAllMocks();
for (const method of Object.values(transport)) method.mockReset();
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
version: "1.2.3",
});
runtime.discoverRuntimeAttachment.mockResolvedValue(ATTACHED);
runtime.inspectBackgroundHealth.mockResolvedValue(null);
});
describe("--data-dir is a global option", () => {
it("points a Runtime-attached command at that data directory", async () => {
transport.get.mockResolvedValue({
status: 200,
data: { request_id: "r", profile: null, elapsed_ms: 1 },
});
const result = await capturedRun(
["profile", "--data-dir", "/synthetic/other", "--json"],
{ ONE_MEMORY_DATA_DIR: "/synthetic/default" },
);
expect(result.code).toBe(0);
expect(runtime.inspectRuntimeStatus).toHaveBeenCalledWith(
expect.objectContaining({ dataDir: "/synthetic/other" }),
);
});
it("is accepted by every command the catalog used to advertise it on", async () => {
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "stopped",
code: "runtime_not_running",
});
runtime.discoverRuntimeAttachment.mockResolvedValue(null);
for (const argv of [
["profile"],
["scan-roots", "list"],
["adapter", "list"],
["scan", "--source", "codex"],
["write", "status", "r-1"],
["backup", "create", "--target", "/synthetic/data/backups/a"],
["memories", "list"],
]) {
const result = await capturedRun([
...argv,
"--data-dir",
"/synthetic/other",
"--json",
]);
expect(result.code, argv.join(" ")).toBe(3);
}
});
it("refuses a relative or repeated --data-dir", async () => {
for (const argv of [
["profile", "--data-dir", "relative"],
["profile", "--data-dir", "/a", "--data-dir", "/b"],
["profile", "--data-dir"],
]) {
expect((await capturedRun(argv)).code).toBe(2);
}
expect(runtime.inspectRuntimeStatus).not.toHaveBeenCalled();
});
});
describe("Core failures are exit 1, not usage errors", () => {
it("adapter probe reports a Core refusal with its code on stdout", async () => {
transport.request.mockResolvedValue({
status: 409,
data: { detail: "source_not_authorized" },
});
const result = await capturedRun([
"adapter",
"probe",
"--source",
"codex",
"--json",
]);
expect(result.code).toBe(1);
expect(result.stderr).toBe("");
expect(JSON.parse(result.stdout)).toMatchObject({
command: "adapter probe",
state: "failed",
code: "source_not_authorized",
exit_code: 1,
});
});
it("adapter probe prints readable text without --json", async () => {
transport.request.mockResolvedValue({
status: 200,
data: { source: "codex", available: true },
});
const result = await capturedRun(["adapter", "probe", "--source", "codex"]);
expect(result.code).toBe(0);
expect(result.stdout).toContain("来源 codex 的探测结果");
expect(result.stdout).toContain("available: true");
});
it("scan-roots add posts directly and reports a refusal as exit 1", async () => {
transport.request.mockResolvedValue({
status: 409,
data: { detail: "scan_root_conflict" },
});
const result = await capturedRun([
"scan-roots",
"add",
"--root",
"/synthetic/root",
"--source",
"codex",
"--json",
]);
expect(result.code).toBe(1);
expect(transport.request).toHaveBeenCalledWith(
expect.objectContaining({
method: "post",
url: "http://127.0.0.1:8765/api/v1/scan-roots",
data: { root: "/synthetic/root", source: "codex", recursive: true },
}),
);
expect(JSON.parse(result.stdout).code).toBe("scan_root_conflict");
});
it("the old `path add` spelling still works", async () => {
transport.request.mockResolvedValue({ status: 200, data: { ok: true } });
const result = await capturedRun([
"path",
"remove",
"--root",
"/synthetic/root",
"--yes",
]);
expect(result.code).toBe(0);
expect(transport.request.mock.calls[0][0]).toMatchObject({
method: "delete",
data: { root: "/synthetic/root" },
});
});
it("scan --source runs directly", async () => {
transport.post.mockResolvedValue({
status: 200,
data: { job_id: "j1", source_id: "codex", status: "pending" },
});
const result = await capturedRun(["scan", "--source", "codex", "--json"]);
expect(result.code).toBe(0);
expect(JSON.parse(result.stdout)).toMatchObject({
command: "scan",
state: "committed",
scan: { job_id: "j1" },
});
// A scan without a source is a usage error, not a preview.
expect((await capturedRun(["scan"])).code).toBe(2);
});
});
describe("write submit keeps its idempotency key", () => {
async function messagesFile(): Promise<string> {
const directory = await mkdtemp(path.join(tmpdir(), "memhub-write-"));
const file = path.join(directory, "messages.json");
await writeFile(file, JSON.stringify([{ role: "user", content: "hi" }]));
return file;
}
it("generates a request id, sends it and prints it with a retry hint", async () => {
transport.post.mockImplementation(
async (_url: string, body: { request_id: string }) => ({
status: 200,
data: {
request_id: body.request_id,
state: "accepted",
created_at: "T",
updated_at: "T",
},
}),
);
const result = await capturedRun([
"write",
"submit",
"--session",
"s1",
"--messages-file",
await messagesFile(),
]);
expect(result.code).toBe(0);
const sent = transport.post.mock.calls[0][1].request_id as string;
expect(sent).toMatch(/^[0-9a-f-]{36}$/);
expect(result.stdout).toContain(`request_id: ${sent}`);
expect(result.stdout).toContain(`--request-id ${sent}`);
});
it("reports a 409 conflict as JSON on stdout with the request id", async () => {
transport.post.mockResolvedValue({
status: 409,
data: { detail: "memory_write_conflict" },
});
const result = await capturedRun([
"write",
"submit",
"--session",
"s1",
"--messages-file",
await messagesFile(),
"--request-id",
"req-1",
"--json",
]);
expect(result.code).toBe(1);
expect(result.stderr).toBe("");
expect(JSON.parse(result.stdout)).toMatchObject({
command: "write submit",
state: "rejected",
request_id: "req-1",
code: "memory_write_conflict",
});
});
it("write status of an unknown id is exit 1 with state not_found", async () => {
transport.get.mockResolvedValue({ status: 404, data: {} });
const error = Object.assign(new Error("404"), {
response: { status: 404 },
});
transport.get.mockRejectedValue(error);
const result = await capturedRun(["task", "--id", "nope", "--json"]);
// `axios.isAxiosError` is mocked false, so the read error surfaces as a
// failed read; either way it is exit 1, never a silent exit 0.
expect(result.code).toBe(1);
});
});
describe("doctor and ui use exit 3 for a stopped Runtime", () => {
it("doctor exits 3 when the Runtime is stopped", async () => {
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "stopped",
code: "runtime_not_running",
});
const result = await capturedRun(["doctor", "--json"]);
expect(result.code).toBe(3);
expect(JSON.parse(result.stdout).schema_version).toBe(1);
});
});
describe("catalog exit codes", () => {
it("lists exit 2 for every command, since every parser can reject", () => {
for (const command of COMMANDS) {
expect(command.exit_codes, command.name).toContain(2);
}
});
it("lists exit 3 for every command that attaches to the Runtime", () => {
for (const name of [
"status",
"doctor",
"init",
"sources",
"sync",
"progress",
"jobs",
"models",
"memories",
"profile",
"graph-width",
"host",
"recall",
"scan-roots",
"adapter",
"scan",
"write",
"backup",
"ui",
"onboarding",
]) {
const command = COMMANDS.find((entry) => entry.name === name);
expect(command?.exit_codes, name).toContain(3);
}
});
it("does not claim exit 3 or 4 for lifecycle commands that never return them", () => {
for (const name of ["start", "stop", "restart"]) {
const command = COMMANDS.find((entry) => entry.name === name);
expect(command?.exit_codes).toEqual([0, 1, 2]);
}
});
it("says stop turns login auto-start off for a managed service", () => {
const stop = COMMANDS.find((entry) => entry.name === "stop");
expect(stop?.summary).toContain("登录时也不会自动启动");
});
});
async function capturedRun(
arguments_: string[],
environment: NodeJS.ProcessEnv = { ONE_MEMORY_DATA_DIR: "/synthetic/data" },
) {
let stdout = "";
let stderr = "";
const stdoutSpy = vi
.spyOn(process.stdout, "write")
.mockImplementation((chunk) => {
stdout += String(chunk);
return true;
});
const stderrSpy = vi
.spyOn(process.stderr, "write")
.mockImplementation((chunk) => {
stderr += String(chunk);
return true;
});
try {
return { code: await run(arguments_, environment), stdout, stderr };
} finally {
stdoutSpy.mockRestore();
stderrSpy.mockRestore();
}
}
+1 -42
View File
@@ -1,7 +1,6 @@
import { mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises";
import { mkdtemp, readdir, rm, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { inflateRawSync } from "node:zlib";
import { DiagnosticLogger } from "@one-memory/runtime-control";
import { afterEach, expect, it, vi } from "vitest";
@@ -30,27 +29,6 @@ async function fixture() {
return dataDir;
}
function readZipEntry(archive: Buffer, expected: string): string {
let offset = 0;
while (archive.readUInt32LE(offset) === 0x04034b50) {
const method = archive.readUInt16LE(offset + 8);
const compressedSize = archive.readUInt32LE(offset + 18);
const nameLength = archive.readUInt16LE(offset + 26);
const extraLength = archive.readUInt16LE(offset + 28);
const nameStart = offset + 30;
const dataStart = nameStart + nameLength + extraLength;
const name = archive.subarray(nameStart, nameStart + nameLength).toString();
const compressed = archive.subarray(dataStart, dataStart + compressedSize);
if (name === expected) {
if (method === 0) return compressed.toString();
if (method === 8) return inflateRawSync(compressed).toString();
throw new Error("unsupported zip compression");
}
offset = dataStart + compressedSize;
}
throw new Error(`missing ${expected}`);
}
it("prints matching diagnostic events as NDJSON without starting Runtime", async () => {
const dataDir = await fixture();
let stdout = "";
@@ -96,25 +74,6 @@ it("exports one support zip to the requested downloads directory", async () => {
});
expect(receipt.path).toMatch(/MemHub-日志-3800-\d{8}-\d{4}\.zip$/);
expect(await readdir(downloads)).toEqual([path.basename(receipt.path)]);
const archive = await readFile(receipt.path);
expect(JSON.parse(readZipEntry(archive, "summary.json")).commit).toBeNull();
});
it("fails closed for an explicitly supplied invalid commit", async () => {
const dataDir = await fixture();
const downloads = await mkdtemp(path.join(os.tmpdir(), "om-cli-downloads-"));
roots.push(downloads);
vi.spyOn(process.stdout, "write").mockImplementation(() => true);
await expect(
run(["logs", "export", "--json"], {
ONE_MEMORY_DATA_DIR: dataDir,
ONE_MEMORY_EXPORT_DIR: downloads,
ONE_MEMORY_BUILD_NUMBER: "3800",
ONE_MEMORY_COMMIT: "unknown",
}),
).rejects.toThrow("diagnostic_commit_invalid");
expect(await readdir(downloads)).toEqual([]);
});
it("never emits a secret from a malformed manually written event", async () => {
+57 -726
View File
@@ -1,18 +1,9 @@
import {
mkdir,
mkdtemp,
readdir,
readFile,
realpath,
rm,
} from "node:fs/promises";
import { mkdtemp, readdir, readFile, realpath, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import {
claimRuntimeServiceForApp,
clearSdkMigrationBlock,
readRuntimeServiceOwner,
recordSdkMigrationBlock,
resolveRuntimeCliConfig,
} from "@one-memory/runtime-control";
import { describe, expect, it, vi } from "vitest";
@@ -66,11 +57,6 @@ const runtimeMocks = vi.hoisted(() => ({
})),
uninstallUserService: vi.fn(async (..._args: unknown[]) => "removed"),
restartUserService: vi.fn(async () => undefined),
runSdkMigration: vi.fn(
async (): Promise<{ state: string; code?: string }> => ({
state: "not_required",
}),
),
rollbackLocalInstall: vi.fn(async () => undefined),
planCliLink: vi.fn(async ({ home, installRoot }) => ({
schema_version: 1,
@@ -140,7 +126,6 @@ vi.mock("@one-memory/runtime-control", async (importOriginal) => {
installUserService: runtimeMocks.installUserService,
uninstallUserService: runtimeMocks.uninstallUserService,
restartUserService: runtimeMocks.restartUserService,
runSdkMigration: runtimeMocks.runSdkMigration,
rollbackLocalInstall: runtimeMocks.rollbackLocalInstall,
planCliLink: runtimeMocks.planCliLink,
installCliLink: runtimeMocks.installCliLink,
@@ -799,7 +784,7 @@ describe("public One Memory CLI", () => {
}
});
it("upgrade restart failure keeps the new version and never restarts the old one", async () => {
it("upgrade failure restarts the restored artifact with the same data identity", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce({
schema_version: 1,
@@ -841,27 +826,25 @@ describe("public One Memory CLI", () => {
command: "upgrade",
code: "restart failed",
});
expect(JSON.parse(streams.stderr())).toMatchObject({
rolled_back: false,
});
// Product rule: the previous Runtime may predate the SDK migration gate,
// so it is never restored after the migration was handed the home.
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(aliasMocks.rollbackManagedCliLink).not.toHaveBeenCalled();
expect(runtimeMocks.restartUserService).toHaveBeenCalledOnce();
expect(
(runtimeMocks.restartUserService.mock.calls as unknown[][])[0]?.[0],
).toMatchObject({
dataDir: "/tmp/custom data",
installRoot: "/tmp/install/current",
});
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
// The CLI manages the objective aliases the package does not own
// (`memhub` + `memhub-cli`); both are rolled back.
expect(aliasMocks.rollbackManagedCliLink).toHaveBeenCalledTimes(2);
expect(runtimeMocks.restartUserService).toHaveBeenCalledTimes(2);
for (const call of runtimeMocks.restartUserService.mock
.calls as unknown[][]) {
expect(call[0]).toMatchObject({
dataDir: "/tmp/custom data",
installRoot: "/tmp/install/current",
});
}
});
// O-22 (2026-09-29 C8 H17): `upgrade --yes` on a manifest-valid but
// crash-looping version reported exit 0 and left `current` on the bad
// version. A restart that merely returns is not proof the new Runtime came
// up, so the CLI now verifies health and rolls back if it never does.
it("fails without rolling back when the upgraded Runtime never becomes healthy", async () => {
it("rolls back and fails when the upgraded Runtime never becomes healthy", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce({
schema_version: 1,
@@ -901,219 +884,9 @@ describe("public One Memory CLI", () => {
command: "upgrade",
code: "upgrade_health_check_failed",
});
// Kept on the new version; the old Runtime is not restarted.
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(runtimeMocks.restartUserService).toHaveBeenCalledOnce();
});
const upgradeArguments = [
"upgrade",
"--source",
"/tmp/dist",
"--home",
"/tmp/home",
"--install-root",
"/tmp/install",
"--data-dir",
"/tmp/custom data",
"--yes",
];
const presentCliLink = {
schema_version: 1 as const,
platform: "linux" as const,
home: "/tmp/home",
install_root: "/tmp/install",
bin_directory: "/tmp/bin",
status: "present" as const,
blocked_reason: null,
link_path: "/tmp/bin/one-memory",
target_path: "/tmp/install/current/bin/one-memory",
};
it("says the data was restored when a failed migration put it back", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
runtimeMocks.runSdkMigration.mockResolvedValueOnce({
state: "deferred",
code: "sdk_migration_memory_unreadable",
dataRestored: true,
} as never);
const streams = captureStreams();
let code: number;
try {
code = await run(
upgradeArguments,
{},
{ waitForRuntimeReady: async () => true },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
const failure = JSON.parse(streams.stderr());
expect(failure).toMatchObject({
code: "sdk_migration_memory_unreadable",
data_restored: true,
log_file: "/tmp/custom data/logs/sdk-migration.log",
});
expect(failure.message).toContain("记忆数据已恢复到升级前的状态。");
expect(failure.message).toContain(
"/tmp/custom data/logs/sdk-migration.log",
);
});
it("migrates the home with the new Core before restarting the Runtime", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
const order: string[] = [];
runtimeMocks.runSdkMigration.mockImplementationOnce(async () => {
order.push("migrate");
return { state: "cutover" };
});
runtimeMocks.restartUserService.mockImplementationOnce(async () => {
order.push("restart");
});
const output = captureStdout();
let code: number;
try {
code = await run(
upgradeArguments,
{},
{ waitForRuntimeReady: async () => true },
);
} finally {
output.restore();
}
expect(code!).toBe(0);
expect(order).toEqual(["migrate", "restart"]);
expect(runtimeMocks.runSdkMigration).toHaveBeenCalledWith(
expect.objectContaining({ dataDir: "/tmp/custom data" }),
);
});
it("stays on the new version, not serving, when the migration does not finish", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
runtimeMocks.runSdkMigration.mockResolvedValueOnce({
state: "deferred",
code: "sdk_migration_disk_space_insufficient",
});
const streams = captureStreams();
let code: number;
try {
code = await run(
upgradeArguments,
{},
{ waitForRuntimeReady: async () => true },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
const failure = JSON.parse(streams.stderr());
expect(failure).toMatchObject({
command: "upgrade",
code: "sdk_migration_disk_space_insufficient",
// The one file support needs, and where to send it.
log_file: "/tmp/custom data/logs/sdk-migration.log",
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
});
expect(failure.message).toContain(
"/tmp/custom data/logs/sdk-migration.log",
);
expect(failure.message).toContain("GitHub");
expect(failure.message).not.toContain("回滚到上一版本");
expect(runtimeMocks.runSdkMigration).toHaveBeenCalledWith(
expect.objectContaining({ caller: "cli-upgrade" }),
);
// No rollback: the old Runtime has no migration gate. The new service is
// (re)started so its gate, not the old Runtime, holds the data dir.
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(aliasMocks.rollbackManagedCliLink).not.toHaveBeenCalled();
expect(runtimeMocks.restartUserService).toHaveBeenCalledOnce();
expect(
(runtimeMocks.restartUserService.mock.calls as unknown[][])[0]?.[0],
).toMatchObject({
installRoot: "/tmp/install/current",
});
});
it("stays on the new version when the migration itself throws", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
runtimeMocks.runSdkMigration.mockRejectedValueOnce(
new Error("sdk_migration_fence_failed"),
);
const streams = captureStreams();
let code: number;
try {
code = await run(
upgradeArguments,
{},
{ waitForRuntimeReady: async () => true },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
expect(JSON.parse(streams.stderr())).toMatchObject({
command: "upgrade",
code: "sdk_migration_fence_failed",
log_file: "/tmp/custom data/logs/sdk-migration.log",
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
});
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
});
it("still rolls back a failure before the migration touched the home", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
aliasMocks.installManagedCliLink.mockRejectedValueOnce(
new Error("cli_link_failed"),
);
const streams = captureStreams();
let code: number;
try {
code = await run(
upgradeArguments,
{},
{ waitForRuntimeReady: async () => true },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
expect(JSON.parse(streams.stderr())).toMatchObject({
command: "upgrade",
code: "cli_link_failed",
});
expect(runtimeMocks.runSdkMigration).not.toHaveBeenCalled();
// Rolled back to the previous version and restarted it.
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
// The previous Runtime was never stopped; nothing is (re)started.
expect(runtimeMocks.restartUserService).not.toHaveBeenCalled();
});
it("keeps the new version when a migrated home is slow to come up", async () => {
vi.clearAllMocks();
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
runtimeMocks.runSdkMigration.mockResolvedValueOnce({ state: "cutover" });
const streams = captureStreams();
let code: number;
try {
code = await run(
upgradeArguments,
{},
{ waitForRuntimeReady: async () => false },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
expect(JSON.parse(streams.stderr())).toMatchObject({
command: "upgrade",
code: "runtime_not_ready_after_sdk_migration",
});
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(runtimeMocks.restartUserService).toHaveBeenCalledTimes(2);
});
it("reports success once the upgraded Runtime is healthy", async () => {
@@ -1611,7 +1384,7 @@ describe("public One Memory CLI", () => {
expect(JSON.parse(output.value()).apply).toBe(true);
});
it("keeps the distribution when service installation fails after the migration ran", async () => {
it("rolls back the distribution when service installation fails", async () => {
runtimeMocks.applyLocalInstall.mockResolvedValueOnce({
schema_version: 1,
source: "/tmp/distribution",
@@ -1652,14 +1425,13 @@ describe("public One Memory CLI", () => {
command: "install",
status: "error",
code: "service failed",
rolled_back: false,
});
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(runtimeMocks.rollbackCliLink).not.toHaveBeenCalled();
expect(aliasMocks.rollbackManagedCliLink).not.toHaveBeenCalled();
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
expect(runtimeMocks.rollbackCliLink).toHaveBeenCalledOnce();
expect(aliasMocks.rollbackManagedCliLink).toHaveBeenCalledTimes(2);
});
it("upgrades only an existing managed CLI and keeps it after a failed restart", async () => {
it("upgrades only an existing managed CLI and rolls back a failed restart", async () => {
runtimeMocks.planCliLink.mockResolvedValueOnce({
schema_version: 1,
platform: "linux",
@@ -1699,7 +1471,7 @@ describe("public One Memory CLI", () => {
status: "error",
code: "restart failed",
});
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
expect(runtimeMocks.installCliLink).not.toHaveBeenCalled();
});
@@ -2011,11 +1783,16 @@ describe("stopped Runtime is exit 3, not a usage error", () => {
["path", "list", "--json"],
["adapter", "list", "--json"],
["task", "--id", "nope", "--json"],
["write", "status", "nope", "--json"],
["backup", "create", "--target", "/tmp/x", "--json"],
["scan-roots", "add", "--root", "/tmp/x", "--json"],
["scan", "--source", "codex", "--json"],
["onboarding", "reset", "--json"],
[
"write",
"submit",
"--session",
"s",
"--messages-file",
"/tmp/messages.json",
"--json",
],
["backup", "create", "--target", "/tmp/x", "--yes", "--json"],
])("%s ... --json exits 3 with runtime_not_running", async (...argv) => {
const streams = captureStreams();
let code: number;
@@ -2025,9 +1802,7 @@ describe("stopped Runtime is exit 3, not a usage error", () => {
streams.restore();
}
expect(code!).toBe(3);
// A result, not an error: on stdout like every other --json document.
expect(streams.stderr()).toBe("");
const document = JSON.parse(streams.stdout());
const document = JSON.parse(streams.stderr());
expect(document).toMatchObject({
code: "runtime_not_running",
exit_code: 3,
@@ -2036,48 +1811,6 @@ describe("stopped Runtime is exit 3, not a usage error", () => {
expect(document.status).not.toBe("usage_error");
});
it("write submit validates its messages file before attaching, then exits 3", async () => {
const directory = await mkdtemp(path.join(tmpdir(), "memhub-write-"));
const file = path.join(directory, "messages.json");
await import("node:fs/promises").then(({ writeFile }) =>
writeFile(file, JSON.stringify([{ role: "user", content: "hi" }])),
);
const streams = captureStreams();
let missing: number;
let code: number;
try {
missing = await run(
[
"write",
"submit",
"--session",
"s",
"--messages-file",
path.join(directory, "absent.json"),
"--json",
],
{},
);
code = await run(
[
"write",
"submit",
"--session",
"s",
"--messages-file",
file,
"--json",
],
{},
);
} finally {
streams.restore();
await rm(directory, { recursive: true, force: true });
}
expect(missing!).toBe(2);
expect(code!).toBe(3);
});
it.each(["pause", "resume"] as const)(
"parses sync %s as the direct sync-set command",
async (action) => {
@@ -2162,189 +1895,6 @@ describe("lifecycle commands accept --json", () => {
});
});
// A Runtime service that refuses to serve an unmigrated home must be reported
// as that, with the migration log and where to send it, not as a generic
// "not ready within budget".
describe("lifecycle commands report a migration block", () => {
async function blockedHome() {
const dataDir = await mkdtemp(path.join(tmpdir(), "memhub-blocked-"));
await recordSdkMigrationBlock(
dataDir,
"sdk_migration_disk_space_insufficient",
"runtime-service",
);
return dataDir;
}
// darwin: kickstart "restarts" but never becomes ready. Elsewhere: no
// service, so the direct spawn fails. Both must surface the block.
const blockedControls = () => ({
restartManagedRuntime: vi.fn(async () => "restarted" as const),
waitForRuntimeReady: vi.fn(async () => false),
stopRuntime: vi.fn(async () => "stopped" as const),
ensureRuntime: vi.fn(async () => {
throw new Error("Timed out waiting for MemHub Runtime");
}),
});
it("restart --json prints the block code, log file and issues URL", async () => {
const dataDir = await blockedHome();
const output = captureStdout();
let code: number;
try {
code = await run(
["restart", "--json"],
{ ONE_MEMORY_DATA_DIR: dataDir },
blockedControls(),
);
} finally {
output.restore();
await rm(dataDir, { recursive: true, force: true });
}
expect(code!).toBe(1);
const lines = output.value().trim().split("\n");
const document = JSON.parse(lines[lines.length - 1]!);
expect(document).toMatchObject({
command: "restart",
state: "blocked",
code: "sdk_migration_disk_space_insufficient",
log_file: path.join(dataDir, "logs", "sdk-migration.log"),
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
});
expect(output.value()).not.toContain("runtime_not_ready_within_budget");
});
it("restart on a frozen home retries the migration in the foreground first", async () => {
const dataDir = await mkdtemp(path.join(tmpdir(), "memhub-frozen-"));
await recordSdkMigrationBlock(
dataDir,
"sdk_migration_memory_unreadable",
"desktop",
{ frozen: { service_was_enabled: true }, data_restored: true },
);
const controls = {
...blockedControls(),
migrateSdkData: vi.fn(async () => ({
state: "deferred" as const,
code: "sdk_migration_memory_unreadable",
dataRestored: true,
})),
};
const output = captureStdout();
let code: number;
try {
code = await run(
["restart", "--json"],
{ ONE_MEMORY_DATA_DIR: dataDir },
controls,
);
} finally {
output.restore();
await rm(dataDir, { recursive: true, force: true });
}
expect(code!).toBe(1);
expect(controls.migrateSdkData).toHaveBeenCalledWith(
expect.objectContaining({ dataDir, caller: "cli-restart" }),
);
// Still frozen: the service is not started to retry on its own.
expect(controls.restartManagedRuntime).not.toHaveBeenCalled();
expect(controls.ensureRuntime).not.toHaveBeenCalled();
expect(JSON.parse(output.value())).toMatchObject({
command: "restart",
state: "blocked",
code: "sdk_migration_memory_unreadable",
data_restored: true,
});
});
it("restart on a frozen home restarts the Runtime once the retry succeeds", async () => {
const dataDir = await mkdtemp(path.join(tmpdir(), "memhub-frozen-"));
await recordSdkMigrationBlock(
dataDir,
"sdk_migration_memory_unreadable",
"desktop",
{ frozen: { service_was_enabled: false } },
);
const order: string[] = [];
const migrateSdkData = vi.fn(async () => {
order.push("migrate");
await clearSdkMigrationBlock(dataDir);
return { state: "cutover" as const };
});
const output = captureStreams();
let code: number;
try {
code = await run(
["restart", "--json"],
{ ONE_MEMORY_DATA_DIR: dataDir },
{
migrateSdkData,
waitForRuntimeReady: vi.fn(async () => true),
restartManagedRuntime: vi.fn(async () => {
order.push("restart");
return "restarted" as const;
}),
stopRuntime: vi.fn(async () => "stopped" as const),
ensureRuntime: vi.fn(async () => {
order.push("restart");
return { pid: 1 } as never;
}),
},
);
} finally {
output.restore();
await rm(dataDir, { recursive: true, force: true });
}
expect(code!).toBe(0);
expect(migrateSdkData).toHaveBeenCalledWith(
expect.objectContaining({ caller: "cli-restart" }),
);
expect(order).toEqual(["migrate", "restart"]);
});
it("restart in text mode names the log file and GitHub issues", async () => {
const dataDir = await blockedHome();
const output = captureStdout();
try {
expect(
await run(
["restart"],
{ ONE_MEMORY_DATA_DIR: dataDir },
blockedControls(),
),
).toBe(1);
} finally {
output.restore();
await rm(dataDir, { recursive: true, force: true });
}
expect(output.value()).toContain("sdk_migration_disk_space_insufficient");
expect(output.value()).toContain(
path.join(dataDir, "logs", "sdk-migration.log"),
);
expect(output.value()).toContain(
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
);
});
it("ui --json reports the block instead of runtime_not_running", async () => {
const dataDir = await blockedHome();
const output = captureStdout();
let code: number;
try {
code = await run(["ui", "--json"], { ONE_MEMORY_DATA_DIR: dataDir });
} finally {
output.restore();
await rm(dataDir, { recursive: true, force: true });
}
expect(code!).not.toBe(0);
expect(JSON.parse(output.value())).toMatchObject({
command: "ui",
state: "blocked",
code: "sdk_migration_disk_space_insufficient",
log_file: path.join(dataDir, "logs", "sdk-migration.log"),
});
});
});
// The objective names `memhub host install <host>` (host as a positional). Both
// that and the `--host <id>` flag form must reach the same command; naming both
// at once is ambiguous and refused rather than guessed.
@@ -2454,7 +2004,7 @@ describe("install gates on Runtime readiness", () => {
});
}
it("fails without rolling back when the installed Runtime never becomes ready", async () => {
it("fails and rolls back when the installed Runtime never becomes ready", async () => {
vi.clearAllMocks();
stubInstallPlan();
const streams = captureStreams();
@@ -2483,10 +2033,10 @@ describe("install gates on Runtime readiness", () => {
}
expect(code!).toBe(1);
expect(streams.stderr()).toContain("install_runtime_not_ready");
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
expect(
lifecycleMocks.rollbackPreparedServiceInstall,
).not.toHaveBeenCalled();
).toHaveBeenCalledOnce();
});
it("takes CLI ownership after a successful install over an App-owned Runtime", async () => {
@@ -2531,7 +2081,7 @@ describe("install gates on Runtime readiness", () => {
}
});
it("never hands the home back to the App Runtime after the migration ran", async () => {
it("restores App ownership before relaunching the old Runtime after install failure", async () => {
vi.clearAllMocks();
stubInstallPlan();
const root = await mkdtemp(path.join(tmpdir(), "memhub-cli-owner-"));
@@ -2541,6 +2091,14 @@ describe("install gates on Runtime readiness", () => {
appBundle: "/Applications/MemHub.app",
appExecutable: "/Applications/MemHub.app/Contents/MacOS/MemHub",
});
lifecycleMocks.rollbackPreparedServiceInstall.mockImplementationOnce(
async (_prepared: unknown, rollback: () => Promise<void>) => {
await rollback();
await expect(readRuntimeServiceOwner(dataDir)).resolves.toMatchObject({
owner: "app",
});
},
);
const streams = captureStreams();
try {
expect(
@@ -2563,114 +2121,15 @@ describe("install gates on Runtime readiness", () => {
{ waitForRuntimeReady: async () => false },
),
).toBe(1);
// The App Runtime may predate the SDK migration gate: CLI keeps it.
await expect(readRuntimeServiceOwner(dataDir)).resolves.toMatchObject({
owner: "cli",
owner: "app",
});
expect(
lifecycleMocks.rollbackPreparedServiceInstall,
).not.toHaveBeenCalled();
} finally {
streams.restore();
await rm(root, { recursive: true, force: true });
}
});
it("keeps the install, not serving, when its migration is deferred", async () => {
vi.clearAllMocks();
stubInstallPlan();
runtimeMocks.runSdkMigration.mockResolvedValueOnce({
state: "deferred",
code: "sdk_migration_memory_unreadable",
});
const waitForRuntimeReady = vi.fn(async () => true);
const streams = captureStreams();
let code: number;
try {
code = await run(
[
"install",
"--source",
"/tmp/dist",
"--home",
"/tmp/home",
"--install-root",
"/tmp/install",
"--bin-dir",
"/tmp/bin",
"--data-dir",
"/tmp/data",
"--yes",
],
{},
{ waitForRuntimeReady },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
const failure = JSON.parse(streams.stderr());
expect(failure).toMatchObject({
command: "install",
code: "sdk_migration_memory_unreadable",
log_file: "/tmp/data/logs/sdk-migration.log",
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
});
expect(failure.message).toContain("/tmp/data/logs/sdk-migration.log");
expect(runtimeMocks.runSdkMigration).toHaveBeenCalledWith(
expect.objectContaining({ caller: "cli-install" }),
);
// The new (gated) service owns the data dir; the previous registration is
// never re-bootstrapped and nothing is rolled back.
expect(lifecycleMocks.applyPreparedServiceInstall).toHaveBeenCalledWith(
expect.objectContaining({ restorePreviousOnFailure: false }),
);
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
expect(
lifecycleMocks.rollbackPreparedServiceInstall,
).not.toHaveBeenCalled();
expect(waitForRuntimeReady).not.toHaveBeenCalled();
});
it("still rolls back an install that failed before its migration ran", async () => {
vi.clearAllMocks();
stubInstallPlan();
runtimeMocks.installCliLink.mockRejectedValueOnce(
new Error("cli_link_failed"),
);
const streams = captureStreams();
let code: number;
try {
code = await run(
[
"install",
"--source",
"/tmp/dist",
"--home",
"/tmp/home",
"--install-root",
"/tmp/install",
"--bin-dir",
"/tmp/bin",
"--data-dir",
"/tmp/data",
"--yes",
],
{},
{ waitForRuntimeReady: async () => true },
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
expect(JSON.parse(streams.stderr())).toMatchObject({
command: "install",
code: "cli_link_failed",
});
expect(runtimeMocks.runSdkMigration).not.toHaveBeenCalled();
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
});
it("succeeds when the installed Runtime becomes ready", async () => {
vi.clearAllMocks();
stubInstallPlan();
@@ -2750,8 +2209,7 @@ describe("ui wires the Core session contract without token leakage", () => {
} finally {
output.restore();
}
// A stopped Runtime is exit 3, like every other command.
expect(code!).toBe(3);
expect(code!).toBe(4);
const document = JSON.parse(output.value());
expect(document).toMatchObject({
schema_version: 1,
@@ -2780,120 +2238,6 @@ describe("ui wires the Core session contract without token leakage", () => {
// refuse while the Runtime owns the data dir, stop+call Core from a
// maintenance process, and never report success from a malformed response.
describe("backup restore drives the Core managed-restore contract", () => {
it("previews without --yes, exits 0 and starts nothing", async () => {
const maintenance = vi.fn();
const output = captureStdout();
let code: number;
try {
code = await run(
["backup", "restore", "--source", "/tmp/x", "--json"],
{},
{ runMaintenanceRestore: maintenance },
);
} finally {
output.restore();
}
expect(code!).toBe(0);
expect(maintenance).not.toHaveBeenCalled();
expect(JSON.parse(output.value())).toMatchObject({
command: "backup restore",
state: "preview",
target: "/tmp/x",
applied: false,
next_step: "memhub backup restore --source /tmp/x --yes --confirm /tmp/x",
});
});
it("deletes data only when --confirm repeats the data directory", async () => {
const home = await mkdtemp(path.join(tmpdir(), "memhub-delete-home-"));
const dataDir = path.join(home, ".one-memory");
await mkdir(path.join(dataDir, "logs"), { recursive: true });
const deleteData = async (...extra: string[]) => {
const output = captureStdout();
try {
const code = await run(
["delete-data", "--data-dir", dataDir, ...extra, "--json"],
{ HOME: home },
);
return { code, document: JSON.parse(output.value()) };
} finally {
output.restore();
}
};
try {
const refused = await deleteData("--yes");
expect(refused.code).toBe(4);
expect(refused.document).toMatchObject({
state: "blocked",
code: "data_deletion_confirmation_required",
});
expect(await readdir(dataDir)).toEqual(["logs"]);
const deleted = await deleteData("--yes", "--confirm", dataDir);
expect(deleted.code).toBe(0);
expect(await readdir(dataDir).catch(() => [])).toEqual([]);
} finally {
await rm(home, { recursive: true, force: true });
}
});
it("refuses --yes without repeating the backup path", async () => {
const maintenance = vi.fn();
const output = captureStdout();
let code: number;
try {
code = await run(
["backup", "restore", "--source", "/tmp/x", "--yes", "--json"],
{},
{ runMaintenanceRestore: maintenance },
);
} finally {
output.restore();
}
expect(code!).toBe(4);
expect(maintenance).not.toHaveBeenCalled();
expect(JSON.parse(output.value())).toMatchObject({
state: "rejected",
code: "backup_restore_confirmation_required",
next_step:
"恢复需要再写一遍备份路径:memhub backup restore --source /tmp/x --yes --confirm /tmp/x",
});
});
it("restores with --yes and the repeated backup path", async () => {
const maintenance = vi.fn(async () => ({
state: "restored" as const,
result: {
state: "restored" as const,
restoredBytes: 1,
previousBackupPath: "/tmp/pre.sqlite3",
sourceSchemaVersion: 34,
},
}));
const output = captureStdout();
let code: number;
try {
code = await run(
[
"backup",
"restore",
"--source",
"/tmp/x",
"--yes",
"--confirm",
"/tmp/x",
"--json",
],
{},
{ runMaintenanceRestore: maintenance },
);
} finally {
output.restore();
}
expect(code!).toBe(0);
expect(maintenance).toHaveBeenCalledTimes(1);
});
it("rejects while the Runtime is running, with a distinct exit code", async () => {
runtimeMocks.inspectRuntimeStatus.mockResolvedValueOnce({
state: "running",
@@ -3010,7 +2354,7 @@ describe("backup restore drives the Core managed-restore contract", () => {
});
});
it("refuses a legacy --confirm that does not repeat the source path", async () => {
it("requires --confirm to repeat the exact source path", async () => {
const maintenance = vi.fn();
const output = captureStdout();
let code: number;
@@ -3038,7 +2382,7 @@ describe("backup restore drives the Core managed-restore contract", () => {
command: "backup",
verb: "restore",
state: "rejected",
code: "backup_restore_confirmation_mismatch",
code: "backup_restore_confirmation_required",
});
});
@@ -3190,7 +2534,7 @@ describe("init onboarding", () => {
// Boolean flags were idempotent, but accepting them twice hid a duplicated
// argv fragment that could also have duplicated a value-bearing flag.
it.each(["--import", "--confirm-overrides"])(
it.each(["--import", "--full-rescan", "--confirm-overrides", "--dry-run"])(
"refuses a repeated init %s flag",
async (flag) => {
expect(await run(["init", "--source", "codex", flag, flag], {})).toBe(2);
@@ -3263,25 +2607,12 @@ describe("init onboarding", () => {
},
);
// `--dry-run` and `--full-rescan` never did anything (P0-2/P0-3): they are
// gone rather than accepted as no-ops.
it.each(["--dry-run", "--full-rescan"])(
"refuses the removed no-op %s flag",
async (flag) => {
expect(
await run(["init", "--source", "codex", "--import", flag], {}),
).toBe(2);
},
);
it("still parses the old `init --source --import` spelling", async () => {
// It stops at the Runtime check (exit 3), which is not a usage failure.
const streams = captureStreams();
try {
expect(await run(["init", "--source", "codex", "--import"], {})).toBe(3);
} finally {
streams.restore();
}
it("accepts --dry-run together with --import", async () => {
// Parsing must accept the pair; it stops at the Runtime check, which is
// not a usage failure.
expect(
await run(["init", "--source", "codex", "--import", "--dry-run"], {}),
).not.toBe(2);
});
});
+2267 -2045
View File
File diff suppressed because it is too large. Load diff
+197 -228
View File
@@ -58,12 +58,6 @@ const CONFIRM_OPTION: CommandOption = {
flag: "--yes",
summary: "确认执行;不加该参数时只输出计划,不做任何改动。",
};
/** The one confirmation convention, for irreversible operations only. */
const DESTRUCTIVE_CONFIRM_OPTION: CommandOption = {
flag: "--yes",
summary:
"确认这次不可逆操作;不加时只说明将要做什么,不做任何改动,exit 0。(旧的 --confirm 仍作为别名接受。)",
};
const JSON_OPTION: CommandOption = {
flag: "--json",
summary: "以 JSON 输出到 stdout,供脚本与 Agent 消费。",
@@ -76,6 +70,21 @@ const INTERVAL_OPTION: CommandOption = {
flag: "--interval <seconds>",
summary: "刷新间隔,默认 5 秒,范围 1..300;必须与 --watch 一起使用。",
};
const PLAN_FILE_OPTION: CommandOption = {
flag: "--plan-file <path>",
summary: "读取先前 --json 保存的有界计划文件。",
};
const EXPLICIT_CONFIRM_OPTION: CommandOption = {
flag: "--confirm",
summary: "确认执行计划文件逐字描述的操作;缺少时绝不写入。",
};
// Same flag, but for commands that name their target directly instead of
// going through a saved plan file; the plan-file wording would be wrong here.
const DIRECT_CONFIRM_OPTION: CommandOption = {
flag: "--confirm",
summary: "确认执行上面预览的这次操作;缺少时绝不写入。",
};
const SERVICE_OPTIONS: readonly CommandOption[] = [
PLATFORM_OPTION,
HOME_OPTION,
@@ -145,77 +154,55 @@ const INSTALL_OPTIONS: readonly CommandOption[] = [
const INSTALL_USAGE =
"[--source <path> | --release-manifest-url <https-url>] [--channel <stable|prerelease>] [--install-root <path>] [--home <path>] [--bin-dir <path>] [--data-dir <path>] [--uid <id>] [--xdg-config-home <path>] [--no-shell-path] [--shell-profile <path>] [--preserve-data-dir] [--allow-downgrade] [--yes]";
const USAGE_EXITS = [EXIT_OK, EXIT_FAILED, EXIT_USAGE] as const;
const RUNTIME_EXITS = [
EXIT_OK,
EXIT_FAILED,
EXIT_USAGE,
EXIT_RUNTIME_STOPPED,
] as const;
const KIND_OPTION: CommandOption = {
flag: "--kind <chat|experience|fact>",
summary:
"记忆所在的命名空间;工作/编码场景的经验和事实需要同时给 --variant。list 输出里每条都带 kind/variant。",
};
const VARIANT_OPTION: CommandOption = {
flag: "--variant <working|coding>",
summary:
"经验或事实属于工作还是编码场景;必须和 --kind experience|fact 一起用。",
};
export const COMMANDS: readonly CommandDescriptor[] = [
{
name: "serve",
summary: "在前台启动完整 Runtime(Core 与 Host Gateway),不注册系统服务。",
usage: "memhub serve",
options: [],
exit_codes: USAGE_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED],
},
{
name: "start",
summary:
"启动本机 Runtime;已有通过身份校验的实例时只连接,不重复启动。受管服务会同时恢复登录时自动启动。",
summary: "启动本机 Runtime;已有通过身份校验的实例时只连接,不重复启动。",
usage: "memhub start [--json]",
options: [JSON_OPTION],
exit_codes: USAGE_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_RUNTIME_STOPPED],
},
{
name: "stop",
summary:
"停止通过身份校验的本机 Runtime;身份不明确时不发送信号。受管服务(launchd/systemd、App)停止后登录时也不会自动启动,直到再次运行 memhub start。",
summary: "停止通过身份校验的本机 Runtime;身份不明确时不发送信号。",
usage: "memhub stop [--json]",
options: [JSON_OPTION],
exit_codes: USAGE_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED],
},
{
name: "restart",
summary: "先停止再启动本机 Runtime,两步串行执行。",
usage: "memhub restart [--json]",
options: [JSON_OPTION],
exit_codes: USAGE_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED],
},
{
name: "status",
summary: "只读查询 Runtime 运行状态;不启动服务、不迁移数据、不改配置。",
usage: "memhub status [--watch] [--interval <seconds>] [--json]",
options: [WATCH_OPTION, INTERVAL_OPTION, JSON_OPTION],
exit_codes: RUNTIME_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_RUNTIME_STOPPED],
},
{
name: "doctor",
summary:
"按诊断口径做只读检查并给出修复建议;默认不修复任何东西。Runtime 未运行时 exit 3。",
summary: "按诊断口径做同样的只读检查;默认不修复任何东西。",
usage: "memhub doctor [--json]",
options: [JSON_OPTION],
exit_codes: RUNTIME_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_RUNTIME_STOPPED],
},
{
name: "logs",
summary:
"读取本机脱敏诊断日志;logs export 会把排查包(含系统崩溃报告)直接存到下载目录。默认显示最近一次会话的 200 条事件。",
"读取本机脱敏诊断日志;logs export 会把排查包直接存到下载目录。默认显示最近一次会话的 200 条事件。",
usage:
"memhub logs export [--json]\n memhub logs [--since <15m|2h|7d|RFC3339>] [--session <id>] [--trace <id>] [--incident <id>] [--operation <id>] [--request <id>] [--job <id>] [--task <id>] [--component <name>] [--event <name>] [--errors-only] [--tail <1..5000>] [--follow] [--json]",
"memhub logs export [--json]\nmemhub logs [--since <15m|2h|7d|RFC3339>] [--session <id>] [--trace <id>] [--incident <id>] [--operation <id>] [--request <id>] [--job <id>] [--task <id>] [--component <name>] [--event <name>] [--errors-only] [--tail <1..5000>] [--follow] [--json]",
options: [
{
flag: "--since <time>",
@@ -239,46 +226,64 @@ export const COMMANDS: readonly CommandDescriptor[] = [
"以 NDJSON 输出到 stdout(每行一个脱敏事件;非单 JSON 文档),供脚本与 Agent 消费。",
},
],
exit_codes: USAGE_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE],
},
{
name: "init",
summary:
"首次接入引导。在终端里(未加 --json)进入交互向导:选择来源 → 是否开启处理 → 配置模型。非终端或加 --json 时不读 stdin,只列出可接入的来源和下一步命令。脚本接入请用 sources connect。",
usage: "memhub init [--json]",
options: [JSON_OPTION],
exit_codes: RUNTIME_EXITS,
},
{
name: "sources",
summary:
"列出来源,直接接入来源,或按 request id 查接入回执。connect 不加 --import 只授权(处理保持冻结:不导入历史、不扫描新会话);加 --import 同时导入历史并开启持续同步,返回 request id。目录来自环境变量覆盖的来源必须加 --confirm-overrides。",
"首次接入:选择来源 → 选择是否导入历史。接入即开启该 Harness 的处理;后续可用 sync pause/resume 冻结或恢复。在终端且不带 --source 时进入交互引导(默认全选);非终端或加 --json 时不读 stdin、只列出可用来源和下一步。--all-detected 一次接入所有已检测来源。",
usage:
"memhub sources list [--json]\n memhub sources connect <id> [<id> ...] [--import] [--confirm-overrides] [--request-id <uuid>] [--json]\n memhub sources connect --all-detected [--import] [--confirm-overrides] [--json]\n memhub sources receipt <request-id> [--json]",
"memhub init [--json]\n memhub init --source <id> [--source <id> ...] [--import] [--dry-run] [--full-rescan] [--confirm-overrides] [--json]\n memhub init --all-detected [--import] [--dry-run] [--json]",
options: [
{
flag: "--import",
summary:
"接入后立即导入历史会话并开启持续同步;不加时只授权,处理保持冻结。",
flag: "--source <id>",
summary: "要接入的来源,可重复;终端下不给则进入交互引导。",
},
{
flag: "--all-detected",
summary:
"接入全部已检测到(location_state=found)的来源;与具体来源互斥。",
"接入并(加 --import 时)导入全部 location_state=found 的来源。",
},
{
flag: "--import",
summary: "接入后立即导入该来源的历史会话(默认只接入不导入)。",
},
{
flag: "--dry-run",
summary: "只出计划,不接入也不导入;用于先看会做什么。",
},
{
flag: "--full-rescan",
summary: "导入时忽略既有游标做全量重扫。",
},
{
flag: "--confirm-overrides",
summary:
"确认接受目录来自环境变量覆盖的来源(如 DSH_HOME 指定的目录);不加时这些来源被拒绝(指定来源时)或跳过并列出(--all-detected 时)。",
},
{
flag: "--request-id <uuid>",
summary:
"--import 的幂等键;用同一个 id 重试只会读回第一次的回执,不会重复导入。",
summary: "确认接受环境覆盖类来源(如 DSH_HOME 指定的目录)。",
},
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "setup",
summary: "生成无头首次导入计划;只读取已有来源,不探测目录、不写授权。",
usage: "memhub setup plan --source <id> [--source <id> ...] [--json]",
options: [
{
flag: "--source <id>",
summary: "计划导入的来源,可重复 1..32 次。",
},
JSON_OPTION,
],
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "sources",
summary: "列出来源、执行已确认的导入计划。",
usage:
"memhub sources list [--json]\n memhub sources import --plan-file <path> --confirm [--json]\n memhub sources receipt <request-id> [--json]",
options: [PLAN_FILE_OPTION, EXPLICIT_CONFIRM_OPTION, JSON_OPTION],
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "sync",
@@ -292,7 +297,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
},
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "progress",
@@ -313,17 +318,17 @@ export const COMMANDS: readonly CommandDescriptor[] = [
INTERVAL_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "jobs",
summary:
"查看或控制导入任务:start 创建 history/sync 任务;pause/resume/cancel 控制已有任务。都直接执行。",
"启动或控制单个导入任务:start 创建 history/sync 作业;pause/resume/cancel 控制已有作业。所有写操作先出计划,必须显式 --confirm;list/show 为只读。",
usage:
"memhub jobs list [--json]\n" +
" memhub jobs show <job-id> [--json]\n" +
" memhub jobs start --source <id> [--operation <history|sync>] [--json]\n" +
" memhub jobs <pause|resume|cancel> --job <id> [--json]",
" memhub jobs start --source <id> [--operation <history|sync>] [--confirm] [--json]\n" +
" memhub jobs <pause|resume|cancel> --job <id> [--confirm] [--json]",
options: [
{
flag: "--source <id>",
@@ -331,116 +336,123 @@ export const COMMANDS: readonly CommandDescriptor[] = [
},
{
flag: "--operation <history|sync>",
summary:
"start 的导入类型,默认 history;sync 与 scan --source 相同,立即增量同步一次。",
summary: "start 的导入类型,默认 history。",
},
{
flag: "--job <id>",
summary: "pause/resume/cancel 的目标导入任务。",
},
DIRECT_CONFIRM_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "models",
summary:
"管理生成模型配置。add/test 直接提交并返回操作 ID(异步,用 receipt 查结果);edit/activate 直接执行;remove 不可逆,需 --yes。<profile-id>/<operation-id> 都是 UUID;脚本模式的凭据只从受控文件读取。",
"交互配置模型,或生成配置/探测计划、执行已确认操作与查询回执;脚本模式的凭据只从受控文件读取。",
usage:
"memhub models list [--json]\n memhub models show <profile-id> [--json]\n memhub models add --config-file <path> [--api-key-file <path>] [--activate] [--json]\n memhub models test <profile-id> [--json]\n memhub models receipt <operation-id> [--json]\n memhub models edit <profile-id> --config-file <path> [--api-key-file <path>] [--json]\n memhub models activate <profile-id> [--json]\n memhub models remove <profile-id> [--yes] [--json]",
"memhub models list [--json]\n memhub models show <profile-id> [--json]\n memhub models plan --config-file <path> [--json]\n memhub models configure --plan-file <path> --confirm [--api-key-file <path>] [--json]\n memhub models receipt <operation-id> [--json]\n memhub models edit <profile-id> --config-file <path> [--confirm] [--api-key-file <path>] [--json]\n memhub models activate <profile-id> [--confirm] [--json]\n memhub models probe-plan <profile-id> [--json]\n memhub models probe --plan-file <path> --confirm [--json]\n memhub models probe-receipt <operation-id> [--json]\n memhub models remove <profile-id> [--confirm] [--json]",
alternative_usage: "memhub models setup",
options: [
{
flag: "--config-file <path>",
summary:
"有界 JSON 模型配置,不能含 api_key;edit 的补丁必须带 expected_revision。",
summary: "不含 api_key 的有界 JSON 模型配置。",
},
PLAN_FILE_OPTION,
EXPLICIT_CONFIRM_OPTION,
{
flag: "--api-key-file <path>",
summary: "从属主专用的普通文件读取凭据;内容不进入参数、输出或错误。",
summary: "从 regular file 读取凭据;内容不进入参数、输出或错误。",
},
{
flag: "--activate",
summary: "add 成功后把新配置设为当前使用。",
},
DESTRUCTIVE_CONFIRM_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "memories",
summary:
"列出、搜索、查看、添加、修改或物理删除记忆(对话、工作、编码三个场景)。add 和 App 的「添加记忆」是同一个入口;delete 不可逆,需 --yes。",
summary: "通过 SDK 数据面列出、查看、修改或物理删除记忆。",
usage:
"memhub memories list [--scene <all|chat|working|coding>] [--source <id>] [--query <text>] [--status <state>] [--offset <n>] [--limit <n>] [--json]\n memhub memories show <memory-id> [--kind <kind>] [--variant <variant>] [--json]\n memhub memories add <text> [--request-id <uuid>] [--json]\n memhub memories add --content-file <path> [--request-id <uuid>] [--json]\n memhub memories add-status <request-id> [--json]\n memhub memories update <memory-id> --content-file <path> [--kind <kind>] [--variant <variant>] [--task <text>] [--json]\n memhub memories delete <memory-id> [--kind <kind>] [--variant <variant>] [--yes] [--json]",
"memhub memories list [--status <state>] [--offset <n>] [--limit <n>] [--json]\n memhub memories show <memory-id> [--json]\n memhub memories update <memory-id> --content-file <path> [--confirm] [--json]\n memhub memories delete <memory-id> [--confirm] [--json]",
options: [
{
flag: "--scene <all|chat|working|coding>",
summary: "只列一个场景,默认 all。working/coding 列的是该场景的经验。",
},
{ flag: "--source <id>", summary: "只列某个来源写入的记忆。" },
{
flag: "--query <text>",
summary:
"关键词搜索(只搜 active 记忆,在最相关的一批里匹配所有关键词,输出 search_window 时表示可能少计)。",
},
{
flag: "--status <state>",
summary: `记忆状态,取值 ${MEMORY_STATUSES.join("、")},默认 active。`,
summary: `记忆状态,取值 ${MEMORY_STATUSES.join("、")}。`,
},
{ flag: "--offset <n>", summary: "跳过前 n 条,默认 0。" },
{
flag: "--limit <n>",
summary: "本页最多返回多少条,默认 100,上限 500。",
},
KIND_OPTION,
VARIANT_OPTION,
{
flag: "--content-file <path>",
summary:
"从有界普通文件读取正文:add 的新记忆(≤12000 字),或 update 的新内容(经验为 solution)。",
"从有界普通文件读取新增/纠正正文;计划只保存摘要,执行时再次校验。",
},
{
flag: "--task <text>",
summary: "update 经验(--kind experience)时同时改写 task。",
},
{
flag: "--request-id <uuid>",
summary:
"add 的幂等键;用同一个 id 重试只会读回第一次的结果,不会重复添加。不给时自动生成并输出。",
},
DESTRUCTIVE_CONFIRM_OPTION,
DIRECT_CONFIRM_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "profile",
summary: "读取 SDK 组装后的用户长期画像(L0 + 已启用的长期画像补充)。",
usage: "memhub profile [--json]",
options: [JSON_OPTION],
exit_codes: RUNTIME_EXITS,
options: [DATA_DIR_OPTION, JSON_OPTION],
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "graph-width",
name: "calls",
summary:
"嵌入维度变化、图谱宽度不匹配时的处置决定。keep_existing 与 export_first 直接执行;rebuild_with_new 会归档旧图谱并重建,需 --yes。",
"清除本机调用记录;不可逆删除,默认只出计划,必须显式 --confirm 才执行。",
usage:
"memhub graph-width --decision <keep_existing|export_first|rebuild_with_new> [--yes] [--json]",
"memhub calls clear --event <id> [--event <id> ...] [--confirm] [--json]\n memhub calls clear --all [--confirm] [--json]",
options: [
{
flag: "--event <id>",
summary: "要清除的调用记录事件 ID,可重复,最多 100 条。",
},
{
flag: "--all",
summary: "清除全部调用记录;与 --event 互斥。",
},
DIRECT_CONFIRM_OPTION,
JSON_OPTION,
],
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "notes",
summary:
"端内新增记忆(memory-notes):从普通文件新增,或显式确认后处理已有笔记。",
usage:
"memhub notes add --content-file <path> [--json]\n memhub notes apply --plan-file <path> --content-file <path> --confirm [--json]\n memhub notes process <note-id> [--confirm] [--json]",
options: [
{
flag: "--content-file <path>",
summary: "从有界普通文件读取新增正文;计划只保存摘要,执行时重新校验。",
},
PLAN_FILE_OPTION,
DIRECT_CONFIRM_OPTION,
JSON_OPTION,
],
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "graph",
summary: "图谱宽度不匹配时的用户决定;默认只出计划,--confirm 后才执行。",
usage:
"memhub graph width --decision <keep_existing|rebuild_with_new|export_first> [--confirm] [--json]",
options: [
{
flag: "--decision <decision>",
summary:
"keep_existing(保持现有图谱)、export_first(先导出)、rebuild_with_new(导出后归档旧图谱并按新维度重建)。",
},
{
...DESTRUCTIVE_CONFIRM_OPTION,
summary: "rebuild_with_new 必须加;其它决定不需要。",
"keep_existing(保持)、export_first(先导出)、rebuild_with_new(导出后重建)。",
},
DIRECT_CONFIRM_OPTION,
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "host",
@@ -468,12 +480,12 @@ export const COMMANDS: readonly CommandDescriptor[] = [
},
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "recall",
summary:
"从原始对话与已沉淀记忆统一检索上下文;本地只读,不调用模型、不触发补索引。文本输出标明每条是「原始对话摘录」还是「记忆」(JSON 里是 kind: raw|memory)。",
"从当前授权的原文与治理记忆统一检索上下文;本地只读,不调用模型、不触发补索引。",
usage:
"memhub recall <query> [--kind <raw|memory>] [--source <id>] [--limit <n>] [--character-budget <n>] [--timeout-ms <n>] [--json]\n memhub recall --query-file <path> [...]\n memhub recall --stdin [...]",
options: [
@@ -504,21 +516,21 @@ export const COMMANDS: readonly CommandDescriptor[] = [
},
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
},
{
name: "service-plan",
summary: "生成用户级服务计划;只输出计划,不写文件也不执行命令。",
usage: `memhub service-plan ${SERVICE_USAGE}`,
options: SERVICE_OPTIONS,
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_PLAN_BLOCKED],
exit_codes: [EXIT_OK, EXIT_PLAN_BLOCKED],
},
{
name: "service-install",
summary: "预览用户级服务安装;只有 --yes 才写入服务文件并注册。",
usage: `memhub service-install ${SERVICE_USAGE} [--yes]`,
options: [...SERVICE_OPTIONS, CONFIRM_OPTION],
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_PLAN_BLOCKED],
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_PLAN_BLOCKED],
},
{
name: "service-uninstall",
@@ -526,15 +538,15 @@ export const COMMANDS: readonly CommandDescriptor[] = [
"预览用户级服务卸载;只有 --yes 才注销并删除逐字匹配的受管服务文件,不删数据。",
usage: `memhub service-uninstall ${SERVICE_USAGE} [--yes]`,
options: [...SERVICE_OPTIONS, CONFIRM_OPTION],
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_PLAN_BLOCKED],
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_PLAN_BLOCKED],
},
{
name: "install",
summary:
"预览发行目录安装、memhub/memhub-cli/one-memory 短链接、PATH 条目与用户服务计划;只有 --yes 才执行。记忆数据迁移开始前失败会回滚;迁移开始后失败保留新版本、拒绝启动,并输出迁移日志与反馈地址。",
"预览发行目录安装、memhub/memhub-cli/one-memory 短链接、PATH 条目与用户服务计划;只有 --yes 才执行并在失败时回滚。",
usage: `memhub install ${INSTALL_USAGE}`,
options: INSTALL_OPTIONS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_PLAN_BLOCKED],
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_PLAN_BLOCKED],
},
{
name: "uninstall",
@@ -547,31 +559,24 @@ export const COMMANDS: readonly CommandDescriptor[] = [
{
name: "upgrade",
summary:
"预览升级;只有 --yes 才切换受管版本并重启服务,不删数据。记忆数据迁移开始前失败会恢复旧版本;迁移开始后失败保留新版本、拒绝启动,并输出迁移日志与反馈地址(不会回滚到旧版本)。",
"预览升级;只有 --yes 才切换受管版本并重启服务,失败时恢复旧版本,不删数据。",
usage: `memhub upgrade ${INSTALL_USAGE}`,
options: INSTALL_OPTIONS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_PLAN_BLOCKED],
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_PLAN_BLOCKED],
},
{
name: "delete-data",
summary:
"删除 MemHub 数据目录(记忆、来源、配置),不可逆。这是唯一会删除数据的命令;uninstall 不会。需要 Runtime 已停止;不加参数时只预览要删除的目录;删除需要 --yes 并用 --confirm 再写一遍同一个目录。",
"独立、二次确认地删除 One Memory Home(数据)。这是唯一会删除数据的命令;uninstall 不会。默认只出预览;需 Runtime 已停止,并显式 --confirm <path> 复述该目录。",
usage:
"memhub delete-data [--data-dir <path>] [--yes --confirm <path>] [--json]",
options: [
{
flag: "--data-dir <path>",
summary: "要删除的数据目录绝对路径;默认当前使用的数据目录。",
},
{
...DESTRUCTIVE_CONFIRM_OPTION,
summary:
"删除时必须加,且要同时给出 --confirm <path>;不加时只预览,exit 0。",
},
DATA_DIR_OPTION,
CONFIRM_OPTION,
{
flag: "--confirm <path>",
summary:
"删除时必须加:再写一遍要删除的数据目录,与预览中的目录完全一致才执行。",
"第二次确认:逐字复述预览里的数据目录;缺少它时即使 --yes 也不删除。",
},
JSON_OPTION,
],
@@ -583,149 +588,113 @@ export const COMMANDS: readonly CommandDescriptor[] = [
"onboarding reset 回到首次运行的引导流程。只清除引导完成状态:不删记忆数据,也不需要卸载重装。",
usage: "memhub onboarding reset [--json]",
options: [JSON_OPTION],
exit_codes: RUNTIME_EXITS,
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE],
},
{
name: "help",
summary: "显示总帮助或单个命令的用法。",
usage: "memhub help [command] [--json]",
options: [JSON_OPTION],
exit_codes: [EXIT_OK, EXIT_USAGE],
exit_codes: [EXIT_OK],
},
{
name: "version",
summary: "显示 CLI 版本。",
usage: "memhub version [--json]",
options: [JSON_OPTION],
exit_codes: [EXIT_OK, EXIT_USAGE],
exit_codes: [EXIT_OK],
},
{
name: "scan-roots",
name: "path",
summary:
"查看或登记被扫描的目录(scan roots)。add/remove 直接执行。(旧名 path 仍可用。)",
"查看或登记被扫描的目录(scan roots)。list 只读;add/remove 预览后加 --yes 才提交;均已接入 Runtime 的 scan-roots 端点。",
usage:
"memhub scan-roots list [--json]\n" +
" memhub scan-roots add --root <path> [--source <id>] [--no-recursive] [--json]\n" +
" memhub scan-roots remove --root <path> [--json]",
options: [
{ flag: "--root <path>", summary: "要登记或移除的目录。" },
{
flag: "--source <id>",
summary: "把目录绑定到某个来源;不给时由 Runtime 自动识别。",
},
{ flag: "--no-recursive", summary: "只扫描该目录本身,不进入子目录。" },
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
"memhub path list [--json]\n" +
" memhub path add --root <path> [--source <id>] [--no-recursive] [--yes]\n" +
" memhub path remove --root <path> [--yes]",
options: [DATA_DIR_OPTION, JSON_OPTION, CONFIRM_OPTION],
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED, EXIT_PLAN_BLOCKED],
},
{
name: "adapter",
summary:
"查看 Harness Adapter 的就绪状态与最近一次探测结果(list),或现场探测某个来源(probe --source <id>);两者均为只读。",
"查看 Harness Adapter 的就绪状态与最近一次探测结果(list),或现场探测某个来源(probe --source <id>);两者均为只读,已接入 Runtime 的 adapters/探测端点。",
usage:
"memhub adapter list [--json]\n" +
" memhub adapter probe --source <id> [--json]",
options: [
{ flag: "--source <id>", summary: "probe 的目标来源。" },
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
options: [DATA_DIR_OPTION, JSON_OPTION],
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED],
},
{
name: "scan",
summary:
"立即增量扫描一个已连接并开启同步的来源,返回导入任务(与 jobs start --operation sync 相同)。接入全部来源请用 sources connect --all-detected --import。",
usage: "memhub scan --source <id> [--json]",
"立即扫描;默认只输出计划,加 --yes 才真正执行。指定 --source 扫描单个来源,--all 一次扫描/导入全部已检测来源(Runtime 按来源寻址,不接受裸路径)。",
usage:
"memhub scan [--source <id>] [--full] [--yes] [--json]\n memhub scan --all [--yes] [--json]",
options: [
{
flag: "--source <id>",
summary: "要扫描的来源。",
summary: "按单个来源扫描;与 --all 互斥。",
},
{
flag: "--all",
summary:
"处理全部 location_state=found 的来源,按来源逐条回执;部分成功返回非 0。",
},
{
flag: "--full",
summary: "忽略既有游标做全量重扫;默认只读取增量。",
},
DATA_DIR_OPTION,
JSON_OPTION,
CONFIRM_OPTION,
],
exit_codes: RUNTIME_EXITS,
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED, EXIT_PLAN_BLOCKED],
},
{
name: "write",
summary:
"提交一段 OpenAI 格式的对话(messages)到写入队列,异步加工成记忆;或按 request id 查询状态。request id 是幂等键:不给时自动生成并输出,重试请带上同一个。新增一条结论式记忆请用 memories add。",
summary: "提交 OpenAI messages 到 SDK 写入队列,或按 request id 查询状态。",
usage:
"memhub write submit --messages-file <path> --session <id> [--source <id>] [--request-id <id>] [--memory-at <RFC3339>] [--live] [--json]\n memhub write status <request-id> [--json]",
options: [
{
flag: "--messages-file <path>",
summary:
"OpenAI messages 数组(或 {messages:[…]}),1..200 条,role 为 user/assistant/tool,至少一条 user。",
},
{ flag: "--session <id>", summary: "会话 ID。" },
{ flag: "--source <id>", summary: "来源应用,默认 memhub-cli。" },
{
flag: "--request-id <id>",
summary:
"幂等键:同 id 同内容不会重复入队,同 id 不同内容被拒绝(409)。",
},
{ flag: "--memory-at <RFC3339>", summary: "对话发生时间,必须带时区。" },
{ flag: "--live", summary: "优先处理。" },
JSON_OPTION,
],
exit_codes: RUNTIME_EXITS,
"memhub write submit --messages-file <path> --session <id> [--source <id>] [--request-id <id>] [--memory-at <RFC3339>] [--live] [--json]\n memhub write status --id <request-id> [--json]",
options: [DATA_DIR_OPTION, JSON_OPTION],
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED],
},
{
name: "task",
summary:
"兼容别名:按 request id 查询写入状态;新脚本请使用 write status。",
usage: "memhub task --id <request-id> [--json]",
options: [DATA_DIR_OPTION, JSON_OPTION],
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED],
},
{
name: "backup",
summary:
"备份与恢复主库(来源、写入队列、配置)。注意:不包含记忆本身——SDK 记忆库 hy-memory/ 不在备份里,restore 也不会替换它。create 的目标要放在 <数据目录>/backups/ 下才能被 restore 使用;restore 覆盖现有主库,需 Runtime 已停止,并加 --yes 与 --confirm <同一个备份路径>。",
"备份与恢复 MemHub 数据(两阶段)。create 调用 Runtime 的备份端点;restore 会覆盖现有数据,要求 Runtime 已停止且 `--confirm` 逐字重复 `--source`,随后由非 Runtime 维护态 Core 完成受管校验、保护性备份和原子替换。",
usage:
"memhub backup create --target <path> [--json]\n" +
" memhub backup restore --source <path> [--yes --confirm <path>] [--json]",
"memhub backup create --target <path> [--yes] [--json]\n" +
" memhub backup restore --source <path> --yes --confirm <path> [--json]",
options: [
{
flag: "--target <path>",
summary: "create 的备份文件路径,必须在数据目录内。",
},
{
flag: "--source <path>",
summary:
"restore 的备份文件,必须是 <数据目录>/backups/ 下的受管备份。",
},
{
...DESTRUCTIVE_CONFIRM_OPTION,
summary:
"restore 必须加,且要同时给出 --confirm <path>;create 不需要。",
},
DATA_DIR_OPTION,
JSON_OPTION,
CONFIRM_OPTION,
{
flag: "--confirm <path>",
summary:
"restore 必须加:再写一遍 --source 的备份路径,完全一致才执行。",
summary: "restore 要求逐字重复 --source 路径;不一致时不做任何改动。",
},
JSON_OPTION,
],
exit_codes: [...RUNTIME_EXITS, EXIT_PLAN_BLOCKED],
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED, EXIT_PLAN_BLOCKED],
},
{
name: "ui",
summary:
"用系统默认浏览器打开 Runtime 托管的 Web Console:先向 Core 换取一次性 loopback 登录码,再交给系统浏览器由 Core 置 HttpOnly/SameSite=Strict session;CLI 不向浏览器暴露裸 Core token。Runtime 未运行时 exit 3;Core 未提供该合同时如实返回 ui_session_contract_unavailable(exit 4)。",
"用系统默认浏览器打开 Runtime 托管的 Web Console:先向 Core 换取一次性 loopback 登录码,再交给系统浏览器由 Core 置 HttpOnly/SameSite=Strict session;CLI 不向浏览器暴露裸 Core token。Core 未提供该合同时如实返回 ui_session_contract_unavailable(exit 4)。",
usage: "memhub ui [--json]",
options: [JSON_OPTION],
exit_codes: [...RUNTIME_EXITS, EXIT_PLAN_BLOCKED],
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_PLAN_BLOCKED],
},
];
/** Old names that still run but are not listed: `path` → `scan-roots`,
* `task` → `write status`, `graph width` → `graph-width`. */
export const HIDDEN_COMMAND_ALIASES: Readonly<Record<string, string>> = {
path: "scan-roots",
task: "write",
graph: "graph-width",
};
export function findCommand(name: string): CommandDescriptor | undefined {
return COMMANDS.find((command) => command.name === name);
}
/** A listed command, or the command a hidden alias now runs as. */
export function findCommandOrAlias(
name: string,
): CommandDescriptor | undefined {
return findCommand(HIDDEN_COMMAND_ALIASES[name] ?? name);
}
+30 -114
View File
@@ -69,6 +69,7 @@ const COMMANDS = [
"doctor",
"logs",
"init",
"setup",
"sources",
"sync",
"progress",
@@ -76,7 +77,10 @@ const COMMANDS = [
"models",
"memories",
"profile",
"graph-width",
// C6 App BLOCK (25f4ecf): UI write operations that had no CLI equivalent.
"calls",
"notes",
"graph",
"host",
"recall",
"service-plan",
@@ -89,20 +93,19 @@ const COMMANDS = [
"onboarding",
"help",
"version",
"scan-roots",
// A6 product commands. Appended, not inserted: the catalog is a stable
// contract, so existing names must not shift position.
"path",
"adapter",
"scan",
"write",
"task",
"backup",
// `ui` is declared but blocked on a Core session contract; it is listed so
// `help` documents it and the blocked shape is machine-readable.
"ui",
] as const;
/** Old names that still run and still answer `--help`, but are not listed. */
const HIDDEN_ALIASES = [
["path", "scan-roots"],
["task", "write"],
] as const;
describe("CLI-22 shared output contract", () => {
describe("help", () => {
it.each([["help"], ["--help"], ["-h"]])(
@@ -143,56 +146,6 @@ describe("CLI-22 shared output contract", () => {
},
);
it.each(HIDDEN_ALIASES)(
"documents the hidden alias %s as %s and keeps it out of the listing",
async (alias, target) => {
const streams = captureStreams();
try {
expect(await run(["help", alias], {})).toBe(0);
expect(await run([alias, "--help"], {})).toBe(0);
} finally {
streams.restore();
}
expect(streams.stdout()).toContain(`${PRODUCT} ${target}`);
const listing = captureStreams();
try {
expect(await run(["help", "--json"], {})).toBe(0);
} finally {
listing.restore();
}
const names = JSON.parse(listing.stdout()).commands.map(
(entry: { name: string }) => entry.name,
);
expect(names).not.toContain(alias);
},
);
it("lists no removed command", async () => {
const streams = captureStreams();
try {
expect(await run(["help", "--json"], {})).toBe(0);
} finally {
streams.restore();
}
const catalog = streams.stdout();
for (const removed of [
'"name":"setup"',
'"name":"notes"',
'"name":"calls"',
'"name":"graph"',
"--plan-file",
"probe-plan",
"probe-receipt",
"models configure",
"sources import",
"--full-rescan",
"--dry-run",
"--full]",
]) {
expect(catalog).not.toContain(removed);
}
});
it("never executes the documented command", async () => {
for (const command of COMMANDS) {
for (const argv of [
@@ -248,63 +201,27 @@ describe("CLI-22 shared output contract", () => {
expect(payload.commands).toBeUndefined();
});
it("documents the one --yes convention for destructive commands", async () => {
const general = captureStreams();
try {
expect(await run(["help"], {})).toBe(0);
} finally {
general.restore();
}
expect(general.stdout()).toContain("确认约定");
for (const [command, usage] of [
[
"backup",
"memhub backup restore --source <path> [--yes --confirm <path>]",
],
["memories", "memhub memories delete <memory-id>"],
["models", "memhub models remove <profile-id> [--yes]"],
[
"delete-data",
"memhub delete-data [--data-dir <path>] [--yes --confirm <path>]",
],
["graph-width", "[--yes]"],
] as const) {
const streams = captureStreams();
try {
expect(await run(["help", command], {})).toBe(0);
} finally {
streams.restore();
}
expect(streams.stdout()).toContain(usage);
// Only the two commands with no way back repeat the target path.
if (command !== "backup" && command !== "delete-data") {
expect(streams.stdout()).not.toContain("--confirm <path>");
}
}
});
it("says backup does not include the memories themselves", async () => {
it("documents the destructive restore confirmation explicitly", async () => {
const streams = captureStreams();
try {
expect(await run(["help", "backup"], {})).toBe(0);
} finally {
streams.restore();
}
expect(streams.stdout()).toContain("不包含记忆本身");
expect(streams.stdout()).toContain(
"memhub backup restore --source <path> --yes --confirm <path>",
);
expect(streams.stdout()).toContain("逐字重复 --source");
});
it("does not promise a rollback after the migration started", async () => {
for (const command of ["install", "upgrade"]) {
const streams = captureStreams();
try {
expect(await run(["help", command], {})).toBe(0);
} finally {
streams.restore();
}
expect(streams.stdout()).toContain("迁移开始后失败保留新版本");
expect(streams.stdout()).not.toContain("失败时恢复旧版本");
expect(streams.stdout()).not.toContain("失败时回滚");
it("does not attach restore's confirmation flag to path", async () => {
const streams = captureStreams();
try {
expect(await run(["help", "path"], {})).toBe(0);
} finally {
streams.restore();
}
expect(streams.stdout()).not.toContain("--confirm <path>");
});
});
@@ -628,23 +545,22 @@ describe("documented commands", () => {
models: [
"list",
"show",
"add",
"test",
"plan",
"configure",
"receipt",
"edit",
"activate",
"probe-plan",
"probe",
"probe-receipt",
"remove",
],
// Source processing is one Freeze/Thaw switch under `sync`; the obsolete
// independent read/write `sources access` command must not return here.
sources: ["list", "connect", "receipt"],
sources: ["list", "import", "receipt"],
jobs: ["list", "show", "start", "<pause|resume|cancel>"],
// `sync <pause|resume>` is documented as one alternation, so it is
// checked as an entry rather than as two literal subcommands.
sync: ["status", "<pause|resume>"],
memories: ["list", "show", "add", "add-status", "update", "delete"],
"scan-roots": ["list", "add", "remove"],
write: ["submit", "status"],
memories: ["list", "show", "update", "delete"],
host: ["list", "status", "install", "uninstall", "sync"],
};
+135 -84
View File
@@ -43,35 +43,61 @@ const ATTACHED = {
};
const RESPONSE = {
schema_version: 2,
schema_version: 1,
request_id: "context-request-1",
state: "grounded",
paths: {
raw: {
status: "ok",
reason: null,
elapsed_ms: 2,
returned_count: 1,
truncated: false,
truncated_by: [],
},
memory: {
status: "ok",
reason: null,
elapsed_ms: 3,
returned_count: 1,
truncated: false,
truncated_by: [],
},
},
items: [
{
kind: "raw",
raw_id: "raw-1",
content: "Synthetic raw citation.",
source: "codex",
session_id: "session-1",
memory_at: "2026-09-19T00:00:00Z",
score: 0.8,
source_id: "codex",
conversation_id: "conversation-1",
anchor_evidence_id: "evidence-1",
messages: [
{
evidence_id: "evidence-1",
revision: 1,
role: "user",
written_by: "user",
created_at: "2026-09-19T00:00:00Z",
text: "Synthetic raw citation.",
truncated: false,
},
],
truncated: false,
},
{
kind: "memory",
memory_id: "memory-1",
content: "Synthetic SDK memory.",
layer: "l2_fact",
status: "active",
source: "codex",
revision: 2,
content: "Synthetic governed memory.",
memory_kind: "working",
updated_at: "2026-09-19T00:00:01Z",
score: 0.7,
evidence_ids: ["evidence-1"],
truncated: false,
},
],
used_characters: 44,
used_characters: 49,
truncated: false,
total_ms: 5,
adoption: "unknown",
};
beforeEach(() => {
@@ -89,17 +115,42 @@ beforeEach(() => {
});
describe("unified context search", () => {
it("accepts an empty raw item in a valid partial context response", async () => {
it("accepts an empty tool message in a valid partial context response", async () => {
transport.post.mockResolvedValue({
status: 200,
data: {
...RESPONSE,
state: "partial",
paths: {
raw: {
...RESPONSE.paths.raw,
status: "degraded",
reason: "lexical_only",
returned_count: 1,
},
memory: {
...RESPONSE.paths.memory,
status: "empty",
returned_count: 0,
},
},
items: [
{
...RESPONSE.items[0],
source: "workbuddy",
content: "",
kind: "raw",
source_id: "workbuddy",
conversation_id: "conversation-1",
anchor_evidence_id: "evidence-1",
messages: [
{
evidence_id: "evidence-1",
revision: 1,
role: "tool",
written_by: "tool",
created_at: "2026-10-05T20:48:12Z",
text: "",
truncated: true,
},
],
truncated: true,
},
],
@@ -114,9 +165,8 @@ describe("unified context search", () => {
streams.restore();
}
expect(JSON.parse(streams.stdout())).toMatchObject({
schema_version: 2,
state: "partial",
items: [{ kind: "raw", content: "" }],
items: [{ kind: "raw", messages: [{ text: "" }] }],
});
});
@@ -128,26 +178,12 @@ describe("unified context search", () => {
streams.restore();
}
const output = streams.stdout();
expect(output).toContain("Synthetic SDK memory.");
expect(output).toContain("Synthetic governed memory.");
expect(output).toContain("Synthetic raw citation.");
expect(output).not.toMatch(/^- raw |memory-1|raw-1|采用状态/m);
expect(output).not.toMatch(/^- raw |memory-1|evidence-1|采用状态/m);
});
it("labels raw hits as original-conversation excerpts, not memories", async () => {
const streams = captureStreams();
try {
expect(await run(["recall", "synthetic question"])).toBe(0);
} finally {
streams.restore();
}
const output = streams.stdout();
expect(output).toContain(
"- [原始对话摘录 · codex] Synthetic raw citation.",
);
expect(output).toContain("- [记忆] Synthetic SDK memory.");
});
it("queries the SDK context endpoint directly and preserves item ids", async () => {
it("queries the current read-only context endpoint and preserves citations", async () => {
const streams = captureStreams();
try {
expect(
@@ -173,7 +209,7 @@ describe("unified context search", () => {
streams.restore();
}
expect(transport.post).toHaveBeenCalledWith(
"/api/v2/context/search",
"/api/v1/context/search",
{
query: "synthetic question",
kinds: ["raw", "memory"],
@@ -186,18 +222,12 @@ describe("unified context search", () => {
);
const output = JSON.parse(streams.stdout());
expect(output).toMatchObject({
schema_version: 2,
state: "grounded",
request_id: "context-request-1",
adoption: "unknown",
});
expect(output).not.toHaveProperty("paths");
expect(output.items[0]).toMatchObject({ kind: "raw", raw_id: "raw-1" });
expect(output.items[1]).toMatchObject({
kind: "memory",
memory_id: "memory-1",
layer: "l2_fact",
});
// No authority pre-query: released Core always serves the SDK data plane.
expect(output.items[0].messages[0].evidence_id).toBe("evidence-1");
expect(output.items[1].evidence_ids).toEqual(["evidence-1"]);
expect(transport.get).not.toHaveBeenCalled();
expect(transport.put).not.toHaveBeenCalled();
expect(transport.patch).not.toHaveBeenCalled();
@@ -212,8 +242,19 @@ describe("unified context search", () => {
status: 200,
data: {
...RESPONSE,
paths: {
...RESPONSE.paths,
raw: {
status: "not_requested",
reason: "not_requested",
elapsed_ms: 0,
returned_count: 0,
truncated: false,
truncated_by: [],
},
},
items: [RESPONSE.items[1]],
used_characters: 21,
used_characters: 26,
},
});
const streams = captureStreams();
@@ -353,7 +394,29 @@ describe("unified context search", () => {
status: 200,
data: {
...RESPONSE,
items: [{ ...RESPONSE.items[0], content: text }],
paths: {
...RESPONSE.paths,
memory: {
...RESPONSE.paths.memory,
status: "empty",
returned_count: 0,
},
},
items: [
{
...(RESPONSE.items[0] as Record<string, unknown>),
messages: [
{
...(
RESPONSE.items[0] as {
messages: Array<Record<string, unknown>>;
}
).messages[0],
text,
},
],
},
],
used_characters: 10,
},
});
@@ -366,7 +429,7 @@ describe("unified context search", () => {
expect(JSON.parse(streams.stdout())).toMatchObject({
state: "grounded",
used_characters: 10,
items: [{ kind: "raw", content: text }],
items: [{ kind: "raw", messages: [{ text }] }],
});
});
@@ -378,7 +441,29 @@ describe("unified context search", () => {
status: 200,
data: {
...RESPONSE,
items: [{ ...RESPONSE.items[0], content: text }],
paths: {
...RESPONSE.paths,
memory: {
...RESPONSE.paths.memory,
status: "empty",
returned_count: 0,
},
},
items: [
{
...(RESPONSE.items[0] as Record<string, unknown>),
messages: [
{
...(
RESPONSE.items[0] as {
messages: Array<Record<string, unknown>>;
}
).messages[0],
text,
},
],
},
],
used_characters: 12,
},
});
@@ -394,40 +479,6 @@ describe("unified context search", () => {
});
});
it.each([
[
"a raw item from a source that was not asked for",
["--source", "codex"],
{ ...RESPONSE.items[0], source: "cursor" },
],
[
"a raw item when only memory was asked for",
["--kind", "memory"],
RESPONSE.items[0],
],
["a score outside 0..1", [], { ...RESPONSE.items[1], score: 1.5 }],
])("rejects %s as an invalid response", async (_label, flags, item) => {
transport.post.mockResolvedValue({
status: 200,
data: {
...RESPONSE,
items: [item],
used_characters: Array.from(String(item.content)).length,
},
});
const streams = captureStreams();
try {
expect(await run(["recall", "query", ...flags, "--json"])).toBe(1);
} finally {
streams.restore();
}
expect(JSON.parse(streams.stdout())).toMatchObject({
schema_version: 2,
state: "invalid",
code: "response_invalid",
});
});
it("reports endpoint denial without exposing the response body", async () => {
transport.post.mockResolvedValue({
status: 403,
+164 -55
View File
@@ -3,6 +3,31 @@ import { PRODUCT } from "./version.js";
export const CONTEXT_QUERY_MAX_CHARACTERS = 2_000;
const CONTEXT_KINDS = ["raw", "memory"] as const;
const PATH_STATES = [
"ok",
"empty",
"not_requested",
"not_authorized",
"unavailable",
"degraded",
] as const;
const PATH_REASONS = [
"not_requested",
"permission_required",
"source_revoked",
"read_unavailable",
"timeout_budget",
"evidence_changed",
"memory_changed",
"lexical_only",
] as const;
const LIMIT_REASONS = [
"count_budget",
"character_budget",
"timeout_budget",
"metadata_budget",
] as const;
export interface ContextSearchInput {
query: string;
kinds: Array<(typeof CONTEXT_KINDS)[number]>;
@@ -12,9 +37,8 @@ export interface ContextSearchInput {
timeout_ms: number;
}
/** The CLI's JSON document mirrors Core's SDK-native context schema v2. */
export interface ContextSearchDocument {
schema_version: 2;
schema_version: 1;
product: string;
state:
| "grounded"
@@ -23,14 +47,15 @@ export interface ContextSearchDocument {
| "unavailable"
| "denied"
| "not_found"
| "rejected"
| "invalid";
code?: string;
request_id?: string;
paths?: Record<string, unknown>;
items: Array<Record<string, unknown>>;
used_characters?: number;
truncated?: boolean;
total_ms?: number;
adoption?: "unknown";
}
export async function searchContext(
@@ -42,7 +67,7 @@ export async function searchContext(
const validated = validateContextResponse(outcome.data, input);
return (
validated ?? {
schema_version: 2,
schema_version: 1,
product: PRODUCT,
state: "invalid",
code: "response_invalid",
@@ -98,9 +123,6 @@ export function renderContextText(document: ContextSearchDocument): string {
if (document.state === "unavailable") return "暂时无法查记忆,请稍后重试。\n";
if (document.state === "denied") return "当前应用没有查记忆权限。\n";
if (document.state === "invalid") return "查记忆返回了无法识别的结果。\n";
if (document.state === "rejected") {
return `查记忆被拒绝(${document.code}),请检查参数。\n`;
}
const lines = [
document.state === "empty"
? "没有查到相关记忆。"
@@ -108,14 +130,14 @@ export function renderContextText(document: ContextSearchDocument): string {
? "查到以下内容,部分内容还在整理中:"
: "查到以下相关内容:",
];
// A raw hit is an excerpt of an original conversation, not a distilled
// memory; say which one each line is so it is not read as a conclusion.
for (const item of document.items) {
const label =
item.kind === "raw"
? `[原始对话摘录 · ${String(item.source)}]`
: "[记忆]";
lines.push(`- ${label} ${String(item.content)}`);
if (item.kind === "memory") {
lines.push(`- ${String(item.content)}`);
} else {
for (const message of item.messages as Array<Record<string, unknown>>) {
lines.push(`- ${String(message.text)}`);
}
}
}
if (document.truncated) lines.push("结果已按请求预算截断。");
return `${lines.join("\n")}\n`;
@@ -127,37 +149,84 @@ function validateContextResponse(
): ContextSearchDocument | null {
const response = record(value);
if (
response?.schema_version !== 2 ||
response?.schema_version !== 1 ||
!oneOf(response.state, ["grounded", "empty", "partial", "unavailable"]) ||
!boundedText(response.request_id, 64) ||
!record(response.paths) ||
!Array.isArray(response.items) ||
response.items.length > request.limit ||
!integer(response.used_characters, 0, request.character_budget) ||
typeof response.truncated !== "boolean" ||
!number(response.total_ms, 0)
!number(response.total_ms, 0) ||
response.adoption !== "unknown"
) {
return null;
}
const paths = response.paths as Record<string, unknown>;
const rawPath = validatePath(paths.raw);
const memoryPath = validatePath(paths.memory);
if (!rawPath || !memoryPath) return null;
const requested = new Set(request.kinds);
if (
(!requested.has("raw") && rawPath.status !== "not_requested") ||
(!requested.has("memory") && memoryPath.status !== "not_requested")
) {
return null;
}
const items: Array<Record<string, unknown>> = [];
let characters = 0;
for (const itemValue of response.items) {
const item = validateItem(itemValue, request);
if (!item || !requested.has(item.kind as "raw" | "memory")) return null;
items.push(item);
characters += characterCount(item.content);
characters +=
item.kind === "memory"
? characterCount(item.content)
: (item.messages as Array<Record<string, unknown>>).reduce(
(sum, message) => sum + characterCount(message.text),
0,
);
}
if (characters !== response.used_characters) return null;
if (response.state === "empty" && items.length !== 0) return null;
return {
schema_version: 2,
schema_version: 1,
product: PRODUCT,
state: response.state,
request_id: response.request_id as string,
paths: { raw: rawPath, memory: memoryPath },
items,
used_characters: response.used_characters as number,
truncated: response.truncated,
total_ms: response.total_ms as number,
adoption: "unknown",
};
}
function validatePath(value: unknown): Record<string, unknown> | null {
const path = record(value);
if (
!path ||
!oneOf(path.status, PATH_STATES) ||
!number(path.elapsed_ms, 0) ||
!integer(path.returned_count, 0, 20) ||
typeof path.truncated !== "boolean" ||
!enumList(path.truncated_by, LIMIT_REASONS) ||
!(
path.reason === null ||
path.reason === undefined ||
oneOf(path.reason, PATH_REASONS)
)
) {
return null;
}
return {
status: path.status,
reason: path.reason ?? null,
elapsed_ms: path.elapsed_ms,
returned_count: path.returned_count,
truncated: path.truncated,
truncated_by: path.truncated_by,
};
}
@@ -166,57 +235,80 @@ function validateItem(
request: ContextSearchInput,
): Record<string, unknown> | null {
const item = record(value);
if (
!item ||
!oneOf(item.kind, CONTEXT_KINDS) ||
boundedText(item.content, 20_000, true) === null ||
!number(item.score, 0) ||
(item.score as number) > 1 ||
typeof item.truncated !== "boolean"
) {
return null;
}
if (!item || !oneOf(item.kind, CONTEXT_KINDS)) return null;
if (item.kind === "memory") {
if (
!boundedText(item.memory_id, 512) ||
boundedText(item.layer, 64, true) === null ||
boundedText(item.status, 32, true) === null ||
!optionalText(item.source, 128) ||
!optionalText(item.updated_at, 64)
!integer(item.revision, 1) ||
boundedText(item.content, 20_000, true) === null ||
boundedText(item.memory_kind, 64, true) === null ||
!boundedText(item.updated_at, 64) ||
!textList(item.evidence_ids, 128, 512) ||
typeof item.truncated !== "boolean"
) {
return null;
}
return {
kind: "memory",
memory_id: item.memory_id,
revision: item.revision,
content: item.content,
layer: item.layer,
status: item.status,
source: item.source ?? null,
updated_at: item.updated_at ?? null,
score: item.score,
memory_kind: item.memory_kind,
updated_at: item.updated_at,
evidence_ids: item.evidence_ids,
truncated: item.truncated,
};
}
if (
!boundedText(item.raw_id, 512) ||
!boundedText(item.source, 128) ||
!boundedText(item.source_id, 128) ||
(request.raw_source_ids !== undefined &&
!request.raw_source_ids.includes(item.source as string)) ||
!boundedText(item.session_id, 512) ||
!optionalText(item.memory_at, 64)
!request.raw_source_ids.includes(item.source_id as string)) ||
!boundedText(item.conversation_id, 512) ||
!boundedText(item.anchor_evidence_id, 512) ||
!Array.isArray(item.messages) ||
item.messages.length < 1 ||
item.messages.length > 3 ||
typeof item.truncated !== "boolean"
) {
return null;
}
const messages = item.messages.map(validateMessage);
if (messages.some((message) => message === null)) return null;
return {
kind: "raw",
source_id: item.source_id,
conversation_id: item.conversation_id,
anchor_evidence_id: item.anchor_evidence_id,
messages,
truncated: item.truncated,
};
}
function validateMessage(value: unknown): Record<string, unknown> | null {
const message = record(value);
if (
!message ||
!boundedText(message.evidence_id, 512) ||
!integer(message.revision, 1) ||
boundedText(message.role, 32, true) === null ||
boundedText(message.written_by, 32, true) === null ||
!(
message.created_at === null ||
boundedText(message.created_at, 64) !== null
) ||
boundedText(message.text, 20_000, true) === null ||
typeof message.truncated !== "boolean"
) {
return null;
}
return {
kind: "raw",
raw_id: item.raw_id,
content: item.content,
source: item.source,
session_id: item.session_id,
memory_at: item.memory_at ?? null,
score: item.score,
truncated: item.truncated,
evidence_id: message.evidence_id,
revision: message.revision,
role: message.role,
written_by: message.written_by,
created_at: message.created_at,
text: message.text,
truncated: message.truncated,
};
}
@@ -224,7 +316,7 @@ function failed(
outcome: Exclude<ReadOutcome<unknown>, { state: "ok" }>,
): ContextSearchDocument {
return {
schema_version: 2,
schema_version: 1,
product: PRODUCT,
state: outcome.state,
code: outcome.code,
@@ -282,10 +374,27 @@ function oneOf<const T extends readonly string[]>(
return typeof value === "string" && values.includes(value as T[number]);
}
function optionalText(value: unknown, maximum: number): boolean {
function enumList(
value: unknown,
values: readonly string[],
): value is string[] {
return (
value === null ||
value === undefined ||
boundedText(value, maximum) !== null
Array.isArray(value) &&
value.length <= 4 &&
new Set(value).size === value.length &&
value.every((item) => oneOf(item, values))
);
}
function textList(
value: unknown,
maximumItems: number,
maximumText: number,
): value is string[] {
return (
Array.isArray(value) &&
value.length <= maximumItems &&
new Set(value).size === value.length &&
value.every((item) => boundedText(item, maximumText) !== null)
);
}
+87 -129
View File
@@ -8,19 +8,18 @@ import axios from "axios";
*/
const READ_ONLY_PATHS = [
"/api/v1/call-events",
"/api/v1/home-summary",
"/api/v1/import-jobs",
"/api/v1/live-sync",
"/api/v1/memory-library",
"/api/v1/memory-processing",
"/api/v1/memory-tasks",
"/api/v1/model-profiles",
"/api/v1/sources",
"/api/v2/home-summary",
"/api/v2/memory-library",
"/api/v2/user-profile",
] as const;
const SDK_MEMORY_DETAIL = /^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}$/;
/** Status of one `POST /api/v2/memories` addition, by its request id. */
const MEMORY_ADDITION =
/^\/api\/v2\/memories\/additions\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
/** Contract (C0, H25): the one-time loopback code exchange for `memhub ui`.
* It is a POST because it mints a short-lived code; the CLI sends the bearer
@@ -68,10 +67,13 @@ export function createUiSessionRequest(attachment: {
};
}
const MEMORY_DETAIL = /^\/api\/v1\/memories\/[A-Za-z0-9][\w.-]{0,127}$/;
const IMPORT_JOB =
/^\/api\/v1\/import-jobs\/[A-Za-z0-9][\w.-]{0,127}(?:\/issues)?$/;
const OPERATION_RECEIPT =
/^\/api\/v1\/(?:source-connections|model-profile-creations|model-probe-operations)\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const MEMORY_MUTATION_RECEIPT =
/^\/api\/v1\/memory-mutations\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
/**
* Core registers these static routes before `/{memory_id}`, so a memory whose
* id happens to be one of them would reach a different handler: `graph`
@@ -82,15 +84,9 @@ export const RESERVED_MEMORY_SEGMENTS = ["ask", "graph"] as const;
/** A response larger than this is refused before it is buffered. */
const MAX_RESPONSE_BYTES = 4 * 1024 * 1024;
/** `rejected` is Core refusing the request itself (400/409/422: a bad filter,
* an ambiguous id). It is not "Core is busy" — only a transport failure or a
* 5xx is `unavailable`. */
export type ReadOutcome<T> =
| { state: "ok"; data: T }
| {
state: "denied" | "unavailable" | "not_found" | "rejected";
code: string;
};
| { state: "denied" | "unavailable" | "not_found"; code: string };
export type CoreReader = <T>(
path: string,
@@ -149,27 +145,10 @@ export function createCoreReader(attachment: {
if (response.status === 404) {
return { state: "not_found" as const, code: "http_404" };
}
if ([400, 409, 422].includes(response.status)) {
return {
state: "rejected" as const,
code: detailCode(response.data, response.status),
};
}
return { state: "unavailable" as const, code: `http_${response.status}` };
};
}
/** Core's stable `detail` code when it sent one, else the bare status. */
function detailCode(value: unknown, status: number): string {
const detail =
value && typeof value === "object" && !Array.isArray(value)
? (value as { detail?: unknown }).detail
: undefined;
return typeof detail === "string" && /^[a-z0-9_.:-]{1,128}$/.test(detail)
? detail
: `http_${status}`;
}
/**
* Context search is semantically read-only but intentionally uses POST so the
* private query and filters are carried in a bounded body, never in a URL.
@@ -194,7 +173,7 @@ export function createCoreContextQuery(attachment: {
let response: { status: number; data: unknown };
try {
response = await client.post(
"/api/v2/context/search",
"/api/v1/context/search",
body,
options?.signal ? { signal: options.signal } : undefined,
);
@@ -218,18 +197,17 @@ export function isReadOnlyPath(path: string): boolean {
if ((READ_ONLY_PATHS as readonly string[]).includes(path)) return true;
if (IMPORT_JOB.test(path)) return true;
if (OPERATION_RECEIPT.test(path)) return true;
if (MEMORY_ADDITION.test(path)) return true;
if (!SDK_MEMORY_DETAIL.test(path)) return false;
const segment = path.slice("/api/v2/memories/".length);
if (MEMORY_MUTATION_RECEIPT.test(path)) return true;
if (SDK_MEMORY_DETAIL.test(path)) return true;
if (!MEMORY_DETAIL.test(path)) return false;
const segment = path.slice("/api/v1/memories/".length);
return !(RESERVED_MEMORY_SEGMENTS as readonly string[]).includes(segment);
}
/** A write accepted by the unified Memory Server entry.
*
* `request_id` is the idempotency key: Core keys the write on it, so a retry
* with the same id and the same messages is answered from the first write, and
* the same id with different messages is refused (409). A retry without the
* original id is a new write.
* `request_id` is what the caller keeps and queries later. Retrying is unsafe:
* HY has no idempotency key, so a resubmit duplicates the memory.
*/
export interface MemoryWriteReceipt {
request_id: string;
@@ -296,11 +274,6 @@ function baseUrlOf(attachment: RuntimeAttachmentLike): string {
return `http://${attachment.host}:${attachment.port}`;
}
export type MemoryWriteOutcome =
| { state: "accepted"; receipt: MemoryWriteReceipt }
| { state: "rejected"; code: string; http_status: number }
| { state: "unknown"; code: string };
export async function submitMemoryWrite(
attachment: RuntimeAttachmentLike,
request: {
@@ -311,48 +284,28 @@ export async function submitMemoryWrite(
memory_at?: string;
priority?: "live" | "normal";
},
): Promise<MemoryWriteOutcome> {
let response: { status: number; data: unknown };
try {
response = await axios.post<MemoryWriteReceipt>(
`${baseUrlOf(attachment)}/api/v2/memory-writes`,
{
request_id: request.request_id,
session_id: request.session,
source_id: request.source,
messages: [...request.messages],
...(request.memory_at === undefined
? {}
: { memory_at: request.memory_at }),
...(request.priority === undefined
? {}
: { priority: request.priority }),
},
{
headers: { Authorization: `Bearer ${attachment.apiToken}` },
timeout: 15_000,
proxy: false,
maxRedirects: 0,
maxContentLength: MAX_RESPONSE_BYTES,
validateStatus: () => true,
},
);
} catch {
// The request may or may not have reached Core; the same request id makes
// a retry safe.
return { state: "unknown", code: "transport_result_unknown" };
}
if (response.status >= 200 && response.status < 300) {
return { state: "accepted", receipt: response.data as MemoryWriteReceipt };
}
if (response.status >= 400 && response.status < 500) {
return {
state: "rejected",
code: detailCode(response.data, response.status),
http_status: response.status,
};
}
return { state: "unknown", code: `http_${response.status}_result_unknown` };
): Promise<MemoryWriteReceipt> {
const response = await axios.post<MemoryWriteReceipt>(
`${baseUrlOf(attachment)}/api/v2/memory-writes`,
{
request_id: request.request_id,
session_id: request.session,
source_id: request.source,
messages: [...request.messages],
...(request.memory_at === undefined
? {}
: { memory_at: request.memory_at }),
...(request.priority === undefined ? {} : { priority: request.priority }),
},
{
headers: { Authorization: `Bearer ${attachment.apiToken}` },
timeout: 15_000,
proxy: false,
maxRedirects: 0,
maxContentLength: MAX_RESPONSE_BYTES,
},
);
return response.data;
}
/** Ask Runtime to scan one source now, returning the import job it started.
@@ -549,52 +502,48 @@ export async function readScanRoots(
}
}
/** A direct Core write whose refusal carries Core's stable code. */
export type CoreWriteResult =
| { ok: true; data: unknown }
| { ok: false; code: string };
async function coreWrite(
attachment: RuntimeAttachmentLike,
method: "post" | "delete",
pathname: string,
body: Record<string, unknown>,
timeout = 10_000,
): Promise<CoreWriteResult> {
let response: { status: number; data: unknown };
try {
response = await axios.request({
method,
url: `${baseUrlOf(attachment)}${pathname}`,
data: body,
headers: { Authorization: `Bearer ${attachment.apiToken}` },
timeout,
proxy: false,
maxRedirects: 0,
maxContentLength: MAX_RESPONSE_BYTES,
validateStatus: () => true,
});
} catch {
return { ok: false, code: "transport_failed" };
}
if (response.status >= 200 && response.status < 300) {
return { ok: true, data: response.data };
}
return { ok: false, code: detailCode(response.data, response.status) };
}
export async function addScanRoot(
attachment: RuntimeAttachmentLike,
body: { root: string; source?: string; recursive?: boolean },
): Promise<CoreWriteResult> {
return coreWrite(attachment, "post", "/api/v1/scan-roots", body);
): Promise<unknown | null> {
try {
const response = await axios.post<unknown>(
`${baseUrlOf(attachment)}/api/v1/scan-roots`,
body,
{
headers: { Authorization: `Bearer ${attachment.apiToken}` },
timeout: 10_000,
proxy: false,
maxRedirects: 0,
maxContentLength: MAX_RESPONSE_BYTES,
},
);
return response.data;
} catch (error) {
return { _error: error instanceof Error ? error.message : String(error) };
}
}
export async function removeScanRoot(
attachment: RuntimeAttachmentLike,
body: { root: string },
): Promise<CoreWriteResult> {
return coreWrite(attachment, "delete", "/api/v1/scan-roots", body);
): Promise<unknown | null> {
try {
const response = await axios.delete<unknown>(
`${baseUrlOf(attachment)}/api/v1/scan-roots`,
{
headers: { Authorization: `Bearer ${attachment.apiToken}` },
data: body,
timeout: 10_000,
proxy: false,
maxRedirects: 0,
maxContentLength: MAX_RESPONSE_BYTES,
},
);
return response.data;
} catch (error) {
return { _error: error instanceof Error ? error.message : String(error) };
}
}
export async function readAdapters(
@@ -620,12 +569,21 @@ export async function readAdapters(
export async function probeAdapter(
attachment: RuntimeAttachmentLike,
source: string,
): Promise<CoreWriteResult> {
return coreWrite(
attachment,
"post",
"/api/v1/adapters/probe",
{ source },
15_000,
);
): Promise<unknown | null> {
try {
const response = await axios.post<unknown>(
`${baseUrlOf(attachment)}/api/v1/adapters/probe`,
{ source },
{
headers: { Authorization: `Bearer ${attachment.apiToken}` },
timeout: 15_000,
proxy: false,
maxRedirects: 0,
maxContentLength: MAX_RESPONSE_BYTES,
},
);
return response.data;
} catch (error) {
return { _error: error instanceof Error ? error.message : String(error) };
}
}
+94 -9
View File
@@ -4,11 +4,9 @@ const WRITE_PATHS = [
"/api/v1/sources/connect",
"/api/v1/model-profiles",
] as const;
/** `POST /api/v2/memories` adds one memory (the app's 添加记忆). */
const MEMORY_ADD_PATH = "/api/v2/memories";
/** One SDK memory; delete names its namespace in the query string. */
const SDK_MEMORY_PATH =
/^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}(?:\?kind=(?:chat|experience|fact)(?:&variant=(?:working|coding))?)?$/;
const MEMORY_WRITE_PATH =
/^\/api\/v1\/memories(?:\/[A-Za-z0-9][\w.-]{0,127}\/(?:correct|scopes|actions\/delete))?$/;
const SDK_MEMORY_PATH = /^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}$/;
const MODEL_PROBE_PATH =
/^\/api\/v1\/model-profiles\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\/probe$/;
const MODEL_PROFILE_PATH =
@@ -24,7 +22,18 @@ const SOURCE_REVOKE_PATH =
const SOURCE_IMPORT_JOB_PATH = "/api/v1/import-jobs";
const IMPORT_JOB_ACTION_PATH =
/^\/api\/v1\/import-jobs\/[A-Za-z0-9][\w.-]{0,127}\/(?:pause|resume|cancel)$/;
const CALL_EVENTS_CLEAR_PATH = "/api/v1/call-events/clear";
const CALL_EVENTS_CLEAR_ALL_PATH = "/api/v1/call-events/clear-all";
const IMPORT_FILE_COMMIT =
/^\/api\/v1\/import-files\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\/commit$/;
const IMPORT_FILE =
/^\/api\/v1\/import-files\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const MEMORY_NOTES_PATH = "/api/v1/memory-notes";
const MEMORY_NOTE_PROCESS =
/^\/api\/v1\/memory-notes\/[A-Za-z0-9][\w.:-]{0,127}\/process$/;
const EXTRACTION_SETTINGS_PATH = "/api/v1/extraction-settings";
const GRAPH_WIDTH_PATH = "/api/v1/graph-width";
const MANUAL_EXTRACT_PATH = "/api/v1/memory-processing/manual-extract";
const ONBOARDING_COMPLETE_PATH = "/api/v1/onboarding/complete";
const MAX_BYTES = 4 * 1024 * 1024;
@@ -61,6 +70,18 @@ export interface CorePatchOperations {
): Promise<OperationOutcome<T>>;
}
export interface CoreUploadOperations {
uploadImportFile<T>(
input: {
bytes: Uint8Array;
filename: string;
inputKind: "raw_conversation" | "memory_content";
operationId: string;
},
signal?: AbortSignal,
): Promise<OperationOutcome<T>>;
}
export function createCoreOperations(attachment: {
host: string;
port: number;
@@ -69,7 +90,8 @@ export function createCoreOperations(attachment: {
}): CoreOperations &
CoreDeleteOperations &
CorePutOperations &
CorePatchOperations {
CorePatchOperations &
CoreUploadOperations {
const client = axios.create({
baseURL: `http://${attachment.host}:${attachment.port}`,
headers: { Authorization: `Bearer ${attachment.apiToken}` },
@@ -80,6 +102,18 @@ export function createCoreOperations(attachment: {
maxBodyLength: 128 * 1024,
validateStatus: () => true,
});
// Uploads carry up to 128 MiB of raw octet-stream, so they need a separate
// body limit and a deadline that leaves room for parsing on the Core side.
const uploadClient = axios.create({
baseURL: `http://${attachment.host}:${attachment.port}`,
headers: { Authorization: `Bearer ${attachment.apiToken}` },
timeout: 120_000,
proxy: false,
maxRedirects: 0,
maxContentLength: MAX_BYTES,
maxBodyLength: 128 * 1024 * 1024,
validateStatus: () => true,
});
return {
async post<T>(
path: string,
@@ -89,13 +123,19 @@ export function createCoreOperations(attachment: {
if (
!(WRITE_PATHS as readonly string[]).includes(path) &&
path !== SOURCE_IMPORT_JOB_PATH &&
path !== MEMORY_ADD_PATH &&
!MEMORY_WRITE_PATH.test(path) &&
!MODEL_PROBE_PATH.test(path) &&
!SOURCE_SYNC_PATH.test(path) &&
!IMPORT_JOB_ACTION_PATH.test(path) &&
!SOURCE_AUTHORIZE_PATH.test(path) &&
!SOURCE_REVOKE_PATH.test(path) &&
path !== CALL_EVENTS_CLEAR_PATH &&
path !== CALL_EVENTS_CLEAR_ALL_PATH &&
!IMPORT_FILE_COMMIT.test(path) &&
path !== MEMORY_NOTES_PATH &&
!MEMORY_NOTE_PROCESS.test(path) &&
path !== GRAPH_WIDTH_PATH &&
path !== MANUAL_EXTRACT_PATH &&
path !== ONBOARDING_COMPLETE_PATH &&
!MODEL_PROFILE_ACTIVATE.test(path)
) {
@@ -131,7 +171,7 @@ export function createCoreOperations(attachment: {
body: Record<string, unknown>,
signal?: AbortSignal,
) {
if (!SDK_MEMORY_PATH.test(path) || path.includes("?")) {
if (path !== EXTRACTION_SETTINGS_PATH && !SDK_MEMORY_PATH.test(path)) {
throw new Error("core_operation_path_not_allowed");
}
let response: { status: number; data: unknown };
@@ -193,7 +233,7 @@ export function createCoreOperations(attachment: {
};
},
async delete<T>(path: string, signal?: AbortSignal) {
if (!SDK_MEMORY_PATH.test(path) && !MODEL_PROFILE_PATH.test(path)) {
if (!IMPORT_FILE.test(path) && !SDK_MEMORY_PATH.test(path)) {
throw new Error("core_operation_path_not_allowed");
}
let response: { status: number; data: unknown };
@@ -217,6 +257,51 @@ export function createCoreOperations(attachment: {
code: `http_${response.status}_result_unknown`,
};
},
async uploadImportFile<T>(
input: {
bytes: Uint8Array;
filename: string;
inputKind: "raw_conversation" | "memory_content";
operationId: string;
},
signal?: AbortSignal,
) {
const query = new URLSearchParams({
filename: input.filename,
input_kind: input.inputKind,
operation_id: input.operationId,
});
let response: { status: number; data: unknown };
try {
response = await uploadClient.post(
`/api/v1/import-files?${query.toString()}`,
input.bytes,
{
headers: {
"Content-Type": "application/octet-stream",
"Content-Length": String(input.bytes.byteLength),
},
...(signal ? { signal } : {}),
},
);
} catch {
return { state: "unknown", code: "transport_result_unknown" };
}
if (response.status >= 200 && response.status < 300) {
return { state: "committed", data: response.data as T };
}
if (response.status >= 400 && response.status < 500) {
return {
state: "rejected",
code: safeErrorCode(response.data, response.status),
http_status: response.status,
};
}
return {
state: "unknown",
code: `http_${response.status}_result_unknown`,
};
},
};
}
+212 -24
View File
@@ -1,20 +1,18 @@
import type {
GovernedMemory,
HomeSummary,
ImportIssue,
ImportJob,
LiveSyncHealth,
MemoryDetail,
MemoryEvidenceExcerpt,
MemoryLibraryPage,
MemoryProcessingSummary,
ModelProfile,
SourceResponse,
} from "@one-memory/contracts";
/** SDK memory states `memhub memories list --status` may ask for. */
export const MEMORY_STATUSES = [
"active",
"superseded",
"shadow",
"negated",
"conflicted",
"archived",
] as const;
/** The only memory states a response may carry, and the only ones a filter may ask for. */
export const MEMORY_STATUSES = ["confirmed", "deleted", "superseded"] as const;
/**
* HTTP 200 is not a contract. Everything here re-checks the shape and the
@@ -25,6 +23,7 @@ export const MEMORY_STATUSES = [
const ID_MAX = 128;
const LABEL_MAX = 256;
const TEXT_MAX = 100_000;
const LIST_MAX = 500;
const LIVE_SYNC_STATES = [
"idle",
@@ -40,6 +39,14 @@ const PROVIDER_KINDS = [
"local_onnx",
] as const;
const CREDENTIAL_STATES = ["available", "missing", "unavailable"] as const;
const MEMORY_KINDS = [
"chat",
"working",
"coding",
"schema",
"intention",
] as const;
const MEMORY_ORIGINS = ["extracted", "answered", "agent", "manual"] as const;
const IMPORT_STATUSES = [
"pending",
"running",
@@ -50,6 +57,16 @@ const IMPORT_STATUSES = [
"cancelled",
] as const;
const SCAN_PHASES = ["discover", "read", "emit", "done"] as const;
const MEMORY_TASK_KINDS = ["extract", "system2", "ultra", "sweeper"] as const;
const MEMORY_TASK_STATUSES = [
"pending",
"running",
"waiting_for_model",
"completed",
"failed",
"cancelled",
"deferred_outside_window",
] as const;
const CAPABILITY_STATES = ["not_implemented", "implemented"] as const;
const VERIFICATION_STATES = ["fixture_only", "real_replayed"] as const;
const SYNC_COMPLETENESS = [
@@ -70,6 +87,16 @@ const LOCATION_STATES = [
] as const;
const LIVE_SYNC_MODES = ["polling", "watch_and_polling"] as const;
export interface ValidatedMemoryTask {
task_id: string;
kind: (typeof MEMORY_TASK_KINDS)[number];
status: (typeof MEMORY_TASK_STATUSES)[number];
attempts: number;
last_error_code: string | null;
created_at: string;
updated_at: string;
}
export interface SourceConnectionCandidate {
source_id: string;
display_name: string;
@@ -325,6 +352,45 @@ export function validateImportIssues(
});
}
export function validateMemoryTasks(
value: unknown,
): ValidatedMemoryTask[] | null {
return list(value, (item) => {
const task = record(item);
if (!task) return null;
const taskId = text(task.task_id, ID_MAX);
if (
task.last_error_code !== null &&
text(task.last_error_code, LABEL_MAX) === null
) {
return null;
}
const error = task.last_error_code as string | null;
const createdAt = text(task.created_at, LABEL_MAX);
const updatedAt = text(task.updated_at, LABEL_MAX);
const attempts = count(task.attempts);
if (
taskId === null ||
createdAt === null ||
updatedAt === null ||
attempts === null ||
!oneOf(task.kind, MEMORY_TASK_KINDS) ||
!oneOf(task.status, MEMORY_TASK_STATUSES)
) {
return null;
}
return {
task_id: taskId,
kind: task.kind,
status: task.status,
attempts,
last_error_code: error,
created_at: createdAt,
updated_at: updatedAt,
};
});
}
export function validateProcessing(
value: unknown,
): MemoryProcessingSummary | null {
@@ -378,32 +444,150 @@ export function validateModelProfiles(value: unknown): ModelProfile[] | null {
});
}
/** `GET /api/v2/home-summary`: the SDK-authority home card. */
export interface SdkHomeSummary {
schema_version: 2;
memory_count: number;
latest: { memory_id: string } | null;
observed_at: string;
}
export function validateHomeSummary(value: unknown): SdkHomeSummary | null {
export function validateHomeSummary(value: unknown): HomeSummary | null {
const summary = record(value);
if (
!summary ||
summary.schema_version !== 2 ||
count(summary.memory_count) === null ||
text(summary.observed_at, LABEL_MAX) === null
) {
return null;
}
const latest = summary.latest;
if (latest !== null) {
const memory = record(latest);
if (!memory || typeof memory.memory_id !== "string" || !memory.memory_id) {
return summary as unknown as HomeSummary;
}
/** The page must be the page that was asked for, not merely a page. */
export function validateMemoryPage(
value: unknown,
request: { offset: number; limit: number },
): MemoryLibraryPage | null {
const page = record(value);
if (!page) return null;
const total = count(page.total);
if (
total === null ||
page.offset !== request.offset ||
page.limit !== request.limit
) {
return null;
}
const items = list(page.items, validateGovernedMemory);
if (items === null || items.length > request.limit) return null;
if (items.length > 0 && total < request.offset + items.length) return null;
return { items, total, offset: page.offset, limit: page.limit };
}
export function validateMemoryDetail(
value: unknown,
memoryId: string,
): MemoryDetail | null {
const detail = record(value);
if (!detail) return null;
const memory = validateGovernedMemory(detail.memory);
if (memory === null || memory.memory_id !== memoryId) return null;
const cited = new Set(memory.evidence_ids);
const returned = new Set<string>();
const evidence = list(detail.evidence ?? [], (item) => {
const excerpt = record(item);
if (!excerpt) return null;
const evidenceId = text(excerpt.evidence_id, ID_MAX);
// Core builds this list from the memory's own citations; anything else is
// another record's excerpt and must not be shown under this memory.
if (
evidenceId === null ||
!cited.has(evidenceId) ||
returned.has(evidenceId) ||
typeof excerpt.available !== "boolean"
) {
return null;
}
returned.add(evidenceId);
const available = excerpt.available === true;
const verbatim = available ? text(excerpt.verbatim ?? "", TEXT_MAX) : "";
if (verbatim === null) return null;
return {
evidence_id: evidenceId,
available,
source_id: optional(excerpt.source_id, ID_MAX),
written_by: optional(excerpt.written_by, LABEL_MAX),
created_at: optional(excerpt.created_at, LABEL_MAX),
// An excerpt the user cannot read has no text, whatever the server sent.
verbatim,
} as unknown as MemoryEvidenceExcerpt;
});
if (evidence === null || evidence.length !== memory.evidence_ids.length) {
return null;
}
return summary as unknown as SdkHomeSummary;
const history = list(detail.history ?? [], (item) => {
const version = record(item);
if (
!version ||
typeof version.revision !== "number" ||
!Number.isSafeInteger(version.revision) ||
version.revision < 1 ||
text(version.action, LABEL_MAX) === null ||
!oneOf(version.status, MEMORY_STATUSES) ||
text(version.content, TEXT_MAX) === null ||
identifiers(version.scopes) === null ||
text(version.created_at, LABEL_MAX) === null
) {
return null;
}
return {
revision: version.revision,
action: version.action,
status: version.status,
content: version.content,
scopes: identifiers(version.scopes) as string[],
created_at: version.created_at,
};
});
if (history === null) return null;
return {
memory,
evidence,
history,
index_sync: null,
} as unknown as MemoryDetail;
}
export function validateGovernedMemory(value: unknown): GovernedMemory | null {
const memory = record(value);
if (
!memory ||
!text(memory.memory_id, ID_MAX) ||
!text(memory.user_id, LABEL_MAX) ||
!text(memory.content, TEXT_MAX) ||
!oneOf(memory.kind, MEMORY_KINDS) ||
!oneOf(memory.origin, MEMORY_ORIGINS) ||
typeof memory.revision !== "number" ||
!Number.isSafeInteger(memory.revision) ||
memory.revision < 1 ||
!oneOf(memory.status, MEMORY_STATUSES) ||
!text(memory.created_at, LABEL_MAX) ||
!text(memory.updated_at, LABEL_MAX)
) {
return null;
}
const scopes = identifiers(memory.scopes);
const evidenceIds = identifiers(memory.evidence_ids);
const tags = identifiers(memory.tags ?? []);
if (
scopes === null ||
scopes.length < 1 ||
evidenceIds === null ||
evidenceIds.length < 1 ||
tags === null ||
tags.length > 32
) {
return null;
}
return {
...memory,
scopes,
evidence_ids: evidenceIds,
tags,
} as unknown as GovernedMemory;
}
function identifiers(value: unknown): string[] | null {
@@ -434,6 +618,10 @@ function text(value: unknown, max: number): string | null {
return typeof value === "string" && value.length <= max ? value : null;
}
function optional(value: unknown, max: number): string | undefined {
return typeof value === "string" && value.length <= max ? value : undefined;
}
function count(value: unknown): number | null {
return typeof value === "number" && Number.isSafeInteger(value) && value >= 0
? value
@@ -134,6 +134,11 @@ function progress(): ProgressDocument {
generation: {
summary_state: { state: "ok" },
summary: { waiting_for_model: 1, failed: 0 },
tasks_state: { state: "ok" },
tasks: [{ task_id: "t1", status: "waiting_for_model" }],
returned: 1,
validated: 1,
output_truncated: false,
},
};
}
+12 -21
View File
@@ -65,9 +65,6 @@ export function renderDoctorText(document: DoctorDocument): string {
model: "模型",
pending_operations: "待处理操作",
};
const migration = document.checks.find(
(check) => check.id === "core" && check.details?.log_file,
);
return [
`MemHub 检查:${doctorState(document.state)}`,
...document.checks.map(
@@ -76,13 +73,7 @@ export function renderDoctorText(document: DoctorDocument): string {
),
...(document.state === "healthy"
? []
: migration
? [
`升级迁移没有完成(错误码:${migration.code}),MemHub 不会启动。`,
`下一步:把迁移日志附在 GitHub 反馈里发给我们:${migration.details?.log_file}`,
`反馈地址:${migration.details?.issues_url}`,
]
: ["下一步:打开 MemHub 的“帮助与诊断”查看并处理异常。"]),
: ["下一步:打开 MemHub 的“帮助与诊断”查看并处理异常。"]),
"",
].join("\n");
}
@@ -116,14 +107,6 @@ function coreCheck(status: StatusDocument): DoctorCheck {
id: "core",
state: status.runtime.state === "stopped" ? "warn" : "fail",
code: status.runtime.code,
...(status.runtime.log_file
? {
details: {
log_file: status.runtime.log_file,
issues_url: status.runtime.issues_url ?? "",
},
}
: {}),
};
}
@@ -217,7 +200,8 @@ function modelCheck(status: StatusDocument): DoctorCheck {
function pendingCheck(progress: ProgressDocument | null): DoctorCheck {
if (
progress?.imports.state !== "ok" ||
progress.generation.summary_state.state !== "ok"
progress.generation.summary_state.state !== "ok" ||
progress.generation.tasks_state.state !== "ok"
) {
return {
id: "pending_operations",
@@ -228,20 +212,27 @@ function pendingCheck(progress: ProgressDocument | null): DoctorCheck {
const jobAttention = progress.imports.jobs.filter((item) =>
["paused", "interrupted", "failed"].includes(text(item.status) ?? ""),
).length;
const taskAttention = progress.generation.tasks.filter((item) =>
["waiting_for_model", "failed"].includes(text(item.status) ?? ""),
).length;
const summary = progress.generation.summary;
const waiting = summary ? number(summary.waiting_for_model) : 0;
const failed = summary ? number(summary.failed) : 0;
const attention = jobAttention + waiting + failed;
const attention = jobAttention + taskAttention + waiting + failed;
return {
id: "pending_operations",
state: attention > 0 ? "warn" : "pass",
code: attention > 0 ? "operations_need_attention" : "pending_operations_ok",
details: {
import_jobs_validated: progress.imports.validated,
generation_tasks_validated: progress.generation.validated,
jobs_need_attention: jobAttention,
tasks_need_attention: taskAttention,
waiting_for_model: waiting,
failed,
output_truncated: progress.imports.output_truncated,
output_truncated:
progress.imports.output_truncated ||
progress.generation.output_truncated,
},
};
}
+24 -11
View File
@@ -23,26 +23,39 @@ function accountHomeDirectory(): string | undefined {
async function main(): Promise<number> {
const arguments_ = process.argv.slice(2);
const command = arguments_[0];
const action = arguments_[1];
// A preview (`--yes` absent) must not create or write any diagnostic file:
// the contract is that it only emits a plan. Diagnostics for these commands
// are initialized only once the caller has confirmed the mutation.
const confirmed = arguments_.includes("--yes");
const confirmed = arguments_.includes("--yes")
? true
: arguments_.includes("--confirm");
const installLike = [
"install",
"upgrade",
"service-install",
"service-uninstall",
].includes(command ?? "");
const isPreview = installLike && !confirmed;
const mutating = [
"start",
"stop",
"restart",
"install",
"upgrade",
"service-install",
"service-uninstall",
].includes(command ?? "");
const planFileApply =
(command === "sources" && action === "import") ||
(command === "models" && (action === "configure" || action === "probe")) ||
(command === "processing" && action === "apply") ||
(command === "memories" && action === "apply");
const isPreview = (installLike || planFileApply) && !confirmed;
const mutating =
[
"start",
"stop",
"restart",
"install",
"upgrade",
"service-install",
"service-uninstall",
].includes(command ?? "") ||
(command === "sources" && action === "import") ||
(command === "models" && ["configure", "probe"].includes(action ?? "")) ||
(command === "processing" && action === "apply") ||
(command === "memories" && action === "apply");
let diagnostics: DiagnosticLogger | null = null;
if (mutating && !isPreview) {
try {
-93
View File
@@ -1,93 +0,0 @@
/**
* `memhub graph-width`: the user's decision when the embedding width changed
* and the memory graph no longer matches it (`POST /api/v1/graph-width`).
*
* `keep_existing` and `export_first` change nothing irreversible and run
* directly. `rebuild_with_new` archives the current graph and rebuilds it, so
* the CLI asks for `--yes` before sending it.
*/
import { parseGraphWidthDecisionReceipt } from "@one-memory/operations";
import type { CoreOperations } from "./core-operations.js";
import { PRODUCT } from "./version.js";
export const GRAPH_WIDTH_DECISIONS = [
"keep_existing",
"rebuild_with_new",
"export_first",
] as const;
export type GraphWidthDecision = (typeof GRAPH_WIDTH_DECISIONS)[number];
/** The one decision that cannot be taken back. */
export function graphWidthDecisionIsDestructive(
decision: GraphWidthDecision,
): boolean {
return decision === "rebuild_with_new";
}
export type GraphWidthDocument =
| {
schema_version: 1;
product: string;
command: "graph-width";
state: "committed";
decision: GraphWidthDecision;
receipt: Record<string, unknown>;
}
| {
schema_version: 1;
product: string;
command: "graph-width";
state: "rejected" | "unknown";
decision: GraphWidthDecision;
code: string;
};
export async function decideGraphWidth(
decision: GraphWidthDecision,
operations: CoreOperations,
): Promise<GraphWidthDocument> {
const base = {
schema_version: 1 as const,
product: PRODUCT,
command: "graph-width" as const,
decision,
};
const result = await operations.post<unknown>("/api/v1/graph-width", {
decision,
});
if (result.state !== "committed") {
return { ...base, state: result.state, code: result.code };
}
const receipt = parseGraphWidthDecisionReceipt(result.data);
if (receipt === null) {
return { ...base, state: "unknown", code: "graph_width_receipt_invalid" };
}
return {
...base,
state: "committed",
receipt: {
archived_path: receipt.archived_path ?? null,
exported_path: receipt.exported_path ?? null,
mismatch: receipt.mismatch ?? null,
},
};
}
export function graphWidthExitCode(document: GraphWidthDocument): number {
return document.state === "committed" ? 0 : 1;
}
export function renderGraphWidthText(document: GraphWidthDocument): string {
if (document.state === "committed") {
const lines = [`已执行图谱宽度决定:${document.decision}`];
if (document.receipt.exported_path) {
lines.push(`导出:${document.receipt.exported_path}`);
}
if (document.receipt.archived_path) {
lines.push(`旧图谱已归档:${document.receipt.archived_path}`);
}
return `${lines.join("\n")}\n`;
}
return `图谱宽度决定未完成:${document.state}(${document.code})\n`;
}
File diff suppressed because it is too large. Load diff
+287 -82
View File
@@ -1,4 +1,4 @@
import { randomUUID } from "node:crypto";
import { createHash, randomUUID } from "node:crypto";
import type { ImportJob, ModelProfile } from "@one-memory/contracts";
import {
buildModelProfileCreateRequest,
@@ -26,7 +26,45 @@ const PROVIDERS = [
"local_onnx",
] as const;
interface SourceImportPlanBody {
schema_version: 1;
kind: "source_import";
request_id: string;
source_ids: string[];
source_roots: Record<string, string>;
source_candidate_revisions: Record<string, number>;
confirm_environment_overrides: boolean;
full_rescan: false;
onboarding: true;
}
export type SourceImportPlan = SourceImportPlanBody & { digest: string };
interface ModelCreatePlanBody {
schema_version: 1;
kind: "model_profile_create";
operation_id: string;
config: Record<string, unknown>;
}
export type ModelCreatePlan = ModelCreatePlanBody & { digest: string };
/** A removal only needs an id: there is no config to review, and inventing one
* to fit the create shape would misrepresent what the command does.
*/
export interface ModelRemovePlan {
schema_version: 1;
kind: "model_profile_remove";
target: string;
}
export type FlowDocument =
| {
schema_version: 1;
product: string;
state: "preview";
plan: SourceImportPlan | ModelCreatePlan | ModelRemovePlan;
}
| {
schema_version: 1;
product: string;
@@ -95,24 +133,10 @@ function classifySourceImport(
return committed(operationId, receipt);
}
/**
* Connect sources and import their history in one direct request.
*
* Reads the current candidates, refuses a source whose location comes from an
* environment override unless the caller confirmed overrides, and posts one
* `/sources/connect` (grant + history import + live sync). `requestId` is the
* idempotency key: re-running with the same id reads the first receipt.
*/
export async function connectSourcesWithImport(
export async function planSourceImport(
reader: CoreReader,
operations: CoreOperations,
input: {
sourceIds: readonly string[];
confirmOverrides: boolean;
requestId?: string;
},
sourceIds: readonly string[],
): Promise<FlowDocument> {
const sourceIds = input.sourceIds;
if (
sourceIds.length < 1 ||
sourceIds.length > 32 ||
@@ -121,8 +145,6 @@ export async function connectSourcesWithImport(
) {
return invalid("invalid_source_selection");
}
const requestId = input.requestId ?? randomUUID();
if (!UUID.test(requestId)) return invalid("invalid_request_id");
const outcome = await reader<unknown>("/api/v1/sources");
if (outcome.state !== "ok") return fromReadFailure(outcome);
const candidates = validateSourceConnectionCandidates(outcome.data);
@@ -134,59 +156,69 @@ export async function connectSourcesWithImport(
return invalid("source_unavailable");
}
const items = selected.map(([candidate]) => candidate);
const needsOverride = items
.filter((item) => item.requires_override_confirmation)
.map((item) => item.source_id);
if (needsOverride.length > 0 && !input.confirmOverrides) {
return {
schema_version: 1,
product: PRODUCT,
state: "rejected",
operation_id: requestId,
code: "environment_override_confirmation_required",
receipt: { override_required_source_ids: needsOverride },
};
}
const body: SourceImportPlanBody = {
schema_version: 1,
kind: "source_import",
request_id: randomUUID(),
source_ids: [...sourceIds],
source_roots: Object.fromEntries(
items.map((item) => [item.source_id, item.candidate_root_id]),
),
source_candidate_revisions: Object.fromEntries(
items.flatMap((item) =>
item.candidate_revision === undefined
? []
: [[item.source_id, item.candidate_revision]],
),
),
confirm_environment_overrides: items.some(
(item) => item.requires_override_confirmation,
),
full_rescan: false,
onboarding: true,
};
return preview(withDigest(body));
}
export async function executeSourceImport(
value: unknown,
reader: CoreReader,
operations: CoreOperations,
): Promise<FlowDocument> {
const plan = validateSourcePlan(value);
if (!plan) return invalid("source_plan_invalid");
// The shared builder owns the connect body's field set and coalescing, so
// the CLI and the Web console send the same request by construction.
let request: ReturnType<typeof buildSourceImportRequest>;
try {
request = buildSourceImportRequest({
sourceIds: [...sourceIds],
sourceRoots: Object.fromEntries(
items.map((item) => [item.source_id, item.candidate_root_id]),
),
sourceCandidateRevisions: Object.fromEntries(
items.flatMap((item) =>
item.candidate_revision === undefined
? []
: [[item.source_id, item.candidate_revision]],
),
),
confirmEnvironmentOverrides: needsOverride.length > 0,
fullRescan: false,
onboarding: true,
sourceIds: plan.source_ids,
sourceRoots: plan.source_roots,
sourceCandidateRevisions: plan.source_candidate_revisions,
confirmEnvironmentOverrides: plan.confirm_environment_overrides,
fullRescan: plan.full_rescan,
onboarding: plan.onboarding,
enableSync: true,
requestId,
requestId: plan.request_id,
});
} catch (error) {
if (error instanceof SourceImportInputError) {
return invalid("invalid_source_selection");
return invalid("source_plan_invalid");
}
throw error;
}
const result = await operations.post<unknown>(request.path, request.body);
if (result.state === "rejected") {
return rejected(requestId, result.code);
return rejected(plan.request_id, result.code);
}
if (result.state === "committed") {
const receipt = validateSourceReceipt(result.data, requestId);
if (!receipt) return unknown(requestId, "source_receipt_invalid");
return classifySourceImport(requestId, receipt);
const receipt = validateSourceReceipt(result.data, plan.request_id);
if (!receipt) return unknown(plan.request_id, "source_receipt_invalid");
return classifySourceImport(plan.request_id, receipt);
}
const recovered = await readSourceReceipt(reader, requestId);
const recovered = await readSourceReceipt(reader, plan.request_id);
if (recovered.state === "unknown" || recovered.state === "not_found") {
return unknown(requestId, result.code);
return unknown(plan.request_id, result.code);
}
// A recovered receipt must be classified the same way as a fresh one: an
// all-rejected or all-busy replay is still not success. Only a terminal
@@ -230,23 +262,26 @@ export async function readSourceReceipt(
return committed(requestId, receipt);
}
/**
* Create a model profile directly from a bounded config (never containing the
* key) plus an optional key read from a controlled file.
*
* `operationId` is Core's creation idempotency key; the receipt can be read
* later with `readModelReceipt`.
*/
export async function createModelProfile(
export function planModelProfile(value: unknown): FlowDocument {
const config = validateModelConfig(value);
if (!config) return invalid("model_config_invalid");
const body: ModelCreatePlanBody = {
schema_version: 1,
kind: "model_profile_create",
operation_id: randomUUID(),
config,
};
return preview(withDigest(body));
}
export async function executeModelProfile(
value: unknown,
apiKey: string | undefined,
reader: CoreReader,
operations: CoreOperations,
operationId: string = randomUUID(),
): Promise<FlowDocument> {
const config = validateModelConfig(value);
if (!config) return invalid("model_config_invalid");
if (!UUID.test(operationId)) return invalid("invalid_operation_id");
const plan = validateModelPlan(value);
if (!plan) return invalid("model_plan_invalid");
// The shared create builder owns the body's allow-list, bounds and the
// creation_operation_id/api_key handling, so the CLI and the Web console
// send the same create request.
@@ -254,31 +289,31 @@ export async function createModelProfile(
try {
request = buildModelProfileCreateRequest(
{
...config,
...(plan.config as Record<string, unknown>),
...(apiKey === undefined ? {} : { api_key: apiKey }),
} as Parameters<typeof buildModelProfileCreateRequest>[0],
operationId,
plan.operation_id,
);
} catch (error) {
if (error instanceof ModelProfileInputError) {
return invalid("model_config_invalid");
return invalid("model_plan_invalid");
}
throw error;
}
const result = await operations.post<unknown>(request.path, request.body);
if (result.state === "rejected") {
return rejected(operationId, result.code);
return rejected(plan.operation_id, result.code);
}
if (result.state === "committed") {
const profile = validateModelProfile(result.data);
return profile
? committed(operationId, projectProfile(profile))
: unknown(operationId, "model_receipt_invalid");
? committed(plan.operation_id, projectProfile(profile))
: unknown(plan.operation_id, "model_receipt_invalid");
}
const recovered = await readModelReceipt(reader, operationId);
const recovered = await readModelReceipt(reader, plan.operation_id);
return recovered.state === "committed"
? { ...recovered, state: "recovered" }
: unknown(operationId, result.code);
: unknown(plan.operation_id, result.code);
}
export async function readModelReceipt(
@@ -327,6 +362,27 @@ export function parseApiKey(bytes: Buffer | null): string | undefined {
}
export function renderFlowText(document: FlowDocument): string {
if (document.state === "preview") {
const plan = document.plan;
// A removal has no config digest or operation id to confirm: the target id
// *is* the thing being reviewed. Rendering fields it does not have would
// print "undefined".
if (plan.kind === "model_profile_remove") {
return [
`预览:${plan.kind}`,
`目标配置:${plan.target}`,
"未执行任何写操作。加 --confirm 才会真正删除。",
"",
].join("\n");
}
return [
`预览:${plan.kind}`,
`操作 ID:${"request_id" in plan ? plan.request_id : plan.operation_id}`,
`确认摘要:${plan.digest}`,
"未执行任何写操作。请把 --json 输出保存为 plan 文件,再用对应命令加 --confirm。",
"",
].join("\n");
}
const lines = [
`操作状态:${document.state}`,
`操作 ID:${document.operation_id ?? "无"}`,
@@ -351,20 +407,98 @@ export function renderFlowText(document: FlowDocument): string {
if (document.state === "partial" && Array.isArray(accepted)) {
lines.push(` 已接入:${accepted.join("、")}`);
}
const overrides = document.receipt.override_required_source_ids;
if (Array.isArray(overrides)) {
lines.push(
` 这些来源的目录来自环境变量覆盖,需要确认:${overrides.join("、")}`,
" 确认目录无误后加 --confirm-overrides 重新运行。",
);
}
}
lines.push("");
return lines.join("\n");
}
export function flowExitCode(document: FlowDocument): number {
return ["committed", "recovered"].includes(document.state) ? 0 : 1;
return ["preview", "committed", "recovered"].includes(document.state) ? 0 : 1;
}
export function inspectExecutionPlan(value: unknown): {
kind: "source_import" | "model_profile_create";
operationId: string;
digest: string;
} | null {
const source = validateSourcePlan(value);
if (source) {
return {
kind: source.kind,
operationId: source.request_id,
digest: source.digest,
};
}
const model = validateModelPlan(value);
return model
? {
kind: model.kind,
operationId: model.operation_id,
digest: model.digest,
}
: null;
}
function validateSourcePlan(value: unknown): SourceImportPlan | null {
const plan = record(value);
const requestId = asString(plan?.request_id);
if (
plan?.schema_version !== 1 ||
plan.kind !== "source_import" ||
!UUID.test(requestId) ||
!Array.isArray(plan.source_ids) ||
plan.source_ids.length < 1 ||
plan.source_ids.length > 32 ||
plan.source_ids.some(
(id) => typeof id !== "string" || !SOURCE_ID.test(id),
) ||
new Set(plan.source_ids).size !== plan.source_ids.length ||
plan.full_rescan !== false ||
plan.onboarding !== true ||
typeof plan.confirm_environment_overrides !== "boolean" ||
typeof plan.digest !== "string"
) {
return null;
}
const sourceIds = plan.source_ids as string[];
const roots = stringMap(plan.source_roots, sourceIds);
const revisions = numberMap(plan.source_candidate_revisions, sourceIds);
if (!roots || !revisions) return null;
const body: SourceImportPlanBody = {
schema_version: 1,
kind: "source_import",
request_id: requestId,
source_ids: [...sourceIds],
source_roots: roots,
source_candidate_revisions: revisions,
confirm_environment_overrides: plan.confirm_environment_overrides,
full_rescan: false,
onboarding: true,
};
const expected = withDigest(body);
return timingSafeText(expected.digest, plan.digest) ? expected : null;
}
function validateModelPlan(value: unknown): ModelCreatePlan | null {
const plan = record(value);
const operationId = asString(plan?.operation_id);
if (
plan?.schema_version !== 1 ||
plan.kind !== "model_profile_create" ||
!UUID.test(operationId) ||
typeof plan.digest !== "string"
) {
return null;
}
const config = validateModelConfig(plan.config);
if (!config) return null;
const expected = withDigest<ModelCreatePlanBody>({
schema_version: 1,
kind: "model_profile_create",
operation_id: operationId,
config,
});
return timingSafeText(expected.digest, plan.digest) ? expected : null;
}
function validateModelConfig(value: unknown): Record<string, unknown> | null {
@@ -510,6 +644,37 @@ function projectJob(job: ImportJob): Record<string, unknown> {
};
}
function withDigest<T extends object>(body: T): T & { digest: string } {
return { ...body, digest: digest(body) };
}
function digest(value: unknown): string {
return createHash("sha256").update(canonical(value)).digest("hex");
}
function canonical(value: unknown): string {
if (Array.isArray(value)) return `[${value.map(canonical).join(",")}]`;
if (value && typeof value === "object") {
return `{${Object.keys(value)
.sort()
.map(
(key) =>
`${JSON.stringify(key)}:${canonical((value as Record<string, unknown>)[key])}`,
)
.join(",")}}`;
}
return JSON.stringify(value);
}
function preview(plan: SourceImportPlan | ModelCreatePlan): FlowDocument {
return {
schema_version: 1,
product: PRODUCT,
state: "preview",
plan,
};
}
function committed(
operationId: string,
receipt: Record<string, unknown>,
@@ -602,3 +767,43 @@ function optionalInteger(
(value as number) <= maximum)
);
}
function stringMap(
value: unknown,
keys: readonly string[],
): Record<string, string> | null {
const map = record(value);
if (
!map ||
Object.keys(map).length !== keys.length ||
keys.some((key) => !boundedText(map[key], 256))
) {
return null;
}
return Object.fromEntries(keys.map((key) => [key, map[key] as string]));
}
function numberMap(
value: unknown,
allowedKeys: readonly string[],
): Record<string, number> | null {
const map = record(value);
if (
!map ||
Object.entries(map).some(
([key, revision]) =>
!allowedKeys.includes(key) ||
!Number.isSafeInteger(revision) ||
(revision as number) < 0,
)
) {
return null;
}
return Object.fromEntries(
Object.entries(map).map(([key, revision]) => [key, revision as number]),
);
}
function timingSafeText(expected: string, actual: unknown): boolean {
return typeof actual === "string" && expected === actual;
}
@@ -0,0 +1,64 @@
import path from "node:path";
import { describe, expect, it } from "vitest";
import { initPlanAttemptPath } from "./cli.js";
import { interactiveInitAttemptPath } from "./init-attempt.js";
// Regression: `init --source <id> --import` used to pass the literal string
// `"init"` to `claimPlanAttempt`, which resolved to `<cwd>/init.one-memory-attempt`.
// Two consequences: the fence was shared across unrelated runs in the same
// directory, and any unrelated file named `init` could make a valid import
// report a misleading usage error (exit 2) instead of reading its receipt.
describe("init import attempt fence", () => {
it("lives under the data dir, not the current working directory", () => {
const fence = initPlanAttemptPath(
"/tmp/example data",
"11111111-2222-4333-8444-555555555555",
);
expect(fence).toBe(
path.join(
"/tmp/example data",
".cli-init-attempt-11111111-2222-4333-8444-555555555555",
),
);
expect(path.isAbsolute(fence)).toBe(true);
expect(fence).not.toBe("init");
// Directly under the data dir: no nested directory that could fail to be
// created and turn a valid import into a usage error.
expect(path.dirname(fence)).toBe("/tmp/example data");
});
it("is per data dir and per operation, so installs/runs cannot collide", () => {
const a = initPlanAttemptPath("/tmp/data-a", "op-1");
const b = initPlanAttemptPath("/tmp/data-b", "op-1");
const c = initPlanAttemptPath("/tmp/data-a", "op-2");
expect(new Set([a, b, c]).size).toBe(3);
});
});
// QA-CLI R3 F1: the interactive wizard passed a CWD-relative singleton
// (`init` or `<journalDir>/init`) to `claimPlanAttempt`. A stale
// `init.one-memory-attempt` in the working directory then made every later
// `memhub init` fail with "交互计划无效 / exit 2".
describe("interactive init attempt fence", () => {
it("lives under the data dir and is unique per operation", () => {
const fence = interactiveInitAttemptPath(
"/tmp/data",
"11111111-2222-4333-8444-555555555555",
);
expect(fence).toBe(
path.join(
"/tmp/data",
".cli-init-attempt-11111111-2222-4333-8444-555555555555",
),
);
expect(path.isAbsolute(fence)).toBe(true);
expect(fence).not.toBe("init");
});
it("does not reuse one fence for two distinct plans", () => {
const first = interactiveInitAttemptPath("/tmp/data", "op-1");
const second = interactiveInitAttemptPath("/tmp/data", "op-2");
expect(first).not.toBe(second);
});
});
+16
View File
@@ -0,0 +1,16 @@
import path from "node:path";
/** Where the interactive `init` wizard keeps its durable attempt fence.
*
* The wizard originally passed a CWD-relative singleton (`init`) to
* `claimPlanAttempt`. That shared one fence across unrelated runs and let a
* stale `init.one-memory-attempt` in the working directory turn every later
* wizard run into "交互计划无效 / exit 2". Keep the fence under the Runtime's
* data dir, keyed by the operation id, exactly like `init --source --import`.
*/
export function interactiveInitAttemptPath(
dataDir: string,
operationId: string,
): string {
return path.join(dataDir, `.cli-init-attempt-${operationId}`);
}
+6 -6
View File
@@ -418,9 +418,9 @@ describe("A2 dual-mode init", () => {
});
// QA-CLI R3 F1: a stale CWD `init.one-memory-attempt` used to poison every
// later wizard run. There are no plan files or attempt fences any more: the
// connect request id is the idempotency key Core keeps.
it("ignores a stale CWD init fence and writes no fence files", async () => {
// later wizard run. The fence now lives under the isolated data dir and is
// keyed by the operation id.
it("ignores a stale CWD init fence and keeps its fence under the data dir", async () => {
transport.get.mockImplementation(async (url: string) =>
url === "/api/v1/sources"
? { status: 200, data: [candidate("codex", "Codex")] }
@@ -471,7 +471,7 @@ describe("A2 dual-mode init", () => {
const dataEntries = await readdir(dataDir);
expect(
dataEntries.some((entry) => entry.startsWith(".cli-init-attempt-")),
).toBe(false);
).toBe(true);
});
it("keeps selected Harnesses frozen without creating history jobs", async () => {
@@ -506,7 +506,7 @@ describe("A2 dual-mode init", () => {
});
});
describe("A19 sources connect --all-detected (old spelling: init --all-detected)", () => {
describe("A19 init --all-detected", () => {
it("connects every found source in one batch call without prompting", async () => {
transport.get.mockImplementation(async (url: string) =>
url === "/api/v1/sources"
@@ -556,7 +556,7 @@ describe("A19 sources connect --all-detected (old spelling: init --all-detected)
expect(prompter.asked).toBe(0);
expect(code).toBe(0);
const document = JSON.parse(streams.value());
expect(document.command).toBe("sources connect");
expect(document.command).toBe("init");
expect(document.state).toBe("committed");
expect(
document.sources.map((row: { source_id: string }) => row.source_id),
+50 -23
View File
@@ -4,11 +4,18 @@ import type {
CoreDeleteOperations,
CoreOperations,
} from "./core-operations.js";
import { createModelProfile, type FlowDocument } from "./headless-flows.js";
import {
executeModelProfile,
type FlowDocument,
planModelProfile,
} from "./headless-flows.js";
import { approvalFingerprint, listHosts, readHost } from "./host-flow.js";
import { parseSelection, type WizardPrompter } from "./init-wizard.js";
import { startModelProbe } from "./model-probe-flow.js";
import { activateModelProfile } from "./model-write-flows.js";
import { executeModelProbe, planModelProbe } from "./model-probe-flow.js";
import {
executeModelProfileActivate,
planModelProfileActivate,
} from "./model-write-flows.js";
type ProviderPreset = {
id: string;
@@ -352,26 +359,30 @@ export async function runInteractiveModelSetup(
);
})
: new Map<string, string>();
const created = await createModelProfile(
{
name: preset.label,
provider_kind: preset.provider_kind,
base_url: baseUrl,
model,
embedding_provider_kind: "local_onnx",
embedding_model: "bge-small-zh-v1.5",
embedding_dimensions: 512,
timeout_seconds: 90,
max_concurrency: 4,
disable_thinking: true,
},
const planned = planModelProfile({
name: preset.label,
provider_kind: preset.provider_kind,
base_url: baseUrl,
model,
embedding_provider_kind: "local_onnx",
embedding_model: "bge-small-zh-v1.5",
embedding_dimensions: 512,
timeout_seconds: 90,
max_concurrency: 4,
disable_thinking: true,
});
if (
planned.state !== "preview" ||
planned.plan.kind !== "model_profile_create"
) {
return { state: "failed", code: "model_config_invalid" };
}
const created = await executeModelProfile(
planned.plan,
apiKey,
clients.reader,
clients.operations,
);
if (created.state === "invalid") {
return { state: "failed", code: "model_config_invalid" };
}
const profile = receiptProfile(created);
if (profile === null) {
prompter.write("模型配置未保存。");
@@ -380,11 +391,18 @@ export async function runInteractiveModelSetup(
}
try {
const probed = await startModelProbe(
const probePlan = await planModelProbe(
clients.reader,
clients.operations,
profile.profile_id,
);
const probed =
probePlan.state === "preview"
? await executeModelProbe(
probePlan.plan,
clients.reader,
clients.operations,
)
: probePlan;
if (!probePassed(probed)) {
await clients.operations.delete(
`/api/v1/model-profiles/${encodeURIComponent(profile.profile_id)}`,
@@ -403,8 +421,17 @@ export async function runInteractiveModelSetup(
throw error;
}
const activated = await activateModelProfile(
profile.profile_id,
const activation = planModelProfileActivate({
profileId: profile.profile_id,
});
if (
activation.state !== "preview" ||
activation.plan.kind !== "model_profile_activate"
) {
return { state: "failed", code: "model_activate_plan_invalid" };
}
const activated = await executeModelProfileActivate(
activation.plan,
clients.operations,
);
if (activated.state !== "committed") {
+13 -13
View File
@@ -101,7 +101,7 @@ describe("memhub jobs", () => {
expect(transport.post).not.toHaveBeenCalled();
});
it("start posts directly, without a preview step", async () => {
it("start previews without a POST", async () => {
const output = captureStdout();
let code: number;
try {
@@ -110,14 +110,14 @@ describe("memhub jobs", () => {
output.restore();
}
expect(code!).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
"/api/v1/import-jobs",
{ source_id: "codex", operation: "history" },
undefined,
);
expect(transport.post).not.toHaveBeenCalled();
expect(JSON.parse(output.value())).toMatchObject({
state: "preview",
plan: { source_id: "codex", operation: "history" },
});
});
it("start still accepts a legacy --confirm", async () => {
it("start --confirm posts the exact Core request", async () => {
const output = captureStdout();
let code: number;
try {
@@ -146,7 +146,7 @@ describe("memhub jobs", () => {
});
it.each(["pause", "resume", "cancel"] as const)(
"%s posts the action endpoint directly",
"%s previews without a POST",
async (action) => {
const output = captureStdout();
let code: number;
@@ -156,11 +156,11 @@ describe("memhub jobs", () => {
output.restore();
}
expect(code!).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
`/api/v1/import-jobs/job-1/${action}`,
{},
undefined,
);
expect(transport.post).not.toHaveBeenCalled();
expect(JSON.parse(output.value())).toMatchObject({
state: "preview",
plan: { job_id: "job-1", action },
});
},
);
+1 -31
View File
@@ -1,4 +1,4 @@
import { mkdtemp, rm, symlink } from "node:fs/promises";
import { mkdtemp, rm } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { DiagnosticLogger } from "@one-memory/runtime-control";
@@ -41,15 +41,11 @@ it("redacts unsafe details and absolute stack frames from the read surface", ()
session_id: "11111111-1111-4111-8111-111111111111",
details: {
count: 1,
reason: "/Users/me/private-query.txt",
status: "sk-live-secret-token",
source_id: "credential.private",
prompt: "secret-token-123",
authorization: "Bearer secret-token-123",
} as unknown as Record<string, string | number | boolean | null>,
safe_stack: [
"packages/runtime-control/src/example.ts:42:7",
"packages/runtime/sk-live-secret-token.ts:1:1",
"/Users/me/private.py:1",
"C:\\Users\\me\\private.py:1",
],
@@ -60,8 +56,6 @@ it("redacts unsafe details and absolute stack frames from the read surface", ()
"packages/runtime-control/src/example.ts:42:7",
]);
expect(JSON.stringify(redacted)).not.toContain("secret-token-123");
expect(JSON.stringify(redacted)).not.toContain("sk-live-secret-token");
expect(JSON.stringify(redacted)).not.toContain("credential.private");
expect(JSON.stringify(redacted)).not.toContain("/Users/me");
expect(JSON.stringify(redacted)).not.toContain("C:\\\\Users");
});
@@ -138,27 +132,3 @@ it("follows a newly appended event without stopping Runtime", async () => {
});
expect(seen).toEqual(["service.ready", "core.failed"]);
});
it("fails follow mode when the diagnostic directory is unsafe", async () => {
const dataDir = await mkdtemp(
path.join(os.tmpdir(), "om-cli-follow-unsafe-"),
);
const external = await mkdtemp(
path.join(os.tmpdir(), "om-cli-follow-external-"),
);
roots.push(dataDir, external);
await symlink(external, path.join(dataDir, "logs"));
const query = parseLogQuery(["--follow"]);
expect(query).not.toBeNull();
if (!query) throw new Error("query_missing");
await expect(
followLogs({
dataDir,
query,
signal: new AbortController().signal,
maxEvents: 1,
emit: () => undefined,
wait: async () => undefined,
}),
).rejects.toThrow("log_directory_unreadable");
});
+102 -6
View File
@@ -3,7 +3,6 @@ import path from "node:path";
import type { DiagnosticEventV1 } from "@one-memory/runtime-control";
import {
diagnosticLogFiles,
projectDiagnosticEventForSafeOutput,
readDiagnosticEvents,
} from "@one-memory/runtime-control";
@@ -25,6 +24,54 @@ export interface LogQuery {
const ID = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,255}$/;
const NAME = /^[a-z][a-z0-9]*(?:[._-][a-z0-9]+){0,15}$/;
const SAFE_DETAIL_KEYS = new Set([
"action",
"attempt",
"count",
"method",
"mode",
"phase",
"platform",
"reason",
"result_count",
"source_id",
"spawned",
"status",
"version",
"parent_session_id",
"invalid_lines",
"truncated",
"runtime_state",
"core_state",
"broker_state",
"instance_id",
"open_file_count",
"memory_bytes",
"thread_count",
"database_error_count",
"file_count",
"sqlite_file_count",
"byte_file_count",
"total_bytes",
"copied_bytes",
"bytes_per_second",
"stored_count",
"revised_count",
"skipped_count",
"error_count",
"pending_count",
"prepared_count",
"queue_depth",
"in_flight_count",
"call_count",
"created_count",
"updated_count",
"superseded_count",
"deleted_count",
"operation_kind",
"file_kind",
]);
export function parseLogQuery(arguments_: readonly string[]): LogQuery | null {
if (arguments_.filter((item) => item === "--json").length > 1) return null;
const values: Partial<LogQuery> = { errorsOnly: false, follow: false };
@@ -190,13 +237,63 @@ export function renderLogText(
return `${lines.join("\n")}\n`;
}
/** Defence in depth for the CLI read surface. Local writers accept arbitrary
* bounded scalar keys, so every outward read surface must reuse the same
* field-and-value projection as the support-bundle exporter. */
/** Defence in depth for the CLI read surface. `DiagnosticLogger` already
* allow-lists keys, but a future writer, a manually edited log, or a malformed
* line must not turn `memhub logs` into an exfiltration path. */
export function redactDiagnosticEvent(
event: DiagnosticEventV1,
): DiagnosticEventV1 {
return projectDiagnosticEventForSafeOutput(event);
const redacted: DiagnosticEventV1 = {
schema_version: 1,
timestamp: event.timestamp,
level: event.level,
service: event.service,
component: event.component,
event: event.event,
outcome: event.outcome,
session_id: event.session_id,
...(event.trace_id === undefined ? {} : { trace_id: event.trace_id }),
...(event.incident_id === undefined
? {}
: { incident_id: event.incident_id }),
...(event.instance_id === undefined
? {}
: { instance_id: event.instance_id }),
...(event.operation_id === undefined
? {}
: { operation_id: event.operation_id }),
...(event.request_id === undefined ? {} : { request_id: event.request_id }),
...(event.job_id === undefined ? {} : { job_id: event.job_id }),
...(event.task_id === undefined ? {} : { task_id: event.task_id }),
...(event.duration_ms === undefined
? {}
: { duration_ms: event.duration_ms }),
...(event.state_before === undefined
? {}
: { state_before: event.state_before }),
...(event.state_after === undefined
? {}
: { state_after: event.state_after }),
...(event.error_code === undefined ? {} : { error_code: event.error_code }),
...(event.error_type === undefined ? {} : { error_type: event.error_type }),
...(event.safe_stack === undefined
? {}
: {
safe_stack: event.safe_stack.filter(
(frame) =>
!frame.startsWith("/") &&
!/^[A-Za-z]:\\/.test(frame) &&
!frame.includes(".."),
),
}),
};
const details = Object.fromEntries(
Object.entries(event.details ?? {}).filter(([key]) =>
SAFE_DETAIL_KEYS.has(key),
),
);
if (Object.keys(details).length > 0) redacted.details = details;
return redacted;
}
export async function followLogs(options: {
@@ -218,7 +315,6 @@ export async function followLogs(options: {
...options.query,
tail: 5_000,
});
if (report.failure) throw new Error(report.failure);
for (const event of report.events) {
const identity = JSON.stringify(event);
if (seen.has(identity)) continue;
-583
View File
@@ -1,583 +0,0 @@
import { mkdtemp, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { run } from "./cli.js";
const transport = vi.hoisted(() => ({
get: vi.fn(),
post: vi.fn(),
put: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
}));
vi.mock("axios", () => ({
default: { create: vi.fn(() => transport), get: vi.fn() },
}));
const runtime = vi.hoisted(() => ({
inspectRuntimeStatus: vi.fn(),
discoverRuntimeAttachment: vi.fn(),
}));
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
...runtime,
}));
const REQUEST = "33333333-3333-4333-8333-333333333333";
const EVIDENCE = "44444444-4444-4444-8444-444444444444";
function addReceipt(state = "pending", memoryIds: string[] = []) {
return {
evidence_id: EVIDENCE,
text: "Prefers tea",
scope: "all-apps",
created_at: "2026-10-27T00:00:00Z",
state,
task_id: null,
memory_ids: memoryIds,
error: null,
};
}
function memoryRecord(overrides: Record<string, unknown> = {}) {
return {
memory_id: "exp-1",
content: "deploy\nrun make",
layer: "experience",
status: "active",
is_latest: true,
supersedes: [],
superseded_by: [],
tags: [],
owner: null,
source_raw_ids: [],
user_id: "local",
agent_id: "",
session_id: null,
source: "codebuddy",
memory_at: null,
gmt_created: 1_790_000_000,
scene: "working",
kind: "experience",
variant: "working",
task: "deploy",
solution: "run make",
...overrides,
};
}
beforeEach(() => {
vi.clearAllMocks();
transport.get.mockReset();
transport.post.mockReset();
transport.put.mockReset();
transport.patch.mockReset();
transport.delete.mockReset();
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
version: "1.2.3",
});
runtime.discoverRuntimeAttachment.mockResolvedValue({
instanceId: "instance-1",
apiToken: "synthetic-runtime-token",
host: "127.0.0.1",
port: 8765,
servicePid: 4242,
health: {
status: "ok",
service: "one-memory-core",
version: "1.2.3",
instance_id: "instance-1",
},
});
});
describe("memories add", () => {
it("adds one memory directly and prints the receipt with its request id", async () => {
transport.post.mockResolvedValue({ status: 201, data: addReceipt() });
const result = await capturedRun([
"memories",
"add",
"Prefers tea",
"--json",
]);
expect(result.code).toBe(0);
const [url, body] = transport.post.mock.calls[0];
expect(url).toBe("/api/v2/memories");
expect(body).toMatchObject({ text: "Prefers tea", scope: "all-apps" });
expect(body.request_id).toMatch(/^[0-9a-f-]{36}$/);
expect(JSON.parse(result.stdout)).toMatchObject({
schema_version: 2,
command: "memories add",
state: "accepted",
request_id: body.request_id,
receipt: { evidence_id: EVIDENCE, state: "pending" },
});
});
it("reuses --request-id so a retry is idempotent, and says so in text", async () => {
transport.post.mockResolvedValue({ status: 201, data: addReceipt() });
const result = await capturedRun([
"memories",
"add",
"Prefers tea",
"--request-id",
REQUEST,
]);
expect(result.code).toBe(0);
expect(transport.post.mock.calls[0][1].request_id).toBe(REQUEST);
expect(result.stdout).toContain(`request_id: ${REQUEST}`);
expect(result.stdout).toContain(`memhub memories add-status ${REQUEST}`);
});
it("reads the text from a bounded file", async () => {
const directory = await mkdtemp(path.join(tmpdir(), "om-memory-add-"));
const file = path.join(directory, "memory.txt");
await writeFile(file, "Prefers tea\n", { mode: 0o600 });
transport.post.mockResolvedValue({ status: 201, data: addReceipt() });
const result = await capturedRun([
"memories",
"add",
"--content-file",
file,
"--json",
]);
expect(result.code).toBe(0);
expect(transport.post.mock.calls[0][1].text).toBe("Prefers tea");
});
it("refuses bad input before contacting Core", async () => {
for (const argv of [
["memories", "add"],
["memories", "add", "a", "--content-file", "/tmp/x"],
["memories", "add", "a", "--request-id", "not-a-uuid"],
["memories", "add", " "],
["memories", "add", "x".repeat(12_001)],
]) {
expect((await capturedRun(argv)).code).toBe(2);
}
expect(transport.post).not.toHaveBeenCalled();
expect(runtime.inspectRuntimeStatus).not.toHaveBeenCalled();
});
it("reports a Core conflict as rejected, on stdout, exit 1", async () => {
transport.post.mockResolvedValue({
status: 409,
data: { detail: "memory_add_conflict" },
});
const result = await capturedRun([
"memories",
"add",
"Other text",
"--request-id",
REQUEST,
"--json",
]);
expect(result.code).toBe(1);
expect(JSON.parse(result.stdout)).toMatchObject({
state: "rejected",
code: "memory_add_conflict",
request_id: REQUEST,
});
});
it("never reports a malformed success body as accepted", async () => {
transport.post.mockResolvedValue({
status: 201,
data: { evidence_id: "x" },
});
const result = await capturedRun(["memories", "add", "tea", "--json"]);
expect(result.code).toBe(1);
expect(JSON.parse(result.stdout).state).toBe("unknown");
});
it("answers a stopped Runtime with exit 3 and writes nothing", async () => {
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "stopped",
code: "runtime_not_running",
});
const result = await capturedRun(["memories", "add", "tea", "--json"]);
expect(result.code).toBe(3);
expect(JSON.parse(result.stdout).code).toBe("runtime_not_running");
expect(transport.post).not.toHaveBeenCalled();
});
});
describe("memories add-status", () => {
it("reads the addition by request id", async () => {
transport.get.mockResolvedValue({
status: 200,
data: addReceipt("ready", ["m-1"]),
});
const result = await capturedRun([
"memories",
"add-status",
REQUEST,
"--json",
]);
expect(result.code).toBe(0);
expect(transport.get).toHaveBeenCalledWith(
`/api/v2/memories/additions/${REQUEST}`,
undefined,
);
expect(JSON.parse(result.stdout)).toMatchObject({
command: "memories add-status",
state: "accepted",
receipt: { state: "ready", memory_ids: ["m-1"] },
});
});
it("reports an unknown request id as not_found, exit 1", async () => {
transport.get.mockResolvedValue({
status: 404,
data: { detail: "memory_add_not_found" },
});
const result = await capturedRun(["memories", "status", REQUEST]);
expect(result.code).toBe(1);
expect(result.stdout).toContain("未找到该 request id");
});
it("refuses a non-UUID request id", async () => {
expect((await capturedRun(["memories", "add-status", "abc"])).code).toBe(2);
expect(transport.get).not.toHaveBeenCalled();
});
});
describe("memories update / delete pass the namespace", () => {
it("updates an experience with kind, variant and task, directly", async () => {
const directory = await mkdtemp(path.join(tmpdir(), "om-memory-update-"));
const file = path.join(directory, "solution.txt");
await writeFile(file, "run make deploy\n", { mode: 0o600 });
transport.put.mockResolvedValue({ status: 200, data: memoryRecord() });
const result = await capturedRun([
"memories",
"update",
"exp-1",
"--content-file",
file,
"--kind",
"experience",
"--variant",
"working",
"--task",
"deploy",
"--json",
]);
expect(result.code).toBe(0);
expect(transport.put).toHaveBeenCalledWith(
"/api/v2/memories/exp-1",
{
content: "run make deploy",
kind: "experience",
variant: "working",
task: "deploy",
},
undefined,
);
expect(JSON.parse(result.stdout)).toMatchObject({
schema_version: 2,
state: "committed",
operation: "update",
});
});
it("refuses --task on anything but an experience, and a lone --variant", async () => {
for (const argv of [
["memories", "update", "m-1", "--content-file", "/tmp/x", "--task", "t"],
[
"memories",
"update",
"m-1",
"--content-file",
"/tmp/x",
"--kind",
"fact",
"--variant",
"coding",
"--task",
"t",
],
["memories", "delete", "m-1", "--variant", "coding"],
["memories", "delete", "m-1", "--kind", "chat", "--variant", "coding"],
["memories", "show", "m-1", "--kind", "nope"],
]) {
expect((await capturedRun(argv)).code).toBe(2);
}
expect(transport.put).not.toHaveBeenCalled();
expect(transport.delete).not.toHaveBeenCalled();
});
it("previews a delete without --yes and changes nothing", async () => {
const result = await capturedRun([
"memories",
"delete",
"fact-1",
"--kind",
"fact",
"--variant",
"coding",
"--json",
]);
expect(result.code).toBe(0);
expect(JSON.parse(result.stdout)).toMatchObject({
schema_version: 2,
command: "memories delete",
state: "preview",
target: "fact-1",
applied: false,
});
expect(JSON.parse(result.stdout).next_step).toContain(
"--kind fact --variant coding --yes",
);
expect(transport.delete).not.toHaveBeenCalled();
expect(runtime.inspectRuntimeStatus).not.toHaveBeenCalled();
});
it("deletes with --yes (or the legacy --confirm) and names the namespace", async () => {
transport.delete.mockResolvedValue({ status: 204, data: "" });
for (const flag of ["--yes", "--confirm"]) {
const result = await capturedRun([
"memories",
"delete",
"fact-1",
"--kind",
"fact",
"--variant",
"coding",
flag,
"--json",
]);
expect(result.code).toBe(0);
expect(JSON.parse(result.stdout)).toMatchObject({
schema_version: 2,
state: "committed",
operation: "delete",
});
}
expect(transport.delete).toHaveBeenCalledWith(
"/api/v2/memories/fact-1?kind=fact&variant=coding",
undefined,
);
});
it("explains an ambiguous id instead of a bare code", async () => {
transport.delete.mockResolvedValue({
status: 409,
data: { detail: "memory_id_ambiguous" },
});
const result = await capturedRun(["memories", "delete", "dup", "--yes"]);
expect(result.code).toBe(1);
expect(result.stdout).toContain("--kind");
});
it("shows a memory in its namespace", async () => {
transport.get.mockResolvedValue({ status: 200, data: memoryRecord() });
const result = await capturedRun([
"memories",
"show",
"exp-1",
"--kind",
"experience",
"--variant",
"working",
"--json",
]);
expect(result.code).toBe(0);
expect(transport.get).toHaveBeenCalledWith("/api/v2/memories/exp-1", {
params: { kind: "experience", variant: "working" },
});
const document = JSON.parse(result.stdout);
expect(document.schema_version).toBe(2);
expect(document).not.toHaveProperty("evidence");
expect(document).not.toHaveProperty("history");
expect(document.memory).toMatchObject({
kind: "experience",
variant: "working",
task: "deploy",
});
});
});
describe("memories list filters", () => {
const page = {
memories: [memoryRecord()],
total: 1,
limit: 100,
offset: 0,
search_window: 200,
window_id: "w1",
window_reset: false,
};
it("sends --scene, --source and --query to the library", async () => {
transport.get.mockResolvedValue({ status: 200, data: page });
const result = await capturedRun([
"memories",
"list",
"--scene",
"working",
"--source",
"codebuddy",
"--query",
"deploy",
"--json",
]);
expect(result.code).toBe(0);
expect(transport.get).toHaveBeenCalledWith("/api/v2/memory-library", {
params: {
offset: 0,
limit: 100,
scene: "working",
source: "codebuddy",
query: "deploy",
},
});
const document = JSON.parse(result.stdout);
expect(document.schema_version).toBe(2);
expect(document.page).toMatchObject({
state: "ok",
scene: "working",
query: "deploy",
search_window: 200,
});
expect(document.memories[0]).toMatchObject({
kind: "experience",
variant: "working",
});
});
it("omits scene for the default all", async () => {
transport.get.mockResolvedValue({ status: 200, data: page });
await capturedRun(["memories", "list", "--scene", "all", "--json"]);
expect(transport.get.mock.calls[0][1].params).not.toHaveProperty("scene");
});
it("validates --status, --scene and --query locally instead of reporting a busy backend", async () => {
for (const argv of [
["memories", "list", "--status", "bogus"],
["memories", "list", "--scene", "normal"],
["memories", "list", "--query", "x", "--status", "archived"],
["memories", "list", "--source", "Bad Source"],
]) {
expect((await capturedRun(argv)).code).toBe(2);
}
expect(transport.get).not.toHaveBeenCalled();
});
it("reports a Core 422 as rejected, not as 后台正忙", async () => {
transport.get.mockResolvedValue({
status: 422,
data: { detail: "sdk_search_supports_active_status_only" },
});
const result = await capturedRun(["memories", "list"]);
expect(result.code).toBe(1);
expect(result.stdout).not.toContain("后台正忙");
expect(result.stdout).toContain("sdk_search_supports_active_status_only");
});
});
describe("graph-width", () => {
it("runs a non-destructive decision directly", async () => {
transport.post.mockResolvedValue({
status: 200,
data: { archived_path: null, exported_path: "/tmp/x", mismatch: null },
});
const result = await capturedRun([
"graph-width",
"--decision",
"export_first",
"--json",
]);
expect(result.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
"/api/v1/graph-width",
{ decision: "export_first" },
undefined,
);
expect(JSON.parse(result.stdout).state).toBe("committed");
});
it("previews rebuild_with_new without --yes and applies it with --yes", async () => {
const preview = await capturedRun([
"graph-width",
"--decision",
"rebuild_with_new",
"--json",
]);
expect(preview.code).toBe(0);
expect(JSON.parse(preview.stdout)).toMatchObject({
state: "preview",
applied: false,
});
expect(transport.post).not.toHaveBeenCalled();
transport.post.mockResolvedValue({
status: 200,
data: { archived_path: "/tmp/old", exported_path: null, mismatch: null },
});
const applied = await capturedRun([
"graph",
"width",
"--decision",
"rebuild_with_new",
"--yes",
"--json",
]);
expect(applied.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
"/api/v1/graph-width",
{ decision: "rebuild_with_new" },
undefined,
);
});
});
describe("removed commands", () => {
it.each([
["notes", "add", "--content-file", "/tmp/x"],
["notes", "process", REQUEST, "--confirm"],
["calls", "clear", "--all", "--confirm"],
["processing", "extract", "--evidence", REQUEST, "--confirm"],
["extraction", "set", "--window", "6m", "--confirm"],
["import-file", "upload", "--file", "/tmp/x", "--confirm"],
])("%s is an unknown command that writes nothing", async (...argv) => {
const result = await capturedRun(argv as string[]);
expect(result.code).toBe(2);
expect(result.stderr).toContain("未知命令");
expect(transport.post).not.toHaveBeenCalled();
});
});
async function capturedRun(arguments_: string[]) {
let stdout = "";
let stderr = "";
const stdoutSpy = vi
.spyOn(process.stdout, "write")
.mockImplementation((chunk) => {
stdout += String(chunk);
return true;
});
const stderrSpy = vi
.spyOn(process.stderr, "write")
.mockImplementation((chunk) => {
stderr += String(chunk);
return true;
});
try {
return {
code: await run(arguments_, {
ONE_MEMORY_DATA_DIR: "/synthetic/data",
}),
stdout,
stderr,
};
} finally {
stdoutSpy.mockRestore();
stderrSpy.mockRestore();
}
}
+700
View File
@@ -0,0 +1,700 @@
import { createHash, randomUUID, timingSafeEqual } from "node:crypto";
import type { GovernedMemory } from "@one-memory/contracts";
import {
buildMemoryActionRequest,
buildMemoryCorrectRequest,
buildMemoryCreateRequest,
parseMemoryReceipt,
} from "@one-memory/operations";
import type { CoreReader } from "./core-client.js";
import type { CoreOperations } from "./core-operations.js";
import { validateMemoryDetail } from "./core-responses.js";
import { isMemoryId } from "./memory-report.js";
import { PRODUCT } from "./version.js";
const MEMORY_KINDS = [
"chat",
"working",
"coding",
"schema",
"intention",
] as const;
const OPERATIONS = ["add", "correct", "delete"] as const;
const MEMORY_ORIGINS = ["extracted", "answered", "agent", "manual"] as const;
const MUTATION_RECEIPT_KEYS = new Set([
"schema_version",
"request_id",
"state",
"operation",
"memory_id",
"result_revision",
"result_status",
"memory_state",
"migrated_from",
]);
const CONTENT_MAX_CHARACTERS = 100_000;
const ID = /^[A-Za-z0-9][\w.-]{0,127}$/;
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
interface PlanBody {
schema_version: 1;
kind: "memory_mutation";
operation: (typeof OPERATIONS)[number];
request_id: string;
memory_id?: string;
expected_revision?: number;
expected_status?: string;
expected_origin?: string;
memory_kind?: (typeof MEMORY_KINDS)[number];
scopes?: string[];
evidence_ids?: string[];
content_sha256?: string;
content_characters?: number;
current?: {
status: string;
origin: string;
revision: number;
scopes: string[];
evidence: Array<{
evidence_id: string;
available: boolean;
source_id?: string;
written_by?: string;
}>;
};
}
export type MemoryMutationPlan = PlanBody & { digest: string };
export type MemoryFlowDocument =
| {
schema_version: 1;
product: string;
state: "preview";
plan: MemoryMutationPlan;
}
| {
schema_version: 1;
product: string;
state: "committed" | "recovered";
operation_id: string;
receipt: Record<string, unknown>;
}
| {
schema_version: 1;
product: string;
state: "rejected" | "unknown" | "invalid";
operation_id?: string;
code: string;
};
export function parseMemoryContent(bytes: Buffer): string | null {
if (bytes.byteLength > CONTENT_MAX_CHARACTERS * 4 + 2) return null;
let decoded: string;
try {
decoded = new TextDecoder("utf-8", { fatal: true }).decode(bytes);
} catch {
return null;
}
const content = decoded.replace(/\r?\n$/, "");
return content.trim().length > 0 &&
[...content].length <= CONTENT_MAX_CHARACTERS &&
![...content].some((character) => {
const code = character.charCodeAt(0);
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
})
? content
: null;
}
export function planMemoryAdd(input: {
content: string;
memoryKind: string;
scopes: string[];
evidenceIds: string[];
}): MemoryFlowDocument {
if (
!validContent(input.content) ||
!oneOf(input.memoryKind, MEMORY_KINDS) ||
!validScopes(input.scopes) ||
!validIds(input.evidenceIds, 500)
) {
return invalid("memory_add_input_invalid");
}
return preview(
withDigest({
schema_version: 1,
kind: "memory_mutation",
operation: "add",
request_id: randomUUID(),
memory_kind: input.memoryKind,
scopes: input.scopes,
evidence_ids: input.evidenceIds,
...contentIdentity(input.content),
}),
);
}
export async function planExistingMemoryMutation(
reader: CoreReader,
input: {
operation: Exclude<(typeof OPERATIONS)[number], "add">;
memoryId: string;
content?: string;
},
): Promise<MemoryFlowDocument> {
if (
!isMemoryId(input.memoryId) ||
(input.operation === "correct" && input.content === undefined) ||
(input.content !== undefined && !validContent(input.content))
) {
return invalid("memory_plan_input_invalid");
}
const outcome = await reader<unknown>(`/api/v1/memories/${input.memoryId}`);
if (outcome.state !== "ok") {
return invalid(`memory_${outcome.state}:${outcome.code}`);
}
const detail = validateMemoryDetail(outcome.data, input.memoryId);
if (!detail) return invalid("memory_response_invalid");
const memory = detail.memory;
if (memory.status !== "confirmed") {
return invalid("memory_operation_not_allowed");
}
const body: PlanBody = {
schema_version: 1,
kind: "memory_mutation",
operation: input.operation,
request_id: randomUUID(),
memory_id: memory.memory_id,
expected_revision: memory.revision,
expected_status: memory.status,
expected_origin: memory.origin,
...(input.content ? contentIdentity(input.content) : {}),
current: {
status: memory.status,
origin: memory.origin,
revision: memory.revision,
scopes: [...memory.scopes],
evidence: detail.evidence.map((item) => ({
evidence_id: item.evidence_id,
available: item.available,
...(item.source_id ? { source_id: item.source_id } : {}),
...(item.written_by ? { written_by: item.written_by } : {}),
})),
},
};
return preview(withDigest(body));
}
export function inspectMemoryPlan(
value: unknown,
): { operationId: string; digest: string; needsContent: boolean } | null {
const plan = validatePlan(value);
return plan
? {
operationId: plan.request_id,
digest: plan.digest,
needsContent: plan.operation === "add" || plan.operation === "correct",
}
: null;
}
export function memoryPlanAcceptsContent(
value: unknown,
content: string | undefined,
): boolean {
const plan = validatePlan(value);
if (!plan) return false;
if (plan.operation !== "add" && plan.operation !== "correct") {
return content === undefined;
}
if (content === undefined) return false;
const identity = contentIdentity(content);
return (
identity.content_sha256 === plan.content_sha256 &&
identity.content_characters === plan.content_characters
);
}
export async function executeMemoryMutation(
value: unknown,
content: string | undefined,
operations: CoreOperations,
reader?: CoreReader,
): Promise<MemoryFlowDocument> {
const plan = validatePlan(value);
if (!plan) return invalid("memory_plan_invalid");
if (!memoryPlanAcceptsContent(plan, content)) {
return invalid("memory_content_mismatch");
}
const { path, body } = operationRequest(plan, content);
const outcome = await operations.post<unknown>(path, body);
if (outcome.state === "rejected") {
return {
schema_version: 1,
product: PRODUCT,
state: "rejected",
operation_id: plan.request_id,
code: outcome.code,
};
}
if (outcome.state === "unknown") {
return reader
? readMemoryMutationReceipt(reader, plan)
: pending(plan.request_id, outcome.code);
}
const memory = validateWriteResponse(outcome.data, plan);
if (!memory || !matchesReceipt(memory, plan, content)) {
return reader
? readMemoryMutationReceipt(reader, plan)
: pending(plan.request_id, "memory_receipt_invalid");
}
return {
schema_version: 1,
product: PRODUCT,
state: "committed",
operation_id: plan.request_id,
receipt: {
memory_id: memory.memory_id,
status: memory.status,
origin: memory.origin,
revision: memory.revision,
scopes: memory.scopes,
evidence_ids: memory.evidence_ids,
content_sha256: contentIdentity(memory.content).content_sha256,
content_characters: [...memory.content].length,
created_at: memory.created_at,
updated_at: memory.updated_at,
},
};
}
/**
* Validate the write response through the shared `parseMemoryReceipt`, which
* owns the GovernedMemory shape and the revision/status binding Core must
* satisfy. The CLI keeps only its own plan binding (`matchesReceipt`, which
* also checks `origin`, `kind` and the echoed content), so the two surfaces
* judge a write response identically and no shape rule is duplicated here.
*
* `add` creates the first revision and expects `confirmed`; the shared parser
* derives that from `expectedRevision: 0`. Every other operation advances the
* revision the caller acted against and expects `confirmed` (correct/scope)
* or `deleted` (delete).
*/
function validateWriteResponse(
value: unknown,
plan: MemoryMutationPlan,
): GovernedMemory | null {
const expectedStatus = plan.operation === "delete" ? "deleted" : "confirmed";
// For `add` there is no id or prior revision to bind yet, so the parser's
// own receipt is the only source for both; the plan binding that follows is
// what proves the response belongs to this request.
const memoryId =
plan.operation === "add"
? String((value as { memory_id?: unknown } | null)?.memory_id ?? "")
: (plan.memory_id as string);
if (memoryId === "") return null;
return parseMemoryReceipt(value, {
memoryId,
expectedRevision:
plan.operation === "add" ? 0 : (plan.expected_revision as number),
expectedStatus,
});
}
export async function readMemoryMutationReceipt(
reader: CoreReader,
value: unknown,
): Promise<MemoryFlowDocument> {
const plan = validatePlan(value);
if (!plan) return invalid("memory_plan_invalid");
const outcome = await reader<unknown>(
`/api/v1/memory-mutations/${encodeURIComponent(plan.request_id)}`,
);
if (outcome.state !== "ok") {
return pending(plan.request_id, `memory_mutation_receipt_${outcome.code}`);
}
const receipt = validateMutationReceipt(outcome.data, plan);
if (receipt === "not_found") {
return pending(
plan.request_id,
"memory_mutation_receipt_not_found_after_attempt",
);
}
if (!receipt) {
return pending(plan.request_id, "memory_mutation_receipt_invalid");
}
return {
schema_version: 1,
product: PRODUCT,
state: "recovered",
operation_id: plan.request_id,
receipt,
};
}
export function memoryFlowExitCode(document: MemoryFlowDocument): number {
return ["preview", "committed", "recovered"].includes(document.state) ? 0 : 1;
}
export function renderMemoryFlowText(document: MemoryFlowDocument): string {
if (document.state === "preview") {
const plan = document.plan;
const lines = [
`记忆操作预览:${plan.operation}`,
`request_id:${plan.request_id}`,
];
if (plan.memory_id) lines.push(`memory_id:${plan.memory_id}`);
if (plan.expected_revision) {
lines.push(
`当前版本:${plan.expected_revision} 状态:${plan.expected_status} 来源方式:${plan.expected_origin}`,
);
}
if (plan.scopes) lines.push(`使用范围:${plan.scopes.join(",")}`);
if (plan.current) {
for (const evidence of plan.current.evidence) {
lines.push(
`证据 ${evidence.evidence_id}:source=${evidence.source_id ?? "unknown"} author=${evidence.written_by ?? "unknown"} available=${evidence.available}`,
);
}
}
if (plan.content_characters !== undefined) {
lines.push(
`正文:${plan.content_characters} 字符,sha256=${plan.content_sha256}`,
);
}
lines.push("尚未写入;保存 JSON 计划后用 memories apply --confirm 执行。");
return `${lines.join("\n")}\n`;
}
if (document.state === "committed" || document.state === "recovered") {
return `记忆操作${document.state === "recovered" ? "已从治理回执恢复" : "已提交"}:${document.operation_id}\n${JSON.stringify(document.receipt)}\n`;
}
return `记忆操作未完成:${document.state}(${"code" in document ? document.code : "memory_operation_state_invalid"})${document.operation_id ? ` request_id=${document.operation_id}` : ""}\n`;
}
function operationRequest(
plan: MemoryMutationPlan,
content: string | undefined,
): { path: string; body: Record<string, unknown> } {
if (plan.operation === "add") {
// The shared create builder is the Web console's manual-add body, which
// always creates `chat` (the Web has no kind picker). The CLI exposes
// `--kind`, so it builds through the same builder and then sets the kind,
// keeping every other field (user_id/scope/evidence/origin) shared.
const request = buildMemoryCreateRequest(
{
content: content as string,
scopes: plan.scopes ?? [],
evidence_ids: plan.evidence_ids ?? [],
origin: "manual",
},
plan.request_id,
);
const draft = request.body.draft as Record<string, unknown>;
draft.kind = plan.memory_kind;
return { path: request.path, body: request.body };
}
if (plan.operation === "correct") {
return buildMemoryCorrectRequest(
plan.memory_id as string,
plan.expected_revision as number,
content as string,
undefined,
plan.request_id,
);
}
return buildMemoryActionRequest(
plan.memory_id as string,
plan.operation,
plan.expected_revision as number,
plan.request_id,
);
}
function matchesReceipt(
memory: GovernedMemory,
plan: MemoryMutationPlan,
content: string | undefined,
): boolean {
if (plan.operation === "add") {
return (
memory.status === "confirmed" &&
memory.origin === "manual" &&
memory.revision === 1 &&
memory.kind === plan.memory_kind &&
same(memory.scopes, plan.scopes ?? []) &&
same(memory.evidence_ids, plan.evidence_ids ?? []) &&
memory.content === content
);
}
if (
memory.memory_id !== plan.memory_id ||
memory.revision !== (plan.expected_revision as number) + 1 ||
memory.origin !== plan.expected_origin
) {
return false;
}
if (plan.operation === "delete" && memory.status !== "deleted") return false;
if (plan.operation === "correct" && memory.status !== plan.expected_status) {
return false;
}
if (plan.operation === "correct" && memory.content !== content) return false;
const expectedScopes =
plan.scopes ?? (plan.current?.scopes as string[] | undefined);
return expectedScopes ? same(memory.scopes, expectedScopes) : true;
}
function validateMutationReceipt(
value: unknown,
plan: MemoryMutationPlan,
): Record<string, unknown> | "not_found" | null {
const receipt = record(value);
if (
!receipt ||
Object.keys(receipt).length !== MUTATION_RECEIPT_KEYS.size ||
Object.keys(receipt).some((key) => !MUTATION_RECEIPT_KEYS.has(key)) ||
receipt.schema_version !== 2 ||
receipt.request_id !== plan.request_id
) {
return null;
}
if (receipt.state === "not_found") {
return [
receipt.operation,
receipt.memory_id,
receipt.result_revision,
receipt.result_status,
receipt.memory_state,
receipt.migrated_from,
].every((item) => item === null)
? "not_found"
: null;
}
const expectedOperation =
plan.operation === "add" ? "create" : plan.operation;
const expectedRevision =
plan.operation === "add" ? 1 : (plan.expected_revision as number) + 1;
const expectedStatus = plan.operation === "delete" ? "deleted" : "confirmed";
// A migrated receipt records an operation from before v2; no plan made
// today can have produced it.
if (
receipt.state !== "committed" ||
receipt.migrated_from !== null ||
receipt.operation !== expectedOperation ||
typeof receipt.memory_id !== "string" ||
!isMemoryId(receipt.memory_id) ||
(plan.memory_id !== undefined && receipt.memory_id !== plan.memory_id) ||
receipt.result_revision !== expectedRevision ||
receipt.result_status !== expectedStatus ||
!["present", "deleted"].includes(String(receipt.memory_state))
) {
return null;
}
return {
schema_version: 2,
request_id: plan.request_id,
state: "committed",
operation: expectedOperation,
memory_id: receipt.memory_id,
result_revision: expectedRevision,
result_status: expectedStatus,
memory_state: receipt.memory_state,
proof_scope: "governance_transaction_only",
downstream_sync_verified: false,
payload_fingerprint_verified: false,
};
}
function validatePlan(value: unknown): MemoryMutationPlan | null {
const plan = record(value);
if (
plan?.schema_version !== 1 ||
plan.kind !== "memory_mutation" ||
!oneOf(plan.operation, OPERATIONS) ||
typeof plan.request_id !== "string" ||
!UUID.test(plan.request_id) ||
typeof plan.digest !== "string"
) {
return null;
}
const body = { ...plan };
delete body.digest;
const expected = withDigest(body as unknown as PlanBody);
if (!safeEqual(expected.digest, plan.digest)) return null;
if (plan.operation === "add") {
if (
!oneOf(plan.memory_kind, MEMORY_KINDS) ||
!validScopes(plan.scopes) ||
!validIds(plan.evidence_ids, 500) ||
!contentFields(plan)
) {
return null;
}
} else if (
typeof plan.memory_id !== "string" ||
!isMemoryId(plan.memory_id) ||
!Number.isSafeInteger(plan.expected_revision) ||
(plan.expected_revision as number) < 1 ||
plan.expected_status !== "confirmed" ||
!oneOf(plan.expected_origin, MEMORY_ORIGINS) ||
!validCurrent(plan) ||
(plan.operation === "correct" && !contentFields(plan))
) {
return null;
}
return plan as unknown as MemoryMutationPlan;
}
function contentFields(plan: Record<string, unknown>): boolean {
return (
typeof plan.content_sha256 === "string" &&
/^[0-9a-f]{64}$/.test(plan.content_sha256) &&
Number.isSafeInteger(plan.content_characters) &&
(plan.content_characters as number) >= 1 &&
(plan.content_characters as number) <= CONTENT_MAX_CHARACTERS
);
}
function validCurrent(plan: Record<string, unknown>): boolean {
const current = record(plan.current);
if (
!current ||
current.status !== plan.expected_status ||
current.origin !== plan.expected_origin ||
current.revision !== plan.expected_revision ||
!validScopes(current.scopes) ||
!Array.isArray(current.evidence) ||
current.evidence.length < 1 ||
current.evidence.length > 500
) {
return false;
}
return current.evidence.every((value) => {
const evidence = record(value);
return (
evidence !== null &&
typeof evidence.evidence_id === "string" &&
ID.test(evidence.evidence_id) &&
typeof evidence.available === "boolean" &&
(evidence.source_id === undefined ||
(typeof evidence.source_id === "string" &&
evidence.source_id.length <= 128)) &&
(evidence.written_by === undefined ||
(typeof evidence.written_by === "string" &&
evidence.written_by.length <= 256))
);
});
}
function contentIdentity(content: string): {
content_sha256: string;
content_characters: number;
} {
return {
content_sha256: createHash("sha256").update(content, "utf8").digest("hex"),
content_characters: [...content].length,
};
}
function validContent(content: string): boolean {
return (
content.trim().length > 0 &&
[...content].length <= CONTENT_MAX_CHARACTERS &&
![...content].some((character) => {
const code = character.charCodeAt(0);
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
})
);
}
function validScopes(value: unknown): value is string[] {
if (
!Array.isArray(value) ||
value.length < 1 ||
value.length > 128 ||
new Set(value).size !== value.length ||
value.some(
(scope) =>
typeof scope !== "string" ||
scope.trim() !== scope ||
scope.length < 1 ||
scope.length > 512 ||
scope === "*",
)
) {
return false;
}
return !value.includes("private") || value.length === 1;
}
function validIds(value: unknown, maximum: number): value is string[] {
return (
Array.isArray(value) &&
value.length >= 1 &&
value.length <= maximum &&
new Set(value).size === value.length &&
value.every((item) => typeof item === "string" && ID.test(item))
);
}
function withDigest(body: PlanBody): MemoryMutationPlan {
return {
...body,
digest: createHash("sha256")
.update(JSON.stringify(body), "utf8")
.digest("hex"),
};
}
function preview(plan: MemoryMutationPlan): MemoryFlowDocument {
return {
schema_version: 1,
product: PRODUCT,
state: "preview",
plan,
};
}
function invalid(code: string): MemoryFlowDocument {
return {
schema_version: 1,
product: PRODUCT,
state: "invalid",
code,
};
}
function pending(operationId: string, code: string): MemoryFlowDocument {
return {
schema_version: 1,
product: PRODUCT,
state: "unknown",
operation_id: operationId,
code,
};
}
function same(left: readonly string[], right: readonly string[]): boolean {
return (
left.length === right.length &&
left.every((value, index) => value === right[index])
);
}
function safeEqual(left: string, right: string): boolean {
if (left.length !== right.length) return false;
return timingSafeEqual(Buffer.from(left), Buffer.from(right));
}
function oneOf<const T extends readonly string[]>(
value: unknown,
values: T,
): value is T[number] {
return typeof value === "string" && values.includes(value as T[number]);
}
function record(value: unknown): Record<string, unknown> | null {
return value !== null && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: null;
}
@@ -0,0 +1,211 @@
import { readFileSync } from "node:fs";
import { describe, expect, it } from "vitest";
import type { CoreReader } from "./core-client.js";
import {
planExistingMemoryMutation,
planMemoryAdd,
readMemoryMutationReceipt,
} from "./memory-flows.js";
// Written by a real Core and checked against it on the Core side, so these are
// the receipts the CLI meets in use rather than ones shaped to fit the parser.
const FIXTURE = JSON.parse(
readFileSync(
new URL(
"../../../packages/contracts/fixtures/memory-mutation-receipts.v2.json",
import.meta.url,
),
"utf8",
),
) as { receipts: Record<"add" | "correct" | "delete", Receipt> };
interface Receipt {
schema_version: number;
request_id: string;
memory_id: string;
result_revision: number;
result_status: string;
operation: string;
}
function reader(data: unknown): CoreReader {
return async <T>() => ({ state: "ok", data: data as T });
}
function detail(sample: Receipt) {
const revision = sample.result_revision - 1;
return {
memory: {
memory_id: sample.memory_id,
user_id: "local",
kind: "chat",
status: "confirmed",
content: "Synthetic memory.",
scopes: ["all-apps"],
evidence_ids: ["evidence-1"],
origin: "manual",
tags: [],
revision,
created_at: "2026-09-19T00:00:00Z",
updated_at: "2026-09-19T00:00:01Z",
deleted_at: null,
},
evidence: [
{
evidence_id: "evidence-1",
available: true,
source_id: "fixture",
written_by: "person",
created_at: "2026-09-19T00:00:00Z",
verbatim: "Synthetic source text.",
},
],
history: [
{
revision,
action: "create",
status: "confirmed",
content: "Synthetic memory.",
scopes: ["all-apps"],
created_at: "2026-09-19T00:00:01Z",
},
],
index_sync: null,
};
}
describe("memory mutation receipt v2 from Core", () => {
it("recovers an add from the receipt Core returns for a create", async () => {
const sample = FIXTURE.receipts.add;
expect(sample.schema_version).toBe(2);
const preview = planMemoryAdd({
content: "Synthetic memory.",
memoryKind: "chat",
scopes: ["all-apps"],
evidenceIds: ["evidence-1"],
});
if (preview.state !== "preview") throw new Error("preview expected");
// The request id is the one binding a receipt to its plan; the plan picks it.
const result = await readMemoryMutationReceipt(
reader({ ...sample, request_id: preview.plan.request_id }),
preview.plan,
);
expect(result).toMatchObject({
state: "recovered",
receipt: {
schema_version: 2,
operation: "create",
memory_id: sample.memory_id,
result_revision: 1,
result_status: "confirmed",
memory_state: "present",
},
});
});
it.each<["correct" | "delete", { content?: string }]>([
["correct", { content: "Corrected synthetic memory." }],
["delete", {}],
])(
"recovers %s from the receipt Core returns for it",
async (operation, extra) => {
const sample = FIXTURE.receipts[operation];
const preview = await planExistingMemoryMutation(reader(detail(sample)), {
operation,
memoryId: sample.memory_id,
...extra,
});
if (preview.state !== "preview") {
throw new Error(`preview expected: ${JSON.stringify(preview)}`);
}
const result = await readMemoryMutationReceipt(
reader({ ...sample, request_id: preview.plan.request_id }),
preview.plan,
);
expect(result).toMatchObject({
state: "recovered",
receipt: {
schema_version: 2,
operation: sample.operation,
memory_id: sample.memory_id,
result_revision: sample.result_revision,
result_status: sample.result_status,
},
});
},
);
it("does not take a v1 receipt for a v2 one", async () => {
const sample = FIXTURE.receipts.delete;
const preview = await planExistingMemoryMutation(reader(detail(sample)), {
operation: "delete",
memoryId: sample.memory_id,
});
if (preview.state !== "preview") throw new Error("preview expected");
const { migrated_from: _dropped, ...v1 } = sample as Receipt & {
migrated_from: null;
};
// As v1 was, and with v2's fields but v1's number: either alone is refused.
for (const receipt of [
{ ...v1, schema_version: 1 },
{ ...sample, schema_version: 1 },
]) {
const result = await readMemoryMutationReceipt(
reader({ ...receipt, request_id: preview.plan.request_id }),
preview.plan,
);
expect(result).toMatchObject({
state: "unknown",
code: "memory_mutation_receipt_invalid",
});
}
});
it("does not take a not-found receipt that names a migrated operation", async () => {
const sample = FIXTURE.receipts.delete;
const preview = await planExistingMemoryMutation(reader(detail(sample)), {
operation: "delete",
memoryId: sample.memory_id,
});
if (preview.state !== "preview") throw new Error("preview expected");
const notFound = {
schema_version: 2,
request_id: preview.plan.request_id,
state: "not_found",
operation: null,
memory_id: null,
result_revision: null,
result_status: null,
memory_state: null,
};
expect(
await readMemoryMutationReceipt(
reader({ ...notFound, migrated_from: null }),
preview.plan,
),
).toMatchObject({
code: "memory_mutation_receipt_not_found_after_attempt",
});
expect(
await readMemoryMutationReceipt(
reader({ ...notFound, migrated_from: "reject" }),
preview.plan,
),
).toMatchObject({ code: "memory_mutation_receipt_invalid" });
});
it("plans nothing on a memory that is not in effect", async () => {
const sample = FIXTURE.receipts.delete;
const superseded = detail(sample);
superseded.memory.status = "superseded";
superseded.history[0].status = "superseded";
const result = await planExistingMemoryMutation(reader(superseded), {
operation: "delete",
memoryId: sample.memory_id,
});
expect(result).toMatchObject({
state: "invalid",
code: "memory_operation_not_allowed",
});
});
});
+21 -267
View File
@@ -1,17 +1,5 @@
import { randomUUID } from "node:crypto";
import type {
MemoryAddReceipt,
SdkMemoryPage,
SdkMemoryRecord,
} from "@one-memory/contracts";
import {
buildMemoryAddRequest,
MemoryAddInputError,
memoryAddStatusPath,
parseMemoryAddReceipt,
} from "@one-memory/operations";
import type { SdkMemoryPage, SdkMemoryRecord } from "@one-memory/contracts";
import { type CoreReader, RESERVED_MEMORY_SEGMENTS } from "./core-client.js";
import type { CoreOperations } from "./core-operations.js";
import { PRODUCT } from "./version.js";
/** Output budget: a long memory is shortened visibly, never dropped silently. */
@@ -27,75 +15,27 @@ export function isMemoryId(value: string): boolean {
);
}
const CONTENT_MAX_CHARACTERS = 100_000;
/** A memory body read from `--content-file`: UTF-8, non-blank, bounded, and
* free of control characters other than tab and newline. */
export function parseMemoryContent(bytes: Buffer): string | null {
if (bytes.byteLength > CONTENT_MAX_CHARACTERS * 4 + 2) return null;
let decoded: string;
try {
decoded = new TextDecoder("utf-8", { fatal: true }).decode(bytes);
} catch {
return null;
}
const content = decoded.replace(/\r?\n$/, "");
return content.trim().length > 0 &&
[...content].length <= CONTENT_MAX_CHARACTERS &&
![...content].some((character) => {
const code = character.charCodeAt(0);
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
})
? content
: null;
}
export const MEMORY_SCENES = ["all", "chat", "working", "coding"] as const;
export type MemoryScene = (typeof MEMORY_SCENES)[number];
export const MEMORY_KINDS = ["chat", "experience", "fact"] as const;
export type MemoryKind = (typeof MEMORY_KINDS)[number];
export const MEMORY_VARIANTS = ["working", "coding"] as const;
export type MemoryVariant = (typeof MEMORY_VARIANTS)[number];
/** Which SDK namespace a memory lives in. A chat memory needs neither; a
* working/coding experience or fact needs both so Core does not have to guess
* (an id present in two namespaces is refused as ambiguous). */
export interface MemoryNamespace {
kind?: MemoryKind;
variant?: MemoryVariant;
}
export function namespaceParameters(
namespace: MemoryNamespace,
): Record<string, string> {
return {
...(namespace.kind === undefined ? {} : { kind: namespace.kind }),
...(namespace.variant === undefined ? {} : { variant: namespace.variant }),
};
}
export interface MemoryQuery {
query?: string;
status?: string;
source?: string;
scene?: MemoryScene;
offset: number;
limit: number;
}
/** Every `memories` document is schema 2: list/show share it with
* add/update/delete, and the v1-era `evidence`/`history` fields are gone. */
export interface MemoryPageDocument {
schema_version: 2;
schema_version: 1;
product: string;
page: Record<string, unknown>;
memories: Record<string, unknown>[];
}
export interface MemoryDetailDocument {
schema_version: 2;
schema_version: 1;
product: string;
memory: Record<string, unknown>;
evidence: Record<string, unknown>[];
history: Record<string, unknown>[];
}
export async function readMemoryPage(
@@ -107,15 +47,10 @@ export async function readMemoryPage(
limit: query.limit,
};
if (query.status !== undefined) parameters.statuses = query.status;
if (query.source !== undefined) parameters.source = query.source;
if (query.query !== undefined) parameters.query = query.query;
if (query.scene !== undefined && query.scene !== "all") {
parameters.scene = query.scene;
}
const outcome = await reader<unknown>("/api/v2/memory-library", parameters);
if (outcome.state !== "ok") {
return {
schema_version: 2,
schema_version: 1,
product: PRODUCT,
page: { state: outcome.state, code: outcome.code },
memories: [],
@@ -124,7 +59,7 @@ export async function readMemoryPage(
const page = sdkMemoryPage(outcome.data);
if (page === null) {
return {
schema_version: 2,
schema_version: 1,
product: PRODUCT,
page: { state: "invalid", code: "response_invalid" },
memories: [],
@@ -132,21 +67,15 @@ export async function readMemoryPage(
}
const items = page.memories;
return {
schema_version: 2,
schema_version: 1,
product: PRODUCT,
page: {
state: "ok",
scene: query.scene ?? "all",
...(query.query === undefined ? {} : { query: query.query }),
total: page.total,
offset: page.offset,
limit: page.limit,
returned: items.length,
has_more: page.offset + items.length < page.total,
// A keyword search only looks among this many most relevant memories.
...(typeof page.search_window === "number"
? { search_window: page.search_window }
: {}),
},
memories: items.map((item) => describeMemory(item)),
};
@@ -155,195 +84,42 @@ export async function readMemoryPage(
export async function readMemoryDetail(
reader: CoreReader,
memoryId: string,
namespace: MemoryNamespace = {},
): Promise<MemoryDetailDocument> {
const parameters = namespaceParameters(namespace);
const outcome = await reader<unknown>(
`/api/v2/memories/${memoryId}`,
Object.keys(parameters).length > 0 ? parameters : undefined,
);
const outcome = await reader<unknown>(`/api/v2/memories/${memoryId}`);
if (outcome.state !== "ok") {
return {
schema_version: 2,
schema_version: 1,
product: PRODUCT,
memory: { state: outcome.state, code: outcome.code },
evidence: [],
history: [],
};
}
const memory = sdkMemory(outcome.data);
if (memory === null || memory.memory_id !== memoryId) {
return {
schema_version: 2,
schema_version: 1,
product: PRODUCT,
memory: { state: "invalid", code: "response_invalid" },
evidence: [],
history: [],
};
}
return {
schema_version: 2,
schema_version: 1,
product: PRODUCT,
memory: { state: "ok", ...describeMemory(memory) },
evidence: [],
history: [],
};
}
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
/** The same bound the app's 添加记忆 box applies. */
export const MEMORY_ADD_MAX_CHARACTERS = 12_000;
export interface MemoryAddDocument {
schema_version: 2;
product: string;
command: "memories add" | "memories add-status";
state: "accepted" | "rejected" | "unknown" | "not_found" | "invalid";
request_id: string;
code?: string;
receipt?: Record<string, unknown>;
}
export function isRequestId(value: string): boolean {
return UUID.test(value);
}
/** A memory text: non-blank, bounded, no control characters except tab and
* newline. */
export function validMemoryText(text: string): boolean {
return (
text.trim().length > 0 &&
[...text].length <= MEMORY_ADD_MAX_CHARACTERS &&
![...text].some((character) => {
const code = character.charCodeAt(0);
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
})
);
}
function addReceipt(receipt: MemoryAddReceipt): Record<string, unknown> {
return {
evidence_id: receipt.evidence_id,
state: receipt.state,
created_at: receipt.created_at,
task_id: receipt.task_id,
memory_ids: receipt.memory_ids,
error: receipt.error,
};
}
/**
* Add one memory directly, the same request as the app's 添加记忆.
*
* `requestId` is Core's idempotency key: retrying with the same id and text
* returns the first receipt; the same id with other text is refused.
*/
export async function addMemory(
operations: CoreOperations,
text: string,
requestId: string = randomUUID(),
): Promise<MemoryAddDocument> {
const base = {
schema_version: 2 as const,
product: PRODUCT,
command: "memories add" as const,
request_id: requestId,
};
if (!validMemoryText(text) || !UUID.test(requestId)) {
return { ...base, state: "invalid", code: "memory_add_invalid" };
}
let request: ReturnType<typeof buildMemoryAddRequest>;
try {
request = buildMemoryAddRequest({
request_id: requestId,
text,
scope: "all-apps",
});
} catch (error) {
if (error instanceof MemoryAddInputError) {
return { ...base, state: "invalid", code: "memory_add_invalid" };
}
throw error;
}
const result = await operations.post<unknown>(request.path, request.body);
if (result.state !== "committed") {
return { ...base, state: result.state, code: result.code };
}
const receipt = parseMemoryAddReceipt(result.data);
return receipt === null
? { ...base, state: "unknown", code: "memory_add_receipt_invalid" }
: { ...base, state: "accepted", receipt: addReceipt(receipt) };
}
/** Read how one addition is doing, by the request id it was submitted with. */
export async function readMemoryAddStatus(
reader: CoreReader,
requestId: string,
): Promise<MemoryAddDocument> {
const base = {
schema_version: 2 as const,
product: PRODUCT,
command: "memories add-status" as const,
request_id: requestId,
};
if (!UUID.test(requestId)) {
return { ...base, state: "invalid", code: "invalid_request_id" };
}
const outcome = await reader<unknown>(memoryAddStatusPath(requestId));
if (outcome.state === "not_found") {
return { ...base, state: "not_found", code: "memory_add_not_found" };
}
if (outcome.state !== "ok") {
return {
...base,
state: outcome.state === "rejected" ? "rejected" : "unknown",
code: outcome.code,
};
}
const receipt = parseMemoryAddReceipt(outcome.data);
return receipt === null
? { ...base, state: "unknown", code: "memory_add_receipt_invalid" }
: { ...base, state: "accepted", receipt: addReceipt(receipt) };
}
export function memoryAddExitCode(document: MemoryAddDocument): number {
return document.state === "accepted" ? 0 : 1;
}
export function renderMemoryAddText(document: MemoryAddDocument): string {
if (document.state === "accepted" && document.receipt) {
return [
`request_id: ${document.request_id}`,
`evidence_id: ${document.receipt.evidence_id}`,
`state: ${document.receipt.state}`,
...((document.receipt.memory_ids as string[]).length > 0
? [`memory_ids: ${(document.receipt.memory_ids as string[]).join(",")}`]
: []),
document.command === "memories add"
? `查询进度:memhub memories add-status ${document.request_id};重试请带上同一个 --request-id,不会重复添加。`
: "",
"",
]
.filter((line, index, all) => line !== "" || index === all.length - 1)
.join("\n");
}
if (document.state === "not_found") {
return `未找到该 request id:${document.request_id}\n`;
}
return [
`添加记忆未完成:${document.state}(${document.code})`,
document.state === "unknown"
? `结果未知;可以用同一个 --request-id ${document.request_id} 安全重试。`
: "",
"",
]
.filter((line, index, all) => line !== "" || index === all.length - 1)
.join("\n");
}
export function renderMemoryPageText(document: MemoryPageDocument): string {
const page = document.page;
if (page.state !== "ok") {
if (page.state === "unavailable") {
return "后台正忙,记忆列表这次没查完,不代表没有记忆,请稍后再试\n";
}
if (page.state === "rejected") {
return `记忆查询被拒绝(${page.code}),请检查参数。\n`;
}
return `记忆查询未完成:${page.state}(${page.code})\n`;
}
const lines = [
@@ -351,16 +127,9 @@ export function renderMemoryPageText(document: MemoryPageDocument): string {
page.has_more ? ",还有更多,请用 --offset 继续" : ""
}`,
];
if (typeof page.search_window === "number") {
lines.push(`(搜索只在最相关的 ${page.search_window} 条里匹配关键词)`);
}
for (const memory of document.memories) {
const namespace =
memory.kind === "chat"
? ""
: ` kind=${memory.kind} variant=${memory.variant}`;
lines.push(
`- ${memory.memory_id} [${memory.scene}] ${memory.status}/${memory.layer} source=${memory.source ?? "未知"}${namespace}`,
`- ${memory.memory_id} ${memory.status}/${memory.layer} source=${memory.source ?? "未知"}`,
` ${memory.content}${memory.content_truncated ? ` …(共 ${memory.content_characters} 字)` : ""}`,
);
}
@@ -375,14 +144,10 @@ export function renderMemoryDetailText(document: MemoryDetailDocument): string {
if (memory.state === "unavailable") {
return "后台正忙,记忆详情这次没查完,不代表没有这条记忆,请稍后再试\n";
}
if (memory.code === "memory_id_ambiguous") {
return "这个 ID 同时存在于多个场景;请加 --kind(以及 --variant)指明是哪一条。\n";
}
return `记忆读取未完成:${memory.state}(${memory.code})\n`;
}
const lines = [
`记忆 ${memory.memory_id}`,
`场景:${memory.scene} 类型:${memory.kind}${memory.variant ? `(${memory.variant})` : ""}`,
`状态:${memory.status} 层:${memory.layer} 来源:${memory.source ?? "未知"}`,
`owner:${memory.owner ?? "未知"} latest:${memory.is_latest}`,
`raw IDs:${(memory.source_raw_ids as string[]).join(",") || "无"}`,
@@ -400,10 +165,6 @@ function describeMemory(memory: SdkMemoryRecord): Record<string, unknown> {
const budgeted = budget(memory.content ?? "");
return {
memory_id: memory.memory_id,
// Pre-2.0.6 Core listed chat memories only and named no scene.
scene: memory.scene ?? "chat",
kind: memory.kind ?? "chat",
variant: memory.variant ?? null,
status: memory.status,
layer: memory.layer,
is_latest: memory.is_latest,
@@ -415,9 +176,6 @@ function describeMemory(memory: SdkMemoryRecord): Record<string, unknown> {
tags: memory.tags ?? [],
memory_at: memory.memory_at,
gmt_created: memory.gmt_created,
...(memory.task === undefined || memory.task === null
? {}
: { task: memory.task }),
content: budgeted.text,
content_truncated: budgeted.truncated,
content_characters: budgeted.characters,
@@ -435,9 +193,6 @@ function sdkMemoryPage(value: unknown): SdkMemoryPage | null {
total: value.total,
limit: value.limit,
offset: value.offset,
...(integer(value.search_window)
? { search_window: value.search_window }
: {}),
};
}
@@ -453,9 +208,8 @@ function sdkMemory(value: unknown): SdkMemoryRecord | null {
!strings(value.superseded_by) ||
!strings(value.tags) ||
!strings(value.source_raw_ids) ||
// Working/coding memories may carry an empty user or agent id.
typeof value.user_id !== "string" ||
typeof value.agent_id !== "string"
!text(value.user_id) ||
!text(value.agent_id)
)
return null;
return value as unknown as SdkMemoryRecord;
+64 -89
View File
@@ -3,9 +3,9 @@ import { describe, expect, it, vi } from "vitest";
import type { CoreReader } from "./core-client.js";
import type { CoreOperations } from "./core-operations.js";
import {
modelProbeExitCode,
executeModelProbe,
planModelProbe,
readModelProbeReceipt,
startModelProbe,
} from "./model-probe-flow.js";
const PROFILE_ID = "22222222-2222-4222-8222-222222222222";
@@ -83,101 +83,67 @@ function completed(operationId: string) {
};
}
const OPERATION_ID = "33333333-3333-4333-8333-333333333333";
/** Reader that answers the profile list, then the queued operation reads. */
function readerWith(...operationReads: unknown[]) {
const reader = vi.fn(async (path: string) => {
if (path === "/api/v1/model-profiles") {
return { state: "ok", data: [PROFILE] };
}
return operationReads.shift() ?? { state: "not_found", code: "http_404" };
});
return reader;
}
describe("model probe operation flow", () => {
it("probes the exact current profile revision once support is proven", async () => {
const reader = readerWith({ state: "ok", data: notFound(OPERATION_ID) });
it("plans against the exact current profile revision", async () => {
const reader = vi.fn().mockResolvedValue({ state: "ok", data: [PROFILE] });
const document = await planModelProbe(reader as CoreReader, PROFILE_ID);
expect(document.state).toBe("preview");
if (document.state !== "preview") throw new Error("expected preview");
expect(document.plan).toMatchObject({
kind: "model_profile_probe",
profile_id: PROFILE_ID,
expected_profile_revision: 3,
});
expect(document.plan.operation_id).toMatch(
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/,
);
expect(document.plan.digest).toMatch(/^[0-9a-f]{64}$/);
});
it("proves operation support with schema-1 not_found before posting", async () => {
const planned = await planModelProbe(
vi.fn().mockResolvedValue({ state: "ok", data: [PROFILE] }) as CoreReader,
PROFILE_ID,
);
if (planned.state !== "preview") throw new Error("expected preview");
const operationId = planned.plan.operation_id;
const reader = vi
.fn()
.mockResolvedValueOnce({ state: "ok", data: notFound(operationId) });
const post = vi
.fn()
.mockResolvedValue({ state: "committed", data: completed(OPERATION_ID) });
.mockResolvedValue({ state: "committed", data: completed(operationId) });
const result = await startModelProbe(
reader as unknown as CoreReader,
const result = await executeModelProbe(
planned.plan,
reader as CoreReader,
{ post } as CoreOperations,
PROFILE_ID,
OPERATION_ID,
);
expect(result.state).toBe("completed");
expect(reader).toHaveBeenCalledWith(
`/api/v1/model-probe-operations/${OPERATION_ID}`,
`/api/v1/model-probe-operations/${operationId}`,
);
expect(post).toHaveBeenCalledWith(
`/api/v1/model-profiles/${PROFILE_ID}/probe`,
{ operation_id: OPERATION_ID, expected_profile_revision: 3 },
{ operation_id: operationId, expected_profile_revision: 3 },
);
});
it("generates its own operation id and treats a pending probe as submitted", async () => {
const post = vi.fn(
async (_path: string, body: Record<string, unknown>) => ({
state: "committed",
data: {
...notFound(String(body.operation_id)),
profile_id: PROFILE_ID,
expected_profile_revision: 3,
state: "pending",
stale: false,
},
}),
);
const reader = vi.fn(async (path: string) =>
path === "/api/v1/model-profiles"
? { state: "ok", data: [PROFILE] }
: {
state: "ok",
data: notFound(path.split("/").at(-1) ?? ""),
},
);
const result = await startModelProbe(
reader as unknown as CoreReader,
{ post } as unknown as CoreOperations,
PROFILE_ID,
);
expect(result.state).toBe("pending");
expect(result.operation_id).toMatch(
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/,
);
expect(modelProbeExitCode(result)).toBe(0);
});
it("refuses an unknown profile without posting", async () => {
const post = vi.fn();
const result = await startModelProbe(
readerWith() as unknown as CoreReader,
{ post } as CoreOperations,
"44444444-4444-4444-8444-444444444444",
OPERATION_ID,
);
expect(result).toMatchObject({
state: "not_found",
code: "model_profile_not_found",
});
expect(post).not.toHaveBeenCalled();
});
it("does not post when the Core lacks the exact operation contract", async () => {
const planned = await planModelProbe(
vi.fn().mockResolvedValue({ state: "ok", data: [PROFILE] }) as CoreReader,
PROFILE_ID,
);
if (planned.state !== "preview") throw new Error("expected preview");
const post = vi.fn();
const result = await startModelProbe(
readerWith({
const result = await executeModelProbe(
planned.plan,
vi.fn().mockResolvedValue({
state: "not_found",
code: "http_404",
}) as unknown as CoreReader,
}) as CoreReader,
{ post } as CoreOperations,
PROFILE_ID,
OPERATION_ID,
);
expect(result).toMatchObject({
state: "disabled",
@@ -187,15 +153,19 @@ describe("model probe operation flow", () => {
});
it("does not post when a legacy Core returns a non-operation body", async () => {
const planned = await planModelProbe(
vi.fn().mockResolvedValue({ state: "ok", data: [PROFILE] }) as CoreReader,
PROFILE_ID,
);
if (planned.state !== "preview") throw new Error("expected preview");
const post = vi.fn();
const result = await startModelProbe(
readerWith({
const result = await executeModelProbe(
planned.plan,
vi.fn().mockResolvedValue({
state: "ok",
data: { detail: "legacy_route_shape" },
}) as unknown as CoreReader,
}) as CoreReader,
{ post } as CoreOperations,
PROFILE_ID,
OPERATION_ID,
);
expect(result).toMatchObject({
state: "disabled",
@@ -205,19 +175,24 @@ describe("model probe operation flow", () => {
});
it("reconciles an unknown POST by GET and never treats not_found as replay permission", async () => {
const reader = readerWith(
{ state: "ok", data: notFound(OPERATION_ID) },
{ state: "ok", data: notFound(OPERATION_ID) },
const planned = await planModelProbe(
vi.fn().mockResolvedValue({ state: "ok", data: [PROFILE] }) as CoreReader,
PROFILE_ID,
);
if (planned.state !== "preview") throw new Error("expected preview");
const operationId = planned.plan.operation_id;
const reader = vi
.fn()
.mockResolvedValueOnce({ state: "ok", data: notFound(operationId) })
.mockResolvedValueOnce({ state: "ok", data: notFound(operationId) });
const post = vi.fn().mockResolvedValue({
state: "unknown",
code: "transport_result_unknown",
});
const result = await startModelProbe(
reader as unknown as CoreReader,
const result = await executeModelProbe(
planned.plan,
reader as CoreReader,
{ post } as CoreOperations,
PROFILE_ID,
OPERATION_ID,
);
expect(result).toMatchObject({
state: "unknown",
+124 -42
View File
@@ -1,10 +1,11 @@
import { randomUUID } from "node:crypto";
import { createHash, randomUUID } from "node:crypto";
import type { CoreReader, ReadOutcome } from "./core-client.js";
import type { CoreOperations } from "./core-operations.js";
import { validateModelProfiles } from "./core-responses.js";
import { PRODUCT } from "./version.js";
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const DIGEST = /^[0-9a-f]{64}$/;
const RECEIPT_KEYS = new Set([
"schema_version",
"operation_id",
@@ -58,7 +59,23 @@ const CODES = new Set([
"embedding_unavailable",
]);
interface ModelProbePlanBody {
schema_version: 1;
kind: "model_profile_probe";
operation_id: string;
profile_id: string;
expected_profile_revision: number;
}
export type ModelProbePlan = ModelProbePlanBody & { digest: string };
export type ModelProbeFlowDocument =
| {
schema_version: 1;
product: string;
state: "preview";
plan: ModelProbePlan;
}
| {
schema_version: 1;
product: string;
@@ -74,25 +91,11 @@ export type ModelProbeFlowDocument =
code: string;
};
/**
* Start one capability probe of a model profile (one real model request).
*
* Reads the profile's current revision so Core can refuse a probe of a
* profile that changed meanwhile, checks that this Core tracks probe
* operations (an old Core would start an untracked request), then submits it.
* The probe runs asynchronously: the answer is usually `pending`, and
* `models receipt <operation-id>` reads the result.
*/
export async function startModelProbe(
export async function planModelProbe(
reader: CoreReader,
operations: CoreOperations,
profileId: string,
operationId: string = randomUUID(),
): Promise<ModelProbeFlowDocument> {
if (!UUID.test(profileId)) return failure("invalid", "invalid_profile_id");
if (!UUID.test(operationId)) {
return failure("invalid", "invalid_probe_operation_id");
}
const outcome = await reader<unknown>("/api/v1/model-profiles");
if (outcome.state !== "ok") return fromReadFailure(outcome);
const profiles = validateModelProfiles(outcome.data);
@@ -102,46 +105,71 @@ export async function startModelProbe(
);
if (matches.length !== 1)
return failure("not_found", "model_profile_not_found");
const revision = matches[0].revision;
if (!Number.isSafeInteger(revision) || revision < 1) {
const profile = matches[0];
if (!Number.isSafeInteger(profile.revision) || profile.revision < 1) {
return failure("invalid", "model_profile_revision_invalid");
}
const plan = withDigest({
schema_version: 1 as const,
kind: "model_profile_probe" as const,
operation_id: randomUUID(),
profile_id: profileId,
expected_profile_revision: profile.revision,
});
return { schema_version: 1, product: PRODUCT, state: "preview", plan };
}
export async function executeModelProbe(
value: unknown,
reader: CoreReader,
operations: CoreOperations,
): Promise<ModelProbeFlowDocument> {
const plan = validatePlan(value);
if (!plan) return failure("invalid", "model_probe_plan_invalid");
// A 404 or a legacy/malformed body is not support. Some old Core versions
// ignore the operation body and start an untracked external model request.
const support = await reader<unknown>(operationPath(operationId));
const support = await reader<unknown>(operationPath(plan.operation_id));
if (support.state !== "ok") {
return failure(
"disabled",
"model_probe_operations_unsupported",
operationId,
plan.operation_id,
);
}
const supportReceipt = validateReceipt(support.data, operationId);
const supportReceipt = validateReceipt(support.data, plan.operation_id);
if (supportReceipt?.state !== "not_found") {
return failure(
"disabled",
"model_probe_operations_unsupported",
operationId,
plan.operation_id,
);
}
const posted = await operations.post<unknown>(
`/api/v1/model-profiles/${profileId}/probe`,
{ operation_id: operationId, expected_profile_revision: revision },
const outcome = await operations.post<unknown>(
`/api/v1/model-profiles/${plan.profile_id}/probe`,
{
operation_id: plan.operation_id,
expected_profile_revision: plan.expected_profile_revision,
},
);
if (posted.state === "rejected") {
return failure("rejected", posted.code, operationId);
if (outcome.state === "rejected") {
return failure("rejected", outcome.code, plan.operation_id);
}
if (posted.state === "committed") {
return receiptDocument(posted.data, operationId, profileId, revision);
if (outcome.state === "committed") {
return receiptDocument(
outcome.data,
plan.operation_id,
plan.profile_id,
plan.expected_profile_revision,
);
}
const reconciled = await readModelProbeReceipt(reader, operationId);
const reconciled = await readModelProbeReceipt(reader, plan.operation_id);
return reconciled.state === "not_found"
? failure(
"unknown",
"model_probe_receipt_not_found_after_attempt",
operationId,
plan.operation_id,
)
: reconciled;
}
@@ -162,24 +190,32 @@ export async function readModelProbeReceipt(
return receiptDocument(outcome.data, operationId);
}
export function inspectModelProbePlan(value: unknown): ModelProbePlan | null {
return validatePlan(value);
}
export function modelProbeExitCode(document: ModelProbeFlowDocument): number {
// `pending` is a submitted probe still running: the submit succeeded.
return ["completed", "pending"].includes(document.state) ? 0 : 1;
return ["preview", "completed"].includes(document.state) ? 0 : 1;
}
export function renderModelProbeText(document: ModelProbeFlowDocument): string {
const lines = [
if (document.state === "preview") {
return [
"预览:模型能力探测",
`模型配置:${document.plan.profile_id}`,
`预期版本:${document.plan.expected_profile_revision}`,
`操作 ID:${document.plan.operation_id}`,
`确认摘要:${document.plan.digest}`,
"未发起模型请求。请保存 --json 输出,再用 models probe --plan-file <path> --confirm。",
"",
].join("\n");
}
return [
`探测状态:${document.state}`,
`操作 ID:${document.operation_id ?? "无"}`,
"code" in document ? `代码:${document.code}` : "已返回有界操作回执。",
];
if (document.state === "pending" && document.operation_id) {
lines.push(
`探测在后台进行;用 memhub models receipt ${document.operation_id} 查看结果。`,
);
}
lines.push("");
return lines.join("\n");
"",
].join("\n");
}
function receiptDocument(
@@ -213,6 +249,31 @@ function receiptDocument(
};
}
function validatePlan(value: unknown): ModelProbePlan | null {
const plan = record(value);
if (
plan?.schema_version !== 1 ||
plan.kind !== "model_profile_probe" ||
typeof plan.operation_id !== "string" ||
!UUID.test(plan.operation_id) ||
typeof plan.profile_id !== "string" ||
!UUID.test(plan.profile_id) ||
!positiveInteger(plan.expected_profile_revision) ||
typeof plan.digest !== "string" ||
!DIGEST.test(plan.digest)
) {
return null;
}
const expected = withDigest({
schema_version: 1 as const,
kind: "model_profile_probe" as const,
operation_id: plan.operation_id,
profile_id: plan.profile_id,
expected_profile_revision: plan.expected_profile_revision,
});
return expected.digest === plan.digest ? expected : null;
}
function validateReceipt(
value: unknown,
operationId: string,
@@ -348,6 +409,27 @@ function failure(
};
}
function withDigest<T extends object>(body: T): T & { digest: string } {
return {
...body,
digest: createHash("sha256").update(canonical(body)).digest("hex"),
};
}
function canonical(value: unknown): string {
if (Array.isArray(value)) return `[${value.map(canonical).join(",")}]`;
if (value && typeof value === "object") {
return `{${Object.keys(value)
.sort()
.map(
(key) =>
`${JSON.stringify(key)}:${canonical((value as Record<string, unknown>)[key])}`,
)
.join(",")}}`;
}
return JSON.stringify(value);
}
function record(value: unknown): Record<string, unknown> | null {
return value !== null && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
+2 -2
View File
@@ -41,7 +41,7 @@ export type ModelReportDocument =
| {
schema_version: 1;
product: string;
state: "denied" | "unavailable" | "not_found" | "invalid" | "rejected";
state: "denied" | "unavailable" | "not_found" | "invalid";
code: string;
};
@@ -136,7 +136,7 @@ function project(profile: ModelProfile): SafeModelProfile {
}
function failure(
state: "denied" | "unavailable" | "not_found" | "invalid" | "rejected",
state: "denied" | "unavailable" | "not_found" | "invalid",
code: string,
): ModelReportDocument {
return { schema_version: 1, product: PRODUCT, state, code };
+44 -17
View File
@@ -126,12 +126,29 @@ describe("models activate", () => {
expect(runtime.inspectRuntimeStatus).not.toHaveBeenCalled();
});
it("posts the activate endpoint directly and reports the profile", async () => {
it("previews without contacting Core", async () => {
const result = await capturedRun(["models", "activate", PROFILE, "--json"]);
expect(result.code).toBe(0);
const document = JSON.parse(result.stdout);
expect(document.state).toBe("preview");
expect(document.plan.kind).toBe("model_profile_activate");
// Activation has no CAS body, so the plan must not pretend to carry one.
expect(document.plan).not.toHaveProperty("expected_revision");
expect(transport.post).not.toHaveBeenCalled();
});
it("posts the activate endpoint with --confirm and reports the profile", async () => {
transport.post.mockResolvedValue({
status: 200,
data: profileResponse(),
});
const result = await capturedRun(["models", "activate", PROFILE, "--json"]);
const result = await capturedRun([
"models",
"activate",
PROFILE,
"--confirm",
"--json",
]);
expect(result.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
`/api/v1/model-profiles/${PROFILE}/activate`,
@@ -179,22 +196,31 @@ describe("models activate", () => {
});
});
describe("models remove", () => {
it("previews without --yes and changes nothing", async () => {
const result = await capturedRun(["models", "remove", PROFILE, "--json"]);
expect(result.code).toBe(0);
expect(JSON.parse(result.stdout)).toMatchObject({
command: "models remove",
state: "preview",
target: PROFILE,
applied: false,
});
expect(runtime.inspectRuntimeStatus).not.toHaveBeenCalled();
});
});
describe("models edit", () => {
it("sends the exact patch body with the CAS revision directly", async () => {
it("previews a patch from a config file without writing", async () => {
const directory = await mkdtemp(path.join(tmpdir(), "om-model-edit-"));
const config = path.join(directory, "patch.json");
await writeFile(
config,
JSON.stringify({ expected_revision: 4, name: "Renamed" }),
{ mode: 0o600 },
);
const result = await capturedRun([
"models",
"edit",
PROFILE,
"--config-file",
config,
"--json",
]);
expect(result.code).toBe(0);
const document = JSON.parse(result.stdout);
expect(document.state).toBe("preview");
expect(document.plan.patch).toEqual({ name: "Renamed" });
expect(transport.patch).not.toHaveBeenCalled();
});
it("sends the exact patch body with the CAS revision on confirm", async () => {
const directory = await mkdtemp(path.join(tmpdir(), "om-model-edit-"));
const config = path.join(directory, "patch.json");
await writeFile(
@@ -216,6 +242,7 @@ describe("models edit", () => {
PROFILE,
"--config-file",
config,
"--confirm",
"--json",
]);
expect(result.code).toBe(0);
+196 -40
View File
@@ -3,12 +3,14 @@
* activate (`POST /model-profiles/{id}/activate`) and edit
* (`PATCH /model-profiles/{id}`).
*
* Core owns the semantics. Both run directly: activation is reversible (activate
* another profile) and an edit is guarded by `expected_revision`, the
* compare-and-swap Core requires, mirroring the Web console's
* `ModelProfilePatch`.
* Core owns the semantics. These flows reuse the same two-phase convention as
* `models configure`: plan without `--confirm` (or produce a plan file), then
* apply exactly that plan. The edit body mirrors the Web console's
* `ModelProfilePatch`, including `expected_revision` as the compare-and-swap
* guard Core requires.
*/
import { createHash, randomUUID } from "node:crypto";
import type { ModelProfile } from "@one-memory/contracts";
import {
buildModelProfileUpdateRequest,
@@ -20,7 +22,29 @@ import { PRODUCT } from "./version.js";
const PROFILE_ID = /^[A-Za-z0-9][\w.-]{0,127}$/;
export type ModelWritePlan =
| {
schema_version: 1;
kind: "model_profile_activate";
profile_id: string;
digest: string;
}
| {
schema_version: 1;
kind: "model_profile_edit";
profile_id: string;
expected_revision: number;
patch: Record<string, unknown>;
digest: string;
};
export type ModelWriteDocument =
| {
schema_version: 1;
product: string;
state: "preview";
plan: ModelWritePlan;
}
| {
schema_version: 1;
product: string;
@@ -36,6 +60,38 @@ export type ModelWriteDocument =
code: string;
};
function canonical(value: unknown): string {
if (Array.isArray(value)) return `[${value.map(canonical).join(",")}]`;
if (value !== null && typeof value === "object") {
return `{${Object.entries(value as Record<string, unknown>)
.sort(([left], [right]) => left.localeCompare(right))
.map(([key, item]) => `${JSON.stringify(key)}:${canonical(item)}`)
.join(",")}}`;
}
if (value === undefined) return "null";
return JSON.stringify(value);
}
function digestOf(value: unknown): string {
return createHash("sha256").update(canonical(value)).digest("hex");
}
function preview(
plan:
| Omit<
Extract<ModelWritePlan, { kind: "model_profile_activate" }>,
"digest"
>
| Omit<Extract<ModelWritePlan, { kind: "model_profile_edit" }>, "digest">,
): ModelWriteDocument {
return {
schema_version: 1,
product: PRODUCT,
state: "preview",
plan: { ...plan, digest: digestOf(plan) } as ModelWritePlan,
};
}
function invalid(code: string): ModelWriteDocument {
return { schema_version: 1, product: PRODUCT, state: "invalid", code };
}
@@ -66,45 +122,87 @@ function projectProfile(profile: ModelProfile): Record<string, unknown> {
};
}
/** Make `profileId` the active profile. Activation has no request body, so
* there is no revision for Core to check. */
export async function activateModelProfile(
profileId: string,
operations: CoreOperations,
): Promise<ModelWriteDocument> {
if (!PROFILE_ID.test(profileId)) {
return invalid("model_profile_activate_input_invalid");
}
const result = await operations.post<unknown>(
`/api/v1/model-profiles/${encodeURIComponent(profileId)}/activate`,
{},
);
return modelProfileResult(profileId, result);
// --- activate -------------------------------------------------------------
/**
* Activation has no request body, so the plan carries only the target. There
* is deliberately no `expected_revision`: Core would never read one, and
* recording a check it does not perform would be a fake guard.
*/
export function planModelProfileActivate(input: {
profileId: string;
}): ModelWriteDocument {
return PROFILE_ID.test(input.profileId)
? preview({
schema_version: 1,
kind: "model_profile_activate",
profile_id: input.profileId,
})
: invalid("model_profile_activate_input_invalid");
}
/** Apply a bounded patch (carrying `expected_revision`) to one profile; the
* key, when given, comes from a controlled file and never from the patch. */
export async function editModelProfile(
profileId: string,
patch: unknown,
apiKey: string | undefined,
operations: CorePatchOperations,
export async function executeModelProfileActivate(
plan: Extract<ModelWritePlan, { kind: "model_profile_activate" }>,
operations: CoreOperations,
): Promise<ModelWriteDocument> {
if (
!PROFILE_ID.test(profileId) ||
patch === null ||
typeof patch !== "object" ||
Array.isArray(patch) ||
"api_key" in patch
) {
return invalid("model_profile_edit_input_invalid");
}
// The shared builder owns patch validation (allow-list, expected_revision
// CAS, field bounds, api_key/clear_api_key exclusivity).
const result = await operations.post<unknown>(
`/api/v1/model-profiles/${encodeURIComponent(plan.profile_id)}/activate`,
{},
);
return modelProfileResult(plan.profile_id, result);
}
// --- edit -----------------------------------------------------------------
export function planModelProfileEdit(input: {
profileId: string;
patch: unknown;
}): ModelWriteDocument {
// The shared builder owns patch validation (14-key allow-list,
// expected_revision CAS, field bounds, api_key/clear_api_key exclusivity).
// A rejected patch is `invalid`, never a plan the caller could confirm.
let request: ReturnType<typeof buildModelProfileUpdateRequest>;
try {
request = buildModelProfileUpdateRequest(profileId, {
...(patch as Record<string, unknown>),
request = buildModelProfileUpdateRequest(input.profileId, input.patch);
} catch (error) {
if (error instanceof ModelProfileInputError) {
return invalid("model_profile_edit_input_invalid");
}
throw error;
}
const patch = request.body as Record<string, unknown>;
if (!PROFILE_ID.test(input.profileId)) {
return invalid("model_profile_edit_input_invalid");
}
return preview({
schema_version: 1,
kind: "model_profile_edit",
profile_id: input.profileId,
expected_revision: patch.expected_revision as number,
// `api_key` is read from a controlled file at apply time, exactly like
// `models configure`; the plan never carries the credential. The revision
// is a top-level field, so it is stripped from the nested patch rather
// than being duplicated in both places.
patch: Object.fromEntries(
Object.entries(patch).filter(
([key]) => key !== "api_key" && key !== "expected_revision",
),
),
});
}
export async function executeModelProfileEdit(
plan: Extract<ModelWritePlan, { kind: "model_profile_edit" }>,
operations: CorePatchOperations,
apiKey: string | undefined,
): Promise<ModelWriteDocument> {
// Re-validate through the shared builder at apply time too, so a hand-edited
// plan cannot smuggle an out-of-bounds patch past the write.
let request: ReturnType<typeof buildModelProfileUpdateRequest>;
try {
request = buildModelProfileUpdateRequest(plan.profile_id, {
...plan.patch,
expected_revision: plan.expected_revision,
...(apiKey === undefined ? {} : { api_key: apiKey }),
});
} catch (error) {
@@ -114,7 +212,7 @@ export async function editModelProfile(
throw error;
}
const result = await operations.patch<unknown>(request.path, request.body);
return modelProfileResult(profileId, result);
return modelProfileResult(plan.profile_id, result);
}
function modelProfileResult(
@@ -156,11 +254,69 @@ function modelProfileResult(
return committed(operationId, profile);
}
export function validateModelWritePlan(value: unknown): ModelWritePlan | null {
if (value === null || typeof value !== "object" || Array.isArray(value)) {
return null;
}
const candidate = value as Record<string, unknown>;
if (candidate.schema_version !== 1 || typeof candidate.kind !== "string") {
return null;
}
const { digest, ...withoutDigest } = candidate;
if (typeof digest !== "string" || digest !== digestOf(withoutDigest)) {
return null;
}
switch (candidate.kind) {
case "model_profile_activate":
return typeof candidate.profile_id === "string" &&
PROFILE_ID.test(candidate.profile_id) &&
Object.keys(candidate).length === 3
? (candidate as unknown as ModelWritePlan)
: null;
case "model_profile_edit": {
if (
typeof candidate.profile_id !== "string" ||
!PROFILE_ID.test(candidate.profile_id) ||
!Number.isSafeInteger(candidate.expected_revision)
) {
return null;
}
// The shared builder is the single judge of a patch body; the plan must
// not carry `api_key` (that is read from a controlled file at apply).
let built: ReturnType<typeof buildModelProfileUpdateRequest>;
try {
built = buildModelProfileUpdateRequest(candidate.profile_id, {
...(candidate.patch as Record<string, unknown>),
expected_revision: candidate.expected_revision,
});
} catch {
return null;
}
if ("api_key" in built.body) return null;
return Object.keys(candidate).length === 6
? (candidate as unknown as ModelWritePlan)
: null;
}
default:
return null;
}
}
export function modelWriteExitCode(document: ModelWriteDocument): number {
return document.state === "committed" ? 0 : 1;
return ["preview", "committed"].includes(document.state) ? 0 : 1;
}
export function renderModelWriteText(document: ModelWriteDocument): string {
if (document.state === "preview") {
const plan = document.plan;
return [
`预览:${plan.kind}`,
`目标配置:${plan.profile_id}`,
`确认摘要:${plan.digest}`,
"未执行任何写操作。请把 --json 输出保存为 plan 文件,再用对应命令加 --confirm。",
"",
].join("\n");
}
if (document.state === "committed") {
return `已保存模型配置:${document.receipt.profile_id}\nrevision=${document.receipt.revision}\n`;
}
+57
View File
@@ -0,0 +1,57 @@
import { constants } from "node:fs";
import { type FileHandle, open } from "node:fs/promises";
import { readBoundedJson } from "./bounded-file.js";
export type PlanAttempt = {
schema_version: 1;
operation_id: string;
plan_digest: string;
};
/**
* Durable client fence written before a mutating request. Once present, the
* same plan may only query its operation receipt; it is never blindly posted
* again after an unknown result.
*/
export async function claimPlanAttempt(
planFile: string,
attempt: PlanAttempt,
): Promise<"first" | "prior" | "invalid"> {
const journal = `${planFile}.one-memory-attempt`;
let handle: FileHandle;
try {
handle = await open(
journal,
constants.O_WRONLY |
constants.O_CREAT |
constants.O_EXCL |
(constants.O_NOFOLLOW ?? 0),
0o600,
);
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "EEXIST") return "invalid";
const current = await readBoundedJson(journal, 1024);
return sameAttempt(current, attempt) ? "prior" : "invalid";
}
try {
await handle.writeFile(`${JSON.stringify(attempt)}\n`, "utf8");
await handle.sync();
return "first";
} catch {
return "invalid";
} finally {
await handle.close().catch(() => undefined);
}
}
function sameAttempt(value: unknown, expected: PlanAttempt): boolean {
return (
value !== null &&
typeof value === "object" &&
!Array.isArray(value) &&
(value as Record<string, unknown>).schema_version === 1 &&
(value as Record<string, unknown>).operation_id === expected.operation_id &&
(value as Record<string, unknown>).plan_digest === expected.plan_digest &&
Object.keys(value).length === 3
);
}
+10 -6
View File
@@ -104,7 +104,7 @@ describe("package matrix harness", () => {
});
});
it("fails closed on every write command without a Runtime", {
it("fails closed on every destructive UI-write command without a Runtime", {
timeout: 120_000,
}, async () => {
const runtimeDist = await mkdtemp(
@@ -140,7 +140,7 @@ describe("package matrix harness", () => {
expect(document.state).toBe("verified");
const results = document.commands.stopped_destructive.results;
// Every destructive command fails closed while Runtime is stopped.
expect(Object.keys(results)).toHaveLength(8);
expect(Object.keys(results)).toHaveLength(7);
for (const [name, result] of Object.entries(results)) {
expect(result, `${name} did not fail closed`).toEqual({
exit: 3,
@@ -164,7 +164,7 @@ describe("package matrix harness", () => {
const shim = [
"#!/usr/bin/env node",
"const a=process.argv.slice(2);",
"if(['graph-width','models','memories','sources'].includes(a[0])){",
"if(['calls','notes','graph','models','memories'].includes(a[0])){",
' process.stdout.write(\'{"schema_version":1,"product":"memhub","state":"committed","operation_id":"x","receipt":{}}\\n\');',
" process.exit(0);",
"}",
@@ -190,7 +190,7 @@ describe("package matrix harness", () => {
expect(failure?.state).toBe("failed");
expect(
failure?.failures.filter((item) => item.includes("did not fail closed")),
).toHaveLength(8);
).toHaveLength(7);
});
it("can require an expected bundle SHA and reports bundle metadata", {
@@ -291,7 +291,9 @@ describe("package matrix harness", () => {
"sync",
"memories",
"profile",
"graph-width",
"calls",
"notes",
"graph",
"models",
"host",
"doctor",
@@ -309,12 +311,14 @@ describe("package matrix harness", () => {
"uninstall",
"delete-data",
"recall",
"setup",
"scan",
"progress",
"jobs",
"scan-roots",
"path",
"adapter",
"write",
"task",
"serve",
];
const catalog = {
@@ -145,15 +145,13 @@ describe("memhub short-command contract", () => {
[command.usage, command.alternative_usage ?? ""].join("\n"),
).join("\n");
for (const shortForm of [
"memhub memories add",
"memhub memories add-status",
"memhub calls clear",
"memhub notes add",
"memhub notes process",
"memhub models setup",
"memhub models add",
"memhub models test",
"memhub models activate",
"memhub models edit",
"memhub graph-width",
"memhub sources connect",
"memhub graph width",
"memhub write submit",
"memhub write status",
"memhub memories update",
@@ -161,10 +159,6 @@ describe("memhub short-command contract", () => {
]) {
expect(flattened).toContain(shortForm);
}
// No "note" concept and no plan files on the CLI surface.
for (const removed of ["memhub notes", "memhub calls", "--plan-file"]) {
expect(flattened).not.toContain(removed);
}
expect(COMMANDS.some((command) => command.name === "permission")).toBe(
false,
);
@@ -217,7 +211,7 @@ describe("memhub short-command contract", () => {
// but an earlier catalog text still claimed they were "尚不可用". A user
// reading `help path` would not try a command that works.
it("does not call the wired path/adapter commands unavailable", () => {
for (const name of ["scan-roots", "adapter"]) {
for (const name of ["path", "adapter"]) {
const command = COMMANDS.find((entry) => entry.name === name);
expect(command).toBeDefined();
const text = [
@@ -283,7 +277,7 @@ describe("memhub short-command contract", () => {
}
});
it("keeps destructive previews free of diagnostic side effects", {
it("keeps plan-file apply previews free of diagnostic side effects", {
timeout: 20_000,
}, async () => {
expect(
@@ -297,10 +291,9 @@ describe("memhub short-command contract", () => {
);
try {
for (const [index, argv] of [
["memories", "delete", "m-1"],
["models", "remove", "44444444-4444-4444-8444-444444444444"],
["backup", "restore", "--source", path.join(root, "backup.sqlite3")],
["graph-width", "--decision", "rebuild_with_new"],
["sources", "import", "--plan-file", path.join(root, "plan.json")],
["models", "configure", "--plan-file", path.join(root, "plan.json")],
["memories", "apply", "--plan-file", path.join(root, "plan.json")],
].entries()) {
const dataDir = path.join(root, `data-${index}`);
await import("node:fs/promises").then(({ mkdir }) =>
@@ -312,8 +305,8 @@ describe("memhub short-command contract", () => {
stdio: "pipe",
});
} catch {
// A failure path is acceptable; the guard is that no preview adds
// a diagnostic file.
// The missing plan or stopped Runtime is expected; the guard is
// that even the failure path did not add a diagnostic file.
}
expect(await readdir(dataDir)).toEqual([]);
}
+265 -123
View File
@@ -1,56 +1,122 @@
// The rules behind the commands that attach to the Runtime directly.
// A6: the rules behind the six product commands.
//
// The parsers are where the contract actually lives — "only irreversible
// operations ask for --yes" and "unknown flags are rejected" are decisions,
// not formatting — so they are tested directly rather than only through the
// CLI.
// The parsers are where the contract actually lives — "needs --yes to change
// anything" and "no write without evidence" are decisions, not formatting — so
// they are tested directly rather than only through the CLI.
import { describe, expect, it } from "vitest";
import { parseOpenAIMessages } from "./core-client.js";
import {
parseBackupCommand,
parseScanCommand,
parseScanRootsCommand,
asPlanDocument,
describe as describePlan,
parseProductCommand,
parseTaskCommand,
parseWriteCommand,
restoreConfirmationMatches,
} from "./product-flows.js";
describe("scan-roots", () => {
it("reads the root, source and recursive flag off add", () => {
expect(
parseScanRootsCommand([
"add",
"--root",
"/tmp/x",
"--source",
"codex",
"--no-recursive",
]),
).toEqual({
kind: "add",
root: "/tmp/x",
source: "codex",
recursive: false,
});
expect(parseScanRootsCommand(["add", "--root", "/tmp/x"])).toEqual({
kind: "add",
root: "/tmp/x",
recursive: true,
});
describe("mutating commands are two-phase", () => {
it("plans without --yes", () => {
const parsed = parseProductCommand("scan", ["--root", "/tmp/a"]);
expect(parsed?.confirmed).toBe(false);
});
it("distinguishes remove from add and refuses add-only flags on remove", () => {
expect(parseScanRootsCommand(["remove", "--root", "/tmp/x"])).toEqual({
kind: "remove",
root: "/tmp/x",
});
it("confirms with --yes", () => {
const parsed = parseProductCommand("scan", ["--root", "/tmp/a", "--yes"]);
expect(parsed?.confirmed).toBe(true);
});
it("says what it would do", () => {
const parsed = parseProductCommand("scan", ["--root", "/tmp/a"]);
// `?.plan!` would let a null parse pass silently; assert the parse first.
if (parsed === null) {
throw new Error("scan with a root must parse");
}
expect(describePlan(parsed.plan)).toContain("scan: /tmp/a");
});
it("marks a plan as not applied", () => {
// Assert the parse itself: `?.` let a null result satisfy this test for as
// long as `backup create` was unparseable.
const parsed = parseProductCommand("backup", [
"create",
"--target",
"/tmp/b",
]);
if (parsed === null) {
throw new Error("backup create must parse");
}
expect(parsed.plan.verb).toBe("create");
expect(asPlanDocument(parsed.plan).applied).toBe(false);
});
it("parses the verb for every command that takes one", () => {
// `path add` / `path remove` / `backup create` were rejected outright
// before the verb was skipped in parsing.
expect(
parseScanRootsCommand(["remove", "--root", "/tmp/x", "--no-recursive"]),
parseProductCommand("path", ["add", "--root", "/tmp/x"])?.plan.action,
).toBe("add scan root");
expect(
parseProductCommand("path", ["remove", "--root", "/tmp/x"])?.plan.action,
).toBe("remove scan root");
expect(
parseProductCommand("backup", ["restore", "--source", "/tmp/b"])?.plan
.action,
).toBe("restore backup");
});
it("defaults a scan to all registered roots", () => {
const parsed = parseProductCommand("scan", []);
expect(parsed?.plan.target).toBe("all registered roots");
});
});
describe("unknown flags are rejected", () => {
it("rejects a typo'd flag rather than ignoring it", () => {
// A silently ignored --yes would turn a real apply into a plan, and the
// user would believe the change had been made.
expect(parseProductCommand("scan", ["--yesx"])).toBeNull();
expect(parseProductCommand("scan", ["--root"])).toBeNull();
expect(parseProductCommand("scan", ["--root", "--yes"])).toBeNull();
});
it("rejects an unknown verb for backup", () => {
expect(
parseProductCommand("backup", ["make", "--target", "/tmp/b"]),
).toBeNull();
});
// `path` and `scan` share one parser. Before this guard, an unknown verb was
// silently treated as `path add` or a default `scan`, so `--yes` executed a
// different operation than the one the caller named.
it("rejects an unknown verb for path without turning it into add", () => {
expect(
parseScanRootsCommand([
parseProductCommand("path", ["frobnicate", "--root", "/tmp/x", "--yes"]),
).toBeNull();
});
it("rejects an unknown verb for scan without turning it into all", () => {
expect(
parseProductCommand("scan", ["frobnicate", "--all", "--yes"]),
).toBeNull();
expect(parseProductCommand("scan", ["frobnicate", "--yes"])).toBeNull();
});
it("still accepts the documented verbless scan", () => {
expect(parseProductCommand("scan", ["--all", "--yes"])?.plan.all).toBe(
true,
);
expect(
parseProductCommand("scan", ["--source", "codex"])?.plan.sourceId,
).toBe("codex");
});
it("rejects flags that the named verb does not consume", () => {
// These were accepted and then silently dropped. An operator who typed
// `path remove --source codex` would believe the source filter applied.
expect(
parseProductCommand("path", [
"remove",
"--root",
"/tmp/x",
@@ -58,103 +124,137 @@ describe("scan-roots", () => {
"codex",
]),
).toBeNull();
});
it("is direct, but keeps accepting a legacy --yes", () => {
expect(parseScanRootsCommand(["add", "--root", "/tmp/x", "--yes"])).toEqual(
{ kind: "add", root: "/tmp/x", recursive: true },
);
});
it("rejects unknown verbs, typos and repeated flags", () => {
expect(parseScanRootsCommand(["create", "--root", "/tmp/x"])).toBeNull();
expect(parseScanRootsCommand(["add", "--rot", "/tmp/x"])).toBeNull();
expect(parseScanRootsCommand(["add"])).toBeNull();
expect(
parseScanRootsCommand(["add", "--root", "/a", "--root", "/b"]),
parseProductCommand("path", [
"remove",
"--root",
"/tmp/x",
"--no-recursive",
]),
).toBeNull();
expect(
parseScanRootsCommand(["add", "--root", "/a", "--yes", "--yes"]),
).toBeNull();
});
});
describe("scan", () => {
it("needs a source: there is no endpoint that scans a bare path", () => {
expect(parseScanCommand(["--source", "codex"])).toEqual({
kind: "source",
sourceId: "codex",
});
expect(parseScanCommand([])).toBeNull();
expect(parseScanCommand(["--root", "/tmp/a"])).toBeNull();
});
it("no longer accepts --full, which was never sent to Core", () => {
expect(parseScanCommand(["--source", "codex", "--full"])).toBeNull();
});
it("keeps --all as a hidden alias and a legacy --yes", () => {
expect(parseScanCommand(["--all", "--yes"])).toEqual({ kind: "all" });
expect(parseScanCommand(["--all", "--source", "codex"])).toBeNull();
expect(parseScanCommand(["--source", "codex", "--yes"])).toEqual({
kind: "source",
sourceId: "codex",
});
});
it("rejects a verb, a bad source id and repeated flags", () => {
expect(parseScanCommand(["all"])).toBeNull();
expect(parseScanCommand(["--source", "Bad Id"])).toBeNull();
expect(
parseScanCommand(["--source", "codex", "--source", "cursor"]),
parseProductCommand("path", ["add", "--root", "/tmp/x", "--full"]),
).toBeNull();
expect(
parseScanCommand(["--source", "codex", "--json", "--json"]),
parseProductCommand("backup", [
"create",
"--target",
"/tmp/b",
"--source",
"/tmp/s",
]),
).toBeNull();
});
});
describe("backup", () => {
it("creates directly and requires a target", () => {
expect(parseBackupCommand(["create", "--target", "/tmp/b"])).toEqual({
verb: "create",
target: "/tmp/b",
});
expect(parseBackupCommand(["create"])).toBeNull();
expect(
parseBackupCommand(["create", "--target", "/tmp/b", "--source", "/x"]),
parseProductCommand("backup", [
"restore",
"--source",
"/tmp/s",
"--target",
"/tmp/t",
]),
).toBeNull();
});
it("restores only with --yes and requires a source", () => {
expect(parseBackupCommand(["restore", "--source", "/tmp/b"])).toEqual({
verb: "restore",
source: "/tmp/b",
confirmed: false,
});
// A later value silently won before this guard. For `scan --source` that
// means one extra shell/make variable can change which source is scanned
// while the operator believes both were applied.
it("rejects repeated value-bearing flags instead of letting the last one win", () => {
expect(
parseBackupCommand(["restore", "--source", "/tmp/b", "--yes"]),
).toMatchObject({ confirmed: true });
expect(parseBackupCommand(["restore", "--yes"])).toBeNull();
parseProductCommand("scan", ["--source", "codex", "--source", "cursor"]),
).toBeNull();
expect(
parseProductCommand("path", [
"add",
"--root",
"/tmp/a",
"--root",
"/tmp/b",
]),
).toBeNull();
expect(
parseProductCommand("backup", [
"create",
"--target",
"/tmp/a",
"--target",
"/tmp/b",
]),
).toBeNull();
});
it("accepts the legacy --confirm <path> and checks it lexically", () => {
const parsed = parseBackupCommand([
it("rejects repeated boolean flags instead of treating them as idempotent", () => {
expect(parseProductCommand("scan", ["--all", "--all", "--yes"])).toBeNull();
expect(
parseProductCommand("scan", ["--full", "--full", "--yes"]),
).toBeNull();
expect(
parseProductCommand("path", [
"add",
"--root",
"/tmp/a",
"--no-recursive",
"--no-recursive",
]),
).toBeNull();
expect(
parseProductCommand("backup", [
"create",
"--target",
"/tmp/a",
"--yes",
"--yes",
]),
).toBeNull();
});
it("rejects a repeated global --json spelling", () => {
expect(
parseProductCommand("scan", ["--root", "/tmp/a", "--json", "--json"]),
).toBeNull();
expect(
parseProductCommand("backup", [
"create",
"--target",
"/tmp/a",
"--json",
"--json",
]),
).toBeNull();
});
it("requires a source when restoring", () => {
expect(
parseProductCommand("backup", ["restore", "--target", "/tmp/b"]),
).toBeNull();
});
it("parses the explicit restore confirmation path", () => {
const parsed = parseProductCommand("backup", [
"restore",
"--source",
"/tmp/b",
"/tmp/backups/a.sqlite3",
"--yes",
"--confirm",
"/tmp/b",
"/tmp/backups/a.sqlite3",
]);
expect(parsed).toMatchObject({ confirmPath: "/tmp/b" });
expect(restoreConfirmationMatches("/tmp/b", "/tmp/b")).toBe(true);
expect(restoreConfirmationMatches("/tmp/b", undefined)).toBe(true);
expect(restoreConfirmationMatches("/tmp/b", "/tmp/./b")).toBe(false);
expect(parsed).not.toBeNull();
expect(parsed?.confirmPath).toBe("/tmp/backups/a.sqlite3");
expect(
restoreConfirmationMatches("/tmp/backups/a.sqlite3", parsed?.confirmPath),
).toBe(true);
expect(
restoreConfirmationMatches("/tmp/backups/a.sqlite3", undefined),
).toBe(false);
expect(
restoreConfirmationMatches(
"/tmp/backups/a.sqlite3",
"/tmp/backups/../backups/a.sqlite3",
),
).toBe(false);
});
it("rejects an unknown verb", () => {
expect(parseBackupCommand(["delete", "--target", "/tmp/b"])).toBeNull();
it("requires a target when creating a backup", () => {
expect(parseProductCommand("backup", ["create"])).toBeNull();
});
});
@@ -205,19 +305,11 @@ describe("write", () => {
).toBeNull();
});
it("parses status under write, positional or --id", () => {
it("parses status under write", () => {
expect(parseWriteCommand(["status", "--id", "request-1"])).toEqual({
kind: "status",
id: "request-1",
});
expect(parseWriteCommand(["status", "request-1"])).toEqual({
kind: "status",
id: "request-1",
});
});
it("rejects a repeated --session instead of letting the last one win", () => {
expect(parseWriteCommand([...base, "--session", "s2"])).toBeNull();
});
it("marks --live as priority live", () => {
@@ -274,3 +366,53 @@ describe("task", () => {
expect(parseTaskCommand(["--id", "req-1", "--json", "--json"])).toBeNull();
});
});
describe("path add and remove carry what the endpoint needs", () => {
it("reads the source and recursive flag off the path add", () => {
const parsed = parseProductCommand("path", [
"add",
"--root",
"/tmp/scanme",
"--source",
"codex",
"--no-recursive",
]);
// `?.plan!` let a null parse pass silently, so assert the parse itself.
if (parsed === null) {
throw new Error("path add must parse");
}
expect(parsed.plan.action).toBe("add scan root");
expect(parsed.plan.target).toBe("/tmp/scanme");
expect(parsed.plan.detail).toContain("source=codex");
expect(parsed.plan.detail).toContain("recursive=false");
});
it("distinguishes remove from add", () => {
const parsed = parseProductCommand("path", [
"remove",
"--root",
"/tmp/scanme",
]);
if (parsed === null) {
throw new Error("path remove must parse");
}
// The action is the discriminator at apply time: add and remove share a
// `--root`, so a mis-set action would register the directory it meant to delete.
expect(parsed.plan.action).toBe("remove scan root");
});
it("does not silently drop --yes", () => {
const parsed = parseProductCommand("path", [
"add",
"--root",
"/tmp/a",
"--yes",
]);
expect(parsed?.confirmed).toBe(true);
// Without this, a typo'd `--yes` would fall through to a plan and the user
// would believe the change had been made.
const without = parseProductCommand("path", ["add", "--root", "/tmp/a"]);
expect(without?.confirmed).toBe(false);
});
});
+272 -153
View File
@@ -1,163 +1,258 @@
// Parsers for the commands that attach to the Runtime directly:
// `scan-roots` (alias `path`), `scan`, `backup`, `write`.
// A6: the product commands Runtime exposes on the CLI.
//
// Rules shared by all of them:
// These are the six commands the objective names: path, adapter, scan, write,
// task, backup. Two rules run through all of them:
//
// - **Only irreversible operations ask for `--yes`.** `backup restore`
// overwrites the database, so without `--yes` it previews and changes
// nothing. Registering a scan root, scanning a source, creating a backup and
// submitting a write are direct. A legacy `--yes` on a direct command is
// accepted and ignored so older scripts keep working.
// - **Unknown or repeated flags are rejected**, never ignored: a dropped typo
// would silently change what the command does.
// - **Output is machine-readable.** `--json` emits one document on stdout.
// - **Mutating commands are two-phase.** Without `--yes` they print a plan and
// change nothing. This is the convention the existing install/service
// commands already follow, and it is why `scan` and `backup` default to a
// plan rather than quietly doing work.
// - **Output is machine-readable.** `--json` emits one document on stdout so an
// agent can consume it; the human form is text. Never both, so a caller that
// parses stdout is not surprised by prose.
//
// The commands talk to the Runtime's HTTP API, not to the database and not to
// the SDK: CLI and desktop are clients of the same Runtime.
// HY. That is the A6 rule: CLI and Electron are clients of the same Runtime.
export type ScanRootsCommand =
| {
readonly kind: "add";
readonly root: string;
readonly source?: string;
readonly recursive: boolean;
}
| { readonly kind: "remove"; readonly root: string };
export type ScanCommand =
| { readonly kind: "source"; readonly sourceId: string }
/** Hidden alias of `sources connect --all-detected --import`. */
| { readonly kind: "all" };
export type BackupCommand =
| { readonly verb: "create"; readonly target: string }
| {
readonly verb: "restore";
readonly source: string;
readonly confirmed: boolean;
/** Legacy `--confirm <path>`: accepted, and must repeat `--source`. */
readonly confirmPath?: string;
};
interface ParsedFlags {
readonly values: Map<string, string>;
readonly switches: Set<string>;
/** What a mutating command would do, before it is allowed to do it. */
export interface Plan {
readonly action: string;
readonly target: string;
readonly detail?: string;
/** Which form of the command this plan is for, when one name has several.
*
* `backup` has `create` and `restore`, and applying the wrong one is not a
* harmless mistake — so the verb travels with the plan the user confirmed.
*/
readonly verb?: string;
/** The source the Runtime endpoint addresses, when the plan names one.
*
* Carried on the plan rather than re-parsed at apply time so the plan the
* user confirmed and the request actually sent cannot disagree.
*/
readonly sourceId?: string;
/** `scan --all`: every source Runtime actually found, not a named one. */
readonly all?: boolean;
}
/** Read `--flag value` pairs and bare switches; null on anything unknown,
* repeated or missing its value. */
function parseFlags(
export function planAddRoot(
root: string,
source: string | undefined,
recursive: boolean,
): Plan {
return {
action: "add scan root",
target: root,
detail: `source=${source ?? "auto"} recursive=${recursive}`,
};
}
export function planRemoveRoot(root: string): Plan {
return { action: "remove scan root", target: root };
}
export function planScan(root: string | undefined, full: boolean): Plan {
return {
action: "scan",
target: root ?? "all registered roots",
detail: `full=${full}`,
};
}
export function planScanSource(sourceId: string, full: boolean): Plan {
return {
action: "scan",
target: `source ${sourceId}`,
detail: `full=${full}`,
sourceId,
};
}
export function planScanAll(full: boolean): Plan {
return {
action: "scan",
target: "all detected sources",
detail: `full=${full}`,
all: true,
};
}
export function planBackup(target: string): Plan {
return { action: "create backup", target, verb: "create" };
}
export function planRestore(source: string): Plan {
return { action: "restore backup", target: source, verb: "restore" };
}
export interface ProductCommand {
readonly plan: Plan;
readonly confirmed: boolean;
/** For destructive verbs with a named artifact, the exact value repeated by
* the user. `backup restore` requires this to match `--source`. */
readonly confirmPath?: string;
}
/** Parse a mutating product command.
*
* Unknown flags are rejected rather than ignored: a typo'd `--yes` that is
* silently dropped would turn a real apply into a plan, and the user would
* believe the change had been made.
*/
export function parseProductCommand(
name: string,
args: readonly string[],
valueFlags: readonly string[],
switchFlags: readonly string[],
): ParsedFlags | null {
const values = new Map<string, string>();
const switches = new Set<string>();
for (let index = 0; index < args.length; index += 1) {
): ProductCommand | null {
let root: string | undefined;
let source: string | undefined;
let target: string | undefined;
let recursive = true;
let full = false;
let all = false;
let confirmed = false;
let confirmPath: string | undefined;
const seen = new Set<string>();
const seenValues = new Set<string>();
const flags = new Set([
"--root",
"--source",
"--target",
"--confirm",
"--no-recursive",
"--full",
"--all",
"--yes",
"--json",
]);
// The first token is the verb for commands that have one (`path add`,
// `backup create`). It used to fall into the flag loop and be rejected as an
// unknown flag, which made every one of these commands permanently
// unparseable — `path add --root X` printed "参数无效" even though `help`
// documented it. The tests missed this because they asserted
// `parsed?.plan!` and null satisfies `?.` + `!`.
const verb = args[0];
const start = verb !== undefined && !verb.startsWith("--") ? 1 : 0;
for (let index = start; index < args.length; index += 1) {
const arg = args[index];
if (switchFlags.includes(arg)) {
if (switches.has(arg)) return null;
switches.add(arg);
if (!flags.has(arg)) return null;
// A repeated flag is a malformed argv, not an idempotent spelling. The
// other command parsers already apply this rule to --json as well.
if (seen.has(arg)) return null;
seen.add(arg);
if (arg === "--yes") {
confirmed = true;
continue;
}
if (arg === "--confirm") {
const value = args[index + 1];
if (value === undefined || value.startsWith("--")) return null;
if (confirmPath !== undefined) return null;
confirmPath = value;
index += 1;
continue;
}
if (arg === "--json") continue;
if (arg === "--no-recursive") {
recursive = false;
continue;
}
if (arg === "--full") {
full = true;
continue;
}
if (arg === "--all") {
all = true;
continue;
}
if (!valueFlags.includes(arg) || values.has(arg)) return null;
const value = args[index + 1];
if (value === undefined || value.startsWith("--")) return null;
values.set(arg, value);
if (seenValues.has(arg)) return null;
seenValues.add(arg);
index += 1;
if (arg === "--root") root = value;
else if (arg === "--source") source = value;
else target = value;
}
return { values, switches };
}
const SOURCE_ID = /^[a-z0-9][a-z0-9-]{0,127}$/;
/** `scan-roots add --root <path> [--source <id>] [--no-recursive]` and
* `scan-roots remove --root <path>` (`list` is answered separately). */
export function parseScanRootsCommand(
args: readonly string[],
): ScanRootsCommand | null {
const verb = args[0];
if (verb === "add") {
const flags = parseFlags(
args.slice(1),
["--root", "--source"],
["--no-recursive", "--yes", "--json"],
);
const root = flags?.values.get("--root");
if (!flags || root === undefined) return null;
const source = flags.values.get("--source");
if (source !== undefined && !SOURCE_ID.test(source)) return null;
if (name === "path") {
if (confirmPath !== undefined) return null;
if (target !== undefined) return null;
if (root === undefined) return null;
if (source === undefined && args.includes("--source")) return null;
if (seen.has("--full") || seen.has("--all")) return null;
if (args[0] !== "add" && args[0] !== "remove") return null;
if (
args[0] === "remove" &&
(seen.has("--source") || seen.has("--no-recursive"))
) {
return null;
}
return {
kind: "add",
root,
...(source === undefined ? {} : { source }),
recursive: !flags.switches.has("--no-recursive"),
plan:
args[0] === "add"
? planAddRoot(root, source, recursive)
: planRemoveRoot(root),
confirmed,
};
}
if (verb === "remove") {
const flags = parseFlags(args.slice(1), ["--root"], ["--yes", "--json"]);
const root = flags?.values.get("--root");
return flags && root !== undefined ? { kind: "remove", root } : null;
if (name === "scan") {
if (confirmPath !== undefined) return null;
if (target !== undefined) return null;
if (seen.has("--no-recursive")) return null;
if (args[0] !== undefined && !args[0].startsWith("--")) return null;
// `--all` means "every source Runtime actually found", so naming one too
// would make the command mean two different things.
if (all && source !== undefined) return null;
if (all) return { plan: planScanAll(full), confirmed };
// A source is the thing the Runtime scan endpoint addresses. A bare root is
// still accepted for planning (it means "all registered roots") but cannot
// be applied, because there is no endpoint that scans a raw path — the CLI
// must not pretend it can.
if (source === undefined) return { plan: planScan(root, full), confirmed };
return { plan: planScanSource(source, full), confirmed };
}
return null;
}
/** `scan --source <id>`. A scan names a source: the Runtime has no endpoint
* that scans a bare path, so a missing `--source` is a usage error rather
* than a preview of something that can never run. */
export function parseScanCommand(args: readonly string[]): ScanCommand | null {
const flags = parseFlags(args, ["--source"], ["--all", "--yes", "--json"]);
if (!flags) return null;
const source = flags.values.get("--source");
if (flags.switches.has("--all")) {
return source === undefined ? { kind: "all" } : null;
}
return source !== undefined && SOURCE_ID.test(source)
? { kind: "source", sourceId: source }
: null;
}
/** `backup create --target <path>` and
* `backup restore --source <path> [--yes]`. */
export function parseBackupCommand(
args: readonly string[],
): BackupCommand | null {
const verb = args[0];
if (verb === "create") {
const flags = parseFlags(args.slice(1), ["--target"], ["--yes", "--json"]);
const target = flags?.values.get("--target");
return flags && target !== undefined ? { verb: "create", target } : null;
}
if (verb === "restore") {
const flags = parseFlags(
args.slice(1),
["--source", "--confirm"],
["--yes", "--json"],
);
const source = flags?.values.get("--source");
if (!flags || source === undefined) return null;
const confirmPath = flags.values.get("--confirm");
if (name === "backup") {
const verb = args[0];
if (verb !== "create" && verb !== "restore") return null;
if (seen.has("--full") || seen.has("--all") || seen.has("--no-recursive")) {
return null;
}
if (verb === "restore") {
if (source === undefined) return null;
if (target !== undefined) return null;
return {
plan: planRestore(source),
confirmed,
...(confirmPath === undefined ? {} : { confirmPath }),
};
}
if (confirmPath !== undefined) return null;
if (source !== undefined) return null;
if (target === undefined) return null;
return {
verb: "restore",
source,
confirmed: flags.switches.has("--yes"),
plan: planBackup(target),
confirmed,
...(confirmPath === undefined ? {} : { confirmPath }),
};
}
return null;
}
/** Whether a legacy `--confirm <path>` names the same restore source.
/** Whether `--confirm <path>` names the same restore source.
*
* The comparison is lexical on purpose: the CLI must not resolve symlinks or
* touch the filesystem before Core has validated the managed source. A
* different string is a different confirmation.
* different string is a different confirmation, so a typo or a path escape
* can never be applied as a side effect of `--yes`.
*/
export function restoreConfirmationMatches(
source: string,
confirmPath: string | undefined,
): boolean {
return confirmPath === undefined || confirmPath === source;
return confirmPath === source;
}
export interface WriteSubmitCommand {
@@ -177,33 +272,41 @@ export interface WriteStatusCommand {
export type WriteCommand = WriteSubmitCommand | WriteStatusCommand;
/** Parse `write submit ...` and `write status <request-id>` (or `--id`). */
/** Parse SDK-native `write submit|status`. */
export function parseWriteCommand(
args: readonly string[],
): WriteCommand | null {
if (args.filter((item) => item === "--json").length > 1) return null;
const meaningful = args.filter((item) => item !== "--json");
if (meaningful[0] === "status") {
const rest = meaningful.slice(1);
if (rest.length === 1 && !rest[0].startsWith("--")) {
return { kind: "status", id: rest[0] };
}
const status = parseTaskCommand(rest);
const status = parseTaskCommand(meaningful.slice(1));
return status === null ? null : { kind: "status", id: status.id };
}
// `write --session ...` without `submit` stays accepted for older scripts.
const input = meaningful[0] === "submit" ? meaningful.slice(1) : meaningful;
const flags = parseFlags(
input,
["--session", "--source", "--messages-file", "--request-id", "--memory-at"],
["--live"],
);
if (!flags) return null;
const session = flags.values.get("--session");
const source = flags.values.get("--source") ?? "memhub-cli";
const messagesFile = flags.values.get("--messages-file");
const requestId = flags.values.get("--request-id");
const memoryAt = flags.values.get("--memory-at");
let session: string | undefined;
let source = "memhub-cli";
let messagesFile: string | undefined;
let requestId: string | undefined;
let memoryAt: string | undefined;
let priority: "live" | "normal" | undefined;
for (let index = 0; index < input.length; index += 1) {
const arg = input[index];
if (arg === "--live") {
priority = "live";
continue;
}
const value = input[index + 1];
if (value === undefined || value.startsWith("--")) return null;
index += 1;
if (arg === "--session") session = value;
else if (arg === "--source") source = value;
else if (arg === "--messages-file") messagesFile = value;
else if (arg === "--request-id") requestId = value;
else if (arg === "--memory-at") memoryAt = value;
else return null;
}
if (session === undefined || messagesFile === undefined || !source)
return null;
if (
@@ -220,23 +323,39 @@ export function parseWriteCommand(
...(requestId === undefined ? {} : { request_id: requestId }),
...(memoryAt === undefined ? {} : { memory_at: memoryAt }),
};
return flags.switches.has("--live")
? { ...command, priority: "live" }
: command;
return priority === undefined ? command : { ...command, priority };
}
export interface TaskCommand {
readonly id: string;
}
/** `--id <request-id>`: the legacy `task` spelling of `write status`. */
export function parseTaskCommand(args: readonly string[]): TaskCommand | null {
if (args.filter((item) => item === "--json").length > 1) return null;
const flags = parseFlags(
args.filter((item) => item !== "--json"),
["--id"],
[],
);
const id = flags?.values.get("--id");
let id: string | undefined;
for (let index = 0; index < args.length; index += 1) {
const arg = args[index];
if (arg === "--json") continue;
if (arg !== "--id") return null;
const value = args[index + 1];
if (value === undefined || value.startsWith("--")) return null;
index += 1;
id = value;
}
return id === undefined ? null : { id };
}
/** The human form. Kept separate from the JSON form on purpose. */
export function describe(plan: Plan): string {
const line = `${plan.action}: ${plan.target}`;
return plan.detail ? `${line} (${plan.detail})` : line;
}
/** The machine form: one document, nothing else on stdout. */
export function asPlanDocument(plan: Plan): {
schema_version: 1;
plan: Plan;
applied: boolean;
} {
return { schema_version: 1, plan, applied: false };
}
+20 -108
View File
@@ -184,16 +184,8 @@ const CORE = new Map<string, { status: number; data: unknown }>([
},
],
[
"/api/v2/home-summary",
{
status: 200,
data: {
schema_version: 2,
memory_count: 42,
latest: null,
observed_at: "T",
},
},
"/api/v1/home-summary",
{ status: 200, data: { memory_count: 42, latest: null, observed_at: "T" } },
],
]);
@@ -202,7 +194,9 @@ const READ_ONLY_PATHS = [
"/api/v1/live-sync",
"/api/v1/memory-processing",
"/api/v1/model-profiles",
"/api/v2/home-summary",
"/api/v1/home-summary",
"/api/v1/memory-library",
"/api/v1/memories/",
"/api/v2/memory-library",
"/api/v2/memories/",
"/api/v2/user-profile",
@@ -697,66 +691,6 @@ describe("CLI-21 memory reads", () => {
expect(report.memories[1].content.length).toBeLessThan(400);
});
it("lists working and coding memories with their scene and kind", async () => {
transport.get.mockResolvedValue({
status: 200,
data: {
total: 2,
offset: 0,
limit: 20,
totals: { chat: 1, working: 1, coding: 0 },
window_id: null,
window_reset: false,
memories: [
memory("m1", "对话记忆"),
{
...memory("e1", "修复测试\n固定随机种子"),
layer: "experience",
agent_id: "",
owner: null,
source_raw_ids: [],
scene: "working",
kind: "experience",
variant: "working",
task: "修复测试",
solution: "固定随机种子",
created_at: 1_790_000_000,
updated_at: 1_790_000_100,
},
],
},
});
const streams = captureStreams();
try {
expect(await run(["memories", "list", "--json"], {})).toBe(0);
} finally {
streams.restore();
}
const report = JSON.parse(streams.stdout());
expect(report.page).toMatchObject({ state: "ok", total: 2, returned: 2 });
expect(report.memories[0]).toMatchObject({
memory_id: "m1",
scene: "chat",
kind: "chat",
variant: null,
});
expect(report.memories[1]).toMatchObject({
memory_id: "e1",
scene: "working",
kind: "experience",
variant: "working",
layer: "experience",
});
const text = captureStreams();
try {
expect(await run(["memories", "list"], {})).toBe(0);
} finally {
text.restore();
}
expect(text.stdout()).toContain("- e1 [working] active/experience");
});
it("passes the requested filters through as query parameters", async () => {
await run(
[
@@ -878,14 +812,11 @@ describe("CLI-21 memory reads", () => {
layer: "L2_FACT",
status: "active",
});
// Schema 2, like add/update/delete; the v1-era always-empty
// `evidence`/`history` fields are gone.
expect(report.schema_version).toBe(2);
expect(report).not.toHaveProperty("evidence");
expect(report).not.toHaveProperty("history");
expect(report.evidence).toEqual([]);
expect(report.history).toEqual([]);
});
// Core registers /api/v2/memories/graph before /{memory_id}, and that static
// Core registers /api/v1/memories/graph before /{memory_id}, and that static
// handler initializes the HY bridge. A memory id that collides with a
// reserved segment must never become a request.
it("refuses a memory id that collides with a static Core route", async () => {
@@ -899,15 +830,18 @@ describe("CLI-21 memory reads", () => {
expect(streams.stdout()).toBe("");
}
expect(transport.get).not.toHaveBeenCalled();
expect(isReadOnlyPath("/api/v2/memories/graph")).toBe(false);
expect(isReadOnlyPath("/api/v2/memories/ask")).toBe(false);
expect(isReadOnlyPath("/api/v2/memories/m1")).toBe(true);
expect(isReadOnlyPath("/api/v1/memories/graph")).toBe(false);
expect(isReadOnlyPath("/api/v1/memories/ask")).toBe(false);
expect(isReadOnlyPath("/api/v1/memories/m1")).toBe(true);
expect(isReadOnlyPath("/api/v2/memory-library")).toBe(true);
expect(isReadOnlyPath("/api/v2/home-summary")).toBe(true);
// The retired v1 data plane is no longer a CLI read.
expect(isReadOnlyPath("/api/v1/memories/m1")).toBe(false);
expect(isReadOnlyPath("/api/v1/home-summary")).toBe(false);
expect(isReadOnlyPath("/api/v2/memories/m1")).toBe(true);
expect(isReadOnlyPath("/api/v1/model-content-permission")).toBe(false);
expect(
isReadOnlyPath(
"/api/v1/memory-mutations/123e4567-e89b-12d3-a456-426614174000",
),
).toBe(true);
expect(isReadOnlyPath("/api/v1/memory-mutations/../sources")).toBe(false);
});
it("refuses a memory id that would escape the memories path", async () => {
@@ -927,26 +861,6 @@ describe("CLI-21 memory reads", () => {
// was cast to the expected type, so a wrong record, an unavailable excerpt, an
// oversized page and a non-string model field all reached stdout as success.
describe("malformed success responses", () => {
it("treats a v1-shaped home summary as invalid, not as a memory count", async () => {
transport.get.mockImplementation(async (url: string) =>
url === "/api/v2/home-summary"
? {
status: 200,
data: { memory_count: 7, latest: null, observed_at: "T" },
}
: (CORE.get(url) ?? { status: 404, data: { detail: "not_found" } }),
);
const streams = captureStreams();
try {
await run(["status", "--json"], {});
} finally {
streams.restore();
}
expect(JSON.parse(streams.stdout()).sections.memories).toMatchObject({
state: "invalid",
});
});
it("refuses a detail that is not the memory that was asked for", async () => {
transport.get.mockResolvedValue({
status: 200,
@@ -1140,16 +1054,14 @@ describe("backup create target refusal is rejected, not usage_error", () => {
let code: number;
try {
code = await run(
["backup", "create", "--target", "/tmp/outside.db", "--json"],
["backup", "create", "--target", "/tmp/outside.db", "--yes", "--json"],
{},
);
} finally {
streams.restore();
}
expect(code!).toBe(1);
// A result, on stdout like every --json document.
expect(streams.stderr()).toBe("");
const document = JSON.parse(streams.stdout());
const document = JSON.parse(streams.stderr());
expect(document.status).not.toBe("usage_error");
expect(document).toMatchObject({
command: "backup",
@@ -315,40 +315,6 @@ describe("macOS service lifecycle", () => {
expect(deps.verifyRegistrationSnapshot).toHaveBeenCalledTimes(2);
});
it("does not re-bootstrap a previous registration that may serve a legacy home", async () => {
const f = fixture();
const oldPath = "/Users/example/Library/LaunchAgents/old-runtime.plist";
const runCommand = vi
.fn()
.mockResolvedValueOnce(undefined)
.mockRejectedValueOnce(new Error("bootstrap failed"));
const deps = dependencies({
inspectRegistration: vi.fn(async () => ({
kind: "managed" as const,
plistPath: oldPath,
content: "old managed plist",
})),
runCommand,
});
const prepared = await prepareServiceInstall(f.layout, f.options, deps);
await expect(
applyPreparedServiceInstall(
{ ...prepared, restorePreviousOnFailure: false },
deps,
),
).rejects.toThrow("runtime_service_replacement_failed");
// bootout + the failed bootstrap only; the old plist is not bootstrapped.
expect(runCommand).toHaveBeenCalledTimes(2);
expect(runCommand).not.toHaveBeenCalledWith([
"launchctl",
"bootstrap",
"gui/501",
oldPath,
]);
});
it("removes the replacement plist and restores the previous registration", async () => {
const f = fixture();
const oldPath = "/Users/example/Library/LaunchAgents/old-runtime.plist";
+1 -16
View File
@@ -72,13 +72,6 @@ export type PreparedServiceInstall = {
mode: "direct" | "bootstrap-existing" | "restart" | "replace";
previous?: ManagedSnapshot;
previouslyEnabled: boolean;
/**
* Re-bootstrap the previous (possibly older) LaunchAgent when the new one
* fails to bootstrap. `install` turns this off once the home has been handed
* to this version's SDK migration: an older Runtime has no migration gate and
* would serve a home that is not on SDK authority.
*/
restorePreviousOnFailure?: boolean;
};
const defaultDependencies: ServiceLifecycleDependencies = {
@@ -198,8 +191,7 @@ export async function applyPreparedServiceInstall(
if (!previous) throw new Error("runtime_service_upgrade_state_invalid");
await verifyPreparedRegistration(prepared, dependencies);
const domain = launchdDomain(options.uid);
let replacementFailure: "upgrade" | "rollback" | "not-restored" | null =
null;
let replacementFailure: "upgrade" | "rollback" | null = null;
const replaceCommand: ServiceCommandRunner = async (command) => {
if (command[0] !== "launchctl" || command[1] !== "bootstrap") {
await dependencies.runCommand(command);
@@ -216,10 +208,6 @@ export async function applyPreparedServiceInstall(
await dependencies.runCommand(command);
} catch {
replacementFailure = "upgrade";
if (prepared.restorePreviousOnFailure === false) {
replacementFailure = "not-restored";
throw new Error("runtime_service_replacement_failed");
}
try {
await dependencies.verifyRegistrationSnapshot(previous);
await dependencies.runCommand([
@@ -246,9 +234,6 @@ export async function applyPreparedServiceInstall(
if (replacementFailure === "upgrade") {
throw new Error("runtime_service_managed_upgrade_failed");
}
if (replacementFailure === "not-restored") {
throw new Error("runtime_service_replacement_failed");
}
throw error;
}
} catch (error) {
+36 -9
View File
@@ -7,10 +7,12 @@ import type {
} from "@one-memory/contracts";
import type { CoreReader, ReadOutcome } from "./core-client.js";
import {
type ValidatedMemoryTask,
validateImportIssues,
validateImportJob,
validateImportJobs,
validateLiveSync,
validateMemoryTasks,
validateProcessing,
validateSources,
} from "./core-responses.js";
@@ -68,6 +70,11 @@ export interface ProgressDocument {
generation: {
summary_state: ReadState;
summary: Record<string, unknown> | null;
tasks_state: ReadState;
tasks: Array<Record<string, unknown>>;
returned: number;
validated: number;
output_truncated: boolean;
};
}
@@ -127,13 +134,12 @@ export async function readProgress(
const importsPath = options.jobId
? `/api/v1/import-jobs/${options.jobId}`
: "/api/v1/import-jobs";
// Generation progress comes from `memory-processing` alone: under SDK
// authority the per-task list (`/api/v1/memory-tasks`) is retired and empty.
const [importsRead, sourcesRead, processingRead, issuesRead] =
const [importsRead, sourcesRead, processingRead, tasksRead, issuesRead] =
await Promise.all([
reader<unknown>(importsPath),
reader<unknown>("/api/v1/sources"),
reader<unknown>("/api/v1/memory-processing"),
reader<unknown>("/api/v1/memory-tasks"),
options.jobId && options.issues
? reader<unknown>(`/api/v1/import-jobs/${options.jobId}/issues`)
: Promise.resolve(null),
@@ -147,6 +153,7 @@ export async function readProgress(
: validated(importsRead, validateImportJobs);
const processing = validated(processingRead, validateProcessing);
const sources = validated(sourcesRead, validateSources);
const tasks = validated(tasksRead, validateMemoryTasks);
const issues =
options.jobId && options.issues && issuesRead
? validated(issuesRead, (value) =>
@@ -155,19 +162,25 @@ export async function readProgress(
: null;
const jobs = imports.state === "ok" ? imports.data : [];
const taskItems = tasks.state === "ok" ? tasks.data : [];
const issueItems = issues?.state === "ok" ? issues.data : [];
const importState = stateOnly(imports);
const summaryState = stateOnly(processing);
const tasksState = stateOnly(tasks);
const hasReadFailure =
importState.state !== "ok" ||
sources.state !== "ok" ||
summaryState.state !== "ok" ||
tasksState.state !== "ok" ||
(issues !== null && issues.state !== "ok");
const hasProblem =
jobs.some((job) =>
["paused", "interrupted", "failed"].includes(job.status),
) ||
taskItems.some((task) =>
["waiting_for_model", "failed"].includes(task.status),
) ||
(processing.state === "ok" &&
(processing.data.failed > 0 ||
processing.data.waiting_for_model > 0 ||
@@ -175,6 +188,7 @@ export async function readProgress(
issueItems.length > 0;
const hasActive =
jobs.some((job) => ["pending", "running"].includes(job.status)) ||
taskItems.some((task) => ["pending", "running"].includes(task.status)) ||
(processing.state === "ok" &&
(processing.data.pending > 0 || processing.data.running > 0));
@@ -215,6 +229,11 @@ export async function readProgress(
summary_state: summaryState,
summary:
processing.state === "ok" ? projectProcessing(processing.data) : null,
tasks_state: tasksState,
tasks: taskItems.slice(0, options.limit).map(projectTask),
returned: Math.min(taskItems.length, options.limit),
validated: taskItems.length,
output_truncated: taskItems.length > options.limit,
},
};
}
@@ -270,7 +289,10 @@ export function renderProgressText(document: ProgressDocument): string {
? `记忆整理:待处理 ${document.generation.summary.pending} · 进行中 ${document.generation.summary.running} · 等待模型 ${document.generation.summary.waiting_for_model} · 失败 ${document.generation.summary.failed}`
: "记忆整理:暂时无法读取",
);
if (document.imports.output_truncated) {
if (
document.imports.output_truncated ||
document.generation.output_truncated
) {
lines.push("这里只显示汇总;完整记录请使用 --json。");
}
lines.push("");
@@ -331,6 +353,11 @@ export function unreadProgress(
generation: {
summary_state: failed,
summary: null,
tasks_state: failed,
tasks: [],
returned: 0,
validated: 0,
output_truncated: false,
},
};
}
@@ -341,11 +368,7 @@ function validated<T>(
):
| { state: "ok"; data: T }
| { state: Exclude<ReadState["state"], "ok">; code: string } {
if (outcome.state !== "ok") {
// A Core refusal of the request itself reads as an invalid answer here.
const state = outcome.state === "rejected" ? "invalid" : outcome.state;
return { state, code: outcome.code };
}
if (outcome.state !== "ok") return outcome;
const data = validate(outcome.data);
return data === null
? { state: "invalid", code: "response_invalid" }
@@ -442,3 +465,7 @@ function projectProcessing(
})),
};
}
function projectTask(task: ValidatedMemoryTask): Record<string, unknown> {
return { ...task };
}
+48 -6
View File
@@ -129,6 +129,18 @@ const RESPONSES = new Map<string, { status: number; data: unknown }>([
},
},
],
[
"/api/v1/memory-tasks",
{
status: 200,
data: [
memoryTask("task-pending", "pending"),
memoryTask("task-running", "running"),
memoryTask("task-model", "waiting_for_model", "model_missing"),
memoryTask("task-failed", "failed", "provider_unavailable"),
],
},
],
[
"/api/v1/import-jobs/job-running",
{ status: 200, data: importJob("job-running", "running") },
@@ -270,9 +282,9 @@ describe("CLI-19 progress and issue reads", () => {
imported_messages: 20,
totals_state: { state: "ok" },
});
// The retired per-task list is no longer read or reported.
expect(report.generation).not.toHaveProperty("tasks");
expect(requestedPaths()).not.toContain("/api/v1/memory-tasks");
expect(
report.generation.tasks.map((task: { status: string }) => task.status),
).toEqual(["pending", "running", "waiting_for_model", "failed"]);
expect(streams.stdout()).not.toMatch(/percent|percentage|%/i);
});
@@ -362,7 +374,7 @@ describe("CLI-19 progress and issue reads", () => {
expect(streams.stdout()).not.toContain("SYNTHETIC_FOREIGN_ISSUE");
});
it("bounds the import job list and emits an explicit truncation receipt", async () => {
it("bounds each task list and emits an explicit truncation receipt", async () => {
const streams = captureStreams();
try {
expect(await run(["progress", "--limit", "1", "--json"], {})).toBe(1);
@@ -376,6 +388,12 @@ describe("CLI-19 progress and issue reads", () => {
validated: 3,
output_truncated: true,
});
expect(report.generation.tasks).toHaveLength(1);
expect(report.generation).toMatchObject({
returned: 1,
validated: 4,
output_truncated: true,
});
});
it("streams bounded progress snapshots as serial NDJSON", async () => {
@@ -410,6 +428,7 @@ describe("CLI-19 progress and issue reads", () => {
"/api/v1/import-jobs",
"/api/v1/sources",
"/api/v1/memory-processing",
"/api/v1/memory-tasks",
]) {
expect(requestedPaths().filter((item) => item === path)).toHaveLength(2);
}
@@ -481,7 +500,7 @@ describe("CLI-19 progress and issue reads", () => {
it("preserves denied, unavailable and malformed sections in one partial report", async () => {
transport.get.mockImplementation(async (url: string) => {
if (url === "/api/v1/import-jobs") return { status: 503, data: {} };
if (url === "/api/v1/sources") return { status: 403, data: {} };
if (url === "/api/v1/memory-tasks") return { status: 403, data: {} };
if (url === "/api/v1/memory-processing") {
return { status: 200, data: { pending: { secret: "nested" } } };
}
@@ -502,7 +521,7 @@ describe("CLI-19 progress and issue reads", () => {
state: "invalid",
code: "response_invalid",
});
expect(report.imports.totals_state).toEqual({
expect(report.generation.tasks_state).toEqual({
state: "denied",
code: "http_403",
});
@@ -555,6 +574,7 @@ describe("CLI-19 progress and issue reads", () => {
"/api/v1/live-sync",
"/api/v1/import-jobs",
"/api/v1/memory-processing",
"/api/v1/memory-tasks",
].some((allowed) => path === allowed || path.startsWith(`${allowed}/`)),
),
).toBe(true);
@@ -614,6 +634,28 @@ function importJob(
};
}
function memoryTask(
taskId: string,
status: string,
error: string | null = null,
) {
return {
task_id: taskId,
request_id: `request-${taskId}`,
user_id: "local",
kind: "extract",
input_ref: "synthetic",
status,
profile_id: "profile-1",
profile_revision: 1,
profile_snapshot: {},
attempts: status === "failed" ? 2 : 0,
last_error_code: error,
created_at: "T1",
updated_at: "T2",
};
}
function requestedPaths(): string[] {
return (transport.get.mock.calls as [string][]).map(([path]) => path);
}
+1 -43
View File
@@ -1,11 +1,6 @@
import { describe, expect, it } from "vitest";
import { composeDoctorReport, renderDoctorText } from "./doctor-report.js";
import {
composeStatusReport,
renderStatusText,
type StatusDocument,
} from "./status-report.js";
import { renderStatusText, type StatusDocument } from "./status-report.js";
describe("status default output", () => {
it("formats blocked task objects recursively instead of printing object coercions", () => {
@@ -40,40 +35,3 @@ describe("status default output", () => {
);
});
});
describe("status and doctor on an unfinished upgrade migration", () => {
it("names the migration log and the issues page", async () => {
const report = await composeStatusReport({
dataDir: "/tmp/isolated/.one-memory",
runtime: { state: "stopped", code: "sdk_migration_required" },
background: null,
reader: null,
});
expect(report.runtime.log_file).toBe(
"/tmp/isolated/.one-memory/logs/sdk-migration.log",
);
expect(report.runtime.issues_url).toBe(
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
);
const text = renderStatusText(report, "MemHub");
expect(text).toContain("sdk_migration_required");
expect(text).toContain("/tmp/isolated/.one-memory/logs/sdk-migration.log");
const doctor = renderDoctorText(composeDoctorReport(report, null));
expect(doctor).toContain(
"/tmp/isolated/.one-memory/logs/sdk-migration.log",
);
expect(doctor).not.toContain("帮助与诊断");
});
it("adds no log path for an ordinary stopped runtime", async () => {
const report = await composeStatusReport({
dataDir: "/tmp/isolated/.one-memory",
runtime: { state: "stopped", code: "runtime_not_running" },
background: null,
reader: null,
});
expect(report.runtime.log_file).toBeUndefined();
});
});
+3 -23
View File
@@ -1,4 +1,5 @@
import type {
HomeSummary,
LiveSyncHealth,
MemoryProcessingSummary,
ModelProfile,
@@ -8,10 +9,8 @@ import type {
BackgroundHealth,
RuntimeStatus,
} from "@one-memory/runtime-control";
import { sdkMigrationLogPath } from "@one-memory/runtime-control";
import type { CoreReader, ReadOutcome } from "./core-client.js";
import {
type SdkHomeSummary,
validateHomeSummary,
validateLiveSync,
validateModelProfiles,
@@ -20,9 +19,6 @@ import {
} from "./core-responses.js";
import { PRODUCT } from "./version.js";
const MIGRATION_ISSUES_URL =
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new";
export type SectionState =
| "ok"
| "denied"
@@ -46,9 +42,6 @@ export interface StatusDocument {
host?: string;
port?: number;
version?: string;
/** Set only when an unfinished upgrade migration is why nothing serves. */
log_file?: string;
issues_url?: string;
};
bridge: {
state: BackgroundHealth["state"] | "not_checked";
@@ -109,7 +102,7 @@ export async function composeStatusReport(options: {
reader<unknown>("/api/v1/live-sync"),
reader<unknown>("/api/v1/memory-processing"),
reader<unknown>("/api/v1/model-profiles"),
reader<unknown>("/api/v2/home-summary"),
reader<unknown>("/api/v1/home-summary"),
]);
sections.sources = section(sources, validateSources, describeSources);
sections.sync = section(sync, validateLiveSync, describeSync);
@@ -160,12 +153,6 @@ export async function composeStatusReport(options: {
host: options.runtime.host,
port: options.runtime.port,
version: options.runtime.version,
...(options.runtime.code.startsWith("sdk_migration_")
? {
log_file: sdkMigrationLogPath(options.dataDir),
issues_url: MIGRATION_ISSUES_URL,
}
: {}),
},
bridge,
sections,
@@ -190,13 +177,6 @@ export function renderStatusText(
): string {
const lines = [
`${heading}:${runtimeLine(report)}`,
...(report.runtime.log_file
? [
`升级迁移没有完成(错误码:${report.runtime.code}),MemHub 不会启动。`,
`请把迁移日志附在 GitHub 反馈里发给我们:${report.runtime.log_file}`,
`反馈地址:${report.runtime.issues_url}`,
]
: []),
`资料空间:${report.data_dir}`,
`应用连接:${BRIDGE_LABELS[report.bridge.state]}`,
];
@@ -367,7 +347,7 @@ function describeModel(profiles: ModelProfile[]): Record<string, unknown> {
};
}
function describeMemories(summary: SdkHomeSummary): Record<string, unknown> {
function describeMemories(summary: HomeSummary): Record<string, unknown> {
return {
memory_count: summary.memory_count,
};
-4
View File
@@ -102,10 +102,6 @@ export function uiSessionFailure(code: string): { code: string } {
export function uiExitCode(result: UiResult): number {
if (result.state === "opened") return 0;
// A stopped Runtime is the same exit 3 every other command reports.
if (result.state === "blocked" && result.code === "runtime_not_running") {
return 3;
}
if (result.state === "blocked") return 4;
return 1;
}
+479
View File
@@ -0,0 +1,479 @@
import { mkdtemp, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { run } from "./cli.js";
const transport = vi.hoisted(() => ({
get: vi.fn(),
post: vi.fn(),
put: vi.fn(),
patch: vi.fn(),
delete: vi.fn(),
}));
vi.mock("axios", () => ({
default: { create: vi.fn(() => transport), get: vi.fn() },
}));
const runtime = vi.hoisted(() => ({
inspectRuntimeStatus: vi.fn(),
discoverRuntimeAttachment: vi.fn(),
}));
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
...runtime,
}));
const EVENT = "11111111-1111-4111-8111-111111111111";
const UPLOAD = "22222222-2222-4222-8222-222222222222";
const NOTE = "33333333-3333-4333-8333-333333333333";
beforeEach(() => {
vi.clearAllMocks();
// clearAllMocks keeps prior mock implementations, so a mockResolvedValue
// from an earlier test would leak into the next one and fake its response.
transport.get.mockReset();
transport.post.mockReset();
transport.put.mockReset();
transport.patch.mockReset();
transport.delete.mockReset();
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "running",
code: "runtime_verified",
instanceId: "instance-1",
host: "127.0.0.1",
port: 8765,
version: "1.2.3",
});
runtime.discoverRuntimeAttachment.mockResolvedValue({
instanceId: "instance-1",
apiToken: "synthetic-runtime-token",
host: "127.0.0.1",
port: 8765,
servicePid: 4242,
health: {
status: "ok",
service: "one-memory-core",
version: "1.2.3",
instance_id: "instance-1",
},
});
});
describe("calls clear", () => {
it("reports runtime_stopped (exit 3) without attempting a write", async () => {
runtime.inspectRuntimeStatus.mockResolvedValue({
state: "stopped",
code: "runtime_not_running",
});
const result = await capturedRun([
"calls",
"clear",
"--all",
"--confirm",
"--json",
]);
expect(result.code).toBe(3);
expect(JSON.parse(result.stdout).code).toBe("runtime_not_running");
expect(transport.post).not.toHaveBeenCalled();
});
it("previews without contacting Core", async () => {
const result = await capturedRun([
"calls",
"clear",
"--event",
EVENT,
"--json",
]);
expect(result.code).toBe(0);
const document = JSON.parse(result.stdout);
expect(document.state).toBe("preview");
expect(document.plan.kind).toBe("call_events_clear");
expect(transport.post).not.toHaveBeenCalled();
expect(transport.delete).not.toHaveBeenCalled();
});
it("sends the exact Core body with --confirm", async () => {
transport.post.mockResolvedValue({ status: 200, data: { removed: 1 } });
const result = await capturedRun([
"calls",
"clear",
"--event",
EVENT,
"--confirm",
"--json",
]);
expect(result.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
"/api/v1/call-events/clear",
{ event_ids: [EVENT], user_confirmed: true },
undefined,
);
expect(JSON.parse(result.stdout).state).toBe("committed");
});
it("clears all without naming events", async () => {
transport.post.mockResolvedValue({ status: 200, data: { removed: 4 } });
const result = await capturedRun([
"calls",
"clear",
"--all",
"--confirm",
"--json",
]);
expect(result.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
"/api/v1/call-events/clear-all",
{ user_confirmed: true },
undefined,
);
});
it("rejects a malformed event id without contacting Core", async () => {
const result = await capturedRun([
"calls",
"clear",
"--event",
"not-a-uuid",
"--json",
]);
expect(result.code).toBe(2);
expect(transport.post).not.toHaveBeenCalled();
});
it("never reports committed for a malformed success body", async () => {
transport.post.mockResolvedValue({ status: 200, data: { wrong: true } });
const result = await capturedRun([
"calls",
"clear",
"--event",
EVENT,
"--confirm",
"--json",
]);
expect(result.code).toBe(1);
const document = JSON.parse(result.stdout);
expect(document.state).toBe("unknown");
expect(document.code).toBe("ui_write_receipt_invalid");
});
});
describe.skip("retired import-file", () => {
it("never uploads without --confirm", async () => {
const directory = await mkdtemp(path.join(tmpdir(), "om-import-"));
const file = path.join(directory, "export.json");
await writeFile(file, '{"conversations":[]}', { mode: 0o600 });
const result = await capturedRun([
"import-file",
"upload",
"--file",
file,
"--json",
]);
expect(result.code).toBe(0);
const document = JSON.parse(result.stdout);
expect(document.state).toBe("preview");
expect(document.plan.operation_id).toBeDefined();
expect(transport.post).not.toHaveBeenCalled();
});
it("uploads the file with a deterministic operation id on confirm", async () => {
const directory = await mkdtemp(path.join(tmpdir(), "om-import-"));
const file = path.join(directory, "export.json");
await writeFile(file, '{"conversations":[]}', { mode: 0o600 });
transport.post.mockResolvedValue({
status: 201,
data: {
schema_version: 1,
operation_id: "ignored-here",
state: "completed",
upload_id: UPLOAD,
upload_state: "staged",
},
});
const preview = await capturedRun([
"import-file",
"upload",
"--file",
file,
"--json",
]);
const plan = JSON.parse(preview.stdout).plan;
transport.post.mockResolvedValue({
status: 201,
data: {
schema_version: 1,
operation_id: plan.operation_id,
state: "completed",
upload_id: UPLOAD,
upload_state: "staged",
},
});
const result = await capturedRun([
"import-file",
"upload",
"--file",
file,
"--confirm",
"--json",
]);
expect(result.code).toBe(0);
const url = transport.post.mock.calls.at(-1)?.[0] as string;
expect(url).toContain("/api/v1/import-files?");
expect(url).toContain(`operation_id=${plan.operation_id}`);
expect(url).toContain("filename=export.json");
expect(JSON.parse(result.stdout).state).toBe("committed");
});
it("commits and discards only with explicit confirmation", async () => {
const preview = await capturedRun([
"import-file",
"commit",
UPLOAD,
"--json",
]);
expect(preview.code).toBe(0);
expect(JSON.parse(preview.stdout).state).toBe("preview");
expect(transport.post).not.toHaveBeenCalled();
transport.post.mockResolvedValue({
status: 200,
data: {
upload_id: UPLOAD,
filename: "x.json",
vendor: "generic",
source_id: "generic-export",
messages: 0,
conversations: 0,
human_messages: 0,
agent_messages: 0,
skipped_records: 0,
ignored_files: 0,
checksum: "a".repeat(64),
created_at: "T",
committed: true,
},
});
const committed = await capturedRun([
"import-file",
"commit",
UPLOAD,
"--confirm",
"--json",
]);
expect(committed.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
`/api/v1/import-files/${UPLOAD}/commit`,
{ confirmed: true },
undefined,
);
transport.delete.mockResolvedValue({ status: 204, data: "" });
const discarded = await capturedRun([
"import-file",
"discard",
UPLOAD,
"--confirm",
"--json",
]);
expect(discarded.code).toBe(0);
expect(JSON.parse(discarded.stdout).state).toBe("committed");
expect(transport.delete).toHaveBeenCalledWith(
`/api/v1/import-files/${UPLOAD}`,
undefined,
);
});
it("maps a Core 404 to a stable rejected code, not a fake success", async () => {
transport.post.mockResolvedValue({
status: 404,
data: { detail: "import_file_not_found" },
});
const result = await capturedRun([
"import-file",
"commit",
UPLOAD,
"--confirm",
"--json",
]);
expect(result.code).toBe(1);
const document = JSON.parse(result.stdout);
expect(document.state).toBe("rejected");
expect(document.code).toBe("import_file_not_found");
});
});
describe("notes / extraction / graph / manual extract", () => {
it("adds a note from a bounded file and re-verifies content on apply", async () => {
const directory = await mkdtemp(path.join(tmpdir(), "om-note-"));
const content = path.join(directory, "note.txt");
await writeFile(content, "记住这个决定", { mode: 0o600 });
const preview = await capturedRun([
"notes",
"add",
"--content-file",
content,
"--json",
]);
expect(preview.code).toBe(0);
const planFile = path.join(directory, "plan.json");
await writeFile(planFile, preview.stdout, { mode: 0o600 });
transport.post.mockResolvedValue({
status: 201,
// A realistic Core `MemoryNoteReceipt`: the shared parser requires
// `scope` and `memory_ids` too, so a minimal `{evidence_id,state}` stub
// is a malformed body, not a success.
data: {
evidence_id: NOTE,
text: "记住这个决定",
scope: "all-apps",
created_at: "2026-09-19T00:00:00Z",
state: "pending",
task_id: null,
memory_ids: [],
error: null,
},
});
const applied = await capturedRun([
"notes",
"apply",
"--plan-file",
planFile,
"--content-file",
content,
"--confirm",
"--json",
]);
expect(applied.code).toBe(0);
const body = transport.post.mock.calls.at(-1)?.[1] as Record<
string,
unknown
>;
expect(body.text).toBe("记住这个决定");
expect(body.scope).toBe("all-apps");
expect(JSON.parse(applied.stdout).state).toBe("committed");
});
it.skip("plans then applies the extraction window", async () => {
const preview = await capturedRun([
"extraction",
"set",
"--window",
"6m",
"--json",
]);
expect(preview.code).toBe(0);
expect(JSON.parse(preview.stdout).state).toBe("preview");
expect(transport.put).not.toHaveBeenCalled();
transport.put.mockResolvedValue({
status: 200,
data: { window_preset: "6m" },
});
const applied = await capturedRun([
"extraction",
"set",
"--window",
"6m",
"--confirm",
"--json",
]);
expect(applied.code).toBe(0);
expect(transport.put).toHaveBeenCalledWith(
"/api/v1/extraction-settings",
{ window_preset: "6m" },
undefined,
);
});
it("does not call graph width without --confirm", async () => {
const preview = await capturedRun([
"graph",
"width",
"--decision",
"rebuild_with_new",
"--json",
]);
expect(preview.code).toBe(0);
expect(transport.post).not.toHaveBeenCalled();
transport.post.mockResolvedValue({
status: 200,
data: { archived_path: "/tmp/old", exported_path: null, mismatch: null },
});
const applied = await capturedRun([
"graph",
"width",
"--decision",
"rebuild_with_new",
"--confirm",
"--json",
]);
expect(applied.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
"/api/v1/graph-width",
{ decision: "rebuild_with_new" },
undefined,
);
});
it.skip("plans manual extraction and applies it only on confirm", async () => {
const preview = await capturedRun([
"processing",
"extract",
"--evidence",
NOTE,
"--json",
]);
expect(preview.code).toBe(0);
expect(JSON.parse(preview.stdout).state).toBe("preview");
expect(transport.post).not.toHaveBeenCalled();
transport.post.mockResolvedValue({
status: 200,
data: { promoted: 1, batches_planned: 2, batches_current: 1 },
});
const applied = await capturedRun([
"processing",
"extract",
"--evidence",
NOTE,
"--confirm",
"--json",
]);
expect(applied.code).toBe(0);
expect(transport.post).toHaveBeenCalledWith(
"/api/v1/memory-processing/manual-extract",
{ evidence_ids: [NOTE] },
undefined,
);
});
});
async function capturedRun(arguments_: string[]) {
let stdout = "";
let stderr = "";
const stdoutSpy = vi
.spyOn(process.stdout, "write")
.mockImplementation((chunk) => {
stdout += String(chunk);
return true;
});
const stderrSpy = vi
.spyOn(process.stderr, "write")
.mockImplementation((chunk) => {
stderr += String(chunk);
return true;
});
try {
return {
code: await run(arguments_, {
ONE_MEMORY_DATA_DIR: "/synthetic/data",
}),
stdout,
stderr,
};
} finally {
stdoutSpy.mockRestore();
stderrSpy.mockRestore();
}
}
@@ -0,0 +1,501 @@
/**
* The remaining Web Console mutating operations the CLI did not expose:
* 端内新增记忆 (memory-notes), extraction window, graph width, and manual
* extraction.
*
* Core owns every rule; this module only validates the input it is about to
* send, performs one explicit two-phase write, and normalises the receipt so
* the CLI never reports a write as committed without evidence.
*/
import { createHash, randomUUID } from "node:crypto";
import {
buildMemoryNoteSaveRequest,
MemoryNoteInputError,
parseExtractionSettingsReceipt,
parseGraphWidthDecisionReceipt,
parseManualExtractReceipt,
parseMemoryNoteReceipt,
} from "@one-memory/operations";
import type { CoreOperations, CorePutOperations } from "./core-operations.js";
import { PRODUCT } from "./version.js";
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const DIGEST = /^[0-9a-f]{64}$/;
const NOTE_MAX_CHARACTERS = 12_000;
const EXTRACTION_PRESETS = ["3m", "6m", "1y", "all"] as const;
const GRAPH_DECISIONS = [
"keep_existing",
"rebuild_with_new",
"export_first",
] as const;
const NOTE_STATES = [
"waiting_for_model",
"waiting_for_permission",
"pending",
"running",
"failed",
"ready",
"no_memories",
] as const;
export type NoteState = (typeof NOTE_STATES)[number];
export type ExtendedWritePlan =
| {
schema_version: 1;
kind: "memory_note_add";
request_id: string;
content_sha256: string;
content_characters: number;
digest: string;
}
| {
schema_version: 1;
kind: "memory_note_process";
evidence_id: string;
digest: string;
}
| {
schema_version: 1;
kind: "extraction_window_set";
window_preset: (typeof EXTRACTION_PRESETS)[number];
digest: string;
}
| {
schema_version: 1;
kind: "graph_width_decide";
decision: (typeof GRAPH_DECISIONS)[number];
digest: string;
}
| {
schema_version: 1;
kind: "manual_extract";
evidence_ids: string[];
digest: string;
};
export type ExtendedWriteOutcome =
| {
schema_version: 1;
product: string;
state: "preview";
plan: ExtendedWritePlan;
}
| {
schema_version: 1;
product: string;
state: "committed" | "recovered";
operation_id: string;
receipt: Record<string, unknown>;
}
| {
schema_version: 1;
product: string;
state: "rejected" | "unknown" | "invalid";
operation_id?: string;
code: string;
};
function canonical(value: unknown): string {
if (Array.isArray(value)) return `[${value.map(canonical).join(",")}]`;
if (value !== null && typeof value === "object") {
return `{${Object.entries(value as Record<string, unknown>)
.sort(([left], [right]) => left.localeCompare(right))
.map(([key, item]) => `${JSON.stringify(key)}:${canonical(item)}`)
.join(",")}}`;
}
if (value === undefined) return "null";
return JSON.stringify(value);
}
function digestOf(value: unknown): string {
return createHash("sha256").update(canonical(value)).digest("hex");
}
type Draft =
| Omit<Extract<ExtendedWritePlan, { kind: "memory_note_add" }>, "digest">
| Omit<Extract<ExtendedWritePlan, { kind: "memory_note_process" }>, "digest">
| Omit<
Extract<ExtendedWritePlan, { kind: "extraction_window_set" }>,
"digest"
>
| Omit<Extract<ExtendedWritePlan, { kind: "graph_width_decide" }>, "digest">
| Omit<Extract<ExtendedWritePlan, { kind: "manual_extract" }>, "digest">;
function preview(plan: Draft): ExtendedWriteOutcome {
return {
schema_version: 1,
product: PRODUCT,
state: "preview",
plan: { ...plan, digest: digestOf(plan) } as ExtendedWritePlan,
};
}
function invalid(code: string): ExtendedWriteOutcome {
return { schema_version: 1, product: PRODUCT, state: "invalid", code };
}
function committed(
operationId: string,
receipt: Record<string, unknown>,
state: "committed" | "recovered" = "committed",
): ExtendedWriteOutcome {
return {
schema_version: 1,
product: PRODUCT,
state,
operation_id: operationId,
receipt,
};
}
function rejected(
operationId: string | undefined,
code: string,
): ExtendedWriteOutcome {
return {
schema_version: 1,
product: PRODUCT,
state: "rejected",
...(operationId === undefined ? {} : { operation_id: operationId }),
code,
};
}
function unknown(
operationId: string | undefined,
code: string,
): ExtendedWriteOutcome {
return {
schema_version: 1,
product: PRODUCT,
state: "unknown",
...(operationId === undefined ? {} : { operation_id: operationId }),
code,
};
}
// --- memory-notes ---------------------------------------------------------
export function planMemoryNoteAdd(text: string): ExtendedWriteOutcome {
// The plan keeps only the note's identity, like `memories plan add`: the
//正文 lives in a bounded file and is re-read and re-hashed at apply time, so
// a plan file never carries the user's private text.
if (!validNoteText(text)) return invalid("memory_note_text_invalid");
const requestId = randomUUID();
return preview({
schema_version: 1,
kind: "memory_note_add",
request_id: requestId,
content_sha256: createHash("sha256").update(text, "utf8").digest("hex"),
content_characters: [...text].length,
});
}
export function validNoteText(text: string): boolean {
return (
text.length > 0 &&
[...text].length <= NOTE_MAX_CHARACTERS &&
text.trim().length > 0 &&
![...text].some((character) => {
const code = character.charCodeAt(0);
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
})
);
}
export function planMemoryNoteProcess(
evidenceId: string,
): ExtendedWriteOutcome {
return UUID.test(evidenceId)
? preview({
schema_version: 1,
kind: "memory_note_process",
evidence_id: evidenceId,
})
: invalid("memory_note_id_invalid");
}
function noteReceipt(
operationId: string,
data: { evidence_id: string; state: NoteState },
): ExtendedWriteOutcome {
return committed(operationId, { ...data });
}
export async function executeMemoryNoteAdd(
plan: Extract<ExtendedWritePlan, { kind: "memory_note_add" }>,
operations: CoreOperations,
text: string,
): Promise<ExtendedWriteOutcome> {
if (
!validNoteText(text) ||
createHash("sha256").update(text, "utf8").digest("hex") !==
plan.content_sha256 ||
[...text].length !== plan.content_characters
) {
return invalid("memory_note_content_mismatch");
}
// The shared builder owns the save body (request_id/text/scope=all-apps), so
// the CLI and the Web console post the same three fields. Its own text check
// already ran above, so a throw here means an unusable plan.
let request: ReturnType<typeof buildMemoryNoteSaveRequest>;
try {
request = buildMemoryNoteSaveRequest({
request_id: plan.request_id,
text,
scope: "all-apps",
} as Parameters<typeof buildMemoryNoteSaveRequest>[0]);
} catch (error) {
if (error instanceof MemoryNoteInputError) {
return invalid("memory_note_content_mismatch");
}
throw error;
}
const result = await operations.post<unknown>(request.path, request.body);
if (result.state === "rejected") {
return rejected(plan.request_id, result.code);
}
if (result.state === "unknown") {
return unknown(plan.request_id, result.code);
}
const receipt = asNoteReceipt(result.data);
return receipt === null
? unknown(plan.request_id, "ui_write_receipt_invalid")
: noteReceipt(plan.request_id, receipt);
}
export async function executeMemoryNoteProcess(
plan: Extract<ExtendedWritePlan, { kind: "memory_note_process" }>,
operations: CoreOperations,
): Promise<ExtendedWriteOutcome> {
const result = await operations.post<unknown>(
`/api/v1/memory-notes/${plan.evidence_id}/process`,
{},
);
if (result.state === "rejected") {
return rejected(plan.evidence_id, result.code);
}
if (result.state === "unknown") {
return unknown(plan.evidence_id, result.code);
}
const receipt = asNoteReceipt(result.data);
return receipt === null
? unknown(plan.evidence_id, "ui_write_receipt_invalid")
: noteReceipt(plan.evidence_id, receipt);
}
function asNoteReceipt(
value: unknown,
): { evidence_id: string; state: NoteState } | null {
// The shared parser is the single judge of a note receipt; a malformed body
// returns null and the caller reports `unknown`, never a saved note.
const receipt = parseMemoryNoteReceipt(value);
return receipt === null
? null
: { evidence_id: receipt.evidence_id, state: receipt.state as NoteState };
}
// --- extraction window / graph width / manual extract ---------------------
export function planExtractionWindowSet(
windowPreset: string,
): ExtendedWriteOutcome {
return (EXTRACTION_PRESETS as readonly string[]).includes(windowPreset)
? preview({
schema_version: 1,
kind: "extraction_window_set",
window_preset: windowPreset as (typeof EXTRACTION_PRESETS)[number],
})
: invalid("extraction_window_preset_invalid");
}
export function planGraphWidthDecide(decision: string): ExtendedWriteOutcome {
return (GRAPH_DECISIONS as readonly string[]).includes(decision)
? preview({
schema_version: 1,
kind: "graph_width_decide",
decision: decision as (typeof GRAPH_DECISIONS)[number],
})
: invalid("graph_width_decision_invalid");
}
export function planManualExtract(evidenceIds: string[]): ExtendedWriteOutcome {
const ids = [...new Set(evidenceIds)];
return ids.length === evidenceIds.length && ids.every((id) => UUID.test(id))
? preview({
schema_version: 1,
kind: "manual_extract",
evidence_ids: ids,
})
: invalid("manual_extract_evidence_ids_invalid");
}
export async function executeExtractionWindowSet(
plan: Extract<ExtendedWritePlan, { kind: "extraction_window_set" }>,
operations: CorePutOperations,
): Promise<ExtendedWriteOutcome> {
const result = await operations.put<unknown>("/api/v1/extraction-settings", {
window_preset: plan.window_preset,
});
if (result.state === "rejected") {
return rejected(plan.window_preset, result.code);
}
if (result.state === "unknown") {
return unknown(plan.window_preset, result.code);
}
// The shared parser also enforces that the receipt echoes the requested
// preset, so a response belonging to another write cannot pass.
const parsed = parseExtractionSettingsReceipt(
result.data,
plan.window_preset,
);
if (parsed === null) {
return unknown(plan.window_preset, "ui_write_receipt_invalid");
}
return committed(plan.window_preset, { window_preset: plan.window_preset });
}
export async function executeGraphWidthDecide(
plan: Extract<ExtendedWritePlan, { kind: "graph_width_decide" }>,
operations: CoreOperations,
): Promise<ExtendedWriteOutcome> {
const result = await operations.post<unknown>("/api/v1/graph-width", {
decision: plan.decision,
});
if (result.state === "rejected") {
return rejected(plan.decision, result.code);
}
if (result.state === "unknown") {
return unknown(plan.decision, result.code);
}
const receipt = parseGraphWidthDecisionReceipt(result.data);
if (receipt === null) {
return unknown(plan.decision, "ui_write_receipt_invalid");
}
return committed(plan.decision, {
archived_path: receipt.archived_path ?? null,
exported_path: receipt.exported_path ?? null,
mismatch: receipt.mismatch ?? null,
});
}
export async function executeManualExtract(
plan: Extract<ExtendedWritePlan, { kind: "manual_extract" }>,
operations: CoreOperations,
): Promise<ExtendedWriteOutcome> {
const result = await operations.post<unknown>(
"/api/v1/memory-processing/manual-extract",
{ evidence_ids: plan.evidence_ids },
);
if (result.state === "rejected") {
return rejected("manual_extract", result.code);
}
if (result.state === "unknown") {
return unknown("manual_extract", result.code);
}
const receipt = parseManualExtractReceipt(result.data);
if (receipt === null) {
return unknown("manual_extract", "ui_write_receipt_invalid");
}
return committed("manual_extract", {
promoted: receipt.promoted,
batches_planned: receipt.batches_planned,
batches_current: receipt.batches_current,
});
}
export function validateExtendedWritePlan(
value: unknown,
): ExtendedWritePlan | null {
if (value === null || typeof value !== "object" || Array.isArray(value)) {
return null;
}
const candidate = value as Record<string, unknown>;
if (candidate.schema_version !== 1 || typeof candidate.kind !== "string") {
return null;
}
const { digest, ...withoutDigest } = candidate;
if (typeof digest !== "string" || digest !== digestOf(withoutDigest)) {
return null;
}
switch (candidate.kind) {
case "memory_note_add":
return typeof candidate.request_id === "string" &&
UUID.test(candidate.request_id) &&
typeof candidate.content_sha256 === "string" &&
DIGEST.test(candidate.content_sha256) &&
Number.isSafeInteger(candidate.content_characters) &&
(candidate.content_characters as number) >= 1 &&
(candidate.content_characters as number) <= NOTE_MAX_CHARACTERS &&
Object.keys(candidate).length === 6
? (candidate as unknown as ExtendedWritePlan)
: null;
case "memory_note_process":
return typeof candidate.evidence_id === "string" &&
UUID.test(candidate.evidence_id) &&
Object.keys(candidate).length === 4
? (candidate as unknown as ExtendedWritePlan)
: null;
case "extraction_window_set":
return (EXTRACTION_PRESETS as readonly string[]).includes(
String(candidate.window_preset),
) && Object.keys(candidate).length === 4
? (candidate as unknown as ExtendedWritePlan)
: null;
case "graph_width_decide":
return (GRAPH_DECISIONS as readonly string[]).includes(
String(candidate.decision),
) && Object.keys(candidate).length === 4
? (candidate as unknown as ExtendedWritePlan)
: null;
case "manual_extract":
return Array.isArray(candidate.evidence_ids) &&
candidate.evidence_ids.every(
(id) => typeof id === "string" && UUID.test(id),
) &&
Object.keys(candidate).length === 4
? (candidate as unknown as ExtendedWritePlan)
: null;
default:
return null;
}
}
export function extendedWriteExitCode(document: ExtendedWriteOutcome): number {
return ["preview", "committed", "recovered"].includes(document.state) ? 0 : 1;
}
export function renderExtendedWriteText(
document: ExtendedWriteOutcome,
): string {
if (document.state === "preview") {
const lines = ["以下操作尚未执行;加 --confirm 后才会写入:"];
const plan = document.plan;
switch (plan.kind) {
case "memory_note_add":
lines.push(`新增端内记忆:${plan.content_characters} 字符`);
break;
case "memory_note_process":
lines.push(`处理端内记忆:${plan.evidence_id}`);
break;
case "extraction_window_set":
lines.push(`设置提取窗口:${plan.window_preset}`);
break;
case "graph_width_decide":
lines.push(`图谱宽度决定:${plan.decision}`);
break;
case "manual_extract":
lines.push(`手动提取:${plan.evidence_ids.length} 条证据`);
break;
}
lines.push(`plan_digest:${plan.digest}`);
return `${lines.join("\n")}\n`;
}
if (document.state === "committed" || document.state === "recovered") {
return `${document.state === "recovered" ? "已从回执恢复" : "已提交"}:${document.operation_id}\n${JSON.stringify(document.receipt)}\n`;
}
const code = "code" in document ? document.code : "ui_write_state_invalid";
return `操作未完成:${document.state}(${code})\n`;
}
+144
View File
@@ -0,0 +1,144 @@
import { describe, expect, it, vi } from "vitest";
import type {
CoreDeleteOperations,
CoreOperations,
} from "./core-operations.js";
import {
executeCallEventsClear,
executeCallEventsClearAll,
executeImportFileCommit,
executeImportFileDiscard,
planCallEventsClear,
planCallEventsClearAll,
planImportFileCommit,
planImportFileDiscard,
planImportFileUpload,
validateUiWritePlan,
} from "./ui-write-flows.js";
const EVENT = "11111111-1111-4111-8111-111111111111";
const UPLOAD = "22222222-2222-4222-8222-222222222222";
function planOf(value: ReturnType<typeof planCallEventsClear>) {
if (value.state !== "preview") throw new Error("expected preview");
return value.plan;
}
describe("UI write plans", () => {
it("permits only valid call-event ids and rejects duplicates", () => {
expect(planCallEventsClear([EVENT]).state).toBe("preview");
expect(planCallEventsClear([]).state).toBe("invalid");
expect(planCallEventsClear([EVENT, EVENT]).state).toBe("invalid");
});
it("plans all remaining UI write operations", () => {
expect(planCallEventsClearAll().state).toBe("preview");
expect(planImportFileCommit(UPLOAD).state).toBe("preview");
expect(planImportFileDiscard(UPLOAD).state).toBe("preview");
expect(
planImportFileUpload({
path: "/tmp/export.json",
bytes: 42,
sha256: "a".repeat(64),
inputKind: "raw_conversation",
}).state,
).toBe("preview");
});
it("rejects a tampered plan digest", async () => {
const plan = planOf(planCallEventsClear([EVENT]));
expect(validateUiWritePlan(plan)).toEqual(plan);
expect(validateUiWritePlan({ ...plan, event_ids: [UPLOAD] })).toBeNull();
});
});
describe("UI write execution", () => {
it("posts clear with Core's confirmation field", async () => {
const post = vi.fn(async () => ({
state: "committed" as const,
data: { removed: 1 },
}));
const plan = planOf(planCallEventsClear([EVENT]));
if (plan.kind !== "call_events_clear") throw new Error("kind");
const result = await executeCallEventsClear(plan, {
post,
} as unknown as CoreOperations);
expect(result.state).toBe("committed");
expect(post).toHaveBeenCalledWith("/api/v1/call-events/clear", {
event_ids: [EVENT],
user_confirmed: true,
});
});
it("posts clear-all with Core's confirmation field", async () => {
const post = vi.fn(async () => ({
state: "committed" as const,
data: { removed: 2 },
}));
const plan = planOf(planCallEventsClearAll());
if (plan.kind !== "call_events_clear_all") throw new Error("kind");
await executeCallEventsClearAll(plan, {
post,
} as unknown as CoreOperations);
expect(post).toHaveBeenCalledWith("/api/v1/call-events/clear-all", {
user_confirmed: true,
});
});
it("uses POST commit and DELETE discard endpoints", async () => {
const post = vi.fn(async () => ({
state: "committed" as const,
data: {
upload_id: UPLOAD,
filename: "x.json",
vendor: "generic",
source_id: "generic-export",
messages: 0,
conversations: 0,
human_messages: 0,
agent_messages: 0,
skipped_records: 0,
ignored_files: 0,
checksum: "a".repeat(64),
created_at: "T",
committed: true,
},
}));
const del = vi.fn(async <T>() => ({
state: "committed" as const,
data: {} as T,
}));
const operations = {
post,
delete: del,
} as unknown as CoreOperations & CoreDeleteOperations;
const commit = planOf(planImportFileCommit(UPLOAD));
if (commit.kind !== "import_file_commit") throw new Error("kind");
await executeImportFileCommit(commit, operations);
expect(post).toHaveBeenCalledWith(`/api/v1/import-files/${UPLOAD}/commit`, {
confirmed: true,
});
const discard = planOf(planImportFileDiscard(UPLOAD));
if (discard.kind !== "import_file_discard") throw new Error("kind");
expect((await executeImportFileDiscard(discard, operations)).state).toBe(
"committed",
);
expect(del).toHaveBeenCalledWith(`/api/v1/import-files/${UPLOAD}`);
});
it("treats a malformed success receipt as unknown", async () => {
const plan = planOf(planCallEventsClear([EVENT]));
if (plan.kind !== "call_events_clear") throw new Error("kind");
const result = await executeCallEventsClear(plan, {
post: vi.fn(async () => ({
state: "committed" as const,
data: { wrong: true },
})),
} as unknown as CoreOperations);
expect(result).toMatchObject({
state: "unknown",
code: "ui_write_receipt_invalid",
});
});
});
+555
View File
@@ -0,0 +1,555 @@
/**
* The Web Console's remaining mutating operations, exposed to the CLI without
* duplicating Core semantics. Each operation is planned first; callers apply
* the exact plan they reviewed.
*/
import { createHash } from "node:crypto";
import { constants } from "node:fs";
import { type FileHandle, open } from "node:fs/promises";
import type {
CoreDeleteOperations,
CoreOperations,
CoreUploadOperations,
} from "./core-operations.js";
import { PRODUCT } from "./version.js";
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
const INPUT_KINDS = ["raw_conversation", "memory_content"] as const;
const MAX_UPLOAD_BYTES = 128 * 1024 * 1024;
export type UiWritePlan =
| {
schema_version: 1;
kind: "call_events_clear";
event_ids: string[];
digest: string;
}
| {
schema_version: 1;
kind: "call_events_clear_all";
digest: string;
}
| {
schema_version: 1;
kind: "import_file_commit";
upload_id: string;
digest: string;
}
| {
schema_version: 1;
kind: "import_file_discard";
upload_id: string;
digest: string;
}
| {
schema_version: 1;
kind: "import_file_upload";
operation_id: string;
path: string;
filename: string;
input_kind: (typeof INPUT_KINDS)[number];
bytes: number;
sha256: string;
digest: string;
};
export type UiWriteOutcome =
| {
schema_version: 1;
product: string;
state: "preview";
plan: UiWritePlan;
}
| {
schema_version: 1;
product: string;
state: "committed" | "recovered";
operation_id: string;
receipt: Record<string, unknown>;
}
| {
schema_version: 1;
product: string;
state: "rejected" | "unknown" | "invalid";
operation_id?: string;
code: string;
receipt?: Record<string, unknown>;
};
function digestOf(value: unknown): string {
return createHash("sha256").update(canonical(value)).digest("hex");
}
function canonical(value: unknown): string {
if (Array.isArray(value)) return `[${value.map(canonical).join(",")}]`;
if (value !== null && typeof value === "object") {
return `{${Object.entries(value as Record<string, unknown>)
.sort(([left], [right]) => left.localeCompare(right))
.map(([key, item]) => `${JSON.stringify(key)}:${canonical(item)}`)
.join(",")}}`;
}
if (value === undefined) return "null";
return JSON.stringify(value);
}
export type UiWritePlanDraft =
| Omit<Extract<UiWritePlan, { kind: "call_events_clear" }>, "digest">
| Omit<Extract<UiWritePlan, { kind: "call_events_clear_all" }>, "digest">
| Omit<Extract<UiWritePlan, { kind: "import_file_commit" }>, "digest">
| Omit<Extract<UiWritePlan, { kind: "import_file_discard" }>, "digest">
| Omit<Extract<UiWritePlan, { kind: "import_file_upload" }>, "digest">;
function preview(plan: UiWritePlanDraft): UiWriteOutcome {
return {
schema_version: 1,
product: PRODUCT,
state: "preview",
plan: { ...plan, digest: digestOf(plan) } as UiWritePlan,
};
}
function invalid(code: string): UiWriteOutcome {
return { schema_version: 1, product: PRODUCT, state: "invalid", code };
}
function committed(
operationId: string,
receipt: Record<string, unknown>,
state: "committed" | "recovered" = "committed",
): UiWriteOutcome {
return {
schema_version: 1,
product: PRODUCT,
state,
operation_id: operationId,
receipt,
};
}
export function planCallEventsClear(
eventIds: readonly string[],
): UiWriteOutcome {
if (
eventIds.length < 1 ||
eventIds.length > 100 ||
new Set(eventIds).size !== eventIds.length ||
eventIds.some((id) => !UUID.test(id))
) {
return invalid("call_event_ids_invalid");
}
return preview({
schema_version: 1,
kind: "call_events_clear",
event_ids: [...eventIds],
});
}
export function planCallEventsClearAll(): UiWriteOutcome {
return preview({ schema_version: 1, kind: "call_events_clear_all" });
}
export function planImportFileCommit(uploadId: string): UiWriteOutcome {
return UUID.test(uploadId)
? preview({
schema_version: 1,
kind: "import_file_commit",
upload_id: uploadId,
})
: invalid("import_file_id_invalid");
}
export function planImportFileDiscard(uploadId: string): UiWriteOutcome {
return UUID.test(uploadId)
? preview({
schema_version: 1,
kind: "import_file_discard",
upload_id: uploadId,
})
: invalid("import_file_id_invalid");
}
export function planImportFileUpload(input: {
path: string;
bytes: number;
sha256: string;
inputKind: "raw_conversation" | "memory_content";
}): UiWriteOutcome {
const filename = input.path.split(/[/\\]/).pop() ?? "";
if (
input.path.length === 0 ||
!Number.isSafeInteger(input.bytes) ||
input.bytes < 0 ||
input.bytes > MAX_UPLOAD_BYTES ||
!/^[0-9a-f]{64}$/.test(input.sha256) ||
!INPUT_KINDS.includes(input.inputKind) ||
filename.length === 0 ||
[...filename].length > 200 ||
!/\.(?:zip|json|md|txt)$/i.test(filename)
) {
return invalid("import_file_upload_invalid");
}
return preview({
schema_version: 1,
kind: "import_file_upload",
operation_id: fileUploadOperationId(
filename,
input.inputKind,
input.sha256,
),
path: input.path,
filename,
input_kind: input.inputKind,
bytes: input.bytes,
sha256: input.sha256,
});
}
/**
* Read one regular, non-symlinked export without buffering past the 128 MiB
* limit. Returns the bytes plus the checksum the plan records, so apply can
* prove the file it uploads is the one the caller reviewed.
*/
export async function readImportFileUpload(
filePath: string,
inputKind: "raw_conversation" | "memory_content",
): Promise<{ plan: UiWritePlan; bytes: Buffer } | null> {
let handle: FileHandle;
try {
handle = await open(
filePath,
constants.O_RDONLY |
(constants.O_NONBLOCK ?? 0) |
(constants.O_NOFOLLOW ?? 0),
);
} catch {
return null;
}
try {
const stats = await handle.stat();
if (!stats.isFile() || stats.nlink !== 1 || stats.size > MAX_UPLOAD_BYTES) {
return null;
}
const buffer = Buffer.alloc(Number(stats.size));
let offset = 0;
while (offset < buffer.length) {
const chunk = await handle.read(
buffer,
offset,
buffer.length - offset,
offset,
);
if (chunk.bytesRead === 0) break;
offset += chunk.bytesRead;
}
if (offset !== buffer.length) return null;
const checksum = createHash("sha256").update(buffer).digest("hex");
const result = planImportFileUpload({
path: filePath,
bytes: buffer.byteLength,
sha256: checksum,
inputKind,
});
if (result.state !== "preview") return null;
return { plan: result.plan, bytes: buffer };
} catch {
return null;
} finally {
await handle.close().catch(() => undefined);
}
}
/**
* Core keys upload idempotency by `operation_id` and rejects a replay whose
* fingerprint differs. Deriving it from the file's own identity means a retry
* after a dropped response resumes the same staging operation instead of
* creating a second upload; a genuinely different payload maps to a different
* operation and cannot overwrite the first.
*/
function fileUploadOperationId(
filename: string,
inputKind: (typeof INPUT_KINDS)[number],
checksum: string,
): string {
const hex = createHash("sha256")
.update(JSON.stringify([filename, inputKind, checksum]))
.digest("hex");
return [
hex.slice(0, 8),
hex.slice(8, 12),
`4${hex.slice(13, 16)}`,
`${(8 + (parseInt(hex[16]!, 16) % 4)).toString(16)}${hex.slice(17, 20)}`,
hex.slice(20, 32),
].join("-");
}
export function validateUiWritePlan(value: unknown): UiWritePlan | null {
if (value === null || typeof value !== "object" || Array.isArray(value)) {
return null;
}
const candidate = value as Record<string, unknown>;
if (candidate.schema_version !== 1 || typeof candidate.kind !== "string") {
return null;
}
const { digest, ...withoutDigest } = candidate;
if (typeof digest !== "string" || digest !== digestOf(withoutDigest)) {
return null;
}
switch (candidate.kind) {
case "call_events_clear":
return Array.isArray(candidate.event_ids) &&
candidate.event_ids.length >= 1 &&
candidate.event_ids.length <= 100 &&
new Set(candidate.event_ids).size === candidate.event_ids.length &&
candidate.event_ids.every(
(id) => typeof id === "string" && UUID.test(id),
)
? (candidate as unknown as UiWritePlan)
: null;
case "call_events_clear_all":
return Object.keys(candidate).length === 3
? (candidate as unknown as UiWritePlan)
: null;
case "import_file_commit":
case "import_file_discard":
return typeof candidate.upload_id === "string" &&
UUID.test(candidate.upload_id) &&
Object.keys(candidate).length === 4
? (candidate as unknown as UiWritePlan)
: null;
case "import_file_upload":
return typeof candidate.operation_id === "string" &&
UUID.test(candidate.operation_id) &&
typeof candidate.path === "string" &&
typeof candidate.filename === "string" &&
candidate.filename === candidate.path.split(/[/\\]/).pop() &&
/^[0-9a-f]{64}$/.test(String(candidate.sha256)) &&
INPUT_KINDS.includes(
candidate.input_kind as (typeof INPUT_KINDS)[number],
) &&
Number.isSafeInteger(candidate.bytes) &&
(candidate.bytes as number) >= 0 &&
(candidate.bytes as number) <= 128 * 1024 * 1024
? (candidate as unknown as UiWritePlan)
: null;
default:
return null;
}
}
export function uiWriteExitCode(document: UiWriteOutcome): number {
return ["preview", "committed", "recovered"].includes(document.state) ? 0 : 1;
}
export function renderUiWriteText(document: UiWriteOutcome): string {
if (document.state === "preview") {
return `以下操作尚未执行;加 --confirm 后才会写入:\n${document.plan.kind}\nplan_digest:${document.plan.digest}\n`;
}
if (document.state === "committed" || document.state === "recovered") {
return `已提交:${document.operation_id}\n${JSON.stringify(document.receipt)}\n`;
}
const code = "code" in document ? document.code : "ui_write_state_invalid";
return `操作未完成:${document.state}(${code})\n`;
}
export async function executeCallEventsClear(
plan: Extract<UiWritePlan, { kind: "call_events_clear" }>,
operations: CoreOperations,
): Promise<UiWriteOutcome> {
const result = await operations.post<unknown>("/api/v1/call-events/clear", {
event_ids: plan.event_ids,
user_confirmed: true,
});
return outcome(result, plan.event_ids.join(","), ["removed"]);
}
export async function executeCallEventsClearAll(
plan: Extract<UiWritePlan, { kind: "call_events_clear_all" }>,
operations: CoreOperations,
): Promise<UiWriteOutcome> {
const result = await operations.post<unknown>(
"/api/v1/call-events/clear-all",
{ user_confirmed: true },
);
return outcome(result, "all", ["removed"]);
}
export async function executeImportFileCommit(
plan: Extract<UiWritePlan, { kind: "import_file_commit" }>,
operations: CoreOperations,
): Promise<UiWriteOutcome> {
const result = await operations.post<unknown>(
`/api/v1/import-files/${plan.upload_id}/commit`,
{ confirmed: true },
);
return outcome(result, plan.upload_id, [
"upload_id",
"filename",
"vendor",
"source_id",
"messages",
"conversations",
"human_messages",
"agent_messages",
"skipped_records",
"ignored_files",
"checksum",
"created_at",
"committed",
]);
}
export async function executeImportFileDiscard(
plan: Extract<UiWritePlan, { kind: "import_file_discard" }>,
operations: CoreDeleteOperations,
): Promise<UiWriteOutcome> {
const result = await operations.delete<unknown>(
`/api/v1/import-files/${plan.upload_id}`,
);
// Core answers 204 with no body. A successful delete is a committed
// operation even though there is no receipt to validate; requiring a body
// here would report every real discard as `ui_write_receipt_invalid`.
return outcome(result, plan.upload_id, null);
}
/** Upload one local file through the same recoverable Core staging flow the
* Web console uses. The operation id is part of the plan, so a retry after a
* dropped response resumes the same staging record instead of duplicating it.
*/
export async function executeImportFileUpload(
plan: Extract<UiWritePlan, { kind: "import_file_upload" }>,
operations: CoreUploadOperations,
): Promise<UiWriteOutcome> {
// Re-read the file at apply time and re-derive its identity. A plan that
// names a path whose content changed after review is refused rather than
// uploading something the caller never saw.
const reread = await readImportFileUpload(plan.path, plan.input_kind);
if (reread === null) {
return invalid("import_file_upload_unreadable");
}
const bytes = reread.bytes;
const checksum = createHash("sha256").update(bytes).digest("hex");
if (
bytes.byteLength !== plan.bytes ||
checksum !== plan.sha256 ||
reread.plan.digest !== plan.digest
) {
return invalid("import_file_upload_content_mismatch");
}
const result = await operations.uploadImportFile<unknown>({
bytes,
filename: plan.filename,
inputKind: plan.input_kind,
operationId: plan.operation_id,
});
if (result.state === "rejected") {
return {
schema_version: 1,
product: PRODUCT,
state: "rejected",
operation_id: plan.operation_id,
code: result.code,
};
}
if (result.state === "unknown") {
return {
schema_version: 1,
product: PRODUCT,
state: "unknown",
operation_id: plan.operation_id,
code: result.code,
};
}
const receipt =
result.data !== null &&
typeof result.data === "object" &&
!Array.isArray(result.data)
? (result.data as Record<string, unknown>)
: null;
if (
receipt === null ||
receipt.operation_id !== plan.operation_id ||
typeof receipt.state !== "string"
) {
return {
schema_version: 1,
product: PRODUCT,
state: "unknown",
operation_id: plan.operation_id,
code: "ui_write_receipt_invalid",
};
}
// Core reports the staged upload as `completed`; `pending`/`not_found` here
// means the operation record did not settle, so the result is not known.
if (receipt.state !== "completed") {
return {
schema_version: 1,
product: PRODUCT,
state: "unknown",
operation_id: plan.operation_id,
code: `import_upload_${receipt.state}`,
};
}
const uploadId =
typeof receipt.upload_id === "string" ? receipt.upload_id : undefined;
if (uploadId === undefined || !UUID.test(uploadId)) {
return {
schema_version: 1,
product: PRODUCT,
state: "unknown",
operation_id: plan.operation_id,
code: "ui_write_receipt_invalid",
};
}
return committed(plan.operation_id, {
operation_id: plan.operation_id,
upload_id: uploadId,
upload_state: receipt.upload_state ?? null,
});
}
function outcome(
result:
| Awaited<ReturnType<CoreOperations["post"]>>
| Awaited<ReturnType<CoreDeleteOperations["delete"]>>,
operationId: string,
required: readonly string[] | null,
): UiWriteOutcome {
if (result.state === "rejected") {
return {
schema_version: 1,
product: PRODUCT,
state: "rejected",
operation_id: operationId,
code: result.code,
};
}
if (result.state === "unknown") {
return {
schema_version: 1,
product: PRODUCT,
state: "unknown",
operation_id: operationId,
code: result.code,
};
}
if (required === null) {
return committed(operationId, {});
}
const data =
result.data !== null &&
typeof result.data === "object" &&
!Array.isArray(result.data)
? (result.data as Record<string, unknown>)
: null;
if (data === null || required.some((key) => !Object.hasOwn(data, key))) {
return {
schema_version: 1,
product: PRODUCT,
state: "unknown",
operation_id: operationId,
code: "ui_write_receipt_invalid",
};
}
return committed(operationId, data);
}
+1 -15
View File
@@ -13,16 +13,7 @@ const GLOBAL_OPTIONS: readonly CommandOption[] = [
summary: "显示帮助;可与 --json 组合输出机器可读目录。",
},
{ flag: "--version, -V", summary: "显示 CLI 版本。" },
{
flag: "--json",
summary:
"以 JSON 输出到 stdout,供脚本与 Agent 消费;结果与失败都在 stdout,只有参数错误写 stderr。",
},
{
flag: "--data-dir <path>",
summary:
"本次使用的 MemHub 数据目录(绝对路径);默认 ONE_MEMORY_DATA_DIR > 安装回执 > ~/.one-memory。",
},
{ flag: "--json", summary: "以 JSON 输出到 stdout,供脚本与 Agent 消费。" },
];
export function renderGeneralHelp(): string {
@@ -42,9 +33,6 @@ export function renderGeneralHelp(): string {
(exit) => ` ${exit.code} ${pad(exit.name, 17)}${exit.summary}`,
),
"",
"确认约定:只有不可逆操作(memories delete、models remove、delete-data、backup restore、graph-width --decision rebuild_with_new)需要 --yes;不加时只说明将要做什么,不做任何改动,exit 0。delete-data 与 backup restore 还要用 --confirm <path> 再写一遍目标路径。install/upgrade/uninstall/service-install/service-uninstall 不加 --yes 时只出计划。其它写命令直接执行。",
"异步操作返回 ID,用对应命令查进度:sources receipt、models receipt、memories add-status、write status。",
"目前仅支持 macOS 与 Linux 连接本机 Runtime。",
"stdout 只输出数据,进度与诊断写入 stderr。",
`运行 ${PRODUCT} help <command> 查看单个命令的参数。`,
"",
@@ -67,8 +55,6 @@ export function renderCommandHelp(command: CommandDescriptor): string {
);
}
lines.push(
"Global options: --json、--data-dir <path>(见 memhub help)。",
"",
"Exit codes:",
...EXIT_CODES.filter((exit) => command.exit_codes.includes(exit.code)).map(
(exit) => ` ${exit.code} ${pad(exit.name, 17)}${exit.summary}`,
+3 -4
View File
@@ -4,7 +4,7 @@ build-backend = "uv_build"
[project]
name = "one-memory"
version = "1.0.5"
version = "1.0.4"
description = "MemHub local-first memory core"
readme = "README.md"
requires-python = ">=3.12"
@@ -12,9 +12,8 @@ dependencies = [
"fastapi>=0.141.1",
"filelock>=3.32.4",
# Public SDK version validated against MemHub's permission and recall contracts.
"hy-memory==2.0.6",
# chromadb arrives through the HY SDK; locked here so an unrelated upgrade
# cannot move the SDK's store. Built-in embedding consumes numpy directly.
"hy-memory==2.0.5.dev3",
# Retrieval's derived vector index directly consumes these locked versions.
"chromadb==1.5.9",
"numpy==2.5.2",
"httpx>=0.28.1",
File diff suppressed because it is too large. Load diff
@@ -15,7 +15,7 @@ from one_memory.call_event_store import (
CallTool,
HostId,
)
from one_memory.context_models import SdkContextSearchResponse
from one_memory.context_models import ContextSearchResponse, MemoryContextItem, RawContextItem
from one_memory.external_memory_access import ExternalMemoryAccess
from one_memory.recall import RecallReceipt
@@ -41,27 +41,75 @@ def _host_id(access: ExternalMemoryAccess | None) -> HostId | None:
return cast(HostId, host) if host in HOSTS else None
def sdk_context_call_event(
result: SdkContextSearchResponse,
def context_call_event(
result: ContextSearchResponse,
access: ExternalMemoryAccess | None,
*,
memory_searched: bool,
timed_out: bool,
) -> CallEvent:
# SDK items carry no Evidence/Governance revision, and the reference
# resolver reads the legacy store, so the event records counts only.
raw_count = sum(item.kind == "raw" for item in result.items)
memory_count = sum(item.kind == "memory" for item in result.items)
state: CallState = "timed_out" if timed_out else result.state
return _metadata_event(
tool="om_recall" if memory_searched else "om_conversations",
state=state,
elapsed_ms=result.total_ms,
access=access,
raw_count=raw_count,
memory_count=memory_count,
event_id=UUID(result.request_id),
references: list[CallReference] = []
seen: set[tuple[str, str, int]] = set()
for item in result.items:
if isinstance(item, RawContextItem):
candidates = [
CallReference(
kind="raw",
id=message.evidence_id,
revision=message.revision,
source_id=item.source_id,
)
for message in item.messages
]
elif isinstance(item, MemoryContextItem):
candidates = [
CallReference(
kind="memory", id=item.memory_id, revision=item.revision, source_id=None
)
]
else:
continue
for reference in candidates:
identity = (reference.kind, reference.id, reference.revision)
if identity not in seen:
seen.add(identity)
references.append(reference)
timed_out = any(
path.reason == "timeout_budget" for path in (result.paths.raw, result.paths.memory)
)
state: CallState = (
"partial" if timed_out and references else "timed_out" if timed_out else result.state
)
return CallEvent.model_validate(
{
"event_id": UUID(result.request_id),
"occurred_at": datetime.now(UTC),
"host_id": _host_id(access),
"installation_id": UUID(access.installation_id) if access else None,
"tool": _context_tool(result),
"state": state,
"stage": "core_result",
"delivery": "unknown",
"adoption": "unknown",
"host_session_id": None,
"injection_elapsed_ms": None,
"elapsed_ms": result.total_ms,
"raw_count": sum(isinstance(item, RawContextItem) for item in result.items),
"memory_count": sum(isinstance(item, MemoryContextItem) for item in result.items),
"references": tuple(references),
}
)
def record_context_result(
store: CallEventStore,
result: ContextSearchResponse,
access: ExternalMemoryAccess | None,
) -> None:
store.record(context_call_event(result, access))
def _context_tool(result: ContextSearchResponse) -> CallTool:
if result.paths.memory.status not in {"not_requested", "not_authorized"}:
return "om_recall"
return "om_conversations"
def _metadata_event(
@@ -94,6 +142,26 @@ def _metadata_event(
)
def conversation_call_event(
result: object,
access: ExternalMemoryAccess | None,
elapsed_ms: float,
) -> CallEvent:
hits = tuple(getattr(result, "hits", ()) or ())
conversations = {
conversation_id
for hit in hits
if (conversation_id := getattr(hit, "conversation_id", None))
}
return _metadata_event(
tool="om_conversations",
state="grounded" if hits else "empty",
elapsed_ms=elapsed_ms,
access=access,
raw_count=min(len(conversations), 20),
)
def ask_call_event(
result: object,
access: ExternalMemoryAccess | None,
@@ -187,9 +255,8 @@ def record_legacy_result(
host_session_id=None,
injection_elapsed_ms=None,
elapsed_ms=elapsed_ms,
# A raw-fallback recall returned conversation excerpts, not memories.
raw_count=len(receipt.hits) if receipt.mode == "raw" else 0,
memory_count=0 if receipt.mode == "raw" else len(receipt.hits),
raw_count=0,
memory_count=len(receipt.hits),
references=(),
)
)
File diff suppressed because it is too large. Load diff
@@ -23,6 +23,10 @@ class Settings(BaseSettings):
live_sync_enabled: bool = True
automatic_memory_processing: bool = True
imports_enabled: bool = True
# Experimental and unreleased: ownership/error shutdown still requires acceptance.
# Resume only explicitly initialized generations; never migrate at startup.
retrieval_index_enabled: bool = False
retrieval_minimum_free_bytes: int = Field(default=256 * 1024 * 1024, ge=0)
# Bounded metadata only: query, original text and answers are never stored.
call_event_retention_days: int | None = Field(default=30, ge=1, le=365)
call_event_max_events: int | None = Field(default=10_000, ge=1, le=100_000)
@@ -47,19 +47,6 @@ class ContextSearchRequest(BaseModel):
return self
class SdkContextSearchRequest(ContextSearchRequest):
"""`POST /api/v2/context/search`: the three SDK recall lanes are set apart.
`limit` is the normal lane's top-k and the cap on returned items;
`schema_limit` and `intention_limit` are the SDK's schema (L6) and
proactive intention (L7) lanes. A caller's values are passed through.
"""
limit: int = Field(default=10, ge=1, le=20, strict=True)
schema_limit: int = Field(default=3, ge=0, le=20, strict=True)
intention_limit: int = Field(default=1, ge=0, le=20, strict=True)
class ContextMessage(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
+411 -35
View File
@@ -1,6 +1,7 @@
"""Authenticated SDK context search route, without generation or reindex side effects."""
"""Authenticated composite read route, without generation or reindex side effects."""
import asyncio
import sqlite3
import time
from collections.abc import Sequence
from datetime import UTC, datetime
@@ -10,15 +11,36 @@ from uuid import uuid4
from fastapi import APIRouter, HTTPException, Request, Response
from fastapi.params import Depends
from one_memory.call_event_recorder import sdk_context_call_event
from one_memory.background_work import (
in_interactive_read,
in_read_pool,
memory_read_pool,
raw_read_pool,
)
from one_memory.call_event_recorder import context_call_event
from one_memory.context_access import resolve_context_authority
from one_memory.context_memory import (
merge_memory_candidates,
read_memory_context,
revalidate_memory_context,
)
from one_memory.context_models import (
ContextItem,
ContextKind,
ContextLaneResult,
ContextSearchRequest,
ContextSearchResponse,
ContextSemanticStatus,
MemoryContextItem,
RawContextItem,
SdkContextItem,
SdkContextSearchRequest,
SdkContextSearchResponse,
SdkMemoryContextItem,
SdkRawContextItem,
)
from one_memory.context_raw import revalidate_raw_context
from one_memory.context_search import search_context
from one_memory.diagnostic_log import diagnostic_event, diagnostic_exception
from one_memory.external_memory_access import external_access
from one_memory.hy_runtime_config import HY_MEMORY_READER
from one_memory.hy_sdk_runtime import sdk_relevance_floor
@@ -28,8 +50,389 @@ from one_memory.hy_sdk_types import (
parse_sdk_search,
)
from one_memory.imports.storage import EvidenceStore
from one_memory.retrieval.raw_lane import read_raw_indexed_context
from one_memory.semantic_deadline import hy_wait_budget, search_within
CALL_EVENT_RESPONSE_BUDGET_MS = 250
RESPONSE_SERIALIZATION_RESERVE_MS = 100
INTERACTIVE_TOTAL_BUDGET_MS = 900
RAW_LANE_BUDGET_SECONDS = 0.3
MEMORY_LANE_BUDGET_SECONDS = 0.75
MEMORY_FINALIZE_RESERVE_SECONDS = 0.1
def context_router(dependencies: Sequence[Depends]) -> APIRouter:
router = APIRouter(prefix="/api/v1/context", dependencies=dependencies)
@router.post("/search", response_model=ContextSearchResponse)
async def search(
value: ContextSearchRequest, request: Request, response: Response
) -> ContextSearchResponse:
response.headers["Cache-Control"] = "no-store"
started = time.monotonic()
search_budget_ms = min(
INTERACTIVE_TOTAL_BUDGET_MS,
max(50, value.timeout_ms - RESPONSE_SERIALIZATION_RESERVE_MS),
)
deadline = started + search_budget_ms / 1000
access = external_access(request.headers)
try:
database = cast(EvidenceStore, request.app.state.evidence_store).database_path
authority = await asyncio.wait_for(
in_interactive_read(
resolve_context_authority, value, access, database, deadline=deadline
),
timeout=max(0, deadline - time.monotonic()),
)
except HTTPException:
raise
except Exception:
# Authority could not be established; do not try an unbounded local lane.
raise HTTPException(503, "context_authority_unavailable") from None
lane_observations: dict[ContextKind, ContextLaneResult] = {}
lane_failures: dict[ContextKind, tuple[str, str]] = {}
stage_timings: dict[str, float] = {}
async def raw(end: float) -> ContextLaneResult:
lane_end = min(end, time.monotonic() + RAW_LANE_BUDGET_SECONDS)
read = asyncio.create_task(
in_read_pool(
raw_read_pool(),
read_raw_indexed_context,
database,
value,
source_ids=authority.raw_sources,
deadline=lane_end,
index=getattr(request.app.state, "retrieval_runtime", None),
)
)
done, _ = await asyncio.wait({read}, timeout=max(0, lane_end - time.monotonic()))
if read not in done:
read.cancel()
read.add_done_callback(
lambda task: task.exception() if not task.cancelled() else None
)
result = ContextLaneResult(
status="degraded",
reason="timeout_budget",
searched_channels=("lexical", "semantic"),
timed_out=True,
truncated_by=("timeout_budget",),
)
else:
result = read.result()
lane_observations["raw"] = result
return result
async def memory(end: float) -> ContextLaneResult:
lane_end = min(end, time.monotonic() + MEMORY_LANE_BUDGET_SECONDS)
target = getattr(request.app.state, "recall_target", None)
lexical = asyncio.create_task(
in_read_pool(
memory_read_pool(),
read_memory_context,
database,
value,
scopes=authority.memory_scopes,
deadline=lane_end,
semantic_ids=(),
semantic_outcome="absent",
)
)
semantic_hits, outcome = await search_within(
getattr(target, "semantic_search", None),
value.query,
hy_wait_budget(lane_end - time.monotonic()),
)
done, _ = await asyncio.wait({lexical}, timeout=max(0, lane_end - time.monotonic()))
if lexical not in done:
lexical.cancel()
lexical.add_done_callback(
lambda task: task.exception() if not task.cancelled() else None
)
result = ContextLaneResult(
status="unavailable",
reason="timeout_budget",
searched_channels=("lexical", "semantic"),
semantic_status=cast(
ContextSemanticStatus,
{
"absent": "not_searched",
"answered": "completed",
"failed": "failed",
"timed_out": "timed_out",
}[outcome],
),
timed_out=True,
truncated_by=("timeout_budget",),
)
lane_observations["memory"] = result
return result
try:
lexical_result = lexical.result()
except (sqlite3.Error, OSError, ValueError) as error:
diagnostic_event(
level="warn",
component="context.search",
event="context.lexical_memory_failed",
outcome="degraded",
error_code="lexical_memory_read_failed",
**diagnostic_exception(error),
)
lexical_result = ContextLaneResult(
status="unavailable",
reason="read_unavailable",
searched_channels=("lexical",),
failure_site="memory_lexical",
error_class=type(error).__name__,
)
if outcome == "absent":
result = lexical_result
elif outcome in {"failed", "timed_out"}:
truncated_by = list(lexical_result.truncated_by)
has_lexical_memory = bool(lexical_result.items)
if (
outcome == "timed_out"
and not has_lexical_memory
and "timeout_budget" not in truncated_by
):
truncated_by.append("timeout_budget")
result = lexical_result.model_copy(
update={
"status": "degraded",
"reason": "lexical_only" if has_lexical_memory else "timeout_budget",
"truncated_by": tuple(truncated_by),
"searched_channels": ("lexical", "semantic"),
"semantic_status": "timed_out" if outcome == "timed_out" else "failed",
"timed_out": outcome == "timed_out" and not has_lexical_memory,
}
)
else:
semantic_request = value.model_copy(update={"query": ""})
enriched = asyncio.create_task(
in_interactive_read(
read_memory_context,
database,
semantic_request,
scopes=authority.memory_scopes,
deadline=lane_end,
semantic_hits=semantic_hits,
semantic_outcome=outcome,
)
)
done, _ = await asyncio.wait(
{enriched},
timeout=max(
0,
lane_end - MEMORY_FINALIZE_RESERVE_SECONDS - time.monotonic(),
),
)
if enriched not in done:
enriched.cancel()
enriched.add_done_callback(
lambda task: task.exception() if not task.cancelled() else None
)
result = lexical_result.model_copy(
update={
"status": "degraded",
"reason": "lexical_only",
"searched_channels": ("lexical", "semantic"),
"semantic_status": "completed",
"timed_out": False,
}
)
else:
try:
semantic_result = enriched.result()
except (sqlite3.Error, OSError, ValueError) as error:
diagnostic_event(
level="warn",
component="context.search",
event="context.semantic_enrichment_failed",
outcome="degraded",
error_code="semantic_enrichment_read_failed",
**diagnostic_exception(error),
)
result = lexical_result.model_copy(
update={
"status": "degraded",
"reason": "lexical_only",
"searched_channels": ("lexical", "semantic"),
"semantic_status": "completed",
"timed_out": False,
}
)
lane_observations["memory"] = result
return result
if semantic_result.status == "unavailable" and lexical_result.items:
result = lexical_result.model_copy(
update={
"status": "degraded",
"reason": "lexical_only",
"searched_channels": ("lexical", "semantic"),
"semantic_status": "completed",
"timed_out": False,
}
)
lane_observations["memory"] = result
return result
semantic_items = tuple(
item
for item in semantic_result.items
if isinstance(item, MemoryContextItem)
)
lexical_items = tuple(
item for item in lexical_result.items if isinstance(item, MemoryContextItem)
)
combined_count = len(
{item.memory_id for item in (*semantic_items, *lexical_items)}
)
merged = merge_memory_candidates(
lexical_items,
semantic_items,
limit=value.limit + 1,
)
truncated_by = list(semantic_result.truncated_by)
if combined_count > len(merged) and "count_budget" not in truncated_by:
truncated_by.append("count_budget")
result = semantic_result.model_copy(
update={
"items": tuple(merged),
# An answered semantic search may legitimately
# return no IDs. Lexical can still have a grounded
# hit; retaining semantic's `empty` status while
# attaching that hit violates the lane contract
# and makes the aggregator reject the whole lane.
"status": "ok" if merged else semantic_result.status,
"truncated_by": tuple(truncated_by),
}
)
lane_observations["memory"] = result
return result
async def finalizer(
kind: ContextKind, items: tuple[ContextItem, ...], end: float
) -> tuple[ContextItem, ...]:
if kind == "raw":
return await in_interactive_read(
revalidate_raw_context,
database,
tuple(item for item in items if isinstance(item, RawContextItem)),
source_ids=authority.raw_sources,
deadline=end,
)
return await in_interactive_read(
revalidate_memory_context,
database,
tuple(item for item in items if isinstance(item, MemoryContextItem)),
scopes=authority.memory_scopes,
deadline=end,
)
result = await search_context(
value,
raw_reader=raw if authority.raw_allowed else None,
memory_reader=memory if authority.memory_allowed else None,
finalizer=finalizer,
deadline=deadline,
timing_observer=stage_timings.update,
failure_observer=lambda kind, site, error_class: lane_failures.__setitem__(
kind, (site, error_class)
),
)
final = result.model_copy(update={"total_ms": (time.monotonic() - started) * 1000})
serialization_started = time.monotonic()
final.model_dump_json()
stage_timings["serialization_ms"] = (time.monotonic() - serialization_started) * 1000
for phase, duration in stage_timings.items():
diagnostic_event(
level="info",
component="context.search",
event="context_stage.completed",
outcome="succeeded",
request_id=final.request_id,
duration_ms=max(0, round(duration)),
details={"phase": phase},
)
for kind in ("raw", "memory"):
path = getattr(final.paths, kind)
observed = lane_observations.get(kind)
failure = lane_failures.get(kind)
lexical_hits = (
sum(
1
for item in observed.items
if isinstance(item, MemoryContextItem) and "lexical" in item.matched_by
)
if observed is not None and kind == "memory"
else 0
)
semantic_hits = (
sum(
1
for item in observed.items
if isinstance(item, MemoryContextItem)
and "governed_semantic" in item.matched_by
)
if observed is not None and kind == "memory"
else 0
)
timed_out = path.reason == "timeout_budget" or (
observed.timed_out if observed is not None else False
)
diagnostic_failure_site = failure[0] if failure is not None else None
diagnostic_error_class = failure[1] if failure is not None else None
if observed is not None:
diagnostic_failure_site = diagnostic_failure_site or observed.failure_site
diagnostic_error_class = diagnostic_error_class or observed.error_class
diagnostic_event(
level="warn" if path.status in {"degraded", "unavailable"} else "info",
component="context.search",
event="context_lane.completed",
outcome=path.status,
request_id=final.request_id,
duration_ms=round(path.elapsed_ms),
details={
"operation_kind": kind,
"mode": ",".join(observed.searched_channels) if observed else "none",
"status": path.status,
"index_complete": observed.index_complete if observed else None,
**(
{"lexical_hits": lexical_hits, "semantic_hits": semantic_hits}
if kind == "memory"
else {}
),
"reason": path.reason,
"truncated": path.truncated,
"timed_out": timed_out,
**(
{
"failure_site": diagnostic_failure_site,
"error_class": diagnostic_error_class,
}
if diagnostic_failure_site is not None
and diagnostic_error_class is not None
else {}
),
},
)
store = getattr(request.app.state, "call_event_store", None)
if store is not None and value.query != "__om_probe__":
event = context_call_event(final, access)
try:
await asyncio.to_thread(
store.record,
event,
busy_timeout_ms=CALL_EVENT_RESPONSE_BUDGET_MS,
)
response.headers["X-One-Memory-Call-Recorded"] = "true"
except Exception:
response.headers["X-One-Memory-Call-Recorded"] = "false"
request.app.state.call_event_recording_error = True
return final
return router
def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
@@ -38,7 +441,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
@router.post("/search", response_model=SdkContextSearchResponse)
async def search(
value: SdkContextSearchRequest,
value: ContextSearchRequest,
request: Request,
response: Response,
) -> SdkContextSearchResponse:
@@ -113,8 +516,8 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
limit=value.limit,
min_score=sdk_relevance_floor(),
schema_min_score=sdk_relevance_floor(),
schema_limit=value.schema_limit,
intention_limit=value.intention_limit,
schema_limit=value.limit,
intention_limit=value.limit,
scene="normal",
reader=HY_MEMORY_READER,
)
@@ -148,29 +551,6 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
)
return tuple(items)
async def record(final: SdkContextSearchResponse, *, timed_out: bool) -> None:
# Same boundary as v2 ask: only a host's call is history. The
# renderer's own searches are the person reading their library.
store = getattr(request.app.state, "call_event_store", None)
if store is None or access is None or value.query == "__om_probe__":
return
event = sdk_context_call_event(
final,
access,
memory_searched=authority.memory_allowed,
timed_out=timed_out,
)
try:
await asyncio.to_thread(
store.record,
event,
busy_timeout_ms=CALL_EVENT_RESPONSE_BUDGET_MS,
)
response.headers["X-One-Memory-Call-Recorded"] = "true"
except Exception:
response.headers["X-One-Memory-Call-Recorded"] = "false"
request.app.state.call_event_recording_error = True
lanes = (raw_lane(), memory_lane())
try:
raw_items, memory_items = await asyncio.wait_for(
@@ -178,7 +558,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
timeout=value.timeout_ms / 1000,
)
except TimeoutError:
final = SdkContextSearchResponse(
return SdkContextSearchResponse(
request_id=str(uuid4()),
state="unavailable",
items=(),
@@ -186,8 +566,6 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
truncated=True,
total_ms=(time.monotonic() - started) * 1000,
)
await record(final, timed_out=True)
return final
partial = isinstance(raw_items, Exception) or isinstance(memory_items, Exception)
raw_candidates = (
() if isinstance(raw_items, Exception) else cast(tuple[SdkContextItem, ...], raw_items)
@@ -219,7 +597,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
truncated = True
selected.append(item)
used += len(item.content)
final = SdkContextSearchResponse(
return SdkContextSearchResponse(
request_id=str(uuid4()),
state=("partial" if partial else "grounded" if selected else "empty"),
items=tuple(selected),
@@ -227,7 +605,5 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
truncated=truncated,
total_ms=(time.monotonic() - started) * 1000,
)
await record(final, timed_out=False)
return final
return router
@@ -0,0 +1,464 @@
"""Bounded two-lane orchestration; readers and finalizers own trusted read access."""
import asyncio
import time
from collections.abc import Awaitable, Callable, Mapping
from dataclasses import dataclass, field
from uuid import uuid4
from pydantic import TypeAdapter
from one_memory.context_models import (
ContextItem,
ContextKind,
ContextLaneResult,
ContextLimit,
ContextPath,
ContextPaths,
ContextSearchRequest,
ContextSearchResponse,
MemoryContextItem,
RawContextItem,
)
Reader = Callable[[float], Awaitable[ContextLaneResult]]
Finalizer = Callable[
[ContextKind, tuple[ContextItem, ...], float], Awaitable[tuple[ContextItem, ...]]
]
TimingObserver = Callable[[Mapping[str, float]], None]
FailureObserver = Callable[[ContextKind, str, str], None]
_KINDS: tuple[ContextKind, ...] = ("raw", "memory")
_ITEMS = TypeAdapter(tuple[ContextItem, ...])
@dataclass
class _Lane:
result: ContextLaneResult
elapsed_ms: float = 0
limits: list[ContextLimit] = field(default_factory=list)
def _identity(item: ContextItem) -> tuple[str, ...]:
if isinstance(item, RawContextItem):
return (item.kind, item.source_id, item.conversation_id, item.anchor_evidence_id)
return (item.kind, item.memory_id)
def _validate_items(kind: ContextKind, items: tuple[ContextItem, ...]) -> None:
if len(items) > 21 or any(item.kind != kind for item in items):
raise ValueError("invalid lane items")
for item in items:
if isinstance(item, RawContextItem) and item.anchor_evidence_id not in {
message.evidence_id for message in item.messages
}:
raise ValueError("missing raw anchor")
async def _read(reader: Reader, kind: ContextKind, deadline: float) -> ContextLaneResult:
value = await reader(deadline)
# Revalidate even BaseModel instances: model_construct/model_copy bypass validation.
result = ContextLaneResult.model_validate(value.model_dump())
_validate_items(kind, result.items)
if result.status not in ("ok", "empty", "degraded", "unavailable"):
raise ValueError("reader cannot decide requested authority")
if result.status in ("empty", "unavailable") and result.items:
raise ValueError("non-result lane has items")
return result
async def _finalize(
finalizer: Finalizer, kind: ContextKind, items: tuple[ContextItem, ...], deadline: float
) -> tuple[ContextItem, ...]:
value = await finalizer(kind, items, deadline)
if not isinstance(value, tuple):
raise ValueError("invalid finalization result")
checked = _ITEMS.validate_python([item.model_dump() for item in value])
_validate_items(kind, checked)
if not {_identity(item) for item in checked} <= {_identity(item) for item in items}:
raise ValueError("finalizer introduced an unselected identity")
return checked
async def _collect(
jobs: dict[ContextKind, Awaitable[object]], deadline: float
) -> tuple[
dict[ContextKind, object],
dict[ContextKind, BaseException],
set[ContextKind],
dict[ContextKind, float],
]:
"""Cancel and await every owned task on deadline, failure or caller cancellation."""
durations: dict[ContextKind, float] = {}
async def measured(kind: ContextKind, job: Awaitable[object]) -> object:
started = time.monotonic()
try:
value = await job
if time.monotonic() >= deadline:
raise TimeoutError
return value
finally:
durations[kind] = (time.monotonic() - started) * 1000
tasks = {kind: asyncio.create_task(measured(kind, job)) for kind, job in jobs.items()}
values: dict[ContextKind, object] = {}
failed: dict[ContextKind, BaseException] = {}
timed_out: set[ContextKind] = set()
try:
if not tasks:
return values, failed, timed_out, durations
done, pending = await asyncio.wait(
tasks.values(), timeout=max(0, deadline - time.monotonic())
)
for kind, task in tasks.items():
if task in pending:
timed_out.add(kind)
elif task in done:
if not task.cancelled() and isinstance(task.exception(), TimeoutError):
timed_out.add(kind)
elif task.cancelled() or task.exception() is not None:
failed[kind] = task.exception() or asyncio.CancelledError()
else:
values[kind] = task.result()
return values, failed, timed_out, durations
finally:
for task in tasks.values():
if not task.done():
task.cancel()
if tasks:
# Cooperative readers finish immediately and still get their
# normal cleanup. A bridge that suppresses cancellation must not
# hold a host response past its hard deadline while an underlying
# SDK thread winds down.
cleanup: asyncio.Future[list[object]] = asyncio.gather(
*tasks.values(), return_exceptions=True
)
current = asyncio.current_task()
if current is not None and current.cancelling():
await cleanup
else:
try:
await asyncio.wait_for(asyncio.shield(cleanup), timeout=0.05)
except TimeoutError:
cleanup.add_done_callback(_consume_cleanup)
def _consume_cleanup(task: asyncio.Future[list[object]]) -> None:
if not task.cancelled():
task.exception()
def _deduplicate(
items: tuple[ContextItem, ...], limits: list[ContextLimit]
) -> tuple[ContextItem, ...]:
unique: dict[tuple[str, ...], ContextItem] = {}
for item in items:
key = _identity(item)
previous = unique.get(key)
if isinstance(previous, MemoryContextItem) and isinstance(item, MemoryContextItem):
if item.revision != previous.revision:
# Never graft older evidence onto a newer memory version.
unique[key] = item if item.revision > previous.revision else previous
continue
evidence = tuple(dict.fromkeys((*previous.evidence_ids, *item.evidence_ids)))
if len(evidence) > 128:
limits.append("metadata_budget")
unique[key] = previous.model_copy(
update={
"evidence_ids": evidence[:128],
"truncated": previous.truncated or item.truncated or len(evidence) > 128,
}
)
elif isinstance(previous, RawContextItem) and isinstance(item, RawContextItem):
messages = {message.evidence_id: message for message in previous.messages}
for message in item.messages:
if (
message.evidence_id not in messages
or message.revision > messages[message.evidence_id].revision
):
messages[message.evidence_id] = message
if len(messages) > 3:
limits.append("metadata_budget")
# The first window already contains the anchor; dict order preserves it.
unique[key] = previous.model_copy(
update={
"messages": tuple(messages.values())[:3],
"truncated": previous.truncated or item.truncated or len(messages) > 3,
}
)
else:
unique[key] = item
# Reserve each raw anchor for its own window before deduplicating context messages.
anchors = {
(item.source_id, item.conversation_id, item.anchor_evidence_id)
for item in unique.values()
if isinstance(item, RawContextItem)
}
seen: set[tuple[str, ...]] = set()
result: list[ContextItem] = []
for item in unique.values():
if isinstance(item, MemoryContextItem):
result.append(
item.model_copy(update={"evidence_ids": tuple(dict.fromkeys(item.evidence_ids))})
)
continue
retained_messages = []
for message in item.messages:
key = (item.source_id, item.conversation_id, message.evidence_id)
if key in seen or (key in anchors and message.evidence_id != item.anchor_evidence_id):
continue
seen.add(key)
retained_messages.append(message)
result.append(item.model_copy(update={"messages": tuple(retained_messages)}))
return tuple(result)
def _characters(item: ContextItem) -> int:
if isinstance(item, MemoryContextItem):
return len(item.content)
return sum(len(message.text) for message in item.messages)
def _shares(lengths: list[int], budget: int) -> list[int]:
"""Water-fill allocations so a large first item cannot consume every character."""
allocations = [0] * len(lengths)
pending = list(range(len(lengths)))
while pending and budget:
share, remainder = divmod(budget, len(pending))
next_pending = []
for position, index in enumerate(pending):
added = min(lengths[index] - allocations[index], share + (position < remainder))
allocations[index] += added
budget -= added
if allocations[index] < lengths[index]:
next_pending.append(index)
pending = next_pending
return allocations
def _clip(item: ContextItem, budget: int) -> ContextItem:
if _characters(item) <= budget:
return item
if isinstance(item, MemoryContextItem):
return item.model_copy(update={"content": item.content[:budget], "truncated": True})
# The first raw message is the lexical anchor. Give it the first share of
# the budget, then water-fill the adjacent context; otherwise a long
# neighbor can consume the whole window and hide the answer.
anchor = item.messages[0]
anchor_budget = min(len(anchor.text), budget)
remaining = budget - anchor_budget
adjacent = list(item.messages[1:])
shares = [anchor_budget, *_shares([len(message.text) for message in adjacent], remaining)]
return item.model_copy(
update={
"truncated": True,
"messages": tuple(
message.model_copy(
update={
"text": message.text[:share],
"truncated": message.truncated or len(message.text) > share,
}
)
for message, share in zip(item.messages, shares, strict=True)
),
}
)
async def search_context(
request: ContextSearchRequest,
*,
raw_reader: Reader | None,
memory_reader: Reader | None,
finalizer: Finalizer,
deadline: float | None = None,
timing_observer: TimingObserver | None = None,
failure_observer: FailureObserver | None = None,
) -> ContextSearchResponse:
request = ContextSearchRequest.model_validate(request.model_dump())
started = time.monotonic()
deadline = min(
deadline if deadline is not None else float("inf"), started + request.timeout_ms / 1000
)
lanes: dict[ContextKind, _Lane] = {}
jobs: dict[ContextKind, Awaitable[object]] = {}
readers: dict[ContextKind, Reader | None] = {"raw": raw_reader, "memory": memory_reader}
for kind, reader in readers.items():
if kind not in request.kinds:
lanes[kind] = _Lane(ContextLaneResult(status="not_requested", reason="not_requested"))
elif reader is None:
lanes[kind] = _Lane(
ContextLaneResult(status="not_authorized", reason="permission_required")
)
elif time.monotonic() >= deadline:
lanes[kind] = _Lane(
ContextLaneResult(status="degraded", reason="timeout_budget", timed_out=True),
limits=["timeout_budget"],
)
else:
jobs[kind] = _read(reader, kind, deadline)
values, failed, timed_out, read_ms = await _collect(jobs, deadline)
for kind in jobs:
result = values.get(kind)
if kind in timed_out:
result = ContextLaneResult(
status="degraded",
reason="timeout_budget",
truncated_by=("timeout_budget",),
timed_out=True,
)
elif kind in failed or not isinstance(result, ContextLaneResult):
error = failed.get(kind)
if failure_observer is not None:
failure_observer(
kind,
"collect",
type(error).__name__ if error is not None else "InvalidLaneResult",
)
result = ContextLaneResult(
status="unavailable",
reason="read_unavailable",
failure_site="collect",
error_class=type(error).__name__ if error is not None else "InvalidLaneResult",
)
lanes[kind] = _Lane(result, read_ms.get(kind, 0), list(dict.fromkeys(result.truncated_by)))
final_jobs: dict[ContextKind, Awaitable[object]] = {}
for kind in jobs:
lane = lanes[kind]
if lane.result.status not in ("ok", "empty", "degraded"):
continue
if time.monotonic() >= deadline:
lane.result = ContextLaneResult(
status="degraded", reason="timeout_budget", timed_out=True
)
lane.limits.append("timeout_budget")
else:
final_jobs[kind] = _finalize(finalizer, kind, lane.result.items, deadline)
final_values, final_failed, final_timed_out, revalidation_ms = await _collect(
final_jobs, deadline
)
for kind in final_jobs:
lane = lanes[kind]
lane.elapsed_ms += revalidation_ms.get(kind, 0)
if kind in final_timed_out:
lane.result = ContextLaneResult(
status="unavailable", reason="timeout_budget", timed_out=True
)
lane.limits.append("timeout_budget")
elif kind in final_failed:
error = final_failed[kind]
if failure_observer is not None:
failure_observer(kind, "finalize", type(error).__name__)
lane.result = ContextLaneResult(
status="unavailable",
reason="read_unavailable",
failure_site="finalize",
error_class=type(error).__name__,
)
else:
final_items = _ITEMS.validate_python(final_values[kind])
changed = {_identity(item): item for item in lane.result.items} != {
_identity(item): item for item in final_items
}
lane.result = lane.result.model_copy(
update={
"items": final_items,
"status": "degraded" if changed else lane.result.status,
"reason": ("evidence_changed" if kind == "raw" else "memory_changed")
if changed
else lane.result.reason,
}
)
# Nothing previously verified may escape once the common release deadline expired.
if time.monotonic() >= deadline:
for kind in jobs:
if lanes[kind].result.status in ("ok", "empty", "degraded"):
lanes[kind].result = ContextLaneResult(
status="degraded", reason="timeout_budget", timed_out=True
)
lanes[kind].limits.append("timeout_budget")
fusion_started = time.monotonic()
queues = {kind: _deduplicate(lanes[kind].result.items, lanes[kind].limits) for kind in _KINDS}
interleaved = [
queues[kind][index]
for index in range(max(map(len, queues.values()), default=0))
for kind in _KINDS
if index < len(queues[kind])
]
selected = interleaved[: request.limit]
for item in interleaved[request.limit :]:
lanes[item.kind].limits.append("count_budget")
shares = _shares([_characters(item) for item in selected], request.character_budget)
items = tuple(_clip(item, share) for item, share in zip(selected, shares, strict=True))
for before, after in zip(selected, items, strict=True):
if _characters(before) != _characters(after):
lanes[after.kind].limits.append("character_budget")
if time.monotonic() >= deadline:
items = ()
for kind in jobs:
if lanes[kind].result.status in ("ok", "empty", "degraded"):
lanes[kind].result = ContextLaneResult(
status="degraded", reason="timeout_budget", timed_out=True
)
lanes[kind].limits.append("timeout_budget")
lanes[kind].elapsed_ms = (time.monotonic() - started) * 1000
paths: dict[ContextKind, ContextPath] = {}
for kind in _KINDS:
lane = lanes[kind]
emitted = [item for item in items if item.kind == kind]
limits = tuple(dict.fromkeys(lane.limits))
truncated = bool(limits) or any(item.truncated for item in emitted)
status = lane.result.status
benign_limits = set(limits) <= {"count_budget", "character_budget"}
benign_degradation = (
not lane.result.timed_out
and benign_limits
and (
lane.result.reason is None
or (kind == "raw" and lane.result.reason == "lexical_only")
)
)
if status in ("ok", "empty") or (status == "degraded" and benign_degradation):
status = "ok" if emitted or lane.result.items else "empty"
paths[kind] = ContextPath(
status=status,
reason=lane.result.reason,
elapsed_ms=lane.elapsed_ms,
returned_count=len(emitted),
truncated=truncated,
truncated_by=limits,
)
requested_paths = [paths[kind] for kind in request.kinds]
completed = any(path.status in ("ok", "empty", "degraded") for path in requested_paths)
incomplete = any(path.status not in ("ok", "empty") for path in requested_paths)
response = ContextSearchResponse(
request_id=str(uuid4()),
state="unavailable"
if not completed
else "partial"
if incomplete
else "grounded"
if items
else "empty",
paths=ContextPaths(raw=paths["raw"], memory=paths["memory"]),
items=items,
used_characters=sum(map(_characters, items)),
truncated=any(path.truncated for path in paths.values()),
total_ms=(time.monotonic() - started) * 1000,
)
checked = ContextSearchResponse.model_validate(response.model_dump())
if timing_observer is not None:
timing_observer(
{
"raw_ms": read_ms.get("raw", 0),
"memory_ms": read_ms.get("memory", 0),
"raw_revalidation_ms": revalidation_ms.get("raw", 0),
"memory_revalidation_ms": revalidation_ms.get("memory", 0),
"fusion_ms": (time.monotonic() - fusion_started) * 1000,
"total_ms": checked.total_ms,
}
)
return checked
@@ -22,7 +22,7 @@ from dataclasses import dataclass
from datetime import UTC, datetime
from functools import partial
from pathlib import Path
from typing import Any, Literal, cast
from typing import Any, Literal, Protocol, cast
from one_memory.background_work import interactive_read_pool
from one_memory.context_expand import ExpansionAuthority, ExpansionRefSigner
@@ -36,8 +36,23 @@ from one_memory.retrieval.models import (
RawEvidence,
RetrievalPath,
RetrievalResponse,
TimeRange,
)
from one_memory.retrieval.query_plan import plan_local_query
from one_memory.retrieval.raw_candidates import RawCandidateBatch
from one_memory.retrieval.raw_evidence import materialize_raw_candidates
class RawIndexLookup(Protocol):
def lookup_raw(
self,
query: str,
*,
source_ids: frozenset[str],
deadline: float,
time_range: TimeRange | None = None,
) -> tuple[RawCandidateBatch, bool]: ...
ExpandRefs = tuple[ExpansionRefSigner, ExpansionAuthority]
@@ -91,6 +106,11 @@ class LaneResult:
coverage_limited: bool = False
def _semantic_only_raw(item: RawEvidence) -> bool:
"""Semantic overlap alone is a candidate, not a confirmed recall item."""
return set(item.matched_by) == {"semantic"}
@dataclass(frozen=True)
class StageTimings:
raw_ms: float = 0.0
@@ -171,6 +191,12 @@ def _managed(
return selected, truncated
def _group_missing_requirements(item: RawEvidence | GovernedEvidence) -> list[str]:
if isinstance(item, RawEvidence) and _semantic_only_raw(item):
return ["relevance_confirmation"]
return []
def _path(
status: str,
reason: str | None,
@@ -270,7 +296,8 @@ def read_lexical_raw_lane(
("lexical", "object") if plan.object_keys else ("lexical",),
truncated,
# Starvation means: this lane matched, then dropped the item whole. An
# ordinary no-match must stay empty.
# ordinary no-match must stay empty, and an unconfigured semantic lane
# is not starvation.
starved and not items,
probe_saturated,
)
@@ -537,6 +564,97 @@ def warm_raw_read_path(
return False
def read_semantic_raw_lane(
index: RawIndexLookup | None,
database_path: Path,
query: CoordinatorQuery,
*,
source_ids: frozenset[str],
deadline: float,
) -> LaneResult:
if not source_ids:
return LaneResult((), "unavailable", "permission_required", (), False)
# No derived index is the no-model baseline, not a failure: the lexical lane
# must still answer. Semantic is genuinely unsearched in that state, so it
# must not claim a channel or a coverage result it never produced.
if index is None:
return LaneResult((), "not_searched", None, (), False)
try:
batch, unpublished = index.lookup_raw(
query.query,
source_ids=source_ids,
deadline=deadline,
time_range=plan_local_query(query.query).time_range,
)
except Exception:
return LaneResult((), "unavailable", "index_unavailable", ("semantic",), False)
if not isinstance(batch, RawCandidateBatch):
return LaneResult((), "unavailable", "index_unavailable", ("semantic",), False)
if batch.reason == "not_authorized":
return LaneResult((), "unavailable", "permission_required", ("semantic",), False)
if batch.reason == "deadline":
return LaneResult((), "unavailable", "timeout_budget", ("semantic",), True)
if batch.reason in {"vector_unavailable", "index_unavailable"}:
return LaneResult((), "unavailable", batch.reason, ("semantic",), False)
store, generation = _index_state(index, database_path)
from one_memory.retrieval.index_store import RetrievalIndexStore
store = cast("RetrievalIndexStore | None", store)
if store is None or generation is None:
return LaneResult((), "unavailable", "index_unavailable", ("semantic",), False)
try:
materialized = materialize_raw_candidates(
store,
batch,
generation_id=generation,
source_ids=source_ids,
deadline=deadline,
character_budget=query.character_budget,
)
except Exception:
return LaneResult((), "unavailable", "index_unavailable", ("semantic",), False)
if materialized.reason == "deadline":
return LaneResult((), "unavailable", "timeout_budget", ("semantic",), True)
if materialized.reason in {"index_unavailable", "not_authorized"}:
return LaneResult((), "unavailable", materialized.reason, ("semantic",), False)
complete = materialized.complete and not unpublished
if materialized.reason == "evidence_changed":
status: Literal["ok", "empty", "unavailable", "not_searched"] = "unavailable"
reason = "evidence_changed"
elif not materialized.items and not complete:
status, reason = "unavailable", "index_incomplete"
elif materialized.reason == "character_budget":
status, reason = "ok", None
else:
status, reason = ("ok" if materialized.items else "empty"), None
return LaneResult(
tuple(materialized.items),
status,
reason,
("semantic",),
materialized.reason == "character_budget",
)
def _index_state(index: RawIndexLookup, database_path: Path) -> tuple[object | None, str | None]:
store = getattr(index, "_store", None)
generation = None
snapshot = getattr(index, "snapshot", None)
if snapshot is not None:
try:
generation = snapshot().generation_id
except Exception:
generation = None
if store is None:
try:
from one_memory.retrieval.index_store import RetrievalIndexStore
store = RetrievalIndexStore(database_path)
except Exception:
store = None
return store, generation
def read_memory_lane(
database_path: Path,
query: CoordinatorQuery,
@@ -739,6 +857,7 @@ def coordinate_context_v2(
query: CoordinatorQuery,
*,
database_path: Path,
index: RawIndexLookup | None = None,
deadline: float,
expand_refs: ExpandRefs | None = None,
) -> RetrievalResponse:
@@ -752,29 +871,62 @@ def coordinate_context_v2(
lexical = read_lexical_raw_lane(
database_path, query, source_ids=authority.raw_sources, deadline=deadline
)
semantic = read_semantic_raw_lane(
index, database_path, query, source_ids=authority.raw_sources, deadline=deadline
)
timings = StageTimings(raw_ms=(time.monotonic() - raw_started) * 1000)
combined: dict[tuple[str, str], RawEvidence] = {}
for item in lexical.items:
if isinstance(item, RawEvidence):
combined.setdefault(_identity(item), item)
for item in (*lexical.items, *semantic.items):
if not isinstance(item, RawEvidence):
continue
identity = _identity(item)
previous = combined.get(identity)
if previous is None or _semantic_only_raw(previous):
combined[identity] = item
# Semantic similarity is a candidate signal, not a verified recall
# result. Keep semantic-only candidates visible in coverage, but do not
# release their bodies for host injection unless the item is
# independently confirmed by the lexical/object lane. A global distance
# threshold is deliberately not used: the measured positive and
# negative distributions overlap.
semantic_only = {_identity(item) for item in combined.values() if _semantic_only_raw(item)}
confirmed = {_identity(item) for item in lexical.items if isinstance(item, RawEvidence)}
deferred_raw = semantic_only - confirmed
if deferred_raw:
combined = {
identity: item
for identity, item in combined.items()
if identity not in deferred_raw
}
channels = tuple(dict.fromkeys((*lexical.channels, *semantic.channels)))
if lexical.status == "unavailable":
status: Literal["ok", "empty", "unavailable", "not_searched", "degraded"] = (
"unavailable"
)
reason = lexical.reason
elif combined or lexical.status == "empty":
status, reason = "ok", None
elif semantic.status == "unavailable":
status = "unavailable"
reason = semantic.reason
elif combined:
status, reason = "ok", semantic.reason if semantic.status != "ok" else None
elif deferred_raw:
status, reason = "degraded", "relevance_confirmation"
elif lexical.status == "empty":
status, reason = ("empty" if semantic.status != "not_searched" else "ok"), None
else:
status, reason = "empty", None
truncated = lexical.truncated or semantic.truncated
raw = LaneResult(
tuple(combined.values()),
status,
reason,
lexical.channels,
lexical.truncated,
channels,
truncated,
lexical.starved,
lexical.coverage_limited,
)
if deferred_raw:
object.__setattr__(raw, "coverage_limited", True)
elif "raw" in query.kinds and not authority.raw_allowed:
raw = LaneResult((), "unavailable", "permission_required", (), False)
if "memory" in query.kinds:
@@ -843,7 +995,9 @@ def coordinate_context_v2(
return _path("unavailable", raw.reason, raw.channels, "not_searched", False, 0)
if dropped and not final_raw:
return _path("unavailable", "permission_required", raw.channels, "unavailable", True, 0)
# The raw lane is complete when the lexical lane actually answered.
# Lexical-only is complete when the lexical lane actually answered. An
# unconfigured semantic lane is a capability absence, not proof that
# less than the authorized range was searched.
complete = (
raw.status in {"ok", "empty"}
and not dropped
@@ -868,7 +1022,9 @@ def coordinate_context_v2(
"count_budget"
if truncated
else "coverage_budget"
if raw.coverage_limited
if raw.coverage_limited and not deferred_raw
else "relevance_confirmation"
if deferred_raw
else "character_budget"
)
return _path(
@@ -956,7 +1112,7 @@ def coordinate_context_v2(
"group_id": f"group-{index}",
"answer_target": query.intent,
"decision_status": "missing_confirmation",
"missing_requirements": [],
"missing_requirements": _group_missing_requirements(item),
"object_refs": _object_refs(item, plan.object_keys),
"items": [group_item.model_dump(mode="json") for group_item in (item,)],
}
@@ -1029,6 +1185,7 @@ async def search_context_v2(
query: CoordinatorQuery,
*,
database_path: Path,
index: RawIndexLookup | None = None,
deadline: float,
expand_refs: ExpandRefs | None = None,
) -> RetrievalResponse:
@@ -1041,6 +1198,7 @@ async def search_context_v2(
authority,
query,
database_path=database_path,
index=index,
deadline=deadline,
expand_refs=expand_refs,
),
@@ -4,10 +4,9 @@ from __future__ import annotations
import os
import re
from collections.abc import Sequence
from dataclasses import dataclass
from one_memory.imports.models import EvidenceContentKind, EvidenceRecord
from one_memory.imports.models import EvidenceContentKind, EvidenceOrigin, EvidenceRecord
from one_memory.memory_tasks import segment_evidence_record
MEMORY_TASK_TIMEOUT_MARGIN_SECONDS = 60.0
MEMORY_TASK_TIMEOUT_MIN_SECONDS = 180.0
@@ -30,13 +29,10 @@ _HOST_ENVELOPE = re.compile(
r")",
re.IGNORECASE | re.DOTALL,
)
#: The write rule (`turn_batches`): every 5 complete turns, or fewer when
#: the next turn would take the batch past 8000 HY-read characters.
TURN_BATCH_TURNS = 5
TURN_BATCH_CHARACTERS = 8_000
#: A live conversation's last turn counts as complete, and a short batch is
#: written, after this long without a new message in that conversation.
TURN_IDLE_FLUSH_SECONDS = 600
_WHOLE_TURN_CHARACTERS = 60_000
_WINDOW_BYTE_CEILING = 2_000_000
_HISTORY_MERGE_TURNS = 8
_HISTORY_MERGE_CHARACTERS = 8_000
def memory_task_timeout_seconds(model_timeout_seconds: float) -> float:
@@ -61,83 +57,180 @@ def _coding_tool_result_max_bytes() -> int:
_HY_TOOL_CHARACTERS = _coding_tool_result_max_bytes()
@dataclass(frozen=True)
class TurnBatches:
"""What a conversation's records become: batches to write, records to keep."""
#: Each batch is whole turns; every batch holds at least one user message.
batches: tuple[tuple[EvidenceRecord, ...], ...]
#: Not written yet: the open last turn, complete turns short of a batch,
#: or records with no user message before them. Fed back in, in order,
#: with the conversation's next records.
carry: tuple[EvidenceRecord, ...]
def conversation_turns(records: Sequence[EvidenceRecord]) -> list[tuple[EvidenceRecord, ...]]:
"""One user message and every record after it up to the next user message.
Records before the first user message are not a turn of their own: they
join the first turn (an assistant proposal and the user turn adopting it
reach HY together). With no user message at all the result is one group
that does not open with a user, which `turn_batches` never writes.
"""
def _conversation_chunks(
records: list[EvidenceRecord],
*,
max_characters: int | None = None,
max_turns: int | None = None,
completed_only: bool = False,
) -> list[tuple[EvidenceRecord, ...]]:
"""Build bounded, complete user/assistant/tool windows."""
turns: list[list[EvidenceRecord]] = []
for record in records:
if not _is_hy_turn_record(record):
continue
# Until the first user message, the group is leading records only.
leading = bool(turns) and not any(_opens_turn(item) for item in turns[-1])
if not turns or (_opens_turn(record) and not leading):
pending_proposal = bool(turns) and not _opens_turn(turns[-1][0])
if not turns or (_opens_turn(record) and not pending_proposal):
turns.append([record])
else:
turns[-1].append(record)
return [tuple(turn) for turn in turns]
if completed_only and turns and _trailing_user_turn_is_open(turns[-1]):
turns.pop()
whole = _whole_turns() and max_characters is None
if whole:
size = _hy_read_length
max_characters = _WHOLE_TURN_CHARACTERS
max_turns = max_turns or 1_000_000
else:
size = lambda record: len(record.verbatim) # noqa: E731
max_characters = max_characters or 12_000
max_turns = max_turns or 24
chunks: list[tuple[EvidenceRecord, ...]] = []
for turn in turns:
original_question = turn[0] if _opens_turn(turn[0]) else None
expanded: list[EvidenceRecord] = []
for record in turn:
reserve = (
size(original_question)
if original_question is not None
and record is not original_question
and size(original_question) < max_characters
else 0
)
segment_limit = max(1, max_characters - reserve)
if size(record) <= segment_limit:
expanded.append(record)
continue
spans = _semantic_segment_spans(record.verbatim, segment_limit)
expanded.extend(
segment_evidence_record(record, index=index, start=start, end=end)
if len(spans) > 1
else record
for index, (start, end) in enumerate(spans)
)
question = expanded[0] if expanded and _opens_turn(expanded[0]) else None
current: list[EvidenceRecord] = []
for record in expanded:
if current and (
len(current) >= max_turns
or sum(size(item) for item in current) + size(record) > max_characters
or sum(_window_bytes(item) for item in current) + _window_bytes(record)
> _WINDOW_BYTE_CEILING
):
chunks.append(tuple(current))
current = (
[question]
if question is not None
and max_turns > 1
and size(question) + size(record) <= max_characters
else []
)
current.append(record)
chunks.append(tuple(current))
return _merge_tool_only_chunks(chunks)
def turn_batches(records: Sequence[EvidenceRecord], *, flush: bool) -> TurnBatches:
"""Group a conversation's records into write batches by the turn rule.
def _semantic_segment_spans(text: str, limit: int) -> list[tuple[int, int]]:
if len(text) <= limit:
return [(0, len(text))]
spans: list[tuple[int, int]] = []
start = 0
separators = ("\n\n", "\n", "。", "!", "?", ". ", "! ", "? ")
while start < len(text):
hard_end = min(len(text), start + limit)
if hard_end == len(text):
end = hard_end
else:
floor = start + max(1, limit // 2)
candidates = [text.rfind(separator, floor, hard_end) for separator in separators]
boundary = max(candidates)
if boundary < floor:
end = hard_end
else:
matched = next(
separator for separator in separators if text.startswith(separator, boundary)
)
end = boundary + len(matched)
spans.append((start, end))
start = end
return spans
- A turn is complete once the next user message follows it; `flush`
completes the last one too (history at end of file, or 10 idle minutes).
- Every 5 complete turns are one batch. Before 5, a turn that would take
the batch past 8000 characters starts the next batch; one turn over
8000 is a batch by itself and is never split.
- Characters are what HY reads: user/assistant text in full, a tool
record up to `_HY_TOOL_CHARACTERS`.
- Fewer than 5 complete turns within 8000 are written only on `flush`.
- No batch is written without a user message; records with none before
them are carried, and dropped on `flush`.
"""
turns = conversation_turns(records)
if turns and not any(_opens_turn(record) for record in turns[0]):
# Records with no user message yet: nothing to write until one comes.
return TurnBatches(batches=(), carry=() if flush else turns[0])
complete = turns if flush else turns[:-1]
open_tail = () if flush or not turns else turns[-1]
batches: list[tuple[EvidenceRecord, ...]] = []
current: list[tuple[EvidenceRecord, ...]] = []
def _merge_tool_only_chunks(
chunks: list[tuple[EvidenceRecord, ...]],
) -> list[tuple[EvidenceRecord, ...]]:
if not any(chunk and all(record.role == "tool" for record in chunk) for chunk in chunks):
return chunks
merged: list[tuple[EvidenceRecord, ...]] = []
leading_tools: list[EvidenceRecord] = []
for chunk in chunks:
if not chunk:
continue
if all(record.role == "tool" for record in chunk):
if merged:
merged[-1] = (*merged[-1], *chunk)
else:
leading_tools.extend(chunk)
continue
if leading_tools:
chunk = (*leading_tools, *chunk)
leading_tools = []
merged.append(chunk)
return [] if leading_tools else merged
def _merge_historical_chunks(
chunks: list[tuple[EvidenceRecord, ...]],
*,
max_turns: int = _HISTORY_MERGE_TURNS,
max_characters: int = _HISTORY_MERGE_CHARACTERS,
) -> list[tuple[EvidenceRecord, ...]]:
if not chunks:
return []
merged: list[tuple[EvidenceRecord, ...]] = []
current: list[EvidenceRecord] = []
current_turns = 0
current_characters = 0
for turn in complete:
characters = sum(_hy_read_length(record) for record in turn)
if current and current_characters + characters > TURN_BATCH_CHARACTERS:
batches.append(tuple(record for item in current for record in item))
current, current_characters = [], 0
current.append(turn)
current_characters += characters
if len(current) >= TURN_BATCH_TURNS or current_characters > TURN_BATCH_CHARACTERS:
batches.append(tuple(record for item in current for record in item))
current, current_characters = [], 0
if current and flush:
batches.append(tuple(record for item in current for record in item))
current = []
carry = tuple(record for item in current for record in item) + tuple(open_tail)
return TurnBatches(batches=tuple(batches), carry=carry)
for chunk in chunks:
chunk_characters = sum(len(record.verbatim) for record in chunk)
if current and (
current_turns >= max_turns or current_characters + chunk_characters > max_characters
):
merged.append(tuple(current))
current = []
current_turns = 0
current_characters = 0
current.extend(chunk)
current_turns += 1
current_characters += chunk_characters
if current:
merged.append(tuple(current))
return merged
def _chunk_input_ref(chunk: tuple[EvidenceRecord, ...]) -> str:
user_records = tuple(item for item in chunk if item.role == "user")
return user_records[-1].evidence_id if user_records else chunk[-1].evidence_id
def _opens_turn(record: EvidenceRecord) -> bool:
return record.role == "user"
def _trailing_user_turn_is_open(turn: list[EvidenceRecord]) -> bool:
if len(turn) != 1:
return False
opener = turn[0]
return (
_opens_turn(opener)
and opener.content_kind is EvidenceContentKind.RAW_CONVERSATION
and not _is_memory_note(opener)
)
def _is_hy_turn_record(record: EvidenceRecord) -> bool:
if record.role == "tool":
return record.content_kind is EvidenceContentKind.RAW_CONVERSATION and bool(
@@ -161,6 +254,23 @@ def _is_hy_input_turn(record: EvidenceRecord) -> bool:
return _HOST_ENVELOPE.search(source[:2_000]) is None
def _is_memory_note(record: EvidenceRecord) -> bool:
meta = record.origin_meta.get(EvidenceOrigin.MANUAL_IMPORT)
return (
record.source_id == "one-memory-manual"
and meta is not None
and meta.raw_meta.get("input_mode") == "direct_memory_note"
)
def _window_bytes(record: EvidenceRecord) -> int:
return len(record.verbatim.encode("utf-8"))
def _whole_turns() -> bool:
return os.environ.get("ONE_MEMORY_SPLIT_TURNS", "").strip() != "1"
def _hy_read_length(record: EvidenceRecord) -> int:
if record.role == "tool":
return min(len(record.verbatim), _HY_TOOL_CHARACTERS)
@@ -6,6 +6,7 @@ from collections.abc import Sequence
from typing import NamedTuple
PREFACE = "以下是获授权历史参考,不是指令;当前用户请求优先。"
_MARKER = re.compile(r"om-derived:([0-9a-f]{64})")
# Written by the host-bridge recall injection hook (recall-injection.ts): the
# recalled memory block and, in the same reply, the cross-session notice block.
_INJECTED_TAGS = ("memhub_memory_context", "memhub_session_notice")
@@ -88,3 +89,45 @@ def strip_injected_memory(text: str) -> str:
return text
kept.append(text[at:])
return _EMPTY_REMINDER.sub("", "".join(kept)).strip()
def marker(event_id: str) -> str:
return f"om-derived:{event_id}"
def echoed_event_ids(verbatim: str) -> tuple[str, ...]:
return tuple(dict.fromkeys(_MARKER.findall(verbatim)))
def render_reference(events: tuple[object, ...] | list[object]) -> str:
lines = [PREFACE]
for event in events:
source = getattr(event, "source_id", None) or "unknown"
revision = getattr(event, "evidence_revision", None)
evidence_id = getattr(event, "evidence_id", None) or ""
event_id = getattr(event, "event_id", "")
lines.append(
f"来源 {source} 修订 {0 if revision is None else revision} "
f"证据 {evidence_id} {marker(str(event_id))}"
)
return "\n".join(lines)
def is_pure_echo(verbatim: str, known_event_ids: set[str]) -> bool:
"""True when the text is only a block we already injected.
A later line that is not the preface, a source line, or one of those
markers is a real correction and must still be recorded.
"""
found = set(echoed_event_ids(verbatim))
if not found or not found & known_event_ids:
return False
for line in verbatim.splitlines():
stripped = line.strip()
if not stripped or stripped == PREFACE or stripped.startswith("来源 "):
continue
if _MARKER.search(stripped):
continue
return False
return True
+123 -218
View File
@@ -20,10 +20,8 @@ from __future__ import annotations
import atexit
import json
import logging
import math
import os
import re
import stat
import sys
import threading
import time
@@ -74,30 +72,80 @@ class _Refused(OSError):
"""The path is not one we are willing to write to, and never will be."""
DiagnosticLevel = Literal["debug", "info", "warn", "error"]
DiagnosticLevel = Literal["info", "warn", "error"]
_IDENTIFIER = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{0,255}$")
_EVENT = re.compile(r"^[a-z][a-z0-9]*(?:[._-][a-z0-9]+){0,15}$")
_DIAGNOSTIC_LOG_NAME = re.compile(
r"^diagnostic-(runtime|core|host-bridge|desktop|cli)\.jsonl(?:\.\d+)?$"
_DETAIL_KEYS = frozenset(
{
"action",
"attempt",
"count",
"method",
"mode",
"phase",
"platform",
"reason",
"result_count",
"source_id",
"spawned",
"status",
"version",
"parent_session_id",
"invalid_lines",
"truncated",
"open_file_count",
"memory_bytes",
"thread_count",
"database_error_count",
"route_path",
"sqlite_errorcode",
"sqlite_errorname",
"file_count",
"sqlite_file_count",
"byte_file_count",
"total_bytes",
"copied_bytes",
"bytes_per_second",
"stored_count",
"revised_count",
"skipped_count",
"error_count",
"pending_count",
"prepared_count",
"queue_depth",
"in_flight_count",
"call_count",
"created_count",
"updated_count",
"superseded_count",
"deleted_count",
"operation_kind",
"index_complete",
"lexical_hits",
"semantic_hits",
"generation",
"timed_out",
"failure_site",
"error_class",
"file_kind",
"file_size",
"file_mtime_ns",
"file_inode",
"cursor_new",
"cursor_old",
"cursor_read_to_end",
"cursor_rewind_branch",
"summed_ms",
"model_ms",
"accounted_ms",
"unaccounted_ms",
"conversation_hash",
# The sedimentation switch: a bounded boolean, the same shape as
# `spawned` and `index_complete`. Left out of this list it did not
# drop a log line, it turned the switch's own request into a 500.
"enabled",
}
)
_DETAIL_KEY = re.compile(r"^[A-Za-z][A-Za-z0-9_.-]{0,63}$")
_CODE_FRAME = re.compile(
r"^(?:(?:apps|packages|scripts)/(?:[A-Za-z0-9_-]+/)*[A-Za-z0-9_.-]+:\d+:\d+"
r"|one_memory/(?:[A-Za-z0-9_-]+/)*[A-Za-z0-9_.-]+:\d+:[A-Za-z_][A-Za-z0-9_]*"
r"|[A-Za-z0-9_][A-Za-z0-9_.-]*\.py:\d+:[A-Za-z_][A-Za-z0-9_]*)$"
)
_SHORT_CREDENTIAL_TOKEN = re.compile(
r"(?<![A-Za-z0-9])([A-Za-z0-9]{24,47})(?:[_-])?(?![A-Za-z0-9])"
)
_REFUSED_DETAIL_KEYS = frozenset({"__proto__", "prototype", "constructor"})
_MAX_DETAIL_KEYS = 32
_MAX_SAFE_INTEGER = (1 << 53) - 1
_EXPORT_SECRET_MARKER = re.compile(
r"(?:^|[._:-])(?:authorization|credential|password|prompt|query|secret|token)(?:$|[._:-])",
re.IGNORECASE,
)
_TRACE_STRING_DETAIL_KEYS = frozenset({"phase", "mode", "status", "reason", "failure_site"})
_TRACE_NUMBER_DETAIL_KEYS = frozenset({"count"})
_trace_id: ContextVar[str | None] = ContextVar("one_memory_trace_id", default=None)
_active: DiagnosticLog | None = None
_database_errors: Counter[tuple[str, str]] = Counter()
@@ -216,12 +264,7 @@ class DiagnosticLog:
}
value.update({key: _identifier(item) for key, item in identifiers.items() if item})
if duration_ms is not None:
if (
not isinstance(duration_ms, int)
or isinstance(duration_ms, bool)
or duration_ms < 0
or duration_ms > _MAX_SAFE_INTEGER
):
if not isinstance(duration_ms, int) or duration_ms < 0:
raise ValueError("diagnostic_duration_invalid")
value["duration_ms"] = duration_ms
for field_name, item in {
@@ -233,9 +276,7 @@ class DiagnosticLog:
if item is not None:
value[field_name] = _identifier(item)
if safe_stack:
if len(safe_stack) > 16:
raise ValueError("diagnostic_stack_invalid")
value["safe_stack"] = [_safe_stack_line(line) for line in safe_stack]
value["safe_stack"] = [_safe_stack_line(line) for line in safe_stack[:256]]
if details:
value["details"] = _details(details)
self._submit(_serialize(value))
@@ -248,8 +289,6 @@ class DiagnosticLog:
event: str,
outcome: str,
) -> dict[str, object]:
if level not in ("debug", "info", "warn", "error"):
raise ValueError("diagnostic_level_invalid")
return {
"schema_version": 1,
"timestamp": datetime.now(UTC).isoformat().replace("+00:00", "Z"),
@@ -612,17 +651,24 @@ def _drain_diagnostics_at_exit() -> None:
def diagnostic_event(**values: object) -> None:
active = _active
if active is None:
return
if values.get("level") == "debug" and not logging.getLogger().isEnabledFor(logging.DEBUG):
return
try:
active = _active
if active is None:
return
if values.get("level") == "debug" and not logging.getLogger().isEnabledFor(logging.DEBUG):
return
active.write(**values) # type: ignore[arg-type]
except Exception:
# Diagnostics are best-effort: validation, serialization, logging, and
# filesystem failures must never become a product outage. BaseException
# deliberately remains visible so process-control signals still work.
except OSError:
# Diagnostics must not become a new product outage when a disk is full
# or the log directory is temporarily unavailable.
return
except ValueError:
# Same promise, same reason, for a rejected payload instead of a failed
# write. An unlisted `details` key once turned every external recall
# into a 500 and then killed the process, and the sedimentation switch
# into a permanent failure. The payload itself is still wrong and is
# still caught in CI by tests/test_diagnostic_payload_keys.py; what
# must not happen is the caller paying for it.
return
@@ -645,95 +691,46 @@ def read_trace_events(
if (request_id is None) == (trace_id is None):
raise ValueError("exactly_one_trace_identity_required")
key, expected = ("request_id", request_id) if request_id is not None else ("trace_id", trace_id)
if expected is None or not _export_identifier(expected):
if expected is None or not _IDENTIFIER.fullmatch(expected):
raise ValueError("invalid_trace_identity")
selected: list[dict[str, object]] = []
directory = data_dir / "logs"
try:
directory_info = directory.lstat()
except FileNotFoundError:
return ()
if stat.S_ISLNK(directory_info.st_mode) or not stat.S_ISDIR(directory_info.st_mode):
raise ValueError("diagnostic_directory_unsafe")
files: list[tuple[Path, float]] = []
for path in directory.iterdir():
if not _DIAGNOSTIC_LOG_NAME.fullmatch(path.name):
allowed_details = {"phase", "mode", "status", "reason", "failure_site", "count"}
files = sorted(
(data_dir / "logs").glob("diagnostic-*.jsonl*"),
key=lambda path: path.stat().st_mtime,
)
for path in files:
if path.is_symlink() or not path.is_file():
continue
try:
info = path.lstat()
except OSError:
continue
if stat.S_ISREG(info.st_mode) and not stat.S_ISLNK(info.st_mode):
files.append((path, info.st_mtime))
for path, _mtime in sorted(files, key=lambda item: item[1]):
for line in _read_regular_file_no_follow(path).splitlines():
for line in path.read_text(encoding="utf-8", errors="replace").splitlines():
try:
event = json.loads(line)
except (TypeError, json.JSONDecodeError):
continue
if not isinstance(event, dict) or event.get(key) != expected:
continue
timestamp = event.get("timestamp")
if not isinstance(timestamp, str):
continue
try:
datetime.fromisoformat(timestamp.replace("Z", "+00:00"))
except ValueError:
continue
component = event.get("component")
event_name = event.get("event")
outcome = event.get("outcome")
projected: dict[str, object] = {
"timestamp": timestamp,
"component": (
component
if isinstance(component, str) and _export_identifier(component)
else "diagnostic.redacted"
),
"event": (
event_name
if isinstance(event_name, str) and _export_identifier(event_name)
else "diagnostic.redacted"
),
"outcome": (
outcome
if isinstance(outcome, str) and _export_identifier(outcome)
else "redacted"
),
}
duration_ms = event.get("duration_ms")
if (
isinstance(duration_ms, int)
and not isinstance(duration_ms, bool)
and 0 <= duration_ms <= _MAX_SAFE_INTEGER
):
projected["duration_ms"] = duration_ms
for name in ("error_code", "error_type"):
value = event.get(name)
if isinstance(value, str) and _export_identifier(value):
projected[name] = value
details = event.get("details")
projected = {
name: event[name]
for name in (
"timestamp",
"component",
"event",
"outcome",
"duration_ms",
"error_code",
"error_type",
)
if name in event
}
if isinstance(details, dict):
safe_details = _trace_details(details)
if safe_details:
projected["details"] = safe_details
projected["details"] = {
name: value for name, value in details.items() if name in allowed_details
}
selected.append(projected)
return tuple(selected[-max(1, min(limit, 500)) :])
def _read_regular_file_no_follow(path: Path) -> str:
descriptor = os.open(path, os.O_RDONLY | _O_NOFOLLOW)
try:
if not stat.S_ISREG(os.fstat(descriptor).st_mode):
raise ValueError("diagnostic_file_unsafe")
with os.fdopen(descriptor, encoding="utf-8", errors="replace") as handle:
descriptor = -1
return handle.read()
finally:
if descriptor >= 0:
os.close(descriptor)
def current_trace_id() -> str | None:
return _trace_id.get()
@@ -804,7 +801,7 @@ def record_database_error(
sqlite_code = getattr(error, "sqlite_errorcode", None)
sqlite_name = getattr(error, "sqlite_errorname", None)
details: dict[str, object] = {"phase": safe_component, "count": count}
if route_path is not None and route_path.startswith(("/api/v1/", "/api/v2/")):
if route_path is not None and route_path.startswith("/api/v1/"):
details["route_path"] = route_path
if isinstance(sqlite_code, int):
details["sqlite_errorcode"] = sqlite_code
@@ -840,77 +837,8 @@ def diagnostic_exception(error: BaseException) -> dict[str, object]:
}
def _contains_credential_material(value: str) -> bool:
return bool(
re.search(
r"(?:^|[^A-Za-z0-9])(?:sk-|sk_|omhg_|ghp_|github_pat_)[A-Za-z0-9._-]+",
value,
re.IGNORECASE,
)
or re.search(r"(?:^|[^A-Za-z0-9])xox[baprs]-[A-Za-z0-9-]+", value, re.IGNORECASE)
or re.search(r"(?:^|[^A-Za-z0-9])(?:AKIA|ASIA)[A-Z0-9]{16}(?![A-Za-z0-9])", value)
or re.search(r"(?:^|[^A-Za-z0-9])AKID[A-Za-z0-9]{12,}", value, re.IGNORECASE)
or re.search(r"(?:^|[^A-Za-z0-9])AIza[A-Za-z0-9_-]{20,}", value)
or re.search(r"(?:^|[^A-Za-z0-9])GOCSPX-[A-Za-z0-9_-]+", value)
or any(
_is_conservative_short_credential(match.group(1))
for match in _SHORT_CREDENTIAL_TOKEN.finditer(value)
)
or re.search(
r"\b[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\b",
value,
)
or any(
len(candidate) >= 48
and any(character.isalpha() for character in candidate)
and any(character.isdigit() for character in candidate)
for candidate in re.findall(r"\b[A-Za-z0-9._-]{48,}\b", value)
)
)
def _is_conservative_short_credential(value: str) -> bool:
return bool(
re.search(r"[A-Z]", value)
and re.search(r"[a-z]", value)
and re.search(r"\d", value)
and not re.fullmatch(r"[A-Fa-f0-9]+", value)
)
def _export_identifier(value: str) -> bool:
return bool(
_IDENTIFIER.fullmatch(value)
and not _EXPORT_SECRET_MARKER.search(value)
and not _contains_credential_material(value)
)
def _trace_details(values: Mapping[object, object]) -> dict[str, str | int | float]:
result: dict[str, str | int | float] = {}
for key, value in values.items():
if (
key in _TRACE_STRING_DETAIL_KEYS
and isinstance(value, str)
and _export_identifier(value)
):
result[str(key)] = value
elif key in _TRACE_NUMBER_DETAIL_KEYS and isinstance(value, (int, float)):
if isinstance(value, bool):
continue
safe_integer = isinstance(value, int) and abs(value) <= _MAX_SAFE_INTEGER
safe_float = (
isinstance(value, float)
and math.isfinite(value)
and (not value.is_integer() or abs(value) <= _MAX_SAFE_INTEGER)
)
if safe_integer or safe_float:
result[str(key)] = value
return result
def _serialize(value: Mapping[str, object]) -> str:
return json.dumps(value, ensure_ascii=True, allow_nan=False, separators=(",", ":")) + "\n"
return json.dumps(value, ensure_ascii=True, separators=(",", ":")) + "\n"
def _uuid(value: str | None) -> str | None:
@@ -936,47 +864,24 @@ def _event_name(value: str) -> str:
def _safe_stack_line(value: str) -> str:
if (
not value
or len(value) > 256
or any(character in value for character in "\0\r\n")
or _contains_credential_material(value)
):
if not value or len(value) > 256 or "\n" in value or "\r" in value:
raise ValueError("diagnostic_stack_invalid")
if _CODE_FRAME.fullmatch(value):
return value
if value.startswith("thread|"):
frames = value.removeprefix("thread|").split(" > ")
if frames and all(_CODE_FRAME.fullmatch(frame) for frame in frames):
return value
raise ValueError("diagnostic_stack_invalid")
if Path(value.split(":", 1)[0]).is_absolute() or "file://" in value:
raise ValueError("diagnostic_stack_invalid")
return value
def _details(
values: Mapping[str, str | int | float | bool | None],
) -> dict[str, str | int | float | bool | None]:
if len(values) > _MAX_DETAIL_KEYS:
raise ValueError("diagnostic_detail_count_invalid")
result: dict[str, str | int | float | bool | None] = {}
for key, value in values.items():
if (
not isinstance(key, str)
or not _DETAIL_KEY.fullmatch(key)
or key in _REFUSED_DETAIL_KEYS
):
if key not in _DETAIL_KEYS:
raise ValueError("diagnostic_detail_key_invalid")
if isinstance(value, str):
if not value or len(value) > 256 or any(character in value for character in "\0\r\n"):
if not value or len(value) > 256 or "\n" in value or "\r" in value:
raise ValueError("diagnostic_detail_value_invalid")
elif isinstance(value, float):
if not math.isfinite(value) or (value.is_integer() and abs(value) > _MAX_SAFE_INTEGER):
raise ValueError("diagnostic_detail_value_invalid")
elif isinstance(value, bool) or value is None:
pass
elif isinstance(value, int):
if abs(value) > _MAX_SAFE_INTEGER:
raise ValueError("diagnostic_detail_value_invalid")
else:
elif value is not None and not isinstance(value, (int, float, bool)):
raise ValueError("diagnostic_detail_value_invalid")
result[key] = value
return result
@@ -3,8 +3,8 @@
Every site that catches `Exception` broadly must call `report_exception`
before (or instead of) continuing, so a failure never looks like success
and a stack is always on disk. Content (conversation text, keys, SQL
parameters, and exception messages) never reaches the diagnostic log: only
exception type, a repeat count, and file:line:function frames.
parameters) never reaches the log: only exception type, a truncated
message, and file:line:function frames.
"""
from __future__ import annotations
@@ -67,7 +67,7 @@ def report_exception(
event=event,
outcome="failed",
**payload,
details={"count": _RECENT[key]},
details={"reason": _safe_message(error), "count": _RECENT[key]},
)
if full:
_logger.error(
@@ -23,9 +23,13 @@ _ENVELOPE_HEADERS = frozenset(
)
_BROKER_ROUTES = {
("POST", "/api/v1/recall"): "om_recall",
("POST", "/api/v1/memories/ask"): "om_ask",
("POST", "/api/v2/memories/ask"): "om_ask",
("POST", "/api/v2/memories/ask/stream"): "om_ask",
("GET", "/api/v1/conversations/search"): "om_conversations",
("POST", "/api/v1/diagnostics/trace"): "om_diagnostic_trace",
("POST", "/api/v1/boundary/peek"): "om_recall",
("POST", "/api/v1/boundary/ack"): "om_recall",
}
_CONCRETE_CALL_EVENT_DELIVERY = re.compile(
r"/api/v1/call-events/[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-"
@@ -124,9 +128,11 @@ def require_external_endpoint(request: Request) -> None:
access = external_access(request.headers)
if access is None:
return
if request.method == "POST" and request.url.path == "/api/v2/context/search":
# om_conversations reaches Core only here, as the raw lane; the route
# resolves each requested lane against its original grant.
if request.method == "POST" and request.url.path in {
"/api/v1/context/search",
"/api/v2/context/search",
}:
# The route resolves each requested lane against its original grant.
# Passing this envelope gate does not authorize either lane by itself.
if access.tools.isdisjoint({"om_recall", "om_conversations"}):
raise HTTPException(403, "external_recall_scope_denied")
@@ -140,3 +146,7 @@ def require_external_endpoint(request: Request) -> None:
raise HTTPException(403, "external_endpoint_denied")
if tool not in access.tools:
raise HTTPException(403, "external_recall_scope_denied")
if tool == "om_conversations":
access.require(tool, frozenset({"all-apps"}))
if not any(scope.startswith("source:") for scope in access.scopes):
raise HTTPException(403, "external_source_permission_required")
@@ -0,0 +1,127 @@
"""Bounded homepage projection without loading every memory into the renderer."""
import sqlite3
import time
from datetime import UTC, datetime
from pathlib import Path
from pydantic import BaseModel, ConfigDict
from one_memory.database_access import main_database
from one_memory.memory_evidence_freshness import register_memory_freshness
class HomeLatestMemory(BaseModel):
model_config = ConfigDict(frozen=True)
memory_id: str
content: str
truncated: bool
revision: int
action: str
updated_at: datetime
source_ids: tuple[str, ...]
class HomeSummary(BaseModel):
model_config = ConfigDict(frozen=True)
memory_count: int
latest: HomeLatestMemory | None
observed_at: datetime
#: 2026-10-12: the read hit its deadline and the numbers below are not a
#: snapshot of reality. Callers should keep their previous summary on
#: screen instead of treating count=0 as "no memories".
stale: bool = False
class HomeSummaryUnavailable(Exception):
"""The homepage projection could not be read inside its own deadline.
Raised instead of letting SQLite's interrupt escape as a bare
`OperationalError`, so the route can answer with an empty projection
rather than a 500.
"""
def read_home_summary(database_path: Path, *, user_id: str = "local") -> HomeSummary:
# The exclusion mirrors SourceReadAccess: one revoked active citation hides
# the whole memory. Permission filtering precedes both count and LIMIT.
# `+link.active` keeps SQLite on the (memory_id, evidence_id) primary key.
# Without the unary plus it picks the (active, ...) index and walks every
# active link once per memory: 2.9 s instead of 0.07 s on a 1,810-memory
# library, enough to time the homepage out whenever the machine is busy.
readable = """
m.user_id = ? AND m.status = 'confirmed' AND m.deleted_at IS NULL
AND NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
JOIN source_grants g ON g.user_id=e.user_id AND g.source_id=e.source_id
WHERE link.memory_id=m.memory_id AND +link.active=1
AND g.revoked_at IS NOT NULL
)
AND NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
LEFT JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.memory_id=m.memory_id AND +link.active=1
AND (e.evidence_id IS NULL OR e.user_id != m.user_id)
)
"""
# The total counts with the cheap permission filters only: the per-row
# freshness UDF scales with the whole library and can burn the whole read
# budget on its own (D22). Product accepted a small drift here; the rows
# that are actually displayed are still checked row by row below.
with main_database(database_path).read() as db:
deadline = time.monotonic() + 5
db.set_progress_handler(lambda: int(time.monotonic() >= deadline), 1000)
db.row_factory = sqlite3.Row
try:
db.execute("BEGIN")
count = db.execute(
f"SELECT COUNT(*) FROM governed_memories m WHERE {readable}", (user_id,)
).fetchone()[0]
readable += " AND " + register_memory_freshness(db)
row = db.execute(
f"""SELECT m.memory_id, substr(m.content,1,320) AS content,
length(m.content)>320 AS truncated, m.revision,
COALESCE(v.action, 'unknown') AS action, m.updated_at
FROM governed_memories m
LEFT JOIN governed_memory_versions v
ON v.memory_id=m.memory_id AND v.revision=m.revision
WHERE {readable}
ORDER BY m.updated_at DESC, m.memory_id LIMIT 1""",
(user_id,),
).fetchone()
latest = None
if row is not None:
sources = db.execute(
"""SELECT DISTINCT e.source_id FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id=link.evidence_id
WHERE link.memory_id=? AND +link.active=1 ORDER BY e.source_id""",
(row["memory_id"],),
).fetchall()
latest = HomeLatestMemory(
memory_id=row["memory_id"],
content=row["content"],
truncated=bool(row["truncated"]),
revision=row["revision"],
action=row["action"],
updated_at=row["updated_at"],
source_ids=tuple(source[0] for source in sources),
)
except sqlite3.Error as error:
# The progress handler above is what interrupts this read, so a
# slow homepage surfaces as SQLITE_INTERRUPT (code 9) and used to
# escape as a 500. 2026-10-12: answer with a `stale` projection
# instead of raising: the caller keeps its last good summary on
# screen (the UI poll retries within 10s), and the homepage never
# zeroes out or errors just because the machine is busy importing.
if getattr(error, "sqlite_errorcode", None) != sqlite3.SQLITE_INTERRUPT:
raise
return HomeSummary(
memory_count=0,
latest=None,
observed_at=datetime.now(UTC),
stale=True,
)
return HomeSummary(memory_count=count, latest=latest, observed_at=datetime.now(UTC))
@@ -14,7 +14,7 @@ from pathlib import Path
from typing import Literal, cast
HY_MEMORY_DISTRIBUTION = "hy-memory"
HY_MEMORY_VERSION = "2.0.6"
HY_MEMORY_VERSION = "2.0.5.dev3"
HY_MEMORY_NAMESPACE = "governed-v2"
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
@@ -1,20 +1,12 @@
"""Offline maintenance of the HY SDK data plane for App migrations.
Runtime product code must use public ``HyMemoryClient`` methods. The startup
migration boundary is the isolated exception: all writers are stopped, so it
may copy, restore and repair SDK-owned local stores.
"""
"""Migration-only access to the HY SDK's public inventory and import APIs."""
from __future__ import annotations
import json
import sqlite3
from collections.abc import Iterator
from contextlib import closing
from dataclasses import dataclass, field
from datetime import UTC, datetime, timedelta
import asyncio
from collections.abc import Callable, Mapping
from datetime import datetime
from pathlib import Path
from typing import Any
from typing import Protocol
from hy_memory import ( # type: ignore[import-untyped]
DataPlaneRestore,
@@ -23,27 +15,189 @@ from hy_memory import ( # type: ignore[import-untyped]
snapshot_local_data_plane,
)
from one_memory.database_access import main_database
from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE
# v2 reads the backfill's governed_memory_evidence links; v1 missed them.
_METADATA_REPAIR_VERSION = 2
_BATCH = 500
from one_memory.hy_sdk_types import (
HySdkMemoryPage,
HySdkMemoryRecord,
HySdkMigrationError,
HySdkRawPage,
HySdkRawRecord,
)
@dataclass(frozen=True)
class RepairMetadata:
source: str | None
memory_at: int | None
created_at: int | None
class MigrationClient(Protocol):
def list_memories(
self,
*,
user_id: str,
agent_id: str | None = None,
statuses: list[str] | None = None,
limit: int = 100,
offset: int = 0,
order: str = "desc",
) -> Mapping[str, object]: ...
def import_memory_exact(self, content: str, **kwargs: object) -> Mapping[str, object]: ...
def list_raw(
self,
*,
user_id: str,
agent_id: str | None = None,
session_id: str | None = None,
source: str | None = None,
limit: int = 100,
offset: int = 0,
order: str = "desc",
) -> Mapping[str, object]: ...
def import_raw(self, content: str, **kwargs: object) -> Mapping[str, object]: ...
def delete(self, memory_id: str) -> Mapping[str, object]: ...
def get_write_outcome(self, request_id: str) -> Mapping[str, object] | None: ...
@dataclass(frozen=True)
class MetadataRepairResult:
migration_id: str | None
mapped: int
vector_rows: int
coding_rows: int
class HySdkMigration:
"""Typed async adapter over the SDK's exact migration primitives."""
def __init__(self, client: MigrationClient) -> None:
self.client = client
async def list_memories(
self,
*,
user_id: str,
agent_id: str | None = None,
statuses: tuple[str, ...] = ("active",),
limit: int = 100,
offset: int = 0,
order: str = "desc",
) -> HySdkMemoryPage:
raw = await _sdk_call(
"hy_sdk_memory_list_failed",
self.client.list_memories,
user_id=user_id,
agent_id=agent_id,
statuses=list(statuses),
limit=limit,
offset=offset,
order=order,
)
return _memory_page(raw, limit=limit, offset=offset)
async def import_memory_exact(
self,
*,
content: str,
memory_id: str,
user_id: str,
agent_id: str,
session_id: str,
source: str,
layer: str,
status: str,
owner: str | None,
tags: tuple[str, ...],
supersedes: tuple[str, ...],
superseded_by: tuple[str, ...],
is_latest: bool,
memory_at: datetime | None,
metadata: Mapping[str, object],
request_id: str,
) -> dict[str, object]:
raw = await _sdk_call(
"hy_sdk_memory_import_failed",
self.client.import_memory_exact,
content,
memory_id=memory_id,
user_id=user_id,
agent_id=agent_id,
session_id=session_id,
source=source,
layer=layer,
status=status,
owner=owner,
tags=list(tags),
supersedes=list(supersedes),
superseded_by=list(superseded_by),
is_latest=is_latest,
memory_at=memory_at,
metadata=dict(metadata),
request_id=request_id,
)
return _successful_result(raw, "hy_sdk_memory_import_failed")
async def list_raw(
self,
*,
user_id: str,
agent_id: str | None = None,
session_id: str | None = None,
source: str | None = None,
limit: int = 100,
offset: int = 0,
order: str = "desc",
) -> HySdkRawPage:
raw = await _sdk_call(
"hy_sdk_raw_list_failed",
self.client.list_raw,
user_id=user_id,
agent_id=agent_id,
session_id=session_id,
source=source,
limit=limit,
offset=offset,
order=order,
)
return _raw_page(raw, limit=limit, offset=offset)
async def import_raw(
self,
*,
content: str,
raw_id: str,
user_id: str,
agent_id: str,
session_id: str,
source: str,
memory_at: datetime | None,
metadata: Mapping[str, object],
request_id: str,
) -> dict[str, object]:
raw = await _sdk_call(
"hy_sdk_raw_import_failed",
self.client.import_raw,
content,
raw_id=raw_id,
user_id=user_id,
agent_id=agent_id,
session_id=session_id,
source=source,
memory_at=memory_at,
metadata=dict(metadata),
request_id=request_id,
)
return _successful_result(raw, "hy_sdk_raw_import_failed")
async def delete(self, memory_id: str) -> dict[str, object]:
raw = await _sdk_call(
"hy_sdk_memory_delete_failed",
self.client.delete,
memory_id,
)
return _successful_result(raw, "hy_sdk_memory_delete_failed")
async def get_write_outcome(self, request_id: str) -> dict[str, object] | None:
raw = await _sdk_call(
"hy_sdk_write_outcome_failed",
self.client.get_write_outcome,
request_id,
)
if raw is None:
return None
if not isinstance(raw, Mapping):
raise HySdkMigrationError("hy_sdk_write_outcome_shape_invalid")
return dict(raw)
def snapshot_data_plane(
@@ -76,400 +230,152 @@ def restore_data_plane(
)
def repair_sdk_metadata_from_bundle(
*,
data_dir: Path,
database_path: Path,
now: datetime | None = None,
) -> MetadataRepairResult:
"""Restore metadata lost by historical cutovers from their verified bundles.
Offline only: no SDK client may be open on this home (see `_repair_vector`).
"""
migration_id = _sdk_migration_id(database_path)
if migration_id is None:
return MetadataRepairResult(None, 0, 0, 0)
receipt = data_dir / f".sdk-metadata-repair-v{_METADATA_REPAIR_VERSION}-{migration_id}.json"
if receipt.is_file():
document = json.loads(receipt.read_text(encoding="utf-8"))
return MetadataRepairResult(
migration_id,
int(document.get("mapped", 0)),
int(document.get("vector_rows", 0)),
int(document.get("coding_rows", 0)),
)
# A home can carry several cutovers (v1 and then v2). A later bundle may
# have been taken after an earlier cutover had already retired the legacy
# tables, so every bundle is read and their links are pooled.
legacy = sorted(
(data_dir / "backups").glob("*-pre-cutover.sdk-bundle/management/one-memory.sqlite3")
)
if not legacy:
# No bundle yet is not a finished repair: try again on the next start.
return MetadataRepairResult(migration_id, 0, 0, 0)
current = now or datetime.now(UTC)
pools: dict[str, _Pool] = {}
for path in legacy:
_merge_pools(pools, _legacy_pools(path, now=current))
mapping = _decide(pools)
vector_rows = _repair_vector(data_dir, mapping, now=current)
coding_rows = _repair_coding(data_dir, mapping)
result = MetadataRepairResult(
migration_id,
len(mapping),
vector_rows,
coding_rows,
)
temporary = receipt.with_suffix(".tmp")
temporary.write_text(
json.dumps(
{
"schema_version": 1,
"migration_id": migration_id,
"mapped": result.mapped,
"vector_rows": vector_rows,
"coding_rows": coding_rows,
},
sort_keys=True,
)
+ "\n",
encoding="utf-8",
)
temporary.replace(receipt)
return result
def _sdk_migration_id(database_path: Path) -> str | None:
with main_database(database_path).read(row_factory=sqlite3.Row) as database:
row = database.execute(
"SELECT authority,migration_id FROM memory_authority_state WHERE singleton=1"
).fetchone()
if row is None or row["authority"] != "sdk":
return None
value = row["migration_id"]
return str(value) if isinstance(value, str) and value else None
@dataclass
class _Pool:
"""Everything the legacy library knew about one SDK memory."""
sources: set[str] = field(default_factory=set)
times: list[datetime] = field(default_factory=list)
created: list[datetime] = field(default_factory=list)
def _legacy_metadata(path: Path, *, now: datetime) -> dict[str, RepairMetadata]:
return _decide(_legacy_pools(path, now=now))
def _legacy_pools(path: Path, *, now: datetime) -> dict[str, _Pool]:
"""Link each SDK memory id to the legacy evidence it was made from.
Three legacy paths wrote memories the SDK now holds:
- the cutover backfill (`import_exact`) copied every governed memory into the
SDK under its legacy id, or under the id `memory_task_outputs` recorded,
without a source or a conversation time; `governed_memory_evidence` is the
link from that memory to its evidence;
- extraction tasks recorded their evidence as `support_refs`;
- the write queue recorded the SDK ids each job produced.
"""
with closing(sqlite3.connect(f"file:{path}?mode=ro", uri=True)) as database:
database.row_factory = sqlite3.Row
tables = {
str(row[0])
for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
}
if "evidence" not in tables:
return {}
links: dict[str, set[str]] = {}
created: dict[str, list[datetime]] = {}
canonical: dict[str, str] = {}
if "memory_task_outputs" in tables:
for row in database.execute("SELECT body FROM memory_task_outputs"):
try:
body = json.loads(str(row["body"]))
except (TypeError, ValueError):
continue
legacy_id = body.get("memory_id")
canonical_id = body.get("hy_memory_id")
refs = body.get("support_refs")
if not isinstance(legacy_id, str) or not isinstance(canonical_id, str):
continue
canonical[legacy_id] = canonical_id
links.setdefault(canonical_id, set()).update(
str(ref["evidence_id"])
for ref in (refs if isinstance(refs, list) else ())
if isinstance(ref, dict) and isinstance(ref.get("evidence_id"), str)
)
if "governed_memories" in tables:
for row in database.execute(
"SELECT memory_id,created_at,updated_at FROM governed_memories"
):
legacy_id = str(row["memory_id"])
canonical_id = canonical.get(legacy_id, legacy_id)
links.setdefault(canonical_id, set())
legacy_created = _plausible_time(
row["created_at"], fallback=row["updated_at"], now=now
)
if legacy_created is not None:
created.setdefault(canonical_id, []).append(legacy_created)
if "governed_memory_evidence" in tables:
for row in database.execute(
"SELECT memory_id,evidence_id FROM governed_memory_evidence"
):
legacy_id = str(row["memory_id"])
links.setdefault(canonical.get(legacy_id, legacy_id), set()).add(
str(row["evidence_id"])
)
if "memory_write_jobs" in tables and "sdk_memory_ids_json" in _columns(
database, "memory_write_jobs"
):
job_evidence: dict[str, set[str]] = {}
if "memory_write_job_evidence" in tables:
for row in database.execute(
"SELECT job_id,evidence_id FROM memory_write_job_evidence"
):
job_evidence.setdefault(str(row["job_id"]), set()).add(str(row["evidence_id"]))
for row in database.execute(
"SELECT job_id,sdk_memory_ids_json FROM memory_write_jobs "
"WHERE sdk_memory_ids_json IS NOT NULL"
):
try:
memory_ids = json.loads(str(row["sdk_memory_ids_json"]))
except (TypeError, ValueError):
continue
for memory_id in memory_ids if isinstance(memory_ids, list) else ():
if isinstance(memory_id, str) and memory_id:
links.setdefault(memory_id, set()).update(
job_evidence.get(str(row["job_id"]), ())
)
evidence: dict[str, tuple[str, datetime | None]] = {}
wanted = sorted({identifier for refs in links.values() for identifier in refs})
for batch in _batches(wanted):
placeholders = ",".join("?" for _ in batch)
for row in database.execute(
"SELECT evidence_id,source_id,created_at,imported_at "
f"FROM evidence WHERE evidence_id IN ({placeholders})",
batch,
):
evidence[str(row["evidence_id"])] = (
str(row["source_id"]),
_plausible_time(row["created_at"], fallback=row["imported_at"], now=now),
)
pools: dict[str, _Pool] = {}
for canonical_id, refs in links.items():
pool = pools.setdefault(canonical_id, _Pool())
for identifier in refs:
if identifier not in evidence:
continue
source, timestamp = evidence[identifier]
if source and source != "default":
pool.sources.add(source)
if timestamp is not None:
pool.times.append(timestamp)
pool.created.extend(created.get(canonical_id, ()))
return pools
def _merge_pools(into: dict[str, _Pool], other: dict[str, _Pool]) -> None:
for canonical_id, pool in other.items():
merged = into.setdefault(canonical_id, _Pool())
merged.sources |= pool.sources
merged.times.extend(pool.times)
merged.created.extend(pool.created)
def _decide(pools: dict[str, _Pool]) -> dict[str, RepairMetadata]:
# Several legacy memories can fold into one SDK memory. Their evidence is
# pooled before deciding, so a disagreeing source stays unknown instead of
# whichever link happened to come last.
return {
canonical_id: RepairMetadata(
source=next(iter(pool.sources)) if len(pool.sources) == 1 else None,
memory_at=int(max(pool.times).timestamp()) if pool.times else None,
created_at=int(min(pool.created).timestamp()) if pool.created else None,
)
for canonical_id, pool in pools.items()
}
def _columns(database: sqlite3.Connection, table: str) -> set[str]:
return {str(row[1]) for row in database.execute(f"PRAGMA table_info({table})")}
def _plausible_time(
value: object,
*,
fallback: object,
now: datetime,
) -> datetime | None:
for candidate in (value, fallback):
if not isinstance(candidate, str) or not candidate:
continue
try:
parsed = datetime.fromisoformat(candidate)
except ValueError:
continue
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=UTC)
parsed = parsed.astimezone(UTC)
if parsed <= now + timedelta(days=1):
return parsed
return None
def _repair_vector(
data_dir: Path,
mapping: dict[str, RepairMetadata],
*,
now: datetime,
) -> int:
directory = data_dir / "hy-memory" / HY_MEMORY_NAMESPACE / "vector"
if not (directory / "chroma.sqlite3").is_file():
return 0
from one_memory.sdk_vector_reattach import open_offline_chroma
client = open_offline_chroma(directory)
# This must be the only Chroma client on the directory: Chroma shares one
# System per path, and this client's close() drops it from under any
# other (KeyError on its next lookup). `sdk-migrate` runs it after the
# cutover with Runtime stopped; Core's startup fallback runs it while
# `hold_stores_offline` keeps every SDK client waiting.
async def _sdk_call[T](
code: str,
call: Callable[..., T],
*args: object,
**kwargs: object,
) -> T:
pending = asyncio.create_task(asyncio.to_thread(call, *args, **kwargs))
try:
repaired = 0
ids = tuple(mapping)
for collection in client.list_collections():
for batch in _batches(ids):
page = collection.get(ids=list(batch), include=["metadatas"])
updated_ids: list[str] = []
updated_metadata: list[dict[str, object]] = []
for memory_id, metadata in zip(
page.get("ids") or (),
page.get("metadatas") or (),
strict=True,
):
if not isinstance(metadata, dict):
continue
desired = mapping.get(str(memory_id))
if desired is None:
continue
next_metadata = dict(metadata)
changed = False
if desired.source and next_metadata.get("source") in {
None,
"",
"default",
}:
next_metadata["source"] = desired.source
changed = True
if (
desired.memory_at is not None
and next_metadata.get("memory_at") != desired.memory_at
):
next_metadata["memory_at"] = desired.memory_at
changed = True
if (
desired.created_at is not None
and next_metadata.get("gmt_created") != desired.created_at
):
next_metadata["gmt_created"] = desired.created_at
changed = True
if changed:
updated_ids.append(str(memory_id))
updated_metadata.append(next_metadata)
if updated_ids:
collection.update(ids=updated_ids, metadatas=updated_metadata)
repaired += len(updated_ids)
repaired += _settle_future_times(collection, now=now)
finally:
client.close()
return repaired
return await asyncio.shield(pending)
except asyncio.CancelledError:
await _finish_cancelled_call(pending)
raise
except Exception as error:
raise HySdkMigrationError(code) from error
def _settle_future_times(collection: Any, *, now: datetime) -> int:
"""Move conversation times that are still in the future back to the write time.
A historical Cursor bug stored a file hash as the message time (years like
2254). The SDK lists newest first by `memory_at`, so those rows hold the
first page. Rows the bundles could date were fixed above; for the rest the
write time is the closest true moment that is known.
"""
cutoff = int((now + timedelta(days=1)).timestamp())
page = collection.get(where={"memory_at": {"$gt": cutoff}}, include=["metadatas"])
updated_ids: list[str] = []
updated_metadata: list[dict[str, object]] = []
for memory_id, metadata in zip(page.get("ids") or (), page.get("metadatas") or (), strict=True):
if not isinstance(metadata, dict):
async def _finish_cancelled_call[T](pending: asyncio.Task[T]) -> None:
while not pending.done():
try:
await asyncio.shield(pending)
except asyncio.CancelledError:
continue
created = metadata.get("gmt_created")
settled = (
created
if isinstance(created, int) and not isinstance(created, bool) and created <= cutoff
else int(now.timestamp())
except Exception:
return
if not pending.cancelled():
pending.exception()
def _memory_page(raw: object, *, limit: int, offset: int) -> HySdkMemoryPage:
if not isinstance(raw, Mapping):
raise HySdkMigrationError("hy_sdk_memory_page_shape_invalid")
rows: list[object] = []
total = 0
for bucket_name, row_name in (("vdb", "memories"), ("graph", "nodes")):
bucket = raw.get(bucket_name)
if bucket is None and bucket_name == "graph":
continue
if not isinstance(bucket, Mapping):
raise HySdkMigrationError("hy_sdk_memory_page_shape_invalid")
bucket_rows = bucket.get(row_name)
bucket_total = bucket.get("total")
if not isinstance(bucket_rows, list) or type(bucket_total) is not int or bucket_total < 0:
raise HySdkMigrationError("hy_sdk_memory_page_shape_invalid")
rows.extend(bucket_rows)
total += bucket_total
records: dict[str, HySdkMemoryRecord] = {}
for row in rows:
record = _memory_record(row)
records.setdefault(record.memory_id, record)
return HySdkMemoryPage(
memories=tuple(records.values()),
total=total,
limit=limit,
offset=offset,
)
def _memory_record(raw: object) -> HySdkMemoryRecord:
if not isinstance(raw, Mapping):
raise HySdkMigrationError("hy_sdk_memory_record_shape_invalid")
required = ("memory_id", "content", "layer", "status", "user_id", "agent_id")
if any(not isinstance(raw.get(name), str) or not raw[name] for name in required):
raise HySdkMigrationError("hy_sdk_memory_record_shape_invalid")
custom = raw.get("custom")
source_raw_ids: list[object] = []
if isinstance(custom, Mapping) and isinstance(custom.get("source_raw_memory_ids"), list):
source_raw_ids.extend(custom["source_raw_memory_ids"])
if isinstance(raw.get("source_raw_memory_id"), str):
source_raw_ids.append(raw["source_raw_memory_id"])
try:
return HySdkMemoryRecord(
memory_id=raw["memory_id"],
content=raw["content"],
layer=raw["layer"],
status=raw["status"],
is_latest=bool(raw.get("is_latest", True)),
supersedes=_string_tuple(raw.get("supersedes")),
superseded_by=_string_tuple(raw.get("superseded_by")),
tags=_string_tuple(raw.get("tags")),
owner=raw.get("owner") if raw.get("owner") in {"user", "agent"} else None,
source_raw_ids=_string_tuple(source_raw_ids),
user_id=raw["user_id"],
agent_id=raw["agent_id"],
session_id=_optional_string(raw.get("session_id")),
source=_optional_string(raw.get("source")),
memory_at=_optional_int(raw.get("memory_at")),
gmt_created=_optional_int(raw.get("gmt_created")),
)
updated_ids.append(str(memory_id))
updated_metadata.append({**metadata, "memory_at": settled})
if updated_ids:
collection.update(ids=updated_ids, metadatas=updated_metadata)
return len(updated_ids)
except (TypeError, ValueError) as error:
raise HySdkMigrationError("hy_sdk_memory_record_shape_invalid") from error
def _repair_coding(
data_dir: Path,
mapping: dict[str, RepairMetadata],
) -> int:
root = data_dir / "hy-memory" / HY_MEMORY_NAMESPACE
repaired = 0
for name in ("coding.sqlite3", "coding_cv.sqlite3"):
path = root / name
if not path.is_file():
continue
with closing(sqlite3.connect(path)) as database:
tables = {
str(row[0])
for row in database.execute("SELECT name FROM sqlite_master WHERE type='table'")
}
for table, identifier in (
("coding_memory_meta", "experience_id"),
("working_fact", "fact_id"),
):
if table not in tables:
continue
rows = database.execute(
f"SELECT {identifier},source,created_at FROM {table}"
).fetchall()
for memory_id, source, created_at in rows:
desired = mapping.get(str(memory_id))
if desired is None:
continue
updates: list[str] = []
values: list[object] = []
if desired.source and source in {None, "", "default"}:
updates.append("source=?")
values.append(desired.source)
if desired.created_at is not None:
desired_created_at = (
datetime.fromtimestamp(desired.created_at, tz=UTC)
.replace(tzinfo=None)
.isoformat()
)
if created_at != desired_created_at:
updates.append("created_at=?")
values.append(desired_created_at)
if not updates:
continue
values.append(memory_id)
database.execute(
f"UPDATE {table} SET {','.join(updates)} WHERE {identifier}=?",
values,
)
repaired += 1
database.commit()
return repaired
def _raw_page(raw: object, *, limit: int, offset: int) -> HySdkRawPage:
if not isinstance(raw, Mapping):
raise HySdkMigrationError("hy_sdk_raw_page_shape_invalid")
rows = raw.get("raw")
total = raw.get("total")
if not isinstance(rows, list) or type(total) is not int or total < 0:
raise HySdkMigrationError("hy_sdk_raw_page_shape_invalid")
return HySdkRawPage(
raw=tuple(_raw_record(row) for row in rows),
total=total,
limit=limit,
offset=offset,
)
def _batches[T](values: list[T] | tuple[T, ...]) -> Iterator[tuple[T, ...]]:
for start in range(0, len(values), _BATCH):
yield tuple(values[start : start + _BATCH])
def _raw_record(raw: object) -> HySdkRawRecord:
if not isinstance(raw, Mapping):
raise HySdkMigrationError("hy_sdk_raw_record_shape_invalid")
required = ("raw_id", "content", "user_id", "agent_id", "session_id", "source")
if any(not isinstance(raw.get(name), str) or not raw[name] for name in required):
raise HySdkMigrationError("hy_sdk_raw_record_shape_invalid")
custom = raw.get("custom")
try:
return HySdkRawRecord(
raw_id=raw["raw_id"],
content=raw["content"],
user_id=raw["user_id"],
agent_id=raw["agent_id"],
session_id=raw["session_id"],
source=raw["source"],
memory_at=_optional_int(raw.get("memory_at")),
gmt_created=_optional_int(raw.get("gmt_created")),
custom=dict(custom) if isinstance(custom, Mapping) else {},
)
except (TypeError, ValueError) as error:
raise HySdkMigrationError("hy_sdk_raw_record_shape_invalid") from error
def _successful_result(raw: object, code: str) -> dict[str, object]:
if not isinstance(raw, Mapping) or raw.get("success") is not True:
raise HySdkMigrationError(code)
return dict(raw)
def _string_tuple(value: object) -> tuple[str, ...]:
if not isinstance(value, list | tuple):
return ()
return tuple(item for item in value if isinstance(item, str) and item)
def _optional_string(value: object) -> str | None:
return value if isinstance(value, str) else None
def _optional_int(value: object) -> int | None:
return value if type(value) is int else None
+16 -131
View File
@@ -17,9 +17,6 @@ from pathlib import Path
from typing import Any, ClassVar
from hy_memory import ( # type: ignore[import-untyped]
AmbiguousMemoryIdError as AmbiguousMemoryIdError, # re-export for api.py
)
from hy_memory import (
HyMemoryClient,
MemoryConfig,
SharedRuntime,
@@ -36,11 +33,7 @@ from one_memory.hy_graph_admin import (
)
from one_memory.hy_sdk_types import HySdkRuntimeError
from one_memory.local_embedding import LocalEmbedder
from one_memory.model_profiles import (
ModelProfileSnapshot,
ProviderKind,
disable_thinking_params,
)
from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind
_RUNTIME_LOCK = threading.RLock()
BUILTIN_RETRIEVAL_PROFILE_ID = "builtin-retrieval"
@@ -56,16 +49,6 @@ class _RuntimeEntry:
_RUNTIMES: dict[Path, _RuntimeEntry] = {}
# Homes whose SDK-owned local stores are held offline for maintenance, and
# the client constructions in flight per home. Chroma keeps one process-wide
# System per path with no lock around create, lookup and release, so a
# second client that opens or closes beside the SDK's pulls the System out
# from under it (KeyError on the next lookup). Maintenance therefore runs
# only while no SDK client exists for that home, and new clients wait.
_STORE_GATE = threading.Condition()
_OFFLINE_STORES: set[Path] = set()
_OPENING: dict[Path, int] = {}
class LocalOnnxEmbeddingService:
"""Public ``EmbeddingService`` implementation backed by MemHub's ONNX model."""
@@ -254,26 +237,24 @@ def build_sdk_config(
root = data_dir.resolve() / "hy-memory" / "governed-v2"
root.mkdir(parents=True, exist_ok=True)
collection = _collection_base(
embedding_provider, embedding_endpoint, embedding_model, dimensions
collection_identity = "\0".join(
(
embedding_provider.value,
embedding_endpoint or "local",
embedding_model,
str(dimensions or "auto"),
)
)
llm: dict[str, Any] = {
"provider": "openai",
"model": profile.model,
"api_key": generation_key,
"base_url": generation_endpoint,
}
# The SDK sends `extra_body` on every chat call, chat and coding/working
# alike. Without it DeepSeek spent the whole agent_max_tokens on thinking
# and answered finish_reason=length with no content.
thinking_off = disable_thinking_params(profile)
if thinking_off:
llm["extra_body"] = thinking_off
collection = "memhub_" + hashlib.sha256(collection_identity.encode()).hexdigest()[:16]
return MemoryConfig.from_dict(
{
"llm": llm,
"llm": {
"provider": "openai",
"model": profile.model,
"api_key": generation_key,
"base_url": generation_endpoint,
},
"embedder": {
"provider": "openai",
"model": (
@@ -317,60 +298,7 @@ def acquire_client(
data_dir: Path,
http_transport_factory: Callable[[], Any] | None = None,
) -> ClientLease:
"""Construct a public SDK client and return its explicit runtime lease.
Waits while `hold_stores_offline` holds this home's stores.
"""
key = data_dir.resolve()
with _STORE_GATE:
while key in _OFFLINE_STORES:
_STORE_GATE.wait()
_OPENING[key] = _OPENING.get(key, 0) + 1
try:
return _acquire_client(
profile,
api_key=api_key,
data_dir=data_dir,
http_transport_factory=http_transport_factory,
)
finally:
with _STORE_GATE:
_OPENING[key] -= 1
if not _OPENING[key]:
del _OPENING[key]
_STORE_GATE.notify_all()
def hold_stores_offline(data_dir: Path) -> bool:
"""Hold this home's SDK stores for offline maintenance; False if a client is open.
While held, `acquire_client` for the home waits. Every SDK client in Core
is built there, so nothing reads or writes Chroma beside the maintenance.
`release_stores` ends the hold; it is idempotent.
"""
key = data_dir.resolve()
with _STORE_GATE:
with _RUNTIME_LOCK:
open_runtime = key in _RUNTIMES
if key in _OFFLINE_STORES or _OPENING.get(key) or open_runtime:
return False
_OFFLINE_STORES.add(key)
return True
def release_stores(data_dir: Path) -> None:
with _STORE_GATE:
_OFFLINE_STORES.discard(data_dir.resolve())
_STORE_GATE.notify_all()
def _acquire_client(
profile: ModelProfileSnapshot,
*,
api_key: str | None,
data_dir: Path,
http_transport_factory: Callable[[], Any] | None,
) -> ClientLease:
"""Construct a public SDK client and return its explicit runtime lease."""
config = build_sdk_config(profile, api_key=api_key, data_dir=data_dir)
config.http_transport_factory = http_transport_factory
mismatch = inspect_graph_width_mismatch(config)
@@ -420,49 +348,6 @@ def _generation_access(
return endpoint, api_key
def _collection_base(
provider: ProviderKind,
endpoint: str | None,
model: str,
dimensions: int | None,
) -> str:
identity = "\0".join((provider.value, endpoint or "local", model, str(dimensions or "auto")))
return "memhub_" + hashlib.sha256(identity.encode()).hexdigest()[:16]
def vector_collection_bases(profile: ModelProfileSnapshot) -> tuple[str, tuple[str, ...]]:
"""The SDK vector collection base for this embedding space, and the older ones.
The SDK derives every vector collection (chat, Coding keys, Working facts,
each variant) by suffixing this base. Builds before #145 named the same
embedding space `one_memory_<12 hex>`, with or without a declared width;
the offline migration renames those so their vectors stay readable.
"""
provider = profile.embedding_provider_kind or profile.provider_kind
local = provider is ProviderKind.LOCAL_ONNX
model = profile.embedding_model or (BUILTIN_EMBEDDING_MODEL if local else None)
if not model:
raise HySdkRuntimeError("hy_sdk_embedding_model_required")
endpoint: str | None
dimensions: int | None
if local:
endpoint, dimensions = None, profile.embedding_dimensions or BUILTIN_EMBEDDING_DIMENSIONS
legacy_endpoint = "local-onnx"
else:
endpoint = _openai_endpoint(provider, profile.embedding_base_url or profile.base_url)
dimensions = profile.embedding_dimensions
legacy_endpoint = endpoint
legacy_model = profile.embedding_model or "bge-small-zh-v1.5"
legacy = tuple(
"one_memory_"
+ hashlib.sha256(
":".join((provider.value, legacy_endpoint, legacy_model, *declared)).encode()
).hexdigest()[:12]
for declared in ((), ("auto",), (str(profile.embedding_dimensions or "auto"),))
)
return _collection_base(provider, endpoint, model, dimensions), tuple(dict.fromkeys(legacy))
def _openai_endpoint(provider: ProviderKind, endpoint: str | None) -> str:
value = (endpoint or "").rstrip("/")
if not value:
+9 -365
View File
@@ -3,8 +3,6 @@
from __future__ import annotations
from collections.abc import Mapping
from datetime import UTC, datetime
from typing import Literal, NamedTuple, cast
from pydantic import BaseModel, ConfigDict, Field
@@ -24,28 +22,14 @@ class HySdkRuntimeError(HySdkError):
class HySdkMigrationError(HySdkError):
"""The privileged offline SDK data-plane migration failed."""
"""A public SDK migration operation failed or returned an invalid shape."""
class HySdkReadError(HySdkError):
"""A public SDK read operation failed or returned an invalid shape."""
MemoryScene = Literal["chat", "working", "coding"]
MemoryKindName = Literal["chat", "experience", "fact"]
MemoryVariantName = Literal["working", "coding"]
class HySdkMemoryRecord(BaseModel):
"""One memory of any scene, in the unified shape the library lists.
Chat memories keep every field they always had. Working/coding memories
(experiences and facts) carry ``scene``/``kind``/``variant`` so a client
can open, edit and delete them again without guessing the namespace; an
experience also keeps ``task`` and ``solution`` apart, and ``content`` is
the readable ``task\nsolution`` text the SDK lists.
"""
model_config = ConfigDict(frozen=True, extra="forbid")
memory_id: str
@@ -64,36 +48,17 @@ class HySdkMemoryRecord(BaseModel):
source: str | None = None
memory_at: int | None = None
gmt_created: int | None = None
scene: MemoryScene = "chat"
kind: MemoryKindName = "chat"
# The productivity namespace ("working" | "coding"); None for chat. Pass it
# back with `kind` on get/update/delete.
variant: MemoryVariantName | None = None
task: str | None = None
solution: str | None = None
# Epoch seconds. `updated_at` falls back to `created_at`.
created_at: int | None = None
updated_at: int | None = None
@classmethod
def from_sdk(cls, raw: object) -> HySdkMemoryRecord:
if not isinstance(raw, Mapping):
raise HySdkReadError("hy_memory_record_shape_unknown")
if raw.get("kind") in {"experience", "fact"}:
return cls._from_productivity(raw)
if any(
not isinstance(raw.get(key), str) or not str(raw[key]).strip()
for key in ("memory_id", "content")
):
raise HySdkReadError("hy_memory_record_shape_unknown")
try:
gmt_created = (
raw.get("gmt_created") if isinstance(raw.get("gmt_created"), int) else None
)
created_at = _epoch(raw.get("created_at"))
if created_at is None:
created_at = gmt_created
updated_at = _epoch(raw.get("updated_at"))
return cls(
memory_id=str(raw["memory_id"]),
content=str(raw["content"]),
@@ -112,69 +77,9 @@ class HySdkMemoryRecord(BaseModel):
),
source=str(raw["source"]) if isinstance(raw.get("source"), str) else None,
memory_at=raw.get("memory_at") if isinstance(raw.get("memory_at"), int) else None,
gmt_created=gmt_created,
created_at=created_at,
updated_at=updated_at if updated_at is not None else created_at,
)
except (KeyError, TypeError, ValueError) as error:
raise HySdkReadError("hy_memory_record_shape_unknown") from error
@classmethod
def _from_productivity(cls, raw: Mapping[str, object]) -> HySdkMemoryRecord:
"""An experience or fact from list/search(scene="all") or get().
get() names the namespace in ``variant`` (its ``scene`` is the legacy
"productivity"); list/search name it in ``scene``.
"""
kind = str(raw["kind"])
variant = next(
(
value
for value in (raw.get("variant"), raw.get("scene"))
if value in {"working", "coding"}
),
None,
)
id_key = "experience_id" if kind == "experience" else "fact_id"
memory_id = raw.get("memory_id") or raw.get(id_key)
if variant is None or not isinstance(memory_id, str) or not memory_id.strip():
raise HySdkReadError("hy_memory_record_shape_unknown")
task: str | None = None
solution: str | None = None
if kind == "experience":
task = str(raw.get("task") or "").strip()
solution = str(raw.get("solution") or "").strip()
content = "\n".join(part for part in (task, solution) if part)
else:
statement = raw.get("statement")
content = str(statement if isinstance(statement, str) else raw.get("content") or "")
if not content.strip():
raise HySdkReadError("hy_memory_record_shape_unknown")
status = str(raw.get("status") or "active")
created_at = _epoch(raw.get("created_at"))
updated_at = _epoch(raw.get("updated_at"))
try:
return cls(
memory_id=memory_id,
content=content,
layer=kind,
status=status,
is_latest=status == "active",
tags=_string_tuple(raw.get("tags")),
user_id=str(raw.get("user_id") or ""),
agent_id=str(raw.get("agent_id") or ""),
session_id=(
str(raw["session_id"]) if isinstance(raw.get("session_id"), str) else None
gmt_created=(
raw.get("gmt_created") if isinstance(raw.get("gmt_created"), int) else None
),
source=str(raw["source"]) if isinstance(raw.get("source"), str) else None,
gmt_created=created_at,
scene=cast(MemoryScene, variant),
kind=cast(MemoryKindName, kind),
variant=cast(MemoryVariantName, variant),
task=task,
solution=solution,
created_at=created_at,
updated_at=updated_at if updated_at is not None else created_at,
)
except (KeyError, TypeError, ValueError) as error:
raise HySdkReadError("hy_memory_record_shape_unknown") from error
@@ -187,52 +92,11 @@ class HySdkMemoryPage(BaseModel):
total: int = Field(ge=0)
limit: int = Field(ge=1)
offset: int = Field(ge=0)
# Set on a search whose relevance window was full: matches were looked for
# only among this many most relevant memories, so `total` may undercount.
search_window: int | None = None
# Keyword search only: the server-side result window this page was cut
# from. Pass it back with the next offset to page the same results.
window_id: str | None = None
# True when the window_id the client sent had expired or did not match the
# query: the search ran again and this is page 1 of the new window.
window_reset: bool = False
# Empty-query listing only: how many memories each scene holds under the
# same filters (`total` is their sum).
totals: dict[str, int] | None = None
@classmethod
def from_sdk(cls, raw: object, *, limit: int, offset: int) -> HySdkMemoryPage:
page, _skipped = cls._parse(raw, limit=limit, offset=offset, skip_invalid=False)
return page
@classmethod
def from_sdk_skipping_invalid(
cls, raw: object, *, limit: int, offset: int
) -> tuple[HySdkMemoryPage, int]:
"""Parse a page, leaving out rows outside the public record contract.
Returns the page and how many rows were left out, so the caller can
report them. The rows themselves are never touched.
"""
return cls._parse(raw, limit=limit, offset=offset, skip_invalid=True)
@classmethod
def _parse(
cls,
raw: object,
*,
limit: int,
offset: int,
skip_invalid: bool,
) -> tuple[HySdkMemoryPage, int]:
if not isinstance(raw, Mapping):
raise HySdkReadError("hy_memory_list_shape_unknown")
if raw.get("scene") == "all":
return cls._parse_all_scenes(raw, limit=limit, offset=offset, skip_invalid=skip_invalid)
if raw.get("scene") == "productivity":
return cls._parse_experiences(
raw, limit=limit, offset=offset, skip_invalid=skip_invalid
)
rows: list[object] = []
total = 0
for bucket_name, rows_name in (("vdb", "memories"), ("graph", "nodes")):
@@ -251,128 +115,15 @@ class HySdkMemoryPage(BaseModel):
total += bucket_total
rows.extend(bucket[rows_name])
deduplicated: dict[str, HySdkMemoryRecord] = {}
skipped = 0
for row in rows:
try:
record = HySdkMemoryRecord.from_sdk(row)
except HySdkReadError:
if not skip_invalid:
raise
skipped += 1
continue
record = HySdkMemoryRecord.from_sdk(row)
deduplicated.setdefault(record.memory_id, record)
page = cls(
return cls(
memories=tuple(deduplicated.values()),
# `total` is the SDK's stable result-set count. Subtracting only
# invalid rows observed on this page would make it change as the
# caller paged. Until the SDK exposes a quarantined count, keep the
# authoritative total and skip malformed rows only in this page.
total=total,
limit=limit,
offset=offset,
)
return page, skipped
@classmethod
def _parse_all_scenes(
cls,
raw: Mapping[str, object],
*,
limit: int,
offset: int,
skip_invalid: bool,
) -> tuple[HySdkMemoryPage, int]:
"""``list_memories(scene="all")``: chat, working and coding, newest first."""
rows = raw.get("memories")
total = raw.get("total")
if (
not isinstance(rows, list)
or not isinstance(total, int)
or isinstance(total, bool)
or total < 0
):
raise HySdkReadError("hy_memory_list_shape_unknown")
totals_raw = raw.get("totals")
totals = (
{
str(scene): count
for scene, count in totals_raw.items()
if isinstance(count, int) and not isinstance(count, bool) and count >= 0
}
if isinstance(totals_raw, Mapping)
else None
)
memories: list[HySdkMemoryRecord] = []
seen: set[tuple[str, str, str | None]] = set()
skipped = 0
for row in rows:
try:
record = HySdkMemoryRecord.from_sdk(row)
except HySdkReadError:
if not skip_invalid:
raise
skipped += 1
continue
key = (record.memory_id, record.kind, record.variant)
if key in seen:
continue
seen.add(key)
memories.append(record)
return (
cls(
memories=tuple(memories),
total=total,
limit=limit,
offset=offset,
totals=totals,
),
skipped,
)
@classmethod
def _parse_experiences(
cls,
raw: Mapping[str, object],
*,
limit: int,
offset: int,
skip_invalid: bool,
) -> tuple[HySdkMemoryPage, int]:
"""``list_memories(scene="working"|"coding")``: that scene's experiences.
The SDK names the scene once, in ``variant``; each row is a bare
experience, so the kind and variant are added before parsing.
"""
rows = raw.get("experiences")
total = raw.get("total")
variant = raw.get("variant")
if (
not isinstance(rows, list)
or not isinstance(total, int)
or isinstance(total, bool)
or total < 0
or variant not in {"working", "coding"}
):
raise HySdkReadError("hy_memory_list_shape_unknown")
memories: list[HySdkMemoryRecord] = []
skipped = 0
for row in rows:
try:
if not isinstance(row, Mapping):
raise HySdkReadError("hy_memory_record_shape_unknown")
record = HySdkMemoryRecord.from_sdk(
{**row, "kind": "experience", "variant": variant}
)
except HySdkReadError:
if not skip_invalid:
raise
skipped += 1
continue
memories.append(record)
return (
cls(memories=tuple(memories), total=total, limit=limit, offset=offset),
skipped,
)
class HySdkRawRecord(BaseModel):
@@ -449,21 +200,11 @@ class HySdkRawPage(BaseModel):
offset: int = Field(ge=0)
@classmethod
def from_memory_list(
cls,
raw: object,
*,
limit: int,
offset: int,
) -> HySdkRawPage:
"""Parse ``list_memories(layers=['l1_raw'])`` into the App raw DTO."""
def from_sdk(cls, raw: object, *, limit: int, offset: int) -> HySdkRawPage:
if not isinstance(raw, Mapping):
raise HySdkReadError("hy_memory_raw_list_shape_unknown")
bucket = raw.get("vdb")
if not isinstance(bucket, Mapping):
raise HySdkReadError("hy_memory_raw_list_shape_unknown")
rows = bucket.get("memories")
total = bucket.get("total")
rows = raw.get("raw")
total = raw.get("total")
if (
not isinstance(rows, list)
or not isinstance(total, int)
@@ -471,13 +212,8 @@ class HySdkRawPage(BaseModel):
or total < 0
):
raise HySdkReadError("hy_memory_raw_list_shape_unknown")
converted = []
for row in rows:
if not isinstance(row, Mapping) or row.get("layer") != "l1_raw":
raise HySdkReadError("hy_memory_raw_list_shape_unknown")
converted.append({**row, "raw_id": row.get("memory_id")})
return cls(
raw=tuple(HySdkRawRecord.from_sdk(row) for row in converted),
raw=tuple(HySdkRawRecord.from_sdk(row) for row in rows),
total=total,
limit=limit,
offset=offset,
@@ -538,79 +274,6 @@ def parse_sdk_search(raw: object, *, error_code: str) -> tuple[SemanticMemoryHit
return tuple(found)
class LibraryHit(NamedTuple):
"""One keyword match in a library search window, with its namespace."""
memory_id: str
kind: MemoryKindName
variant: MemoryVariantName | None
def library_keyword_hits(raw: object, query: str) -> tuple[tuple[LibraryHit, ...], int]:
"""Memories in an SDK search window whose text contains every query term.
The local SDK store has no keyword channel (Chroma's keyword search is
empty), so search is pure vector similarity: a word in the library can
score below the floor, and a word in none of it still scores above. The
library search asks the SDK for its most relevant window and keeps only the
rows that actually contain the words, most relevant first. Returns those
hits and how many distinct rows the window held.
A ``search(scene="all")`` window mixes chat memories with working/coding
experiences and facts; their text is the unified ``content`` (an
experience's task and solution, a fact's statement). The fused order is
kept as the SDK ranked it.
"""
terms = query.casefold().split()
if isinstance(raw, Mapping) and raw.get("scene") == "all":
rows = raw.get("memories")
if not isinstance(rows, list) or any(not isinstance(row, Mapping) for row in rows):
raise HySdkReadError("hy_memory_search_shape_unknown")
ordered: list[Mapping[str, object]] = list(rows)
else:
ordered = sorted(
_reply_rows(raw, "hy_memory_search_shape_unknown"), key=_score, reverse=True
)
seen: set[LibraryHit] = set()
matched: list[LibraryHit] = []
for item in ordered:
hit = _library_hit(item)
if hit is None or hit in seen:
continue
seen.add(hit)
tags = item.get("tags")
parts = [item.get(key) for key in ("content", "task", "solution", "statement")] + list(
tags if isinstance(tags, list) else ()
)
text = " ".join(part for part in parts if isinstance(part, str)).casefold()
if all(term in text for term in terms):
matched.append(hit)
return tuple(matched), len(seen)
def _library_hit(item: Mapping[str, object]) -> LibraryHit | None:
kind = item.get("kind")
if kind in {"experience", "fact"}:
variant = item.get("variant") if item.get("variant") in {"working", "coding"} else None
variant = variant or (
item.get("scene") if item.get("scene") in {"working", "coding"} else None
)
memory_id = item.get("memory_id") or item.get(
"experience_id" if kind == "experience" else "fact_id"
)
if variant is None or not isinstance(memory_id, str) or not memory_id:
return None
return LibraryHit(
memory_id,
"experience" if kind == "experience" else "fact",
"coding" if variant == "coding" else "working",
)
memory_id = item.get("memory_id") or item.get("id")
if not isinstance(memory_id, str) or not memory_id:
return None
return LibraryHit(memory_id, "chat", None)
def parse_sdk_raw_search(raw: object) -> tuple[HySdkRawHit, ...]:
return tuple(
HySdkRawHit.from_sdk_search(row)
@@ -618,25 +281,6 @@ def parse_sdk_raw_search(raw: object) -> tuple[HySdkRawHit, ...]:
)
def _epoch(value: object) -> int | None:
"""Epoch seconds from an int or an ISO-8601 string (naive means UTC)."""
if isinstance(value, bool):
return None
if isinstance(value, int):
return value
if isinstance(value, float):
return int(value)
if isinstance(value, str) and value.strip():
try:
parsed = datetime.fromisoformat(value.strip())
except ValueError:
return None
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=UTC)
return int(parsed.timestamp())
return None
def _string_tuple(value: object) -> tuple[str, ...]:
if not isinstance(value, list):
return ()
@@ -50,10 +50,6 @@ class AuthorizedSourceFile:
size: int
modified_ns: int
changed_ns: int
# Source-specific change token (for example Cursor's WAL identity). It is
# cursor metadata only; modified_ns remains the real filesystem mtime used
# as a fallback conversation timestamp.
cursor_version: str | None = None
root_identity: str | None = None
requested_root: Path | None = None
access_check: Callable[[], None] | None = field(default=None, repr=False, compare=False)
@@ -557,10 +557,12 @@ class PollingLiveSyncSupervisor:
)
def _note_internal_error(self, source_id: str, error: Exception) -> None:
# The ordinary local log keeps the exception message for debugging.
# The structured diagnostic event deliberately keeps only the stable
# error code, source id, type, and relative stack: exception messages
# can contain paths, query text, or credentials and are exportable data.
# Type alone used to be all this said. It is not enough to act on:
# `OperationalError` covers a locked database, a missing table and a
# corrupt page alike, and without the message and the frames two dozen
# of these in a log could only be guessed at. The stack is the
# redacted, `one_memory`-relative one, so it names the call site
# without printing anyone's paths or query text.
logging.getLogger(__name__).warning(
"live_sync_internal_error %s: %s", type(error).__name__, error
)
@@ -570,7 +572,10 @@ class PollingLiveSyncSupervisor:
event="live_sync.internal_error",
outcome="failed",
error_code="live_sync_internal_error",
details={"source_id": source_id},
# `reason` is the listed key that can carry the message; see
# `diagnostic_log._DETAIL_KEYS`. Truncated because a detail value
# longer than 256 characters is rejected.
details={"source_id": source_id, "reason": str(error)[:200]},
**diagnostic_exception(error),
)
if isinstance(error, sqlite3.Error):
@@ -662,15 +662,11 @@ class ImportRunner:
and self.sdk_queue is not None
and event.cursor.read_to_end
):
# History ends at this end of file, so its last turn
# and a short batch are written. Live polling keeps
# them for the next increment or the idle flush.
self.sdk_queue.finalize_sdk_file(
user_id=current.user_id,
source_id=current.source_id,
file_key=event.cursor.file_key,
connection=connection,
flush=not self._live_import,
)
continue
if self.on_stable_evidence is not None and event.cursor.read_to_end:
@@ -871,7 +867,10 @@ class ImportRunner:
outcome="failed",
job_id=job_id,
error_code="import_heartbeat_failed",
details={"phase": "heartbeat"},
# `reason` is the listed key for this; see
# `diagnostic_log._DETAIL_KEYS`. Truncated: a detail value
# over 256 characters is rejected.
details={"reason": str(error)[:200]},
**diagnostic_exception(error),
)
@@ -644,7 +644,7 @@ def is_unchanged(source_file: AuthorizedSourceFile, cursors: dict[str, SourceCur
return (
previous is not None
and previous.read_to_end
and previous.file_identity == _cursor_file_identity(source_file)
and previous.file_identity == source_file.file_identity
and previous.observed_size == source_file.size
and previous.observed_modified_ns == source_file.modified_ns
# Chmod alone leaves size, mtime and inode untouched, so a file that
@@ -691,7 +691,7 @@ def touched_cursor(
previous = options.cursors.get(source_file.safe_id)
keep = (
previous is not None
and _same_file(previous, source_file)
and previous.file_identity == source_file.file_identity
and previous.observed_size <= source_file.size
)
# A kept cursor means the next scan resumes exactly where this one ended:
@@ -871,7 +871,7 @@ def resume_checkpoint(
previous = options.cursors.get(source_file.safe_id)
if previous is None:
return None
if not _same_file(previous, source_file):
if previous.file_identity != source_file.file_identity:
return None
if source_file.size < previous.observed_size:
return None
@@ -1046,7 +1046,7 @@ def source_cursor(
) -> SourceCursor:
return SourceCursor(
file_key=source_file.safe_id,
file_identity=_cursor_file_identity(source_file),
file_identity=source_file.file_identity,
checkpoint=checkpoint,
checkpoint_anchor=checkpoint_anchor,
observed_size=source_file.size,
@@ -1054,27 +1054,6 @@ def source_cursor(
)
def _cursor_file_identity(source_file: AuthorizedSourceFile) -> str:
return (
f"{source_file.file_identity}{_VERSION_SEPARATOR}{source_file.cursor_version}"
if source_file.cursor_version
else source_file.file_identity
)
_VERSION_SEPARATOR = ":v:"
def _same_file(previous: SourceCursor, source_file: AuthorizedSourceFile) -> bool:
"""Whether a stored cursor still points into this file.
The stored identity may carry a change token (Cursor's WAL version). The
token says the file changed, which `is_unchanged` handles; it does not mean
the file was replaced, so resuming compares the bare `dev:ino` only.
"""
return previous.file_identity.partition(_VERSION_SEPARATOR)[0] == source_file.file_identity
_POSITION = re.compile(r"(?<=:)\d+(?=:|$)")
@@ -533,8 +533,8 @@ def _with_write_ahead_log(database: AuthorizedSourceFile) -> AuthorizedSourceFil
version = hashlib.blake2b(
f"{database.modified_ns}:{journal.st_mtime_ns}:{journal.st_size}:{journal.st_ino}".encode(),
digest_size=8,
).hexdigest()
return replace(database, cursor_version=version)
).digest()
return replace(database, modified_ns=int.from_bytes(version, "big") & ((1 << 63) - 1))
def _live_baseline(listing: _CursorListing) -> dict[str, SourceCursor]:
@@ -1,113 +0,0 @@
"""Short-lived result windows for the memory library's keyword search.
A library search asks the SDK once for a bounded relevance window and keeps
the keyword matches. Paging must cut every page from that same list: running
the search again per page can rank differently and duplicate or skip memories
between pages. The matches are kept here under an opaque ``window_id`` that
the client passes back with the next offset.
A window is bound to the filters it was searched with (query, source,
statuses, scene). It expires after ``ttl_seconds`` without being read, and the
store keeps at most ``max_windows`` windows, dropping the least recently read
first.
Only memory identities are kept, never content: each page is read fresh from
the SDK, so an edit or a delete shows on the next page read.
"""
from __future__ import annotations
import secrets
import threading
import time
from collections import OrderedDict
from collections.abc import Callable
from dataclasses import dataclass, field
from one_memory.hy_sdk_types import LibraryHit
DEFAULT_WINDOW_TTL_SECONDS = 300.0
DEFAULT_MAX_WINDOWS = 32
WindowKey = tuple[str, str | None, tuple[str, ...], str]
def window_key(
query: str, source: str | None, statuses: list[str], scene: str = "all"
) -> WindowKey:
return (query, source, tuple(sorted(set(statuses))), scene)
@dataclass(frozen=True)
class SearchWindow:
window_id: str
key: WindowKey
hits: tuple[LibraryHit, ...]
# Set when the SDK window was full, so matches outside it were not looked
# for and `total` may undercount.
search_window: int | None
@dataclass
class _Entry:
window: SearchWindow
last_read: float = field(default=0.0)
class LibrarySearchWindows:
def __init__(
self,
*,
ttl_seconds: float = DEFAULT_WINDOW_TTL_SECONDS,
max_windows: int = DEFAULT_MAX_WINDOWS,
clock: Callable[[], float] = time.monotonic,
) -> None:
self._ttl = ttl_seconds
self._max = max(1, max_windows)
self._clock = clock
self._entries: OrderedDict[str, _Entry] = OrderedDict()
self._lock = threading.Lock()
def get(self, window_id: str, key: WindowKey) -> SearchWindow | None:
"""The window, if it is still live and was searched with `key`."""
with self._lock:
now = self._clock()
self._expire(now)
entry = self._entries.get(window_id)
if entry is None or entry.window.key != key:
return None
entry.last_read = now
self._entries.move_to_end(window_id)
return entry.window
def put(
self,
key: WindowKey,
hits: tuple[LibraryHit, ...],
search_window: int | None,
) -> SearchWindow:
window = SearchWindow(
window_id=secrets.token_urlsafe(12),
key=key,
hits=hits,
search_window=search_window,
)
with self._lock:
now = self._clock()
self._expire(now)
self._entries[window.window_id] = _Entry(window=window, last_read=now)
while len(self._entries) > self._max:
self._entries.popitem(last=False)
return window
def __len__(self) -> int:
with self._lock:
return len(self._entries)
def _expire(self, now: float) -> None:
stale = [
window_id
for window_id, entry in self._entries.items()
if now - entry.last_read >= self._ttl
]
for window_id in stale:
del self._entries[window_id]
+17 -46
View File
@@ -1,7 +1,6 @@
"""Write visible hook text into Evidence. Tool output stays out."""
import json
import os
import time
from datetime import UTC, datetime
from enum import StrEnum
@@ -75,13 +74,12 @@ _FILE_FALLBACK = {
}
_FALLBACK_EVENTS = frozenset({"Stop", "stop", "SubagentStop", "StopFailure"})
_TEXT_BLOCKS = frozenset({"text"})
SDK_MIGRATION_CAPTURE_FENCE = ".sdk-migration-capture-fence.json"
def database_path() -> Path:
from one_memory.config import load_settings
from one_memory.workbuddy_boundary import database_path as boundary_database
return load_settings().data_dir / "one-memory.sqlite3"
return boundary_database()
def capture_stdio(source_id: str, stdin: TextIO, output: TextIO) -> int:
@@ -90,49 +88,17 @@ def capture_stdio(source_id: str, stdin: TextIO, output: TextIO) -> int:
del output
if source_id not in CAPTURE_SOURCES:
return 2
database = database_path()
if sdk_migration_capture_fenced(database.parent):
return 0
try:
payload = json.loads(stdin.read() or "")
except json.JSONDecodeError:
return 0
try:
capture_hook(source_id, payload, database)
capture_hook(source_id, payload, database_path())
except Exception:
return 0
return 0
def sdk_migration_capture_fenced(data_dir: Path) -> bool:
"""Ignore stale fences; a live migration or a failed one pauses hook writes.
A failed upgrade migration leaves the fence `frozen` (no pid): the home was
put back as it was before the upgrade and takes no writes until the next
migration attempt, which replaces the fence and lifts it on success.
"""
marker = data_dir / SDK_MIGRATION_CAPTURE_FENCE
try:
document = json.loads(marker.read_text(encoding="utf-8"))
if isinstance(document, dict) and document.get("frozen") is True:
return True
pid = int(document["pid"])
if pid < 1:
raise ValueError("invalid migration fence pid")
os.kill(pid, 0)
return True
except FileNotFoundError:
return False
except ProcessLookupError:
marker.unlink(missing_ok=True)
return False
except PermissionError:
return True
except (KeyError, TypeError, ValueError, json.JSONDecodeError, OSError):
marker.unlink(missing_ok=True)
return False
def capture_hook(source_id: str, payload: object, database: Path) -> bool:
if source_id not in CAPTURE_SOURCES or not isinstance(payload, dict):
return False
@@ -256,19 +222,24 @@ def record_capture_rejection(
def _submit_live_memory(store: EvidenceStore, record: EvidenceRecord, *, changed: bool) -> None:
"""Add the record to its conversation's live assembling row.
The turn rule decides what is written (`turn_batches`): a turn is
complete when the next user message arrives, batches are 5 turns or
8000 characters, and Core writes the rest after 10 quiet minutes.
"""
if not changed or not _sync_open(store.database_path, record.source_id):
return
from one_memory.conversation_processing import _conversation_chunks
from one_memory.memory_server.durable_queue import DurableWriteQueue
from one_memory.memory_server.unified_entry import UnifiedWriteIntake
DurableWriteQueue(store.database_path, evidence=store).assemble_live_records(
(record,), file_key=f"hook:{record.conversation_id}"
)
intake = UnifiedWriteIntake(DurableWriteQueue(store.database_path, evidence=store))
for chunk in _conversation_chunks(
store.list_conversation(record.evidence_id),
max_turns=12,
completed_only=True,
):
if (
record in chunk
and any(item.role == "user" for item in chunk)
and any(item.role == "assistant" for item in chunk)
):
intake.submit_records(chunk, priority="live")
def _conversation(payload: dict[str, object]) -> str | None:
@@ -10,12 +10,16 @@ weights alone.
from __future__ import annotations
import threading
from dataclasses import dataclass
from hashlib import sha256
from importlib import import_module
from typing import TYPE_CHECKING, Any
from one_memory.embedding_assets import (
BUILTIN_EMBEDDING_DIMENSIONS,
BUILTIN_EMBEDDING_MAX_TOKENS,
EMBEDDING_ASSETS,
AssetVerificationError,
MissingEmbeddingAssetsError,
assets_present,
resolve_asset_dir,
@@ -30,6 +34,28 @@ _FETCH_HINT = (
)
@dataclass(frozen=True)
class TokenOffset:
"""Half-open Python Unicode code point offsets; synthetic specials use [0,0)."""
start_cp: int
end_cp: int
special: bool = False
@dataclass(frozen=True)
class TokenizedText:
"""Untruncated, unpadded encoding, including tokenizer postprocessor specials."""
token_ids: tuple[int, ...]
offsets: tuple[TokenOffset, ...]
def __post_init__(self) -> None:
# Defensive copies also keep runtime callers from retaining mutable lists.
object.__setattr__(self, "token_ids", tuple(self.token_ids))
object.__setattr__(self, "offsets", tuple(self.offsets))
class LocalEmbedder:
"""Encodes text with the built-in model, loading it on first use."""
@@ -42,6 +68,43 @@ class LocalEmbedder:
self._session: Any | None = None
self._tokenizer: Any | None = None
self._input_names: frozenset[str] = frozenset()
self._offset_lock = threading.Lock()
self._offset_tokenizer: Any | None = None
def tokenize_with_offsets(self, text: str) -> TokenizedText:
"""Tokenize original text without encode()'s truncation or batch padding.
The Python tokenizers binding returns code point offsets into the input,
even when its model normalizer changes spelling. This separate lazy
tokenizer uses the exact locked asset; it never loads an ONNX session or
mutates the tokenizer shared by encode(). Offsets are not UTF-16 indices.
"""
if not isinstance(text, str):
raise TypeError("embedding_offsets_require_text")
with self._offset_lock:
if self._offset_tokenizer is None:
directory = resolve_asset_dir()
path = directory / "tokenizer.json"
if not path.is_file():
raise MissingEmbeddingAssetsError(f"{_FETCH_HINT} Looked in {directory}.")
data = path.read_bytes()
expected = next(a.sha256 for a in EMBEDDING_ASSETS if a.filename == path.name)
if sha256(data).hexdigest() != expected:
raise AssetVerificationError("tokenizer.json: locked digest mismatch")
from tokenizers import Tokenizer
tokenizer = Tokenizer.from_str(data.decode("utf-8"))
tokenizer.no_truncation()
tokenizer.no_padding()
self._offset_tokenizer = tokenizer
encoding = self._offset_tokenizer.encode(text, add_special_tokens=True)
offsets = tuple(
TokenOffset(start, end, bool(special))
for (start, end), special in zip(
encoding.offsets, encoding.special_tokens_mask, strict=True
)
)
return TokenizedText(tuple(encoding.ids), offsets)
def _load(self) -> tuple[Any, Any]:
# Double-checked under a lock: memory processing encodes from worker
@@ -14,7 +14,6 @@ stuck in it.
"""
import logging
import re
import sys
import threading
import time
@@ -24,7 +23,6 @@ from types import FrameType
logger = logging.getLogger("one_memory.loop_watchdog")
DEFAULT_STALL_SECONDS = 2.0
_SAFE_FALLBACK_FILE = re.compile(r"^[A-Za-z0-9_][A-Za-z0-9_.-]*\.py$")
class LoopWatchdog:
@@ -122,28 +120,22 @@ def _safe_stack_for_main_thread() -> tuple[str, ...]:
def _safe_stacks_for_all_threads() -> tuple[str, ...]:
"""Return bounded stack summaries without recording thread names."""
"""Return one content-free, bounded stack summary for every live thread."""
names = {thread.ident: thread.name for thread in threading.enumerate()}
lines: list[str] = []
for _ident, frame in sorted(sys._current_frames().items()):
for ident, frame in sorted(sys._current_frames().items()):
stack: list[str] = []
# Most specific frame first; append only complete frames so truncation
# can never turn a valid location into free text.
# Most specific frame first so the 256-character privacy bound keeps
# the likely culprit rather than only the thread bootstrap frames.
for entry in reversed(traceback.extract_stack(frame)[-8:]):
parts = entry.filename.replace("\\", "/").split("/")
try:
root = parts.index("one_memory")
relative = "/".join(parts[root:])
except ValueError:
fallback = parts[-1]
relative = fallback if _SAFE_FALLBACK_FILE.fullmatch(fallback) else "unknown.py"
relative = parts[-1]
name = entry.name if entry.name.isidentifier() else "call"
candidate = f"{relative}:{entry.lineno}:{name}"
separator = "" if not stack else " > "
if len("thread|") + len(" > ".join(stack)) + len(separator) + len(candidate) > 256:
break
stack.append(candidate)
if stack:
lines.append(f"thread|{' > '.join(stack)}")
if len(lines) == 16:
break
stack.append(f"{relative}:{entry.lineno}:{name}")
thread_name = names.get(ident, "unknown").replace("\n", " ").replace("\r", " ")
lines.append(f"thread={thread_name[:48]}|{' > '.join(stack)}"[:256])
return tuple(lines)
@@ -0,0 +1,342 @@
"""Read-only local library projection over the existing governed memory store."""
import json
import os
import sqlite3
import time
from pathlib import Path
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field
from one_memory.database_access import main_database
from one_memory.legacy_lifecycle_types import HyLifecycle
from one_memory.memory_evidence_freshness import register_memory_freshness
from one_memory.memory_governance import (
GovernedMemory,
MemoryGovernanceStore,
MemoryStatus,
lifecycle_predicate,
)
#: Read budget for one library page. 2026-10-12: raised from 5s (which starved
#: under a concurrent history import and surfaced as 503) to 15s, still well
#: under the 30s client timeout; past it the read degrades to a `stale` page.
READ_BUDGET_SECONDS = 15
class MemorySourceTime(BaseModel):
"""When the conversation a memory came from happened (D96).
`basis` is `conversation` when the cited messages carry their own time,
`import` when only the import time is known; the UI words them apart.
"""
model_config = ConfigDict(frozen=True, extra="forbid")
at: str
basis: Literal["conversation", "import"]
class MemoryLibraryPage(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
items: list[GovernedMemory]
total: int = Field(ge=0)
offset: int = Field(ge=0)
limit: int = Field(ge=1, le=500)
source_ids_by_memory: dict[str, list[str]] = Field(default_factory=dict)
source_time_by_memory: dict[str, MemorySourceTime] = Field(default_factory=dict)
#: True when the read budget ran out mid-page (typically during a history
#: import): the counts and rows above are a partial snapshot, not an
#: authoritative view. Clients should show "importing" instead of an error.
stale: bool = False
def _sort_by_conversation() -> bool:
"""D99: the list reads as a timeline of the conversations, newest first.
A memory generated today from a three-week-old dialogue belongs next to
that week, not at the top. ONE_MEMORY_LIBRARY_SORT=updated brings back the
old "newest generated first" order without a code change.
"""
return os.environ.get("ONE_MEMORY_LIBRARY_SORT", "").strip().lower() != "updated"
def _origin_join() -> str:
"""The newest cited message per memory, as one joined aggregate.
A correlated subquery would re-scan the link table for every candidate row;
this is a single pass. Memories with no readable citation keep their old
position through COALESCE.
"""
if not _sort_by_conversation():
return ""
return """LEFT JOIN (
SELECT link.memory_id AS memory_id,
MAX(COALESCE(e.created_at, e.imported_at)) AS origin_at
FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.active = 1
GROUP BY link.memory_id
) origin ON origin.memory_id = m.memory_id"""
def _order_by() -> str:
if not _sort_by_conversation():
return "m.updated_at DESC, m.memory_id ASC"
# Conversations that share a timestamp keep the generated order between
# themselves; ties would otherwise fall back to the id and look arbitrary.
return "COALESCE(origin.origin_at, m.updated_at) DESC, m.updated_at DESC, m.memory_id ASC"
class _LibraryQuery(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
user_id: str = Field(min_length=1, max_length=256)
query: str | None = None
status: MemoryStatus | None = None
active_only: bool = False
source: str | None = Field(default=None, min_length=1)
offset: int = Field(default=0, ge=0, le=2**63 - 1, strict=True)
limit: int = Field(default=100, ge=1, le=500, strict=True)
def read_conversation_memories(
database_path: Path,
*,
user_id: str,
source_id: str,
conversation_id: str,
workspace: str | None,
limit: int = 100,
) -> list[GovernedMemory]:
"""Current memories backed by any active Evidence in one conversation."""
with main_database(database_path).read() as db:
db.row_factory = sqlite3.Row
freshness = register_memory_freshness(db)
rows = db.execute(
f"""
SELECT DISTINCT m.* FROM governed_memories m
JOIN governed_memory_evidence link ON link.memory_id=m.memory_id AND link.active=1
JOIN evidence e ON e.evidence_id=link.evidence_id
WHERE m.user_id=? AND m.status='confirmed' AND m.deleted_at IS NULL
AND e.user_id=? AND e.source_id=? AND e.conversation_id=?
AND e.workspace IS ? AND {freshness}
ORDER BY m.updated_at DESC, m.memory_id
LIMIT ?
""",
(user_id, user_id, source_id, conversation_id, workspace, limit),
).fetchall()
return MemoryGovernanceStore.memories_from_rows(db, rows)
def read_memory_library(
database_path: Path,
*,
user_id: str = "local",
query: str | None = None,
status: MemoryStatus | None = None,
active_only: bool = False,
source: str | None = None,
offset: int = 0,
limit: int = 100,
lifecycle: HyLifecycle | None = None,
) -> MemoryLibraryPage:
"""Read one page and its exact readable total from the same SQLite snapshot.
This is the local UI policy, not an external agent authorization entry point.
Missing grants do not revoke already stored content; explicit revocations do.
Read/parse failures propagate rather than masquerading as an empty library.
Which memories are current is HY's `lifecycle` when given, as in
`MemoryGovernanceStore.list`.
"""
request = _LibraryQuery(
user_id=user_id,
query=query,
status=status,
active_only=active_only,
source=source,
offset=offset,
limit=limit,
)
if request.active_only and request.status is not None:
raise ValueError("active_only_conflicts_with_status")
clauses = ["m.user_id = ?"]
values: list[str | int] = [request.user_id]
selected, selected_values = lifecycle_predicate(
lifecycle, MemoryStatus.CONFIRMED if request.active_only else request.status, alias="m"
)
if selected is not None:
clauses.append(selected)
values.extend(selected_values)
if request.status != MemoryStatus.DELETED:
clauses.append("m.deleted_at IS NULL")
# SourceReadAccess hides the whole memory if any active citation is revoked.
# Do this before COUNT/LIMIT; a readable citation cannot rescue a mixed memory.
clauses.append("""NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
JOIN source_grants g ON g.user_id = e.user_id AND g.source_id = e.source_id
WHERE link.memory_id = m.memory_id AND +link.active = 1
AND g.revoked_at IS NOT NULL
)""")
# Governance normally guarantees ownership and referential integrity. Fail
# closed for broken active links instead of exposing another user's citation.
clauses.append("""NOT EXISTS (
SELECT 1 FROM governed_memory_evidence link
LEFT JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.memory_id = m.memory_id AND +link.active = 1
AND (e.evidence_id IS NULL OR e.user_id != m.user_id)
)""")
if request.query:
clauses.append("m.content LIKE ? ESCAPE '\\'")
escaped = request.query.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
values.append(f"%{escaped}%")
if request.source is not None:
# Start from the compact active-link index, then resolve each Evidence
# identity by primary key. The old correlated EXISTS let SQLite scan
# every Evidence row for every candidate memory on large libraries.
clauses.append("""m.memory_id IN (
SELECT link.memory_id FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.active = 1 AND e.user_id = ? AND e.source_id = ?
)""")
values.extend((request.user_id, request.source))
predicate = " AND ".join(clauses)
def _interrupted(error: Exception) -> bool:
return isinstance(error, sqlite3.OperationalError) and "interrupted" in str(error).lower()
with main_database(database_path).read() as db:
# 2026-10-12: 5s starved the page whenever a history import saturated
# SQLite, and the caller turned the deadline abort into a 503, so the
# memory page read as broken for the whole import. 15s stays well under
# the 30s client timeout; past it we return a `stale` page instead of
# an error — an incomplete list is honest, an error is not.
deadline = time.monotonic() + READ_BUDGET_SECONDS
db.set_progress_handler(lambda: int(time.monotonic() >= deadline), 1000)
db.row_factory = sqlite3.Row
db.execute("BEGIN")
# The total counts with the cheap permission filters only: the per-row
# freshness UDF scales with the whole library and can burn the whole
# read budget on its own (D22). Product accepted a small drift here;
# the rows that are actually displayed are still checked row by row.
try:
total = db.execute(
f"SELECT COUNT(*) FROM governed_memories m WHERE {predicate}", values
).fetchone()[0]
except sqlite3.OperationalError as error:
if not _interrupted(error):
raise
return MemoryLibraryPage(
items=[],
total=0,
offset=request.offset,
limit=request.limit,
stale=True,
)
predicate += " AND " + register_memory_freshness(db)
try:
if request.query:
current = db.execute(
f"""SELECT m.* FROM governed_memories m {_origin_join()}
WHERE {predicate}
ORDER BY {_order_by()}""",
values,
).fetchall()
rows = current[request.offset : request.offset + request.limit]
else:
rows = db.execute(
f"""SELECT m.* FROM governed_memories m {_origin_join()}
WHERE {predicate}
ORDER BY {_order_by()} LIMIT ? OFFSET ?""",
[*values, request.limit, request.offset],
).fetchall()
except sqlite3.OperationalError as error:
if not _interrupted(error):
raise
# The count already succeeded: report it, flag the page stale.
return MemoryLibraryPage(
items=[],
total=total,
offset=request.offset,
limit=request.limit,
stale=True,
)
# Static read helper only: constructing the Store would initialize schema.
items = MemoryGovernanceStore.memories_from_rows(db, rows, lifecycle=lifecycle)
# Only the authorized page participates; source filters select memories,
# not a subset of their provenance. Keep this in the count/page snapshot.
source_ids_by_memory: dict[str, list[str]] = {item.memory_id: [] for item in items}
# D96: the list shows when the cited conversation happened, not when
# the memory was generated (which is "today" for a fresh import). Read
# in the same query as the sources, so the page costs no extra round
# trip and only this page's IDs are looked at, never the whole library.
conversation_at: dict[str, str] = {}
imported_at: dict[str, str] = {}
if items:
try:
for link in db.execute(
# D104: the conversation time is when the person spoke, not
# when the turn ended. A memory's evidence covers the whole
# round (question, tool calls, reply), so MAX() lands on the
# reply and reads as a later, contradictory time next to the
# cited source. Prefer the earliest person-side evidence; only
# fall back to the earliest of any kind when the round has no
# person-authored row at all.
"""SELECT link.memory_id, e.source_id,
COALESCE(
MIN(
CASE WHEN e.role = 'user'
AND e.role = 'user'
THEN e.created_at END
),
MIN(e.created_at)
) AS conversation_at,
MAX(e.imported_at) AS imported_at
FROM governed_memory_evidence link
JOIN evidence e ON e.evidence_id = link.evidence_id
WHERE link.active = 1 AND e.user_id = ?
AND link.memory_id IN (SELECT value FROM json_each(?))
GROUP BY link.memory_id, e.source_id
ORDER BY link.memory_id, e.source_id""",
(request.user_id, json.dumps(list(source_ids_by_memory))),
):
memory_id = link["memory_id"]
source_ids_by_memory[memory_id].append(link["source_id"])
for seen, value in (
(conversation_at, link["conversation_at"]),
(imported_at, link["imported_at"]),
):
if value and str(value) > seen.get(memory_id, ""):
seen[memory_id] = str(value)
except sqlite3.OperationalError as error:
if not _interrupted(error):
raise
# Page rows are already read; ship them without source maps
# rather than failing the whole request at the last step.
return MemoryLibraryPage(
items=items,
total=total,
offset=request.offset,
limit=request.limit,
source_ids_by_memory=source_ids_by_memory,
stale=True,
)
source_time_by_memory = {
memory_id: MemorySourceTime(at=conversation_at[memory_id], basis="conversation")
if memory_id in conversation_at
else MemorySourceTime(at=imported_at[memory_id], basis="import")
for memory_id in source_ids_by_memory
if memory_id in conversation_at or memory_id in imported_at
}
return MemoryLibraryPage(
items=items,
total=total,
offset=request.offset,
limit=request.limit,
source_ids_by_memory=source_ids_by_memory,
source_time_by_memory=source_time_by_memory,
)
@@ -0,0 +1,180 @@
"""Bounded, permission-aware receipt projection; never initialize a write store."""
import sqlite3
from pathlib import Path
from typing import Literal
from pydantic import BaseModel, ConfigDict
from one_memory.database_access import main_database
#: `legacy` is a confirmation recorded before memories stopped waiting for
#: one; it changed nothing that still exists, and `migrated_from` says so.
Operation = Literal["create", "correct", "set_scopes", "undo", "delete", "hy_reconcile", "legacy"]
ResultStatus = Literal["confirmed", "deleted"]
MigratedFrom = Literal["confirm", "reject"]
OPERATIONS = {"create", "correct", "set_scopes", "undo", "delete", "hy_reconcile", "legacy"}
STATUSES = {"confirmed", "deleted"}
#: Which operation each pre-v2 one now reads as.
MIGRATED_OPERATIONS = {"confirm": "legacy", "reject": "delete"}
#: A memory may move on to these after an earlier receipt was written; the
#: receipt still reports what that request committed, not the current status.
CURRENT_STATUSES = STATUSES | {"superseded"}
class MemoryMutationReceiptForbidden(RuntimeError):
def __init__(self) -> None:
super().__init__("memory_mutation_receipt_forbidden")
class MemoryMutationReceiptUnavailable(RuntimeError):
def __init__(self) -> None:
super().__init__("memory_mutation_receipt_unavailable")
class MemoryMutationReceipt(BaseModel):
model_config = ConfigDict(frozen=True, extra="forbid")
schema_version: Literal[2] = 2
request_id: str
state: Literal["not_found", "committed"]
operation: Operation | None = None
memory_id: str | None = None
result_revision: int | None = None
result_status: ResultStatus | None = None
memory_state: Literal["present", "deleted"] | None = None
#: The operation as it was stored before v2, when it has no v2 name of its own.
migrated_from: MigratedFrom | None = None
def _identity(value: str, maximum: int) -> bool:
return isinstance(value, str) and 0 < len(value) <= maximum
def _read_snapshot(
database_path: Path, request_id: str, user_id: str
) -> tuple[MemoryMutationReceipt, tuple[object, ...]]:
with main_database(database_path).read(row_factory=sqlite3.Row) as db:
db.execute("PRAGMA query_only=ON")
db.execute("BEGIN")
rows = db.execute(
"SELECT user_id,request_id,memory_id,operation,result_revision "
"FROM governed_memory_mutations WHERE user_id=? AND request_id=? LIMIT 2",
(user_id, request_id),
).fetchall()
if not rows:
return MemoryMutationReceipt(request_id=request_id, state="not_found"), ()
if len(rows) != 1:
raise MemoryMutationReceiptUnavailable()
mutation = rows[0]
memory_id, operation, revision = (
mutation["memory_id"],
mutation["operation"],
mutation["result_revision"],
)
if (
not _identity(memory_id, 512)
or operation not in OPERATIONS
or type(revision) is not int
or not 1 <= revision <= 2**53 - 1
):
raise MemoryMutationReceiptUnavailable()
current = db.execute(
"SELECT user_id,revision,status,deleted_at FROM governed_memories WHERE memory_id=?",
(memory_id,),
).fetchone()
if (
current is None
or current["user_id"] != user_id
or type(current["revision"]) is not int
or current["revision"] < revision
or current["status"] not in CURRENT_STATUSES
or (current["status"] == "deleted") != (current["deleted_at"] is not None)
):
raise MemoryMutationReceiptUnavailable()
# Versions have no historical evidence snapshot. Check every retained link,
# not only active ones, before disclosing even operation/ID metadata.
links = db.execute(
"SELECT l.evidence_id,l.active,e.user_id,e.source_id,e.revision,"
"g.granted_at,g.revoked_at FROM governed_memory_evidence l "
"LEFT JOIN evidence e ON e.evidence_id=l.evidence_id "
"LEFT JOIN source_grants g ON g.user_id=e.user_id AND g.source_id=e.source_id "
"WHERE l.memory_id=? ORDER BY l.evidence_id LIMIT 10001",
(memory_id,),
).fetchall()
if not links or len(links) > 10000:
raise MemoryMutationReceiptUnavailable()
for link in links:
if (
link["user_id"] != user_id
or link["source_id"] is None
or link["active"] not in (0, 1)
):
raise MemoryMutationReceiptUnavailable()
if link["revoked_at"] is not None:
raise MemoryMutationReceiptForbidden()
versions = db.execute(
"SELECT memory_id,revision,action,status,migrated_from FROM governed_memory_versions "
"WHERE memory_id=? AND revision=? LIMIT 2",
(memory_id, revision),
).fetchall()
if len(versions) != 1:
raise MemoryMutationReceiptUnavailable()
version = versions[0]
if version["action"] != operation or version["status"] not in STATUSES:
raise MemoryMutationReceiptUnavailable()
migrated_from = version["migrated_from"]
if migrated_from is not None and MIGRATED_OPERATIONS.get(migrated_from) != operation:
raise MemoryMutationReceiptUnavailable()
if (operation == "legacy") != (migrated_from == "confirm"):
raise MemoryMutationReceiptUnavailable()
required_status = {
"create": "confirmed",
"legacy": "confirmed",
"delete": "deleted",
}
if operation in required_status and version["status"] != required_status[operation]:
raise MemoryMutationReceiptUnavailable()
if (operation == "create") != (revision == 1):
raise MemoryMutationReceiptUnavailable()
if (
operation in {"correct", "set_scopes", "hy_reconcile", "undo"}
and version["status"] != "confirmed"
):
raise MemoryMutationReceiptUnavailable()
receipt = MemoryMutationReceipt(
request_id=request_id,
state="committed",
operation=operation,
memory_id=memory_id,
result_revision=revision,
result_status=version["status"],
memory_state="deleted" if current["status"] == "deleted" else "present",
migrated_from=migrated_from,
)
return receipt, (
tuple(mutation),
tuple(current),
tuple(version),
tuple(tuple(link) for link in links),
)
def read_memory_mutation_receipt(
database_path: Path, *, request_id: str, user_id: str = "local"
) -> MemoryMutationReceipt:
"""History only, never current authorization, HY cleanup or safe-retry proof."""
if not _identity(request_id, 256):
raise ValueError("invalid_memory_mutation_request_id")
if not _identity(user_id, 256):
raise ValueError("invalid_memory_mutation_user_id")
try:
first = _read_snapshot(database_path, request_id, user_id)
current = _read_snapshot(database_path, request_id, user_id)
if first != current:
raise MemoryMutationReceiptUnavailable()
return current[0]
except MemoryMutationReceiptForbidden:
raise
except (sqlite3.Error, ValueError, TypeError, KeyError, IndexError, OSError):
raise MemoryMutationReceiptUnavailable() from None
Loaded 100 of 447 files, more files were not shown because too many files have changed in this diff. Show more