Compare commits
64
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
32539e7a82 | ||
|
|
cfae020a31 | ||
|
|
5d51513ec8 | ||
|
|
9db5b49ccb | ||
|
|
cb7df2a022 | ||
|
|
812d1c3846 | ||
|
|
c8b660651e | ||
|
|
bdd2d9cf84 | ||
|
|
fc2e51a8cc | ||
|
|
d4fca0639e | ||
|
|
f7961f340a | ||
|
|
cfebae118d | ||
|
|
b169906fcb | ||
|
|
f3b417ced4 | ||
|
|
aaebe64244 | ||
|
|
f3d13c4351 | ||
|
|
3e8a006f65 | ||
|
|
a70830970d | ||
|
|
556d95df84 | ||
|
|
ad05303d23 | ||
|
|
423cb00dbf | ||
|
|
e09fbd16f9 | ||
|
|
dd3809a34b | ||
|
|
efe7df4048 | ||
|
|
072e0e5e90 | ||
|
|
a0221bc434 | ||
|
|
754555e3c3 | ||
|
|
b43c967baa | ||
|
|
dac2c15b3f | ||
|
|
0b775d7d61 | ||
|
|
4e217f845e | ||
|
|
2102a298d4 | ||
|
|
e540386db1 | ||
|
|
1374ac26b7 | ||
|
|
47dafbb102 | ||
|
|
49ad1b3086 | ||
|
|
3064515b43 | ||
|
|
e5c8cc3819 | ||
|
|
27f64f1b7a | ||
|
|
3c5451db24 | ||
|
|
a08f69a747 | ||
|
|
099f5ae7c0 | ||
|
|
e7625032d8 | ||
|
|
3aa3b5957d | ||
|
|
409ab9a9b4 | ||
|
|
c2f50dce06 | ||
|
|
2735d8693a | ||
|
|
47bbfb4b27 | ||
|
|
3f169c9951 | ||
|
|
24f699a726 | ||
|
|
cb97aa5822 | ||
|
|
38a4da6849 | ||
|
|
48a5d70026 | ||
|
|
b21c929728 | ||
|
|
b9b3c587ff | ||
|
|
5a5650fc94 | ||
|
|
0e149b6c9c | ||
|
|
0dd2bcd6b1 | ||
|
|
567a8b349d | ||
|
|
d89afb3887 | ||
|
|
0abf87d4b6 | ||
|
|
4e5a44b8dc | ||
|
|
bf36ba765b | ||
|
|
e9cce69650 |
No files matched your search
@@ -4,6 +4,21 @@ Hy Memory and MemHub are versioned and released independently.
|
||||
|
||||
## Unreleased
|
||||
|
||||
- MemHub shows when working and coding experiences were talked about. After
|
||||
the SDK upgrade every one of them read "时间未知" (time unknown) because the
|
||||
SDK never stored their conversation time; hy-memory 2.0.7 stores it and
|
||||
Core now passes it through the memory list, detail, search and home
|
||||
summary. An experience stored before 2.0.7 shows its write time, labelled
|
||||
"写入于" / "写入时间", instead of "时间未知". Metadata repair also writes
|
||||
the conversation time it recovers from the migration bundle into the SDK's
|
||||
coding/working stores when they already have the column.
|
||||
- The SDK migration no longer fails its first run on a home with legacy
|
||||
vectors: reattaching them left a Chroma instance open with its own settings,
|
||||
and the readback that followed could not open the same store
|
||||
("An instance of Chroma already exists ... with different settings").
|
||||
- MemHub "添加记忆" (add a memory) no longer waits behind unprocessed history.
|
||||
Notes are claimed ahead of the event-time window that keeps history strictly
|
||||
old to new; they still run one at a time and in order within their session.
|
||||
- The SDK memory page now reads its list, count, source and time labels and
|
||||
graph from one page response for one source filter, so rows, totals and the
|
||||
map no longer disagree. Memories with no recorded source are shown and
|
||||
@@ -61,6 +76,78 @@ Hy Memory and MemHub are versioned and released independently.
|
||||
`hy_node_permissions` proof table. Evidence, governed memories, model
|
||||
profiles, SDK stores, and source/host grants are preserved.
|
||||
|
||||
## Hy Memory 2.0.7 — 2026-10-28
|
||||
|
||||
- Published
|
||||
[`hy-memory==2.0.7`](https://pypi.org/project/hy-memory/2.0.7/).
|
||||
- Coding and working experiences and facts now keep their conversation time.
|
||||
`add(..., memory_at=...)` reached only the chat path before; the
|
||||
productivity path dropped it, so every experience had only its write time
|
||||
and MemHub showed them as "时间未知" (time unknown). `coding_memory_meta`
|
||||
and `working_fact` gain a nullable `memory_at` column (epoch seconds) in
|
||||
both the working and coding stores; existing databases get it on open, and
|
||||
their old rows stay `None`. Without `memory_at` a write uses the time the
|
||||
request arrived, as chat does. When an experience is updated by a later
|
||||
conversation it keeps the later of the two times.
|
||||
- `list_memories` (`scene="coding"`, `"working"` and `"all"`), `get` and
|
||||
`search` return `memory_at` for experiences and facts in the same field and
|
||||
format as chat (epoch seconds, `None` when unknown). Experience search
|
||||
results now also carry `created_at`.
|
||||
|
||||
## Hy Memory 2.0.6 — 2026-10-27
|
||||
|
||||
- Published
|
||||
[`hy-memory==2.0.6`](https://pypi.org/project/hy-memory/2.0.6/).
|
||||
- `search(..., rerank=False)` skips every reranking stage for that one
|
||||
call (the local cross-encoder, the memory-graph rerank, and the BM25 rerank of
|
||||
working/coding facts), so results come back in retrieval-score order. The
|
||||
flag travels on the read request, so concurrent searches do not affect each
|
||||
other. The default (`True`) keeps current behavior; `/api/v1/search` accepts
|
||||
it too.
|
||||
- MMR diversity selection now runs only when the requested `limit` is
|
||||
below 20 (`MMR_MAX_TOP_K`) and is skipped for larger limits, where its
|
||||
quadratic token-overlap pass is slow and gains little. Previously it ran for
|
||||
every limit. `MEMORY_HYBRID_V2_MMR_LAMBDA` still sets the weight.
|
||||
Near-duplicate demotion follows the same gate: it compares each candidate
|
||||
with every earlier non-duplicate, which is quadratic in a pool that grows
|
||||
with `limit`, and large result pages should show similar items anyway.
|
||||
- `search(scene="all")` searches chat, working and coding at the same time
|
||||
and merges them with Reciprocal Rank Fusion (k=60) over each scene's own
|
||||
ranking, so no scene is crowded out by the other scenes' score scales. Each
|
||||
result says which scene and kind it came from. A scene that fails or is turned
|
||||
off is reported in `errors` and does not fail the search. Source filters apply
|
||||
to all three.
|
||||
- `list_memories(scene="all")` lists chat, working and coding memories
|
||||
together, newest update first, with `limit`/`offset` paging over the merged
|
||||
order and `total` summing all three. `scene="coding"` and `"working"` now
|
||||
honor `include_sources` / `exclude_sources`, and `total` counts only the
|
||||
filtered memories.
|
||||
- `update()` and `delete()` now work on working/coding experiences and
|
||||
facts, not just chat memories. They take the same optional `kind` / `variant`
|
||||
as `get()`, and an ID found in more than one place raises
|
||||
`AmbiguousMemoryIdError` instead of guessing. Updating an experience replaces
|
||||
its solution and keeps its task, unless a new `task=` is also passed; updating
|
||||
a fact replaces its statement. Deleting removes the stored row and its vectors.
|
||||
- `list_memories(scene="chat" | "coding" | "working")` lists one kind of
|
||||
memory. `coding` and `working` list that scene's whole store, newest update
|
||||
first (`workspace_id` still narrows it), and `total` is now the real count
|
||||
rather than the page size. The new `memory_stats()` returns the count and the
|
||||
latest created/updated time for each scene and their total, without reading
|
||||
any memory text. Previously coding and working memories could not be listed
|
||||
or counted without a workspace, so a library of 45 memories listed 22.
|
||||
- Reconcile keeps time order. Both reconcile prompts now state that only
|
||||
`memory_at` decides which memory is newer, that an older memory must never
|
||||
supersede or overwrite a newer one, and that an unknown time is not newer;
|
||||
the Chinese prompt now lists `memory_at` among the new-memory fields. The
|
||||
reconciler also enforces this in code: a SUPERSEDE or UPDATE aimed at a memory
|
||||
with a later `memory_at` becomes an ADD, so the newer memory stays current.
|
||||
- The HTTP server returns 409 `ambiguous_memory_id` and 400 for a bad
|
||||
`kind`/`variant` instead of 500, passes `kind`/`variant` (and `task` on PUT)
|
||||
to get/update/delete, and no longer reads a PUT/DELETE query string as part
|
||||
of the memory id.
|
||||
- `memory_stats()` reads working/coding times, which are stored as naive UTC,
|
||||
as UTC rather than local time.
|
||||
|
||||
## Hy Memory 2.0.5 — 2026-10-22
|
||||
|
||||
- Published
|
||||
|
||||
@@ -743,6 +743,7 @@ RECONCILE_PROMPT_ZH = """你是一个记忆整合系统(Memory Reconciler)
|
||||
|
||||
* content
|
||||
* owner
|
||||
* memory_at(发生时间)
|
||||
* tags
|
||||
|
||||
new memories 没有 ID。
|
||||
@@ -791,6 +792,7 @@ new memories 没有 ID。
|
||||
* 无损合并
|
||||
* 保留所有有效信息
|
||||
* 不要产生重复 memory
|
||||
* 两者在某个值上不一致时,保留 memory_at 更晚的那个值;绝不用更早的 memory 覆盖更新的值
|
||||
|
||||
### 同一次事件的补充 vs 另一次发生
|
||||
|
||||
@@ -823,6 +825,7 @@ new memories 没有 ID。
|
||||
* 新 memory 与旧 memory 在同一维度冲突
|
||||
* 两者不能同时为真
|
||||
* 旧 memory 已不再代表当前状态
|
||||
* 新 memory 的 memory_at 不早于旧 memory 的 memory_at
|
||||
|
||||
例如:
|
||||
|
||||
@@ -837,6 +840,7 @@ new memories 没有 ID。
|
||||
* wording 差异
|
||||
* 同义表达
|
||||
* 可共存偏好
|
||||
* 新 memory 的 memory_at 早于旧 memory:它是更早的状态,绝不能取代更新的状态(见「时间规则」)
|
||||
|
||||
要求:
|
||||
|
||||
@@ -859,7 +863,7 @@ new memories 没有 ID。
|
||||
|
||||
### 把多条旧记忆折叠进同一条链
|
||||
|
||||
如果**两条或更多**已有记忆描述的是**同一维度**,而新记忆是该维度上的最新变化,
|
||||
如果**两条或更多**已有记忆描述的是**同一维度**,而新记忆是该维度上的最新变化(它的 memory_at 晚于所有被折叠的记忆),
|
||||
你可以把它们折叠进同一条演化链。此时用 `memory_ids`(有序列表,**旧→新**)
|
||||
代替单个 `memory_id`:
|
||||
|
||||
@@ -951,13 +955,16 @@ SUPERSEDE 不需要复制旧内容,因为旧节点仍存在。
|
||||
|
||||
## 时间规则
|
||||
|
||||
`memory_at: null`
|
||||
每条 memory(新的和已有的)都带 memory_at,表示它发生的时间。记忆**不一定按发生顺序写入**:
|
||||
历史可能晚导入,导入还没处理完时用户也可能手动添加记忆。所以一条 new memory 可能**比已有记忆更早**。
|
||||
|
||||
表示时间未知。
|
||||
|
||||
不要编造时间顺序。
|
||||
|
||||
不要仅因为“看起来更新”就 supersede。
|
||||
* 新旧**只看 memory_at**。"New memories" 只表示"正在写入",不表示"更近"。
|
||||
* 更早的 memory 绝不能取代更新的 memory:不能 SUPERSEDE 一条更新的记忆,也不能用 UPDATE 拿更早的值覆盖更新的值。
|
||||
* 如果 new memory 比与它冲突的已有记忆更早:保留已有记忆作为当前状态;把 new memory 以"过去状态"ADD
|
||||
(用过去时,并带上时间,例如"2023 年用户住在上海"),没有新信息就不输出。
|
||||
* `memory_at: null` 表示时间未知。未知**不算更新**:不要用 memory_at 为 null 的 memory 去 SUPERSEDE 一条有时间的记忆。
|
||||
* 不要编造时间顺序。
|
||||
* 不要仅因为“看起来更新”就 supersede。
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -221,6 +221,8 @@ Rules:
|
||||
* merge losslessly
|
||||
* preserve all useful information
|
||||
* do NOT create duplicates
|
||||
* if the two disagree on a value, keep the value from the memory with the later
|
||||
`memory_at`; never let an older memory overwrite a newer value
|
||||
|
||||
### Same occurrence vs. another occurrence
|
||||
|
||||
@@ -255,6 +257,7 @@ Use ONLY when:
|
||||
* the new memory contradicts an existing memory
|
||||
* both cannot be true at the same time
|
||||
* the old memory would now give the wrong current-state answer
|
||||
* the new memory's `memory_at` is the same as or LATER than the existing memory's `memory_at`
|
||||
|
||||
Examples:
|
||||
|
||||
@@ -268,6 +271,8 @@ Do NOT use for:
|
||||
* compatible preferences
|
||||
* wording differences
|
||||
* semantic equivalence
|
||||
* a new memory whose `memory_at` is EARLIER than the existing memory's — it is an
|
||||
older state and must never replace a newer one (see "Time order")
|
||||
|
||||
Rules:
|
||||
|
||||
@@ -289,7 +294,8 @@ Rules:
|
||||
### Folding several existing memories into one chain
|
||||
|
||||
If TWO OR MORE existing memories track the SAME dimension and the new memory is
|
||||
the latest change on that dimension, you may fold them into a single evolution
|
||||
the latest change on that dimension — its `memory_at` is later than every folded
|
||||
memory's `memory_at` — you may fold them into a single evolution
|
||||
chain. Use `memory_ids` (an ordered list, OLDEST → NEWEST) instead of a single
|
||||
`memory_id`:
|
||||
|
||||
@@ -318,8 +324,7 @@ older memories' text into it; they remain queryable in the chain history.
|
||||
* Separate dated occurrences stay separate ADDs; use `same_series` only for the
|
||||
same subject and same concrete activity/event type.
|
||||
* Keep each memory focused on one coherent topic.
|
||||
* Do not invent chronology.
|
||||
* `memory_at: null` means unknown time.
|
||||
* Follow "Time order" below.
|
||||
* **NEVER merge or supersede across different subjects.** The subject is whoever
|
||||
the content is about: "The user ..." vs "John ..." vs "Mary ...". Memories about
|
||||
different people are independent facts even when the sentence is otherwise
|
||||
@@ -332,6 +337,27 @@ older memories' text into it; they remain queryable in the chain history.
|
||||
|
||||
---
|
||||
|
||||
# Time order
|
||||
|
||||
Every memory, new or existing, has `memory_at`: when it happened. Memories are
|
||||
NOT always written in the order they happened — history can be imported late,
|
||||
and a memory can be added while older history is still being processed. A new
|
||||
memory can therefore be OLDER than an existing one.
|
||||
|
||||
* Decide which memory is newer ONLY by `memory_at`. "New memories" means "being
|
||||
written now", not "more recent".
|
||||
* An older memory must never replace a newer one: no SUPERSEDE of a newer
|
||||
memory, and no UPDATE that overwrites a newer value with an older one.
|
||||
* If a new memory is older than an existing memory it conflicts with, keep the
|
||||
existing memory as the current state. ADD the new one as a past state, worded
|
||||
in the past with its time (for example "In 2023 the user lived in Shanghai"),
|
||||
or output nothing if it adds no information.
|
||||
* `memory_at: null` means unknown time. Unknown is NOT newer: never SUPERSEDE a
|
||||
memory that has a known `memory_at` with one whose `memory_at` is null.
|
||||
* Do not invent chronology.
|
||||
|
||||
---
|
||||
|
||||
# Optional: `related`
|
||||
|
||||
On any op you MAY add a `related` array recording how the memory being written
|
||||
@@ -441,6 +467,81 @@ def append_reconcile_structure_prompt(
|
||||
return prompt + RECONCILE_STRUCTURE_METADATA_PROMPT_EN
|
||||
|
||||
|
||||
def _as_aware(value: Any) -> Optional[datetime]:
|
||||
"""memory_at as a comparable aware datetime; None when unknown or unreadable."""
|
||||
if isinstance(value, str) and value:
|
||||
try:
|
||||
value = datetime.fromisoformat(value)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
if not isinstance(value, datetime):
|
||||
return None
|
||||
# Stored nodes rebuild memory_at from epoch seconds as naive local time
|
||||
# (MemoryNode.from_dict), so a naive value is local, not UTC.
|
||||
return value if value.tzinfo is not None else value.astimezone()
|
||||
|
||||
|
||||
def enforce_time_order(
|
||||
ops: List["ReconcileOp"],
|
||||
new_memory_at: Any,
|
||||
existing: Dict[str, Any],
|
||||
) -> List["ReconcileOp"]:
|
||||
"""An older memory must never replace a newer one, whatever the model says.
|
||||
|
||||
Memories are not always written in the order they happened: history is
|
||||
imported late, and a person can add a memory while older history is still
|
||||
being processed. The prompt tells the model to compare `memory_at`; this
|
||||
makes it hold. A SUPERSEDE or UPDATE aimed at an existing memory whose
|
||||
`memory_at` is later than the memory being written is turned into an ADD,
|
||||
so the newer memory stays the current state. Unknown times are not
|
||||
compared. A content-less UPDATE (the legacy DELETE mapping) is dropped.
|
||||
"""
|
||||
written_at = _as_aware(new_memory_at)
|
||||
if written_at is None:
|
||||
return ops
|
||||
kept: List[ReconcileOp] = []
|
||||
for op in ops:
|
||||
if op.op not in ("SUPERSEDE", "UPDATE"):
|
||||
kept.append(op)
|
||||
continue
|
||||
targets = list(op.memory_ids) if op.memory_ids else [op.memory_id]
|
||||
newer = [
|
||||
target for target in targets
|
||||
if target and (node := existing.get(target)) is not None
|
||||
and (target_at := _as_aware(getattr(node, "memory_at", None))) is not None
|
||||
and target_at > written_at
|
||||
]
|
||||
if not newer:
|
||||
kept.append(op)
|
||||
continue
|
||||
logger.warning(
|
||||
"[reconciler] %s blocked: memory_at %s is older than existing %s; "
|
||||
"kept the newer memory",
|
||||
op.op, written_at.isoformat(), ",".join(newer),
|
||||
)
|
||||
if not op.content:
|
||||
continue
|
||||
kept.append(ReconcileOp(
|
||||
op="ADD",
|
||||
content=op.content,
|
||||
layer=op.layer,
|
||||
owner=op.owner,
|
||||
tags=list(op.tags),
|
||||
tag_ners=list(op.tag_ners),
|
||||
topic_paths=list(op.topic_paths),
|
||||
event_facet=dict(op.event_facet),
|
||||
topic_assignments=list(op.topic_assignments),
|
||||
structure_references=list(op.structure_references),
|
||||
topic_paths_provided=op.topic_paths_provided,
|
||||
event_facet_provided=op.event_facet_provided,
|
||||
topic_assignments_provided=op.topic_assignments_provided,
|
||||
speculate=op.speculate,
|
||||
related=list(op.related),
|
||||
reason=f"time_order: older than {','.join(newer)}; {op.reason}"[:500],
|
||||
))
|
||||
return kept
|
||||
|
||||
|
||||
def build_reconcile_prompt(
|
||||
*,
|
||||
new_memories: List[str],
|
||||
@@ -1399,6 +1500,7 @@ class MemoryReconciler:
|
||||
_tok_total += response.tokens_used
|
||||
raw_response = response.content
|
||||
ops = self._parse_ops(raw_response)
|
||||
ops = enforce_time_order(ops, current_time, candidate_map)
|
||||
ops = self._inherit_exact_input_structure(
|
||||
ops, new_memories_with_meta,
|
||||
)
|
||||
|
||||
+1062
-78
File diff suppressed because it is too large.
Load diff
@@ -454,9 +454,10 @@ class WorkingFactReconciler:
|
||||
workspace_id: Optional[str] = None,
|
||||
branch: Optional[str] = None,
|
||||
source: str = "default",
|
||||
memory_at: Optional[datetime] = None,
|
||||
) -> Tuple[List[WorkingFact], List[FactReconcileOp]]:
|
||||
"""
|
||||
执行 reconcile + 持久化。
|
||||
执行 reconcile + 持久化。memory_at 是这段对话的时间,记到新事实上。
|
||||
|
||||
Returns:
|
||||
(真正落库的新事实, 执行过的 ops)。LLM 挂掉时退化为全 INSERT 不覆盖,
|
||||
@@ -493,6 +494,7 @@ class WorkingFactReconciler:
|
||||
workspace_id=workspace_id,
|
||||
branch=branch,
|
||||
source=source,
|
||||
memory_at=memory_at,
|
||||
)
|
||||
logger.info(
|
||||
f"[fact-reconcile] drafts={len(drafts)} inserted={len(facts)} "
|
||||
@@ -781,6 +783,7 @@ class WorkingFactReconciler:
|
||||
workspace_id: Optional[str],
|
||||
branch: Optional[str] = None,
|
||||
source: str = "default",
|
||||
memory_at: Optional[datetime] = None,
|
||||
) -> Tuple[List[WorkingFact], List[FactReconcileOp]]:
|
||||
inserted: List[WorkingFact] = []
|
||||
ops: List[FactReconcileOp] = []
|
||||
@@ -801,6 +804,7 @@ class WorkingFactReconciler:
|
||||
subject=draft.subject,
|
||||
details=draft.details,
|
||||
status="active",
|
||||
memory_at=memory_at,
|
||||
)
|
||||
|
||||
allowed = {c.fact_id for c in (candidates[idx] if idx < len(candidates) else [])}
|
||||
|
||||
@@ -577,6 +577,7 @@ class CodingMemoryReconciler:
|
||||
confidence=d.confidence,
|
||||
source=d.source,
|
||||
extraction_source=d.extraction_source,
|
||||
memory_at=d.memory_at,
|
||||
)
|
||||
|
||||
|
||||
|
||||
+203
-9
@@ -86,6 +86,7 @@ CREATE TABLE IF NOT EXISTS coding_memory_meta (
|
||||
confidence REAL NOT NULL DEFAULT 0.7,
|
||||
source TEXT NOT NULL DEFAULT 'default',
|
||||
extraction_source TEXT NOT NULL DEFAULT 'auto_extract',
|
||||
memory_at INTEGER,
|
||||
created_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL
|
||||
);
|
||||
@@ -109,6 +110,7 @@ CREATE TABLE IF NOT EXISTS working_fact (
|
||||
derived_experience_ids TEXT NOT NULL DEFAULT '[]',
|
||||
status TEXT NOT NULL DEFAULT 'active',
|
||||
fact_chain_summary TEXT NOT NULL DEFAULT '',
|
||||
memory_at INTEGER,
|
||||
created_at TEXT NOT NULL
|
||||
);
|
||||
-- 反查前驱(谁的 fact_link 指向我)走这个索引,写演变总结时要顺着链往回走
|
||||
@@ -164,6 +166,8 @@ _ADDED_COLUMNS = {
|
||||
"coding_memory_meta": {
|
||||
"source_fact_ids": "TEXT NOT NULL DEFAULT '[]'",
|
||||
"extraction_source": "TEXT NOT NULL DEFAULT 'auto_extract'",
|
||||
# 对话时间,秒级 epoch(同 chat)。老库里的行是 NULL:那时没存过。
|
||||
"memory_at": "INTEGER",
|
||||
},
|
||||
# 早期版本存过一列 fact_chain(机械累积的历代说法),已废弃:同样的信息顺着
|
||||
# fact_link 反查就有。老库里那一列留着不管,有默认值不影响插入。
|
||||
@@ -177,6 +181,7 @@ _ADDED_COLUMNS = {
|
||||
"subject": "TEXT NOT NULL DEFAULT ''",
|
||||
"details": "TEXT NOT NULL DEFAULT ''",
|
||||
"source": "TEXT NOT NULL DEFAULT 'default'",
|
||||
"memory_at": "INTEGER",
|
||||
},
|
||||
}
|
||||
|
||||
@@ -217,6 +222,21 @@ def _in_clause(column: str, values: Optional[List[str]]) -> Tuple[str, List[Any]
|
||||
return f"{column} IN ({placeholders})", list(values)
|
||||
|
||||
|
||||
def _source_where(
|
||||
include_sources: Optional[List[str]], exclude_sources: Optional[List[str]],
|
||||
) -> Tuple[str, List[Any]]:
|
||||
"""Caller-source filter as SQL (``1`` when unfiltered); exclusion wins."""
|
||||
clauses: List[str] = []
|
||||
params: List[Any] = []
|
||||
if include_sources:
|
||||
clauses.append(f"source IN ({','.join('?' for _ in include_sources)})")
|
||||
params.extend(include_sources)
|
||||
if exclude_sources:
|
||||
clauses.append(f"source NOT IN ({','.join('?' for _ in exclude_sources)})")
|
||||
params.extend(exclude_sources)
|
||||
return (" AND ".join(clauses) or "1"), params
|
||||
|
||||
|
||||
def _scope_matches(
|
||||
row_workspace_id: Optional[str],
|
||||
row_branch: Optional[str],
|
||||
@@ -275,6 +295,29 @@ def _parse_dt(s: Optional[str]) -> Optional[datetime]:
|
||||
return None
|
||||
|
||||
|
||||
def _from_epoch(value: Any) -> Optional[datetime]:
|
||||
"""memory_at 列(秒级 epoch)→ datetime;NULL 或坏值返回 None。"""
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
return datetime.fromtimestamp(int(value))
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _to_epoch(value: Optional[datetime]) -> Optional[int]:
|
||||
return int(value.timestamp()) if isinstance(value, datetime) else None
|
||||
|
||||
|
||||
def _later(a: Optional[datetime], b: Optional[datetime]) -> Optional[datetime]:
|
||||
"""两个对话时间取较晚的;一边缺就取另一边。"""
|
||||
if a is None:
|
||||
return b
|
||||
if b is None:
|
||||
return a
|
||||
return a if _to_epoch(a) >= _to_epoch(b) else b
|
||||
|
||||
|
||||
def _row_to_memory(row: sqlite3.Row) -> CodingMemory:
|
||||
"""SQLite row → CodingMemory"""
|
||||
return CodingMemory(
|
||||
@@ -293,6 +336,7 @@ def _row_to_memory(row: sqlite3.Row) -> CodingMemory:
|
||||
confidence=row["confidence"],
|
||||
source=row["source"],
|
||||
extraction_source=row["extraction_source"],
|
||||
memory_at=_from_epoch(row["memory_at"]),
|
||||
created_at=_parse_dt(row["created_at"]),
|
||||
updated_at=_parse_dt(row["updated_at"]),
|
||||
)
|
||||
@@ -319,6 +363,7 @@ def _row_to_fact(row: sqlite3.Row) -> WorkingFact:
|
||||
# 老库刚 ADD COLUMN 时可能取不到这一列,用 keys() 兜一下
|
||||
fact_chain_summary=(row["fact_chain_summary"] or ""
|
||||
if "fact_chain_summary" in row.keys() else ""),
|
||||
memory_at=_from_epoch(row["memory_at"]),
|
||||
created_at=_parse_dt(row["created_at"]),
|
||||
)
|
||||
|
||||
@@ -582,6 +627,8 @@ class CodingMemoryStore:
|
||||
memory.updated_at = datetime.utcnow()
|
||||
if memory.created_at is None:
|
||||
memory.created_at = prev.created_at
|
||||
# 合并后的经验讲的是两段对话,对话时间取较晚的那段
|
||||
memory.memory_at = _later(prev.memory_at, memory.memory_at)
|
||||
await _run_in_pool(self._upsert_sql_sync, memory)
|
||||
# 删旧 VDB
|
||||
await self._delete_vec_keys_by_count(memory.experience_id, prev_key_count)
|
||||
@@ -603,6 +650,86 @@ class CodingMemoryStore:
|
||||
logger.info(f"[coding-store] deleted experience_id={experience_id}")
|
||||
return True
|
||||
|
||||
async def edit_experience(
|
||||
self,
|
||||
experience_id: str,
|
||||
*,
|
||||
user_id: Optional[str] = None,
|
||||
task: Optional[str] = None,
|
||||
solution: Optional[str] = None,
|
||||
search_keys: Optional[List[str]] = None,
|
||||
) -> Optional[CodingMemory]:
|
||||
"""调用方手动改一条经验(client.update 用)。返回改后的经验;不存在返回 None。
|
||||
|
||||
与 update() 的区别:update 是写入链路的覆盖式重写,带写授权校验(同
|
||||
agent/session/workspace 才能改);这里是用户对已有条目的直接编辑,只动
|
||||
给定的字段,其余原样保留,因此不做授权校验。
|
||||
|
||||
向量行只嵌 task + search_keys(solution 不进向量),所以只改 solution 时
|
||||
不重嵌、不碰向量库;改了 task / search_keys 才删旧 key 行重写。
|
||||
updated_at 总是刷新。
|
||||
"""
|
||||
prev = await self.get_by_id(experience_id, user_id=user_id)
|
||||
if prev is None:
|
||||
return None
|
||||
updated = deepcopy(prev)
|
||||
if task is not None:
|
||||
updated.task = task
|
||||
if solution is not None:
|
||||
updated.solution = solution
|
||||
if search_keys is not None:
|
||||
updated.search_keys = list(search_keys)
|
||||
updated.updated_at = datetime.utcnow()
|
||||
await _run_in_pool(self._upsert_sql_sync, updated)
|
||||
if updated.task != prev.task or updated.search_keys != prev.search_keys:
|
||||
await self._delete_vec_keys_by_count(experience_id, 1 + len(prev.search_keys))
|
||||
await self._upsert_vec_keys(updated)
|
||||
logger.info(f"[coding-store] edited experience_id={experience_id}")
|
||||
return updated
|
||||
|
||||
async def edit_fact(
|
||||
self,
|
||||
fact_id: str,
|
||||
*,
|
||||
statement: str,
|
||||
user_id: Optional[str] = None,
|
||||
) -> Optional[WorkingFact]:
|
||||
"""调用方手动改一条事实的 statement。返回改后的事实;不存在返回 None。
|
||||
|
||||
statement 是事实层的检索文本,所以 active 事实会重嵌并覆盖它那一行向量
|
||||
(node_id 就是 fact_id,upsert 即覆盖)。superseded 事实的向量行早已撤掉,
|
||||
这里不把它加回去,否则会让过期结论重新占召回名额。
|
||||
"""
|
||||
prev = await self.get_fact(fact_id, user_id=user_id)
|
||||
if prev is None:
|
||||
return None
|
||||
updated = deepcopy(prev)
|
||||
updated.statement = statement
|
||||
await _run_in_pool(self._upsert_fact_sql_sync, updated)
|
||||
if updated.status == "active":
|
||||
await self._upsert_fact_vec(updated)
|
||||
logger.info(f"[coding-store] edited fact_id={fact_id}")
|
||||
return updated
|
||||
|
||||
async def delete_fact(self, fact_id: str, user_id: Optional[str] = None) -> bool:
|
||||
"""物理删除一条事实(SQLite 行 + 事实向量行),返回是否真有删到。
|
||||
|
||||
别处留下的引用(经验的 source_fact_ids、后继事实的 fact_link)不清:
|
||||
召回侧本来就按存在性过滤死链(get_many / get_facts 查不到自然掉队)。
|
||||
"""
|
||||
prev = await self.get_fact(fact_id, user_id=user_id)
|
||||
if prev is None:
|
||||
logger.info(f"[coding-store] delete fact skipped (not found): {fact_id}")
|
||||
return False
|
||||
if self._fact_vec is not None:
|
||||
try:
|
||||
await self._fact_vec.delete(fact_id)
|
||||
except Exception as e:
|
||||
logger.debug(f"[coding-store] delete fact vec {fact_id} ignored: {e}")
|
||||
await _run_in_pool(self._delete_fact_sql_sync, fact_id, prev.user_id)
|
||||
logger.info(f"[coding-store] deleted fact_id={fact_id}")
|
||||
return True
|
||||
|
||||
# ------------------------------------------------------------
|
||||
# 读取
|
||||
# ------------------------------------------------------------
|
||||
@@ -687,29 +814,84 @@ class CodingMemoryStore:
|
||||
branch: Optional[str] = None,
|
||||
limit: int = 100,
|
||||
offset: int = 0,
|
||||
include_sources: Optional[List[str]] = None,
|
||||
exclude_sources: Optional[List[str]] = None,
|
||||
) -> List["CodingMemory"]:
|
||||
"""按 workspace_id(+可选 branch)纯过滤列出该 user 的完整 coding memory。
|
||||
|
||||
作用域规则见 _scope_where(与 search 路径共用同一套)。无语义排序,按
|
||||
updated_at 倒序(最近修改优先)。
|
||||
updated_at 倒序(最近修改优先)。include_sources / exclude_sources 按
|
||||
source 列在 SQL 里过滤(排除优先),所以 LIMIT/OFFSET 在过滤之后生效。
|
||||
"""
|
||||
return await _run_in_pool(
|
||||
self._list_by_workspace_sync, user_id, workspace_id, branch, limit, offset
|
||||
self._list_by_workspace_sync, user_id, workspace_id, branch, limit, offset,
|
||||
include_sources, exclude_sources,
|
||||
)
|
||||
|
||||
def _list_by_workspace_sync(
|
||||
self, user_id: str, workspace_id: str, branch: Optional[str],
|
||||
limit: int, offset: int,
|
||||
include_sources: Optional[List[str]] = None,
|
||||
exclude_sources: Optional[List[str]] = None,
|
||||
) -> List["CodingMemory"]:
|
||||
scope_sql, scope_params = _scope_where(workspace_id, branch)
|
||||
source_sql, source_params = _source_where(include_sources, exclude_sources)
|
||||
with self._lock:
|
||||
assert self._conn is not None
|
||||
sql = (f"SELECT * FROM coding_memory_meta WHERE user_id = ? AND {scope_sql} "
|
||||
f"ORDER BY updated_at DESC LIMIT ? OFFSET ?")
|
||||
params: List[Any] = [user_id] + scope_params + [limit, offset]
|
||||
f"AND {source_sql} "
|
||||
f"ORDER BY updated_at DESC, experience_id ASC LIMIT ? OFFSET ?")
|
||||
params: List[Any] = ([user_id] + scope_params + source_params
|
||||
+ [limit, offset])
|
||||
cur = self._conn.execute(sql, params)
|
||||
return [_row_to_memory(r) for r in cur.fetchall()]
|
||||
|
||||
async def stats(
|
||||
self,
|
||||
user_id: str,
|
||||
workspace_id: Optional[str] = None,
|
||||
branch: Optional[str] = None,
|
||||
include_sources: Optional[List[str]] = None,
|
||||
exclude_sources: Optional[List[str]] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""经验数、最新创建/修改时间、active 事实数。纯聚合,不读正文。
|
||||
|
||||
作用域与 list_by_workspace 相同:workspace_id 为空表示该 user 全部;
|
||||
来源过滤同样与 list_by_workspace 一致,count 因此恒等于分页可翻到的总数。
|
||||
"""
|
||||
return await _run_in_pool(
|
||||
self._stats_sync, user_id, workspace_id, branch,
|
||||
include_sources, exclude_sources,
|
||||
)
|
||||
|
||||
def _stats_sync(
|
||||
self, user_id: str, workspace_id: Optional[str], branch: Optional[str],
|
||||
include_sources: Optional[List[str]] = None,
|
||||
exclude_sources: Optional[List[str]] = None,
|
||||
) -> Dict[str, Any]:
|
||||
scope_sql, scope_params = _scope_where(workspace_id, branch)
|
||||
source_sql, source_params = _source_where(include_sources, exclude_sources)
|
||||
with self._lock:
|
||||
assert self._conn is not None
|
||||
row = self._conn.execute(
|
||||
f"SELECT COUNT(*) AS n, MAX(created_at) AS created, "
|
||||
f"MAX(updated_at) AS updated FROM coding_memory_meta "
|
||||
f"WHERE user_id = ? AND {scope_sql} AND {source_sql}",
|
||||
[user_id] + scope_params + source_params,
|
||||
).fetchone()
|
||||
facts = self._conn.execute(
|
||||
f"SELECT COUNT(*) AS n FROM working_fact "
|
||||
f"WHERE user_id = ? AND status = 'active' AND {scope_sql} "
|
||||
f"AND {source_sql}",
|
||||
[user_id] + scope_params + source_params,
|
||||
).fetchone()
|
||||
return {
|
||||
"count": int(row["n"] or 0),
|
||||
"latest_created_at": _parse_dt(row["created"]) if row["created"] else None,
|
||||
"latest_updated_at": _parse_dt(row["updated"]) if row["updated"] else None,
|
||||
"facts": int(facts["n"] or 0),
|
||||
}
|
||||
|
||||
# ------------------------------------------------------------
|
||||
# 召回
|
||||
# ------------------------------------------------------------
|
||||
@@ -1049,8 +1231,8 @@ class CodingMemoryStore:
|
||||
experience_id, user_id, agent_id, task, search_keys, solution,
|
||||
boundary_envs, workspace_id, branch, session_id,
|
||||
files, source_fact_ids, confidence, source, extraction_source,
|
||||
created_at, updated_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
memory_at, created_at, updated_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(experience_id) DO UPDATE SET
|
||||
user_id = excluded.user_id,
|
||||
agent_id = excluded.agent_id,
|
||||
@@ -1066,6 +1248,7 @@ class CodingMemoryStore:
|
||||
confidence = excluded.confidence,
|
||||
source = excluded.source,
|
||||
extraction_source = excluded.extraction_source,
|
||||
memory_at = excluded.memory_at,
|
||||
updated_at = excluded.updated_at
|
||||
""",
|
||||
(
|
||||
@@ -1076,6 +1259,7 @@ class CodingMemoryStore:
|
||||
json.dumps(memory.files, ensure_ascii=False),
|
||||
json.dumps(memory.source_fact_ids, ensure_ascii=False),
|
||||
memory.confidence, memory.source, memory.extraction_source,
|
||||
_to_epoch(memory.memory_at),
|
||||
(memory.created_at or datetime.utcnow()).isoformat(),
|
||||
(memory.updated_at or datetime.utcnow()).isoformat(),
|
||||
),
|
||||
@@ -1088,6 +1272,14 @@ class CodingMemoryStore:
|
||||
(experience_id, user_id),
|
||||
)
|
||||
|
||||
def _delete_fact_sql_sync(self, fact_id: str, user_id: str) -> None:
|
||||
with self._lock:
|
||||
assert self._conn is not None
|
||||
self._conn.execute(
|
||||
"DELETE FROM working_fact WHERE fact_id = ? AND user_id = ?",
|
||||
(fact_id, user_id),
|
||||
)
|
||||
|
||||
def _get_by_id_sync(
|
||||
self, experience_id: str, user_id: Optional[str]
|
||||
) -> Optional[CodingMemory]:
|
||||
@@ -1208,8 +1400,8 @@ class CodingMemoryStore:
|
||||
fact_id, user_id, agent_id, session_id, workspace_id, branch,
|
||||
source, statement, files, subject, details,
|
||||
fact_link, derived_experience_ids,
|
||||
status, fact_chain_summary, created_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
status, fact_chain_summary, memory_at, created_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(fact_id) DO UPDATE SET
|
||||
user_id = excluded.user_id,
|
||||
agent_id = excluded.agent_id,
|
||||
@@ -1224,7 +1416,8 @@ class CodingMemoryStore:
|
||||
fact_link = excluded.fact_link,
|
||||
derived_experience_ids = excluded.derived_experience_ids,
|
||||
status = excluded.status,
|
||||
fact_chain_summary = excluded.fact_chain_summary
|
||||
fact_chain_summary = excluded.fact_chain_summary,
|
||||
memory_at = excluded.memory_at
|
||||
""",
|
||||
(
|
||||
fact.fact_id, fact.user_id, fact.agent_id,
|
||||
@@ -1237,6 +1430,7 @@ class CodingMemoryStore:
|
||||
json.dumps(fact.derived_experience_ids, ensure_ascii=False),
|
||||
fact.status,
|
||||
fact.fact_chain_summary or "",
|
||||
_to_epoch(fact.memory_at),
|
||||
(fact.created_at or datetime.utcnow()).isoformat(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -25,6 +25,11 @@ FactStatus = Literal["active", "superseded"]
|
||||
MAX_FACT_CHAIN_ENTRIES = 8
|
||||
|
||||
|
||||
def _epoch(value: Optional[datetime]) -> Optional[int]:
|
||||
"""memory_at 对外是秒级 epoch,与 chat 一致。"""
|
||||
return int(value.timestamp()) if isinstance(value, datetime) else None
|
||||
|
||||
|
||||
# ================================================================
|
||||
# 草稿(LLM extract 阶段输出)
|
||||
# ================================================================
|
||||
@@ -53,6 +58,8 @@ class CodingMemoryDraft:
|
||||
files: List[str] = field(default_factory=list)
|
||||
source: str = "default"
|
||||
extraction_source: str = "auto_extract"
|
||||
# 对话时间(SDK 注入,同 chat 的 memory_at)
|
||||
memory_at: Optional[datetime] = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
from ..utils.source_filter import normalize_source
|
||||
@@ -73,6 +80,7 @@ class CodingMemoryDraft:
|
||||
"files": list(self.files),
|
||||
"source": self.source,
|
||||
"extraction_source": self.extraction_source,
|
||||
"memory_at": _epoch(self.memory_at),
|
||||
}
|
||||
|
||||
|
||||
@@ -111,6 +119,9 @@ class CodingMemory:
|
||||
extraction_source: str = "auto_extract"
|
||||
|
||||
# 时间
|
||||
# memory_at 是对话时间(同 chat),created_at / updated_at 是写库时间。
|
||||
# UPDATE 合并时取两者中较晚的那个对话时间,见 store.update。
|
||||
memory_at: Optional[datetime] = None
|
||||
created_at: Optional[datetime] = None
|
||||
updated_at: Optional[datetime] = None
|
||||
|
||||
@@ -135,6 +146,7 @@ class CodingMemory:
|
||||
"confidence": self.confidence,
|
||||
"source": self.source,
|
||||
"extraction_source": self.extraction_source,
|
||||
"memory_at": _epoch(self.memory_at),
|
||||
"created_at": self.created_at.isoformat() if self.created_at else None,
|
||||
"updated_at": self.updated_at.isoformat() if self.updated_at else None,
|
||||
}
|
||||
@@ -249,6 +261,7 @@ class WorkingFact:
|
||||
fact_chain_summary: str = ""
|
||||
|
||||
# 时间
|
||||
memory_at: Optional[datetime] = None # 对话时间(同 chat),没传时是收到请求的时间
|
||||
created_at: Optional[datetime] = None # 产生时间,新的优先
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
@@ -272,6 +285,7 @@ class WorkingFact:
|
||||
"derived_experience_ids": list(self.derived_experience_ids),
|
||||
"status": self.status,
|
||||
"fact_chain_summary": self.fact_chain_summary,
|
||||
"memory_at": _epoch(self.memory_at),
|
||||
"created_at": self.created_at.isoformat() if self.created_at else None,
|
||||
}
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@ CodingWriter - 写入端协调器
|
||||
import logging
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from ..agent.llm_provider import LLMProvider
|
||||
@@ -79,9 +80,11 @@ class CodingWriter:
|
||||
session_id: Optional[str] = None,
|
||||
request_id: Optional[str] = None,
|
||||
source: str = "default",
|
||||
memory_at: Optional[datetime] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
working/coding 段写入端到端。返回响应 dict(兼容 client.add 返回结构):
|
||||
working/coding 段写入端到端。memory_at 是对话时间,记到这次写出的经验和
|
||||
事实上;不传则用收到请求的时间(与 chat 链路一致)。返回响应 dict(兼容 client.add 返回结构):
|
||||
{
|
||||
"success": True,
|
||||
"scene": "productivity",
|
||||
@@ -101,6 +104,8 @@ class CodingWriter:
|
||||
"""
|
||||
request_id = request_id or str(uuid.uuid4())
|
||||
t0 = time.perf_counter()
|
||||
if memory_at is None:
|
||||
memory_at = datetime.now()
|
||||
|
||||
if not messages:
|
||||
return self._empty_response(request_id, t0)
|
||||
@@ -172,6 +177,7 @@ class CodingWriter:
|
||||
# extractor 只负责内容;调用来源由 SDK 入口注入,不能交给 LLM 生成。
|
||||
for draft in drafts:
|
||||
draft.source = source
|
||||
draft.memory_at = memory_at
|
||||
|
||||
# 3. 经验层 reconcile + 持久化
|
||||
ops: List[ReconcileOp] = (
|
||||
@@ -197,6 +203,7 @@ class CodingWriter:
|
||||
workspace_id=workspace_id,
|
||||
branch=branch,
|
||||
source=source,
|
||||
memory_at=memory_at,
|
||||
)
|
||||
|
||||
experience_ids = [op.target_experience_id for op in ops
|
||||
|
||||
@@ -160,6 +160,28 @@ def near_dup_demote(
|
||||
return out
|
||||
|
||||
|
||||
# MMR 只在调用方请求的 top-k(search 的 limit)小于这个值时自动启用。
|
||||
# mmr_select 的贪心是 O(k^2 * pool) 次 Jaccard,k 大时代价陡增,而大 k 本身已
|
||||
# 足够容纳多样性,收益很小。权重仍由 MEMORY_HYBRID_V2_MMR_LAMBDA 控制。
|
||||
MMR_MAX_TOP_K = 20
|
||||
|
||||
|
||||
def mmr_applies(top_k: int, lambda_: float) -> bool:
|
||||
"""本次检索是否走 MMR:λ 在有效区间内,且 top_k < MMR_MAX_TOP_K。"""
|
||||
return 0.0 < float(lambda_) < 0.999 and int(top_k) < MMR_MAX_TOP_K
|
||||
|
||||
|
||||
def near_dup_applies(top_k: int) -> bool:
|
||||
"""本次检索是否走近重复降权:与 MMR 同一个 top-k 门槛。
|
||||
|
||||
near_dup_demote 让每个候选和此前所有"非重复"候选两两算 Jaccard,候选
|
||||
互不相似时是 O(pool^2);池子随 limit 放大(约 max(3*limit, 30) + 关键词
|
||||
命中),limit 达到 MMR_MAX_TOP_K 时代价同样陡增。大 k 的调用方(如记忆
|
||||
面板翻页)要的是全部相关项,相似项也应照常出现,降权没有收益。
|
||||
"""
|
||||
return int(top_k) < MMR_MAX_TOP_K
|
||||
|
||||
|
||||
def mmr_select(
|
||||
hits: Sequence[Dict[str, Any]],
|
||||
limit: int,
|
||||
|
||||
@@ -401,6 +401,11 @@ class ReadRequest:
|
||||
intention_limit: int = 0 # Intention 路的 topk,默认 0 关闭
|
||||
|
||||
|
||||
# 重排开关(按次生效)。False 时跳过全部重排阶段:本地 cross-encoder
|
||||
# (recall.local_rerank_enabled) 与记忆关系图重排 (_apply_graph_rerank),
|
||||
# 结果按召回分排序。走请求对象而非环境变量,并发请求互不影响。
|
||||
rerank: bool = True
|
||||
|
||||
# 时间过滤
|
||||
created_after: Optional[float] = None # Unix timestamp (float),只返回 gmt_created >= 此值的记忆
|
||||
|
||||
|
||||
@@ -57,7 +57,9 @@ from ._retrieval.ner import (
|
||||
memory_matches_time_hints,
|
||||
)
|
||||
from ._retrieval.diversity import (
|
||||
near_dup_applies,
|
||||
near_dup_demote,
|
||||
mmr_applies,
|
||||
mmr_select,
|
||||
social_boilerplate_demote,
|
||||
pool_df_demote,
|
||||
@@ -1279,7 +1281,10 @@ class BaseReadPipeline(ReadPipeline):
|
||||
SAME_SERIES_TIMELINE_ON = bool(
|
||||
getattr(self.config.recall, "same_series_timeline_enabled", False)
|
||||
)
|
||||
LOCAL_RERANK_ON = bool(
|
||||
# 按次的重排开关(search(rerank=False)):关掉本地 cross-encoder 与图重排,
|
||||
# 结果按召回分排序。只读请求对象,不碰共享状态,并发请求互不影响。
|
||||
RERANK_ON = bool(getattr(request, "rerank", True))
|
||||
LOCAL_RERANK_ON = RERANK_ON and bool(
|
||||
getattr(self.config.recall, "local_rerank_enabled", False)
|
||||
)
|
||||
LOCAL_RERANK_POOL_FACTOR = max(1, int(
|
||||
@@ -1431,6 +1436,11 @@ class BaseReadPipeline(ReadPipeline):
|
||||
MIN_SCORE = _resolve_normal_min_score(request.min_score)
|
||||
|
||||
final_limit = request.limit if request.limit > 0 else 10
|
||||
# MMR 只在调用方的 top-k(final_limit,即 search 的 limit)< MMR_MAX_TOP_K
|
||||
# 时启用;池子(LOCAL_RERANK 下是 final_limit 的若干倍)不参与判定。
|
||||
MMR_ON = mmr_applies(final_limit, MMR_LAMBDA)
|
||||
# 近重复降权同样是候选两两比较,用同一个 top-k 门槛。
|
||||
NEAR_DUP_ON = near_dup_applies(final_limit)
|
||||
vdb_sem_limit = max(final_limit * 3, 30)
|
||||
vdb_kw_limit = max(final_limit * 5, 80)
|
||||
graph_limit = max(
|
||||
@@ -2462,7 +2472,7 @@ class BaseReadPipeline(ReadPipeline):
|
||||
# 只把已选出的 top-N 换个顺序收益很小。
|
||||
relation_paths: List[Dict[str, Any]] = []
|
||||
_graph_stats: Dict[str, Any] = {}
|
||||
if self._expansion_on():
|
||||
if RERANK_ON and self._expansion_on():
|
||||
_t_graph = datetime.now()
|
||||
_pool_before = len(normal_results)
|
||||
normal_results, relation_paths, _graph_stats = await self._apply_graph_rerank(
|
||||
@@ -2525,7 +2535,7 @@ class BaseReadPipeline(ReadPipeline):
|
||||
df_thr=POOL_DF_THR,
|
||||
penalty=POOL_DF_PENALTY,
|
||||
)
|
||||
if NEAR_DUP_THR > 0 and NEAR_DUP_PENALTY > 0:
|
||||
if NEAR_DUP_ON and NEAR_DUP_THR > 0 and NEAR_DUP_PENALTY > 0:
|
||||
normal_results = near_dup_demote(
|
||||
normal_results,
|
||||
jaccard_thr=NEAR_DUP_THR,
|
||||
@@ -2538,7 +2548,7 @@ class BaseReadPipeline(ReadPipeline):
|
||||
))
|
||||
if LOCAL_RERANK_ON else final_limit
|
||||
)
|
||||
if 0.0 < MMR_LAMBDA < 0.999 and len(normal_results) > _pre_rerank_limit:
|
||||
if MMR_ON and len(normal_results) > _pre_rerank_limit:
|
||||
normal_results = mmr_select(
|
||||
normal_results,
|
||||
_pre_rerank_limit,
|
||||
@@ -2715,7 +2725,7 @@ class BaseReadPipeline(ReadPipeline):
|
||||
reverse=True,
|
||||
)
|
||||
if (
|
||||
0.0 < MMR_LAMBDA < 0.999
|
||||
MMR_ON
|
||||
and len(multi_seed_pool) > _pre_rerank_limit
|
||||
):
|
||||
multi_seed_pool = mmr_select(
|
||||
@@ -3544,6 +3554,9 @@ class BaseReadPipeline(ReadPipeline):
|
||||
"episode_capsule_render_mode": EPISODE_CAPSULE_RENDER_MODE,
|
||||
"same_series_timeline_enabled": SAME_SERIES_TIMELINE_ON,
|
||||
"local_rerank_enabled": LOCAL_RERANK_ON,
|
||||
"rerank": RERANK_ON,
|
||||
"mmr_applied": MMR_ON,
|
||||
"near_dup_applied": NEAR_DUP_ON,
|
||||
"local_rerank_model": getattr(
|
||||
self.config.recall,
|
||||
"local_rerank_model",
|
||||
|
||||
+45
-7
@@ -33,9 +33,28 @@ import threading
|
||||
from datetime import datetime
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from typing import Any, Dict, Optional
|
||||
from urllib.parse import parse_qs, urlparse
|
||||
|
||||
from .retrieval import AmbiguousMemoryIdError
|
||||
|
||||
logger = logging.getLogger("hy_memory.server")
|
||||
|
||||
|
||||
def _namespace_args(values: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""kind / variant from a query string (lists) or a JSON body (scalars).
|
||||
|
||||
They disambiguate an id that exists in more than one namespace (chat,
|
||||
experience, fact; working or coding). Absent means "resolve it".
|
||||
"""
|
||||
args: Dict[str, Any] = {}
|
||||
for key in ("kind", "variant"):
|
||||
value = values.get(key)
|
||||
if isinstance(value, list):
|
||||
value = value[0] if value else None
|
||||
if value:
|
||||
args[key] = str(value)
|
||||
return args
|
||||
|
||||
# Lazy-init global client
|
||||
_client = None
|
||||
_client_lock = None
|
||||
@@ -148,7 +167,6 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
|
||||
if path == "/api/v1/metrics":
|
||||
minutes = 5
|
||||
if "?" in self.path:
|
||||
from urllib.parse import parse_qs, urlparse
|
||||
qs = parse_qs(urlparse(self.path).query)
|
||||
minutes = int(qs.get("minutes", ["5"])[0])
|
||||
client = _get_client()
|
||||
@@ -161,7 +179,8 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
|
||||
if m:
|
||||
memory_id = m.group(1)
|
||||
client = _get_client()
|
||||
result = client.get(memory_id)
|
||||
query = parse_qs(urlparse(self.path).query)
|
||||
result = client.get(memory_id, **_namespace_args(query))
|
||||
if result is None:
|
||||
_json_response(self, 404, {"error": "not_found", "memory_id": memory_id})
|
||||
else:
|
||||
@@ -170,6 +189,10 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
|
||||
|
||||
_json_response(self, 404, {"error": "not_found", "path": path})
|
||||
|
||||
except AmbiguousMemoryIdError as e:
|
||||
_json_response(self, 409, {"error": "ambiguous_memory_id", "detail": str(e)})
|
||||
except ValueError as e:
|
||||
_json_response(self, 400, {"error": str(e)})
|
||||
except Exception as e:
|
||||
logger.error(f"[server] GET {path} error: {e}", exc_info=True)
|
||||
_json_response(self, 500, {"error": str(e)})
|
||||
@@ -213,7 +236,7 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
|
||||
_json_response(self, 500, {"error": str(e)})
|
||||
|
||||
def do_PUT(self):
|
||||
path = self.path.rstrip("/")
|
||||
path = self.path.split("?")[0].rstrip("/")
|
||||
body = _read_json_body(self)
|
||||
if body is None:
|
||||
_json_response(self, 400, {"error": "invalid JSON body"})
|
||||
@@ -228,29 +251,43 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
|
||||
_json_response(self, 400, {"error": "content is required"})
|
||||
return
|
||||
client = _get_client()
|
||||
result = client.update(memory_id, content)
|
||||
result = client.update(
|
||||
memory_id,
|
||||
content,
|
||||
**_namespace_args(body),
|
||||
**({"task": body["task"]} if body.get("task") else {}),
|
||||
)
|
||||
_json_response(self, 200, result)
|
||||
return
|
||||
|
||||
_json_response(self, 404, {"error": "not_found", "path": path})
|
||||
|
||||
except AmbiguousMemoryIdError as e:
|
||||
_json_response(self, 409, {"error": "ambiguous_memory_id", "detail": str(e)})
|
||||
except ValueError as e:
|
||||
_json_response(self, 400, {"error": str(e)})
|
||||
except Exception as e:
|
||||
logger.error(f"[server] PUT {path} error: {e}", exc_info=True)
|
||||
_json_response(self, 500, {"error": str(e)})
|
||||
|
||||
def do_DELETE(self):
|
||||
path = self.path.rstrip("/")
|
||||
path = self.path.split("?")[0].rstrip("/")
|
||||
try:
|
||||
m = re.match(r"^/api/v1/memories/([^/]+)$", path)
|
||||
if m:
|
||||
memory_id = m.group(1)
|
||||
client = _get_client()
|
||||
result = client.delete(memory_id)
|
||||
query = parse_qs(urlparse(self.path).query)
|
||||
result = client.delete(memory_id, **_namespace_args(query))
|
||||
_json_response(self, 200, result)
|
||||
return
|
||||
|
||||
_json_response(self, 404, {"error": "not_found", "path": path})
|
||||
|
||||
except AmbiguousMemoryIdError as e:
|
||||
_json_response(self, 409, {"error": "ambiguous_memory_id", "detail": str(e)})
|
||||
except ValueError as e:
|
||||
_json_response(self, 400, {"error": str(e)})
|
||||
except Exception as e:
|
||||
logger.error(f"[server] DELETE {path} error: {e}", exc_info=True)
|
||||
_json_response(self, 500, {"error": str(e)})
|
||||
@@ -455,7 +492,8 @@ class MemoryHTTPHandler(BaseHTTPRequestHandler):
|
||||
if key in body:
|
||||
kwargs[key] = body[key]
|
||||
|
||||
for key in ("limit", "min_score", "schema_min_score", "schema_limit", "reader"):
|
||||
for key in ("limit", "min_score", "schema_min_score", "schema_limit", "reader",
|
||||
"rerank"):
|
||||
if key in body:
|
||||
kwargs[key] = body[key]
|
||||
|
||||
|
||||
@@ -172,6 +172,20 @@ result = client.add_extracted(
|
||||
该入口仍走 SDK reconcile/持久化;MemHub 只做管理面门禁与可选
|
||||
`memory_id` 记录。
|
||||
|
||||
应用的「添加记忆」、MCP `MemHub_write`(经 broker `POST /memories` 以本机可信身份转发)
|
||||
共用同一个本机 HTTP 合同,直接新增一条记忆:
|
||||
|
||||
```text
|
||||
POST /api/v2/memories {request_id, text, scope: "all-apps"}
|
||||
GET /api/v2/memories/additions/{request_id} # 该次新增的状态
|
||||
```
|
||||
|
||||
两者返回同一回执(`evidence_id`、`text`、`scope`、`created_at`、`state`、`task_id`、
|
||||
`memory_ids`、`error`)。同一 `request_id` 重放返回原回执;内容不同返回 409
|
||||
`memory_add_conflict`,未知 `request_id` 返回 404 `memory_add_not_found`。新增走持久
|
||||
写入队列的优先通道,最终调用 `add_extracted` + reconcile。带外部读取信封的请求
|
||||
一律拒绝(403 `external_endpoint_denied`)。
|
||||
|
||||
## Search:搜索记忆
|
||||
|
||||
```python
|
||||
|
||||
+5
-4
@@ -18,7 +18,7 @@ If you run into a problem, please open an Issue with the version, platform,repro
|
||||
|
||||
## 发行与平台合同
|
||||
|
||||
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.5`。MemHub 源码版本为 `1.0.5`
|
||||
源码当前固定使用公开 PyPI 正式版 `hy-memory==2.0.7`。MemHub 源码版本为 `1.0.5`
|
||||
(build 14),GitHub 仓库为
|
||||
[`Tencent-Hunyuan/Hy-Memory`](https://github.com/Tencent-Hunyuan/Hy-Memory),npm 包名为
|
||||
`memhub-cli`。当前代码架构见
|
||||
@@ -109,15 +109,16 @@ Node、Host Bridge 或模型。用户显式执行 `memhub install` 后,CLI 才
|
||||
Python Core 和运行所需的 Python 依赖位于 MemHub 受管 Runtime,不写入系统 Python。
|
||||
MemHub 的 SQLite 数据默认位于 `~/.one-memory`,不在 npm 包或 Runtime 版本目录中。
|
||||
`npm uninstall` 只移除 npm CLI;`memhub uninstall` 移除服务、受管 Runtime 和 CLI 链接,
|
||||
默认保留 Memory。只有独立的 `memhub delete-data` 在 Runtime 停止且二次确认精确路径后
|
||||
才会删除数据。
|
||||
默认保留 Memory。只有独立的 `memhub delete-data` 在 Runtime 停止、显式加 `--yes` 并用
|
||||
`--confirm <数据目录>` 再写一遍同一个目录后才会删除数据(不加参数只预览要删除的目录)。`memhub stop` 停止受管服务时也会关闭
|
||||
登录自动启动,再次 `memhub start` 会恢复。
|
||||
|
||||
当前平台能力和安装边界以源码、CI 与发布说明为准。未完成真机验收的平台不会仅凭跨平台
|
||||
测试宣称可用。
|
||||
|
||||
## Hy Memory
|
||||
|
||||
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.5`。SDK 配置、权限边界及 search
|
||||
SDK 依赖固定为公开 PyPI 正式版 `hy-memory==2.0.7`。SDK 配置、权限边界及 search
|
||||
返回格式见 [`MEMORY.md`](MEMORY.md)。HY search 返回的 `content` 已包含 SDK 统一生成的
|
||||
逻辑时间和演化上下文;调用方不应再次转换时间戳或重复拼接 `When`。
|
||||
|
||||
|
||||
@@ -2352,7 +2352,7 @@
|
||||
{
|
||||
"SPDXID": "SPDXRef-Package-hy-memory",
|
||||
"name": "hy-memory",
|
||||
"versionInfo": "2.0.5",
|
||||
"versionInfo": "2.0.7",
|
||||
"downloadLocation": "NOASSERTION",
|
||||
"filesAnalyzed": false,
|
||||
"licenseConcluded": "Apache-2.0",
|
||||
|
||||
@@ -9,7 +9,7 @@ Per-package license texts are not reproduced here yet; the license
|
||||
column is filled in as part of release review, and the SBOM records
|
||||
`NOASSERTION` until then rather than guessing.
|
||||
|
||||
## hy-memory 2.0.5
|
||||
## hy-memory 2.0.7
|
||||
|
||||
License: Apache-2.0
|
||||
|
||||
|
||||
@@ -21,18 +21,21 @@ const TOKEN = "probe-token-not-a-real-credential";
|
||||
|
||||
const MEMORY = {
|
||||
memory_id: "probe-1",
|
||||
user_id: "local",
|
||||
kind: "chat",
|
||||
status: "confirmed",
|
||||
content: "部署脚本要先跑 verify_merge_plan。",
|
||||
scopes: ["all-apps"],
|
||||
evidence_ids: ["probe-e1"],
|
||||
origin: "extracted",
|
||||
layer: "l2_fact",
|
||||
status: "active",
|
||||
is_latest: true,
|
||||
supersedes: [],
|
||||
superseded_by: [],
|
||||
tags: [],
|
||||
revision: 2,
|
||||
created_at: "2026-09-19T00:00:00Z",
|
||||
updated_at: "2026-09-19T00:00:00Z",
|
||||
deleted_at: null,
|
||||
owner: null,
|
||||
source_raw_ids: ["probe-raw-1"],
|
||||
user_id: "local",
|
||||
agent_id: "codex",
|
||||
session_id: "probe-session",
|
||||
source: "codex",
|
||||
memory_at: 1758240000,
|
||||
gmt_created: 1758240000,
|
||||
};
|
||||
|
||||
const ROUTES = new Map([
|
||||
@@ -124,26 +127,6 @@ const ROUTES = new Map([
|
||||
},
|
||||
],
|
||||
],
|
||||
[
|
||||
"/api/v1/memory-tasks",
|
||||
[
|
||||
{
|
||||
task_id: "probe-task",
|
||||
request_id: "probe-request",
|
||||
user_id: "local",
|
||||
kind: "extract",
|
||||
input_ref: "probe",
|
||||
status: "waiting_for_model",
|
||||
profile_id: "probe-profile",
|
||||
profile_revision: 1,
|
||||
profile_snapshot: {},
|
||||
attempts: 0,
|
||||
last_error_code: "model_missing",
|
||||
created_at: "2026-09-19T00:00:00Z",
|
||||
updated_at: "2026-09-19T00:00:01Z",
|
||||
},
|
||||
],
|
||||
],
|
||||
[
|
||||
"/api/v1/model-profiles",
|
||||
[
|
||||
@@ -159,28 +142,20 @@ const ROUTES = new Map([
|
||||
},
|
||||
],
|
||||
],
|
||||
["/api/v1/home-summary", { memory_count: 1, latest: null, observed_at: "T" }],
|
||||
[
|
||||
"/api/v1/memory-library",
|
||||
{ total: 1, offset: 0, limit: 100, items: [MEMORY] },
|
||||
],
|
||||
[
|
||||
"/api/v1/memories/probe-1",
|
||||
"/api/v2/home-summary",
|
||||
{
|
||||
memory: MEMORY,
|
||||
evidence: [
|
||||
{
|
||||
evidence_id: "probe-e1",
|
||||
available: true,
|
||||
source_id: "codex",
|
||||
written_by: "user",
|
||||
created_at: "2026-09-19T00:00:00Z",
|
||||
verbatim: "部署前先跑 verify_merge_plan,别靠读 diff 猜冲突。",
|
||||
},
|
||||
],
|
||||
history: [{ memory_id: "probe-1", revision: 2, action: "confirm" }],
|
||||
schema_version: 2,
|
||||
memory_count: 1,
|
||||
latest: MEMORY,
|
||||
observed_at: "2026-09-19T00:00:01Z",
|
||||
},
|
||||
],
|
||||
[
|
||||
"/api/v2/memory-library",
|
||||
{ memories: [MEMORY], total: 1, offset: 0, limit: 100 },
|
||||
],
|
||||
["/api/v2/memories/probe-1", MEMORY],
|
||||
]);
|
||||
|
||||
const seen = [];
|
||||
|
||||
@@ -30,9 +30,7 @@ const REQUIRED_COMMAND_SURFACE = [
|
||||
"sync",
|
||||
"memories",
|
||||
"profile",
|
||||
"calls",
|
||||
"notes",
|
||||
"graph",
|
||||
"graph-width",
|
||||
"models",
|
||||
"host",
|
||||
"doctor",
|
||||
@@ -50,19 +48,17 @@ const REQUIRED_COMMAND_SURFACE = [
|
||||
"uninstall",
|
||||
"delete-data",
|
||||
"recall",
|
||||
"setup",
|
||||
"scan",
|
||||
"progress",
|
||||
"jobs",
|
||||
"path",
|
||||
"scan-roots",
|
||||
"adapter",
|
||||
"write",
|
||||
"task",
|
||||
"serve",
|
||||
];
|
||||
|
||||
const REQUIRED_MARKERS = [
|
||||
"backup_restore_confirmation_required",
|
||||
"backup_restore_confirmation_mismatch",
|
||||
"backup_restore_runtime_state_unknown",
|
||||
"backup_restore_maintenance_identity_invalid",
|
||||
"runtime_service_activation_unverified",
|
||||
@@ -79,16 +75,18 @@ const REQUIRED_MARKERS = [
|
||||
* exact literal the catalog usage string contains.
|
||||
*/
|
||||
const REQUIRED_SUBCOMMANDS = [
|
||||
{ command: "calls", marker: "memhub calls clear" },
|
||||
{ command: "notes", marker: "memhub notes add" },
|
||||
{ command: "notes", marker: "memhub notes process" },
|
||||
{ command: "memories", marker: "memhub memories add" },
|
||||
{ command: "memories", marker: "memhub memories add-status" },
|
||||
{ command: "memories", marker: "memhub memories update" },
|
||||
{ command: "memories", marker: "memhub memories delete" },
|
||||
{ command: "write", marker: "memhub write submit" },
|
||||
{ command: "write", marker: "memhub write status" },
|
||||
{ command: "graph", marker: "memhub graph width" },
|
||||
{ command: "graph-width", marker: "memhub graph-width --decision" },
|
||||
{ command: "models", marker: "memhub models add" },
|
||||
{ command: "models", marker: "memhub models test" },
|
||||
{ command: "models", marker: "memhub models edit" },
|
||||
{ command: "models", marker: "memhub models activate" },
|
||||
{ command: "sources", marker: "memhub sources connect" },
|
||||
{ command: "jobs", marker: "memhub jobs start" },
|
||||
{ command: "jobs", marker: "memhub jobs <pause|resume|cancel>" },
|
||||
];
|
||||
@@ -100,57 +98,30 @@ const REQUIRED_SUBCOMMANDS = [
|
||||
* counterexamples on the packaged Runtime") had no machine-checkable probe.
|
||||
* These run without `--lifecycle`, so they are safe against any package.
|
||||
*/
|
||||
const PROFILE_ID = "44444444-4444-4444-8444-444444444444";
|
||||
const STOPPED_DESTRUCTIVE_PROBES = [
|
||||
{ name: "memories_add", args: ["memories", "add", "synthetic memory"] },
|
||||
{
|
||||
name: "calls_clear_preview",
|
||||
args: ["calls", "clear", "--event", "11111111-1111-4111-8111-111111111111"],
|
||||
},
|
||||
{
|
||||
name: "calls_clear_apply",
|
||||
args: [
|
||||
"calls",
|
||||
"clear",
|
||||
"--event",
|
||||
"11111111-1111-4111-8111-111111111111",
|
||||
"--confirm",
|
||||
],
|
||||
},
|
||||
{
|
||||
name: "calls_clear_all_apply",
|
||||
args: ["calls", "clear", "--all", "--confirm"],
|
||||
},
|
||||
{
|
||||
name: "notes_process_apply",
|
||||
args: [
|
||||
"notes",
|
||||
"process",
|
||||
"33333333-3333-4333-8333-333333333333",
|
||||
"--confirm",
|
||||
],
|
||||
name: "memories_delete_apply",
|
||||
args: ["memories", "delete", "m-1", "--yes"],
|
||||
},
|
||||
{
|
||||
name: "graph_width_apply",
|
||||
args: ["graph", "width", "--decision", "keep_existing", "--confirm"],
|
||||
args: ["graph-width", "--decision", "keep_existing"],
|
||||
},
|
||||
{
|
||||
name: "models_activate_apply",
|
||||
args: [
|
||||
"models",
|
||||
"activate",
|
||||
"44444444-4444-4444-8444-444444444444",
|
||||
"--confirm",
|
||||
],
|
||||
name: "graph_width_rebuild_apply",
|
||||
args: ["graph-width", "--decision", "rebuild_with_new", "--yes"],
|
||||
},
|
||||
{ name: "models_activate_apply", args: ["models", "activate", PROFILE_ID] },
|
||||
{ name: "models_test", args: ["models", "test", PROFILE_ID] },
|
||||
{
|
||||
name: "models_remove_apply",
|
||||
args: ["models", "remove", PROFILE_ID, "--yes"],
|
||||
},
|
||||
{
|
||||
name: "models_edit_apply",
|
||||
args: [
|
||||
"models",
|
||||
"edit",
|
||||
"44444444-4444-4444-8444-444444444444",
|
||||
"--config-file",
|
||||
"/nonexistent/patch.json",
|
||||
"--confirm",
|
||||
],
|
||||
name: "sources_connect_import",
|
||||
args: ["sources", "connect", "codex", "--import"],
|
||||
},
|
||||
];
|
||||
|
||||
|
||||
@@ -119,6 +119,7 @@ describe("A19 batch execution", () => {
|
||||
readerFor([]),
|
||||
operations,
|
||||
planned.notFound,
|
||||
{ confirmOverrides: false },
|
||||
);
|
||||
// Exactly one request for two sources — the batch endpoint, per contract.
|
||||
expect(post).toHaveBeenCalledTimes(1);
|
||||
@@ -165,6 +166,7 @@ describe("A19 batch execution", () => {
|
||||
readerFor([]),
|
||||
operations,
|
||||
planned.notFound,
|
||||
{ confirmOverrides: false },
|
||||
);
|
||||
expect(document.state).toBe("partial");
|
||||
const byId = Object.fromEntries(
|
||||
@@ -206,6 +208,7 @@ describe("A19 batch execution", () => {
|
||||
readerFor([]),
|
||||
operations,
|
||||
[],
|
||||
{ confirmOverrides: false },
|
||||
);
|
||||
expect(document.sources.every((row) => row.status === "failed")).toBe(true);
|
||||
expect(
|
||||
@@ -249,6 +252,7 @@ describe("A19 batch execution", () => {
|
||||
readerFor([]),
|
||||
operations,
|
||||
[],
|
||||
{ confirmOverrides: false },
|
||||
);
|
||||
const busy = document.sources.find((row) => row.source_id === "cursor");
|
||||
expect(document.state).toBe("partial");
|
||||
@@ -288,6 +292,7 @@ describe("A19 batch execution", () => {
|
||||
readerFor([]),
|
||||
operations,
|
||||
planned.notFound,
|
||||
{ confirmOverrides: false },
|
||||
);
|
||||
const notFoundRow = document.sources.find(
|
||||
(row) => row.source_id === "cursor",
|
||||
@@ -296,3 +301,82 @@ describe("A19 batch execution", () => {
|
||||
expect(document.not_found_source_ids).toContain("cursor");
|
||||
});
|
||||
});
|
||||
|
||||
describe("environment-override sources need --confirm-overrides", () => {
|
||||
it("leaves them out and names them unless confirmed", async () => {
|
||||
const planned = await planBatchImport(
|
||||
readerFor([
|
||||
found("codex"),
|
||||
found("dsh", {
|
||||
candidate_kind: "environment_override",
|
||||
requires_override_confirmation: true,
|
||||
}),
|
||||
]),
|
||||
);
|
||||
if (planned.state !== "preview") throw new Error("expected preview");
|
||||
const post = vi.fn(
|
||||
async (_path: string, body: Record<string, unknown>) => ({
|
||||
state: "committed" as const,
|
||||
data: {
|
||||
request_id: body.request_id,
|
||||
accepted_source_ids: body.source_ids,
|
||||
failed_sources: {},
|
||||
jobs: [],
|
||||
},
|
||||
}),
|
||||
);
|
||||
const operations = { post } as unknown as CoreOperations;
|
||||
const skipped = await executeBatchImport(
|
||||
planned.candidates,
|
||||
readerFor([]),
|
||||
operations,
|
||||
planned.notFound,
|
||||
{ confirmOverrides: false },
|
||||
);
|
||||
expect(post.mock.calls[0]![1]).toMatchObject({
|
||||
source_ids: ["codex"],
|
||||
confirm_environment_overrides: false,
|
||||
});
|
||||
expect(skipped.override_required_source_ids).toEqual(["dsh"]);
|
||||
expect(renderBatchScanText(skipped)).toContain("--confirm-overrides");
|
||||
|
||||
const confirmed = await executeBatchImport(
|
||||
planned.candidates,
|
||||
readerFor([]),
|
||||
operations,
|
||||
planned.notFound,
|
||||
{ confirmOverrides: true },
|
||||
);
|
||||
expect(post.mock.calls[1]![1]).toMatchObject({
|
||||
source_ids: ["codex", "dsh"],
|
||||
confirm_environment_overrides: true,
|
||||
});
|
||||
expect(confirmed.override_required_source_ids).toBeUndefined();
|
||||
});
|
||||
|
||||
it("refuses without a request when every detected source needs confirmation", async () => {
|
||||
const planned = await planBatchImport(
|
||||
readerFor([
|
||||
found("dsh", {
|
||||
candidate_kind: "environment_override",
|
||||
requires_override_confirmation: true,
|
||||
}),
|
||||
]),
|
||||
);
|
||||
if (planned.state !== "preview") throw new Error("expected preview");
|
||||
const post = vi.fn();
|
||||
const document = await executeBatchImport(
|
||||
planned.candidates,
|
||||
readerFor([]),
|
||||
{ post } as unknown as CoreOperations,
|
||||
planned.notFound,
|
||||
{ confirmOverrides: false },
|
||||
);
|
||||
expect(post).not.toHaveBeenCalled();
|
||||
expect(document).toMatchObject({
|
||||
state: "rejected",
|
||||
code: "environment_override_confirmation_required",
|
||||
});
|
||||
expect(batchScanExitCode(document)).toBe(1);
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,5 @@
|
||||
/** A19: one-shot scan/import of every *detected* source.
|
||||
/** A19: one-shot connect/import of every *detected* source
|
||||
* (`sources connect --all-detected --import`).
|
||||
*
|
||||
* The engine already takes many sources in a single `POST /api/v1/sources/connect`
|
||||
* (`source_ids` is a 1..32 tuple) and answers per source. The CLI used to reject
|
||||
@@ -37,13 +38,16 @@ export interface BatchSourceRow {
|
||||
export interface BatchScanDocument {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
command: "scan" | "init";
|
||||
command: "sources connect";
|
||||
state: "committed" | "partial" | "rejected" | "unknown" | "no_sources";
|
||||
request_id: string | null;
|
||||
accepted_source_ids: string[];
|
||||
sources: BatchSourceRow[];
|
||||
/** `location_state != found` rows are listed so the user can see what was skipped. */
|
||||
not_found_source_ids: string[];
|
||||
/** Detected sources whose location comes from an environment override and
|
||||
* that were left out because `--confirm-overrides` was not given. */
|
||||
override_required_source_ids?: string[];
|
||||
code?: string;
|
||||
}
|
||||
|
||||
@@ -118,12 +122,39 @@ export async function planBatchImport(reader: CoreReader): Promise<
|
||||
|
||||
/** Run the batch connect once and fold the per-source receipt into rows. */
|
||||
export async function executeBatchImport(
|
||||
candidates: readonly SourceConnectionCandidate[],
|
||||
detected: readonly SourceConnectionCandidate[],
|
||||
reader: CoreReader,
|
||||
operations: CoreOperations,
|
||||
notFound: readonly string[],
|
||||
options: { confirmOverrides: boolean },
|
||||
): Promise<BatchScanDocument> {
|
||||
const requestId = randomUUID();
|
||||
// An environment-override location is a choice the user has to confirm:
|
||||
// without `--confirm-overrides` those sources are left out and named.
|
||||
const skipped = options.confirmOverrides
|
||||
? []
|
||||
: detected
|
||||
.filter((candidate) => candidate.requires_override_confirmation)
|
||||
.map((candidate) => candidate.source_id);
|
||||
const overrideFields =
|
||||
skipped.length > 0 ? { override_required_source_ids: skipped } : {};
|
||||
const candidates = detected.filter(
|
||||
(candidate) => !skipped.includes(candidate.source_id),
|
||||
);
|
||||
if (candidates.length === 0) {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
command: "sources connect",
|
||||
state: "rejected",
|
||||
request_id: null,
|
||||
accepted_source_ids: [],
|
||||
sources: [],
|
||||
not_found_source_ids: [...notFound],
|
||||
...overrideFields,
|
||||
code: "environment_override_confirmation_required",
|
||||
};
|
||||
}
|
||||
// Same shared builder as the single-source import and the Web console, so a
|
||||
// batch connect cannot drift from either.
|
||||
const request = buildSourceImportRequest({
|
||||
@@ -141,9 +172,9 @@ export async function executeBatchImport(
|
||||
: [[candidate.source_id, candidate.candidate_revision]],
|
||||
),
|
||||
),
|
||||
confirmEnvironmentOverrides: candidates.some(
|
||||
(candidate) => candidate.requires_override_confirmation,
|
||||
),
|
||||
confirmEnvironmentOverrides:
|
||||
options.confirmOverrides &&
|
||||
candidates.some((candidate) => candidate.requires_override_confirmation),
|
||||
fullRescan: false,
|
||||
onboarding: true,
|
||||
enableSync: true,
|
||||
@@ -154,7 +185,7 @@ export async function executeBatchImport(
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
command: "scan",
|
||||
command: "sources connect",
|
||||
state: result.state,
|
||||
request_id: requestId,
|
||||
accepted_source_ids: [],
|
||||
@@ -168,6 +199,7 @@ export async function executeBatchImport(
|
||||
failure_reason: result.code,
|
||||
})),
|
||||
not_found_source_ids: [...notFound],
|
||||
...overrideFields,
|
||||
code: result.code,
|
||||
};
|
||||
}
|
||||
@@ -254,7 +286,7 @@ export async function executeBatchImport(
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
command: "scan",
|
||||
command: "sources connect",
|
||||
state:
|
||||
failedCount === 0
|
||||
? "committed"
|
||||
@@ -270,6 +302,7 @@ export async function executeBatchImport(
|
||||
accepted_source_ids: accepted,
|
||||
sources: rows,
|
||||
not_found_source_ids: [...notFound],
|
||||
...overrideFields,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -301,5 +334,10 @@ export function renderBatchScanText(document: BatchScanDocument): string {
|
||||
: document.state === "partial"
|
||||
? `部分完成:${document.accepted_source_ids.length} 个成功,其余见上方`
|
||||
: `未完成:${document.code ?? "unknown"}`;
|
||||
return `${summary}\n${lines.join("\n")}\n`;
|
||||
const overrides = document.override_required_source_ids ?? [];
|
||||
const hint =
|
||||
overrides.length > 0
|
||||
? `\n未接入(目录来自环境变量覆盖,需确认):${overrides.join("、")}\n确认目录无误后加 --confirm-overrides 重新运行。`
|
||||
: "";
|
||||
return `${summary}\n${lines.join("\n")}${hint}\n`;
|
||||
}
|
||||
@@ -0,0 +1,371 @@
|
||||
// Contract fixes from the CLI review: exit codes, where JSON goes, the global
|
||||
// `--data-dir`, and the write idempotency key.
|
||||
import { mkdtemp, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { run } from "./cli.js";
|
||||
import { COMMANDS } from "./command-catalog.js";
|
||||
|
||||
const transport = vi.hoisted(() => ({
|
||||
get: vi.fn(),
|
||||
post: vi.fn(),
|
||||
put: vi.fn(),
|
||||
patch: vi.fn(),
|
||||
delete: vi.fn(),
|
||||
request: vi.fn(),
|
||||
}));
|
||||
vi.mock("axios", () => ({
|
||||
default: {
|
||||
create: vi.fn(() => transport),
|
||||
get: transport.get,
|
||||
post: transport.post,
|
||||
request: transport.request,
|
||||
isAxiosError: () => false,
|
||||
},
|
||||
}));
|
||||
|
||||
const runtime = vi.hoisted(() => ({
|
||||
inspectRuntimeStatus: vi.fn(),
|
||||
discoverRuntimeAttachment: vi.fn(),
|
||||
inspectBackgroundHealth: vi.fn(),
|
||||
}));
|
||||
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
|
||||
...runtime,
|
||||
}));
|
||||
|
||||
const ATTACHED = {
|
||||
instanceId: "instance-1",
|
||||
apiToken: "synthetic-runtime-token",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
servicePid: 4242,
|
||||
health: {
|
||||
status: "ok",
|
||||
service: "one-memory-core",
|
||||
version: "1.2.3",
|
||||
instance_id: "instance-1",
|
||||
},
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
for (const method of Object.values(transport)) method.mockReset();
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "running",
|
||||
code: "runtime_verified",
|
||||
instanceId: "instance-1",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
version: "1.2.3",
|
||||
});
|
||||
runtime.discoverRuntimeAttachment.mockResolvedValue(ATTACHED);
|
||||
runtime.inspectBackgroundHealth.mockResolvedValue(null);
|
||||
});
|
||||
|
||||
describe("--data-dir is a global option", () => {
|
||||
it("points a Runtime-attached command at that data directory", async () => {
|
||||
transport.get.mockResolvedValue({
|
||||
status: 200,
|
||||
data: { request_id: "r", profile: null, elapsed_ms: 1 },
|
||||
});
|
||||
const result = await capturedRun(
|
||||
["profile", "--data-dir", "/synthetic/other", "--json"],
|
||||
{ ONE_MEMORY_DATA_DIR: "/synthetic/default" },
|
||||
);
|
||||
expect(result.code).toBe(0);
|
||||
expect(runtime.inspectRuntimeStatus).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ dataDir: "/synthetic/other" }),
|
||||
);
|
||||
});
|
||||
|
||||
it("is accepted by every command the catalog used to advertise it on", async () => {
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "stopped",
|
||||
code: "runtime_not_running",
|
||||
});
|
||||
runtime.discoverRuntimeAttachment.mockResolvedValue(null);
|
||||
for (const argv of [
|
||||
["profile"],
|
||||
["scan-roots", "list"],
|
||||
["adapter", "list"],
|
||||
["scan", "--source", "codex"],
|
||||
["write", "status", "r-1"],
|
||||
["backup", "create", "--target", "/synthetic/data/backups/a"],
|
||||
["memories", "list"],
|
||||
]) {
|
||||
const result = await capturedRun([
|
||||
...argv,
|
||||
"--data-dir",
|
||||
"/synthetic/other",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code, argv.join(" ")).toBe(3);
|
||||
}
|
||||
});
|
||||
|
||||
it("refuses a relative or repeated --data-dir", async () => {
|
||||
for (const argv of [
|
||||
["profile", "--data-dir", "relative"],
|
||||
["profile", "--data-dir", "/a", "--data-dir", "/b"],
|
||||
["profile", "--data-dir"],
|
||||
]) {
|
||||
expect((await capturedRun(argv)).code).toBe(2);
|
||||
}
|
||||
expect(runtime.inspectRuntimeStatus).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe("Core failures are exit 1, not usage errors", () => {
|
||||
it("adapter probe reports a Core refusal with its code on stdout", async () => {
|
||||
transport.request.mockResolvedValue({
|
||||
status: 409,
|
||||
data: { detail: "source_not_authorized" },
|
||||
});
|
||||
const result = await capturedRun([
|
||||
"adapter",
|
||||
"probe",
|
||||
"--source",
|
||||
"codex",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(result.stderr).toBe("");
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
command: "adapter probe",
|
||||
state: "failed",
|
||||
code: "source_not_authorized",
|
||||
exit_code: 1,
|
||||
});
|
||||
});
|
||||
|
||||
it("adapter probe prints readable text without --json", async () => {
|
||||
transport.request.mockResolvedValue({
|
||||
status: 200,
|
||||
data: { source: "codex", available: true },
|
||||
});
|
||||
const result = await capturedRun(["adapter", "probe", "--source", "codex"]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(result.stdout).toContain("来源 codex 的探测结果");
|
||||
expect(result.stdout).toContain("available: true");
|
||||
});
|
||||
|
||||
it("scan-roots add posts directly and reports a refusal as exit 1", async () => {
|
||||
transport.request.mockResolvedValue({
|
||||
status: 409,
|
||||
data: { detail: "scan_root_conflict" },
|
||||
});
|
||||
const result = await capturedRun([
|
||||
"scan-roots",
|
||||
"add",
|
||||
"--root",
|
||||
"/synthetic/root",
|
||||
"--source",
|
||||
"codex",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(transport.request).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
method: "post",
|
||||
url: "http://127.0.0.1:8765/api/v1/scan-roots",
|
||||
data: { root: "/synthetic/root", source: "codex", recursive: true },
|
||||
}),
|
||||
);
|
||||
expect(JSON.parse(result.stdout).code).toBe("scan_root_conflict");
|
||||
});
|
||||
|
||||
it("the old `path add` spelling still works", async () => {
|
||||
transport.request.mockResolvedValue({ status: 200, data: { ok: true } });
|
||||
const result = await capturedRun([
|
||||
"path",
|
||||
"remove",
|
||||
"--root",
|
||||
"/synthetic/root",
|
||||
"--yes",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.request.mock.calls[0][0]).toMatchObject({
|
||||
method: "delete",
|
||||
data: { root: "/synthetic/root" },
|
||||
});
|
||||
});
|
||||
|
||||
it("scan --source runs directly", async () => {
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: { job_id: "j1", source_id: "codex", status: "pending" },
|
||||
});
|
||||
const result = await capturedRun(["scan", "--source", "codex", "--json"]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
command: "scan",
|
||||
state: "committed",
|
||||
scan: { job_id: "j1" },
|
||||
});
|
||||
// A scan without a source is a usage error, not a preview.
|
||||
expect((await capturedRun(["scan"])).code).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe("write submit keeps its idempotency key", () => {
|
||||
async function messagesFile(): Promise<string> {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "memhub-write-"));
|
||||
const file = path.join(directory, "messages.json");
|
||||
await writeFile(file, JSON.stringify([{ role: "user", content: "hi" }]));
|
||||
return file;
|
||||
}
|
||||
|
||||
it("generates a request id, sends it and prints it with a retry hint", async () => {
|
||||
transport.post.mockImplementation(
|
||||
async (_url: string, body: { request_id: string }) => ({
|
||||
status: 200,
|
||||
data: {
|
||||
request_id: body.request_id,
|
||||
state: "accepted",
|
||||
created_at: "T",
|
||||
updated_at: "T",
|
||||
},
|
||||
}),
|
||||
);
|
||||
const result = await capturedRun([
|
||||
"write",
|
||||
"submit",
|
||||
"--session",
|
||||
"s1",
|
||||
"--messages-file",
|
||||
await messagesFile(),
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
const sent = transport.post.mock.calls[0][1].request_id as string;
|
||||
expect(sent).toMatch(/^[0-9a-f-]{36}$/);
|
||||
expect(result.stdout).toContain(`request_id: ${sent}`);
|
||||
expect(result.stdout).toContain(`--request-id ${sent}`);
|
||||
});
|
||||
|
||||
it("reports a 409 conflict as JSON on stdout with the request id", async () => {
|
||||
transport.post.mockResolvedValue({
|
||||
status: 409,
|
||||
data: { detail: "memory_write_conflict" },
|
||||
});
|
||||
const result = await capturedRun([
|
||||
"write",
|
||||
"submit",
|
||||
"--session",
|
||||
"s1",
|
||||
"--messages-file",
|
||||
await messagesFile(),
|
||||
"--request-id",
|
||||
"req-1",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(result.stderr).toBe("");
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
command: "write submit",
|
||||
state: "rejected",
|
||||
request_id: "req-1",
|
||||
code: "memory_write_conflict",
|
||||
});
|
||||
});
|
||||
|
||||
it("write status of an unknown id is exit 1 with state not_found", async () => {
|
||||
transport.get.mockResolvedValue({ status: 404, data: {} });
|
||||
const error = Object.assign(new Error("404"), {
|
||||
response: { status: 404 },
|
||||
});
|
||||
transport.get.mockRejectedValue(error);
|
||||
const result = await capturedRun(["task", "--id", "nope", "--json"]);
|
||||
// `axios.isAxiosError` is mocked false, so the read error surfaces as a
|
||||
// failed read; either way it is exit 1, never a silent exit 0.
|
||||
expect(result.code).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("doctor and ui use exit 3 for a stopped Runtime", () => {
|
||||
it("doctor exits 3 when the Runtime is stopped", async () => {
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "stopped",
|
||||
code: "runtime_not_running",
|
||||
});
|
||||
const result = await capturedRun(["doctor", "--json"]);
|
||||
expect(result.code).toBe(3);
|
||||
expect(JSON.parse(result.stdout).schema_version).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("catalog exit codes", () => {
|
||||
it("lists exit 2 for every command, since every parser can reject", () => {
|
||||
for (const command of COMMANDS) {
|
||||
expect(command.exit_codes, command.name).toContain(2);
|
||||
}
|
||||
});
|
||||
|
||||
it("lists exit 3 for every command that attaches to the Runtime", () => {
|
||||
for (const name of [
|
||||
"status",
|
||||
"doctor",
|
||||
"init",
|
||||
"sources",
|
||||
"sync",
|
||||
"progress",
|
||||
"jobs",
|
||||
"models",
|
||||
"memories",
|
||||
"profile",
|
||||
"graph-width",
|
||||
"host",
|
||||
"recall",
|
||||
"scan-roots",
|
||||
"adapter",
|
||||
"scan",
|
||||
"write",
|
||||
"backup",
|
||||
"ui",
|
||||
"onboarding",
|
||||
]) {
|
||||
const command = COMMANDS.find((entry) => entry.name === name);
|
||||
expect(command?.exit_codes, name).toContain(3);
|
||||
}
|
||||
});
|
||||
|
||||
it("does not claim exit 3 or 4 for lifecycle commands that never return them", () => {
|
||||
for (const name of ["start", "stop", "restart"]) {
|
||||
const command = COMMANDS.find((entry) => entry.name === name);
|
||||
expect(command?.exit_codes).toEqual([0, 1, 2]);
|
||||
}
|
||||
});
|
||||
|
||||
it("says stop turns login auto-start off for a managed service", () => {
|
||||
const stop = COMMANDS.find((entry) => entry.name === "stop");
|
||||
expect(stop?.summary).toContain("登录时也不会自动启动");
|
||||
});
|
||||
});
|
||||
|
||||
async function capturedRun(
|
||||
arguments_: string[],
|
||||
environment: NodeJS.ProcessEnv = { ONE_MEMORY_DATA_DIR: "/synthetic/data" },
|
||||
) {
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
const stdoutSpy = vi
|
||||
.spyOn(process.stdout, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stdout += String(chunk);
|
||||
return true;
|
||||
});
|
||||
const stderrSpy = vi
|
||||
.spyOn(process.stderr, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stderr += String(chunk);
|
||||
return true;
|
||||
});
|
||||
try {
|
||||
return { code: await run(arguments_, environment), stdout, stderr };
|
||||
} finally {
|
||||
stdoutSpy.mockRestore();
|
||||
stderrSpy.mockRestore();
|
||||
}
|
||||
}
|
||||
+621
-60
@@ -1,9 +1,18 @@
|
||||
import { mkdtemp, readdir, readFile, realpath, rm } from "node:fs/promises";
|
||||
import {
|
||||
mkdir,
|
||||
mkdtemp,
|
||||
readdir,
|
||||
readFile,
|
||||
realpath,
|
||||
rm,
|
||||
} from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import {
|
||||
claimRuntimeServiceForApp,
|
||||
clearSdkMigrationBlock,
|
||||
readRuntimeServiceOwner,
|
||||
recordSdkMigrationBlock,
|
||||
resolveRuntimeCliConfig,
|
||||
} from "@one-memory/runtime-control";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
@@ -790,7 +799,7 @@ describe("public One Memory CLI", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("upgrade failure restarts the restored artifact with the same data identity", async () => {
|
||||
it("upgrade restart failure keeps the new version and never restarts the old one", async () => {
|
||||
vi.clearAllMocks();
|
||||
runtimeMocks.planCliLink.mockResolvedValueOnce({
|
||||
schema_version: 1,
|
||||
@@ -832,25 +841,27 @@ describe("public One Memory CLI", () => {
|
||||
command: "upgrade",
|
||||
code: "restart failed",
|
||||
});
|
||||
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
|
||||
// The CLI manages the objective aliases the package does not own
|
||||
// (`memhub` + `memhub-cli`); both are rolled back.
|
||||
expect(aliasMocks.rollbackManagedCliLink).toHaveBeenCalledTimes(2);
|
||||
expect(runtimeMocks.restartUserService).toHaveBeenCalledTimes(2);
|
||||
for (const call of runtimeMocks.restartUserService.mock
|
||||
.calls as unknown[][]) {
|
||||
expect(call[0]).toMatchObject({
|
||||
dataDir: "/tmp/custom data",
|
||||
installRoot: "/tmp/install/current",
|
||||
});
|
||||
}
|
||||
expect(JSON.parse(streams.stderr())).toMatchObject({
|
||||
rolled_back: false,
|
||||
});
|
||||
// Product rule: the previous Runtime may predate the SDK migration gate,
|
||||
// so it is never restored after the migration was handed the home.
|
||||
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
|
||||
expect(aliasMocks.rollbackManagedCliLink).not.toHaveBeenCalled();
|
||||
expect(runtimeMocks.restartUserService).toHaveBeenCalledOnce();
|
||||
expect(
|
||||
(runtimeMocks.restartUserService.mock.calls as unknown[][])[0]?.[0],
|
||||
).toMatchObject({
|
||||
dataDir: "/tmp/custom data",
|
||||
installRoot: "/tmp/install/current",
|
||||
});
|
||||
});
|
||||
|
||||
// O-22 (2026-09-29 C8 H17): `upgrade --yes` on a manifest-valid but
|
||||
// crash-looping version reported exit 0 and left `current` on the bad
|
||||
// version. A restart that merely returns is not proof the new Runtime came
|
||||
// up, so the CLI now verifies health and rolls back if it never does.
|
||||
it("rolls back and fails when the upgraded Runtime never becomes healthy", async () => {
|
||||
it("fails without rolling back when the upgraded Runtime never becomes healthy", async () => {
|
||||
vi.clearAllMocks();
|
||||
runtimeMocks.planCliLink.mockResolvedValueOnce({
|
||||
schema_version: 1,
|
||||
@@ -890,9 +901,9 @@ describe("public One Memory CLI", () => {
|
||||
command: "upgrade",
|
||||
code: "upgrade_health_check_failed",
|
||||
});
|
||||
// Rolled back to the previous version and restarted it.
|
||||
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
|
||||
expect(runtimeMocks.restartUserService).toHaveBeenCalledTimes(2);
|
||||
// Kept on the new version; the old Runtime is not restarted.
|
||||
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
|
||||
expect(runtimeMocks.restartUserService).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
const upgradeArguments = [
|
||||
@@ -919,6 +930,38 @@ describe("public One Memory CLI", () => {
|
||||
target_path: "/tmp/install/current/bin/one-memory",
|
||||
};
|
||||
|
||||
it("says the data was restored when a failed migration put it back", async () => {
|
||||
vi.clearAllMocks();
|
||||
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
|
||||
runtimeMocks.runSdkMigration.mockResolvedValueOnce({
|
||||
state: "deferred",
|
||||
code: "sdk_migration_memory_unreadable",
|
||||
dataRestored: true,
|
||||
} as never);
|
||||
const streams = captureStreams();
|
||||
let code: number;
|
||||
try {
|
||||
code = await run(
|
||||
upgradeArguments,
|
||||
{},
|
||||
{ waitForRuntimeReady: async () => true },
|
||||
);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(code!).toBe(1);
|
||||
const failure = JSON.parse(streams.stderr());
|
||||
expect(failure).toMatchObject({
|
||||
code: "sdk_migration_memory_unreadable",
|
||||
data_restored: true,
|
||||
log_file: "/tmp/custom data/logs/sdk-migration.log",
|
||||
});
|
||||
expect(failure.message).toContain("记忆数据已恢复到升级前的状态。");
|
||||
expect(failure.message).toContain(
|
||||
"/tmp/custom data/logs/sdk-migration.log",
|
||||
);
|
||||
});
|
||||
|
||||
it("migrates the home with the new Core before restarting the Runtime", async () => {
|
||||
vi.clearAllMocks();
|
||||
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
|
||||
@@ -948,7 +991,7 @@ describe("public One Memory CLI", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("rolls back to the previous version when the migration does not finish", async () => {
|
||||
it("stays on the new version, not serving, when the migration does not finish", async () => {
|
||||
vi.clearAllMocks();
|
||||
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
|
||||
runtimeMocks.runSdkMigration.mockResolvedValueOnce({
|
||||
@@ -967,11 +1010,87 @@ describe("public One Memory CLI", () => {
|
||||
streams.restore();
|
||||
}
|
||||
expect(code!).toBe(1);
|
||||
expect(JSON.parse(streams.stderr())).toMatchObject({
|
||||
const failure = JSON.parse(streams.stderr());
|
||||
expect(failure).toMatchObject({
|
||||
command: "upgrade",
|
||||
code: "sdk_migration_disk_space_insufficient",
|
||||
// The one file support needs, and where to send it.
|
||||
log_file: "/tmp/custom data/logs/sdk-migration.log",
|
||||
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
|
||||
});
|
||||
expect(failure.message).toContain(
|
||||
"/tmp/custom data/logs/sdk-migration.log",
|
||||
);
|
||||
expect(failure.message).toContain("GitHub");
|
||||
expect(failure.message).not.toContain("回滚到上一版本");
|
||||
expect(runtimeMocks.runSdkMigration).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ caller: "cli-upgrade" }),
|
||||
);
|
||||
// No rollback: the old Runtime has no migration gate. The new service is
|
||||
// (re)started so its gate, not the old Runtime, holds the data dir.
|
||||
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
|
||||
expect(aliasMocks.rollbackManagedCliLink).not.toHaveBeenCalled();
|
||||
expect(runtimeMocks.restartUserService).toHaveBeenCalledOnce();
|
||||
expect(
|
||||
(runtimeMocks.restartUserService.mock.calls as unknown[][])[0]?.[0],
|
||||
).toMatchObject({
|
||||
installRoot: "/tmp/install/current",
|
||||
});
|
||||
});
|
||||
|
||||
it("stays on the new version when the migration itself throws", async () => {
|
||||
vi.clearAllMocks();
|
||||
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
|
||||
runtimeMocks.runSdkMigration.mockRejectedValueOnce(
|
||||
new Error("sdk_migration_fence_failed"),
|
||||
);
|
||||
const streams = captureStreams();
|
||||
let code: number;
|
||||
try {
|
||||
code = await run(
|
||||
upgradeArguments,
|
||||
{},
|
||||
{ waitForRuntimeReady: async () => true },
|
||||
);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(code!).toBe(1);
|
||||
expect(JSON.parse(streams.stderr())).toMatchObject({
|
||||
command: "upgrade",
|
||||
code: "sdk_migration_fence_failed",
|
||||
log_file: "/tmp/custom data/logs/sdk-migration.log",
|
||||
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
|
||||
});
|
||||
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("still rolls back a failure before the migration touched the home", async () => {
|
||||
vi.clearAllMocks();
|
||||
runtimeMocks.planCliLink.mockResolvedValueOnce(presentCliLink);
|
||||
aliasMocks.installManagedCliLink.mockRejectedValueOnce(
|
||||
new Error("cli_link_failed"),
|
||||
);
|
||||
const streams = captureStreams();
|
||||
let code: number;
|
||||
try {
|
||||
code = await run(
|
||||
upgradeArguments,
|
||||
{},
|
||||
{ waitForRuntimeReady: async () => true },
|
||||
);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(code!).toBe(1);
|
||||
expect(JSON.parse(streams.stderr())).toMatchObject({
|
||||
command: "upgrade",
|
||||
code: "cli_link_failed",
|
||||
});
|
||||
expect(runtimeMocks.runSdkMigration).not.toHaveBeenCalled();
|
||||
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
|
||||
// The previous Runtime was never stopped; nothing is (re)started.
|
||||
expect(runtimeMocks.restartUserService).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("keeps the new version when a migrated home is slow to come up", async () => {
|
||||
@@ -1492,7 +1611,7 @@ describe("public One Memory CLI", () => {
|
||||
expect(JSON.parse(output.value()).apply).toBe(true);
|
||||
});
|
||||
|
||||
it("rolls back the distribution when service installation fails", async () => {
|
||||
it("keeps the distribution when service installation fails after the migration ran", async () => {
|
||||
runtimeMocks.applyLocalInstall.mockResolvedValueOnce({
|
||||
schema_version: 1,
|
||||
source: "/tmp/distribution",
|
||||
@@ -1533,13 +1652,14 @@ describe("public One Memory CLI", () => {
|
||||
command: "install",
|
||||
status: "error",
|
||||
code: "service failed",
|
||||
rolled_back: false,
|
||||
});
|
||||
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
|
||||
expect(runtimeMocks.rollbackCliLink).toHaveBeenCalledOnce();
|
||||
expect(aliasMocks.rollbackManagedCliLink).toHaveBeenCalledTimes(2);
|
||||
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
|
||||
expect(runtimeMocks.rollbackCliLink).not.toHaveBeenCalled();
|
||||
expect(aliasMocks.rollbackManagedCliLink).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("upgrades only an existing managed CLI and rolls back a failed restart", async () => {
|
||||
it("upgrades only an existing managed CLI and keeps it after a failed restart", async () => {
|
||||
runtimeMocks.planCliLink.mockResolvedValueOnce({
|
||||
schema_version: 1,
|
||||
platform: "linux",
|
||||
@@ -1579,7 +1699,7 @@ describe("public One Memory CLI", () => {
|
||||
status: "error",
|
||||
code: "restart failed",
|
||||
});
|
||||
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
|
||||
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
|
||||
expect(runtimeMocks.installCliLink).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
@@ -1891,16 +2011,11 @@ describe("stopped Runtime is exit 3, not a usage error", () => {
|
||||
["path", "list", "--json"],
|
||||
["adapter", "list", "--json"],
|
||||
["task", "--id", "nope", "--json"],
|
||||
[
|
||||
"write",
|
||||
"submit",
|
||||
"--session",
|
||||
"s",
|
||||
"--messages-file",
|
||||
"/tmp/messages.json",
|
||||
"--json",
|
||||
],
|
||||
["backup", "create", "--target", "/tmp/x", "--yes", "--json"],
|
||||
["write", "status", "nope", "--json"],
|
||||
["backup", "create", "--target", "/tmp/x", "--json"],
|
||||
["scan-roots", "add", "--root", "/tmp/x", "--json"],
|
||||
["scan", "--source", "codex", "--json"],
|
||||
["onboarding", "reset", "--json"],
|
||||
])("%s ... --json exits 3 with runtime_not_running", async (...argv) => {
|
||||
const streams = captureStreams();
|
||||
let code: number;
|
||||
@@ -1910,7 +2025,9 @@ describe("stopped Runtime is exit 3, not a usage error", () => {
|
||||
streams.restore();
|
||||
}
|
||||
expect(code!).toBe(3);
|
||||
const document = JSON.parse(streams.stderr());
|
||||
// A result, not an error: on stdout like every other --json document.
|
||||
expect(streams.stderr()).toBe("");
|
||||
const document = JSON.parse(streams.stdout());
|
||||
expect(document).toMatchObject({
|
||||
code: "runtime_not_running",
|
||||
exit_code: 3,
|
||||
@@ -1919,6 +2036,48 @@ describe("stopped Runtime is exit 3, not a usage error", () => {
|
||||
expect(document.status).not.toBe("usage_error");
|
||||
});
|
||||
|
||||
it("write submit validates its messages file before attaching, then exits 3", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "memhub-write-"));
|
||||
const file = path.join(directory, "messages.json");
|
||||
await import("node:fs/promises").then(({ writeFile }) =>
|
||||
writeFile(file, JSON.stringify([{ role: "user", content: "hi" }])),
|
||||
);
|
||||
const streams = captureStreams();
|
||||
let missing: number;
|
||||
let code: number;
|
||||
try {
|
||||
missing = await run(
|
||||
[
|
||||
"write",
|
||||
"submit",
|
||||
"--session",
|
||||
"s",
|
||||
"--messages-file",
|
||||
path.join(directory, "absent.json"),
|
||||
"--json",
|
||||
],
|
||||
{},
|
||||
);
|
||||
code = await run(
|
||||
[
|
||||
"write",
|
||||
"submit",
|
||||
"--session",
|
||||
"s",
|
||||
"--messages-file",
|
||||
file,
|
||||
"--json",
|
||||
],
|
||||
{},
|
||||
);
|
||||
} finally {
|
||||
streams.restore();
|
||||
await rm(directory, { recursive: true, force: true });
|
||||
}
|
||||
expect(missing!).toBe(2);
|
||||
expect(code!).toBe(3);
|
||||
});
|
||||
|
||||
it.each(["pause", "resume"] as const)(
|
||||
"parses sync %s as the direct sync-set command",
|
||||
async (action) => {
|
||||
@@ -2003,6 +2162,189 @@ describe("lifecycle commands accept --json", () => {
|
||||
});
|
||||
});
|
||||
|
||||
// A Runtime service that refuses to serve an unmigrated home must be reported
|
||||
// as that, with the migration log and where to send it, not as a generic
|
||||
// "not ready within budget".
|
||||
describe("lifecycle commands report a migration block", () => {
|
||||
async function blockedHome() {
|
||||
const dataDir = await mkdtemp(path.join(tmpdir(), "memhub-blocked-"));
|
||||
await recordSdkMigrationBlock(
|
||||
dataDir,
|
||||
"sdk_migration_disk_space_insufficient",
|
||||
"runtime-service",
|
||||
);
|
||||
return dataDir;
|
||||
}
|
||||
// darwin: kickstart "restarts" but never becomes ready. Elsewhere: no
|
||||
// service, so the direct spawn fails. Both must surface the block.
|
||||
const blockedControls = () => ({
|
||||
restartManagedRuntime: vi.fn(async () => "restarted" as const),
|
||||
waitForRuntimeReady: vi.fn(async () => false),
|
||||
stopRuntime: vi.fn(async () => "stopped" as const),
|
||||
ensureRuntime: vi.fn(async () => {
|
||||
throw new Error("Timed out waiting for MemHub Runtime");
|
||||
}),
|
||||
});
|
||||
|
||||
it("restart --json prints the block code, log file and issues URL", async () => {
|
||||
const dataDir = await blockedHome();
|
||||
const output = captureStdout();
|
||||
let code: number;
|
||||
try {
|
||||
code = await run(
|
||||
["restart", "--json"],
|
||||
{ ONE_MEMORY_DATA_DIR: dataDir },
|
||||
blockedControls(),
|
||||
);
|
||||
} finally {
|
||||
output.restore();
|
||||
await rm(dataDir, { recursive: true, force: true });
|
||||
}
|
||||
expect(code!).toBe(1);
|
||||
const lines = output.value().trim().split("\n");
|
||||
const document = JSON.parse(lines[lines.length - 1]!);
|
||||
expect(document).toMatchObject({
|
||||
command: "restart",
|
||||
state: "blocked",
|
||||
code: "sdk_migration_disk_space_insufficient",
|
||||
log_file: path.join(dataDir, "logs", "sdk-migration.log"),
|
||||
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
|
||||
});
|
||||
expect(output.value()).not.toContain("runtime_not_ready_within_budget");
|
||||
});
|
||||
|
||||
it("restart on a frozen home retries the migration in the foreground first", async () => {
|
||||
const dataDir = await mkdtemp(path.join(tmpdir(), "memhub-frozen-"));
|
||||
await recordSdkMigrationBlock(
|
||||
dataDir,
|
||||
"sdk_migration_memory_unreadable",
|
||||
"desktop",
|
||||
{ frozen: { service_was_enabled: true }, data_restored: true },
|
||||
);
|
||||
const controls = {
|
||||
...blockedControls(),
|
||||
migrateSdkData: vi.fn(async () => ({
|
||||
state: "deferred" as const,
|
||||
code: "sdk_migration_memory_unreadable",
|
||||
dataRestored: true,
|
||||
})),
|
||||
};
|
||||
const output = captureStdout();
|
||||
let code: number;
|
||||
try {
|
||||
code = await run(
|
||||
["restart", "--json"],
|
||||
{ ONE_MEMORY_DATA_DIR: dataDir },
|
||||
controls,
|
||||
);
|
||||
} finally {
|
||||
output.restore();
|
||||
await rm(dataDir, { recursive: true, force: true });
|
||||
}
|
||||
expect(code!).toBe(1);
|
||||
expect(controls.migrateSdkData).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ dataDir, caller: "cli-restart" }),
|
||||
);
|
||||
// Still frozen: the service is not started to retry on its own.
|
||||
expect(controls.restartManagedRuntime).not.toHaveBeenCalled();
|
||||
expect(controls.ensureRuntime).not.toHaveBeenCalled();
|
||||
expect(JSON.parse(output.value())).toMatchObject({
|
||||
command: "restart",
|
||||
state: "blocked",
|
||||
code: "sdk_migration_memory_unreadable",
|
||||
data_restored: true,
|
||||
});
|
||||
});
|
||||
|
||||
it("restart on a frozen home restarts the Runtime once the retry succeeds", async () => {
|
||||
const dataDir = await mkdtemp(path.join(tmpdir(), "memhub-frozen-"));
|
||||
await recordSdkMigrationBlock(
|
||||
dataDir,
|
||||
"sdk_migration_memory_unreadable",
|
||||
"desktop",
|
||||
{ frozen: { service_was_enabled: false } },
|
||||
);
|
||||
const order: string[] = [];
|
||||
const migrateSdkData = vi.fn(async () => {
|
||||
order.push("migrate");
|
||||
await clearSdkMigrationBlock(dataDir);
|
||||
return { state: "cutover" as const };
|
||||
});
|
||||
const output = captureStreams();
|
||||
let code: number;
|
||||
try {
|
||||
code = await run(
|
||||
["restart", "--json"],
|
||||
{ ONE_MEMORY_DATA_DIR: dataDir },
|
||||
{
|
||||
migrateSdkData,
|
||||
waitForRuntimeReady: vi.fn(async () => true),
|
||||
restartManagedRuntime: vi.fn(async () => {
|
||||
order.push("restart");
|
||||
return "restarted" as const;
|
||||
}),
|
||||
stopRuntime: vi.fn(async () => "stopped" as const),
|
||||
ensureRuntime: vi.fn(async () => {
|
||||
order.push("restart");
|
||||
return { pid: 1 } as never;
|
||||
}),
|
||||
},
|
||||
);
|
||||
} finally {
|
||||
output.restore();
|
||||
await rm(dataDir, { recursive: true, force: true });
|
||||
}
|
||||
expect(code!).toBe(0);
|
||||
expect(migrateSdkData).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ caller: "cli-restart" }),
|
||||
);
|
||||
expect(order).toEqual(["migrate", "restart"]);
|
||||
});
|
||||
|
||||
it("restart in text mode names the log file and GitHub issues", async () => {
|
||||
const dataDir = await blockedHome();
|
||||
const output = captureStdout();
|
||||
try {
|
||||
expect(
|
||||
await run(
|
||||
["restart"],
|
||||
{ ONE_MEMORY_DATA_DIR: dataDir },
|
||||
blockedControls(),
|
||||
),
|
||||
).toBe(1);
|
||||
} finally {
|
||||
output.restore();
|
||||
await rm(dataDir, { recursive: true, force: true });
|
||||
}
|
||||
expect(output.value()).toContain("sdk_migration_disk_space_insufficient");
|
||||
expect(output.value()).toContain(
|
||||
path.join(dataDir, "logs", "sdk-migration.log"),
|
||||
);
|
||||
expect(output.value()).toContain(
|
||||
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
|
||||
);
|
||||
});
|
||||
|
||||
it("ui --json reports the block instead of runtime_not_running", async () => {
|
||||
const dataDir = await blockedHome();
|
||||
const output = captureStdout();
|
||||
let code: number;
|
||||
try {
|
||||
code = await run(["ui", "--json"], { ONE_MEMORY_DATA_DIR: dataDir });
|
||||
} finally {
|
||||
output.restore();
|
||||
await rm(dataDir, { recursive: true, force: true });
|
||||
}
|
||||
expect(code!).not.toBe(0);
|
||||
expect(JSON.parse(output.value())).toMatchObject({
|
||||
command: "ui",
|
||||
state: "blocked",
|
||||
code: "sdk_migration_disk_space_insufficient",
|
||||
log_file: path.join(dataDir, "logs", "sdk-migration.log"),
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
// The objective names `memhub host install <host>` (host as a positional). Both
|
||||
// that and the `--host <id>` flag form must reach the same command; naming both
|
||||
// at once is ambiguous and refused rather than guessed.
|
||||
@@ -2112,7 +2454,7 @@ describe("install gates on Runtime readiness", () => {
|
||||
});
|
||||
}
|
||||
|
||||
it("fails and rolls back when the installed Runtime never becomes ready", async () => {
|
||||
it("fails without rolling back when the installed Runtime never becomes ready", async () => {
|
||||
vi.clearAllMocks();
|
||||
stubInstallPlan();
|
||||
const streams = captureStreams();
|
||||
@@ -2141,10 +2483,10 @@ describe("install gates on Runtime readiness", () => {
|
||||
}
|
||||
expect(code!).toBe(1);
|
||||
expect(streams.stderr()).toContain("install_runtime_not_ready");
|
||||
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
|
||||
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
|
||||
expect(
|
||||
lifecycleMocks.rollbackPreparedServiceInstall,
|
||||
).toHaveBeenCalledOnce();
|
||||
).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("takes CLI ownership after a successful install over an App-owned Runtime", async () => {
|
||||
@@ -2189,7 +2531,7 @@ describe("install gates on Runtime readiness", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("restores App ownership before relaunching the old Runtime after install failure", async () => {
|
||||
it("never hands the home back to the App Runtime after the migration ran", async () => {
|
||||
vi.clearAllMocks();
|
||||
stubInstallPlan();
|
||||
const root = await mkdtemp(path.join(tmpdir(), "memhub-cli-owner-"));
|
||||
@@ -2199,14 +2541,6 @@ describe("install gates on Runtime readiness", () => {
|
||||
appBundle: "/Applications/MemHub.app",
|
||||
appExecutable: "/Applications/MemHub.app/Contents/MacOS/MemHub",
|
||||
});
|
||||
lifecycleMocks.rollbackPreparedServiceInstall.mockImplementationOnce(
|
||||
async (_prepared: unknown, rollback: () => Promise<void>) => {
|
||||
await rollback();
|
||||
await expect(readRuntimeServiceOwner(dataDir)).resolves.toMatchObject({
|
||||
owner: "app",
|
||||
});
|
||||
},
|
||||
);
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(
|
||||
@@ -2229,15 +2563,114 @@ describe("install gates on Runtime readiness", () => {
|
||||
{ waitForRuntimeReady: async () => false },
|
||||
),
|
||||
).toBe(1);
|
||||
// The App Runtime may predate the SDK migration gate: CLI keeps it.
|
||||
await expect(readRuntimeServiceOwner(dataDir)).resolves.toMatchObject({
|
||||
owner: "app",
|
||||
owner: "cli",
|
||||
});
|
||||
expect(
|
||||
lifecycleMocks.rollbackPreparedServiceInstall,
|
||||
).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
streams.restore();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps the install, not serving, when its migration is deferred", async () => {
|
||||
vi.clearAllMocks();
|
||||
stubInstallPlan();
|
||||
runtimeMocks.runSdkMigration.mockResolvedValueOnce({
|
||||
state: "deferred",
|
||||
code: "sdk_migration_memory_unreadable",
|
||||
});
|
||||
const waitForRuntimeReady = vi.fn(async () => true);
|
||||
const streams = captureStreams();
|
||||
let code: number;
|
||||
try {
|
||||
code = await run(
|
||||
[
|
||||
"install",
|
||||
"--source",
|
||||
"/tmp/dist",
|
||||
"--home",
|
||||
"/tmp/home",
|
||||
"--install-root",
|
||||
"/tmp/install",
|
||||
"--bin-dir",
|
||||
"/tmp/bin",
|
||||
"--data-dir",
|
||||
"/tmp/data",
|
||||
"--yes",
|
||||
],
|
||||
{},
|
||||
{ waitForRuntimeReady },
|
||||
);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(code!).toBe(1);
|
||||
const failure = JSON.parse(streams.stderr());
|
||||
expect(failure).toMatchObject({
|
||||
command: "install",
|
||||
code: "sdk_migration_memory_unreadable",
|
||||
log_file: "/tmp/data/logs/sdk-migration.log",
|
||||
issues_url: "https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
|
||||
});
|
||||
expect(failure.message).toContain("/tmp/data/logs/sdk-migration.log");
|
||||
expect(runtimeMocks.runSdkMigration).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ caller: "cli-install" }),
|
||||
);
|
||||
// The new (gated) service owns the data dir; the previous registration is
|
||||
// never re-bootstrapped and nothing is rolled back.
|
||||
expect(lifecycleMocks.applyPreparedServiceInstall).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ restorePreviousOnFailure: false }),
|
||||
);
|
||||
expect(runtimeMocks.rollbackLocalInstall).not.toHaveBeenCalled();
|
||||
expect(
|
||||
lifecycleMocks.rollbackPreparedServiceInstall,
|
||||
).not.toHaveBeenCalled();
|
||||
expect(waitForRuntimeReady).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("still rolls back an install that failed before its migration ran", async () => {
|
||||
vi.clearAllMocks();
|
||||
stubInstallPlan();
|
||||
runtimeMocks.installCliLink.mockRejectedValueOnce(
|
||||
new Error("cli_link_failed"),
|
||||
);
|
||||
const streams = captureStreams();
|
||||
let code: number;
|
||||
try {
|
||||
code = await run(
|
||||
[
|
||||
"install",
|
||||
"--source",
|
||||
"/tmp/dist",
|
||||
"--home",
|
||||
"/tmp/home",
|
||||
"--install-root",
|
||||
"/tmp/install",
|
||||
"--bin-dir",
|
||||
"/tmp/bin",
|
||||
"--data-dir",
|
||||
"/tmp/data",
|
||||
"--yes",
|
||||
],
|
||||
{},
|
||||
{ waitForRuntimeReady: async () => true },
|
||||
);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(code!).toBe(1);
|
||||
expect(JSON.parse(streams.stderr())).toMatchObject({
|
||||
command: "install",
|
||||
code: "cli_link_failed",
|
||||
});
|
||||
expect(runtimeMocks.runSdkMigration).not.toHaveBeenCalled();
|
||||
expect(runtimeMocks.rollbackLocalInstall).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("succeeds when the installed Runtime becomes ready", async () => {
|
||||
vi.clearAllMocks();
|
||||
stubInstallPlan();
|
||||
@@ -2317,7 +2750,8 @@ describe("ui wires the Core session contract without token leakage", () => {
|
||||
} finally {
|
||||
output.restore();
|
||||
}
|
||||
expect(code!).toBe(4);
|
||||
// A stopped Runtime is exit 3, like every other command.
|
||||
expect(code!).toBe(3);
|
||||
const document = JSON.parse(output.value());
|
||||
expect(document).toMatchObject({
|
||||
schema_version: 1,
|
||||
@@ -2346,6 +2780,120 @@ describe("ui wires the Core session contract without token leakage", () => {
|
||||
// refuse while the Runtime owns the data dir, stop+call Core from a
|
||||
// maintenance process, and never report success from a malformed response.
|
||||
describe("backup restore drives the Core managed-restore contract", () => {
|
||||
it("previews without --yes, exits 0 and starts nothing", async () => {
|
||||
const maintenance = vi.fn();
|
||||
const output = captureStdout();
|
||||
let code: number;
|
||||
try {
|
||||
code = await run(
|
||||
["backup", "restore", "--source", "/tmp/x", "--json"],
|
||||
{},
|
||||
{ runMaintenanceRestore: maintenance },
|
||||
);
|
||||
} finally {
|
||||
output.restore();
|
||||
}
|
||||
expect(code!).toBe(0);
|
||||
expect(maintenance).not.toHaveBeenCalled();
|
||||
expect(JSON.parse(output.value())).toMatchObject({
|
||||
command: "backup restore",
|
||||
state: "preview",
|
||||
target: "/tmp/x",
|
||||
applied: false,
|
||||
next_step: "memhub backup restore --source /tmp/x --yes --confirm /tmp/x",
|
||||
});
|
||||
});
|
||||
|
||||
it("deletes data only when --confirm repeats the data directory", async () => {
|
||||
const home = await mkdtemp(path.join(tmpdir(), "memhub-delete-home-"));
|
||||
const dataDir = path.join(home, ".one-memory");
|
||||
await mkdir(path.join(dataDir, "logs"), { recursive: true });
|
||||
const deleteData = async (...extra: string[]) => {
|
||||
const output = captureStdout();
|
||||
try {
|
||||
const code = await run(
|
||||
["delete-data", "--data-dir", dataDir, ...extra, "--json"],
|
||||
{ HOME: home },
|
||||
);
|
||||
return { code, document: JSON.parse(output.value()) };
|
||||
} finally {
|
||||
output.restore();
|
||||
}
|
||||
};
|
||||
try {
|
||||
const refused = await deleteData("--yes");
|
||||
expect(refused.code).toBe(4);
|
||||
expect(refused.document).toMatchObject({
|
||||
state: "blocked",
|
||||
code: "data_deletion_confirmation_required",
|
||||
});
|
||||
expect(await readdir(dataDir)).toEqual(["logs"]);
|
||||
|
||||
const deleted = await deleteData("--yes", "--confirm", dataDir);
|
||||
expect(deleted.code).toBe(0);
|
||||
expect(await readdir(dataDir).catch(() => [])).toEqual([]);
|
||||
} finally {
|
||||
await rm(home, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("refuses --yes without repeating the backup path", async () => {
|
||||
const maintenance = vi.fn();
|
||||
const output = captureStdout();
|
||||
let code: number;
|
||||
try {
|
||||
code = await run(
|
||||
["backup", "restore", "--source", "/tmp/x", "--yes", "--json"],
|
||||
{},
|
||||
{ runMaintenanceRestore: maintenance },
|
||||
);
|
||||
} finally {
|
||||
output.restore();
|
||||
}
|
||||
expect(code!).toBe(4);
|
||||
expect(maintenance).not.toHaveBeenCalled();
|
||||
expect(JSON.parse(output.value())).toMatchObject({
|
||||
state: "rejected",
|
||||
code: "backup_restore_confirmation_required",
|
||||
next_step:
|
||||
"恢复需要再写一遍备份路径:memhub backup restore --source /tmp/x --yes --confirm /tmp/x",
|
||||
});
|
||||
});
|
||||
|
||||
it("restores with --yes and the repeated backup path", async () => {
|
||||
const maintenance = vi.fn(async () => ({
|
||||
state: "restored" as const,
|
||||
result: {
|
||||
state: "restored" as const,
|
||||
restoredBytes: 1,
|
||||
previousBackupPath: "/tmp/pre.sqlite3",
|
||||
sourceSchemaVersion: 34,
|
||||
},
|
||||
}));
|
||||
const output = captureStdout();
|
||||
let code: number;
|
||||
try {
|
||||
code = await run(
|
||||
[
|
||||
"backup",
|
||||
"restore",
|
||||
"--source",
|
||||
"/tmp/x",
|
||||
"--yes",
|
||||
"--confirm",
|
||||
"/tmp/x",
|
||||
"--json",
|
||||
],
|
||||
{},
|
||||
{ runMaintenanceRestore: maintenance },
|
||||
);
|
||||
} finally {
|
||||
output.restore();
|
||||
}
|
||||
expect(code!).toBe(0);
|
||||
expect(maintenance).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("rejects while the Runtime is running, with a distinct exit code", async () => {
|
||||
runtimeMocks.inspectRuntimeStatus.mockResolvedValueOnce({
|
||||
state: "running",
|
||||
@@ -2462,7 +3010,7 @@ describe("backup restore drives the Core managed-restore contract", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("requires --confirm to repeat the exact source path", async () => {
|
||||
it("refuses a legacy --confirm that does not repeat the source path", async () => {
|
||||
const maintenance = vi.fn();
|
||||
const output = captureStdout();
|
||||
let code: number;
|
||||
@@ -2490,7 +3038,7 @@ describe("backup restore drives the Core managed-restore contract", () => {
|
||||
command: "backup",
|
||||
verb: "restore",
|
||||
state: "rejected",
|
||||
code: "backup_restore_confirmation_required",
|
||||
code: "backup_restore_confirmation_mismatch",
|
||||
});
|
||||
});
|
||||
|
||||
@@ -2642,7 +3190,7 @@ describe("init onboarding", () => {
|
||||
|
||||
// Boolean flags were idempotent, but accepting them twice hid a duplicated
|
||||
// argv fragment that could also have duplicated a value-bearing flag.
|
||||
it.each(["--import", "--full-rescan", "--confirm-overrides", "--dry-run"])(
|
||||
it.each(["--import", "--confirm-overrides"])(
|
||||
"refuses a repeated init %s flag",
|
||||
async (flag) => {
|
||||
expect(await run(["init", "--source", "codex", flag, flag], {})).toBe(2);
|
||||
@@ -2715,12 +3263,25 @@ describe("init onboarding", () => {
|
||||
},
|
||||
);
|
||||
|
||||
it("accepts --dry-run together with --import", async () => {
|
||||
// Parsing must accept the pair; it stops at the Runtime check, which is
|
||||
// not a usage failure.
|
||||
expect(
|
||||
await run(["init", "--source", "codex", "--import", "--dry-run"], {}),
|
||||
).not.toBe(2);
|
||||
// `--dry-run` and `--full-rescan` never did anything (P0-2/P0-3): they are
|
||||
// gone rather than accepted as no-ops.
|
||||
it.each(["--dry-run", "--full-rescan"])(
|
||||
"refuses the removed no-op %s flag",
|
||||
async (flag) => {
|
||||
expect(
|
||||
await run(["init", "--source", "codex", "--import", flag], {}),
|
||||
).toBe(2);
|
||||
},
|
||||
);
|
||||
|
||||
it("still parses the old `init --source --import` spelling", async () => {
|
||||
// It stops at the Runtime check (exit 3), which is not a usage failure.
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["init", "--source", "codex", "--import"], {})).toBe(3);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
+1921
-2284
File diff suppressed because it is too large.
Load diff
@@ -58,6 +58,12 @@ const CONFIRM_OPTION: CommandOption = {
|
||||
flag: "--yes",
|
||||
summary: "确认执行;不加该参数时只输出计划,不做任何改动。",
|
||||
};
|
||||
/** The one confirmation convention, for irreversible operations only. */
|
||||
const DESTRUCTIVE_CONFIRM_OPTION: CommandOption = {
|
||||
flag: "--yes",
|
||||
summary:
|
||||
"确认这次不可逆操作;不加时只说明将要做什么,不做任何改动,exit 0。(旧的 --confirm 仍作为别名接受。)",
|
||||
};
|
||||
const JSON_OPTION: CommandOption = {
|
||||
flag: "--json",
|
||||
summary: "以 JSON 输出到 stdout,供脚本与 Agent 消费。",
|
||||
@@ -70,21 +76,6 @@ const INTERVAL_OPTION: CommandOption = {
|
||||
flag: "--interval <seconds>",
|
||||
summary: "刷新间隔,默认 5 秒,范围 1..300;必须与 --watch 一起使用。",
|
||||
};
|
||||
const PLAN_FILE_OPTION: CommandOption = {
|
||||
flag: "--plan-file <path>",
|
||||
summary: "读取先前 --json 保存的有界计划文件。",
|
||||
};
|
||||
const EXPLICIT_CONFIRM_OPTION: CommandOption = {
|
||||
flag: "--confirm",
|
||||
summary: "确认执行计划文件逐字描述的操作;缺少时绝不写入。",
|
||||
};
|
||||
// Same flag, but for commands that name their target directly instead of
|
||||
// going through a saved plan file; the plan-file wording would be wrong here.
|
||||
const DIRECT_CONFIRM_OPTION: CommandOption = {
|
||||
flag: "--confirm",
|
||||
summary: "确认执行上面预览的这次操作;缺少时绝不写入。",
|
||||
};
|
||||
|
||||
const SERVICE_OPTIONS: readonly CommandOption[] = [
|
||||
PLATFORM_OPTION,
|
||||
HOME_OPTION,
|
||||
@@ -154,55 +145,77 @@ const INSTALL_OPTIONS: readonly CommandOption[] = [
|
||||
const INSTALL_USAGE =
|
||||
"[--source <path> | --release-manifest-url <https-url>] [--channel <stable|prerelease>] [--install-root <path>] [--home <path>] [--bin-dir <path>] [--data-dir <path>] [--uid <id>] [--xdg-config-home <path>] [--no-shell-path] [--shell-profile <path>] [--preserve-data-dir] [--allow-downgrade] [--yes]";
|
||||
|
||||
const USAGE_EXITS = [EXIT_OK, EXIT_FAILED, EXIT_USAGE] as const;
|
||||
const RUNTIME_EXITS = [
|
||||
EXIT_OK,
|
||||
EXIT_FAILED,
|
||||
EXIT_USAGE,
|
||||
EXIT_RUNTIME_STOPPED,
|
||||
] as const;
|
||||
|
||||
const KIND_OPTION: CommandOption = {
|
||||
flag: "--kind <chat|experience|fact>",
|
||||
summary:
|
||||
"记忆所在的命名空间;工作/编码场景的经验和事实需要同时给 --variant。list 输出里每条都带 kind/variant。",
|
||||
};
|
||||
const VARIANT_OPTION: CommandOption = {
|
||||
flag: "--variant <working|coding>",
|
||||
summary:
|
||||
"经验或事实属于工作还是编码场景;必须和 --kind experience|fact 一起用。",
|
||||
};
|
||||
|
||||
export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
{
|
||||
name: "serve",
|
||||
summary: "在前台启动完整 Runtime(Core 与 Host Gateway),不注册系统服务。",
|
||||
usage: "memhub serve",
|
||||
options: [],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED],
|
||||
exit_codes: USAGE_EXITS,
|
||||
},
|
||||
{
|
||||
name: "start",
|
||||
summary: "启动本机 Runtime;已有通过身份校验的实例时只连接,不重复启动。",
|
||||
summary:
|
||||
"启动本机 Runtime;已有通过身份校验的实例时只连接,不重复启动。受管服务会同时恢复登录时自动启动。",
|
||||
usage: "memhub start [--json]",
|
||||
options: [JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_RUNTIME_STOPPED],
|
||||
exit_codes: USAGE_EXITS,
|
||||
},
|
||||
{
|
||||
name: "stop",
|
||||
summary: "停止通过身份校验的本机 Runtime;身份不明确时不发送信号。",
|
||||
summary:
|
||||
"停止通过身份校验的本机 Runtime;身份不明确时不发送信号。受管服务(launchd/systemd、App)停止后登录时也不会自动启动,直到再次运行 memhub start。",
|
||||
usage: "memhub stop [--json]",
|
||||
options: [JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED],
|
||||
exit_codes: USAGE_EXITS,
|
||||
},
|
||||
{
|
||||
name: "restart",
|
||||
summary: "先停止再启动本机 Runtime,两步串行执行。",
|
||||
usage: "memhub restart [--json]",
|
||||
options: [JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED],
|
||||
exit_codes: USAGE_EXITS,
|
||||
},
|
||||
{
|
||||
name: "status",
|
||||
summary: "只读查询 Runtime 运行状态;不启动服务、不迁移数据、不改配置。",
|
||||
usage: "memhub status [--watch] [--interval <seconds>] [--json]",
|
||||
options: [WATCH_OPTION, INTERVAL_OPTION, JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_RUNTIME_STOPPED],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "doctor",
|
||||
summary: "按诊断口径做同样的只读检查;默认不修复任何东西。",
|
||||
summary:
|
||||
"按诊断口径做只读检查并给出修复建议;默认不修复任何东西。Runtime 未运行时 exit 3。",
|
||||
usage: "memhub doctor [--json]",
|
||||
options: [JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_RUNTIME_STOPPED],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "logs",
|
||||
summary:
|
||||
"读取本机脱敏诊断日志;logs export 会把排查包直接存到下载目录。默认显示最近一次会话的 200 条事件。",
|
||||
"读取本机脱敏诊断日志;logs export 会把排查包(含系统崩溃报告)直接存到下载目录。默认显示最近一次会话的 200 条事件。",
|
||||
usage:
|
||||
"memhub logs export [--json]\nmemhub logs [--since <15m|2h|7d|RFC3339>] [--session <id>] [--trace <id>] [--incident <id>] [--operation <id>] [--request <id>] [--job <id>] [--task <id>] [--component <name>] [--event <name>] [--errors-only] [--tail <1..5000>] [--follow] [--json]",
|
||||
"memhub logs export [--json]\n memhub logs [--since <15m|2h|7d|RFC3339>] [--session <id>] [--trace <id>] [--incident <id>] [--operation <id>] [--request <id>] [--job <id>] [--task <id>] [--component <name>] [--event <name>] [--errors-only] [--tail <1..5000>] [--follow] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--since <time>",
|
||||
@@ -226,64 +239,46 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
"以 NDJSON 输出到 stdout(每行一个脱敏事件;非单 JSON 文档),供脚本与 Agent 消费。",
|
||||
},
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE],
|
||||
exit_codes: USAGE_EXITS,
|
||||
},
|
||||
{
|
||||
name: "init",
|
||||
summary:
|
||||
"首次接入:选择来源 → 选择是否导入历史。接入即开启该 Harness 的处理;后续可用 sync pause/resume 冻结或恢复。在终端且不带 --source 时进入交互引导(默认全选);非终端或加 --json 时不读 stdin、只列出可用来源和下一步。--all-detected 一次接入所有已检测来源。",
|
||||
"首次接入引导。在终端里(未加 --json)进入交互向导:选择来源 → 是否开启处理 → 配置模型。非终端或加 --json 时不读 stdin,只列出可接入的来源和下一步命令。脚本接入请用 sources connect。",
|
||||
usage: "memhub init [--json]",
|
||||
options: [JSON_OPTION],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "sources",
|
||||
summary:
|
||||
"列出来源,直接接入来源,或按 request id 查接入回执。connect 不加 --import 只授权(处理保持冻结:不导入历史、不扫描新会话);加 --import 同时导入历史并开启持续同步,返回 request id。目录来自环境变量覆盖的来源必须加 --confirm-overrides。",
|
||||
usage:
|
||||
"memhub init [--json]\n memhub init --source <id> [--source <id> ...] [--import] [--dry-run] [--full-rescan] [--confirm-overrides] [--json]\n memhub init --all-detected [--import] [--dry-run] [--json]",
|
||||
"memhub sources list [--json]\n memhub sources connect <id> [<id> ...] [--import] [--confirm-overrides] [--request-id <uuid>] [--json]\n memhub sources connect --all-detected [--import] [--confirm-overrides] [--json]\n memhub sources receipt <request-id> [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--source <id>",
|
||||
summary: "要接入的来源,可重复;终端下不给则进入交互引导。",
|
||||
flag: "--import",
|
||||
summary:
|
||||
"接入后立即导入历史会话并开启持续同步;不加时只授权,处理保持冻结。",
|
||||
},
|
||||
{
|
||||
flag: "--all-detected",
|
||||
summary:
|
||||
"接入并(加 --import 时)导入全部 location_state=found 的来源。",
|
||||
},
|
||||
{
|
||||
flag: "--import",
|
||||
summary: "接入后立即导入该来源的历史会话(默认只接入不导入)。",
|
||||
},
|
||||
{
|
||||
flag: "--dry-run",
|
||||
summary: "只出计划,不接入也不导入;用于先看会做什么。",
|
||||
},
|
||||
{
|
||||
flag: "--full-rescan",
|
||||
summary: "导入时忽略既有游标做全量重扫。",
|
||||
"接入全部已检测到(location_state=found)的来源;与具体来源互斥。",
|
||||
},
|
||||
{
|
||||
flag: "--confirm-overrides",
|
||||
summary: "确认接受环境覆盖类来源(如 DSH_HOME 指定的目录)。",
|
||||
summary:
|
||||
"确认接受目录来自环境变量覆盖的来源(如 DSH_HOME 指定的目录);不加时这些来源被拒绝(指定来源时)或跳过并列出(--all-detected 时)。",
|
||||
},
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "setup",
|
||||
summary: "生成无头首次导入计划;只读取已有来源,不探测目录、不写授权。",
|
||||
usage: "memhub setup plan --source <id> [--source <id> ...] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--source <id>",
|
||||
summary: "计划导入的来源,可重复 1..32 次。",
|
||||
flag: "--request-id <uuid>",
|
||||
summary:
|
||||
"--import 的幂等键;用同一个 id 重试只会读回第一次的回执,不会重复导入。",
|
||||
},
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "sources",
|
||||
summary: "列出来源、执行已确认的导入计划。",
|
||||
usage:
|
||||
"memhub sources list [--json]\n memhub sources import --plan-file <path> --confirm [--json]\n memhub sources receipt <request-id> [--json]",
|
||||
options: [PLAN_FILE_OPTION, EXPLICIT_CONFIRM_OPTION, JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "sync",
|
||||
@@ -297,7 +292,7 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
},
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "progress",
|
||||
@@ -318,17 +313,17 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
INTERVAL_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "jobs",
|
||||
summary:
|
||||
"启动或控制单个导入任务:start 创建 history/sync 作业;pause/resume/cancel 控制已有作业。所有写操作先出计划,必须显式 --confirm;list/show 为只读。",
|
||||
"查看或控制导入任务:start 创建 history/sync 任务;pause/resume/cancel 控制已有任务。都直接执行。",
|
||||
usage:
|
||||
"memhub jobs list [--json]\n" +
|
||||
" memhub jobs show <job-id> [--json]\n" +
|
||||
" memhub jobs start --source <id> [--operation <history|sync>] [--confirm] [--json]\n" +
|
||||
" memhub jobs <pause|resume|cancel> --job <id> [--confirm] [--json]",
|
||||
" memhub jobs start --source <id> [--operation <history|sync>] [--json]\n" +
|
||||
" memhub jobs <pause|resume|cancel> --job <id> [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--source <id>",
|
||||
@@ -336,123 +331,116 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
},
|
||||
{
|
||||
flag: "--operation <history|sync>",
|
||||
summary: "start 的导入类型,默认 history。",
|
||||
summary:
|
||||
"start 的导入类型,默认 history;sync 与 scan --source 相同,立即增量同步一次。",
|
||||
},
|
||||
{
|
||||
flag: "--job <id>",
|
||||
summary: "pause/resume/cancel 的目标导入任务。",
|
||||
},
|
||||
DIRECT_CONFIRM_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "models",
|
||||
summary:
|
||||
"交互配置模型,或生成配置/探测计划、执行已确认操作与查询回执;脚本模式的凭据只从受控文件读取。",
|
||||
"管理生成模型配置。add/test 直接提交并返回操作 ID(异步,用 receipt 查结果);edit/activate 直接执行;remove 不可逆,需 --yes。<profile-id>/<operation-id> 都是 UUID;脚本模式的凭据只从受控文件读取。",
|
||||
usage:
|
||||
"memhub models list [--json]\n memhub models show <profile-id> [--json]\n memhub models plan --config-file <path> [--json]\n memhub models configure --plan-file <path> --confirm [--api-key-file <path>] [--json]\n memhub models receipt <operation-id> [--json]\n memhub models edit <profile-id> --config-file <path> [--confirm] [--api-key-file <path>] [--json]\n memhub models activate <profile-id> [--confirm] [--json]\n memhub models probe-plan <profile-id> [--json]\n memhub models probe --plan-file <path> --confirm [--json]\n memhub models probe-receipt <operation-id> [--json]\n memhub models remove <profile-id> [--confirm] [--json]",
|
||||
"memhub models list [--json]\n memhub models show <profile-id> [--json]\n memhub models add --config-file <path> [--api-key-file <path>] [--activate] [--json]\n memhub models test <profile-id> [--json]\n memhub models receipt <operation-id> [--json]\n memhub models edit <profile-id> --config-file <path> [--api-key-file <path>] [--json]\n memhub models activate <profile-id> [--json]\n memhub models remove <profile-id> [--yes] [--json]",
|
||||
alternative_usage: "memhub models setup",
|
||||
options: [
|
||||
{
|
||||
flag: "--config-file <path>",
|
||||
summary: "不含 api_key 的有界 JSON 模型配置。",
|
||||
summary:
|
||||
"有界 JSON 模型配置,不能含 api_key;edit 的补丁必须带 expected_revision。",
|
||||
},
|
||||
PLAN_FILE_OPTION,
|
||||
EXPLICIT_CONFIRM_OPTION,
|
||||
{
|
||||
flag: "--api-key-file <path>",
|
||||
summary: "从 regular file 读取凭据;内容不进入参数、输出或错误。",
|
||||
summary: "从属主专用的普通文件读取凭据;内容不进入参数、输出或错误。",
|
||||
},
|
||||
{
|
||||
flag: "--activate",
|
||||
summary: "add 成功后把新配置设为当前使用。",
|
||||
},
|
||||
DESTRUCTIVE_CONFIRM_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "memories",
|
||||
summary: "通过 SDK 数据面列出、查看、修改或物理删除记忆。",
|
||||
summary:
|
||||
"列出、搜索、查看、添加、修改或物理删除记忆(对话、工作、编码三个场景)。add 和 App 的「添加记忆」是同一个入口;delete 不可逆,需 --yes。",
|
||||
usage:
|
||||
"memhub memories list [--status <state>] [--offset <n>] [--limit <n>] [--json]\n memhub memories show <memory-id> [--json]\n memhub memories update <memory-id> --content-file <path> [--confirm] [--json]\n memhub memories delete <memory-id> [--confirm] [--json]",
|
||||
"memhub memories list [--scene <all|chat|working|coding>] [--source <id>] [--query <text>] [--status <state>] [--offset <n>] [--limit <n>] [--json]\n memhub memories show <memory-id> [--kind <kind>] [--variant <variant>] [--json]\n memhub memories add <text> [--request-id <uuid>] [--json]\n memhub memories add --content-file <path> [--request-id <uuid>] [--json]\n memhub memories add-status <request-id> [--json]\n memhub memories update <memory-id> --content-file <path> [--kind <kind>] [--variant <variant>] [--task <text>] [--json]\n memhub memories delete <memory-id> [--kind <kind>] [--variant <variant>] [--yes] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--scene <all|chat|working|coding>",
|
||||
summary: "只列一个场景,默认 all。working/coding 列的是该场景的经验。",
|
||||
},
|
||||
{ flag: "--source <id>", summary: "只列某个来源写入的记忆。" },
|
||||
{
|
||||
flag: "--query <text>",
|
||||
summary:
|
||||
"关键词搜索(只搜 active 记忆,在最相关的一批里匹配所有关键词,输出 search_window 时表示可能少计)。",
|
||||
},
|
||||
{
|
||||
flag: "--status <state>",
|
||||
summary: `记忆状态,取值 ${MEMORY_STATUSES.join("、")}。`,
|
||||
summary: `记忆状态,取值 ${MEMORY_STATUSES.join("、")},默认 active。`,
|
||||
},
|
||||
{ flag: "--offset <n>", summary: "跳过前 n 条,默认 0。" },
|
||||
{
|
||||
flag: "--limit <n>",
|
||||
summary: "本页最多返回多少条,默认 100,上限 500。",
|
||||
},
|
||||
KIND_OPTION,
|
||||
VARIANT_OPTION,
|
||||
{
|
||||
flag: "--content-file <path>",
|
||||
summary:
|
||||
"从有界普通文件读取新增/纠正正文;计划只保存摘要,执行时再次校验。",
|
||||
"从有界普通文件读取正文:add 的新记忆(≤12000 字),或 update 的新内容(经验为 solution)。",
|
||||
},
|
||||
DIRECT_CONFIRM_OPTION,
|
||||
{
|
||||
flag: "--task <text>",
|
||||
summary: "update 经验(--kind experience)时同时改写 task。",
|
||||
},
|
||||
{
|
||||
flag: "--request-id <uuid>",
|
||||
summary:
|
||||
"add 的幂等键;用同一个 id 重试只会读回第一次的结果,不会重复添加。不给时自动生成并输出。",
|
||||
},
|
||||
DESTRUCTIVE_CONFIRM_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "profile",
|
||||
summary: "读取 SDK 组装后的用户长期画像(L0 + 已启用的长期画像补充)。",
|
||||
usage: "memhub profile [--json]",
|
||||
options: [DATA_DIR_OPTION, JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
options: [JSON_OPTION],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "calls",
|
||||
name: "graph-width",
|
||||
summary:
|
||||
"清除本机调用记录;不可逆删除,默认只出计划,必须显式 --confirm 才执行。",
|
||||
"嵌入维度变化、图谱宽度不匹配时的处置决定。keep_existing 与 export_first 直接执行;rebuild_with_new 会归档旧图谱并重建,需 --yes。",
|
||||
usage:
|
||||
"memhub calls clear --event <id> [--event <id> ...] [--confirm] [--json]\n memhub calls clear --all [--confirm] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--event <id>",
|
||||
summary: "要清除的调用记录事件 ID,可重复,最多 100 条。",
|
||||
},
|
||||
{
|
||||
flag: "--all",
|
||||
summary: "清除全部调用记录;与 --event 互斥。",
|
||||
},
|
||||
DIRECT_CONFIRM_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "notes",
|
||||
summary:
|
||||
"端内新增记忆(memory-notes):从普通文件新增,或显式确认后处理已有笔记。",
|
||||
usage:
|
||||
"memhub notes add --content-file <path> [--json]\n memhub notes apply --plan-file <path> --content-file <path> --confirm [--json]\n memhub notes process <note-id> [--confirm] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--content-file <path>",
|
||||
summary: "从有界普通文件读取新增正文;计划只保存摘要,执行时重新校验。",
|
||||
},
|
||||
PLAN_FILE_OPTION,
|
||||
DIRECT_CONFIRM_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
},
|
||||
{
|
||||
name: "graph",
|
||||
summary: "图谱宽度不匹配时的用户决定;默认只出计划,--confirm 后才执行。",
|
||||
usage:
|
||||
"memhub graph width --decision <keep_existing|rebuild_with_new|export_first> [--confirm] [--json]",
|
||||
"memhub graph-width --decision <keep_existing|export_first|rebuild_with_new> [--yes] [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--decision <decision>",
|
||||
summary:
|
||||
"keep_existing(保持)、export_first(先导出)、rebuild_with_new(导出后重建)。",
|
||||
"keep_existing(保持现有图谱)、export_first(先导出)、rebuild_with_new(导出后归档旧图谱并按新维度重建)。",
|
||||
},
|
||||
{
|
||||
...DESTRUCTIVE_CONFIRM_OPTION,
|
||||
summary: "rebuild_with_new 必须加;其它决定不需要。",
|
||||
},
|
||||
DIRECT_CONFIRM_OPTION,
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "host",
|
||||
@@ -480,12 +468,12 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
},
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "recall",
|
||||
summary:
|
||||
"从当前授权的原文与治理记忆统一检索上下文;本地只读,不调用模型、不触发补索引。",
|
||||
"从原始对话与已沉淀记忆统一检索上下文;本地只读,不调用模型、不触发补索引。文本输出标明每条是「原始对话摘录」还是「记忆」(JSON 里是 kind: raw|memory)。",
|
||||
usage:
|
||||
"memhub recall <query> [--kind <raw|memory>] [--source <id>] [--limit <n>] [--character-budget <n>] [--timeout-ms <n>] [--json]\n memhub recall --query-file <path> [...]\n memhub recall --stdin [...]",
|
||||
options: [
|
||||
@@ -516,21 +504,21 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
},
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_RUNTIME_STOPPED],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "service-plan",
|
||||
summary: "生成用户级服务计划;只输出计划,不写文件也不执行命令。",
|
||||
usage: `memhub service-plan ${SERVICE_USAGE}`,
|
||||
options: SERVICE_OPTIONS,
|
||||
exit_codes: [EXIT_OK, EXIT_PLAN_BLOCKED],
|
||||
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_PLAN_BLOCKED],
|
||||
},
|
||||
{
|
||||
name: "service-install",
|
||||
summary: "预览用户级服务安装;只有 --yes 才写入服务文件并注册。",
|
||||
usage: `memhub service-install ${SERVICE_USAGE} [--yes]`,
|
||||
options: [...SERVICE_OPTIONS, CONFIRM_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_PLAN_BLOCKED],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_PLAN_BLOCKED],
|
||||
},
|
||||
{
|
||||
name: "service-uninstall",
|
||||
@@ -538,15 +526,15 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
"预览用户级服务卸载;只有 --yes 才注销并删除逐字匹配的受管服务文件,不删数据。",
|
||||
usage: `memhub service-uninstall ${SERVICE_USAGE} [--yes]`,
|
||||
options: [...SERVICE_OPTIONS, CONFIRM_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_PLAN_BLOCKED],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_PLAN_BLOCKED],
|
||||
},
|
||||
{
|
||||
name: "install",
|
||||
summary:
|
||||
"预览发行目录安装、memhub/memhub-cli/one-memory 短链接、PATH 条目与用户服务计划;只有 --yes 才执行并在失败时回滚。",
|
||||
"预览发行目录安装、memhub/memhub-cli/one-memory 短链接、PATH 条目与用户服务计划;只有 --yes 才执行。记忆数据迁移开始前失败会回滚;迁移开始后失败保留新版本、拒绝启动,并输出迁移日志与反馈地址。",
|
||||
usage: `memhub install ${INSTALL_USAGE}`,
|
||||
options: INSTALL_OPTIONS,
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_PLAN_BLOCKED],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_PLAN_BLOCKED],
|
||||
},
|
||||
{
|
||||
name: "uninstall",
|
||||
@@ -559,24 +547,31 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
{
|
||||
name: "upgrade",
|
||||
summary:
|
||||
"预览升级;只有 --yes 才切换受管版本并重启服务,失败时恢复旧版本,不删数据。",
|
||||
"预览升级;只有 --yes 才切换受管版本并重启服务,不删数据。记忆数据迁移开始前失败会恢复旧版本;迁移开始后失败保留新版本、拒绝启动,并输出迁移日志与反馈地址(不会回滚到旧版本)。",
|
||||
usage: `memhub upgrade ${INSTALL_USAGE}`,
|
||||
options: INSTALL_OPTIONS,
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_PLAN_BLOCKED],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_PLAN_BLOCKED],
|
||||
},
|
||||
{
|
||||
name: "delete-data",
|
||||
summary:
|
||||
"独立、二次确认地删除 One Memory Home(数据)。这是唯一会删除数据的命令;uninstall 不会。默认只出预览;需 Runtime 已停止,并显式 --confirm <path> 复述该目录。",
|
||||
"删除 MemHub 数据目录(记忆、来源、配置),不可逆。这是唯一会删除数据的命令;uninstall 不会。需要 Runtime 已停止;不加参数时只预览要删除的目录;删除需要 --yes 并用 --confirm 再写一遍同一个目录。",
|
||||
usage:
|
||||
"memhub delete-data [--data-dir <path>] [--yes --confirm <path>] [--json]",
|
||||
options: [
|
||||
DATA_DIR_OPTION,
|
||||
CONFIRM_OPTION,
|
||||
{
|
||||
flag: "--data-dir <path>",
|
||||
summary: "要删除的数据目录绝对路径;默认当前使用的数据目录。",
|
||||
},
|
||||
{
|
||||
...DESTRUCTIVE_CONFIRM_OPTION,
|
||||
summary:
|
||||
"删除时必须加,且要同时给出 --confirm <path>;不加时只预览,exit 0。",
|
||||
},
|
||||
{
|
||||
flag: "--confirm <path>",
|
||||
summary:
|
||||
"第二次确认:逐字复述预览里的数据目录;缺少它时即使 --yes 也不删除。",
|
||||
"删除时必须加:再写一遍要删除的数据目录,与预览中的目录完全一致才执行。",
|
||||
},
|
||||
JSON_OPTION,
|
||||
],
|
||||
@@ -588,113 +583,149 @@ export const COMMANDS: readonly CommandDescriptor[] = [
|
||||
"onboarding reset 回到首次运行的引导流程。只清除引导完成状态:不删记忆数据,也不需要卸载重装。",
|
||||
usage: "memhub onboarding reset [--json]",
|
||||
options: [JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "help",
|
||||
summary: "显示总帮助或单个命令的用法。",
|
||||
usage: "memhub help [command] [--json]",
|
||||
options: [JSON_OPTION],
|
||||
exit_codes: [EXIT_OK],
|
||||
exit_codes: [EXIT_OK, EXIT_USAGE],
|
||||
},
|
||||
{
|
||||
name: "version",
|
||||
summary: "显示 CLI 版本。",
|
||||
usage: "memhub version [--json]",
|
||||
options: [JSON_OPTION],
|
||||
exit_codes: [EXIT_OK],
|
||||
exit_codes: [EXIT_OK, EXIT_USAGE],
|
||||
},
|
||||
{
|
||||
name: "path",
|
||||
name: "scan-roots",
|
||||
summary:
|
||||
"查看或登记被扫描的目录(scan roots)。list 只读;add/remove 预览后加 --yes 才提交;均已接入 Runtime 的 scan-roots 端点。",
|
||||
"查看或登记被扫描的目录(scan roots)。add/remove 直接执行。(旧名 path 仍可用。)",
|
||||
usage:
|
||||
"memhub path list [--json]\n" +
|
||||
" memhub path add --root <path> [--source <id>] [--no-recursive] [--yes]\n" +
|
||||
" memhub path remove --root <path> [--yes]",
|
||||
options: [DATA_DIR_OPTION, JSON_OPTION, CONFIRM_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED, EXIT_PLAN_BLOCKED],
|
||||
"memhub scan-roots list [--json]\n" +
|
||||
" memhub scan-roots add --root <path> [--source <id>] [--no-recursive] [--json]\n" +
|
||||
" memhub scan-roots remove --root <path> [--json]",
|
||||
options: [
|
||||
{ flag: "--root <path>", summary: "要登记或移除的目录。" },
|
||||
{
|
||||
flag: "--source <id>",
|
||||
summary: "把目录绑定到某个来源;不给时由 Runtime 自动识别。",
|
||||
},
|
||||
{ flag: "--no-recursive", summary: "只扫描该目录本身,不进入子目录。" },
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "adapter",
|
||||
summary:
|
||||
"查看 Harness Adapter 的就绪状态与最近一次探测结果(list),或现场探测某个来源(probe --source <id>);两者均为只读,已接入 Runtime 的 adapters/探测端点。",
|
||||
"查看 Harness Adapter 的就绪状态与最近一次探测结果(list),或现场探测某个来源(probe --source <id>);两者均为只读。",
|
||||
usage:
|
||||
"memhub adapter list [--json]\n" +
|
||||
" memhub adapter probe --source <id> [--json]",
|
||||
options: [DATA_DIR_OPTION, JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED],
|
||||
options: [
|
||||
{ flag: "--source <id>", summary: "probe 的目标来源。" },
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "scan",
|
||||
summary:
|
||||
"立即扫描;默认只输出计划,加 --yes 才真正执行。指定 --source 扫描单个来源,--all 一次扫描/导入全部已检测来源(Runtime 按来源寻址,不接受裸路径)。",
|
||||
usage:
|
||||
"memhub scan [--source <id>] [--full] [--yes] [--json]\n memhub scan --all [--yes] [--json]",
|
||||
"立即增量扫描一个已连接并开启同步的来源,返回导入任务(与 jobs start --operation sync 相同)。接入全部来源请用 sources connect --all-detected --import。",
|
||||
usage: "memhub scan --source <id> [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--source <id>",
|
||||
summary: "按单个来源扫描;与 --all 互斥。",
|
||||
summary: "要扫描的来源。",
|
||||
},
|
||||
{
|
||||
flag: "--all",
|
||||
summary:
|
||||
"处理全部 location_state=found 的来源,按来源逐条回执;部分成功返回非 0。",
|
||||
},
|
||||
{
|
||||
flag: "--full",
|
||||
summary: "忽略既有游标做全量重扫;默认只读取增量。",
|
||||
},
|
||||
DATA_DIR_OPTION,
|
||||
JSON_OPTION,
|
||||
CONFIRM_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED, EXIT_PLAN_BLOCKED],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "write",
|
||||
summary: "提交 OpenAI messages 到 SDK 写入队列,或按 request id 查询状态。",
|
||||
usage:
|
||||
"memhub write submit --messages-file <path> --session <id> [--source <id>] [--request-id <id>] [--memory-at <RFC3339>] [--live] [--json]\n memhub write status --id <request-id> [--json]",
|
||||
options: [DATA_DIR_OPTION, JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED],
|
||||
},
|
||||
{
|
||||
name: "task",
|
||||
summary:
|
||||
"兼容别名:按 request id 查询写入状态;新脚本请使用 write status。",
|
||||
usage: "memhub task --id <request-id> [--json]",
|
||||
options: [DATA_DIR_OPTION, JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED],
|
||||
"提交一段 OpenAI 格式的对话(messages)到写入队列,异步加工成记忆;或按 request id 查询状态。request id 是幂等键:不给时自动生成并输出,重试请带上同一个。新增一条结论式记忆请用 memories add。",
|
||||
usage:
|
||||
"memhub write submit --messages-file <path> --session <id> [--source <id>] [--request-id <id>] [--memory-at <RFC3339>] [--live] [--json]\n memhub write status <request-id> [--json]",
|
||||
options: [
|
||||
{
|
||||
flag: "--messages-file <path>",
|
||||
summary:
|
||||
"OpenAI messages 数组(或 {messages:[…]}),1..200 条,role 为 user/assistant/tool,至少一条 user。",
|
||||
},
|
||||
{ flag: "--session <id>", summary: "会话 ID。" },
|
||||
{ flag: "--source <id>", summary: "来源应用,默认 memhub-cli。" },
|
||||
{
|
||||
flag: "--request-id <id>",
|
||||
summary:
|
||||
"幂等键:同 id 同内容不会重复入队,同 id 不同内容被拒绝(409)。",
|
||||
},
|
||||
{ flag: "--memory-at <RFC3339>", summary: "对话发生时间,必须带时区。" },
|
||||
{ flag: "--live", summary: "优先处理。" },
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: RUNTIME_EXITS,
|
||||
},
|
||||
{
|
||||
name: "backup",
|
||||
summary:
|
||||
"备份与恢复 MemHub 数据(两阶段)。create 调用 Runtime 的备份端点;restore 会覆盖现有数据,要求 Runtime 已停止且 `--confirm` 逐字重复 `--source`,随后由非 Runtime 维护态 Core 完成受管校验、保护性备份和原子替换。",
|
||||
"备份与恢复主库(来源、写入队列、配置)。注意:不包含记忆本身——SDK 记忆库 hy-memory/ 不在备份里,restore 也不会替换它。create 的目标要放在 <数据目录>/backups/ 下才能被 restore 使用;restore 覆盖现有主库,需 Runtime 已停止,并加 --yes 与 --confirm <同一个备份路径>。",
|
||||
usage:
|
||||
"memhub backup create --target <path> [--yes] [--json]\n" +
|
||||
" memhub backup restore --source <path> --yes --confirm <path> [--json]",
|
||||
"memhub backup create --target <path> [--json]\n" +
|
||||
" memhub backup restore --source <path> [--yes --confirm <path>] [--json]",
|
||||
options: [
|
||||
DATA_DIR_OPTION,
|
||||
JSON_OPTION,
|
||||
CONFIRM_OPTION,
|
||||
{
|
||||
flag: "--target <path>",
|
||||
summary: "create 的备份文件路径,必须在数据目录内。",
|
||||
},
|
||||
{
|
||||
flag: "--source <path>",
|
||||
summary:
|
||||
"restore 的备份文件,必须是 <数据目录>/backups/ 下的受管备份。",
|
||||
},
|
||||
{
|
||||
...DESTRUCTIVE_CONFIRM_OPTION,
|
||||
summary:
|
||||
"restore 必须加,且要同时给出 --confirm <path>;create 不需要。",
|
||||
},
|
||||
{
|
||||
flag: "--confirm <path>",
|
||||
summary: "restore 要求逐字重复 --source 路径;不一致时不做任何改动。",
|
||||
summary:
|
||||
"restore 必须加:再写一遍 --source 的备份路径,完全一致才执行。",
|
||||
},
|
||||
JSON_OPTION,
|
||||
],
|
||||
exit_codes: [EXIT_OK, EXIT_USAGE, EXIT_FAILED, EXIT_PLAN_BLOCKED],
|
||||
exit_codes: [...RUNTIME_EXITS, EXIT_PLAN_BLOCKED],
|
||||
},
|
||||
{
|
||||
name: "ui",
|
||||
summary:
|
||||
"用系统默认浏览器打开 Runtime 托管的 Web Console:先向 Core 换取一次性 loopback 登录码,再交给系统浏览器由 Core 置 HttpOnly/SameSite=Strict session;CLI 不向浏览器暴露裸 Core token。Core 未提供该合同时如实返回 ui_session_contract_unavailable(exit 4)。",
|
||||
"用系统默认浏览器打开 Runtime 托管的 Web Console:先向 Core 换取一次性 loopback 登录码,再交给系统浏览器由 Core 置 HttpOnly/SameSite=Strict session;CLI 不向浏览器暴露裸 Core token。Runtime 未运行时 exit 3;Core 未提供该合同时如实返回 ui_session_contract_unavailable(exit 4)。",
|
||||
usage: "memhub ui [--json]",
|
||||
options: [JSON_OPTION],
|
||||
exit_codes: [EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_PLAN_BLOCKED],
|
||||
exit_codes: [...RUNTIME_EXITS, EXIT_PLAN_BLOCKED],
|
||||
},
|
||||
];
|
||||
|
||||
/** Old names that still run but are not listed: `path` → `scan-roots`,
|
||||
* `task` → `write status`, `graph width` → `graph-width`. */
|
||||
export const HIDDEN_COMMAND_ALIASES: Readonly<Record<string, string>> = {
|
||||
path: "scan-roots",
|
||||
task: "write",
|
||||
graph: "graph-width",
|
||||
};
|
||||
|
||||
export function findCommand(name: string): CommandDescriptor | undefined {
|
||||
return COMMANDS.find((command) => command.name === name);
|
||||
}
|
||||
|
||||
/** A listed command, or the command a hidden alias now runs as. */
|
||||
export function findCommandOrAlias(
|
||||
name: string,
|
||||
): CommandDescriptor | undefined {
|
||||
return findCommand(HIDDEN_COMMAND_ALIASES[name] ?? name);
|
||||
}
|
||||
@@ -69,7 +69,6 @@ const COMMANDS = [
|
||||
"doctor",
|
||||
"logs",
|
||||
"init",
|
||||
"setup",
|
||||
"sources",
|
||||
"sync",
|
||||
"progress",
|
||||
@@ -77,10 +76,7 @@ const COMMANDS = [
|
||||
"models",
|
||||
"memories",
|
||||
"profile",
|
||||
// C6 App BLOCK (25f4ecf): UI write operations that had no CLI equivalent.
|
||||
"calls",
|
||||
"notes",
|
||||
"graph",
|
||||
"graph-width",
|
||||
"host",
|
||||
"recall",
|
||||
"service-plan",
|
||||
@@ -93,19 +89,20 @@ const COMMANDS = [
|
||||
"onboarding",
|
||||
"help",
|
||||
"version",
|
||||
// A6 product commands. Appended, not inserted: the catalog is a stable
|
||||
// contract, so existing names must not shift position.
|
||||
"path",
|
||||
"scan-roots",
|
||||
"adapter",
|
||||
"scan",
|
||||
"write",
|
||||
"task",
|
||||
"backup",
|
||||
// `ui` is declared but blocked on a Core session contract; it is listed so
|
||||
// `help` documents it and the blocked shape is machine-readable.
|
||||
"ui",
|
||||
] as const;
|
||||
|
||||
/** Old names that still run and still answer `--help`, but are not listed. */
|
||||
const HIDDEN_ALIASES = [
|
||||
["path", "scan-roots"],
|
||||
["task", "write"],
|
||||
] as const;
|
||||
|
||||
describe("CLI-22 shared output contract", () => {
|
||||
describe("help", () => {
|
||||
it.each([["help"], ["--help"], ["-h"]])(
|
||||
@@ -146,6 +143,56 @@ describe("CLI-22 shared output contract", () => {
|
||||
},
|
||||
);
|
||||
|
||||
it.each(HIDDEN_ALIASES)(
|
||||
"documents the hidden alias %s as %s and keeps it out of the listing",
|
||||
async (alias, target) => {
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["help", alias], {})).toBe(0);
|
||||
expect(await run([alias, "--help"], {})).toBe(0);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(streams.stdout()).toContain(`${PRODUCT} ${target}`);
|
||||
const listing = captureStreams();
|
||||
try {
|
||||
expect(await run(["help", "--json"], {})).toBe(0);
|
||||
} finally {
|
||||
listing.restore();
|
||||
}
|
||||
const names = JSON.parse(listing.stdout()).commands.map(
|
||||
(entry: { name: string }) => entry.name,
|
||||
);
|
||||
expect(names).not.toContain(alias);
|
||||
},
|
||||
);
|
||||
|
||||
it("lists no removed command", async () => {
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["help", "--json"], {})).toBe(0);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
const catalog = streams.stdout();
|
||||
for (const removed of [
|
||||
'"name":"setup"',
|
||||
'"name":"notes"',
|
||||
'"name":"calls"',
|
||||
'"name":"graph"',
|
||||
"--plan-file",
|
||||
"probe-plan",
|
||||
"probe-receipt",
|
||||
"models configure",
|
||||
"sources import",
|
||||
"--full-rescan",
|
||||
"--dry-run",
|
||||
"--full]",
|
||||
]) {
|
||||
expect(catalog).not.toContain(removed);
|
||||
}
|
||||
});
|
||||
|
||||
it("never executes the documented command", async () => {
|
||||
for (const command of COMMANDS) {
|
||||
for (const argv of [
|
||||
@@ -201,27 +248,63 @@ describe("CLI-22 shared output contract", () => {
|
||||
expect(payload.commands).toBeUndefined();
|
||||
});
|
||||
|
||||
it("documents the destructive restore confirmation explicitly", async () => {
|
||||
it("documents the one --yes convention for destructive commands", async () => {
|
||||
const general = captureStreams();
|
||||
try {
|
||||
expect(await run(["help"], {})).toBe(0);
|
||||
} finally {
|
||||
general.restore();
|
||||
}
|
||||
expect(general.stdout()).toContain("确认约定");
|
||||
for (const [command, usage] of [
|
||||
[
|
||||
"backup",
|
||||
"memhub backup restore --source <path> [--yes --confirm <path>]",
|
||||
],
|
||||
["memories", "memhub memories delete <memory-id>"],
|
||||
["models", "memhub models remove <profile-id> [--yes]"],
|
||||
[
|
||||
"delete-data",
|
||||
"memhub delete-data [--data-dir <path>] [--yes --confirm <path>]",
|
||||
],
|
||||
["graph-width", "[--yes]"],
|
||||
] as const) {
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["help", command], {})).toBe(0);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(streams.stdout()).toContain(usage);
|
||||
// Only the two commands with no way back repeat the target path.
|
||||
if (command !== "backup" && command !== "delete-data") {
|
||||
expect(streams.stdout()).not.toContain("--confirm <path>");
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("says backup does not include the memories themselves", async () => {
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["help", "backup"], {})).toBe(0);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(streams.stdout()).toContain(
|
||||
"memhub backup restore --source <path> --yes --confirm <path>",
|
||||
);
|
||||
expect(streams.stdout()).toContain("逐字重复 --source");
|
||||
expect(streams.stdout()).toContain("不包含记忆本身");
|
||||
});
|
||||
|
||||
it("does not attach restore's confirmation flag to path", async () => {
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["help", "path"], {})).toBe(0);
|
||||
} finally {
|
||||
streams.restore();
|
||||
it("does not promise a rollback after the migration started", async () => {
|
||||
for (const command of ["install", "upgrade"]) {
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["help", command], {})).toBe(0);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(streams.stdout()).toContain("迁移开始后失败保留新版本");
|
||||
expect(streams.stdout()).not.toContain("失败时恢复旧版本");
|
||||
expect(streams.stdout()).not.toContain("失败时回滚");
|
||||
}
|
||||
expect(streams.stdout()).not.toContain("--confirm <path>");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -545,22 +628,23 @@ describe("documented commands", () => {
|
||||
models: [
|
||||
"list",
|
||||
"show",
|
||||
"plan",
|
||||
"configure",
|
||||
"add",
|
||||
"test",
|
||||
"receipt",
|
||||
"probe-plan",
|
||||
"probe",
|
||||
"probe-receipt",
|
||||
"edit",
|
||||
"activate",
|
||||
"remove",
|
||||
],
|
||||
// Source processing is one Freeze/Thaw switch under `sync`; the obsolete
|
||||
// independent read/write `sources access` command must not return here.
|
||||
sources: ["list", "import", "receipt"],
|
||||
sources: ["list", "connect", "receipt"],
|
||||
jobs: ["list", "show", "start", "<pause|resume|cancel>"],
|
||||
// `sync <pause|resume>` is documented as one alternation, so it is
|
||||
// checked as an entry rather than as two literal subcommands.
|
||||
sync: ["status", "<pause|resume>"],
|
||||
memories: ["list", "show", "update", "delete"],
|
||||
memories: ["list", "show", "add", "add-status", "update", "delete"],
|
||||
"scan-roots": ["list", "add", "remove"],
|
||||
write: ["submit", "status"],
|
||||
host: ["list", "status", "install", "uninstall", "sync"],
|
||||
};
|
||||
|
||||
|
||||
@@ -43,61 +43,35 @@ const ATTACHED = {
|
||||
};
|
||||
|
||||
const RESPONSE = {
|
||||
schema_version: 1,
|
||||
schema_version: 2,
|
||||
request_id: "context-request-1",
|
||||
state: "grounded",
|
||||
paths: {
|
||||
raw: {
|
||||
status: "ok",
|
||||
reason: null,
|
||||
elapsed_ms: 2,
|
||||
returned_count: 1,
|
||||
truncated: false,
|
||||
truncated_by: [],
|
||||
},
|
||||
memory: {
|
||||
status: "ok",
|
||||
reason: null,
|
||||
elapsed_ms: 3,
|
||||
returned_count: 1,
|
||||
truncated: false,
|
||||
truncated_by: [],
|
||||
},
|
||||
},
|
||||
items: [
|
||||
{
|
||||
kind: "raw",
|
||||
source_id: "codex",
|
||||
conversation_id: "conversation-1",
|
||||
anchor_evidence_id: "evidence-1",
|
||||
messages: [
|
||||
{
|
||||
evidence_id: "evidence-1",
|
||||
revision: 1,
|
||||
role: "user",
|
||||
written_by: "user",
|
||||
created_at: "2026-09-19T00:00:00Z",
|
||||
text: "Synthetic raw citation.",
|
||||
truncated: false,
|
||||
},
|
||||
],
|
||||
raw_id: "raw-1",
|
||||
content: "Synthetic raw citation.",
|
||||
source: "codex",
|
||||
session_id: "session-1",
|
||||
memory_at: "2026-09-19T00:00:00Z",
|
||||
score: 0.8,
|
||||
truncated: false,
|
||||
},
|
||||
{
|
||||
kind: "memory",
|
||||
memory_id: "memory-1",
|
||||
revision: 2,
|
||||
content: "Synthetic governed memory.",
|
||||
memory_kind: "working",
|
||||
content: "Synthetic SDK memory.",
|
||||
layer: "l2_fact",
|
||||
status: "active",
|
||||
source: "codex",
|
||||
updated_at: "2026-09-19T00:00:01Z",
|
||||
evidence_ids: ["evidence-1"],
|
||||
score: 0.7,
|
||||
truncated: false,
|
||||
},
|
||||
],
|
||||
used_characters: 49,
|
||||
used_characters: 44,
|
||||
truncated: false,
|
||||
total_ms: 5,
|
||||
adoption: "unknown",
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
@@ -115,42 +89,17 @@ beforeEach(() => {
|
||||
});
|
||||
|
||||
describe("unified context search", () => {
|
||||
it("accepts an empty tool message in a valid partial context response", async () => {
|
||||
it("accepts an empty raw item in a valid partial context response", async () => {
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: {
|
||||
...RESPONSE,
|
||||
state: "partial",
|
||||
paths: {
|
||||
raw: {
|
||||
...RESPONSE.paths.raw,
|
||||
status: "degraded",
|
||||
reason: "lexical_only",
|
||||
returned_count: 1,
|
||||
},
|
||||
memory: {
|
||||
...RESPONSE.paths.memory,
|
||||
status: "empty",
|
||||
returned_count: 0,
|
||||
},
|
||||
},
|
||||
items: [
|
||||
{
|
||||
kind: "raw",
|
||||
source_id: "workbuddy",
|
||||
conversation_id: "conversation-1",
|
||||
anchor_evidence_id: "evidence-1",
|
||||
messages: [
|
||||
{
|
||||
evidence_id: "evidence-1",
|
||||
revision: 1,
|
||||
role: "tool",
|
||||
written_by: "tool",
|
||||
created_at: "2026-10-05T20:48:12Z",
|
||||
text: "",
|
||||
truncated: true,
|
||||
},
|
||||
],
|
||||
...RESPONSE.items[0],
|
||||
source: "workbuddy",
|
||||
content: "",
|
||||
truncated: true,
|
||||
},
|
||||
],
|
||||
@@ -165,8 +114,9 @@ describe("unified context search", () => {
|
||||
streams.restore();
|
||||
}
|
||||
expect(JSON.parse(streams.stdout())).toMatchObject({
|
||||
schema_version: 2,
|
||||
state: "partial",
|
||||
items: [{ kind: "raw", messages: [{ text: "" }] }],
|
||||
items: [{ kind: "raw", content: "" }],
|
||||
});
|
||||
});
|
||||
|
||||
@@ -178,12 +128,26 @@ describe("unified context search", () => {
|
||||
streams.restore();
|
||||
}
|
||||
const output = streams.stdout();
|
||||
expect(output).toContain("Synthetic governed memory.");
|
||||
expect(output).toContain("Synthetic SDK memory.");
|
||||
expect(output).toContain("Synthetic raw citation.");
|
||||
expect(output).not.toMatch(/^- raw |memory-1|evidence-1|采用状态/m);
|
||||
expect(output).not.toMatch(/^- raw |memory-1|raw-1|采用状态/m);
|
||||
});
|
||||
|
||||
it("queries the current read-only context endpoint and preserves citations", async () => {
|
||||
it("labels raw hits as original-conversation excerpts, not memories", async () => {
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["recall", "synthetic question"])).toBe(0);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
const output = streams.stdout();
|
||||
expect(output).toContain(
|
||||
"- [原始对话摘录 · codex] Synthetic raw citation.",
|
||||
);
|
||||
expect(output).toContain("- [记忆] Synthetic SDK memory.");
|
||||
});
|
||||
|
||||
it("queries the SDK context endpoint directly and preserves item ids", async () => {
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(
|
||||
@@ -209,7 +173,7 @@ describe("unified context search", () => {
|
||||
streams.restore();
|
||||
}
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
"/api/v1/context/search",
|
||||
"/api/v2/context/search",
|
||||
{
|
||||
query: "synthetic question",
|
||||
kinds: ["raw", "memory"],
|
||||
@@ -222,12 +186,18 @@ describe("unified context search", () => {
|
||||
);
|
||||
const output = JSON.parse(streams.stdout());
|
||||
expect(output).toMatchObject({
|
||||
schema_version: 2,
|
||||
state: "grounded",
|
||||
request_id: "context-request-1",
|
||||
adoption: "unknown",
|
||||
});
|
||||
expect(output.items[0].messages[0].evidence_id).toBe("evidence-1");
|
||||
expect(output.items[1].evidence_ids).toEqual(["evidence-1"]);
|
||||
expect(output).not.toHaveProperty("paths");
|
||||
expect(output.items[0]).toMatchObject({ kind: "raw", raw_id: "raw-1" });
|
||||
expect(output.items[1]).toMatchObject({
|
||||
kind: "memory",
|
||||
memory_id: "memory-1",
|
||||
layer: "l2_fact",
|
||||
});
|
||||
// No authority pre-query: released Core always serves the SDK data plane.
|
||||
expect(transport.get).not.toHaveBeenCalled();
|
||||
expect(transport.put).not.toHaveBeenCalled();
|
||||
expect(transport.patch).not.toHaveBeenCalled();
|
||||
@@ -242,19 +212,8 @@ describe("unified context search", () => {
|
||||
status: 200,
|
||||
data: {
|
||||
...RESPONSE,
|
||||
paths: {
|
||||
...RESPONSE.paths,
|
||||
raw: {
|
||||
status: "not_requested",
|
||||
reason: "not_requested",
|
||||
elapsed_ms: 0,
|
||||
returned_count: 0,
|
||||
truncated: false,
|
||||
truncated_by: [],
|
||||
},
|
||||
},
|
||||
items: [RESPONSE.items[1]],
|
||||
used_characters: 26,
|
||||
used_characters: 21,
|
||||
},
|
||||
});
|
||||
const streams = captureStreams();
|
||||
@@ -394,29 +353,7 @@ describe("unified context search", () => {
|
||||
status: 200,
|
||||
data: {
|
||||
...RESPONSE,
|
||||
paths: {
|
||||
...RESPONSE.paths,
|
||||
memory: {
|
||||
...RESPONSE.paths.memory,
|
||||
status: "empty",
|
||||
returned_count: 0,
|
||||
},
|
||||
},
|
||||
items: [
|
||||
{
|
||||
...(RESPONSE.items[0] as Record<string, unknown>),
|
||||
messages: [
|
||||
{
|
||||
...(
|
||||
RESPONSE.items[0] as {
|
||||
messages: Array<Record<string, unknown>>;
|
||||
}
|
||||
).messages[0],
|
||||
text,
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
items: [{ ...RESPONSE.items[0], content: text }],
|
||||
used_characters: 10,
|
||||
},
|
||||
});
|
||||
@@ -429,7 +366,7 @@ describe("unified context search", () => {
|
||||
expect(JSON.parse(streams.stdout())).toMatchObject({
|
||||
state: "grounded",
|
||||
used_characters: 10,
|
||||
items: [{ kind: "raw", messages: [{ text }] }],
|
||||
items: [{ kind: "raw", content: text }],
|
||||
});
|
||||
});
|
||||
|
||||
@@ -441,29 +378,7 @@ describe("unified context search", () => {
|
||||
status: 200,
|
||||
data: {
|
||||
...RESPONSE,
|
||||
paths: {
|
||||
...RESPONSE.paths,
|
||||
memory: {
|
||||
...RESPONSE.paths.memory,
|
||||
status: "empty",
|
||||
returned_count: 0,
|
||||
},
|
||||
},
|
||||
items: [
|
||||
{
|
||||
...(RESPONSE.items[0] as Record<string, unknown>),
|
||||
messages: [
|
||||
{
|
||||
...(
|
||||
RESPONSE.items[0] as {
|
||||
messages: Array<Record<string, unknown>>;
|
||||
}
|
||||
).messages[0],
|
||||
text,
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
items: [{ ...RESPONSE.items[0], content: text }],
|
||||
used_characters: 12,
|
||||
},
|
||||
});
|
||||
@@ -479,6 +394,40 @@ describe("unified context search", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it.each([
|
||||
[
|
||||
"a raw item from a source that was not asked for",
|
||||
["--source", "codex"],
|
||||
{ ...RESPONSE.items[0], source: "cursor" },
|
||||
],
|
||||
[
|
||||
"a raw item when only memory was asked for",
|
||||
["--kind", "memory"],
|
||||
RESPONSE.items[0],
|
||||
],
|
||||
["a score outside 0..1", [], { ...RESPONSE.items[1], score: 1.5 }],
|
||||
])("rejects %s as an invalid response", async (_label, flags, item) => {
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: {
|
||||
...RESPONSE,
|
||||
items: [item],
|
||||
used_characters: Array.from(String(item.content)).length,
|
||||
},
|
||||
});
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["recall", "query", ...flags, "--json"])).toBe(1);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(JSON.parse(streams.stdout())).toMatchObject({
|
||||
schema_version: 2,
|
||||
state: "invalid",
|
||||
code: "response_invalid",
|
||||
});
|
||||
});
|
||||
|
||||
it("reports endpoint denial without exposing the response body", async () => {
|
||||
transport.post.mockResolvedValue({
|
||||
status: 403,
|
||||
|
||||
@@ -3,31 +3,6 @@ import { PRODUCT } from "./version.js";
|
||||
|
||||
export const CONTEXT_QUERY_MAX_CHARACTERS = 2_000;
|
||||
const CONTEXT_KINDS = ["raw", "memory"] as const;
|
||||
const PATH_STATES = [
|
||||
"ok",
|
||||
"empty",
|
||||
"not_requested",
|
||||
"not_authorized",
|
||||
"unavailable",
|
||||
"degraded",
|
||||
] as const;
|
||||
const PATH_REASONS = [
|
||||
"not_requested",
|
||||
"permission_required",
|
||||
"source_revoked",
|
||||
"read_unavailable",
|
||||
"timeout_budget",
|
||||
"evidence_changed",
|
||||
"memory_changed",
|
||||
"lexical_only",
|
||||
] as const;
|
||||
const LIMIT_REASONS = [
|
||||
"count_budget",
|
||||
"character_budget",
|
||||
"timeout_budget",
|
||||
"metadata_budget",
|
||||
] as const;
|
||||
|
||||
export interface ContextSearchInput {
|
||||
query: string;
|
||||
kinds: Array<(typeof CONTEXT_KINDS)[number]>;
|
||||
@@ -37,8 +12,9 @@ export interface ContextSearchInput {
|
||||
timeout_ms: number;
|
||||
}
|
||||
|
||||
/** The CLI's JSON document mirrors Core's SDK-native context schema v2. */
|
||||
export interface ContextSearchDocument {
|
||||
schema_version: 1;
|
||||
schema_version: 2;
|
||||
product: string;
|
||||
state:
|
||||
| "grounded"
|
||||
@@ -47,15 +23,14 @@ export interface ContextSearchDocument {
|
||||
| "unavailable"
|
||||
| "denied"
|
||||
| "not_found"
|
||||
| "rejected"
|
||||
| "invalid";
|
||||
code?: string;
|
||||
request_id?: string;
|
||||
paths?: Record<string, unknown>;
|
||||
items: Array<Record<string, unknown>>;
|
||||
used_characters?: number;
|
||||
truncated?: boolean;
|
||||
total_ms?: number;
|
||||
adoption?: "unknown";
|
||||
}
|
||||
|
||||
export async function searchContext(
|
||||
@@ -67,7 +42,7 @@ export async function searchContext(
|
||||
const validated = validateContextResponse(outcome.data, input);
|
||||
return (
|
||||
validated ?? {
|
||||
schema_version: 1,
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
state: "invalid",
|
||||
code: "response_invalid",
|
||||
@@ -123,6 +98,9 @@ export function renderContextText(document: ContextSearchDocument): string {
|
||||
if (document.state === "unavailable") return "暂时无法查记忆,请稍后重试。\n";
|
||||
if (document.state === "denied") return "当前应用没有查记忆权限。\n";
|
||||
if (document.state === "invalid") return "查记忆返回了无法识别的结果。\n";
|
||||
if (document.state === "rejected") {
|
||||
return `查记忆被拒绝(${document.code}),请检查参数。\n`;
|
||||
}
|
||||
const lines = [
|
||||
document.state === "empty"
|
||||
? "没有查到相关记忆。"
|
||||
@@ -130,14 +108,14 @@ export function renderContextText(document: ContextSearchDocument): string {
|
||||
? "查到以下内容,部分内容还在整理中:"
|
||||
: "查到以下相关内容:",
|
||||
];
|
||||
// A raw hit is an excerpt of an original conversation, not a distilled
|
||||
// memory; say which one each line is so it is not read as a conclusion.
|
||||
for (const item of document.items) {
|
||||
if (item.kind === "memory") {
|
||||
lines.push(`- ${String(item.content)}`);
|
||||
} else {
|
||||
for (const message of item.messages as Array<Record<string, unknown>>) {
|
||||
lines.push(`- ${String(message.text)}`);
|
||||
}
|
||||
}
|
||||
const label =
|
||||
item.kind === "raw"
|
||||
? `[原始对话摘录 · ${String(item.source)}]`
|
||||
: "[记忆]";
|
||||
lines.push(`- ${label} ${String(item.content)}`);
|
||||
}
|
||||
if (document.truncated) lines.push("结果已按请求预算截断。");
|
||||
return `${lines.join("\n")}\n`;
|
||||
@@ -149,84 +127,37 @@ function validateContextResponse(
|
||||
): ContextSearchDocument | null {
|
||||
const response = record(value);
|
||||
if (
|
||||
response?.schema_version !== 1 ||
|
||||
response?.schema_version !== 2 ||
|
||||
!oneOf(response.state, ["grounded", "empty", "partial", "unavailable"]) ||
|
||||
!boundedText(response.request_id, 64) ||
|
||||
!record(response.paths) ||
|
||||
!Array.isArray(response.items) ||
|
||||
response.items.length > request.limit ||
|
||||
!integer(response.used_characters, 0, request.character_budget) ||
|
||||
typeof response.truncated !== "boolean" ||
|
||||
!number(response.total_ms, 0) ||
|
||||
response.adoption !== "unknown"
|
||||
!number(response.total_ms, 0)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const paths = response.paths as Record<string, unknown>;
|
||||
const rawPath = validatePath(paths.raw);
|
||||
const memoryPath = validatePath(paths.memory);
|
||||
if (!rawPath || !memoryPath) return null;
|
||||
const requested = new Set(request.kinds);
|
||||
if (
|
||||
(!requested.has("raw") && rawPath.status !== "not_requested") ||
|
||||
(!requested.has("memory") && memoryPath.status !== "not_requested")
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const items: Array<Record<string, unknown>> = [];
|
||||
let characters = 0;
|
||||
for (const itemValue of response.items) {
|
||||
const item = validateItem(itemValue, request);
|
||||
if (!item || !requested.has(item.kind as "raw" | "memory")) return null;
|
||||
items.push(item);
|
||||
characters +=
|
||||
item.kind === "memory"
|
||||
? characterCount(item.content)
|
||||
: (item.messages as Array<Record<string, unknown>>).reduce(
|
||||
(sum, message) => sum + characterCount(message.text),
|
||||
0,
|
||||
);
|
||||
characters += characterCount(item.content);
|
||||
}
|
||||
if (characters !== response.used_characters) return null;
|
||||
if (response.state === "empty" && items.length !== 0) return null;
|
||||
return {
|
||||
schema_version: 1,
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
state: response.state,
|
||||
request_id: response.request_id as string,
|
||||
paths: { raw: rawPath, memory: memoryPath },
|
||||
items,
|
||||
used_characters: response.used_characters as number,
|
||||
truncated: response.truncated,
|
||||
total_ms: response.total_ms as number,
|
||||
adoption: "unknown",
|
||||
};
|
||||
}
|
||||
|
||||
function validatePath(value: unknown): Record<string, unknown> | null {
|
||||
const path = record(value);
|
||||
if (
|
||||
!path ||
|
||||
!oneOf(path.status, PATH_STATES) ||
|
||||
!number(path.elapsed_ms, 0) ||
|
||||
!integer(path.returned_count, 0, 20) ||
|
||||
typeof path.truncated !== "boolean" ||
|
||||
!enumList(path.truncated_by, LIMIT_REASONS) ||
|
||||
!(
|
||||
path.reason === null ||
|
||||
path.reason === undefined ||
|
||||
oneOf(path.reason, PATH_REASONS)
|
||||
)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
status: path.status,
|
||||
reason: path.reason ?? null,
|
||||
elapsed_ms: path.elapsed_ms,
|
||||
returned_count: path.returned_count,
|
||||
truncated: path.truncated,
|
||||
truncated_by: path.truncated_by,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -235,88 +166,65 @@ function validateItem(
|
||||
request: ContextSearchInput,
|
||||
): Record<string, unknown> | null {
|
||||
const item = record(value);
|
||||
if (!item || !oneOf(item.kind, CONTEXT_KINDS)) return null;
|
||||
if (
|
||||
!item ||
|
||||
!oneOf(item.kind, CONTEXT_KINDS) ||
|
||||
boundedText(item.content, 20_000, true) === null ||
|
||||
!number(item.score, 0) ||
|
||||
(item.score as number) > 1 ||
|
||||
typeof item.truncated !== "boolean"
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
if (item.kind === "memory") {
|
||||
if (
|
||||
!boundedText(item.memory_id, 512) ||
|
||||
!integer(item.revision, 1) ||
|
||||
boundedText(item.content, 20_000, true) === null ||
|
||||
boundedText(item.memory_kind, 64, true) === null ||
|
||||
!boundedText(item.updated_at, 64) ||
|
||||
!textList(item.evidence_ids, 128, 512) ||
|
||||
typeof item.truncated !== "boolean"
|
||||
boundedText(item.layer, 64, true) === null ||
|
||||
boundedText(item.status, 32, true) === null ||
|
||||
!optionalText(item.source, 128) ||
|
||||
!optionalText(item.updated_at, 64)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
kind: "memory",
|
||||
memory_id: item.memory_id,
|
||||
revision: item.revision,
|
||||
content: item.content,
|
||||
memory_kind: item.memory_kind,
|
||||
updated_at: item.updated_at,
|
||||
evidence_ids: item.evidence_ids,
|
||||
layer: item.layer,
|
||||
status: item.status,
|
||||
source: item.source ?? null,
|
||||
updated_at: item.updated_at ?? null,
|
||||
score: item.score,
|
||||
truncated: item.truncated,
|
||||
};
|
||||
}
|
||||
if (
|
||||
!boundedText(item.source_id, 128) ||
|
||||
!boundedText(item.raw_id, 512) ||
|
||||
!boundedText(item.source, 128) ||
|
||||
(request.raw_source_ids !== undefined &&
|
||||
!request.raw_source_ids.includes(item.source_id as string)) ||
|
||||
!boundedText(item.conversation_id, 512) ||
|
||||
!boundedText(item.anchor_evidence_id, 512) ||
|
||||
!Array.isArray(item.messages) ||
|
||||
item.messages.length < 1 ||
|
||||
item.messages.length > 3 ||
|
||||
typeof item.truncated !== "boolean"
|
||||
!request.raw_source_ids.includes(item.source as string)) ||
|
||||
!boundedText(item.session_id, 512) ||
|
||||
!optionalText(item.memory_at, 64)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const messages = item.messages.map(validateMessage);
|
||||
if (messages.some((message) => message === null)) return null;
|
||||
return {
|
||||
kind: "raw",
|
||||
source_id: item.source_id,
|
||||
conversation_id: item.conversation_id,
|
||||
anchor_evidence_id: item.anchor_evidence_id,
|
||||
messages,
|
||||
raw_id: item.raw_id,
|
||||
content: item.content,
|
||||
source: item.source,
|
||||
session_id: item.session_id,
|
||||
memory_at: item.memory_at ?? null,
|
||||
score: item.score,
|
||||
truncated: item.truncated,
|
||||
};
|
||||
}
|
||||
|
||||
function validateMessage(value: unknown): Record<string, unknown> | null {
|
||||
const message = record(value);
|
||||
if (
|
||||
!message ||
|
||||
!boundedText(message.evidence_id, 512) ||
|
||||
!integer(message.revision, 1) ||
|
||||
boundedText(message.role, 32, true) === null ||
|
||||
boundedText(message.written_by, 32, true) === null ||
|
||||
!(
|
||||
message.created_at === null ||
|
||||
boundedText(message.created_at, 64) !== null
|
||||
) ||
|
||||
boundedText(message.text, 20_000, true) === null ||
|
||||
typeof message.truncated !== "boolean"
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
evidence_id: message.evidence_id,
|
||||
revision: message.revision,
|
||||
role: message.role,
|
||||
written_by: message.written_by,
|
||||
created_at: message.created_at,
|
||||
text: message.text,
|
||||
truncated: message.truncated,
|
||||
};
|
||||
}
|
||||
|
||||
function failed(
|
||||
outcome: Exclude<ReadOutcome<unknown>, { state: "ok" }>,
|
||||
): ContextSearchDocument {
|
||||
return {
|
||||
schema_version: 1,
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
state: outcome.state,
|
||||
code: outcome.code,
|
||||
@@ -374,27 +282,10 @@ function oneOf<const T extends readonly string[]>(
|
||||
return typeof value === "string" && values.includes(value as T[number]);
|
||||
}
|
||||
|
||||
function enumList(
|
||||
value: unknown,
|
||||
values: readonly string[],
|
||||
): value is string[] {
|
||||
function optionalText(value: unknown, maximum: number): boolean {
|
||||
return (
|
||||
Array.isArray(value) &&
|
||||
value.length <= 4 &&
|
||||
new Set(value).size === value.length &&
|
||||
value.every((item) => oneOf(item, values))
|
||||
);
|
||||
}
|
||||
|
||||
function textList(
|
||||
value: unknown,
|
||||
maximumItems: number,
|
||||
maximumText: number,
|
||||
): value is string[] {
|
||||
return (
|
||||
Array.isArray(value) &&
|
||||
value.length <= maximumItems &&
|
||||
new Set(value).size === value.length &&
|
||||
value.every((item) => boundedText(item, maximumText) !== null)
|
||||
value === null ||
|
||||
value === undefined ||
|
||||
boundedText(value, maximum) !== null
|
||||
);
|
||||
}
|
||||
@@ -8,18 +8,19 @@ import axios from "axios";
|
||||
*/
|
||||
const READ_ONLY_PATHS = [
|
||||
"/api/v1/call-events",
|
||||
"/api/v1/home-summary",
|
||||
"/api/v1/import-jobs",
|
||||
"/api/v1/live-sync",
|
||||
"/api/v1/memory-library",
|
||||
"/api/v1/memory-processing",
|
||||
"/api/v1/memory-tasks",
|
||||
"/api/v1/model-profiles",
|
||||
"/api/v1/sources",
|
||||
"/api/v2/home-summary",
|
||||
"/api/v2/memory-library",
|
||||
"/api/v2/user-profile",
|
||||
] as const;
|
||||
const SDK_MEMORY_DETAIL = /^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}$/;
|
||||
/** Status of one `POST /api/v2/memories` addition, by its request id. */
|
||||
const MEMORY_ADDITION =
|
||||
/^\/api\/v2\/memories\/additions\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
|
||||
/** Contract (C0, H25): the one-time loopback code exchange for `memhub ui`.
|
||||
* It is a POST because it mints a short-lived code; the CLI sends the bearer
|
||||
@@ -67,13 +68,10 @@ export function createUiSessionRequest(attachment: {
|
||||
};
|
||||
}
|
||||
|
||||
const MEMORY_DETAIL = /^\/api\/v1\/memories\/[A-Za-z0-9][\w.-]{0,127}$/;
|
||||
const IMPORT_JOB =
|
||||
/^\/api\/v1\/import-jobs\/[A-Za-z0-9][\w.-]{0,127}(?:\/issues)?$/;
|
||||
const OPERATION_RECEIPT =
|
||||
/^\/api\/v1\/(?:source-connections|model-profile-creations|model-probe-operations)\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
const MEMORY_MUTATION_RECEIPT =
|
||||
/^\/api\/v1\/memory-mutations\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
/**
|
||||
* Core registers these static routes before `/{memory_id}`, so a memory whose
|
||||
* id happens to be one of them would reach a different handler: `graph`
|
||||
@@ -84,9 +82,15 @@ export const RESERVED_MEMORY_SEGMENTS = ["ask", "graph"] as const;
|
||||
/** A response larger than this is refused before it is buffered. */
|
||||
const MAX_RESPONSE_BYTES = 4 * 1024 * 1024;
|
||||
|
||||
/** `rejected` is Core refusing the request itself (400/409/422: a bad filter,
|
||||
* an ambiguous id). It is not "Core is busy" — only a transport failure or a
|
||||
* 5xx is `unavailable`. */
|
||||
export type ReadOutcome<T> =
|
||||
| { state: "ok"; data: T }
|
||||
| { state: "denied" | "unavailable" | "not_found"; code: string };
|
||||
| {
|
||||
state: "denied" | "unavailable" | "not_found" | "rejected";
|
||||
code: string;
|
||||
};
|
||||
|
||||
export type CoreReader = <T>(
|
||||
path: string,
|
||||
@@ -145,10 +149,27 @@ export function createCoreReader(attachment: {
|
||||
if (response.status === 404) {
|
||||
return { state: "not_found" as const, code: "http_404" };
|
||||
}
|
||||
if ([400, 409, 422].includes(response.status)) {
|
||||
return {
|
||||
state: "rejected" as const,
|
||||
code: detailCode(response.data, response.status),
|
||||
};
|
||||
}
|
||||
return { state: "unavailable" as const, code: `http_${response.status}` };
|
||||
};
|
||||
}
|
||||
|
||||
/** Core's stable `detail` code when it sent one, else the bare status. */
|
||||
function detailCode(value: unknown, status: number): string {
|
||||
const detail =
|
||||
value && typeof value === "object" && !Array.isArray(value)
|
||||
? (value as { detail?: unknown }).detail
|
||||
: undefined;
|
||||
return typeof detail === "string" && /^[a-z0-9_.:-]{1,128}$/.test(detail)
|
||||
? detail
|
||||
: `http_${status}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Context search is semantically read-only but intentionally uses POST so the
|
||||
* private query and filters are carried in a bounded body, never in a URL.
|
||||
@@ -173,7 +194,7 @@ export function createCoreContextQuery(attachment: {
|
||||
let response: { status: number; data: unknown };
|
||||
try {
|
||||
response = await client.post(
|
||||
"/api/v1/context/search",
|
||||
"/api/v2/context/search",
|
||||
body,
|
||||
options?.signal ? { signal: options.signal } : undefined,
|
||||
);
|
||||
@@ -197,17 +218,18 @@ export function isReadOnlyPath(path: string): boolean {
|
||||
if ((READ_ONLY_PATHS as readonly string[]).includes(path)) return true;
|
||||
if (IMPORT_JOB.test(path)) return true;
|
||||
if (OPERATION_RECEIPT.test(path)) return true;
|
||||
if (MEMORY_MUTATION_RECEIPT.test(path)) return true;
|
||||
if (SDK_MEMORY_DETAIL.test(path)) return true;
|
||||
if (!MEMORY_DETAIL.test(path)) return false;
|
||||
const segment = path.slice("/api/v1/memories/".length);
|
||||
if (MEMORY_ADDITION.test(path)) return true;
|
||||
if (!SDK_MEMORY_DETAIL.test(path)) return false;
|
||||
const segment = path.slice("/api/v2/memories/".length);
|
||||
return !(RESERVED_MEMORY_SEGMENTS as readonly string[]).includes(segment);
|
||||
}
|
||||
|
||||
/** A write accepted by the unified Memory Server entry.
|
||||
*
|
||||
* `request_id` is what the caller keeps and queries later. Retrying is unsafe:
|
||||
* HY has no idempotency key, so a resubmit duplicates the memory.
|
||||
* `request_id` is the idempotency key: Core keys the write on it, so a retry
|
||||
* with the same id and the same messages is answered from the first write, and
|
||||
* the same id with different messages is refused (409). A retry without the
|
||||
* original id is a new write.
|
||||
*/
|
||||
export interface MemoryWriteReceipt {
|
||||
request_id: string;
|
||||
@@ -274,6 +296,11 @@ function baseUrlOf(attachment: RuntimeAttachmentLike): string {
|
||||
return `http://${attachment.host}:${attachment.port}`;
|
||||
}
|
||||
|
||||
export type MemoryWriteOutcome =
|
||||
| { state: "accepted"; receipt: MemoryWriteReceipt }
|
||||
| { state: "rejected"; code: string; http_status: number }
|
||||
| { state: "unknown"; code: string };
|
||||
|
||||
export async function submitMemoryWrite(
|
||||
attachment: RuntimeAttachmentLike,
|
||||
request: {
|
||||
@@ -284,28 +311,48 @@ export async function submitMemoryWrite(
|
||||
memory_at?: string;
|
||||
priority?: "live" | "normal";
|
||||
},
|
||||
): Promise<MemoryWriteReceipt> {
|
||||
const response = await axios.post<MemoryWriteReceipt>(
|
||||
`${baseUrlOf(attachment)}/api/v2/memory-writes`,
|
||||
{
|
||||
request_id: request.request_id,
|
||||
session_id: request.session,
|
||||
source_id: request.source,
|
||||
messages: [...request.messages],
|
||||
...(request.memory_at === undefined
|
||||
? {}
|
||||
: { memory_at: request.memory_at }),
|
||||
...(request.priority === undefined ? {} : { priority: request.priority }),
|
||||
},
|
||||
{
|
||||
headers: { Authorization: `Bearer ${attachment.apiToken}` },
|
||||
timeout: 15_000,
|
||||
proxy: false,
|
||||
maxRedirects: 0,
|
||||
maxContentLength: MAX_RESPONSE_BYTES,
|
||||
},
|
||||
);
|
||||
return response.data;
|
||||
): Promise<MemoryWriteOutcome> {
|
||||
let response: { status: number; data: unknown };
|
||||
try {
|
||||
response = await axios.post<MemoryWriteReceipt>(
|
||||
`${baseUrlOf(attachment)}/api/v2/memory-writes`,
|
||||
{
|
||||
request_id: request.request_id,
|
||||
session_id: request.session,
|
||||
source_id: request.source,
|
||||
messages: [...request.messages],
|
||||
...(request.memory_at === undefined
|
||||
? {}
|
||||
: { memory_at: request.memory_at }),
|
||||
...(request.priority === undefined
|
||||
? {}
|
||||
: { priority: request.priority }),
|
||||
},
|
||||
{
|
||||
headers: { Authorization: `Bearer ${attachment.apiToken}` },
|
||||
timeout: 15_000,
|
||||
proxy: false,
|
||||
maxRedirects: 0,
|
||||
maxContentLength: MAX_RESPONSE_BYTES,
|
||||
validateStatus: () => true,
|
||||
},
|
||||
);
|
||||
} catch {
|
||||
// The request may or may not have reached Core; the same request id makes
|
||||
// a retry safe.
|
||||
return { state: "unknown", code: "transport_result_unknown" };
|
||||
}
|
||||
if (response.status >= 200 && response.status < 300) {
|
||||
return { state: "accepted", receipt: response.data as MemoryWriteReceipt };
|
||||
}
|
||||
if (response.status >= 400 && response.status < 500) {
|
||||
return {
|
||||
state: "rejected",
|
||||
code: detailCode(response.data, response.status),
|
||||
http_status: response.status,
|
||||
};
|
||||
}
|
||||
return { state: "unknown", code: `http_${response.status}_result_unknown` };
|
||||
}
|
||||
|
||||
/** Ask Runtime to scan one source now, returning the import job it started.
|
||||
@@ -502,48 +549,52 @@ export async function readScanRoots(
|
||||
}
|
||||
}
|
||||
|
||||
/** A direct Core write whose refusal carries Core's stable code. */
|
||||
export type CoreWriteResult =
|
||||
| { ok: true; data: unknown }
|
||||
| { ok: false; code: string };
|
||||
|
||||
async function coreWrite(
|
||||
attachment: RuntimeAttachmentLike,
|
||||
method: "post" | "delete",
|
||||
pathname: string,
|
||||
body: Record<string, unknown>,
|
||||
timeout = 10_000,
|
||||
): Promise<CoreWriteResult> {
|
||||
let response: { status: number; data: unknown };
|
||||
try {
|
||||
response = await axios.request({
|
||||
method,
|
||||
url: `${baseUrlOf(attachment)}${pathname}`,
|
||||
data: body,
|
||||
headers: { Authorization: `Bearer ${attachment.apiToken}` },
|
||||
timeout,
|
||||
proxy: false,
|
||||
maxRedirects: 0,
|
||||
maxContentLength: MAX_RESPONSE_BYTES,
|
||||
validateStatus: () => true,
|
||||
});
|
||||
} catch {
|
||||
return { ok: false, code: "transport_failed" };
|
||||
}
|
||||
if (response.status >= 200 && response.status < 300) {
|
||||
return { ok: true, data: response.data };
|
||||
}
|
||||
return { ok: false, code: detailCode(response.data, response.status) };
|
||||
}
|
||||
|
||||
export async function addScanRoot(
|
||||
attachment: RuntimeAttachmentLike,
|
||||
body: { root: string; source?: string; recursive?: boolean },
|
||||
): Promise<unknown | null> {
|
||||
try {
|
||||
const response = await axios.post<unknown>(
|
||||
`${baseUrlOf(attachment)}/api/v1/scan-roots`,
|
||||
body,
|
||||
{
|
||||
headers: { Authorization: `Bearer ${attachment.apiToken}` },
|
||||
timeout: 10_000,
|
||||
proxy: false,
|
||||
maxRedirects: 0,
|
||||
maxContentLength: MAX_RESPONSE_BYTES,
|
||||
},
|
||||
);
|
||||
return response.data;
|
||||
} catch (error) {
|
||||
return { _error: error instanceof Error ? error.message : String(error) };
|
||||
}
|
||||
): Promise<CoreWriteResult> {
|
||||
return coreWrite(attachment, "post", "/api/v1/scan-roots", body);
|
||||
}
|
||||
|
||||
export async function removeScanRoot(
|
||||
attachment: RuntimeAttachmentLike,
|
||||
body: { root: string },
|
||||
): Promise<unknown | null> {
|
||||
try {
|
||||
const response = await axios.delete<unknown>(
|
||||
`${baseUrlOf(attachment)}/api/v1/scan-roots`,
|
||||
{
|
||||
headers: { Authorization: `Bearer ${attachment.apiToken}` },
|
||||
data: body,
|
||||
timeout: 10_000,
|
||||
proxy: false,
|
||||
maxRedirects: 0,
|
||||
maxContentLength: MAX_RESPONSE_BYTES,
|
||||
},
|
||||
);
|
||||
return response.data;
|
||||
} catch (error) {
|
||||
return { _error: error instanceof Error ? error.message : String(error) };
|
||||
}
|
||||
): Promise<CoreWriteResult> {
|
||||
return coreWrite(attachment, "delete", "/api/v1/scan-roots", body);
|
||||
}
|
||||
|
||||
export async function readAdapters(
|
||||
@@ -569,21 +620,12 @@ export async function readAdapters(
|
||||
export async function probeAdapter(
|
||||
attachment: RuntimeAttachmentLike,
|
||||
source: string,
|
||||
): Promise<unknown | null> {
|
||||
try {
|
||||
const response = await axios.post<unknown>(
|
||||
`${baseUrlOf(attachment)}/api/v1/adapters/probe`,
|
||||
{ source },
|
||||
{
|
||||
headers: { Authorization: `Bearer ${attachment.apiToken}` },
|
||||
timeout: 15_000,
|
||||
proxy: false,
|
||||
maxRedirects: 0,
|
||||
maxContentLength: MAX_RESPONSE_BYTES,
|
||||
},
|
||||
);
|
||||
return response.data;
|
||||
} catch (error) {
|
||||
return { _error: error instanceof Error ? error.message : String(error) };
|
||||
}
|
||||
): Promise<CoreWriteResult> {
|
||||
return coreWrite(
|
||||
attachment,
|
||||
"post",
|
||||
"/api/v1/adapters/probe",
|
||||
{ source },
|
||||
15_000,
|
||||
);
|
||||
}
|
||||
@@ -4,9 +4,11 @@ const WRITE_PATHS = [
|
||||
"/api/v1/sources/connect",
|
||||
"/api/v1/model-profiles",
|
||||
] as const;
|
||||
const MEMORY_WRITE_PATH =
|
||||
/^\/api\/v1\/memories(?:\/[A-Za-z0-9][\w.-]{0,127}\/(?:correct|scopes|actions\/delete))?$/;
|
||||
const SDK_MEMORY_PATH = /^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}$/;
|
||||
/** `POST /api/v2/memories` adds one memory (the app's 添加记忆). */
|
||||
const MEMORY_ADD_PATH = "/api/v2/memories";
|
||||
/** One SDK memory; delete names its namespace in the query string. */
|
||||
const SDK_MEMORY_PATH =
|
||||
/^\/api\/v2\/memories\/[A-Za-z0-9][\w.-]{0,127}(?:\?kind=(?:chat|experience|fact)(?:&variant=(?:working|coding))?)?$/;
|
||||
const MODEL_PROBE_PATH =
|
||||
/^\/api\/v1\/model-profiles\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\/probe$/;
|
||||
const MODEL_PROFILE_PATH =
|
||||
@@ -22,18 +24,7 @@ const SOURCE_REVOKE_PATH =
|
||||
const SOURCE_IMPORT_JOB_PATH = "/api/v1/import-jobs";
|
||||
const IMPORT_JOB_ACTION_PATH =
|
||||
/^\/api\/v1\/import-jobs\/[A-Za-z0-9][\w.-]{0,127}\/(?:pause|resume|cancel)$/;
|
||||
const CALL_EVENTS_CLEAR_PATH = "/api/v1/call-events/clear";
|
||||
const CALL_EVENTS_CLEAR_ALL_PATH = "/api/v1/call-events/clear-all";
|
||||
const IMPORT_FILE_COMMIT =
|
||||
/^\/api\/v1\/import-files\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\/commit$/;
|
||||
const IMPORT_FILE =
|
||||
/^\/api\/v1\/import-files\/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
const MEMORY_NOTES_PATH = "/api/v1/memory-notes";
|
||||
const MEMORY_NOTE_PROCESS =
|
||||
/^\/api\/v1\/memory-notes\/[A-Za-z0-9][\w.:-]{0,127}\/process$/;
|
||||
const EXTRACTION_SETTINGS_PATH = "/api/v1/extraction-settings";
|
||||
const GRAPH_WIDTH_PATH = "/api/v1/graph-width";
|
||||
const MANUAL_EXTRACT_PATH = "/api/v1/memory-processing/manual-extract";
|
||||
const ONBOARDING_COMPLETE_PATH = "/api/v1/onboarding/complete";
|
||||
const MAX_BYTES = 4 * 1024 * 1024;
|
||||
|
||||
@@ -70,18 +61,6 @@ export interface CorePatchOperations {
|
||||
): Promise<OperationOutcome<T>>;
|
||||
}
|
||||
|
||||
export interface CoreUploadOperations {
|
||||
uploadImportFile<T>(
|
||||
input: {
|
||||
bytes: Uint8Array;
|
||||
filename: string;
|
||||
inputKind: "raw_conversation" | "memory_content";
|
||||
operationId: string;
|
||||
},
|
||||
signal?: AbortSignal,
|
||||
): Promise<OperationOutcome<T>>;
|
||||
}
|
||||
|
||||
export function createCoreOperations(attachment: {
|
||||
host: string;
|
||||
port: number;
|
||||
@@ -90,8 +69,7 @@ export function createCoreOperations(attachment: {
|
||||
}): CoreOperations &
|
||||
CoreDeleteOperations &
|
||||
CorePutOperations &
|
||||
CorePatchOperations &
|
||||
CoreUploadOperations {
|
||||
CorePatchOperations {
|
||||
const client = axios.create({
|
||||
baseURL: `http://${attachment.host}:${attachment.port}`,
|
||||
headers: { Authorization: `Bearer ${attachment.apiToken}` },
|
||||
@@ -102,18 +80,6 @@ export function createCoreOperations(attachment: {
|
||||
maxBodyLength: 128 * 1024,
|
||||
validateStatus: () => true,
|
||||
});
|
||||
// Uploads carry up to 128 MiB of raw octet-stream, so they need a separate
|
||||
// body limit and a deadline that leaves room for parsing on the Core side.
|
||||
const uploadClient = axios.create({
|
||||
baseURL: `http://${attachment.host}:${attachment.port}`,
|
||||
headers: { Authorization: `Bearer ${attachment.apiToken}` },
|
||||
timeout: 120_000,
|
||||
proxy: false,
|
||||
maxRedirects: 0,
|
||||
maxContentLength: MAX_BYTES,
|
||||
maxBodyLength: 128 * 1024 * 1024,
|
||||
validateStatus: () => true,
|
||||
});
|
||||
return {
|
||||
async post<T>(
|
||||
path: string,
|
||||
@@ -123,19 +89,13 @@ export function createCoreOperations(attachment: {
|
||||
if (
|
||||
!(WRITE_PATHS as readonly string[]).includes(path) &&
|
||||
path !== SOURCE_IMPORT_JOB_PATH &&
|
||||
!MEMORY_WRITE_PATH.test(path) &&
|
||||
path !== MEMORY_ADD_PATH &&
|
||||
!MODEL_PROBE_PATH.test(path) &&
|
||||
!SOURCE_SYNC_PATH.test(path) &&
|
||||
!IMPORT_JOB_ACTION_PATH.test(path) &&
|
||||
!SOURCE_AUTHORIZE_PATH.test(path) &&
|
||||
!SOURCE_REVOKE_PATH.test(path) &&
|
||||
path !== CALL_EVENTS_CLEAR_PATH &&
|
||||
path !== CALL_EVENTS_CLEAR_ALL_PATH &&
|
||||
!IMPORT_FILE_COMMIT.test(path) &&
|
||||
path !== MEMORY_NOTES_PATH &&
|
||||
!MEMORY_NOTE_PROCESS.test(path) &&
|
||||
path !== GRAPH_WIDTH_PATH &&
|
||||
path !== MANUAL_EXTRACT_PATH &&
|
||||
path !== ONBOARDING_COMPLETE_PATH &&
|
||||
!MODEL_PROFILE_ACTIVATE.test(path)
|
||||
) {
|
||||
@@ -171,7 +131,7 @@ export function createCoreOperations(attachment: {
|
||||
body: Record<string, unknown>,
|
||||
signal?: AbortSignal,
|
||||
) {
|
||||
if (path !== EXTRACTION_SETTINGS_PATH && !SDK_MEMORY_PATH.test(path)) {
|
||||
if (!SDK_MEMORY_PATH.test(path) || path.includes("?")) {
|
||||
throw new Error("core_operation_path_not_allowed");
|
||||
}
|
||||
let response: { status: number; data: unknown };
|
||||
@@ -233,7 +193,7 @@ export function createCoreOperations(attachment: {
|
||||
};
|
||||
},
|
||||
async delete<T>(path: string, signal?: AbortSignal) {
|
||||
if (!IMPORT_FILE.test(path) && !SDK_MEMORY_PATH.test(path)) {
|
||||
if (!SDK_MEMORY_PATH.test(path) && !MODEL_PROFILE_PATH.test(path)) {
|
||||
throw new Error("core_operation_path_not_allowed");
|
||||
}
|
||||
let response: { status: number; data: unknown };
|
||||
@@ -257,51 +217,6 @@ export function createCoreOperations(attachment: {
|
||||
code: `http_${response.status}_result_unknown`,
|
||||
};
|
||||
},
|
||||
async uploadImportFile<T>(
|
||||
input: {
|
||||
bytes: Uint8Array;
|
||||
filename: string;
|
||||
inputKind: "raw_conversation" | "memory_content";
|
||||
operationId: string;
|
||||
},
|
||||
signal?: AbortSignal,
|
||||
) {
|
||||
const query = new URLSearchParams({
|
||||
filename: input.filename,
|
||||
input_kind: input.inputKind,
|
||||
operation_id: input.operationId,
|
||||
});
|
||||
let response: { status: number; data: unknown };
|
||||
try {
|
||||
response = await uploadClient.post(
|
||||
`/api/v1/import-files?${query.toString()}`,
|
||||
input.bytes,
|
||||
{
|
||||
headers: {
|
||||
"Content-Type": "application/octet-stream",
|
||||
"Content-Length": String(input.bytes.byteLength),
|
||||
},
|
||||
...(signal ? { signal } : {}),
|
||||
},
|
||||
);
|
||||
} catch {
|
||||
return { state: "unknown", code: "transport_result_unknown" };
|
||||
}
|
||||
if (response.status >= 200 && response.status < 300) {
|
||||
return { state: "committed", data: response.data as T };
|
||||
}
|
||||
if (response.status >= 400 && response.status < 500) {
|
||||
return {
|
||||
state: "rejected",
|
||||
code: safeErrorCode(response.data, response.status),
|
||||
http_status: response.status,
|
||||
};
|
||||
}
|
||||
return {
|
||||
state: "unknown",
|
||||
code: `http_${response.status}_result_unknown`,
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -1,18 +1,20 @@
|
||||
import type {
|
||||
GovernedMemory,
|
||||
HomeSummary,
|
||||
ImportIssue,
|
||||
ImportJob,
|
||||
LiveSyncHealth,
|
||||
MemoryDetail,
|
||||
MemoryEvidenceExcerpt,
|
||||
MemoryLibraryPage,
|
||||
MemoryProcessingSummary,
|
||||
ModelProfile,
|
||||
SourceResponse,
|
||||
} from "@one-memory/contracts";
|
||||
/** The only memory states a response may carry, and the only ones a filter may ask for. */
|
||||
export const MEMORY_STATUSES = ["confirmed", "deleted", "superseded"] as const;
|
||||
/** SDK memory states `memhub memories list --status` may ask for. */
|
||||
export const MEMORY_STATUSES = [
|
||||
"active",
|
||||
"superseded",
|
||||
"shadow",
|
||||
"negated",
|
||||
"conflicted",
|
||||
"archived",
|
||||
] as const;
|
||||
|
||||
/**
|
||||
* HTTP 200 is not a contract. Everything here re-checks the shape and the
|
||||
@@ -23,7 +25,6 @@ export const MEMORY_STATUSES = ["confirmed", "deleted", "superseded"] as const;
|
||||
|
||||
const ID_MAX = 128;
|
||||
const LABEL_MAX = 256;
|
||||
const TEXT_MAX = 100_000;
|
||||
const LIST_MAX = 500;
|
||||
const LIVE_SYNC_STATES = [
|
||||
"idle",
|
||||
@@ -39,14 +40,6 @@ const PROVIDER_KINDS = [
|
||||
"local_onnx",
|
||||
] as const;
|
||||
const CREDENTIAL_STATES = ["available", "missing", "unavailable"] as const;
|
||||
const MEMORY_KINDS = [
|
||||
"chat",
|
||||
"working",
|
||||
"coding",
|
||||
"schema",
|
||||
"intention",
|
||||
] as const;
|
||||
const MEMORY_ORIGINS = ["extracted", "answered", "agent", "manual"] as const;
|
||||
const IMPORT_STATUSES = [
|
||||
"pending",
|
||||
"running",
|
||||
@@ -57,16 +50,6 @@ const IMPORT_STATUSES = [
|
||||
"cancelled",
|
||||
] as const;
|
||||
const SCAN_PHASES = ["discover", "read", "emit", "done"] as const;
|
||||
const MEMORY_TASK_KINDS = ["extract", "system2", "ultra", "sweeper"] as const;
|
||||
const MEMORY_TASK_STATUSES = [
|
||||
"pending",
|
||||
"running",
|
||||
"waiting_for_model",
|
||||
"completed",
|
||||
"failed",
|
||||
"cancelled",
|
||||
"deferred_outside_window",
|
||||
] as const;
|
||||
const CAPABILITY_STATES = ["not_implemented", "implemented"] as const;
|
||||
const VERIFICATION_STATES = ["fixture_only", "real_replayed"] as const;
|
||||
const SYNC_COMPLETENESS = [
|
||||
@@ -87,16 +70,6 @@ const LOCATION_STATES = [
|
||||
] as const;
|
||||
const LIVE_SYNC_MODES = ["polling", "watch_and_polling"] as const;
|
||||
|
||||
export interface ValidatedMemoryTask {
|
||||
task_id: string;
|
||||
kind: (typeof MEMORY_TASK_KINDS)[number];
|
||||
status: (typeof MEMORY_TASK_STATUSES)[number];
|
||||
attempts: number;
|
||||
last_error_code: string | null;
|
||||
created_at: string;
|
||||
updated_at: string;
|
||||
}
|
||||
|
||||
export interface SourceConnectionCandidate {
|
||||
source_id: string;
|
||||
display_name: string;
|
||||
@@ -352,45 +325,6 @@ export function validateImportIssues(
|
||||
});
|
||||
}
|
||||
|
||||
export function validateMemoryTasks(
|
||||
value: unknown,
|
||||
): ValidatedMemoryTask[] | null {
|
||||
return list(value, (item) => {
|
||||
const task = record(item);
|
||||
if (!task) return null;
|
||||
const taskId = text(task.task_id, ID_MAX);
|
||||
if (
|
||||
task.last_error_code !== null &&
|
||||
text(task.last_error_code, LABEL_MAX) === null
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const error = task.last_error_code as string | null;
|
||||
const createdAt = text(task.created_at, LABEL_MAX);
|
||||
const updatedAt = text(task.updated_at, LABEL_MAX);
|
||||
const attempts = count(task.attempts);
|
||||
if (
|
||||
taskId === null ||
|
||||
createdAt === null ||
|
||||
updatedAt === null ||
|
||||
attempts === null ||
|
||||
!oneOf(task.kind, MEMORY_TASK_KINDS) ||
|
||||
!oneOf(task.status, MEMORY_TASK_STATUSES)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
task_id: taskId,
|
||||
kind: task.kind,
|
||||
status: task.status,
|
||||
attempts,
|
||||
last_error_code: error,
|
||||
created_at: createdAt,
|
||||
updated_at: updatedAt,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
export function validateProcessing(
|
||||
value: unknown,
|
||||
): MemoryProcessingSummary | null {
|
||||
@@ -444,150 +378,32 @@ export function validateModelProfiles(value: unknown): ModelProfile[] | null {
|
||||
});
|
||||
}
|
||||
|
||||
export function validateHomeSummary(value: unknown): HomeSummary | null {
|
||||
/** `GET /api/v2/home-summary`: the SDK-authority home card. */
|
||||
export interface SdkHomeSummary {
|
||||
schema_version: 2;
|
||||
memory_count: number;
|
||||
latest: { memory_id: string } | null;
|
||||
observed_at: string;
|
||||
}
|
||||
|
||||
export function validateHomeSummary(value: unknown): SdkHomeSummary | null {
|
||||
const summary = record(value);
|
||||
if (
|
||||
!summary ||
|
||||
summary.schema_version !== 2 ||
|
||||
count(summary.memory_count) === null ||
|
||||
text(summary.observed_at, LABEL_MAX) === null
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return summary as unknown as HomeSummary;
|
||||
}
|
||||
|
||||
/** The page must be the page that was asked for, not merely a page. */
|
||||
export function validateMemoryPage(
|
||||
value: unknown,
|
||||
request: { offset: number; limit: number },
|
||||
): MemoryLibraryPage | null {
|
||||
const page = record(value);
|
||||
if (!page) return null;
|
||||
const total = count(page.total);
|
||||
if (
|
||||
total === null ||
|
||||
page.offset !== request.offset ||
|
||||
page.limit !== request.limit
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const items = list(page.items, validateGovernedMemory);
|
||||
if (items === null || items.length > request.limit) return null;
|
||||
if (items.length > 0 && total < request.offset + items.length) return null;
|
||||
return { items, total, offset: page.offset, limit: page.limit };
|
||||
}
|
||||
|
||||
export function validateMemoryDetail(
|
||||
value: unknown,
|
||||
memoryId: string,
|
||||
): MemoryDetail | null {
|
||||
const detail = record(value);
|
||||
if (!detail) return null;
|
||||
const memory = validateGovernedMemory(detail.memory);
|
||||
if (memory === null || memory.memory_id !== memoryId) return null;
|
||||
const cited = new Set(memory.evidence_ids);
|
||||
const returned = new Set<string>();
|
||||
const evidence = list(detail.evidence ?? [], (item) => {
|
||||
const excerpt = record(item);
|
||||
if (!excerpt) return null;
|
||||
const evidenceId = text(excerpt.evidence_id, ID_MAX);
|
||||
// Core builds this list from the memory's own citations; anything else is
|
||||
// another record's excerpt and must not be shown under this memory.
|
||||
if (
|
||||
evidenceId === null ||
|
||||
!cited.has(evidenceId) ||
|
||||
returned.has(evidenceId) ||
|
||||
typeof excerpt.available !== "boolean"
|
||||
) {
|
||||
const latest = summary.latest;
|
||||
if (latest !== null) {
|
||||
const memory = record(latest);
|
||||
if (!memory || typeof memory.memory_id !== "string" || !memory.memory_id) {
|
||||
return null;
|
||||
}
|
||||
returned.add(evidenceId);
|
||||
const available = excerpt.available === true;
|
||||
const verbatim = available ? text(excerpt.verbatim ?? "", TEXT_MAX) : "";
|
||||
if (verbatim === null) return null;
|
||||
return {
|
||||
evidence_id: evidenceId,
|
||||
available,
|
||||
source_id: optional(excerpt.source_id, ID_MAX),
|
||||
written_by: optional(excerpt.written_by, LABEL_MAX),
|
||||
created_at: optional(excerpt.created_at, LABEL_MAX),
|
||||
// An excerpt the user cannot read has no text, whatever the server sent.
|
||||
verbatim,
|
||||
} as unknown as MemoryEvidenceExcerpt;
|
||||
});
|
||||
if (evidence === null || evidence.length !== memory.evidence_ids.length) {
|
||||
return null;
|
||||
}
|
||||
const history = list(detail.history ?? [], (item) => {
|
||||
const version = record(item);
|
||||
if (
|
||||
!version ||
|
||||
typeof version.revision !== "number" ||
|
||||
!Number.isSafeInteger(version.revision) ||
|
||||
version.revision < 1 ||
|
||||
text(version.action, LABEL_MAX) === null ||
|
||||
!oneOf(version.status, MEMORY_STATUSES) ||
|
||||
text(version.content, TEXT_MAX) === null ||
|
||||
identifiers(version.scopes) === null ||
|
||||
text(version.created_at, LABEL_MAX) === null
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
revision: version.revision,
|
||||
action: version.action,
|
||||
status: version.status,
|
||||
content: version.content,
|
||||
scopes: identifiers(version.scopes) as string[],
|
||||
created_at: version.created_at,
|
||||
};
|
||||
});
|
||||
if (history === null) return null;
|
||||
return {
|
||||
memory,
|
||||
evidence,
|
||||
history,
|
||||
index_sync: null,
|
||||
} as unknown as MemoryDetail;
|
||||
}
|
||||
|
||||
export function validateGovernedMemory(value: unknown): GovernedMemory | null {
|
||||
const memory = record(value);
|
||||
if (
|
||||
!memory ||
|
||||
!text(memory.memory_id, ID_MAX) ||
|
||||
!text(memory.user_id, LABEL_MAX) ||
|
||||
!text(memory.content, TEXT_MAX) ||
|
||||
!oneOf(memory.kind, MEMORY_KINDS) ||
|
||||
!oneOf(memory.origin, MEMORY_ORIGINS) ||
|
||||
typeof memory.revision !== "number" ||
|
||||
!Number.isSafeInteger(memory.revision) ||
|
||||
memory.revision < 1 ||
|
||||
!oneOf(memory.status, MEMORY_STATUSES) ||
|
||||
!text(memory.created_at, LABEL_MAX) ||
|
||||
!text(memory.updated_at, LABEL_MAX)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const scopes = identifiers(memory.scopes);
|
||||
const evidenceIds = identifiers(memory.evidence_ids);
|
||||
const tags = identifiers(memory.tags ?? []);
|
||||
if (
|
||||
scopes === null ||
|
||||
scopes.length < 1 ||
|
||||
evidenceIds === null ||
|
||||
evidenceIds.length < 1 ||
|
||||
tags === null ||
|
||||
tags.length > 32
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
...memory,
|
||||
scopes,
|
||||
evidence_ids: evidenceIds,
|
||||
tags,
|
||||
} as unknown as GovernedMemory;
|
||||
return summary as unknown as SdkHomeSummary;
|
||||
}
|
||||
|
||||
function identifiers(value: unknown): string[] | null {
|
||||
@@ -618,10 +434,6 @@ function text(value: unknown, max: number): string | null {
|
||||
return typeof value === "string" && value.length <= max ? value : null;
|
||||
}
|
||||
|
||||
function optional(value: unknown, max: number): string | undefined {
|
||||
return typeof value === "string" && value.length <= max ? value : undefined;
|
||||
}
|
||||
|
||||
function count(value: unknown): number | null {
|
||||
return typeof value === "number" && Number.isSafeInteger(value) && value >= 0
|
||||
? value
|
||||
|
||||
@@ -134,11 +134,6 @@ function progress(): ProgressDocument {
|
||||
generation: {
|
||||
summary_state: { state: "ok" },
|
||||
summary: { waiting_for_model: 1, failed: 0 },
|
||||
tasks_state: { state: "ok" },
|
||||
tasks: [{ task_id: "t1", status: "waiting_for_model" }],
|
||||
returned: 1,
|
||||
validated: 1,
|
||||
output_truncated: false,
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -65,6 +65,9 @@ export function renderDoctorText(document: DoctorDocument): string {
|
||||
model: "模型",
|
||||
pending_operations: "待处理操作",
|
||||
};
|
||||
const migration = document.checks.find(
|
||||
(check) => check.id === "core" && check.details?.log_file,
|
||||
);
|
||||
return [
|
||||
`MemHub 检查:${doctorState(document.state)}`,
|
||||
...document.checks.map(
|
||||
@@ -73,7 +76,13 @@ export function renderDoctorText(document: DoctorDocument): string {
|
||||
),
|
||||
...(document.state === "healthy"
|
||||
? []
|
||||
: ["下一步:打开 MemHub 的“帮助与诊断”查看并处理异常。"]),
|
||||
: migration
|
||||
? [
|
||||
`升级迁移没有完成(错误码:${migration.code}),MemHub 不会启动。`,
|
||||
`下一步:把迁移日志附在 GitHub 反馈里发给我们:${migration.details?.log_file}`,
|
||||
`反馈地址:${migration.details?.issues_url}`,
|
||||
]
|
||||
: ["下一步:打开 MemHub 的“帮助与诊断”查看并处理异常。"]),
|
||||
"",
|
||||
].join("\n");
|
||||
}
|
||||
@@ -107,6 +116,14 @@ function coreCheck(status: StatusDocument): DoctorCheck {
|
||||
id: "core",
|
||||
state: status.runtime.state === "stopped" ? "warn" : "fail",
|
||||
code: status.runtime.code,
|
||||
...(status.runtime.log_file
|
||||
? {
|
||||
details: {
|
||||
log_file: status.runtime.log_file,
|
||||
issues_url: status.runtime.issues_url ?? "",
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -200,8 +217,7 @@ function modelCheck(status: StatusDocument): DoctorCheck {
|
||||
function pendingCheck(progress: ProgressDocument | null): DoctorCheck {
|
||||
if (
|
||||
progress?.imports.state !== "ok" ||
|
||||
progress.generation.summary_state.state !== "ok" ||
|
||||
progress.generation.tasks_state.state !== "ok"
|
||||
progress.generation.summary_state.state !== "ok"
|
||||
) {
|
||||
return {
|
||||
id: "pending_operations",
|
||||
@@ -212,27 +228,20 @@ function pendingCheck(progress: ProgressDocument | null): DoctorCheck {
|
||||
const jobAttention = progress.imports.jobs.filter((item) =>
|
||||
["paused", "interrupted", "failed"].includes(text(item.status) ?? ""),
|
||||
).length;
|
||||
const taskAttention = progress.generation.tasks.filter((item) =>
|
||||
["waiting_for_model", "failed"].includes(text(item.status) ?? ""),
|
||||
).length;
|
||||
const summary = progress.generation.summary;
|
||||
const waiting = summary ? number(summary.waiting_for_model) : 0;
|
||||
const failed = summary ? number(summary.failed) : 0;
|
||||
const attention = jobAttention + taskAttention + waiting + failed;
|
||||
const attention = jobAttention + waiting + failed;
|
||||
return {
|
||||
id: "pending_operations",
|
||||
state: attention > 0 ? "warn" : "pass",
|
||||
code: attention > 0 ? "operations_need_attention" : "pending_operations_ok",
|
||||
details: {
|
||||
import_jobs_validated: progress.imports.validated,
|
||||
generation_tasks_validated: progress.generation.validated,
|
||||
jobs_need_attention: jobAttention,
|
||||
tasks_need_attention: taskAttention,
|
||||
waiting_for_model: waiting,
|
||||
failed,
|
||||
output_truncated:
|
||||
progress.imports.output_truncated ||
|
||||
progress.generation.output_truncated,
|
||||
output_truncated: progress.imports.output_truncated,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -23,39 +23,26 @@ function accountHomeDirectory(): string | undefined {
|
||||
async function main(): Promise<number> {
|
||||
const arguments_ = process.argv.slice(2);
|
||||
const command = arguments_[0];
|
||||
const action = arguments_[1];
|
||||
// A preview (`--yes` absent) must not create or write any diagnostic file:
|
||||
// the contract is that it only emits a plan. Diagnostics for these commands
|
||||
// are initialized only once the caller has confirmed the mutation.
|
||||
const confirmed = arguments_.includes("--yes")
|
||||
? true
|
||||
: arguments_.includes("--confirm");
|
||||
const confirmed = arguments_.includes("--yes");
|
||||
const installLike = [
|
||||
"install",
|
||||
"upgrade",
|
||||
"service-install",
|
||||
"service-uninstall",
|
||||
].includes(command ?? "");
|
||||
const planFileApply =
|
||||
(command === "sources" && action === "import") ||
|
||||
(command === "models" && (action === "configure" || action === "probe")) ||
|
||||
(command === "processing" && action === "apply") ||
|
||||
(command === "memories" && action === "apply");
|
||||
const isPreview = (installLike || planFileApply) && !confirmed;
|
||||
const mutating =
|
||||
[
|
||||
"start",
|
||||
"stop",
|
||||
"restart",
|
||||
"install",
|
||||
"upgrade",
|
||||
"service-install",
|
||||
"service-uninstall",
|
||||
].includes(command ?? "") ||
|
||||
(command === "sources" && action === "import") ||
|
||||
(command === "models" && ["configure", "probe"].includes(action ?? "")) ||
|
||||
(command === "processing" && action === "apply") ||
|
||||
(command === "memories" && action === "apply");
|
||||
const isPreview = installLike && !confirmed;
|
||||
const mutating = [
|
||||
"start",
|
||||
"stop",
|
||||
"restart",
|
||||
"install",
|
||||
"upgrade",
|
||||
"service-install",
|
||||
"service-uninstall",
|
||||
].includes(command ?? "");
|
||||
let diagnostics: DiagnosticLogger | null = null;
|
||||
if (mutating && !isPreview) {
|
||||
try {
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
/**
|
||||
* `memhub graph-width`: the user's decision when the embedding width changed
|
||||
* and the memory graph no longer matches it (`POST /api/v1/graph-width`).
|
||||
*
|
||||
* `keep_existing` and `export_first` change nothing irreversible and run
|
||||
* directly. `rebuild_with_new` archives the current graph and rebuilds it, so
|
||||
* the CLI asks for `--yes` before sending it.
|
||||
*/
|
||||
|
||||
import { parseGraphWidthDecisionReceipt } from "@one-memory/operations";
|
||||
import type { CoreOperations } from "./core-operations.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
export const GRAPH_WIDTH_DECISIONS = [
|
||||
"keep_existing",
|
||||
"rebuild_with_new",
|
||||
"export_first",
|
||||
] as const;
|
||||
export type GraphWidthDecision = (typeof GRAPH_WIDTH_DECISIONS)[number];
|
||||
|
||||
/** The one decision that cannot be taken back. */
|
||||
export function graphWidthDecisionIsDestructive(
|
||||
decision: GraphWidthDecision,
|
||||
): boolean {
|
||||
return decision === "rebuild_with_new";
|
||||
}
|
||||
|
||||
export type GraphWidthDocument =
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
command: "graph-width";
|
||||
state: "committed";
|
||||
decision: GraphWidthDecision;
|
||||
receipt: Record<string, unknown>;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
command: "graph-width";
|
||||
state: "rejected" | "unknown";
|
||||
decision: GraphWidthDecision;
|
||||
code: string;
|
||||
};
|
||||
|
||||
export async function decideGraphWidth(
|
||||
decision: GraphWidthDecision,
|
||||
operations: CoreOperations,
|
||||
): Promise<GraphWidthDocument> {
|
||||
const base = {
|
||||
schema_version: 1 as const,
|
||||
product: PRODUCT,
|
||||
command: "graph-width" as const,
|
||||
decision,
|
||||
};
|
||||
const result = await operations.post<unknown>("/api/v1/graph-width", {
|
||||
decision,
|
||||
});
|
||||
if (result.state !== "committed") {
|
||||
return { ...base, state: result.state, code: result.code };
|
||||
}
|
||||
const receipt = parseGraphWidthDecisionReceipt(result.data);
|
||||
if (receipt === null) {
|
||||
return { ...base, state: "unknown", code: "graph_width_receipt_invalid" };
|
||||
}
|
||||
return {
|
||||
...base,
|
||||
state: "committed",
|
||||
receipt: {
|
||||
archived_path: receipt.archived_path ?? null,
|
||||
exported_path: receipt.exported_path ?? null,
|
||||
mismatch: receipt.mismatch ?? null,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function graphWidthExitCode(document: GraphWidthDocument): number {
|
||||
return document.state === "committed" ? 0 : 1;
|
||||
}
|
||||
|
||||
export function renderGraphWidthText(document: GraphWidthDocument): string {
|
||||
if (document.state === "committed") {
|
||||
const lines = [`已执行图谱宽度决定:${document.decision}`];
|
||||
if (document.receipt.exported_path) {
|
||||
lines.push(`导出:${document.receipt.exported_path}`);
|
||||
}
|
||||
if (document.receipt.archived_path) {
|
||||
lines.push(`旧图谱已归档:${document.receipt.archived_path}`);
|
||||
}
|
||||
return `${lines.join("\n")}\n`;
|
||||
}
|
||||
return `图谱宽度决定未完成:${document.state}(${document.code})\n`;
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,4 +1,4 @@
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import type { ImportJob, ModelProfile } from "@one-memory/contracts";
|
||||
import {
|
||||
buildModelProfileCreateRequest,
|
||||
@@ -26,45 +26,7 @@ const PROVIDERS = [
|
||||
"local_onnx",
|
||||
] as const;
|
||||
|
||||
interface SourceImportPlanBody {
|
||||
schema_version: 1;
|
||||
kind: "source_import";
|
||||
request_id: string;
|
||||
source_ids: string[];
|
||||
source_roots: Record<string, string>;
|
||||
source_candidate_revisions: Record<string, number>;
|
||||
confirm_environment_overrides: boolean;
|
||||
full_rescan: false;
|
||||
onboarding: true;
|
||||
}
|
||||
|
||||
export type SourceImportPlan = SourceImportPlanBody & { digest: string };
|
||||
|
||||
interface ModelCreatePlanBody {
|
||||
schema_version: 1;
|
||||
kind: "model_profile_create";
|
||||
operation_id: string;
|
||||
config: Record<string, unknown>;
|
||||
}
|
||||
|
||||
export type ModelCreatePlan = ModelCreatePlanBody & { digest: string };
|
||||
|
||||
/** A removal only needs an id: there is no config to review, and inventing one
|
||||
* to fit the create shape would misrepresent what the command does.
|
||||
*/
|
||||
export interface ModelRemovePlan {
|
||||
schema_version: 1;
|
||||
kind: "model_profile_remove";
|
||||
target: string;
|
||||
}
|
||||
|
||||
export type FlowDocument =
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "preview";
|
||||
plan: SourceImportPlan | ModelCreatePlan | ModelRemovePlan;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
@@ -133,10 +95,24 @@ function classifySourceImport(
|
||||
return committed(operationId, receipt);
|
||||
}
|
||||
|
||||
export async function planSourceImport(
|
||||
/**
|
||||
* Connect sources and import their history in one direct request.
|
||||
*
|
||||
* Reads the current candidates, refuses a source whose location comes from an
|
||||
* environment override unless the caller confirmed overrides, and posts one
|
||||
* `/sources/connect` (grant + history import + live sync). `requestId` is the
|
||||
* idempotency key: re-running with the same id reads the first receipt.
|
||||
*/
|
||||
export async function connectSourcesWithImport(
|
||||
reader: CoreReader,
|
||||
sourceIds: readonly string[],
|
||||
operations: CoreOperations,
|
||||
input: {
|
||||
sourceIds: readonly string[];
|
||||
confirmOverrides: boolean;
|
||||
requestId?: string;
|
||||
},
|
||||
): Promise<FlowDocument> {
|
||||
const sourceIds = input.sourceIds;
|
||||
if (
|
||||
sourceIds.length < 1 ||
|
||||
sourceIds.length > 32 ||
|
||||
@@ -145,6 +121,8 @@ export async function planSourceImport(
|
||||
) {
|
||||
return invalid("invalid_source_selection");
|
||||
}
|
||||
const requestId = input.requestId ?? randomUUID();
|
||||
if (!UUID.test(requestId)) return invalid("invalid_request_id");
|
||||
const outcome = await reader<unknown>("/api/v1/sources");
|
||||
if (outcome.state !== "ok") return fromReadFailure(outcome);
|
||||
const candidates = validateSourceConnectionCandidates(outcome.data);
|
||||
@@ -156,69 +134,59 @@ export async function planSourceImport(
|
||||
return invalid("source_unavailable");
|
||||
}
|
||||
const items = selected.map(([candidate]) => candidate);
|
||||
const body: SourceImportPlanBody = {
|
||||
schema_version: 1,
|
||||
kind: "source_import",
|
||||
request_id: randomUUID(),
|
||||
source_ids: [...sourceIds],
|
||||
source_roots: Object.fromEntries(
|
||||
items.map((item) => [item.source_id, item.candidate_root_id]),
|
||||
),
|
||||
source_candidate_revisions: Object.fromEntries(
|
||||
items.flatMap((item) =>
|
||||
item.candidate_revision === undefined
|
||||
? []
|
||||
: [[item.source_id, item.candidate_revision]],
|
||||
),
|
||||
),
|
||||
confirm_environment_overrides: items.some(
|
||||
(item) => item.requires_override_confirmation,
|
||||
),
|
||||
full_rescan: false,
|
||||
onboarding: true,
|
||||
};
|
||||
return preview(withDigest(body));
|
||||
}
|
||||
|
||||
export async function executeSourceImport(
|
||||
value: unknown,
|
||||
reader: CoreReader,
|
||||
operations: CoreOperations,
|
||||
): Promise<FlowDocument> {
|
||||
const plan = validateSourcePlan(value);
|
||||
if (!plan) return invalid("source_plan_invalid");
|
||||
const needsOverride = items
|
||||
.filter((item) => item.requires_override_confirmation)
|
||||
.map((item) => item.source_id);
|
||||
if (needsOverride.length > 0 && !input.confirmOverrides) {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "rejected",
|
||||
operation_id: requestId,
|
||||
code: "environment_override_confirmation_required",
|
||||
receipt: { override_required_source_ids: needsOverride },
|
||||
};
|
||||
}
|
||||
// The shared builder owns the connect body's field set and coalescing, so
|
||||
// the CLI and the Web console send the same request by construction.
|
||||
let request: ReturnType<typeof buildSourceImportRequest>;
|
||||
try {
|
||||
request = buildSourceImportRequest({
|
||||
sourceIds: plan.source_ids,
|
||||
sourceRoots: plan.source_roots,
|
||||
sourceCandidateRevisions: plan.source_candidate_revisions,
|
||||
confirmEnvironmentOverrides: plan.confirm_environment_overrides,
|
||||
fullRescan: plan.full_rescan,
|
||||
onboarding: plan.onboarding,
|
||||
sourceIds: [...sourceIds],
|
||||
sourceRoots: Object.fromEntries(
|
||||
items.map((item) => [item.source_id, item.candidate_root_id]),
|
||||
),
|
||||
sourceCandidateRevisions: Object.fromEntries(
|
||||
items.flatMap((item) =>
|
||||
item.candidate_revision === undefined
|
||||
? []
|
||||
: [[item.source_id, item.candidate_revision]],
|
||||
),
|
||||
),
|
||||
confirmEnvironmentOverrides: needsOverride.length > 0,
|
||||
fullRescan: false,
|
||||
onboarding: true,
|
||||
enableSync: true,
|
||||
requestId: plan.request_id,
|
||||
requestId,
|
||||
});
|
||||
} catch (error) {
|
||||
if (error instanceof SourceImportInputError) {
|
||||
return invalid("source_plan_invalid");
|
||||
return invalid("invalid_source_selection");
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
const result = await operations.post<unknown>(request.path, request.body);
|
||||
if (result.state === "rejected") {
|
||||
return rejected(plan.request_id, result.code);
|
||||
return rejected(requestId, result.code);
|
||||
}
|
||||
if (result.state === "committed") {
|
||||
const receipt = validateSourceReceipt(result.data, plan.request_id);
|
||||
if (!receipt) return unknown(plan.request_id, "source_receipt_invalid");
|
||||
return classifySourceImport(plan.request_id, receipt);
|
||||
const receipt = validateSourceReceipt(result.data, requestId);
|
||||
if (!receipt) return unknown(requestId, "source_receipt_invalid");
|
||||
return classifySourceImport(requestId, receipt);
|
||||
}
|
||||
const recovered = await readSourceReceipt(reader, plan.request_id);
|
||||
const recovered = await readSourceReceipt(reader, requestId);
|
||||
if (recovered.state === "unknown" || recovered.state === "not_found") {
|
||||
return unknown(plan.request_id, result.code);
|
||||
return unknown(requestId, result.code);
|
||||
}
|
||||
// A recovered receipt must be classified the same way as a fresh one: an
|
||||
// all-rejected or all-busy replay is still not success. Only a terminal
|
||||
@@ -262,26 +230,23 @@ export async function readSourceReceipt(
|
||||
return committed(requestId, receipt);
|
||||
}
|
||||
|
||||
export function planModelProfile(value: unknown): FlowDocument {
|
||||
const config = validateModelConfig(value);
|
||||
if (!config) return invalid("model_config_invalid");
|
||||
const body: ModelCreatePlanBody = {
|
||||
schema_version: 1,
|
||||
kind: "model_profile_create",
|
||||
operation_id: randomUUID(),
|
||||
config,
|
||||
};
|
||||
return preview(withDigest(body));
|
||||
}
|
||||
|
||||
export async function executeModelProfile(
|
||||
/**
|
||||
* Create a model profile directly from a bounded config (never containing the
|
||||
* key) plus an optional key read from a controlled file.
|
||||
*
|
||||
* `operationId` is Core's creation idempotency key; the receipt can be read
|
||||
* later with `readModelReceipt`.
|
||||
*/
|
||||
export async function createModelProfile(
|
||||
value: unknown,
|
||||
apiKey: string | undefined,
|
||||
reader: CoreReader,
|
||||
operations: CoreOperations,
|
||||
operationId: string = randomUUID(),
|
||||
): Promise<FlowDocument> {
|
||||
const plan = validateModelPlan(value);
|
||||
if (!plan) return invalid("model_plan_invalid");
|
||||
const config = validateModelConfig(value);
|
||||
if (!config) return invalid("model_config_invalid");
|
||||
if (!UUID.test(operationId)) return invalid("invalid_operation_id");
|
||||
// The shared create builder owns the body's allow-list, bounds and the
|
||||
// creation_operation_id/api_key handling, so the CLI and the Web console
|
||||
// send the same create request.
|
||||
@@ -289,31 +254,31 @@ export async function executeModelProfile(
|
||||
try {
|
||||
request = buildModelProfileCreateRequest(
|
||||
{
|
||||
...(plan.config as Record<string, unknown>),
|
||||
...config,
|
||||
...(apiKey === undefined ? {} : { api_key: apiKey }),
|
||||
} as Parameters<typeof buildModelProfileCreateRequest>[0],
|
||||
plan.operation_id,
|
||||
operationId,
|
||||
);
|
||||
} catch (error) {
|
||||
if (error instanceof ModelProfileInputError) {
|
||||
return invalid("model_plan_invalid");
|
||||
return invalid("model_config_invalid");
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
const result = await operations.post<unknown>(request.path, request.body);
|
||||
if (result.state === "rejected") {
|
||||
return rejected(plan.operation_id, result.code);
|
||||
return rejected(operationId, result.code);
|
||||
}
|
||||
if (result.state === "committed") {
|
||||
const profile = validateModelProfile(result.data);
|
||||
return profile
|
||||
? committed(plan.operation_id, projectProfile(profile))
|
||||
: unknown(plan.operation_id, "model_receipt_invalid");
|
||||
? committed(operationId, projectProfile(profile))
|
||||
: unknown(operationId, "model_receipt_invalid");
|
||||
}
|
||||
const recovered = await readModelReceipt(reader, plan.operation_id);
|
||||
const recovered = await readModelReceipt(reader, operationId);
|
||||
return recovered.state === "committed"
|
||||
? { ...recovered, state: "recovered" }
|
||||
: unknown(plan.operation_id, result.code);
|
||||
: unknown(operationId, result.code);
|
||||
}
|
||||
|
||||
export async function readModelReceipt(
|
||||
@@ -362,27 +327,6 @@ export function parseApiKey(bytes: Buffer | null): string | undefined {
|
||||
}
|
||||
|
||||
export function renderFlowText(document: FlowDocument): string {
|
||||
if (document.state === "preview") {
|
||||
const plan = document.plan;
|
||||
// A removal has no config digest or operation id to confirm: the target id
|
||||
// *is* the thing being reviewed. Rendering fields it does not have would
|
||||
// print "undefined".
|
||||
if (plan.kind === "model_profile_remove") {
|
||||
return [
|
||||
`预览:${plan.kind}`,
|
||||
`目标配置:${plan.target}`,
|
||||
"未执行任何写操作。加 --confirm 才会真正删除。",
|
||||
"",
|
||||
].join("\n");
|
||||
}
|
||||
return [
|
||||
`预览:${plan.kind}`,
|
||||
`操作 ID:${"request_id" in plan ? plan.request_id : plan.operation_id}`,
|
||||
`确认摘要:${plan.digest}`,
|
||||
"未执行任何写操作。请把 --json 输出保存为 plan 文件,再用对应命令加 --confirm。",
|
||||
"",
|
||||
].join("\n");
|
||||
}
|
||||
const lines = [
|
||||
`操作状态:${document.state}`,
|
||||
`操作 ID:${document.operation_id ?? "无"}`,
|
||||
@@ -407,98 +351,20 @@ export function renderFlowText(document: FlowDocument): string {
|
||||
if (document.state === "partial" && Array.isArray(accepted)) {
|
||||
lines.push(` 已接入:${accepted.join("、")}`);
|
||||
}
|
||||
const overrides = document.receipt.override_required_source_ids;
|
||||
if (Array.isArray(overrides)) {
|
||||
lines.push(
|
||||
` 这些来源的目录来自环境变量覆盖,需要确认:${overrides.join("、")}`,
|
||||
" 确认目录无误后加 --confirm-overrides 重新运行。",
|
||||
);
|
||||
}
|
||||
}
|
||||
lines.push("");
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
export function flowExitCode(document: FlowDocument): number {
|
||||
return ["preview", "committed", "recovered"].includes(document.state) ? 0 : 1;
|
||||
}
|
||||
|
||||
export function inspectExecutionPlan(value: unknown): {
|
||||
kind: "source_import" | "model_profile_create";
|
||||
operationId: string;
|
||||
digest: string;
|
||||
} | null {
|
||||
const source = validateSourcePlan(value);
|
||||
if (source) {
|
||||
return {
|
||||
kind: source.kind,
|
||||
operationId: source.request_id,
|
||||
digest: source.digest,
|
||||
};
|
||||
}
|
||||
const model = validateModelPlan(value);
|
||||
return model
|
||||
? {
|
||||
kind: model.kind,
|
||||
operationId: model.operation_id,
|
||||
digest: model.digest,
|
||||
}
|
||||
: null;
|
||||
}
|
||||
|
||||
function validateSourcePlan(value: unknown): SourceImportPlan | null {
|
||||
const plan = record(value);
|
||||
const requestId = asString(plan?.request_id);
|
||||
if (
|
||||
plan?.schema_version !== 1 ||
|
||||
plan.kind !== "source_import" ||
|
||||
!UUID.test(requestId) ||
|
||||
!Array.isArray(plan.source_ids) ||
|
||||
plan.source_ids.length < 1 ||
|
||||
plan.source_ids.length > 32 ||
|
||||
plan.source_ids.some(
|
||||
(id) => typeof id !== "string" || !SOURCE_ID.test(id),
|
||||
) ||
|
||||
new Set(plan.source_ids).size !== plan.source_ids.length ||
|
||||
plan.full_rescan !== false ||
|
||||
plan.onboarding !== true ||
|
||||
typeof plan.confirm_environment_overrides !== "boolean" ||
|
||||
typeof plan.digest !== "string"
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const sourceIds = plan.source_ids as string[];
|
||||
const roots = stringMap(plan.source_roots, sourceIds);
|
||||
const revisions = numberMap(plan.source_candidate_revisions, sourceIds);
|
||||
if (!roots || !revisions) return null;
|
||||
const body: SourceImportPlanBody = {
|
||||
schema_version: 1,
|
||||
kind: "source_import",
|
||||
request_id: requestId,
|
||||
source_ids: [...sourceIds],
|
||||
source_roots: roots,
|
||||
source_candidate_revisions: revisions,
|
||||
confirm_environment_overrides: plan.confirm_environment_overrides,
|
||||
full_rescan: false,
|
||||
onboarding: true,
|
||||
};
|
||||
const expected = withDigest(body);
|
||||
return timingSafeText(expected.digest, plan.digest) ? expected : null;
|
||||
}
|
||||
|
||||
function validateModelPlan(value: unknown): ModelCreatePlan | null {
|
||||
const plan = record(value);
|
||||
const operationId = asString(plan?.operation_id);
|
||||
if (
|
||||
plan?.schema_version !== 1 ||
|
||||
plan.kind !== "model_profile_create" ||
|
||||
!UUID.test(operationId) ||
|
||||
typeof plan.digest !== "string"
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const config = validateModelConfig(plan.config);
|
||||
if (!config) return null;
|
||||
const expected = withDigest<ModelCreatePlanBody>({
|
||||
schema_version: 1,
|
||||
kind: "model_profile_create",
|
||||
operation_id: operationId,
|
||||
config,
|
||||
});
|
||||
return timingSafeText(expected.digest, plan.digest) ? expected : null;
|
||||
return ["committed", "recovered"].includes(document.state) ? 0 : 1;
|
||||
}
|
||||
|
||||
function validateModelConfig(value: unknown): Record<string, unknown> | null {
|
||||
@@ -644,37 +510,6 @@ function projectJob(job: ImportJob): Record<string, unknown> {
|
||||
};
|
||||
}
|
||||
|
||||
function withDigest<T extends object>(body: T): T & { digest: string } {
|
||||
return { ...body, digest: digest(body) };
|
||||
}
|
||||
|
||||
function digest(value: unknown): string {
|
||||
return createHash("sha256").update(canonical(value)).digest("hex");
|
||||
}
|
||||
|
||||
function canonical(value: unknown): string {
|
||||
if (Array.isArray(value)) return `[${value.map(canonical).join(",")}]`;
|
||||
if (value && typeof value === "object") {
|
||||
return `{${Object.keys(value)
|
||||
.sort()
|
||||
.map(
|
||||
(key) =>
|
||||
`${JSON.stringify(key)}:${canonical((value as Record<string, unknown>)[key])}`,
|
||||
)
|
||||
.join(",")}}`;
|
||||
}
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
|
||||
function preview(plan: SourceImportPlan | ModelCreatePlan): FlowDocument {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "preview",
|
||||
plan,
|
||||
};
|
||||
}
|
||||
|
||||
function committed(
|
||||
operationId: string,
|
||||
receipt: Record<string, unknown>,
|
||||
@@ -767,43 +602,3 @@ function optionalInteger(
|
||||
(value as number) <= maximum)
|
||||
);
|
||||
}
|
||||
|
||||
function stringMap(
|
||||
value: unknown,
|
||||
keys: readonly string[],
|
||||
): Record<string, string> | null {
|
||||
const map = record(value);
|
||||
if (
|
||||
!map ||
|
||||
Object.keys(map).length !== keys.length ||
|
||||
keys.some((key) => !boundedText(map[key], 256))
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return Object.fromEntries(keys.map((key) => [key, map[key] as string]));
|
||||
}
|
||||
|
||||
function numberMap(
|
||||
value: unknown,
|
||||
allowedKeys: readonly string[],
|
||||
): Record<string, number> | null {
|
||||
const map = record(value);
|
||||
if (
|
||||
!map ||
|
||||
Object.entries(map).some(
|
||||
([key, revision]) =>
|
||||
!allowedKeys.includes(key) ||
|
||||
!Number.isSafeInteger(revision) ||
|
||||
(revision as number) < 0,
|
||||
)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return Object.fromEntries(
|
||||
Object.entries(map).map(([key, revision]) => [key, revision as number]),
|
||||
);
|
||||
}
|
||||
|
||||
function timingSafeText(expected: string, actual: unknown): boolean {
|
||||
return typeof actual === "string" && expected === actual;
|
||||
}
|
||||
@@ -1,64 +0,0 @@
|
||||
import path from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { initPlanAttemptPath } from "./cli.js";
|
||||
import { interactiveInitAttemptPath } from "./init-attempt.js";
|
||||
|
||||
// Regression: `init --source <id> --import` used to pass the literal string
|
||||
// `"init"` to `claimPlanAttempt`, which resolved to `<cwd>/init.one-memory-attempt`.
|
||||
// Two consequences: the fence was shared across unrelated runs in the same
|
||||
// directory, and any unrelated file named `init` could make a valid import
|
||||
// report a misleading usage error (exit 2) instead of reading its receipt.
|
||||
describe("init import attempt fence", () => {
|
||||
it("lives under the data dir, not the current working directory", () => {
|
||||
const fence = initPlanAttemptPath(
|
||||
"/tmp/example data",
|
||||
"11111111-2222-4333-8444-555555555555",
|
||||
);
|
||||
expect(fence).toBe(
|
||||
path.join(
|
||||
"/tmp/example data",
|
||||
".cli-init-attempt-11111111-2222-4333-8444-555555555555",
|
||||
),
|
||||
);
|
||||
expect(path.isAbsolute(fence)).toBe(true);
|
||||
expect(fence).not.toBe("init");
|
||||
// Directly under the data dir: no nested directory that could fail to be
|
||||
// created and turn a valid import into a usage error.
|
||||
expect(path.dirname(fence)).toBe("/tmp/example data");
|
||||
});
|
||||
|
||||
it("is per data dir and per operation, so installs/runs cannot collide", () => {
|
||||
const a = initPlanAttemptPath("/tmp/data-a", "op-1");
|
||||
const b = initPlanAttemptPath("/tmp/data-b", "op-1");
|
||||
const c = initPlanAttemptPath("/tmp/data-a", "op-2");
|
||||
expect(new Set([a, b, c]).size).toBe(3);
|
||||
});
|
||||
});
|
||||
|
||||
// QA-CLI R3 F1: the interactive wizard passed a CWD-relative singleton
|
||||
// (`init` or `<journalDir>/init`) to `claimPlanAttempt`. A stale
|
||||
// `init.one-memory-attempt` in the working directory then made every later
|
||||
// `memhub init` fail with "交互计划无效 / exit 2".
|
||||
describe("interactive init attempt fence", () => {
|
||||
it("lives under the data dir and is unique per operation", () => {
|
||||
const fence = interactiveInitAttemptPath(
|
||||
"/tmp/data",
|
||||
"11111111-2222-4333-8444-555555555555",
|
||||
);
|
||||
expect(fence).toBe(
|
||||
path.join(
|
||||
"/tmp/data",
|
||||
".cli-init-attempt-11111111-2222-4333-8444-555555555555",
|
||||
),
|
||||
);
|
||||
expect(path.isAbsolute(fence)).toBe(true);
|
||||
expect(fence).not.toBe("init");
|
||||
});
|
||||
|
||||
it("does not reuse one fence for two distinct plans", () => {
|
||||
const first = interactiveInitAttemptPath("/tmp/data", "op-1");
|
||||
const second = interactiveInitAttemptPath("/tmp/data", "op-2");
|
||||
expect(first).not.toBe(second);
|
||||
});
|
||||
});
|
||||
@@ -1,16 +0,0 @@
|
||||
import path from "node:path";
|
||||
|
||||
/** Where the interactive `init` wizard keeps its durable attempt fence.
|
||||
*
|
||||
* The wizard originally passed a CWD-relative singleton (`init`) to
|
||||
* `claimPlanAttempt`. That shared one fence across unrelated runs and let a
|
||||
* stale `init.one-memory-attempt` in the working directory turn every later
|
||||
* wizard run into "交互计划无效 / exit 2". Keep the fence under the Runtime's
|
||||
* data dir, keyed by the operation id, exactly like `init --source --import`.
|
||||
*/
|
||||
export function interactiveInitAttemptPath(
|
||||
dataDir: string,
|
||||
operationId: string,
|
||||
): string {
|
||||
return path.join(dataDir, `.cli-init-attempt-${operationId}`);
|
||||
}
|
||||
@@ -418,9 +418,9 @@ describe("A2 dual-mode init", () => {
|
||||
});
|
||||
|
||||
// QA-CLI R3 F1: a stale CWD `init.one-memory-attempt` used to poison every
|
||||
// later wizard run. The fence now lives under the isolated data dir and is
|
||||
// keyed by the operation id.
|
||||
it("ignores a stale CWD init fence and keeps its fence under the data dir", async () => {
|
||||
// later wizard run. There are no plan files or attempt fences any more: the
|
||||
// connect request id is the idempotency key Core keeps.
|
||||
it("ignores a stale CWD init fence and writes no fence files", async () => {
|
||||
transport.get.mockImplementation(async (url: string) =>
|
||||
url === "/api/v1/sources"
|
||||
? { status: 200, data: [candidate("codex", "Codex")] }
|
||||
@@ -471,7 +471,7 @@ describe("A2 dual-mode init", () => {
|
||||
const dataEntries = await readdir(dataDir);
|
||||
expect(
|
||||
dataEntries.some((entry) => entry.startsWith(".cli-init-attempt-")),
|
||||
).toBe(true);
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("keeps selected Harnesses frozen without creating history jobs", async () => {
|
||||
@@ -506,7 +506,7 @@ describe("A2 dual-mode init", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("A19 init --all-detected", () => {
|
||||
describe("A19 sources connect --all-detected (old spelling: init --all-detected)", () => {
|
||||
it("connects every found source in one batch call without prompting", async () => {
|
||||
transport.get.mockImplementation(async (url: string) =>
|
||||
url === "/api/v1/sources"
|
||||
@@ -556,7 +556,7 @@ describe("A19 init --all-detected", () => {
|
||||
expect(prompter.asked).toBe(0);
|
||||
expect(code).toBe(0);
|
||||
const document = JSON.parse(streams.value());
|
||||
expect(document.command).toBe("init");
|
||||
expect(document.command).toBe("sources connect");
|
||||
expect(document.state).toBe("committed");
|
||||
expect(
|
||||
document.sources.map((row: { source_id: string }) => row.source_id),
|
||||
|
||||
@@ -4,18 +4,11 @@ import type {
|
||||
CoreDeleteOperations,
|
||||
CoreOperations,
|
||||
} from "./core-operations.js";
|
||||
import {
|
||||
executeModelProfile,
|
||||
type FlowDocument,
|
||||
planModelProfile,
|
||||
} from "./headless-flows.js";
|
||||
import { createModelProfile, type FlowDocument } from "./headless-flows.js";
|
||||
import { approvalFingerprint, listHosts, readHost } from "./host-flow.js";
|
||||
import { parseSelection, type WizardPrompter } from "./init-wizard.js";
|
||||
import { executeModelProbe, planModelProbe } from "./model-probe-flow.js";
|
||||
import {
|
||||
executeModelProfileActivate,
|
||||
planModelProfileActivate,
|
||||
} from "./model-write-flows.js";
|
||||
import { startModelProbe } from "./model-probe-flow.js";
|
||||
import { activateModelProfile } from "./model-write-flows.js";
|
||||
|
||||
type ProviderPreset = {
|
||||
id: string;
|
||||
@@ -359,30 +352,26 @@ export async function runInteractiveModelSetup(
|
||||
);
|
||||
})
|
||||
: new Map<string, string>();
|
||||
const planned = planModelProfile({
|
||||
name: preset.label,
|
||||
provider_kind: preset.provider_kind,
|
||||
base_url: baseUrl,
|
||||
model,
|
||||
embedding_provider_kind: "local_onnx",
|
||||
embedding_model: "bge-small-zh-v1.5",
|
||||
embedding_dimensions: 512,
|
||||
timeout_seconds: 90,
|
||||
max_concurrency: 4,
|
||||
disable_thinking: true,
|
||||
});
|
||||
if (
|
||||
planned.state !== "preview" ||
|
||||
planned.plan.kind !== "model_profile_create"
|
||||
) {
|
||||
return { state: "failed", code: "model_config_invalid" };
|
||||
}
|
||||
const created = await executeModelProfile(
|
||||
planned.plan,
|
||||
const created = await createModelProfile(
|
||||
{
|
||||
name: preset.label,
|
||||
provider_kind: preset.provider_kind,
|
||||
base_url: baseUrl,
|
||||
model,
|
||||
embedding_provider_kind: "local_onnx",
|
||||
embedding_model: "bge-small-zh-v1.5",
|
||||
embedding_dimensions: 512,
|
||||
timeout_seconds: 90,
|
||||
max_concurrency: 4,
|
||||
disable_thinking: true,
|
||||
},
|
||||
apiKey,
|
||||
clients.reader,
|
||||
clients.operations,
|
||||
);
|
||||
if (created.state === "invalid") {
|
||||
return { state: "failed", code: "model_config_invalid" };
|
||||
}
|
||||
const profile = receiptProfile(created);
|
||||
if (profile === null) {
|
||||
prompter.write("模型配置未保存。");
|
||||
@@ -391,18 +380,11 @@ export async function runInteractiveModelSetup(
|
||||
}
|
||||
|
||||
try {
|
||||
const probePlan = await planModelProbe(
|
||||
const probed = await startModelProbe(
|
||||
clients.reader,
|
||||
clients.operations,
|
||||
profile.profile_id,
|
||||
);
|
||||
const probed =
|
||||
probePlan.state === "preview"
|
||||
? await executeModelProbe(
|
||||
probePlan.plan,
|
||||
clients.reader,
|
||||
clients.operations,
|
||||
)
|
||||
: probePlan;
|
||||
if (!probePassed(probed)) {
|
||||
await clients.operations.delete(
|
||||
`/api/v1/model-profiles/${encodeURIComponent(profile.profile_id)}`,
|
||||
@@ -421,17 +403,8 @@ export async function runInteractiveModelSetup(
|
||||
throw error;
|
||||
}
|
||||
|
||||
const activation = planModelProfileActivate({
|
||||
profileId: profile.profile_id,
|
||||
});
|
||||
if (
|
||||
activation.state !== "preview" ||
|
||||
activation.plan.kind !== "model_profile_activate"
|
||||
) {
|
||||
return { state: "failed", code: "model_activate_plan_invalid" };
|
||||
}
|
||||
const activated = await executeModelProfileActivate(
|
||||
activation.plan,
|
||||
const activated = await activateModelProfile(
|
||||
profile.profile_id,
|
||||
clients.operations,
|
||||
);
|
||||
if (activated.state !== "committed") {
|
||||
|
||||
@@ -101,7 +101,7 @@ describe("memhub jobs", () => {
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("start previews without a POST", async () => {
|
||||
it("start posts directly, without a preview step", async () => {
|
||||
const output = captureStdout();
|
||||
let code: number;
|
||||
try {
|
||||
@@ -110,14 +110,14 @@ describe("memhub jobs", () => {
|
||||
output.restore();
|
||||
}
|
||||
expect(code!).toBe(0);
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
expect(JSON.parse(output.value())).toMatchObject({
|
||||
state: "preview",
|
||||
plan: { source_id: "codex", operation: "history" },
|
||||
});
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
"/api/v1/import-jobs",
|
||||
{ source_id: "codex", operation: "history" },
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
it("start --confirm posts the exact Core request", async () => {
|
||||
it("start still accepts a legacy --confirm", async () => {
|
||||
const output = captureStdout();
|
||||
let code: number;
|
||||
try {
|
||||
@@ -146,7 +146,7 @@ describe("memhub jobs", () => {
|
||||
});
|
||||
|
||||
it.each(["pause", "resume", "cancel"] as const)(
|
||||
"%s previews without a POST",
|
||||
"%s posts the action endpoint directly",
|
||||
async (action) => {
|
||||
const output = captureStdout();
|
||||
let code: number;
|
||||
@@ -156,11 +156,11 @@ describe("memhub jobs", () => {
|
||||
output.restore();
|
||||
}
|
||||
expect(code!).toBe(0);
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
expect(JSON.parse(output.value())).toMatchObject({
|
||||
state: "preview",
|
||||
plan: { job_id: "job-1", action },
|
||||
});
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
`/api/v1/import-jobs/job-1/${action}`,
|
||||
{},
|
||||
undefined,
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
|
||||
@@ -0,0 +1,583 @@
|
||||
import { mkdtemp, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { run } from "./cli.js";
|
||||
|
||||
const transport = vi.hoisted(() => ({
|
||||
get: vi.fn(),
|
||||
post: vi.fn(),
|
||||
put: vi.fn(),
|
||||
patch: vi.fn(),
|
||||
delete: vi.fn(),
|
||||
}));
|
||||
vi.mock("axios", () => ({
|
||||
default: { create: vi.fn(() => transport), get: vi.fn() },
|
||||
}));
|
||||
|
||||
const runtime = vi.hoisted(() => ({
|
||||
inspectRuntimeStatus: vi.fn(),
|
||||
discoverRuntimeAttachment: vi.fn(),
|
||||
}));
|
||||
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
|
||||
...runtime,
|
||||
}));
|
||||
|
||||
const REQUEST = "33333333-3333-4333-8333-333333333333";
|
||||
const EVIDENCE = "44444444-4444-4444-8444-444444444444";
|
||||
|
||||
function addReceipt(state = "pending", memoryIds: string[] = []) {
|
||||
return {
|
||||
evidence_id: EVIDENCE,
|
||||
text: "Prefers tea",
|
||||
scope: "all-apps",
|
||||
created_at: "2026-10-27T00:00:00Z",
|
||||
state,
|
||||
task_id: null,
|
||||
memory_ids: memoryIds,
|
||||
error: null,
|
||||
};
|
||||
}
|
||||
|
||||
function memoryRecord(overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
memory_id: "exp-1",
|
||||
content: "deploy\nrun make",
|
||||
layer: "experience",
|
||||
status: "active",
|
||||
is_latest: true,
|
||||
supersedes: [],
|
||||
superseded_by: [],
|
||||
tags: [],
|
||||
owner: null,
|
||||
source_raw_ids: [],
|
||||
user_id: "local",
|
||||
agent_id: "",
|
||||
session_id: null,
|
||||
source: "codebuddy",
|
||||
memory_at: null,
|
||||
gmt_created: 1_790_000_000,
|
||||
scene: "working",
|
||||
kind: "experience",
|
||||
variant: "working",
|
||||
task: "deploy",
|
||||
solution: "run make",
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
transport.get.mockReset();
|
||||
transport.post.mockReset();
|
||||
transport.put.mockReset();
|
||||
transport.patch.mockReset();
|
||||
transport.delete.mockReset();
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "running",
|
||||
code: "runtime_verified",
|
||||
instanceId: "instance-1",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
version: "1.2.3",
|
||||
});
|
||||
runtime.discoverRuntimeAttachment.mockResolvedValue({
|
||||
instanceId: "instance-1",
|
||||
apiToken: "synthetic-runtime-token",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
servicePid: 4242,
|
||||
health: {
|
||||
status: "ok",
|
||||
service: "one-memory-core",
|
||||
version: "1.2.3",
|
||||
instance_id: "instance-1",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
describe("memories add", () => {
|
||||
it("adds one memory directly and prints the receipt with its request id", async () => {
|
||||
transport.post.mockResolvedValue({ status: 201, data: addReceipt() });
|
||||
const result = await capturedRun([
|
||||
"memories",
|
||||
"add",
|
||||
"Prefers tea",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
const [url, body] = transport.post.mock.calls[0];
|
||||
expect(url).toBe("/api/v2/memories");
|
||||
expect(body).toMatchObject({ text: "Prefers tea", scope: "all-apps" });
|
||||
expect(body.request_id).toMatch(/^[0-9a-f-]{36}$/);
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
schema_version: 2,
|
||||
command: "memories add",
|
||||
state: "accepted",
|
||||
request_id: body.request_id,
|
||||
receipt: { evidence_id: EVIDENCE, state: "pending" },
|
||||
});
|
||||
});
|
||||
|
||||
it("reuses --request-id so a retry is idempotent, and says so in text", async () => {
|
||||
transport.post.mockResolvedValue({ status: 201, data: addReceipt() });
|
||||
const result = await capturedRun([
|
||||
"memories",
|
||||
"add",
|
||||
"Prefers tea",
|
||||
"--request-id",
|
||||
REQUEST,
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.post.mock.calls[0][1].request_id).toBe(REQUEST);
|
||||
expect(result.stdout).toContain(`request_id: ${REQUEST}`);
|
||||
expect(result.stdout).toContain(`memhub memories add-status ${REQUEST}`);
|
||||
});
|
||||
|
||||
it("reads the text from a bounded file", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "om-memory-add-"));
|
||||
const file = path.join(directory, "memory.txt");
|
||||
await writeFile(file, "Prefers tea\n", { mode: 0o600 });
|
||||
transport.post.mockResolvedValue({ status: 201, data: addReceipt() });
|
||||
const result = await capturedRun([
|
||||
"memories",
|
||||
"add",
|
||||
"--content-file",
|
||||
file,
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.post.mock.calls[0][1].text).toBe("Prefers tea");
|
||||
});
|
||||
|
||||
it("refuses bad input before contacting Core", async () => {
|
||||
for (const argv of [
|
||||
["memories", "add"],
|
||||
["memories", "add", "a", "--content-file", "/tmp/x"],
|
||||
["memories", "add", "a", "--request-id", "not-a-uuid"],
|
||||
["memories", "add", " "],
|
||||
["memories", "add", "x".repeat(12_001)],
|
||||
]) {
|
||||
expect((await capturedRun(argv)).code).toBe(2);
|
||||
}
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
expect(runtime.inspectRuntimeStatus).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("reports a Core conflict as rejected, on stdout, exit 1", async () => {
|
||||
transport.post.mockResolvedValue({
|
||||
status: 409,
|
||||
data: { detail: "memory_add_conflict" },
|
||||
});
|
||||
const result = await capturedRun([
|
||||
"memories",
|
||||
"add",
|
||||
"Other text",
|
||||
"--request-id",
|
||||
REQUEST,
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
state: "rejected",
|
||||
code: "memory_add_conflict",
|
||||
request_id: REQUEST,
|
||||
});
|
||||
});
|
||||
|
||||
it("never reports a malformed success body as accepted", async () => {
|
||||
transport.post.mockResolvedValue({
|
||||
status: 201,
|
||||
data: { evidence_id: "x" },
|
||||
});
|
||||
const result = await capturedRun(["memories", "add", "tea", "--json"]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(JSON.parse(result.stdout).state).toBe("unknown");
|
||||
});
|
||||
|
||||
it("answers a stopped Runtime with exit 3 and writes nothing", async () => {
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "stopped",
|
||||
code: "runtime_not_running",
|
||||
});
|
||||
const result = await capturedRun(["memories", "add", "tea", "--json"]);
|
||||
expect(result.code).toBe(3);
|
||||
expect(JSON.parse(result.stdout).code).toBe("runtime_not_running");
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe("memories add-status", () => {
|
||||
it("reads the addition by request id", async () => {
|
||||
transport.get.mockResolvedValue({
|
||||
status: 200,
|
||||
data: addReceipt("ready", ["m-1"]),
|
||||
});
|
||||
const result = await capturedRun([
|
||||
"memories",
|
||||
"add-status",
|
||||
REQUEST,
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.get).toHaveBeenCalledWith(
|
||||
`/api/v2/memories/additions/${REQUEST}`,
|
||||
undefined,
|
||||
);
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
command: "memories add-status",
|
||||
state: "accepted",
|
||||
receipt: { state: "ready", memory_ids: ["m-1"] },
|
||||
});
|
||||
});
|
||||
|
||||
it("reports an unknown request id as not_found, exit 1", async () => {
|
||||
transport.get.mockResolvedValue({
|
||||
status: 404,
|
||||
data: { detail: "memory_add_not_found" },
|
||||
});
|
||||
const result = await capturedRun(["memories", "status", REQUEST]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(result.stdout).toContain("未找到该 request id");
|
||||
});
|
||||
|
||||
it("refuses a non-UUID request id", async () => {
|
||||
expect((await capturedRun(["memories", "add-status", "abc"])).code).toBe(2);
|
||||
expect(transport.get).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe("memories update / delete pass the namespace", () => {
|
||||
it("updates an experience with kind, variant and task, directly", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "om-memory-update-"));
|
||||
const file = path.join(directory, "solution.txt");
|
||||
await writeFile(file, "run make deploy\n", { mode: 0o600 });
|
||||
transport.put.mockResolvedValue({ status: 200, data: memoryRecord() });
|
||||
const result = await capturedRun([
|
||||
"memories",
|
||||
"update",
|
||||
"exp-1",
|
||||
"--content-file",
|
||||
file,
|
||||
"--kind",
|
||||
"experience",
|
||||
"--variant",
|
||||
"working",
|
||||
"--task",
|
||||
"deploy",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.put).toHaveBeenCalledWith(
|
||||
"/api/v2/memories/exp-1",
|
||||
{
|
||||
content: "run make deploy",
|
||||
kind: "experience",
|
||||
variant: "working",
|
||||
task: "deploy",
|
||||
},
|
||||
undefined,
|
||||
);
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
schema_version: 2,
|
||||
state: "committed",
|
||||
operation: "update",
|
||||
});
|
||||
});
|
||||
|
||||
it("refuses --task on anything but an experience, and a lone --variant", async () => {
|
||||
for (const argv of [
|
||||
["memories", "update", "m-1", "--content-file", "/tmp/x", "--task", "t"],
|
||||
[
|
||||
"memories",
|
||||
"update",
|
||||
"m-1",
|
||||
"--content-file",
|
||||
"/tmp/x",
|
||||
"--kind",
|
||||
"fact",
|
||||
"--variant",
|
||||
"coding",
|
||||
"--task",
|
||||
"t",
|
||||
],
|
||||
["memories", "delete", "m-1", "--variant", "coding"],
|
||||
["memories", "delete", "m-1", "--kind", "chat", "--variant", "coding"],
|
||||
["memories", "show", "m-1", "--kind", "nope"],
|
||||
]) {
|
||||
expect((await capturedRun(argv)).code).toBe(2);
|
||||
}
|
||||
expect(transport.put).not.toHaveBeenCalled();
|
||||
expect(transport.delete).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("previews a delete without --yes and changes nothing", async () => {
|
||||
const result = await capturedRun([
|
||||
"memories",
|
||||
"delete",
|
||||
"fact-1",
|
||||
"--kind",
|
||||
"fact",
|
||||
"--variant",
|
||||
"coding",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
schema_version: 2,
|
||||
command: "memories delete",
|
||||
state: "preview",
|
||||
target: "fact-1",
|
||||
applied: false,
|
||||
});
|
||||
expect(JSON.parse(result.stdout).next_step).toContain(
|
||||
"--kind fact --variant coding --yes",
|
||||
);
|
||||
expect(transport.delete).not.toHaveBeenCalled();
|
||||
expect(runtime.inspectRuntimeStatus).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("deletes with --yes (or the legacy --confirm) and names the namespace", async () => {
|
||||
transport.delete.mockResolvedValue({ status: 204, data: "" });
|
||||
for (const flag of ["--yes", "--confirm"]) {
|
||||
const result = await capturedRun([
|
||||
"memories",
|
||||
"delete",
|
||||
"fact-1",
|
||||
"--kind",
|
||||
"fact",
|
||||
"--variant",
|
||||
"coding",
|
||||
flag,
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
schema_version: 2,
|
||||
state: "committed",
|
||||
operation: "delete",
|
||||
});
|
||||
}
|
||||
expect(transport.delete).toHaveBeenCalledWith(
|
||||
"/api/v2/memories/fact-1?kind=fact&variant=coding",
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
it("explains an ambiguous id instead of a bare code", async () => {
|
||||
transport.delete.mockResolvedValue({
|
||||
status: 409,
|
||||
data: { detail: "memory_id_ambiguous" },
|
||||
});
|
||||
const result = await capturedRun(["memories", "delete", "dup", "--yes"]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(result.stdout).toContain("--kind");
|
||||
});
|
||||
|
||||
it("shows a memory in its namespace", async () => {
|
||||
transport.get.mockResolvedValue({ status: 200, data: memoryRecord() });
|
||||
const result = await capturedRun([
|
||||
"memories",
|
||||
"show",
|
||||
"exp-1",
|
||||
"--kind",
|
||||
"experience",
|
||||
"--variant",
|
||||
"working",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.get).toHaveBeenCalledWith("/api/v2/memories/exp-1", {
|
||||
params: { kind: "experience", variant: "working" },
|
||||
});
|
||||
const document = JSON.parse(result.stdout);
|
||||
expect(document.schema_version).toBe(2);
|
||||
expect(document).not.toHaveProperty("evidence");
|
||||
expect(document).not.toHaveProperty("history");
|
||||
expect(document.memory).toMatchObject({
|
||||
kind: "experience",
|
||||
variant: "working",
|
||||
task: "deploy",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("memories list filters", () => {
|
||||
const page = {
|
||||
memories: [memoryRecord()],
|
||||
total: 1,
|
||||
limit: 100,
|
||||
offset: 0,
|
||||
search_window: 200,
|
||||
window_id: "w1",
|
||||
window_reset: false,
|
||||
};
|
||||
|
||||
it("sends --scene, --source and --query to the library", async () => {
|
||||
transport.get.mockResolvedValue({ status: 200, data: page });
|
||||
const result = await capturedRun([
|
||||
"memories",
|
||||
"list",
|
||||
"--scene",
|
||||
"working",
|
||||
"--source",
|
||||
"codebuddy",
|
||||
"--query",
|
||||
"deploy",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.get).toHaveBeenCalledWith("/api/v2/memory-library", {
|
||||
params: {
|
||||
offset: 0,
|
||||
limit: 100,
|
||||
scene: "working",
|
||||
source: "codebuddy",
|
||||
query: "deploy",
|
||||
},
|
||||
});
|
||||
const document = JSON.parse(result.stdout);
|
||||
expect(document.schema_version).toBe(2);
|
||||
expect(document.page).toMatchObject({
|
||||
state: "ok",
|
||||
scene: "working",
|
||||
query: "deploy",
|
||||
search_window: 200,
|
||||
});
|
||||
expect(document.memories[0]).toMatchObject({
|
||||
kind: "experience",
|
||||
variant: "working",
|
||||
});
|
||||
});
|
||||
|
||||
it("omits scene for the default all", async () => {
|
||||
transport.get.mockResolvedValue({ status: 200, data: page });
|
||||
await capturedRun(["memories", "list", "--scene", "all", "--json"]);
|
||||
expect(transport.get.mock.calls[0][1].params).not.toHaveProperty("scene");
|
||||
});
|
||||
|
||||
it("validates --status, --scene and --query locally instead of reporting a busy backend", async () => {
|
||||
for (const argv of [
|
||||
["memories", "list", "--status", "bogus"],
|
||||
["memories", "list", "--scene", "normal"],
|
||||
["memories", "list", "--query", "x", "--status", "archived"],
|
||||
["memories", "list", "--source", "Bad Source"],
|
||||
]) {
|
||||
expect((await capturedRun(argv)).code).toBe(2);
|
||||
}
|
||||
expect(transport.get).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("reports a Core 422 as rejected, not as 后台正忙", async () => {
|
||||
transport.get.mockResolvedValue({
|
||||
status: 422,
|
||||
data: { detail: "sdk_search_supports_active_status_only" },
|
||||
});
|
||||
const result = await capturedRun(["memories", "list"]);
|
||||
expect(result.code).toBe(1);
|
||||
expect(result.stdout).not.toContain("后台正忙");
|
||||
expect(result.stdout).toContain("sdk_search_supports_active_status_only");
|
||||
});
|
||||
});
|
||||
|
||||
describe("graph-width", () => {
|
||||
it("runs a non-destructive decision directly", async () => {
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: { archived_path: null, exported_path: "/tmp/x", mismatch: null },
|
||||
});
|
||||
const result = await capturedRun([
|
||||
"graph-width",
|
||||
"--decision",
|
||||
"export_first",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
"/api/v1/graph-width",
|
||||
{ decision: "export_first" },
|
||||
undefined,
|
||||
);
|
||||
expect(JSON.parse(result.stdout).state).toBe("committed");
|
||||
});
|
||||
|
||||
it("previews rebuild_with_new without --yes and applies it with --yes", async () => {
|
||||
const preview = await capturedRun([
|
||||
"graph-width",
|
||||
"--decision",
|
||||
"rebuild_with_new",
|
||||
"--json",
|
||||
]);
|
||||
expect(preview.code).toBe(0);
|
||||
expect(JSON.parse(preview.stdout)).toMatchObject({
|
||||
state: "preview",
|
||||
applied: false,
|
||||
});
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: { archived_path: "/tmp/old", exported_path: null, mismatch: null },
|
||||
});
|
||||
const applied = await capturedRun([
|
||||
"graph",
|
||||
"width",
|
||||
"--decision",
|
||||
"rebuild_with_new",
|
||||
"--yes",
|
||||
"--json",
|
||||
]);
|
||||
expect(applied.code).toBe(0);
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
"/api/v1/graph-width",
|
||||
{ decision: "rebuild_with_new" },
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("removed commands", () => {
|
||||
it.each([
|
||||
["notes", "add", "--content-file", "/tmp/x"],
|
||||
["notes", "process", REQUEST, "--confirm"],
|
||||
["calls", "clear", "--all", "--confirm"],
|
||||
["processing", "extract", "--evidence", REQUEST, "--confirm"],
|
||||
["extraction", "set", "--window", "6m", "--confirm"],
|
||||
["import-file", "upload", "--file", "/tmp/x", "--confirm"],
|
||||
])("%s is an unknown command that writes nothing", async (...argv) => {
|
||||
const result = await capturedRun(argv as string[]);
|
||||
expect(result.code).toBe(2);
|
||||
expect(result.stderr).toContain("未知命令");
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
async function capturedRun(arguments_: string[]) {
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
const stdoutSpy = vi
|
||||
.spyOn(process.stdout, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stdout += String(chunk);
|
||||
return true;
|
||||
});
|
||||
const stderrSpy = vi
|
||||
.spyOn(process.stderr, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stderr += String(chunk);
|
||||
return true;
|
||||
});
|
||||
try {
|
||||
return {
|
||||
code: await run(arguments_, {
|
||||
ONE_MEMORY_DATA_DIR: "/synthetic/data",
|
||||
}),
|
||||
stdout,
|
||||
stderr,
|
||||
};
|
||||
} finally {
|
||||
stdoutSpy.mockRestore();
|
||||
stderrSpy.mockRestore();
|
||||
}
|
||||
}
|
||||
@@ -1,700 +0,0 @@
|
||||
import { createHash, randomUUID, timingSafeEqual } from "node:crypto";
|
||||
import type { GovernedMemory } from "@one-memory/contracts";
|
||||
import {
|
||||
buildMemoryActionRequest,
|
||||
buildMemoryCorrectRequest,
|
||||
buildMemoryCreateRequest,
|
||||
parseMemoryReceipt,
|
||||
} from "@one-memory/operations";
|
||||
import type { CoreReader } from "./core-client.js";
|
||||
import type { CoreOperations } from "./core-operations.js";
|
||||
import { validateMemoryDetail } from "./core-responses.js";
|
||||
import { isMemoryId } from "./memory-report.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
const MEMORY_KINDS = [
|
||||
"chat",
|
||||
"working",
|
||||
"coding",
|
||||
"schema",
|
||||
"intention",
|
||||
] as const;
|
||||
const OPERATIONS = ["add", "correct", "delete"] as const;
|
||||
const MEMORY_ORIGINS = ["extracted", "answered", "agent", "manual"] as const;
|
||||
const MUTATION_RECEIPT_KEYS = new Set([
|
||||
"schema_version",
|
||||
"request_id",
|
||||
"state",
|
||||
"operation",
|
||||
"memory_id",
|
||||
"result_revision",
|
||||
"result_status",
|
||||
"memory_state",
|
||||
"migrated_from",
|
||||
]);
|
||||
const CONTENT_MAX_CHARACTERS = 100_000;
|
||||
const ID = /^[A-Za-z0-9][\w.-]{0,127}$/;
|
||||
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
|
||||
interface PlanBody {
|
||||
schema_version: 1;
|
||||
kind: "memory_mutation";
|
||||
operation: (typeof OPERATIONS)[number];
|
||||
request_id: string;
|
||||
memory_id?: string;
|
||||
expected_revision?: number;
|
||||
expected_status?: string;
|
||||
expected_origin?: string;
|
||||
memory_kind?: (typeof MEMORY_KINDS)[number];
|
||||
scopes?: string[];
|
||||
evidence_ids?: string[];
|
||||
content_sha256?: string;
|
||||
content_characters?: number;
|
||||
current?: {
|
||||
status: string;
|
||||
origin: string;
|
||||
revision: number;
|
||||
scopes: string[];
|
||||
evidence: Array<{
|
||||
evidence_id: string;
|
||||
available: boolean;
|
||||
source_id?: string;
|
||||
written_by?: string;
|
||||
}>;
|
||||
};
|
||||
}
|
||||
|
||||
export type MemoryMutationPlan = PlanBody & { digest: string };
|
||||
|
||||
export type MemoryFlowDocument =
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "preview";
|
||||
plan: MemoryMutationPlan;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "committed" | "recovered";
|
||||
operation_id: string;
|
||||
receipt: Record<string, unknown>;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "rejected" | "unknown" | "invalid";
|
||||
operation_id?: string;
|
||||
code: string;
|
||||
};
|
||||
|
||||
export function parseMemoryContent(bytes: Buffer): string | null {
|
||||
if (bytes.byteLength > CONTENT_MAX_CHARACTERS * 4 + 2) return null;
|
||||
let decoded: string;
|
||||
try {
|
||||
decoded = new TextDecoder("utf-8", { fatal: true }).decode(bytes);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
const content = decoded.replace(/\r?\n$/, "");
|
||||
return content.trim().length > 0 &&
|
||||
[...content].length <= CONTENT_MAX_CHARACTERS &&
|
||||
![...content].some((character) => {
|
||||
const code = character.charCodeAt(0);
|
||||
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
|
||||
})
|
||||
? content
|
||||
: null;
|
||||
}
|
||||
|
||||
export function planMemoryAdd(input: {
|
||||
content: string;
|
||||
memoryKind: string;
|
||||
scopes: string[];
|
||||
evidenceIds: string[];
|
||||
}): MemoryFlowDocument {
|
||||
if (
|
||||
!validContent(input.content) ||
|
||||
!oneOf(input.memoryKind, MEMORY_KINDS) ||
|
||||
!validScopes(input.scopes) ||
|
||||
!validIds(input.evidenceIds, 500)
|
||||
) {
|
||||
return invalid("memory_add_input_invalid");
|
||||
}
|
||||
return preview(
|
||||
withDigest({
|
||||
schema_version: 1,
|
||||
kind: "memory_mutation",
|
||||
operation: "add",
|
||||
request_id: randomUUID(),
|
||||
memory_kind: input.memoryKind,
|
||||
scopes: input.scopes,
|
||||
evidence_ids: input.evidenceIds,
|
||||
...contentIdentity(input.content),
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
export async function planExistingMemoryMutation(
|
||||
reader: CoreReader,
|
||||
input: {
|
||||
operation: Exclude<(typeof OPERATIONS)[number], "add">;
|
||||
memoryId: string;
|
||||
content?: string;
|
||||
},
|
||||
): Promise<MemoryFlowDocument> {
|
||||
if (
|
||||
!isMemoryId(input.memoryId) ||
|
||||
(input.operation === "correct" && input.content === undefined) ||
|
||||
(input.content !== undefined && !validContent(input.content))
|
||||
) {
|
||||
return invalid("memory_plan_input_invalid");
|
||||
}
|
||||
const outcome = await reader<unknown>(`/api/v1/memories/${input.memoryId}`);
|
||||
if (outcome.state !== "ok") {
|
||||
return invalid(`memory_${outcome.state}:${outcome.code}`);
|
||||
}
|
||||
const detail = validateMemoryDetail(outcome.data, input.memoryId);
|
||||
if (!detail) return invalid("memory_response_invalid");
|
||||
const memory = detail.memory;
|
||||
if (memory.status !== "confirmed") {
|
||||
return invalid("memory_operation_not_allowed");
|
||||
}
|
||||
const body: PlanBody = {
|
||||
schema_version: 1,
|
||||
kind: "memory_mutation",
|
||||
operation: input.operation,
|
||||
request_id: randomUUID(),
|
||||
memory_id: memory.memory_id,
|
||||
expected_revision: memory.revision,
|
||||
expected_status: memory.status,
|
||||
expected_origin: memory.origin,
|
||||
...(input.content ? contentIdentity(input.content) : {}),
|
||||
current: {
|
||||
status: memory.status,
|
||||
origin: memory.origin,
|
||||
revision: memory.revision,
|
||||
scopes: [...memory.scopes],
|
||||
evidence: detail.evidence.map((item) => ({
|
||||
evidence_id: item.evidence_id,
|
||||
available: item.available,
|
||||
...(item.source_id ? { source_id: item.source_id } : {}),
|
||||
...(item.written_by ? { written_by: item.written_by } : {}),
|
||||
})),
|
||||
},
|
||||
};
|
||||
return preview(withDigest(body));
|
||||
}
|
||||
|
||||
export function inspectMemoryPlan(
|
||||
value: unknown,
|
||||
): { operationId: string; digest: string; needsContent: boolean } | null {
|
||||
const plan = validatePlan(value);
|
||||
return plan
|
||||
? {
|
||||
operationId: plan.request_id,
|
||||
digest: plan.digest,
|
||||
needsContent: plan.operation === "add" || plan.operation === "correct",
|
||||
}
|
||||
: null;
|
||||
}
|
||||
|
||||
export function memoryPlanAcceptsContent(
|
||||
value: unknown,
|
||||
content: string | undefined,
|
||||
): boolean {
|
||||
const plan = validatePlan(value);
|
||||
if (!plan) return false;
|
||||
if (plan.operation !== "add" && plan.operation !== "correct") {
|
||||
return content === undefined;
|
||||
}
|
||||
if (content === undefined) return false;
|
||||
const identity = contentIdentity(content);
|
||||
return (
|
||||
identity.content_sha256 === plan.content_sha256 &&
|
||||
identity.content_characters === plan.content_characters
|
||||
);
|
||||
}
|
||||
|
||||
export async function executeMemoryMutation(
|
||||
value: unknown,
|
||||
content: string | undefined,
|
||||
operations: CoreOperations,
|
||||
reader?: CoreReader,
|
||||
): Promise<MemoryFlowDocument> {
|
||||
const plan = validatePlan(value);
|
||||
if (!plan) return invalid("memory_plan_invalid");
|
||||
if (!memoryPlanAcceptsContent(plan, content)) {
|
||||
return invalid("memory_content_mismatch");
|
||||
}
|
||||
const { path, body } = operationRequest(plan, content);
|
||||
const outcome = await operations.post<unknown>(path, body);
|
||||
if (outcome.state === "rejected") {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "rejected",
|
||||
operation_id: plan.request_id,
|
||||
code: outcome.code,
|
||||
};
|
||||
}
|
||||
if (outcome.state === "unknown") {
|
||||
return reader
|
||||
? readMemoryMutationReceipt(reader, plan)
|
||||
: pending(plan.request_id, outcome.code);
|
||||
}
|
||||
const memory = validateWriteResponse(outcome.data, plan);
|
||||
if (!memory || !matchesReceipt(memory, plan, content)) {
|
||||
return reader
|
||||
? readMemoryMutationReceipt(reader, plan)
|
||||
: pending(plan.request_id, "memory_receipt_invalid");
|
||||
}
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "committed",
|
||||
operation_id: plan.request_id,
|
||||
receipt: {
|
||||
memory_id: memory.memory_id,
|
||||
status: memory.status,
|
||||
origin: memory.origin,
|
||||
revision: memory.revision,
|
||||
scopes: memory.scopes,
|
||||
evidence_ids: memory.evidence_ids,
|
||||
content_sha256: contentIdentity(memory.content).content_sha256,
|
||||
content_characters: [...memory.content].length,
|
||||
created_at: memory.created_at,
|
||||
updated_at: memory.updated_at,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate the write response through the shared `parseMemoryReceipt`, which
|
||||
* owns the GovernedMemory shape and the revision/status binding Core must
|
||||
* satisfy. The CLI keeps only its own plan binding (`matchesReceipt`, which
|
||||
* also checks `origin`, `kind` and the echoed content), so the two surfaces
|
||||
* judge a write response identically and no shape rule is duplicated here.
|
||||
*
|
||||
* `add` creates the first revision and expects `confirmed`; the shared parser
|
||||
* derives that from `expectedRevision: 0`. Every other operation advances the
|
||||
* revision the caller acted against and expects `confirmed` (correct/scope)
|
||||
* or `deleted` (delete).
|
||||
*/
|
||||
function validateWriteResponse(
|
||||
value: unknown,
|
||||
plan: MemoryMutationPlan,
|
||||
): GovernedMemory | null {
|
||||
const expectedStatus = plan.operation === "delete" ? "deleted" : "confirmed";
|
||||
// For `add` there is no id or prior revision to bind yet, so the parser's
|
||||
// own receipt is the only source for both; the plan binding that follows is
|
||||
// what proves the response belongs to this request.
|
||||
const memoryId =
|
||||
plan.operation === "add"
|
||||
? String((value as { memory_id?: unknown } | null)?.memory_id ?? "")
|
||||
: (plan.memory_id as string);
|
||||
if (memoryId === "") return null;
|
||||
return parseMemoryReceipt(value, {
|
||||
memoryId,
|
||||
expectedRevision:
|
||||
plan.operation === "add" ? 0 : (plan.expected_revision as number),
|
||||
expectedStatus,
|
||||
});
|
||||
}
|
||||
|
||||
export async function readMemoryMutationReceipt(
|
||||
reader: CoreReader,
|
||||
value: unknown,
|
||||
): Promise<MemoryFlowDocument> {
|
||||
const plan = validatePlan(value);
|
||||
if (!plan) return invalid("memory_plan_invalid");
|
||||
const outcome = await reader<unknown>(
|
||||
`/api/v1/memory-mutations/${encodeURIComponent(plan.request_id)}`,
|
||||
);
|
||||
if (outcome.state !== "ok") {
|
||||
return pending(plan.request_id, `memory_mutation_receipt_${outcome.code}`);
|
||||
}
|
||||
const receipt = validateMutationReceipt(outcome.data, plan);
|
||||
if (receipt === "not_found") {
|
||||
return pending(
|
||||
plan.request_id,
|
||||
"memory_mutation_receipt_not_found_after_attempt",
|
||||
);
|
||||
}
|
||||
if (!receipt) {
|
||||
return pending(plan.request_id, "memory_mutation_receipt_invalid");
|
||||
}
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "recovered",
|
||||
operation_id: plan.request_id,
|
||||
receipt,
|
||||
};
|
||||
}
|
||||
|
||||
export function memoryFlowExitCode(document: MemoryFlowDocument): number {
|
||||
return ["preview", "committed", "recovered"].includes(document.state) ? 0 : 1;
|
||||
}
|
||||
|
||||
export function renderMemoryFlowText(document: MemoryFlowDocument): string {
|
||||
if (document.state === "preview") {
|
||||
const plan = document.plan;
|
||||
const lines = [
|
||||
`记忆操作预览:${plan.operation}`,
|
||||
`request_id:${plan.request_id}`,
|
||||
];
|
||||
if (plan.memory_id) lines.push(`memory_id:${plan.memory_id}`);
|
||||
if (plan.expected_revision) {
|
||||
lines.push(
|
||||
`当前版本:${plan.expected_revision} 状态:${plan.expected_status} 来源方式:${plan.expected_origin}`,
|
||||
);
|
||||
}
|
||||
if (plan.scopes) lines.push(`使用范围:${plan.scopes.join(",")}`);
|
||||
if (plan.current) {
|
||||
for (const evidence of plan.current.evidence) {
|
||||
lines.push(
|
||||
`证据 ${evidence.evidence_id}:source=${evidence.source_id ?? "unknown"} author=${evidence.written_by ?? "unknown"} available=${evidence.available}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
if (plan.content_characters !== undefined) {
|
||||
lines.push(
|
||||
`正文:${plan.content_characters} 字符,sha256=${plan.content_sha256}`,
|
||||
);
|
||||
}
|
||||
lines.push("尚未写入;保存 JSON 计划后用 memories apply --confirm 执行。");
|
||||
return `${lines.join("\n")}\n`;
|
||||
}
|
||||
if (document.state === "committed" || document.state === "recovered") {
|
||||
return `记忆操作${document.state === "recovered" ? "已从治理回执恢复" : "已提交"}:${document.operation_id}\n${JSON.stringify(document.receipt)}\n`;
|
||||
}
|
||||
return `记忆操作未完成:${document.state}(${"code" in document ? document.code : "memory_operation_state_invalid"})${document.operation_id ? ` request_id=${document.operation_id}` : ""}\n`;
|
||||
}
|
||||
|
||||
function operationRequest(
|
||||
plan: MemoryMutationPlan,
|
||||
content: string | undefined,
|
||||
): { path: string; body: Record<string, unknown> } {
|
||||
if (plan.operation === "add") {
|
||||
// The shared create builder is the Web console's manual-add body, which
|
||||
// always creates `chat` (the Web has no kind picker). The CLI exposes
|
||||
// `--kind`, so it builds through the same builder and then sets the kind,
|
||||
// keeping every other field (user_id/scope/evidence/origin) shared.
|
||||
const request = buildMemoryCreateRequest(
|
||||
{
|
||||
content: content as string,
|
||||
scopes: plan.scopes ?? [],
|
||||
evidence_ids: plan.evidence_ids ?? [],
|
||||
origin: "manual",
|
||||
},
|
||||
plan.request_id,
|
||||
);
|
||||
const draft = request.body.draft as Record<string, unknown>;
|
||||
draft.kind = plan.memory_kind;
|
||||
return { path: request.path, body: request.body };
|
||||
}
|
||||
if (plan.operation === "correct") {
|
||||
return buildMemoryCorrectRequest(
|
||||
plan.memory_id as string,
|
||||
plan.expected_revision as number,
|
||||
content as string,
|
||||
undefined,
|
||||
plan.request_id,
|
||||
);
|
||||
}
|
||||
return buildMemoryActionRequest(
|
||||
plan.memory_id as string,
|
||||
plan.operation,
|
||||
plan.expected_revision as number,
|
||||
plan.request_id,
|
||||
);
|
||||
}
|
||||
|
||||
function matchesReceipt(
|
||||
memory: GovernedMemory,
|
||||
plan: MemoryMutationPlan,
|
||||
content: string | undefined,
|
||||
): boolean {
|
||||
if (plan.operation === "add") {
|
||||
return (
|
||||
memory.status === "confirmed" &&
|
||||
memory.origin === "manual" &&
|
||||
memory.revision === 1 &&
|
||||
memory.kind === plan.memory_kind &&
|
||||
same(memory.scopes, plan.scopes ?? []) &&
|
||||
same(memory.evidence_ids, plan.evidence_ids ?? []) &&
|
||||
memory.content === content
|
||||
);
|
||||
}
|
||||
if (
|
||||
memory.memory_id !== plan.memory_id ||
|
||||
memory.revision !== (plan.expected_revision as number) + 1 ||
|
||||
memory.origin !== plan.expected_origin
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (plan.operation === "delete" && memory.status !== "deleted") return false;
|
||||
if (plan.operation === "correct" && memory.status !== plan.expected_status) {
|
||||
return false;
|
||||
}
|
||||
if (plan.operation === "correct" && memory.content !== content) return false;
|
||||
const expectedScopes =
|
||||
plan.scopes ?? (plan.current?.scopes as string[] | undefined);
|
||||
return expectedScopes ? same(memory.scopes, expectedScopes) : true;
|
||||
}
|
||||
|
||||
function validateMutationReceipt(
|
||||
value: unknown,
|
||||
plan: MemoryMutationPlan,
|
||||
): Record<string, unknown> | "not_found" | null {
|
||||
const receipt = record(value);
|
||||
if (
|
||||
!receipt ||
|
||||
Object.keys(receipt).length !== MUTATION_RECEIPT_KEYS.size ||
|
||||
Object.keys(receipt).some((key) => !MUTATION_RECEIPT_KEYS.has(key)) ||
|
||||
receipt.schema_version !== 2 ||
|
||||
receipt.request_id !== plan.request_id
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
if (receipt.state === "not_found") {
|
||||
return [
|
||||
receipt.operation,
|
||||
receipt.memory_id,
|
||||
receipt.result_revision,
|
||||
receipt.result_status,
|
||||
receipt.memory_state,
|
||||
receipt.migrated_from,
|
||||
].every((item) => item === null)
|
||||
? "not_found"
|
||||
: null;
|
||||
}
|
||||
const expectedOperation =
|
||||
plan.operation === "add" ? "create" : plan.operation;
|
||||
const expectedRevision =
|
||||
plan.operation === "add" ? 1 : (plan.expected_revision as number) + 1;
|
||||
const expectedStatus = plan.operation === "delete" ? "deleted" : "confirmed";
|
||||
// A migrated receipt records an operation from before v2; no plan made
|
||||
// today can have produced it.
|
||||
if (
|
||||
receipt.state !== "committed" ||
|
||||
receipt.migrated_from !== null ||
|
||||
receipt.operation !== expectedOperation ||
|
||||
typeof receipt.memory_id !== "string" ||
|
||||
!isMemoryId(receipt.memory_id) ||
|
||||
(plan.memory_id !== undefined && receipt.memory_id !== plan.memory_id) ||
|
||||
receipt.result_revision !== expectedRevision ||
|
||||
receipt.result_status !== expectedStatus ||
|
||||
!["present", "deleted"].includes(String(receipt.memory_state))
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
schema_version: 2,
|
||||
request_id: plan.request_id,
|
||||
state: "committed",
|
||||
operation: expectedOperation,
|
||||
memory_id: receipt.memory_id,
|
||||
result_revision: expectedRevision,
|
||||
result_status: expectedStatus,
|
||||
memory_state: receipt.memory_state,
|
||||
proof_scope: "governance_transaction_only",
|
||||
downstream_sync_verified: false,
|
||||
payload_fingerprint_verified: false,
|
||||
};
|
||||
}
|
||||
|
||||
function validatePlan(value: unknown): MemoryMutationPlan | null {
|
||||
const plan = record(value);
|
||||
if (
|
||||
plan?.schema_version !== 1 ||
|
||||
plan.kind !== "memory_mutation" ||
|
||||
!oneOf(plan.operation, OPERATIONS) ||
|
||||
typeof plan.request_id !== "string" ||
|
||||
!UUID.test(plan.request_id) ||
|
||||
typeof plan.digest !== "string"
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const body = { ...plan };
|
||||
delete body.digest;
|
||||
const expected = withDigest(body as unknown as PlanBody);
|
||||
if (!safeEqual(expected.digest, plan.digest)) return null;
|
||||
if (plan.operation === "add") {
|
||||
if (
|
||||
!oneOf(plan.memory_kind, MEMORY_KINDS) ||
|
||||
!validScopes(plan.scopes) ||
|
||||
!validIds(plan.evidence_ids, 500) ||
|
||||
!contentFields(plan)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
} else if (
|
||||
typeof plan.memory_id !== "string" ||
|
||||
!isMemoryId(plan.memory_id) ||
|
||||
!Number.isSafeInteger(plan.expected_revision) ||
|
||||
(plan.expected_revision as number) < 1 ||
|
||||
plan.expected_status !== "confirmed" ||
|
||||
!oneOf(plan.expected_origin, MEMORY_ORIGINS) ||
|
||||
!validCurrent(plan) ||
|
||||
(plan.operation === "correct" && !contentFields(plan))
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return plan as unknown as MemoryMutationPlan;
|
||||
}
|
||||
|
||||
function contentFields(plan: Record<string, unknown>): boolean {
|
||||
return (
|
||||
typeof plan.content_sha256 === "string" &&
|
||||
/^[0-9a-f]{64}$/.test(plan.content_sha256) &&
|
||||
Number.isSafeInteger(plan.content_characters) &&
|
||||
(plan.content_characters as number) >= 1 &&
|
||||
(plan.content_characters as number) <= CONTENT_MAX_CHARACTERS
|
||||
);
|
||||
}
|
||||
|
||||
function validCurrent(plan: Record<string, unknown>): boolean {
|
||||
const current = record(plan.current);
|
||||
if (
|
||||
!current ||
|
||||
current.status !== plan.expected_status ||
|
||||
current.origin !== plan.expected_origin ||
|
||||
current.revision !== plan.expected_revision ||
|
||||
!validScopes(current.scopes) ||
|
||||
!Array.isArray(current.evidence) ||
|
||||
current.evidence.length < 1 ||
|
||||
current.evidence.length > 500
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
return current.evidence.every((value) => {
|
||||
const evidence = record(value);
|
||||
return (
|
||||
evidence !== null &&
|
||||
typeof evidence.evidence_id === "string" &&
|
||||
ID.test(evidence.evidence_id) &&
|
||||
typeof evidence.available === "boolean" &&
|
||||
(evidence.source_id === undefined ||
|
||||
(typeof evidence.source_id === "string" &&
|
||||
evidence.source_id.length <= 128)) &&
|
||||
(evidence.written_by === undefined ||
|
||||
(typeof evidence.written_by === "string" &&
|
||||
evidence.written_by.length <= 256))
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
function contentIdentity(content: string): {
|
||||
content_sha256: string;
|
||||
content_characters: number;
|
||||
} {
|
||||
return {
|
||||
content_sha256: createHash("sha256").update(content, "utf8").digest("hex"),
|
||||
content_characters: [...content].length,
|
||||
};
|
||||
}
|
||||
|
||||
function validContent(content: string): boolean {
|
||||
return (
|
||||
content.trim().length > 0 &&
|
||||
[...content].length <= CONTENT_MAX_CHARACTERS &&
|
||||
![...content].some((character) => {
|
||||
const code = character.charCodeAt(0);
|
||||
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
function validScopes(value: unknown): value is string[] {
|
||||
if (
|
||||
!Array.isArray(value) ||
|
||||
value.length < 1 ||
|
||||
value.length > 128 ||
|
||||
new Set(value).size !== value.length ||
|
||||
value.some(
|
||||
(scope) =>
|
||||
typeof scope !== "string" ||
|
||||
scope.trim() !== scope ||
|
||||
scope.length < 1 ||
|
||||
scope.length > 512 ||
|
||||
scope === "*",
|
||||
)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
return !value.includes("private") || value.length === 1;
|
||||
}
|
||||
|
||||
function validIds(value: unknown, maximum: number): value is string[] {
|
||||
return (
|
||||
Array.isArray(value) &&
|
||||
value.length >= 1 &&
|
||||
value.length <= maximum &&
|
||||
new Set(value).size === value.length &&
|
||||
value.every((item) => typeof item === "string" && ID.test(item))
|
||||
);
|
||||
}
|
||||
|
||||
function withDigest(body: PlanBody): MemoryMutationPlan {
|
||||
return {
|
||||
...body,
|
||||
digest: createHash("sha256")
|
||||
.update(JSON.stringify(body), "utf8")
|
||||
.digest("hex"),
|
||||
};
|
||||
}
|
||||
|
||||
function preview(plan: MemoryMutationPlan): MemoryFlowDocument {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "preview",
|
||||
plan,
|
||||
};
|
||||
}
|
||||
|
||||
function invalid(code: string): MemoryFlowDocument {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "invalid",
|
||||
code,
|
||||
};
|
||||
}
|
||||
|
||||
function pending(operationId: string, code: string): MemoryFlowDocument {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "unknown",
|
||||
operation_id: operationId,
|
||||
code,
|
||||
};
|
||||
}
|
||||
|
||||
function same(left: readonly string[], right: readonly string[]): boolean {
|
||||
return (
|
||||
left.length === right.length &&
|
||||
left.every((value, index) => value === right[index])
|
||||
);
|
||||
}
|
||||
|
||||
function safeEqual(left: string, right: string): boolean {
|
||||
if (left.length !== right.length) return false;
|
||||
return timingSafeEqual(Buffer.from(left), Buffer.from(right));
|
||||
}
|
||||
|
||||
function oneOf<const T extends readonly string[]>(
|
||||
value: unknown,
|
||||
values: T,
|
||||
): value is T[number] {
|
||||
return typeof value === "string" && values.includes(value as T[number]);
|
||||
}
|
||||
|
||||
function record(value: unknown): Record<string, unknown> | null {
|
||||
return value !== null && typeof value === "object" && !Array.isArray(value)
|
||||
? (value as Record<string, unknown>)
|
||||
: null;
|
||||
}
|
||||
@@ -1,211 +0,0 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { CoreReader } from "./core-client.js";
|
||||
import {
|
||||
planExistingMemoryMutation,
|
||||
planMemoryAdd,
|
||||
readMemoryMutationReceipt,
|
||||
} from "./memory-flows.js";
|
||||
|
||||
// Written by a real Core and checked against it on the Core side, so these are
|
||||
// the receipts the CLI meets in use rather than ones shaped to fit the parser.
|
||||
const FIXTURE = JSON.parse(
|
||||
readFileSync(
|
||||
new URL(
|
||||
"../../../packages/contracts/fixtures/memory-mutation-receipts.v2.json",
|
||||
import.meta.url,
|
||||
),
|
||||
"utf8",
|
||||
),
|
||||
) as { receipts: Record<"add" | "correct" | "delete", Receipt> };
|
||||
|
||||
interface Receipt {
|
||||
schema_version: number;
|
||||
request_id: string;
|
||||
memory_id: string;
|
||||
result_revision: number;
|
||||
result_status: string;
|
||||
operation: string;
|
||||
}
|
||||
|
||||
function reader(data: unknown): CoreReader {
|
||||
return async <T>() => ({ state: "ok", data: data as T });
|
||||
}
|
||||
|
||||
function detail(sample: Receipt) {
|
||||
const revision = sample.result_revision - 1;
|
||||
return {
|
||||
memory: {
|
||||
memory_id: sample.memory_id,
|
||||
user_id: "local",
|
||||
kind: "chat",
|
||||
status: "confirmed",
|
||||
content: "Synthetic memory.",
|
||||
scopes: ["all-apps"],
|
||||
evidence_ids: ["evidence-1"],
|
||||
origin: "manual",
|
||||
tags: [],
|
||||
revision,
|
||||
created_at: "2026-09-19T00:00:00Z",
|
||||
updated_at: "2026-09-19T00:00:01Z",
|
||||
deleted_at: null,
|
||||
},
|
||||
evidence: [
|
||||
{
|
||||
evidence_id: "evidence-1",
|
||||
available: true,
|
||||
source_id: "fixture",
|
||||
written_by: "person",
|
||||
created_at: "2026-09-19T00:00:00Z",
|
||||
verbatim: "Synthetic source text.",
|
||||
},
|
||||
],
|
||||
history: [
|
||||
{
|
||||
revision,
|
||||
action: "create",
|
||||
status: "confirmed",
|
||||
content: "Synthetic memory.",
|
||||
scopes: ["all-apps"],
|
||||
created_at: "2026-09-19T00:00:01Z",
|
||||
},
|
||||
],
|
||||
index_sync: null,
|
||||
};
|
||||
}
|
||||
|
||||
describe("memory mutation receipt v2 from Core", () => {
|
||||
it("recovers an add from the receipt Core returns for a create", async () => {
|
||||
const sample = FIXTURE.receipts.add;
|
||||
expect(sample.schema_version).toBe(2);
|
||||
const preview = planMemoryAdd({
|
||||
content: "Synthetic memory.",
|
||||
memoryKind: "chat",
|
||||
scopes: ["all-apps"],
|
||||
evidenceIds: ["evidence-1"],
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
// The request id is the one binding a receipt to its plan; the plan picks it.
|
||||
const result = await readMemoryMutationReceipt(
|
||||
reader({ ...sample, request_id: preview.plan.request_id }),
|
||||
preview.plan,
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
state: "recovered",
|
||||
receipt: {
|
||||
schema_version: 2,
|
||||
operation: "create",
|
||||
memory_id: sample.memory_id,
|
||||
result_revision: 1,
|
||||
result_status: "confirmed",
|
||||
memory_state: "present",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it.each<["correct" | "delete", { content?: string }]>([
|
||||
["correct", { content: "Corrected synthetic memory." }],
|
||||
["delete", {}],
|
||||
])(
|
||||
"recovers %s from the receipt Core returns for it",
|
||||
async (operation, extra) => {
|
||||
const sample = FIXTURE.receipts[operation];
|
||||
const preview = await planExistingMemoryMutation(reader(detail(sample)), {
|
||||
operation,
|
||||
memoryId: sample.memory_id,
|
||||
...extra,
|
||||
});
|
||||
if (preview.state !== "preview") {
|
||||
throw new Error(`preview expected: ${JSON.stringify(preview)}`);
|
||||
}
|
||||
const result = await readMemoryMutationReceipt(
|
||||
reader({ ...sample, request_id: preview.plan.request_id }),
|
||||
preview.plan,
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
state: "recovered",
|
||||
receipt: {
|
||||
schema_version: 2,
|
||||
operation: sample.operation,
|
||||
memory_id: sample.memory_id,
|
||||
result_revision: sample.result_revision,
|
||||
result_status: sample.result_status,
|
||||
},
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("does not take a v1 receipt for a v2 one", async () => {
|
||||
const sample = FIXTURE.receipts.delete;
|
||||
const preview = await planExistingMemoryMutation(reader(detail(sample)), {
|
||||
operation: "delete",
|
||||
memoryId: sample.memory_id,
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
const { migrated_from: _dropped, ...v1 } = sample as Receipt & {
|
||||
migrated_from: null;
|
||||
};
|
||||
// As v1 was, and with v2's fields but v1's number: either alone is refused.
|
||||
for (const receipt of [
|
||||
{ ...v1, schema_version: 1 },
|
||||
{ ...sample, schema_version: 1 },
|
||||
]) {
|
||||
const result = await readMemoryMutationReceipt(
|
||||
reader({ ...receipt, request_id: preview.plan.request_id }),
|
||||
preview.plan,
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
state: "unknown",
|
||||
code: "memory_mutation_receipt_invalid",
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it("does not take a not-found receipt that names a migrated operation", async () => {
|
||||
const sample = FIXTURE.receipts.delete;
|
||||
const preview = await planExistingMemoryMutation(reader(detail(sample)), {
|
||||
operation: "delete",
|
||||
memoryId: sample.memory_id,
|
||||
});
|
||||
if (preview.state !== "preview") throw new Error("preview expected");
|
||||
const notFound = {
|
||||
schema_version: 2,
|
||||
request_id: preview.plan.request_id,
|
||||
state: "not_found",
|
||||
operation: null,
|
||||
memory_id: null,
|
||||
result_revision: null,
|
||||
result_status: null,
|
||||
memory_state: null,
|
||||
};
|
||||
expect(
|
||||
await readMemoryMutationReceipt(
|
||||
reader({ ...notFound, migrated_from: null }),
|
||||
preview.plan,
|
||||
),
|
||||
).toMatchObject({
|
||||
code: "memory_mutation_receipt_not_found_after_attempt",
|
||||
});
|
||||
expect(
|
||||
await readMemoryMutationReceipt(
|
||||
reader({ ...notFound, migrated_from: "reject" }),
|
||||
preview.plan,
|
||||
),
|
||||
).toMatchObject({ code: "memory_mutation_receipt_invalid" });
|
||||
});
|
||||
|
||||
it("plans nothing on a memory that is not in effect", async () => {
|
||||
const sample = FIXTURE.receipts.delete;
|
||||
const superseded = detail(sample);
|
||||
superseded.memory.status = "superseded";
|
||||
superseded.history[0].status = "superseded";
|
||||
const result = await planExistingMemoryMutation(reader(superseded), {
|
||||
operation: "delete",
|
||||
memoryId: sample.memory_id,
|
||||
});
|
||||
expect(result).toMatchObject({
|
||||
state: "invalid",
|
||||
code: "memory_operation_not_allowed",
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -1,5 +1,17 @@
|
||||
import type { SdkMemoryPage, SdkMemoryRecord } from "@one-memory/contracts";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import type {
|
||||
MemoryAddReceipt,
|
||||
SdkMemoryPage,
|
||||
SdkMemoryRecord,
|
||||
} from "@one-memory/contracts";
|
||||
import {
|
||||
buildMemoryAddRequest,
|
||||
MemoryAddInputError,
|
||||
memoryAddStatusPath,
|
||||
parseMemoryAddReceipt,
|
||||
} from "@one-memory/operations";
|
||||
import { type CoreReader, RESERVED_MEMORY_SEGMENTS } from "./core-client.js";
|
||||
import type { CoreOperations } from "./core-operations.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
/** Output budget: a long memory is shortened visibly, never dropped silently. */
|
||||
@@ -15,27 +27,75 @@ export function isMemoryId(value: string): boolean {
|
||||
);
|
||||
}
|
||||
|
||||
const CONTENT_MAX_CHARACTERS = 100_000;
|
||||
|
||||
/** A memory body read from `--content-file`: UTF-8, non-blank, bounded, and
|
||||
* free of control characters other than tab and newline. */
|
||||
export function parseMemoryContent(bytes: Buffer): string | null {
|
||||
if (bytes.byteLength > CONTENT_MAX_CHARACTERS * 4 + 2) return null;
|
||||
let decoded: string;
|
||||
try {
|
||||
decoded = new TextDecoder("utf-8", { fatal: true }).decode(bytes);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
const content = decoded.replace(/\r?\n$/, "");
|
||||
return content.trim().length > 0 &&
|
||||
[...content].length <= CONTENT_MAX_CHARACTERS &&
|
||||
![...content].some((character) => {
|
||||
const code = character.charCodeAt(0);
|
||||
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
|
||||
})
|
||||
? content
|
||||
: null;
|
||||
}
|
||||
|
||||
export const MEMORY_SCENES = ["all", "chat", "working", "coding"] as const;
|
||||
export type MemoryScene = (typeof MEMORY_SCENES)[number];
|
||||
export const MEMORY_KINDS = ["chat", "experience", "fact"] as const;
|
||||
export type MemoryKind = (typeof MEMORY_KINDS)[number];
|
||||
export const MEMORY_VARIANTS = ["working", "coding"] as const;
|
||||
export type MemoryVariant = (typeof MEMORY_VARIANTS)[number];
|
||||
|
||||
/** Which SDK namespace a memory lives in. A chat memory needs neither; a
|
||||
* working/coding experience or fact needs both so Core does not have to guess
|
||||
* (an id present in two namespaces is refused as ambiguous). */
|
||||
export interface MemoryNamespace {
|
||||
kind?: MemoryKind;
|
||||
variant?: MemoryVariant;
|
||||
}
|
||||
|
||||
export function namespaceParameters(
|
||||
namespace: MemoryNamespace,
|
||||
): Record<string, string> {
|
||||
return {
|
||||
...(namespace.kind === undefined ? {} : { kind: namespace.kind }),
|
||||
...(namespace.variant === undefined ? {} : { variant: namespace.variant }),
|
||||
};
|
||||
}
|
||||
|
||||
export interface MemoryQuery {
|
||||
query?: string;
|
||||
status?: string;
|
||||
source?: string;
|
||||
scene?: MemoryScene;
|
||||
offset: number;
|
||||
limit: number;
|
||||
}
|
||||
|
||||
/** Every `memories` document is schema 2: list/show share it with
|
||||
* add/update/delete, and the v1-era `evidence`/`history` fields are gone. */
|
||||
export interface MemoryPageDocument {
|
||||
schema_version: 1;
|
||||
schema_version: 2;
|
||||
product: string;
|
||||
page: Record<string, unknown>;
|
||||
memories: Record<string, unknown>[];
|
||||
}
|
||||
|
||||
export interface MemoryDetailDocument {
|
||||
schema_version: 1;
|
||||
schema_version: 2;
|
||||
product: string;
|
||||
memory: Record<string, unknown>;
|
||||
evidence: Record<string, unknown>[];
|
||||
history: Record<string, unknown>[];
|
||||
}
|
||||
|
||||
export async function readMemoryPage(
|
||||
@@ -47,10 +107,15 @@ export async function readMemoryPage(
|
||||
limit: query.limit,
|
||||
};
|
||||
if (query.status !== undefined) parameters.statuses = query.status;
|
||||
if (query.source !== undefined) parameters.source = query.source;
|
||||
if (query.query !== undefined) parameters.query = query.query;
|
||||
if (query.scene !== undefined && query.scene !== "all") {
|
||||
parameters.scene = query.scene;
|
||||
}
|
||||
const outcome = await reader<unknown>("/api/v2/memory-library", parameters);
|
||||
if (outcome.state !== "ok") {
|
||||
return {
|
||||
schema_version: 1,
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
page: { state: outcome.state, code: outcome.code },
|
||||
memories: [],
|
||||
@@ -59,7 +124,7 @@ export async function readMemoryPage(
|
||||
const page = sdkMemoryPage(outcome.data);
|
||||
if (page === null) {
|
||||
return {
|
||||
schema_version: 1,
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
page: { state: "invalid", code: "response_invalid" },
|
||||
memories: [],
|
||||
@@ -67,15 +132,21 @@ export async function readMemoryPage(
|
||||
}
|
||||
const items = page.memories;
|
||||
return {
|
||||
schema_version: 1,
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
page: {
|
||||
state: "ok",
|
||||
scene: query.scene ?? "all",
|
||||
...(query.query === undefined ? {} : { query: query.query }),
|
||||
total: page.total,
|
||||
offset: page.offset,
|
||||
limit: page.limit,
|
||||
returned: items.length,
|
||||
has_more: page.offset + items.length < page.total,
|
||||
// A keyword search only looks among this many most relevant memories.
|
||||
...(typeof page.search_window === "number"
|
||||
? { search_window: page.search_window }
|
||||
: {}),
|
||||
},
|
||||
memories: items.map((item) => describeMemory(item)),
|
||||
};
|
||||
@@ -84,42 +155,195 @@ export async function readMemoryPage(
|
||||
export async function readMemoryDetail(
|
||||
reader: CoreReader,
|
||||
memoryId: string,
|
||||
namespace: MemoryNamespace = {},
|
||||
): Promise<MemoryDetailDocument> {
|
||||
const outcome = await reader<unknown>(`/api/v2/memories/${memoryId}`);
|
||||
const parameters = namespaceParameters(namespace);
|
||||
const outcome = await reader<unknown>(
|
||||
`/api/v2/memories/${memoryId}`,
|
||||
Object.keys(parameters).length > 0 ? parameters : undefined,
|
||||
);
|
||||
if (outcome.state !== "ok") {
|
||||
return {
|
||||
schema_version: 1,
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
memory: { state: outcome.state, code: outcome.code },
|
||||
evidence: [],
|
||||
history: [],
|
||||
};
|
||||
}
|
||||
const memory = sdkMemory(outcome.data);
|
||||
if (memory === null || memory.memory_id !== memoryId) {
|
||||
return {
|
||||
schema_version: 1,
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
memory: { state: "invalid", code: "response_invalid" },
|
||||
evidence: [],
|
||||
history: [],
|
||||
};
|
||||
}
|
||||
return {
|
||||
schema_version: 1,
|
||||
schema_version: 2,
|
||||
product: PRODUCT,
|
||||
memory: { state: "ok", ...describeMemory(memory) },
|
||||
evidence: [],
|
||||
history: [],
|
||||
};
|
||||
}
|
||||
|
||||
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
/** The same bound the app's 添加记忆 box applies. */
|
||||
export const MEMORY_ADD_MAX_CHARACTERS = 12_000;
|
||||
|
||||
export interface MemoryAddDocument {
|
||||
schema_version: 2;
|
||||
product: string;
|
||||
command: "memories add" | "memories add-status";
|
||||
state: "accepted" | "rejected" | "unknown" | "not_found" | "invalid";
|
||||
request_id: string;
|
||||
code?: string;
|
||||
receipt?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
export function isRequestId(value: string): boolean {
|
||||
return UUID.test(value);
|
||||
}
|
||||
|
||||
/** A memory text: non-blank, bounded, no control characters except tab and
|
||||
* newline. */
|
||||
export function validMemoryText(text: string): boolean {
|
||||
return (
|
||||
text.trim().length > 0 &&
|
||||
[...text].length <= MEMORY_ADD_MAX_CHARACTERS &&
|
||||
![...text].some((character) => {
|
||||
const code = character.charCodeAt(0);
|
||||
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
function addReceipt(receipt: MemoryAddReceipt): Record<string, unknown> {
|
||||
return {
|
||||
evidence_id: receipt.evidence_id,
|
||||
state: receipt.state,
|
||||
created_at: receipt.created_at,
|
||||
task_id: receipt.task_id,
|
||||
memory_ids: receipt.memory_ids,
|
||||
error: receipt.error,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Add one memory directly, the same request as the app's 添加记忆.
|
||||
*
|
||||
* `requestId` is Core's idempotency key: retrying with the same id and text
|
||||
* returns the first receipt; the same id with other text is refused.
|
||||
*/
|
||||
export async function addMemory(
|
||||
operations: CoreOperations,
|
||||
text: string,
|
||||
requestId: string = randomUUID(),
|
||||
): Promise<MemoryAddDocument> {
|
||||
const base = {
|
||||
schema_version: 2 as const,
|
||||
product: PRODUCT,
|
||||
command: "memories add" as const,
|
||||
request_id: requestId,
|
||||
};
|
||||
if (!validMemoryText(text) || !UUID.test(requestId)) {
|
||||
return { ...base, state: "invalid", code: "memory_add_invalid" };
|
||||
}
|
||||
let request: ReturnType<typeof buildMemoryAddRequest>;
|
||||
try {
|
||||
request = buildMemoryAddRequest({
|
||||
request_id: requestId,
|
||||
text,
|
||||
scope: "all-apps",
|
||||
});
|
||||
} catch (error) {
|
||||
if (error instanceof MemoryAddInputError) {
|
||||
return { ...base, state: "invalid", code: "memory_add_invalid" };
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
const result = await operations.post<unknown>(request.path, request.body);
|
||||
if (result.state !== "committed") {
|
||||
return { ...base, state: result.state, code: result.code };
|
||||
}
|
||||
const receipt = parseMemoryAddReceipt(result.data);
|
||||
return receipt === null
|
||||
? { ...base, state: "unknown", code: "memory_add_receipt_invalid" }
|
||||
: { ...base, state: "accepted", receipt: addReceipt(receipt) };
|
||||
}
|
||||
|
||||
/** Read how one addition is doing, by the request id it was submitted with. */
|
||||
export async function readMemoryAddStatus(
|
||||
reader: CoreReader,
|
||||
requestId: string,
|
||||
): Promise<MemoryAddDocument> {
|
||||
const base = {
|
||||
schema_version: 2 as const,
|
||||
product: PRODUCT,
|
||||
command: "memories add-status" as const,
|
||||
request_id: requestId,
|
||||
};
|
||||
if (!UUID.test(requestId)) {
|
||||
return { ...base, state: "invalid", code: "invalid_request_id" };
|
||||
}
|
||||
const outcome = await reader<unknown>(memoryAddStatusPath(requestId));
|
||||
if (outcome.state === "not_found") {
|
||||
return { ...base, state: "not_found", code: "memory_add_not_found" };
|
||||
}
|
||||
if (outcome.state !== "ok") {
|
||||
return {
|
||||
...base,
|
||||
state: outcome.state === "rejected" ? "rejected" : "unknown",
|
||||
code: outcome.code,
|
||||
};
|
||||
}
|
||||
const receipt = parseMemoryAddReceipt(outcome.data);
|
||||
return receipt === null
|
||||
? { ...base, state: "unknown", code: "memory_add_receipt_invalid" }
|
||||
: { ...base, state: "accepted", receipt: addReceipt(receipt) };
|
||||
}
|
||||
|
||||
export function memoryAddExitCode(document: MemoryAddDocument): number {
|
||||
return document.state === "accepted" ? 0 : 1;
|
||||
}
|
||||
|
||||
export function renderMemoryAddText(document: MemoryAddDocument): string {
|
||||
if (document.state === "accepted" && document.receipt) {
|
||||
return [
|
||||
`request_id: ${document.request_id}`,
|
||||
`evidence_id: ${document.receipt.evidence_id}`,
|
||||
`state: ${document.receipt.state}`,
|
||||
...((document.receipt.memory_ids as string[]).length > 0
|
||||
? [`memory_ids: ${(document.receipt.memory_ids as string[]).join(",")}`]
|
||||
: []),
|
||||
document.command === "memories add"
|
||||
? `查询进度:memhub memories add-status ${document.request_id};重试请带上同一个 --request-id,不会重复添加。`
|
||||
: "",
|
||||
"",
|
||||
]
|
||||
.filter((line, index, all) => line !== "" || index === all.length - 1)
|
||||
.join("\n");
|
||||
}
|
||||
if (document.state === "not_found") {
|
||||
return `未找到该 request id:${document.request_id}\n`;
|
||||
}
|
||||
return [
|
||||
`添加记忆未完成:${document.state}(${document.code})`,
|
||||
document.state === "unknown"
|
||||
? `结果未知;可以用同一个 --request-id ${document.request_id} 安全重试。`
|
||||
: "",
|
||||
"",
|
||||
]
|
||||
.filter((line, index, all) => line !== "" || index === all.length - 1)
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
export function renderMemoryPageText(document: MemoryPageDocument): string {
|
||||
const page = document.page;
|
||||
if (page.state !== "ok") {
|
||||
if (page.state === "unavailable") {
|
||||
return "后台正忙,记忆列表这次没查完,不代表没有记忆,请稍后再试\n";
|
||||
}
|
||||
if (page.state === "rejected") {
|
||||
return `记忆查询被拒绝(${page.code}),请检查参数。\n`;
|
||||
}
|
||||
return `记忆查询未完成:${page.state}(${page.code})\n`;
|
||||
}
|
||||
const lines = [
|
||||
@@ -127,9 +351,16 @@ export function renderMemoryPageText(document: MemoryPageDocument): string {
|
||||
page.has_more ? ",还有更多,请用 --offset 继续" : ""
|
||||
}`,
|
||||
];
|
||||
if (typeof page.search_window === "number") {
|
||||
lines.push(`(搜索只在最相关的 ${page.search_window} 条里匹配关键词)`);
|
||||
}
|
||||
for (const memory of document.memories) {
|
||||
const namespace =
|
||||
memory.kind === "chat"
|
||||
? ""
|
||||
: ` kind=${memory.kind} variant=${memory.variant}`;
|
||||
lines.push(
|
||||
`- ${memory.memory_id} ${memory.status}/${memory.layer} source=${memory.source ?? "未知"}`,
|
||||
`- ${memory.memory_id} [${memory.scene}] ${memory.status}/${memory.layer} source=${memory.source ?? "未知"}${namespace}`,
|
||||
` ${memory.content}${memory.content_truncated ? ` …(共 ${memory.content_characters} 字)` : ""}`,
|
||||
);
|
||||
}
|
||||
@@ -144,10 +375,14 @@ export function renderMemoryDetailText(document: MemoryDetailDocument): string {
|
||||
if (memory.state === "unavailable") {
|
||||
return "后台正忙,记忆详情这次没查完,不代表没有这条记忆,请稍后再试\n";
|
||||
}
|
||||
if (memory.code === "memory_id_ambiguous") {
|
||||
return "这个 ID 同时存在于多个场景;请加 --kind(以及 --variant)指明是哪一条。\n";
|
||||
}
|
||||
return `记忆读取未完成:${memory.state}(${memory.code})\n`;
|
||||
}
|
||||
const lines = [
|
||||
`记忆 ${memory.memory_id}`,
|
||||
`场景:${memory.scene} 类型:${memory.kind}${memory.variant ? `(${memory.variant})` : ""}`,
|
||||
`状态:${memory.status} 层:${memory.layer} 来源:${memory.source ?? "未知"}`,
|
||||
`owner:${memory.owner ?? "未知"} latest:${memory.is_latest}`,
|
||||
`raw IDs:${(memory.source_raw_ids as string[]).join(",") || "无"}`,
|
||||
@@ -165,6 +400,10 @@ function describeMemory(memory: SdkMemoryRecord): Record<string, unknown> {
|
||||
const budgeted = budget(memory.content ?? "");
|
||||
return {
|
||||
memory_id: memory.memory_id,
|
||||
// Pre-2.0.6 Core listed chat memories only and named no scene.
|
||||
scene: memory.scene ?? "chat",
|
||||
kind: memory.kind ?? "chat",
|
||||
variant: memory.variant ?? null,
|
||||
status: memory.status,
|
||||
layer: memory.layer,
|
||||
is_latest: memory.is_latest,
|
||||
@@ -176,6 +415,9 @@ function describeMemory(memory: SdkMemoryRecord): Record<string, unknown> {
|
||||
tags: memory.tags ?? [],
|
||||
memory_at: memory.memory_at,
|
||||
gmt_created: memory.gmt_created,
|
||||
...(memory.task === undefined || memory.task === null
|
||||
? {}
|
||||
: { task: memory.task }),
|
||||
content: budgeted.text,
|
||||
content_truncated: budgeted.truncated,
|
||||
content_characters: budgeted.characters,
|
||||
@@ -193,6 +435,9 @@ function sdkMemoryPage(value: unknown): SdkMemoryPage | null {
|
||||
total: value.total,
|
||||
limit: value.limit,
|
||||
offset: value.offset,
|
||||
...(integer(value.search_window)
|
||||
? { search_window: value.search_window }
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -208,8 +453,9 @@ function sdkMemory(value: unknown): SdkMemoryRecord | null {
|
||||
!strings(value.superseded_by) ||
|
||||
!strings(value.tags) ||
|
||||
!strings(value.source_raw_ids) ||
|
||||
!text(value.user_id) ||
|
||||
!text(value.agent_id)
|
||||
// Working/coding memories may carry an empty user or agent id.
|
||||
typeof value.user_id !== "string" ||
|
||||
typeof value.agent_id !== "string"
|
||||
)
|
||||
return null;
|
||||
return value as unknown as SdkMemoryRecord;
|
||||
|
||||
@@ -3,9 +3,9 @@ import { describe, expect, it, vi } from "vitest";
|
||||
import type { CoreReader } from "./core-client.js";
|
||||
import type { CoreOperations } from "./core-operations.js";
|
||||
import {
|
||||
executeModelProbe,
|
||||
planModelProbe,
|
||||
modelProbeExitCode,
|
||||
readModelProbeReceipt,
|
||||
startModelProbe,
|
||||
} from "./model-probe-flow.js";
|
||||
|
||||
const PROFILE_ID = "22222222-2222-4222-8222-222222222222";
|
||||
@@ -83,67 +83,101 @@ function completed(operationId: string) {
|
||||
};
|
||||
}
|
||||
|
||||
describe("model probe operation flow", () => {
|
||||
it("plans against the exact current profile revision", async () => {
|
||||
const reader = vi.fn().mockResolvedValue({ state: "ok", data: [PROFILE] });
|
||||
const document = await planModelProbe(reader as CoreReader, PROFILE_ID);
|
||||
expect(document.state).toBe("preview");
|
||||
if (document.state !== "preview") throw new Error("expected preview");
|
||||
expect(document.plan).toMatchObject({
|
||||
kind: "model_profile_probe",
|
||||
profile_id: PROFILE_ID,
|
||||
expected_profile_revision: 3,
|
||||
});
|
||||
expect(document.plan.operation_id).toMatch(
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/,
|
||||
);
|
||||
expect(document.plan.digest).toMatch(/^[0-9a-f]{64}$/);
|
||||
});
|
||||
const OPERATION_ID = "33333333-3333-4333-8333-333333333333";
|
||||
|
||||
it("proves operation support with schema-1 not_found before posting", async () => {
|
||||
const planned = await planModelProbe(
|
||||
vi.fn().mockResolvedValue({ state: "ok", data: [PROFILE] }) as CoreReader,
|
||||
PROFILE_ID,
|
||||
);
|
||||
if (planned.state !== "preview") throw new Error("expected preview");
|
||||
const operationId = planned.plan.operation_id;
|
||||
const reader = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({ state: "ok", data: notFound(operationId) });
|
||||
/** Reader that answers the profile list, then the queued operation reads. */
|
||||
function readerWith(...operationReads: unknown[]) {
|
||||
const reader = vi.fn(async (path: string) => {
|
||||
if (path === "/api/v1/model-profiles") {
|
||||
return { state: "ok", data: [PROFILE] };
|
||||
}
|
||||
return operationReads.shift() ?? { state: "not_found", code: "http_404" };
|
||||
});
|
||||
return reader;
|
||||
}
|
||||
|
||||
describe("model probe operation flow", () => {
|
||||
it("probes the exact current profile revision once support is proven", async () => {
|
||||
const reader = readerWith({ state: "ok", data: notFound(OPERATION_ID) });
|
||||
const post = vi
|
||||
.fn()
|
||||
.mockResolvedValue({ state: "committed", data: completed(operationId) });
|
||||
.mockResolvedValue({ state: "committed", data: completed(OPERATION_ID) });
|
||||
|
||||
const result = await executeModelProbe(
|
||||
planned.plan,
|
||||
reader as CoreReader,
|
||||
const result = await startModelProbe(
|
||||
reader as unknown as CoreReader,
|
||||
{ post } as CoreOperations,
|
||||
PROFILE_ID,
|
||||
OPERATION_ID,
|
||||
);
|
||||
|
||||
expect(result.state).toBe("completed");
|
||||
expect(reader).toHaveBeenCalledWith(
|
||||
`/api/v1/model-probe-operations/${operationId}`,
|
||||
`/api/v1/model-probe-operations/${OPERATION_ID}`,
|
||||
);
|
||||
expect(post).toHaveBeenCalledWith(
|
||||
`/api/v1/model-profiles/${PROFILE_ID}/probe`,
|
||||
{ operation_id: operationId, expected_profile_revision: 3 },
|
||||
{ operation_id: OPERATION_ID, expected_profile_revision: 3 },
|
||||
);
|
||||
});
|
||||
|
||||
it("does not post when the Core lacks the exact operation contract", async () => {
|
||||
const planned = await planModelProbe(
|
||||
vi.fn().mockResolvedValue({ state: "ok", data: [PROFILE] }) as CoreReader,
|
||||
it("generates its own operation id and treats a pending probe as submitted", async () => {
|
||||
const post = vi.fn(
|
||||
async (_path: string, body: Record<string, unknown>) => ({
|
||||
state: "committed",
|
||||
data: {
|
||||
...notFound(String(body.operation_id)),
|
||||
profile_id: PROFILE_ID,
|
||||
expected_profile_revision: 3,
|
||||
state: "pending",
|
||||
stale: false,
|
||||
},
|
||||
}),
|
||||
);
|
||||
const reader = vi.fn(async (path: string) =>
|
||||
path === "/api/v1/model-profiles"
|
||||
? { state: "ok", data: [PROFILE] }
|
||||
: {
|
||||
state: "ok",
|
||||
data: notFound(path.split("/").at(-1) ?? ""),
|
||||
},
|
||||
);
|
||||
const result = await startModelProbe(
|
||||
reader as unknown as CoreReader,
|
||||
{ post } as unknown as CoreOperations,
|
||||
PROFILE_ID,
|
||||
);
|
||||
if (planned.state !== "preview") throw new Error("expected preview");
|
||||
expect(result.state).toBe("pending");
|
||||
expect(result.operation_id).toMatch(
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/,
|
||||
);
|
||||
expect(modelProbeExitCode(result)).toBe(0);
|
||||
});
|
||||
|
||||
it("refuses an unknown profile without posting", async () => {
|
||||
const post = vi.fn();
|
||||
const result = await executeModelProbe(
|
||||
planned.plan,
|
||||
vi.fn().mockResolvedValue({
|
||||
const result = await startModelProbe(
|
||||
readerWith() as unknown as CoreReader,
|
||||
{ post } as CoreOperations,
|
||||
"44444444-4444-4444-8444-444444444444",
|
||||
OPERATION_ID,
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
state: "not_found",
|
||||
code: "model_profile_not_found",
|
||||
});
|
||||
expect(post).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not post when the Core lacks the exact operation contract", async () => {
|
||||
const post = vi.fn();
|
||||
const result = await startModelProbe(
|
||||
readerWith({
|
||||
state: "not_found",
|
||||
code: "http_404",
|
||||
}) as CoreReader,
|
||||
}) as unknown as CoreReader,
|
||||
{ post } as CoreOperations,
|
||||
PROFILE_ID,
|
||||
OPERATION_ID,
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
state: "disabled",
|
||||
@@ -153,19 +187,15 @@ describe("model probe operation flow", () => {
|
||||
});
|
||||
|
||||
it("does not post when a legacy Core returns a non-operation body", async () => {
|
||||
const planned = await planModelProbe(
|
||||
vi.fn().mockResolvedValue({ state: "ok", data: [PROFILE] }) as CoreReader,
|
||||
PROFILE_ID,
|
||||
);
|
||||
if (planned.state !== "preview") throw new Error("expected preview");
|
||||
const post = vi.fn();
|
||||
const result = await executeModelProbe(
|
||||
planned.plan,
|
||||
vi.fn().mockResolvedValue({
|
||||
const result = await startModelProbe(
|
||||
readerWith({
|
||||
state: "ok",
|
||||
data: { detail: "legacy_route_shape" },
|
||||
}) as CoreReader,
|
||||
}) as unknown as CoreReader,
|
||||
{ post } as CoreOperations,
|
||||
PROFILE_ID,
|
||||
OPERATION_ID,
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
state: "disabled",
|
||||
@@ -175,24 +205,19 @@ describe("model probe operation flow", () => {
|
||||
});
|
||||
|
||||
it("reconciles an unknown POST by GET and never treats not_found as replay permission", async () => {
|
||||
const planned = await planModelProbe(
|
||||
vi.fn().mockResolvedValue({ state: "ok", data: [PROFILE] }) as CoreReader,
|
||||
PROFILE_ID,
|
||||
const reader = readerWith(
|
||||
{ state: "ok", data: notFound(OPERATION_ID) },
|
||||
{ state: "ok", data: notFound(OPERATION_ID) },
|
||||
);
|
||||
if (planned.state !== "preview") throw new Error("expected preview");
|
||||
const operationId = planned.plan.operation_id;
|
||||
const reader = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({ state: "ok", data: notFound(operationId) })
|
||||
.mockResolvedValueOnce({ state: "ok", data: notFound(operationId) });
|
||||
const post = vi.fn().mockResolvedValue({
|
||||
state: "unknown",
|
||||
code: "transport_result_unknown",
|
||||
});
|
||||
const result = await executeModelProbe(
|
||||
planned.plan,
|
||||
reader as CoreReader,
|
||||
const result = await startModelProbe(
|
||||
reader as unknown as CoreReader,
|
||||
{ post } as CoreOperations,
|
||||
PROFILE_ID,
|
||||
OPERATION_ID,
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
state: "unknown",
|
||||
|
||||
@@ -1,11 +1,10 @@
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import type { CoreReader, ReadOutcome } from "./core-client.js";
|
||||
import type { CoreOperations } from "./core-operations.js";
|
||||
import { validateModelProfiles } from "./core-responses.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
const DIGEST = /^[0-9a-f]{64}$/;
|
||||
const RECEIPT_KEYS = new Set([
|
||||
"schema_version",
|
||||
"operation_id",
|
||||
@@ -59,23 +58,7 @@ const CODES = new Set([
|
||||
"embedding_unavailable",
|
||||
]);
|
||||
|
||||
interface ModelProbePlanBody {
|
||||
schema_version: 1;
|
||||
kind: "model_profile_probe";
|
||||
operation_id: string;
|
||||
profile_id: string;
|
||||
expected_profile_revision: number;
|
||||
}
|
||||
|
||||
export type ModelProbePlan = ModelProbePlanBody & { digest: string };
|
||||
|
||||
export type ModelProbeFlowDocument =
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "preview";
|
||||
plan: ModelProbePlan;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
@@ -91,11 +74,25 @@ export type ModelProbeFlowDocument =
|
||||
code: string;
|
||||
};
|
||||
|
||||
export async function planModelProbe(
|
||||
/**
|
||||
* Start one capability probe of a model profile (one real model request).
|
||||
*
|
||||
* Reads the profile's current revision so Core can refuse a probe of a
|
||||
* profile that changed meanwhile, checks that this Core tracks probe
|
||||
* operations (an old Core would start an untracked request), then submits it.
|
||||
* The probe runs asynchronously: the answer is usually `pending`, and
|
||||
* `models receipt <operation-id>` reads the result.
|
||||
*/
|
||||
export async function startModelProbe(
|
||||
reader: CoreReader,
|
||||
operations: CoreOperations,
|
||||
profileId: string,
|
||||
operationId: string = randomUUID(),
|
||||
): Promise<ModelProbeFlowDocument> {
|
||||
if (!UUID.test(profileId)) return failure("invalid", "invalid_profile_id");
|
||||
if (!UUID.test(operationId)) {
|
||||
return failure("invalid", "invalid_probe_operation_id");
|
||||
}
|
||||
const outcome = await reader<unknown>("/api/v1/model-profiles");
|
||||
if (outcome.state !== "ok") return fromReadFailure(outcome);
|
||||
const profiles = validateModelProfiles(outcome.data);
|
||||
@@ -105,71 +102,46 @@ export async function planModelProbe(
|
||||
);
|
||||
if (matches.length !== 1)
|
||||
return failure("not_found", "model_profile_not_found");
|
||||
const profile = matches[0];
|
||||
if (!Number.isSafeInteger(profile.revision) || profile.revision < 1) {
|
||||
const revision = matches[0].revision;
|
||||
if (!Number.isSafeInteger(revision) || revision < 1) {
|
||||
return failure("invalid", "model_profile_revision_invalid");
|
||||
}
|
||||
const plan = withDigest({
|
||||
schema_version: 1 as const,
|
||||
kind: "model_profile_probe" as const,
|
||||
operation_id: randomUUID(),
|
||||
profile_id: profileId,
|
||||
expected_profile_revision: profile.revision,
|
||||
});
|
||||
return { schema_version: 1, product: PRODUCT, state: "preview", plan };
|
||||
}
|
||||
|
||||
export async function executeModelProbe(
|
||||
value: unknown,
|
||||
reader: CoreReader,
|
||||
operations: CoreOperations,
|
||||
): Promise<ModelProbeFlowDocument> {
|
||||
const plan = validatePlan(value);
|
||||
if (!plan) return failure("invalid", "model_probe_plan_invalid");
|
||||
|
||||
// A 404 or a legacy/malformed body is not support. Some old Core versions
|
||||
// ignore the operation body and start an untracked external model request.
|
||||
const support = await reader<unknown>(operationPath(plan.operation_id));
|
||||
const support = await reader<unknown>(operationPath(operationId));
|
||||
if (support.state !== "ok") {
|
||||
return failure(
|
||||
"disabled",
|
||||
"model_probe_operations_unsupported",
|
||||
plan.operation_id,
|
||||
operationId,
|
||||
);
|
||||
}
|
||||
const supportReceipt = validateReceipt(support.data, plan.operation_id);
|
||||
const supportReceipt = validateReceipt(support.data, operationId);
|
||||
if (supportReceipt?.state !== "not_found") {
|
||||
return failure(
|
||||
"disabled",
|
||||
"model_probe_operations_unsupported",
|
||||
plan.operation_id,
|
||||
operationId,
|
||||
);
|
||||
}
|
||||
|
||||
const outcome = await operations.post<unknown>(
|
||||
`/api/v1/model-profiles/${plan.profile_id}/probe`,
|
||||
{
|
||||
operation_id: plan.operation_id,
|
||||
expected_profile_revision: plan.expected_profile_revision,
|
||||
},
|
||||
const posted = await operations.post<unknown>(
|
||||
`/api/v1/model-profiles/${profileId}/probe`,
|
||||
{ operation_id: operationId, expected_profile_revision: revision },
|
||||
);
|
||||
if (outcome.state === "rejected") {
|
||||
return failure("rejected", outcome.code, plan.operation_id);
|
||||
if (posted.state === "rejected") {
|
||||
return failure("rejected", posted.code, operationId);
|
||||
}
|
||||
if (outcome.state === "committed") {
|
||||
return receiptDocument(
|
||||
outcome.data,
|
||||
plan.operation_id,
|
||||
plan.profile_id,
|
||||
plan.expected_profile_revision,
|
||||
);
|
||||
if (posted.state === "committed") {
|
||||
return receiptDocument(posted.data, operationId, profileId, revision);
|
||||
}
|
||||
const reconciled = await readModelProbeReceipt(reader, plan.operation_id);
|
||||
const reconciled = await readModelProbeReceipt(reader, operationId);
|
||||
return reconciled.state === "not_found"
|
||||
? failure(
|
||||
"unknown",
|
||||
"model_probe_receipt_not_found_after_attempt",
|
||||
plan.operation_id,
|
||||
operationId,
|
||||
)
|
||||
: reconciled;
|
||||
}
|
||||
@@ -190,32 +162,24 @@ export async function readModelProbeReceipt(
|
||||
return receiptDocument(outcome.data, operationId);
|
||||
}
|
||||
|
||||
export function inspectModelProbePlan(value: unknown): ModelProbePlan | null {
|
||||
return validatePlan(value);
|
||||
}
|
||||
|
||||
export function modelProbeExitCode(document: ModelProbeFlowDocument): number {
|
||||
return ["preview", "completed"].includes(document.state) ? 0 : 1;
|
||||
// `pending` is a submitted probe still running: the submit succeeded.
|
||||
return ["completed", "pending"].includes(document.state) ? 0 : 1;
|
||||
}
|
||||
|
||||
export function renderModelProbeText(document: ModelProbeFlowDocument): string {
|
||||
if (document.state === "preview") {
|
||||
return [
|
||||
"预览:模型能力探测",
|
||||
`模型配置:${document.plan.profile_id}`,
|
||||
`预期版本:${document.plan.expected_profile_revision}`,
|
||||
`操作 ID:${document.plan.operation_id}`,
|
||||
`确认摘要:${document.plan.digest}`,
|
||||
"未发起模型请求。请保存 --json 输出,再用 models probe --plan-file <path> --confirm。",
|
||||
"",
|
||||
].join("\n");
|
||||
}
|
||||
return [
|
||||
const lines = [
|
||||
`探测状态:${document.state}`,
|
||||
`操作 ID:${document.operation_id ?? "无"}`,
|
||||
"code" in document ? `代码:${document.code}` : "已返回有界操作回执。",
|
||||
"",
|
||||
].join("\n");
|
||||
];
|
||||
if (document.state === "pending" && document.operation_id) {
|
||||
lines.push(
|
||||
`探测在后台进行;用 memhub models receipt ${document.operation_id} 查看结果。`,
|
||||
);
|
||||
}
|
||||
lines.push("");
|
||||
return lines.join("\n");
|
||||
}
|
||||
|
||||
function receiptDocument(
|
||||
@@ -249,31 +213,6 @@ function receiptDocument(
|
||||
};
|
||||
}
|
||||
|
||||
function validatePlan(value: unknown): ModelProbePlan | null {
|
||||
const plan = record(value);
|
||||
if (
|
||||
plan?.schema_version !== 1 ||
|
||||
plan.kind !== "model_profile_probe" ||
|
||||
typeof plan.operation_id !== "string" ||
|
||||
!UUID.test(plan.operation_id) ||
|
||||
typeof plan.profile_id !== "string" ||
|
||||
!UUID.test(plan.profile_id) ||
|
||||
!positiveInteger(plan.expected_profile_revision) ||
|
||||
typeof plan.digest !== "string" ||
|
||||
!DIGEST.test(plan.digest)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const expected = withDigest({
|
||||
schema_version: 1 as const,
|
||||
kind: "model_profile_probe" as const,
|
||||
operation_id: plan.operation_id,
|
||||
profile_id: plan.profile_id,
|
||||
expected_profile_revision: plan.expected_profile_revision,
|
||||
});
|
||||
return expected.digest === plan.digest ? expected : null;
|
||||
}
|
||||
|
||||
function validateReceipt(
|
||||
value: unknown,
|
||||
operationId: string,
|
||||
@@ -409,27 +348,6 @@ function failure(
|
||||
};
|
||||
}
|
||||
|
||||
function withDigest<T extends object>(body: T): T & { digest: string } {
|
||||
return {
|
||||
...body,
|
||||
digest: createHash("sha256").update(canonical(body)).digest("hex"),
|
||||
};
|
||||
}
|
||||
|
||||
function canonical(value: unknown): string {
|
||||
if (Array.isArray(value)) return `[${value.map(canonical).join(",")}]`;
|
||||
if (value && typeof value === "object") {
|
||||
return `{${Object.keys(value)
|
||||
.sort()
|
||||
.map(
|
||||
(key) =>
|
||||
`${JSON.stringify(key)}:${canonical((value as Record<string, unknown>)[key])}`,
|
||||
)
|
||||
.join(",")}}`;
|
||||
}
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
|
||||
function record(value: unknown): Record<string, unknown> | null {
|
||||
return value !== null && typeof value === "object" && !Array.isArray(value)
|
||||
? (value as Record<string, unknown>)
|
||||
|
||||
@@ -41,7 +41,7 @@ export type ModelReportDocument =
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "denied" | "unavailable" | "not_found" | "invalid";
|
||||
state: "denied" | "unavailable" | "not_found" | "invalid" | "rejected";
|
||||
code: string;
|
||||
};
|
||||
|
||||
@@ -136,7 +136,7 @@ function project(profile: ModelProfile): SafeModelProfile {
|
||||
}
|
||||
|
||||
function failure(
|
||||
state: "denied" | "unavailable" | "not_found" | "invalid",
|
||||
state: "denied" | "unavailable" | "not_found" | "invalid" | "rejected",
|
||||
code: string,
|
||||
): ModelReportDocument {
|
||||
return { schema_version: 1, product: PRODUCT, state, code };
|
||||
|
||||
@@ -126,29 +126,12 @@ describe("models activate", () => {
|
||||
expect(runtime.inspectRuntimeStatus).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("previews without contacting Core", async () => {
|
||||
const result = await capturedRun(["models", "activate", PROFILE, "--json"]);
|
||||
expect(result.code).toBe(0);
|
||||
const document = JSON.parse(result.stdout);
|
||||
expect(document.state).toBe("preview");
|
||||
expect(document.plan.kind).toBe("model_profile_activate");
|
||||
// Activation has no CAS body, so the plan must not pretend to carry one.
|
||||
expect(document.plan).not.toHaveProperty("expected_revision");
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("posts the activate endpoint with --confirm and reports the profile", async () => {
|
||||
it("posts the activate endpoint directly and reports the profile", async () => {
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: profileResponse(),
|
||||
});
|
||||
const result = await capturedRun([
|
||||
"models",
|
||||
"activate",
|
||||
PROFILE,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
const result = await capturedRun(["models", "activate", PROFILE, "--json"]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
`/api/v1/model-profiles/${PROFILE}/activate`,
|
||||
@@ -196,31 +179,22 @@ describe("models activate", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("models edit", () => {
|
||||
it("previews a patch from a config file without writing", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "om-model-edit-"));
|
||||
const config = path.join(directory, "patch.json");
|
||||
await writeFile(
|
||||
config,
|
||||
JSON.stringify({ expected_revision: 4, name: "Renamed" }),
|
||||
{ mode: 0o600 },
|
||||
);
|
||||
const result = await capturedRun([
|
||||
"models",
|
||||
"edit",
|
||||
PROFILE,
|
||||
"--config-file",
|
||||
config,
|
||||
"--json",
|
||||
]);
|
||||
describe("models remove", () => {
|
||||
it("previews without --yes and changes nothing", async () => {
|
||||
const result = await capturedRun(["models", "remove", PROFILE, "--json"]);
|
||||
expect(result.code).toBe(0);
|
||||
const document = JSON.parse(result.stdout);
|
||||
expect(document.state).toBe("preview");
|
||||
expect(document.plan.patch).toEqual({ name: "Renamed" });
|
||||
expect(transport.patch).not.toHaveBeenCalled();
|
||||
expect(JSON.parse(result.stdout)).toMatchObject({
|
||||
command: "models remove",
|
||||
state: "preview",
|
||||
target: PROFILE,
|
||||
applied: false,
|
||||
});
|
||||
expect(runtime.inspectRuntimeStatus).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
it("sends the exact patch body with the CAS revision on confirm", async () => {
|
||||
describe("models edit", () => {
|
||||
it("sends the exact patch body with the CAS revision directly", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "om-model-edit-"));
|
||||
const config = path.join(directory, "patch.json");
|
||||
await writeFile(
|
||||
@@ -242,7 +216,6 @@ describe("models edit", () => {
|
||||
PROFILE,
|
||||
"--config-file",
|
||||
config,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
|
||||
@@ -3,14 +3,12 @@
|
||||
* activate (`POST /model-profiles/{id}/activate`) and edit
|
||||
* (`PATCH /model-profiles/{id}`).
|
||||
*
|
||||
* Core owns the semantics. These flows reuse the same two-phase convention as
|
||||
* `models configure`: plan without `--confirm` (or produce a plan file), then
|
||||
* apply exactly that plan. The edit body mirrors the Web console's
|
||||
* `ModelProfilePatch`, including `expected_revision` as the compare-and-swap
|
||||
* guard Core requires.
|
||||
* Core owns the semantics. Both run directly: activation is reversible (activate
|
||||
* another profile) and an edit is guarded by `expected_revision`, the
|
||||
* compare-and-swap Core requires, mirroring the Web console's
|
||||
* `ModelProfilePatch`.
|
||||
*/
|
||||
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import type { ModelProfile } from "@one-memory/contracts";
|
||||
import {
|
||||
buildModelProfileUpdateRequest,
|
||||
@@ -22,29 +20,7 @@ import { PRODUCT } from "./version.js";
|
||||
|
||||
const PROFILE_ID = /^[A-Za-z0-9][\w.-]{0,127}$/;
|
||||
|
||||
export type ModelWritePlan =
|
||||
| {
|
||||
schema_version: 1;
|
||||
kind: "model_profile_activate";
|
||||
profile_id: string;
|
||||
digest: string;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
kind: "model_profile_edit";
|
||||
profile_id: string;
|
||||
expected_revision: number;
|
||||
patch: Record<string, unknown>;
|
||||
digest: string;
|
||||
};
|
||||
|
||||
export type ModelWriteDocument =
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "preview";
|
||||
plan: ModelWritePlan;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
@@ -60,38 +36,6 @@ export type ModelWriteDocument =
|
||||
code: string;
|
||||
};
|
||||
|
||||
function canonical(value: unknown): string {
|
||||
if (Array.isArray(value)) return `[${value.map(canonical).join(",")}]`;
|
||||
if (value !== null && typeof value === "object") {
|
||||
return `{${Object.entries(value as Record<string, unknown>)
|
||||
.sort(([left], [right]) => left.localeCompare(right))
|
||||
.map(([key, item]) => `${JSON.stringify(key)}:${canonical(item)}`)
|
||||
.join(",")}}`;
|
||||
}
|
||||
if (value === undefined) return "null";
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
|
||||
function digestOf(value: unknown): string {
|
||||
return createHash("sha256").update(canonical(value)).digest("hex");
|
||||
}
|
||||
|
||||
function preview(
|
||||
plan:
|
||||
| Omit<
|
||||
Extract<ModelWritePlan, { kind: "model_profile_activate" }>,
|
||||
"digest"
|
||||
>
|
||||
| Omit<Extract<ModelWritePlan, { kind: "model_profile_edit" }>, "digest">,
|
||||
): ModelWriteDocument {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "preview",
|
||||
plan: { ...plan, digest: digestOf(plan) } as ModelWritePlan,
|
||||
};
|
||||
}
|
||||
|
||||
function invalid(code: string): ModelWriteDocument {
|
||||
return { schema_version: 1, product: PRODUCT, state: "invalid", code };
|
||||
}
|
||||
@@ -122,87 +66,45 @@ function projectProfile(profile: ModelProfile): Record<string, unknown> {
|
||||
};
|
||||
}
|
||||
|
||||
// --- activate -------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Activation has no request body, so the plan carries only the target. There
|
||||
* is deliberately no `expected_revision`: Core would never read one, and
|
||||
* recording a check it does not perform would be a fake guard.
|
||||
*/
|
||||
export function planModelProfileActivate(input: {
|
||||
profileId: string;
|
||||
}): ModelWriteDocument {
|
||||
return PROFILE_ID.test(input.profileId)
|
||||
? preview({
|
||||
schema_version: 1,
|
||||
kind: "model_profile_activate",
|
||||
profile_id: input.profileId,
|
||||
})
|
||||
: invalid("model_profile_activate_input_invalid");
|
||||
}
|
||||
|
||||
export async function executeModelProfileActivate(
|
||||
plan: Extract<ModelWritePlan, { kind: "model_profile_activate" }>,
|
||||
/** Make `profileId` the active profile. Activation has no request body, so
|
||||
* there is no revision for Core to check. */
|
||||
export async function activateModelProfile(
|
||||
profileId: string,
|
||||
operations: CoreOperations,
|
||||
): Promise<ModelWriteDocument> {
|
||||
if (!PROFILE_ID.test(profileId)) {
|
||||
return invalid("model_profile_activate_input_invalid");
|
||||
}
|
||||
const result = await operations.post<unknown>(
|
||||
`/api/v1/model-profiles/${encodeURIComponent(plan.profile_id)}/activate`,
|
||||
`/api/v1/model-profiles/${encodeURIComponent(profileId)}/activate`,
|
||||
{},
|
||||
);
|
||||
return modelProfileResult(plan.profile_id, result);
|
||||
return modelProfileResult(profileId, result);
|
||||
}
|
||||
|
||||
// --- edit -----------------------------------------------------------------
|
||||
|
||||
export function planModelProfileEdit(input: {
|
||||
profileId: string;
|
||||
patch: unknown;
|
||||
}): ModelWriteDocument {
|
||||
// The shared builder owns patch validation (14-key allow-list,
|
||||
// expected_revision CAS, field bounds, api_key/clear_api_key exclusivity).
|
||||
// A rejected patch is `invalid`, never a plan the caller could confirm.
|
||||
let request: ReturnType<typeof buildModelProfileUpdateRequest>;
|
||||
try {
|
||||
request = buildModelProfileUpdateRequest(input.profileId, input.patch);
|
||||
} catch (error) {
|
||||
if (error instanceof ModelProfileInputError) {
|
||||
return invalid("model_profile_edit_input_invalid");
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
const patch = request.body as Record<string, unknown>;
|
||||
if (!PROFILE_ID.test(input.profileId)) {
|
||||
/** Apply a bounded patch (carrying `expected_revision`) to one profile; the
|
||||
* key, when given, comes from a controlled file and never from the patch. */
|
||||
export async function editModelProfile(
|
||||
profileId: string,
|
||||
patch: unknown,
|
||||
apiKey: string | undefined,
|
||||
operations: CorePatchOperations,
|
||||
): Promise<ModelWriteDocument> {
|
||||
if (
|
||||
!PROFILE_ID.test(profileId) ||
|
||||
patch === null ||
|
||||
typeof patch !== "object" ||
|
||||
Array.isArray(patch) ||
|
||||
"api_key" in patch
|
||||
) {
|
||||
return invalid("model_profile_edit_input_invalid");
|
||||
}
|
||||
return preview({
|
||||
schema_version: 1,
|
||||
kind: "model_profile_edit",
|
||||
profile_id: input.profileId,
|
||||
expected_revision: patch.expected_revision as number,
|
||||
// `api_key` is read from a controlled file at apply time, exactly like
|
||||
// `models configure`; the plan never carries the credential. The revision
|
||||
// is a top-level field, so it is stripped from the nested patch rather
|
||||
// than being duplicated in both places.
|
||||
patch: Object.fromEntries(
|
||||
Object.entries(patch).filter(
|
||||
([key]) => key !== "api_key" && key !== "expected_revision",
|
||||
),
|
||||
),
|
||||
});
|
||||
}
|
||||
|
||||
export async function executeModelProfileEdit(
|
||||
plan: Extract<ModelWritePlan, { kind: "model_profile_edit" }>,
|
||||
operations: CorePatchOperations,
|
||||
apiKey: string | undefined,
|
||||
): Promise<ModelWriteDocument> {
|
||||
// Re-validate through the shared builder at apply time too, so a hand-edited
|
||||
// plan cannot smuggle an out-of-bounds patch past the write.
|
||||
// The shared builder owns patch validation (allow-list, expected_revision
|
||||
// CAS, field bounds, api_key/clear_api_key exclusivity).
|
||||
let request: ReturnType<typeof buildModelProfileUpdateRequest>;
|
||||
try {
|
||||
request = buildModelProfileUpdateRequest(plan.profile_id, {
|
||||
...plan.patch,
|
||||
expected_revision: plan.expected_revision,
|
||||
request = buildModelProfileUpdateRequest(profileId, {
|
||||
...(patch as Record<string, unknown>),
|
||||
...(apiKey === undefined ? {} : { api_key: apiKey }),
|
||||
});
|
||||
} catch (error) {
|
||||
@@ -212,7 +114,7 @@ export async function executeModelProfileEdit(
|
||||
throw error;
|
||||
}
|
||||
const result = await operations.patch<unknown>(request.path, request.body);
|
||||
return modelProfileResult(plan.profile_id, result);
|
||||
return modelProfileResult(profileId, result);
|
||||
}
|
||||
|
||||
function modelProfileResult(
|
||||
@@ -254,69 +156,11 @@ function modelProfileResult(
|
||||
return committed(operationId, profile);
|
||||
}
|
||||
|
||||
export function validateModelWritePlan(value: unknown): ModelWritePlan | null {
|
||||
if (value === null || typeof value !== "object" || Array.isArray(value)) {
|
||||
return null;
|
||||
}
|
||||
const candidate = value as Record<string, unknown>;
|
||||
if (candidate.schema_version !== 1 || typeof candidate.kind !== "string") {
|
||||
return null;
|
||||
}
|
||||
const { digest, ...withoutDigest } = candidate;
|
||||
if (typeof digest !== "string" || digest !== digestOf(withoutDigest)) {
|
||||
return null;
|
||||
}
|
||||
switch (candidate.kind) {
|
||||
case "model_profile_activate":
|
||||
return typeof candidate.profile_id === "string" &&
|
||||
PROFILE_ID.test(candidate.profile_id) &&
|
||||
Object.keys(candidate).length === 3
|
||||
? (candidate as unknown as ModelWritePlan)
|
||||
: null;
|
||||
case "model_profile_edit": {
|
||||
if (
|
||||
typeof candidate.profile_id !== "string" ||
|
||||
!PROFILE_ID.test(candidate.profile_id) ||
|
||||
!Number.isSafeInteger(candidate.expected_revision)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
// The shared builder is the single judge of a patch body; the plan must
|
||||
// not carry `api_key` (that is read from a controlled file at apply).
|
||||
let built: ReturnType<typeof buildModelProfileUpdateRequest>;
|
||||
try {
|
||||
built = buildModelProfileUpdateRequest(candidate.profile_id, {
|
||||
...(candidate.patch as Record<string, unknown>),
|
||||
expected_revision: candidate.expected_revision,
|
||||
});
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
if ("api_key" in built.body) return null;
|
||||
return Object.keys(candidate).length === 6
|
||||
? (candidate as unknown as ModelWritePlan)
|
||||
: null;
|
||||
}
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function modelWriteExitCode(document: ModelWriteDocument): number {
|
||||
return ["preview", "committed"].includes(document.state) ? 0 : 1;
|
||||
return document.state === "committed" ? 0 : 1;
|
||||
}
|
||||
|
||||
export function renderModelWriteText(document: ModelWriteDocument): string {
|
||||
if (document.state === "preview") {
|
||||
const plan = document.plan;
|
||||
return [
|
||||
`预览:${plan.kind}`,
|
||||
`目标配置:${plan.profile_id}`,
|
||||
`确认摘要:${plan.digest}`,
|
||||
"未执行任何写操作。请把 --json 输出保存为 plan 文件,再用对应命令加 --confirm。",
|
||||
"",
|
||||
].join("\n");
|
||||
}
|
||||
if (document.state === "committed") {
|
||||
return `已保存模型配置:${document.receipt.profile_id}\nrevision=${document.receipt.revision}\n`;
|
||||
}
|
||||
|
||||
@@ -1,57 +0,0 @@
|
||||
import { constants } from "node:fs";
|
||||
import { type FileHandle, open } from "node:fs/promises";
|
||||
import { readBoundedJson } from "./bounded-file.js";
|
||||
|
||||
export type PlanAttempt = {
|
||||
schema_version: 1;
|
||||
operation_id: string;
|
||||
plan_digest: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* Durable client fence written before a mutating request. Once present, the
|
||||
* same plan may only query its operation receipt; it is never blindly posted
|
||||
* again after an unknown result.
|
||||
*/
|
||||
export async function claimPlanAttempt(
|
||||
planFile: string,
|
||||
attempt: PlanAttempt,
|
||||
): Promise<"first" | "prior" | "invalid"> {
|
||||
const journal = `${planFile}.one-memory-attempt`;
|
||||
let handle: FileHandle;
|
||||
try {
|
||||
handle = await open(
|
||||
journal,
|
||||
constants.O_WRONLY |
|
||||
constants.O_CREAT |
|
||||
constants.O_EXCL |
|
||||
(constants.O_NOFOLLOW ?? 0),
|
||||
0o600,
|
||||
);
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code !== "EEXIST") return "invalid";
|
||||
const current = await readBoundedJson(journal, 1024);
|
||||
return sameAttempt(current, attempt) ? "prior" : "invalid";
|
||||
}
|
||||
try {
|
||||
await handle.writeFile(`${JSON.stringify(attempt)}\n`, "utf8");
|
||||
await handle.sync();
|
||||
return "first";
|
||||
} catch {
|
||||
return "invalid";
|
||||
} finally {
|
||||
await handle.close().catch(() => undefined);
|
||||
}
|
||||
}
|
||||
|
||||
function sameAttempt(value: unknown, expected: PlanAttempt): boolean {
|
||||
return (
|
||||
value !== null &&
|
||||
typeof value === "object" &&
|
||||
!Array.isArray(value) &&
|
||||
(value as Record<string, unknown>).schema_version === 1 &&
|
||||
(value as Record<string, unknown>).operation_id === expected.operation_id &&
|
||||
(value as Record<string, unknown>).plan_digest === expected.plan_digest &&
|
||||
Object.keys(value).length === 3
|
||||
);
|
||||
}
|
||||
@@ -104,7 +104,7 @@ describe("package matrix harness", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("fails closed on every destructive UI-write command without a Runtime", {
|
||||
it("fails closed on every write command without a Runtime", {
|
||||
timeout: 120_000,
|
||||
}, async () => {
|
||||
const runtimeDist = await mkdtemp(
|
||||
@@ -140,7 +140,7 @@ describe("package matrix harness", () => {
|
||||
expect(document.state).toBe("verified");
|
||||
const results = document.commands.stopped_destructive.results;
|
||||
// Every destructive command fails closed while Runtime is stopped.
|
||||
expect(Object.keys(results)).toHaveLength(7);
|
||||
expect(Object.keys(results)).toHaveLength(8);
|
||||
for (const [name, result] of Object.entries(results)) {
|
||||
expect(result, `${name} did not fail closed`).toEqual({
|
||||
exit: 3,
|
||||
@@ -164,7 +164,7 @@ describe("package matrix harness", () => {
|
||||
const shim = [
|
||||
"#!/usr/bin/env node",
|
||||
"const a=process.argv.slice(2);",
|
||||
"if(['calls','notes','graph','models','memories'].includes(a[0])){",
|
||||
"if(['graph-width','models','memories','sources'].includes(a[0])){",
|
||||
' process.stdout.write(\'{"schema_version":1,"product":"memhub","state":"committed","operation_id":"x","receipt":{}}\\n\');',
|
||||
" process.exit(0);",
|
||||
"}",
|
||||
@@ -190,7 +190,7 @@ describe("package matrix harness", () => {
|
||||
expect(failure?.state).toBe("failed");
|
||||
expect(
|
||||
failure?.failures.filter((item) => item.includes("did not fail closed")),
|
||||
).toHaveLength(7);
|
||||
).toHaveLength(8);
|
||||
});
|
||||
|
||||
it("can require an expected bundle SHA and reports bundle metadata", {
|
||||
@@ -291,9 +291,7 @@ describe("package matrix harness", () => {
|
||||
"sync",
|
||||
"memories",
|
||||
"profile",
|
||||
"calls",
|
||||
"notes",
|
||||
"graph",
|
||||
"graph-width",
|
||||
"models",
|
||||
"host",
|
||||
"doctor",
|
||||
@@ -311,14 +309,12 @@ describe("package matrix harness", () => {
|
||||
"uninstall",
|
||||
"delete-data",
|
||||
"recall",
|
||||
"setup",
|
||||
"scan",
|
||||
"progress",
|
||||
"jobs",
|
||||
"path",
|
||||
"scan-roots",
|
||||
"adapter",
|
||||
"write",
|
||||
"task",
|
||||
"serve",
|
||||
];
|
||||
const catalog = {
|
||||
|
||||
@@ -145,13 +145,15 @@ describe("memhub short-command contract", () => {
|
||||
[command.usage, command.alternative_usage ?? ""].join("\n"),
|
||||
).join("\n");
|
||||
for (const shortForm of [
|
||||
"memhub calls clear",
|
||||
"memhub notes add",
|
||||
"memhub notes process",
|
||||
"memhub memories add",
|
||||
"memhub memories add-status",
|
||||
"memhub models setup",
|
||||
"memhub models add",
|
||||
"memhub models test",
|
||||
"memhub models activate",
|
||||
"memhub models edit",
|
||||
"memhub graph width",
|
||||
"memhub graph-width",
|
||||
"memhub sources connect",
|
||||
"memhub write submit",
|
||||
"memhub write status",
|
||||
"memhub memories update",
|
||||
@@ -159,6 +161,10 @@ describe("memhub short-command contract", () => {
|
||||
]) {
|
||||
expect(flattened).toContain(shortForm);
|
||||
}
|
||||
// No "note" concept and no plan files on the CLI surface.
|
||||
for (const removed of ["memhub notes", "memhub calls", "--plan-file"]) {
|
||||
expect(flattened).not.toContain(removed);
|
||||
}
|
||||
expect(COMMANDS.some((command) => command.name === "permission")).toBe(
|
||||
false,
|
||||
);
|
||||
@@ -211,7 +217,7 @@ describe("memhub short-command contract", () => {
|
||||
// but an earlier catalog text still claimed they were "尚不可用". A user
|
||||
// reading `help path` would not try a command that works.
|
||||
it("does not call the wired path/adapter commands unavailable", () => {
|
||||
for (const name of ["path", "adapter"]) {
|
||||
for (const name of ["scan-roots", "adapter"]) {
|
||||
const command = COMMANDS.find((entry) => entry.name === name);
|
||||
expect(command).toBeDefined();
|
||||
const text = [
|
||||
@@ -277,7 +283,7 @@ describe("memhub short-command contract", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps plan-file apply previews free of diagnostic side effects", {
|
||||
it("keeps destructive previews free of diagnostic side effects", {
|
||||
timeout: 20_000,
|
||||
}, async () => {
|
||||
expect(
|
||||
@@ -291,9 +297,10 @@ describe("memhub short-command contract", () => {
|
||||
);
|
||||
try {
|
||||
for (const [index, argv] of [
|
||||
["sources", "import", "--plan-file", path.join(root, "plan.json")],
|
||||
["models", "configure", "--plan-file", path.join(root, "plan.json")],
|
||||
["memories", "apply", "--plan-file", path.join(root, "plan.json")],
|
||||
["memories", "delete", "m-1"],
|
||||
["models", "remove", "44444444-4444-4444-8444-444444444444"],
|
||||
["backup", "restore", "--source", path.join(root, "backup.sqlite3")],
|
||||
["graph-width", "--decision", "rebuild_with_new"],
|
||||
].entries()) {
|
||||
const dataDir = path.join(root, `data-${index}`);
|
||||
await import("node:fs/promises").then(({ mkdir }) =>
|
||||
@@ -305,8 +312,8 @@ describe("memhub short-command contract", () => {
|
||||
stdio: "pipe",
|
||||
});
|
||||
} catch {
|
||||
// The missing plan or stopped Runtime is expected; the guard is
|
||||
// that even the failure path did not add a diagnostic file.
|
||||
// A failure path is acceptable; the guard is that no preview adds
|
||||
// a diagnostic file.
|
||||
}
|
||||
expect(await readdir(dataDir)).toEqual([]);
|
||||
}
|
||||
|
||||
@@ -1,122 +1,56 @@
|
||||
// A6: the rules behind the six product commands.
|
||||
// The rules behind the commands that attach to the Runtime directly.
|
||||
//
|
||||
// The parsers are where the contract actually lives — "needs --yes to change
|
||||
// anything" and "no write without evidence" are decisions, not formatting — so
|
||||
// they are tested directly rather than only through the CLI.
|
||||
// The parsers are where the contract actually lives — "only irreversible
|
||||
// operations ask for --yes" and "unknown flags are rejected" are decisions,
|
||||
// not formatting — so they are tested directly rather than only through the
|
||||
// CLI.
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { parseOpenAIMessages } from "./core-client.js";
|
||||
import {
|
||||
asPlanDocument,
|
||||
describe as describePlan,
|
||||
parseProductCommand,
|
||||
parseBackupCommand,
|
||||
parseScanCommand,
|
||||
parseScanRootsCommand,
|
||||
parseTaskCommand,
|
||||
parseWriteCommand,
|
||||
restoreConfirmationMatches,
|
||||
} from "./product-flows.js";
|
||||
|
||||
describe("mutating commands are two-phase", () => {
|
||||
it("plans without --yes", () => {
|
||||
const parsed = parseProductCommand("scan", ["--root", "/tmp/a"]);
|
||||
expect(parsed?.confirmed).toBe(false);
|
||||
});
|
||||
|
||||
it("confirms with --yes", () => {
|
||||
const parsed = parseProductCommand("scan", ["--root", "/tmp/a", "--yes"]);
|
||||
expect(parsed?.confirmed).toBe(true);
|
||||
});
|
||||
|
||||
it("says what it would do", () => {
|
||||
const parsed = parseProductCommand("scan", ["--root", "/tmp/a"]);
|
||||
// `?.plan!` would let a null parse pass silently; assert the parse first.
|
||||
if (parsed === null) {
|
||||
throw new Error("scan with a root must parse");
|
||||
}
|
||||
expect(describePlan(parsed.plan)).toContain("scan: /tmp/a");
|
||||
});
|
||||
|
||||
it("marks a plan as not applied", () => {
|
||||
// Assert the parse itself: `?.` let a null result satisfy this test for as
|
||||
// long as `backup create` was unparseable.
|
||||
const parsed = parseProductCommand("backup", [
|
||||
"create",
|
||||
"--target",
|
||||
"/tmp/b",
|
||||
]);
|
||||
if (parsed === null) {
|
||||
throw new Error("backup create must parse");
|
||||
}
|
||||
expect(parsed.plan.verb).toBe("create");
|
||||
expect(asPlanDocument(parsed.plan).applied).toBe(false);
|
||||
});
|
||||
|
||||
it("parses the verb for every command that takes one", () => {
|
||||
// `path add` / `path remove` / `backup create` were rejected outright
|
||||
// before the verb was skipped in parsing.
|
||||
describe("scan-roots", () => {
|
||||
it("reads the root, source and recursive flag off add", () => {
|
||||
expect(
|
||||
parseProductCommand("path", ["add", "--root", "/tmp/x"])?.plan.action,
|
||||
).toBe("add scan root");
|
||||
expect(
|
||||
parseProductCommand("path", ["remove", "--root", "/tmp/x"])?.plan.action,
|
||||
).toBe("remove scan root");
|
||||
expect(
|
||||
parseProductCommand("backup", ["restore", "--source", "/tmp/b"])?.plan
|
||||
.action,
|
||||
).toBe("restore backup");
|
||||
parseScanRootsCommand([
|
||||
"add",
|
||||
"--root",
|
||||
"/tmp/x",
|
||||
"--source",
|
||||
"codex",
|
||||
"--no-recursive",
|
||||
]),
|
||||
).toEqual({
|
||||
kind: "add",
|
||||
root: "/tmp/x",
|
||||
source: "codex",
|
||||
recursive: false,
|
||||
});
|
||||
expect(parseScanRootsCommand(["add", "--root", "/tmp/x"])).toEqual({
|
||||
kind: "add",
|
||||
root: "/tmp/x",
|
||||
recursive: true,
|
||||
});
|
||||
});
|
||||
|
||||
it("defaults a scan to all registered roots", () => {
|
||||
const parsed = parseProductCommand("scan", []);
|
||||
expect(parsed?.plan.target).toBe("all registered roots");
|
||||
});
|
||||
});
|
||||
|
||||
describe("unknown flags are rejected", () => {
|
||||
it("rejects a typo'd flag rather than ignoring it", () => {
|
||||
// A silently ignored --yes would turn a real apply into a plan, and the
|
||||
// user would believe the change had been made.
|
||||
expect(parseProductCommand("scan", ["--yesx"])).toBeNull();
|
||||
expect(parseProductCommand("scan", ["--root"])).toBeNull();
|
||||
expect(parseProductCommand("scan", ["--root", "--yes"])).toBeNull();
|
||||
});
|
||||
|
||||
it("rejects an unknown verb for backup", () => {
|
||||
it("distinguishes remove from add and refuses add-only flags on remove", () => {
|
||||
expect(parseScanRootsCommand(["remove", "--root", "/tmp/x"])).toEqual({
|
||||
kind: "remove",
|
||||
root: "/tmp/x",
|
||||
});
|
||||
expect(
|
||||
parseProductCommand("backup", ["make", "--target", "/tmp/b"]),
|
||||
parseScanRootsCommand(["remove", "--root", "/tmp/x", "--no-recursive"]),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
// `path` and `scan` share one parser. Before this guard, an unknown verb was
|
||||
// silently treated as `path add` or a default `scan`, so `--yes` executed a
|
||||
// different operation than the one the caller named.
|
||||
it("rejects an unknown verb for path without turning it into add", () => {
|
||||
expect(
|
||||
parseProductCommand("path", ["frobnicate", "--root", "/tmp/x", "--yes"]),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("rejects an unknown verb for scan without turning it into all", () => {
|
||||
expect(
|
||||
parseProductCommand("scan", ["frobnicate", "--all", "--yes"]),
|
||||
).toBeNull();
|
||||
expect(parseProductCommand("scan", ["frobnicate", "--yes"])).toBeNull();
|
||||
});
|
||||
|
||||
it("still accepts the documented verbless scan", () => {
|
||||
expect(parseProductCommand("scan", ["--all", "--yes"])?.plan.all).toBe(
|
||||
true,
|
||||
);
|
||||
expect(
|
||||
parseProductCommand("scan", ["--source", "codex"])?.plan.sourceId,
|
||||
).toBe("codex");
|
||||
});
|
||||
|
||||
it("rejects flags that the named verb does not consume", () => {
|
||||
// These were accepted and then silently dropped. An operator who typed
|
||||
// `path remove --source codex` would believe the source filter applied.
|
||||
expect(
|
||||
parseProductCommand("path", [
|
||||
parseScanRootsCommand([
|
||||
"remove",
|
||||
"--root",
|
||||
"/tmp/x",
|
||||
@@ -124,137 +58,103 @@ describe("unknown flags are rejected", () => {
|
||||
"codex",
|
||||
]),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("is direct, but keeps accepting a legacy --yes", () => {
|
||||
expect(parseScanRootsCommand(["add", "--root", "/tmp/x", "--yes"])).toEqual(
|
||||
{ kind: "add", root: "/tmp/x", recursive: true },
|
||||
);
|
||||
});
|
||||
|
||||
it("rejects unknown verbs, typos and repeated flags", () => {
|
||||
expect(parseScanRootsCommand(["create", "--root", "/tmp/x"])).toBeNull();
|
||||
expect(parseScanRootsCommand(["add", "--rot", "/tmp/x"])).toBeNull();
|
||||
expect(parseScanRootsCommand(["add"])).toBeNull();
|
||||
expect(
|
||||
parseProductCommand("path", [
|
||||
"remove",
|
||||
"--root",
|
||||
"/tmp/x",
|
||||
"--no-recursive",
|
||||
]),
|
||||
parseScanRootsCommand(["add", "--root", "/a", "--root", "/b"]),
|
||||
).toBeNull();
|
||||
expect(
|
||||
parseProductCommand("path", ["add", "--root", "/tmp/x", "--full"]),
|
||||
parseScanRootsCommand(["add", "--root", "/a", "--yes", "--yes"]),
|
||||
).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("scan", () => {
|
||||
it("needs a source: there is no endpoint that scans a bare path", () => {
|
||||
expect(parseScanCommand(["--source", "codex"])).toEqual({
|
||||
kind: "source",
|
||||
sourceId: "codex",
|
||||
});
|
||||
expect(parseScanCommand([])).toBeNull();
|
||||
expect(parseScanCommand(["--root", "/tmp/a"])).toBeNull();
|
||||
});
|
||||
|
||||
it("no longer accepts --full, which was never sent to Core", () => {
|
||||
expect(parseScanCommand(["--source", "codex", "--full"])).toBeNull();
|
||||
});
|
||||
|
||||
it("keeps --all as a hidden alias and a legacy --yes", () => {
|
||||
expect(parseScanCommand(["--all", "--yes"])).toEqual({ kind: "all" });
|
||||
expect(parseScanCommand(["--all", "--source", "codex"])).toBeNull();
|
||||
expect(parseScanCommand(["--source", "codex", "--yes"])).toEqual({
|
||||
kind: "source",
|
||||
sourceId: "codex",
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects a verb, a bad source id and repeated flags", () => {
|
||||
expect(parseScanCommand(["all"])).toBeNull();
|
||||
expect(parseScanCommand(["--source", "Bad Id"])).toBeNull();
|
||||
expect(
|
||||
parseScanCommand(["--source", "codex", "--source", "cursor"]),
|
||||
).toBeNull();
|
||||
expect(
|
||||
parseProductCommand("backup", [
|
||||
"create",
|
||||
"--target",
|
||||
"/tmp/b",
|
||||
"--source",
|
||||
"/tmp/s",
|
||||
]),
|
||||
parseScanCommand(["--source", "codex", "--json", "--json"]),
|
||||
).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("backup", () => {
|
||||
it("creates directly and requires a target", () => {
|
||||
expect(parseBackupCommand(["create", "--target", "/tmp/b"])).toEqual({
|
||||
verb: "create",
|
||||
target: "/tmp/b",
|
||||
});
|
||||
expect(parseBackupCommand(["create"])).toBeNull();
|
||||
expect(
|
||||
parseProductCommand("backup", [
|
||||
"restore",
|
||||
"--source",
|
||||
"/tmp/s",
|
||||
"--target",
|
||||
"/tmp/t",
|
||||
]),
|
||||
parseBackupCommand(["create", "--target", "/tmp/b", "--source", "/x"]),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
// A later value silently won before this guard. For `scan --source` that
|
||||
// means one extra shell/make variable can change which source is scanned
|
||||
// while the operator believes both were applied.
|
||||
it("rejects repeated value-bearing flags instead of letting the last one win", () => {
|
||||
it("restores only with --yes and requires a source", () => {
|
||||
expect(parseBackupCommand(["restore", "--source", "/tmp/b"])).toEqual({
|
||||
verb: "restore",
|
||||
source: "/tmp/b",
|
||||
confirmed: false,
|
||||
});
|
||||
expect(
|
||||
parseProductCommand("scan", ["--source", "codex", "--source", "cursor"]),
|
||||
).toBeNull();
|
||||
expect(
|
||||
parseProductCommand("path", [
|
||||
"add",
|
||||
"--root",
|
||||
"/tmp/a",
|
||||
"--root",
|
||||
"/tmp/b",
|
||||
]),
|
||||
).toBeNull();
|
||||
expect(
|
||||
parseProductCommand("backup", [
|
||||
"create",
|
||||
"--target",
|
||||
"/tmp/a",
|
||||
"--target",
|
||||
"/tmp/b",
|
||||
]),
|
||||
).toBeNull();
|
||||
parseBackupCommand(["restore", "--source", "/tmp/b", "--yes"]),
|
||||
).toMatchObject({ confirmed: true });
|
||||
expect(parseBackupCommand(["restore", "--yes"])).toBeNull();
|
||||
});
|
||||
|
||||
it("rejects repeated boolean flags instead of treating them as idempotent", () => {
|
||||
expect(parseProductCommand("scan", ["--all", "--all", "--yes"])).toBeNull();
|
||||
expect(
|
||||
parseProductCommand("scan", ["--full", "--full", "--yes"]),
|
||||
).toBeNull();
|
||||
expect(
|
||||
parseProductCommand("path", [
|
||||
"add",
|
||||
"--root",
|
||||
"/tmp/a",
|
||||
"--no-recursive",
|
||||
"--no-recursive",
|
||||
]),
|
||||
).toBeNull();
|
||||
expect(
|
||||
parseProductCommand("backup", [
|
||||
"create",
|
||||
"--target",
|
||||
"/tmp/a",
|
||||
"--yes",
|
||||
"--yes",
|
||||
]),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("rejects a repeated global --json spelling", () => {
|
||||
expect(
|
||||
parseProductCommand("scan", ["--root", "/tmp/a", "--json", "--json"]),
|
||||
).toBeNull();
|
||||
expect(
|
||||
parseProductCommand("backup", [
|
||||
"create",
|
||||
"--target",
|
||||
"/tmp/a",
|
||||
"--json",
|
||||
"--json",
|
||||
]),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("requires a source when restoring", () => {
|
||||
expect(
|
||||
parseProductCommand("backup", ["restore", "--target", "/tmp/b"]),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("parses the explicit restore confirmation path", () => {
|
||||
const parsed = parseProductCommand("backup", [
|
||||
it("accepts the legacy --confirm <path> and checks it lexically", () => {
|
||||
const parsed = parseBackupCommand([
|
||||
"restore",
|
||||
"--source",
|
||||
"/tmp/backups/a.sqlite3",
|
||||
"/tmp/b",
|
||||
"--yes",
|
||||
"--confirm",
|
||||
"/tmp/backups/a.sqlite3",
|
||||
"/tmp/b",
|
||||
]);
|
||||
expect(parsed).not.toBeNull();
|
||||
expect(parsed?.confirmPath).toBe("/tmp/backups/a.sqlite3");
|
||||
expect(
|
||||
restoreConfirmationMatches("/tmp/backups/a.sqlite3", parsed?.confirmPath),
|
||||
).toBe(true);
|
||||
expect(
|
||||
restoreConfirmationMatches("/tmp/backups/a.sqlite3", undefined),
|
||||
).toBe(false);
|
||||
expect(
|
||||
restoreConfirmationMatches(
|
||||
"/tmp/backups/a.sqlite3",
|
||||
"/tmp/backups/../backups/a.sqlite3",
|
||||
),
|
||||
).toBe(false);
|
||||
expect(parsed).toMatchObject({ confirmPath: "/tmp/b" });
|
||||
expect(restoreConfirmationMatches("/tmp/b", "/tmp/b")).toBe(true);
|
||||
expect(restoreConfirmationMatches("/tmp/b", undefined)).toBe(true);
|
||||
expect(restoreConfirmationMatches("/tmp/b", "/tmp/./b")).toBe(false);
|
||||
});
|
||||
|
||||
it("requires a target when creating a backup", () => {
|
||||
expect(parseProductCommand("backup", ["create"])).toBeNull();
|
||||
it("rejects an unknown verb", () => {
|
||||
expect(parseBackupCommand(["delete", "--target", "/tmp/b"])).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -305,11 +205,19 @@ describe("write", () => {
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("parses status under write", () => {
|
||||
it("parses status under write, positional or --id", () => {
|
||||
expect(parseWriteCommand(["status", "--id", "request-1"])).toEqual({
|
||||
kind: "status",
|
||||
id: "request-1",
|
||||
});
|
||||
expect(parseWriteCommand(["status", "request-1"])).toEqual({
|
||||
kind: "status",
|
||||
id: "request-1",
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects a repeated --session instead of letting the last one win", () => {
|
||||
expect(parseWriteCommand([...base, "--session", "s2"])).toBeNull();
|
||||
});
|
||||
|
||||
it("marks --live as priority live", () => {
|
||||
@@ -366,53 +274,3 @@ describe("task", () => {
|
||||
expect(parseTaskCommand(["--id", "req-1", "--json", "--json"])).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("path add and remove carry what the endpoint needs", () => {
|
||||
it("reads the source and recursive flag off the path add", () => {
|
||||
const parsed = parseProductCommand("path", [
|
||||
"add",
|
||||
"--root",
|
||||
"/tmp/scanme",
|
||||
"--source",
|
||||
"codex",
|
||||
"--no-recursive",
|
||||
]);
|
||||
// `?.plan!` let a null parse pass silently, so assert the parse itself.
|
||||
if (parsed === null) {
|
||||
throw new Error("path add must parse");
|
||||
}
|
||||
expect(parsed.plan.action).toBe("add scan root");
|
||||
expect(parsed.plan.target).toBe("/tmp/scanme");
|
||||
expect(parsed.plan.detail).toContain("source=codex");
|
||||
expect(parsed.plan.detail).toContain("recursive=false");
|
||||
});
|
||||
|
||||
it("distinguishes remove from add", () => {
|
||||
const parsed = parseProductCommand("path", [
|
||||
"remove",
|
||||
"--root",
|
||||
"/tmp/scanme",
|
||||
]);
|
||||
if (parsed === null) {
|
||||
throw new Error("path remove must parse");
|
||||
}
|
||||
// The action is the discriminator at apply time: add and remove share a
|
||||
// `--root`, so a mis-set action would register the directory it meant to delete.
|
||||
expect(parsed.plan.action).toBe("remove scan root");
|
||||
});
|
||||
|
||||
it("does not silently drop --yes", () => {
|
||||
const parsed = parseProductCommand("path", [
|
||||
"add",
|
||||
"--root",
|
||||
"/tmp/a",
|
||||
"--yes",
|
||||
]);
|
||||
expect(parsed?.confirmed).toBe(true);
|
||||
// Without this, a typo'd `--yes` would fall through to a plan and the user
|
||||
// would believe the change had been made.
|
||||
|
||||
const without = parseProductCommand("path", ["add", "--root", "/tmp/a"]);
|
||||
expect(without?.confirmed).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -1,258 +1,163 @@
|
||||
// A6: the product commands Runtime exposes on the CLI.
|
||||
// Parsers for the commands that attach to the Runtime directly:
|
||||
// `scan-roots` (alias `path`), `scan`, `backup`, `write`.
|
||||
//
|
||||
// These are the six commands the objective names: path, adapter, scan, write,
|
||||
// task, backup. Two rules run through all of them:
|
||||
// Rules shared by all of them:
|
||||
//
|
||||
// - **Mutating commands are two-phase.** Without `--yes` they print a plan and
|
||||
// change nothing. This is the convention the existing install/service
|
||||
// commands already follow, and it is why `scan` and `backup` default to a
|
||||
// plan rather than quietly doing work.
|
||||
// - **Output is machine-readable.** `--json` emits one document on stdout so an
|
||||
// agent can consume it; the human form is text. Never both, so a caller that
|
||||
// parses stdout is not surprised by prose.
|
||||
// - **Only irreversible operations ask for `--yes`.** `backup restore`
|
||||
// overwrites the database, so without `--yes` it previews and changes
|
||||
// nothing. Registering a scan root, scanning a source, creating a backup and
|
||||
// submitting a write are direct. A legacy `--yes` on a direct command is
|
||||
// accepted and ignored so older scripts keep working.
|
||||
// - **Unknown or repeated flags are rejected**, never ignored: a dropped typo
|
||||
// would silently change what the command does.
|
||||
// - **Output is machine-readable.** `--json` emits one document on stdout.
|
||||
//
|
||||
// The commands talk to the Runtime's HTTP API, not to the database and not to
|
||||
// HY. That is the A6 rule: CLI and Electron are clients of the same Runtime.
|
||||
// the SDK: CLI and desktop are clients of the same Runtime.
|
||||
|
||||
/** What a mutating command would do, before it is allowed to do it. */
|
||||
export interface Plan {
|
||||
readonly action: string;
|
||||
readonly target: string;
|
||||
readonly detail?: string;
|
||||
/** Which form of the command this plan is for, when one name has several.
|
||||
*
|
||||
* `backup` has `create` and `restore`, and applying the wrong one is not a
|
||||
* harmless mistake — so the verb travels with the plan the user confirmed.
|
||||
*/
|
||||
readonly verb?: string;
|
||||
/** The source the Runtime endpoint addresses, when the plan names one.
|
||||
*
|
||||
* Carried on the plan rather than re-parsed at apply time so the plan the
|
||||
* user confirmed and the request actually sent cannot disagree.
|
||||
*/
|
||||
readonly sourceId?: string;
|
||||
/** `scan --all`: every source Runtime actually found, not a named one. */
|
||||
readonly all?: boolean;
|
||||
export type ScanRootsCommand =
|
||||
| {
|
||||
readonly kind: "add";
|
||||
readonly root: string;
|
||||
readonly source?: string;
|
||||
readonly recursive: boolean;
|
||||
}
|
||||
| { readonly kind: "remove"; readonly root: string };
|
||||
|
||||
export type ScanCommand =
|
||||
| { readonly kind: "source"; readonly sourceId: string }
|
||||
/** Hidden alias of `sources connect --all-detected --import`. */
|
||||
| { readonly kind: "all" };
|
||||
|
||||
export type BackupCommand =
|
||||
| { readonly verb: "create"; readonly target: string }
|
||||
| {
|
||||
readonly verb: "restore";
|
||||
readonly source: string;
|
||||
readonly confirmed: boolean;
|
||||
/** Legacy `--confirm <path>`: accepted, and must repeat `--source`. */
|
||||
readonly confirmPath?: string;
|
||||
};
|
||||
|
||||
interface ParsedFlags {
|
||||
readonly values: Map<string, string>;
|
||||
readonly switches: Set<string>;
|
||||
}
|
||||
|
||||
export function planAddRoot(
|
||||
root: string,
|
||||
source: string | undefined,
|
||||
recursive: boolean,
|
||||
): Plan {
|
||||
return {
|
||||
action: "add scan root",
|
||||
target: root,
|
||||
detail: `source=${source ?? "auto"} recursive=${recursive}`,
|
||||
};
|
||||
}
|
||||
|
||||
export function planRemoveRoot(root: string): Plan {
|
||||
return { action: "remove scan root", target: root };
|
||||
}
|
||||
|
||||
export function planScan(root: string | undefined, full: boolean): Plan {
|
||||
return {
|
||||
action: "scan",
|
||||
target: root ?? "all registered roots",
|
||||
detail: `full=${full}`,
|
||||
};
|
||||
}
|
||||
|
||||
export function planScanSource(sourceId: string, full: boolean): Plan {
|
||||
return {
|
||||
action: "scan",
|
||||
target: `source ${sourceId}`,
|
||||
detail: `full=${full}`,
|
||||
sourceId,
|
||||
};
|
||||
}
|
||||
|
||||
export function planScanAll(full: boolean): Plan {
|
||||
return {
|
||||
action: "scan",
|
||||
target: "all detected sources",
|
||||
detail: `full=${full}`,
|
||||
all: true,
|
||||
};
|
||||
}
|
||||
|
||||
export function planBackup(target: string): Plan {
|
||||
return { action: "create backup", target, verb: "create" };
|
||||
}
|
||||
|
||||
export function planRestore(source: string): Plan {
|
||||
return { action: "restore backup", target: source, verb: "restore" };
|
||||
}
|
||||
|
||||
export interface ProductCommand {
|
||||
readonly plan: Plan;
|
||||
readonly confirmed: boolean;
|
||||
/** For destructive verbs with a named artifact, the exact value repeated by
|
||||
* the user. `backup restore` requires this to match `--source`. */
|
||||
readonly confirmPath?: string;
|
||||
}
|
||||
|
||||
/** Parse a mutating product command.
|
||||
*
|
||||
* Unknown flags are rejected rather than ignored: a typo'd `--yes` that is
|
||||
* silently dropped would turn a real apply into a plan, and the user would
|
||||
* believe the change had been made.
|
||||
*/
|
||||
export function parseProductCommand(
|
||||
name: string,
|
||||
/** Read `--flag value` pairs and bare switches; null on anything unknown,
|
||||
* repeated or missing its value. */
|
||||
function parseFlags(
|
||||
args: readonly string[],
|
||||
): ProductCommand | null {
|
||||
let root: string | undefined;
|
||||
let source: string | undefined;
|
||||
let target: string | undefined;
|
||||
let recursive = true;
|
||||
let full = false;
|
||||
let all = false;
|
||||
let confirmed = false;
|
||||
let confirmPath: string | undefined;
|
||||
const seen = new Set<string>();
|
||||
const seenValues = new Set<string>();
|
||||
|
||||
const flags = new Set([
|
||||
"--root",
|
||||
"--source",
|
||||
"--target",
|
||||
"--confirm",
|
||||
"--no-recursive",
|
||||
"--full",
|
||||
"--all",
|
||||
"--yes",
|
||||
"--json",
|
||||
]);
|
||||
|
||||
// The first token is the verb for commands that have one (`path add`,
|
||||
// `backup create`). It used to fall into the flag loop and be rejected as an
|
||||
// unknown flag, which made every one of these commands permanently
|
||||
// unparseable — `path add --root X` printed "参数无效" even though `help`
|
||||
// documented it. The tests missed this because they asserted
|
||||
// `parsed?.plan!` and null satisfies `?.` + `!`.
|
||||
const verb = args[0];
|
||||
const start = verb !== undefined && !verb.startsWith("--") ? 1 : 0;
|
||||
|
||||
for (let index = start; index < args.length; index += 1) {
|
||||
valueFlags: readonly string[],
|
||||
switchFlags: readonly string[],
|
||||
): ParsedFlags | null {
|
||||
const values = new Map<string, string>();
|
||||
const switches = new Set<string>();
|
||||
for (let index = 0; index < args.length; index += 1) {
|
||||
const arg = args[index];
|
||||
if (!flags.has(arg)) return null;
|
||||
// A repeated flag is a malformed argv, not an idempotent spelling. The
|
||||
// other command parsers already apply this rule to --json as well.
|
||||
if (seen.has(arg)) return null;
|
||||
seen.add(arg);
|
||||
if (arg === "--yes") {
|
||||
confirmed = true;
|
||||
continue;
|
||||
}
|
||||
if (arg === "--confirm") {
|
||||
const value = args[index + 1];
|
||||
if (value === undefined || value.startsWith("--")) return null;
|
||||
if (confirmPath !== undefined) return null;
|
||||
confirmPath = value;
|
||||
index += 1;
|
||||
continue;
|
||||
}
|
||||
if (arg === "--json") continue;
|
||||
if (arg === "--no-recursive") {
|
||||
recursive = false;
|
||||
continue;
|
||||
}
|
||||
if (arg === "--full") {
|
||||
full = true;
|
||||
continue;
|
||||
}
|
||||
if (arg === "--all") {
|
||||
all = true;
|
||||
if (switchFlags.includes(arg)) {
|
||||
if (switches.has(arg)) return null;
|
||||
switches.add(arg);
|
||||
continue;
|
||||
}
|
||||
if (!valueFlags.includes(arg) || values.has(arg)) return null;
|
||||
const value = args[index + 1];
|
||||
if (value === undefined || value.startsWith("--")) return null;
|
||||
if (seenValues.has(arg)) return null;
|
||||
seenValues.add(arg);
|
||||
values.set(arg, value);
|
||||
index += 1;
|
||||
if (arg === "--root") root = value;
|
||||
else if (arg === "--source") source = value;
|
||||
else target = value;
|
||||
}
|
||||
return { values, switches };
|
||||
}
|
||||
|
||||
if (name === "path") {
|
||||
if (confirmPath !== undefined) return null;
|
||||
if (target !== undefined) return null;
|
||||
if (root === undefined) return null;
|
||||
if (source === undefined && args.includes("--source")) return null;
|
||||
if (seen.has("--full") || seen.has("--all")) return null;
|
||||
if (args[0] !== "add" && args[0] !== "remove") return null;
|
||||
if (
|
||||
args[0] === "remove" &&
|
||||
(seen.has("--source") || seen.has("--no-recursive"))
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const SOURCE_ID = /^[a-z0-9][a-z0-9-]{0,127}$/;
|
||||
|
||||
/** `scan-roots add --root <path> [--source <id>] [--no-recursive]` and
|
||||
* `scan-roots remove --root <path>` (`list` is answered separately). */
|
||||
export function parseScanRootsCommand(
|
||||
args: readonly string[],
|
||||
): ScanRootsCommand | null {
|
||||
const verb = args[0];
|
||||
if (verb === "add") {
|
||||
const flags = parseFlags(
|
||||
args.slice(1),
|
||||
["--root", "--source"],
|
||||
["--no-recursive", "--yes", "--json"],
|
||||
);
|
||||
const root = flags?.values.get("--root");
|
||||
if (!flags || root === undefined) return null;
|
||||
const source = flags.values.get("--source");
|
||||
if (source !== undefined && !SOURCE_ID.test(source)) return null;
|
||||
return {
|
||||
plan:
|
||||
args[0] === "add"
|
||||
? planAddRoot(root, source, recursive)
|
||||
: planRemoveRoot(root),
|
||||
confirmed,
|
||||
kind: "add",
|
||||
root,
|
||||
...(source === undefined ? {} : { source }),
|
||||
recursive: !flags.switches.has("--no-recursive"),
|
||||
};
|
||||
}
|
||||
if (name === "scan") {
|
||||
if (confirmPath !== undefined) return null;
|
||||
if (target !== undefined) return null;
|
||||
if (seen.has("--no-recursive")) return null;
|
||||
if (args[0] !== undefined && !args[0].startsWith("--")) return null;
|
||||
// `--all` means "every source Runtime actually found", so naming one too
|
||||
// would make the command mean two different things.
|
||||
if (all && source !== undefined) return null;
|
||||
if (all) return { plan: planScanAll(full), confirmed };
|
||||
// A source is the thing the Runtime scan endpoint addresses. A bare root is
|
||||
// still accepted for planning (it means "all registered roots") but cannot
|
||||
// be applied, because there is no endpoint that scans a raw path — the CLI
|
||||
// must not pretend it can.
|
||||
if (source === undefined) return { plan: planScan(root, full), confirmed };
|
||||
return { plan: planScanSource(source, full), confirmed };
|
||||
if (verb === "remove") {
|
||||
const flags = parseFlags(args.slice(1), ["--root"], ["--yes", "--json"]);
|
||||
const root = flags?.values.get("--root");
|
||||
return flags && root !== undefined ? { kind: "remove", root } : null;
|
||||
}
|
||||
if (name === "backup") {
|
||||
const verb = args[0];
|
||||
if (verb !== "create" && verb !== "restore") return null;
|
||||
if (seen.has("--full") || seen.has("--all") || seen.has("--no-recursive")) {
|
||||
return null;
|
||||
}
|
||||
if (verb === "restore") {
|
||||
if (source === undefined) return null;
|
||||
if (target !== undefined) return null;
|
||||
return {
|
||||
plan: planRestore(source),
|
||||
confirmed,
|
||||
...(confirmPath === undefined ? {} : { confirmPath }),
|
||||
};
|
||||
}
|
||||
if (confirmPath !== undefined) return null;
|
||||
if (source !== undefined) return null;
|
||||
if (target === undefined) return null;
|
||||
return null;
|
||||
}
|
||||
|
||||
/** `scan --source <id>`. A scan names a source: the Runtime has no endpoint
|
||||
* that scans a bare path, so a missing `--source` is a usage error rather
|
||||
* than a preview of something that can never run. */
|
||||
export function parseScanCommand(args: readonly string[]): ScanCommand | null {
|
||||
const flags = parseFlags(args, ["--source"], ["--all", "--yes", "--json"]);
|
||||
if (!flags) return null;
|
||||
const source = flags.values.get("--source");
|
||||
if (flags.switches.has("--all")) {
|
||||
return source === undefined ? { kind: "all" } : null;
|
||||
}
|
||||
return source !== undefined && SOURCE_ID.test(source)
|
||||
? { kind: "source", sourceId: source }
|
||||
: null;
|
||||
}
|
||||
|
||||
/** `backup create --target <path>` and
|
||||
* `backup restore --source <path> [--yes]`. */
|
||||
export function parseBackupCommand(
|
||||
args: readonly string[],
|
||||
): BackupCommand | null {
|
||||
const verb = args[0];
|
||||
if (verb === "create") {
|
||||
const flags = parseFlags(args.slice(1), ["--target"], ["--yes", "--json"]);
|
||||
const target = flags?.values.get("--target");
|
||||
return flags && target !== undefined ? { verb: "create", target } : null;
|
||||
}
|
||||
if (verb === "restore") {
|
||||
const flags = parseFlags(
|
||||
args.slice(1),
|
||||
["--source", "--confirm"],
|
||||
["--yes", "--json"],
|
||||
);
|
||||
const source = flags?.values.get("--source");
|
||||
if (!flags || source === undefined) return null;
|
||||
const confirmPath = flags.values.get("--confirm");
|
||||
return {
|
||||
plan: planBackup(target),
|
||||
confirmed,
|
||||
verb: "restore",
|
||||
source,
|
||||
confirmed: flags.switches.has("--yes"),
|
||||
...(confirmPath === undefined ? {} : { confirmPath }),
|
||||
};
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Whether `--confirm <path>` names the same restore source.
|
||||
/** Whether a legacy `--confirm <path>` names the same restore source.
|
||||
*
|
||||
* The comparison is lexical on purpose: the CLI must not resolve symlinks or
|
||||
* touch the filesystem before Core has validated the managed source. A
|
||||
* different string is a different confirmation, so a typo or a path escape
|
||||
* can never be applied as a side effect of `--yes`.
|
||||
* different string is a different confirmation.
|
||||
*/
|
||||
export function restoreConfirmationMatches(
|
||||
source: string,
|
||||
confirmPath: string | undefined,
|
||||
): boolean {
|
||||
return confirmPath === source;
|
||||
return confirmPath === undefined || confirmPath === source;
|
||||
}
|
||||
|
||||
export interface WriteSubmitCommand {
|
||||
@@ -272,41 +177,33 @@ export interface WriteStatusCommand {
|
||||
|
||||
export type WriteCommand = WriteSubmitCommand | WriteStatusCommand;
|
||||
|
||||
/** Parse SDK-native `write submit|status`. */
|
||||
/** Parse `write submit ...` and `write status <request-id>` (or `--id`). */
|
||||
export function parseWriteCommand(
|
||||
args: readonly string[],
|
||||
): WriteCommand | null {
|
||||
if (args.filter((item) => item === "--json").length > 1) return null;
|
||||
const meaningful = args.filter((item) => item !== "--json");
|
||||
if (meaningful[0] === "status") {
|
||||
const status = parseTaskCommand(meaningful.slice(1));
|
||||
const rest = meaningful.slice(1);
|
||||
if (rest.length === 1 && !rest[0].startsWith("--")) {
|
||||
return { kind: "status", id: rest[0] };
|
||||
}
|
||||
const status = parseTaskCommand(rest);
|
||||
return status === null ? null : { kind: "status", id: status.id };
|
||||
}
|
||||
// `write --session ...` without `submit` stays accepted for older scripts.
|
||||
const input = meaningful[0] === "submit" ? meaningful.slice(1) : meaningful;
|
||||
let session: string | undefined;
|
||||
let source = "memhub-cli";
|
||||
let messagesFile: string | undefined;
|
||||
let requestId: string | undefined;
|
||||
let memoryAt: string | undefined;
|
||||
let priority: "live" | "normal" | undefined;
|
||||
|
||||
for (let index = 0; index < input.length; index += 1) {
|
||||
const arg = input[index];
|
||||
if (arg === "--live") {
|
||||
priority = "live";
|
||||
continue;
|
||||
}
|
||||
const value = input[index + 1];
|
||||
if (value === undefined || value.startsWith("--")) return null;
|
||||
index += 1;
|
||||
if (arg === "--session") session = value;
|
||||
else if (arg === "--source") source = value;
|
||||
else if (arg === "--messages-file") messagesFile = value;
|
||||
else if (arg === "--request-id") requestId = value;
|
||||
else if (arg === "--memory-at") memoryAt = value;
|
||||
else return null;
|
||||
}
|
||||
|
||||
const flags = parseFlags(
|
||||
input,
|
||||
["--session", "--source", "--messages-file", "--request-id", "--memory-at"],
|
||||
["--live"],
|
||||
);
|
||||
if (!flags) return null;
|
||||
const session = flags.values.get("--session");
|
||||
const source = flags.values.get("--source") ?? "memhub-cli";
|
||||
const messagesFile = flags.values.get("--messages-file");
|
||||
const requestId = flags.values.get("--request-id");
|
||||
const memoryAt = flags.values.get("--memory-at");
|
||||
if (session === undefined || messagesFile === undefined || !source)
|
||||
return null;
|
||||
if (
|
||||
@@ -323,39 +220,23 @@ export function parseWriteCommand(
|
||||
...(requestId === undefined ? {} : { request_id: requestId }),
|
||||
...(memoryAt === undefined ? {} : { memory_at: memoryAt }),
|
||||
};
|
||||
return priority === undefined ? command : { ...command, priority };
|
||||
return flags.switches.has("--live")
|
||||
? { ...command, priority: "live" }
|
||||
: command;
|
||||
}
|
||||
|
||||
export interface TaskCommand {
|
||||
readonly id: string;
|
||||
}
|
||||
|
||||
/** `--id <request-id>`: the legacy `task` spelling of `write status`. */
|
||||
export function parseTaskCommand(args: readonly string[]): TaskCommand | null {
|
||||
if (args.filter((item) => item === "--json").length > 1) return null;
|
||||
let id: string | undefined;
|
||||
for (let index = 0; index < args.length; index += 1) {
|
||||
const arg = args[index];
|
||||
if (arg === "--json") continue;
|
||||
if (arg !== "--id") return null;
|
||||
const value = args[index + 1];
|
||||
if (value === undefined || value.startsWith("--")) return null;
|
||||
index += 1;
|
||||
id = value;
|
||||
}
|
||||
const flags = parseFlags(
|
||||
args.filter((item) => item !== "--json"),
|
||||
["--id"],
|
||||
[],
|
||||
);
|
||||
const id = flags?.values.get("--id");
|
||||
return id === undefined ? null : { id };
|
||||
}
|
||||
|
||||
/** The human form. Kept separate from the JSON form on purpose. */
|
||||
export function describe(plan: Plan): string {
|
||||
const line = `${plan.action}: ${plan.target}`;
|
||||
return plan.detail ? `${line} (${plan.detail})` : line;
|
||||
}
|
||||
|
||||
/** The machine form: one document, nothing else on stdout. */
|
||||
export function asPlanDocument(plan: Plan): {
|
||||
schema_version: 1;
|
||||
plan: Plan;
|
||||
applied: boolean;
|
||||
} {
|
||||
return { schema_version: 1, plan, applied: false };
|
||||
}
|
||||
@@ -184,8 +184,16 @@ const CORE = new Map<string, { status: number; data: unknown }>([
|
||||
},
|
||||
],
|
||||
[
|
||||
"/api/v1/home-summary",
|
||||
{ status: 200, data: { memory_count: 42, latest: null, observed_at: "T" } },
|
||||
"/api/v2/home-summary",
|
||||
{
|
||||
status: 200,
|
||||
data: {
|
||||
schema_version: 2,
|
||||
memory_count: 42,
|
||||
latest: null,
|
||||
observed_at: "T",
|
||||
},
|
||||
},
|
||||
],
|
||||
]);
|
||||
|
||||
@@ -194,9 +202,7 @@ const READ_ONLY_PATHS = [
|
||||
"/api/v1/live-sync",
|
||||
"/api/v1/memory-processing",
|
||||
"/api/v1/model-profiles",
|
||||
"/api/v1/home-summary",
|
||||
"/api/v1/memory-library",
|
||||
"/api/v1/memories/",
|
||||
"/api/v2/home-summary",
|
||||
"/api/v2/memory-library",
|
||||
"/api/v2/memories/",
|
||||
"/api/v2/user-profile",
|
||||
@@ -691,6 +697,66 @@ describe("CLI-21 memory reads", () => {
|
||||
expect(report.memories[1].content.length).toBeLessThan(400);
|
||||
});
|
||||
|
||||
it("lists working and coding memories with their scene and kind", async () => {
|
||||
transport.get.mockResolvedValue({
|
||||
status: 200,
|
||||
data: {
|
||||
total: 2,
|
||||
offset: 0,
|
||||
limit: 20,
|
||||
totals: { chat: 1, working: 1, coding: 0 },
|
||||
window_id: null,
|
||||
window_reset: false,
|
||||
memories: [
|
||||
memory("m1", "对话记忆"),
|
||||
{
|
||||
...memory("e1", "修复测试\n固定随机种子"),
|
||||
layer: "experience",
|
||||
agent_id: "",
|
||||
owner: null,
|
||||
source_raw_ids: [],
|
||||
scene: "working",
|
||||
kind: "experience",
|
||||
variant: "working",
|
||||
task: "修复测试",
|
||||
solution: "固定随机种子",
|
||||
created_at: 1_790_000_000,
|
||||
updated_at: 1_790_000_100,
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["memories", "list", "--json"], {})).toBe(0);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
const report = JSON.parse(streams.stdout());
|
||||
expect(report.page).toMatchObject({ state: "ok", total: 2, returned: 2 });
|
||||
expect(report.memories[0]).toMatchObject({
|
||||
memory_id: "m1",
|
||||
scene: "chat",
|
||||
kind: "chat",
|
||||
variant: null,
|
||||
});
|
||||
expect(report.memories[1]).toMatchObject({
|
||||
memory_id: "e1",
|
||||
scene: "working",
|
||||
kind: "experience",
|
||||
variant: "working",
|
||||
layer: "experience",
|
||||
});
|
||||
|
||||
const text = captureStreams();
|
||||
try {
|
||||
expect(await run(["memories", "list"], {})).toBe(0);
|
||||
} finally {
|
||||
text.restore();
|
||||
}
|
||||
expect(text.stdout()).toContain("- e1 [working] active/experience");
|
||||
});
|
||||
|
||||
it("passes the requested filters through as query parameters", async () => {
|
||||
await run(
|
||||
[
|
||||
@@ -812,11 +878,14 @@ describe("CLI-21 memory reads", () => {
|
||||
layer: "L2_FACT",
|
||||
status: "active",
|
||||
});
|
||||
expect(report.evidence).toEqual([]);
|
||||
expect(report.history).toEqual([]);
|
||||
// Schema 2, like add/update/delete; the v1-era always-empty
|
||||
// `evidence`/`history` fields are gone.
|
||||
expect(report.schema_version).toBe(2);
|
||||
expect(report).not.toHaveProperty("evidence");
|
||||
expect(report).not.toHaveProperty("history");
|
||||
});
|
||||
|
||||
// Core registers /api/v1/memories/graph before /{memory_id}, and that static
|
||||
// Core registers /api/v2/memories/graph before /{memory_id}, and that static
|
||||
// handler initializes the HY bridge. A memory id that collides with a
|
||||
// reserved segment must never become a request.
|
||||
it("refuses a memory id that collides with a static Core route", async () => {
|
||||
@@ -830,18 +899,15 @@ describe("CLI-21 memory reads", () => {
|
||||
expect(streams.stdout()).toBe("");
|
||||
}
|
||||
expect(transport.get).not.toHaveBeenCalled();
|
||||
expect(isReadOnlyPath("/api/v1/memories/graph")).toBe(false);
|
||||
expect(isReadOnlyPath("/api/v1/memories/ask")).toBe(false);
|
||||
expect(isReadOnlyPath("/api/v1/memories/m1")).toBe(true);
|
||||
expect(isReadOnlyPath("/api/v2/memory-library")).toBe(true);
|
||||
expect(isReadOnlyPath("/api/v2/memories/graph")).toBe(false);
|
||||
expect(isReadOnlyPath("/api/v2/memories/ask")).toBe(false);
|
||||
expect(isReadOnlyPath("/api/v2/memories/m1")).toBe(true);
|
||||
expect(isReadOnlyPath("/api/v2/memory-library")).toBe(true);
|
||||
expect(isReadOnlyPath("/api/v2/home-summary")).toBe(true);
|
||||
// The retired v1 data plane is no longer a CLI read.
|
||||
expect(isReadOnlyPath("/api/v1/memories/m1")).toBe(false);
|
||||
expect(isReadOnlyPath("/api/v1/home-summary")).toBe(false);
|
||||
expect(isReadOnlyPath("/api/v1/model-content-permission")).toBe(false);
|
||||
expect(
|
||||
isReadOnlyPath(
|
||||
"/api/v1/memory-mutations/123e4567-e89b-12d3-a456-426614174000",
|
||||
),
|
||||
).toBe(true);
|
||||
expect(isReadOnlyPath("/api/v1/memory-mutations/../sources")).toBe(false);
|
||||
});
|
||||
|
||||
it("refuses a memory id that would escape the memories path", async () => {
|
||||
@@ -861,6 +927,26 @@ describe("CLI-21 memory reads", () => {
|
||||
// was cast to the expected type, so a wrong record, an unavailable excerpt, an
|
||||
// oversized page and a non-string model field all reached stdout as success.
|
||||
describe("malformed success responses", () => {
|
||||
it("treats a v1-shaped home summary as invalid, not as a memory count", async () => {
|
||||
transport.get.mockImplementation(async (url: string) =>
|
||||
url === "/api/v2/home-summary"
|
||||
? {
|
||||
status: 200,
|
||||
data: { memory_count: 7, latest: null, observed_at: "T" },
|
||||
}
|
||||
: (CORE.get(url) ?? { status: 404, data: { detail: "not_found" } }),
|
||||
);
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
await run(["status", "--json"], {});
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(JSON.parse(streams.stdout()).sections.memories).toMatchObject({
|
||||
state: "invalid",
|
||||
});
|
||||
});
|
||||
|
||||
it("refuses a detail that is not the memory that was asked for", async () => {
|
||||
transport.get.mockResolvedValue({
|
||||
status: 200,
|
||||
@@ -1054,14 +1140,16 @@ describe("backup create target refusal is rejected, not usage_error", () => {
|
||||
let code: number;
|
||||
try {
|
||||
code = await run(
|
||||
["backup", "create", "--target", "/tmp/outside.db", "--yes", "--json"],
|
||||
["backup", "create", "--target", "/tmp/outside.db", "--json"],
|
||||
{},
|
||||
);
|
||||
} finally {
|
||||
streams.restore();
|
||||
}
|
||||
expect(code!).toBe(1);
|
||||
const document = JSON.parse(streams.stderr());
|
||||
// A result, on stdout like every --json document.
|
||||
expect(streams.stderr()).toBe("");
|
||||
const document = JSON.parse(streams.stdout());
|
||||
expect(document.status).not.toBe("usage_error");
|
||||
expect(document).toMatchObject({
|
||||
command: "backup",
|
||||
|
||||
@@ -315,6 +315,40 @@ describe("macOS service lifecycle", () => {
|
||||
expect(deps.verifyRegistrationSnapshot).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it("does not re-bootstrap a previous registration that may serve a legacy home", async () => {
|
||||
const f = fixture();
|
||||
const oldPath = "/Users/example/Library/LaunchAgents/old-runtime.plist";
|
||||
const runCommand = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(undefined)
|
||||
.mockRejectedValueOnce(new Error("bootstrap failed"));
|
||||
const deps = dependencies({
|
||||
inspectRegistration: vi.fn(async () => ({
|
||||
kind: "managed" as const,
|
||||
plistPath: oldPath,
|
||||
content: "old managed plist",
|
||||
})),
|
||||
runCommand,
|
||||
});
|
||||
|
||||
const prepared = await prepareServiceInstall(f.layout, f.options, deps);
|
||||
await expect(
|
||||
applyPreparedServiceInstall(
|
||||
{ ...prepared, restorePreviousOnFailure: false },
|
||||
deps,
|
||||
),
|
||||
).rejects.toThrow("runtime_service_replacement_failed");
|
||||
|
||||
// bootout + the failed bootstrap only; the old plist is not bootstrapped.
|
||||
expect(runCommand).toHaveBeenCalledTimes(2);
|
||||
expect(runCommand).not.toHaveBeenCalledWith([
|
||||
"launchctl",
|
||||
"bootstrap",
|
||||
"gui/501",
|
||||
oldPath,
|
||||
]);
|
||||
});
|
||||
|
||||
it("removes the replacement plist and restores the previous registration", async () => {
|
||||
const f = fixture();
|
||||
const oldPath = "/Users/example/Library/LaunchAgents/old-runtime.plist";
|
||||
|
||||
@@ -72,6 +72,13 @@ export type PreparedServiceInstall = {
|
||||
mode: "direct" | "bootstrap-existing" | "restart" | "replace";
|
||||
previous?: ManagedSnapshot;
|
||||
previouslyEnabled: boolean;
|
||||
/**
|
||||
* Re-bootstrap the previous (possibly older) LaunchAgent when the new one
|
||||
* fails to bootstrap. `install` turns this off once the home has been handed
|
||||
* to this version's SDK migration: an older Runtime has no migration gate and
|
||||
* would serve a home that is not on SDK authority.
|
||||
*/
|
||||
restorePreviousOnFailure?: boolean;
|
||||
};
|
||||
|
||||
const defaultDependencies: ServiceLifecycleDependencies = {
|
||||
@@ -191,7 +198,8 @@ export async function applyPreparedServiceInstall(
|
||||
if (!previous) throw new Error("runtime_service_upgrade_state_invalid");
|
||||
await verifyPreparedRegistration(prepared, dependencies);
|
||||
const domain = launchdDomain(options.uid);
|
||||
let replacementFailure: "upgrade" | "rollback" | null = null;
|
||||
let replacementFailure: "upgrade" | "rollback" | "not-restored" | null =
|
||||
null;
|
||||
const replaceCommand: ServiceCommandRunner = async (command) => {
|
||||
if (command[0] !== "launchctl" || command[1] !== "bootstrap") {
|
||||
await dependencies.runCommand(command);
|
||||
@@ -208,6 +216,10 @@ export async function applyPreparedServiceInstall(
|
||||
await dependencies.runCommand(command);
|
||||
} catch {
|
||||
replacementFailure = "upgrade";
|
||||
if (prepared.restorePreviousOnFailure === false) {
|
||||
replacementFailure = "not-restored";
|
||||
throw new Error("runtime_service_replacement_failed");
|
||||
}
|
||||
try {
|
||||
await dependencies.verifyRegistrationSnapshot(previous);
|
||||
await dependencies.runCommand([
|
||||
@@ -234,6 +246,9 @@ export async function applyPreparedServiceInstall(
|
||||
if (replacementFailure === "upgrade") {
|
||||
throw new Error("runtime_service_managed_upgrade_failed");
|
||||
}
|
||||
if (replacementFailure === "not-restored") {
|
||||
throw new Error("runtime_service_replacement_failed");
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
} catch (error) {
|
||||
|
||||
@@ -7,12 +7,10 @@ import type {
|
||||
} from "@one-memory/contracts";
|
||||
import type { CoreReader, ReadOutcome } from "./core-client.js";
|
||||
import {
|
||||
type ValidatedMemoryTask,
|
||||
validateImportIssues,
|
||||
validateImportJob,
|
||||
validateImportJobs,
|
||||
validateLiveSync,
|
||||
validateMemoryTasks,
|
||||
validateProcessing,
|
||||
validateSources,
|
||||
} from "./core-responses.js";
|
||||
@@ -70,11 +68,6 @@ export interface ProgressDocument {
|
||||
generation: {
|
||||
summary_state: ReadState;
|
||||
summary: Record<string, unknown> | null;
|
||||
tasks_state: ReadState;
|
||||
tasks: Array<Record<string, unknown>>;
|
||||
returned: number;
|
||||
validated: number;
|
||||
output_truncated: boolean;
|
||||
};
|
||||
}
|
||||
|
||||
@@ -134,12 +127,13 @@ export async function readProgress(
|
||||
const importsPath = options.jobId
|
||||
? `/api/v1/import-jobs/${options.jobId}`
|
||||
: "/api/v1/import-jobs";
|
||||
const [importsRead, sourcesRead, processingRead, tasksRead, issuesRead] =
|
||||
// Generation progress comes from `memory-processing` alone: under SDK
|
||||
// authority the per-task list (`/api/v1/memory-tasks`) is retired and empty.
|
||||
const [importsRead, sourcesRead, processingRead, issuesRead] =
|
||||
await Promise.all([
|
||||
reader<unknown>(importsPath),
|
||||
reader<unknown>("/api/v1/sources"),
|
||||
reader<unknown>("/api/v1/memory-processing"),
|
||||
reader<unknown>("/api/v1/memory-tasks"),
|
||||
options.jobId && options.issues
|
||||
? reader<unknown>(`/api/v1/import-jobs/${options.jobId}/issues`)
|
||||
: Promise.resolve(null),
|
||||
@@ -153,7 +147,6 @@ export async function readProgress(
|
||||
: validated(importsRead, validateImportJobs);
|
||||
const processing = validated(processingRead, validateProcessing);
|
||||
const sources = validated(sourcesRead, validateSources);
|
||||
const tasks = validated(tasksRead, validateMemoryTasks);
|
||||
const issues =
|
||||
options.jobId && options.issues && issuesRead
|
||||
? validated(issuesRead, (value) =>
|
||||
@@ -162,25 +155,19 @@ export async function readProgress(
|
||||
: null;
|
||||
|
||||
const jobs = imports.state === "ok" ? imports.data : [];
|
||||
const taskItems = tasks.state === "ok" ? tasks.data : [];
|
||||
const issueItems = issues?.state === "ok" ? issues.data : [];
|
||||
const importState = stateOnly(imports);
|
||||
const summaryState = stateOnly(processing);
|
||||
const tasksState = stateOnly(tasks);
|
||||
|
||||
const hasReadFailure =
|
||||
importState.state !== "ok" ||
|
||||
sources.state !== "ok" ||
|
||||
summaryState.state !== "ok" ||
|
||||
tasksState.state !== "ok" ||
|
||||
(issues !== null && issues.state !== "ok");
|
||||
const hasProblem =
|
||||
jobs.some((job) =>
|
||||
["paused", "interrupted", "failed"].includes(job.status),
|
||||
) ||
|
||||
taskItems.some((task) =>
|
||||
["waiting_for_model", "failed"].includes(task.status),
|
||||
) ||
|
||||
(processing.state === "ok" &&
|
||||
(processing.data.failed > 0 ||
|
||||
processing.data.waiting_for_model > 0 ||
|
||||
@@ -188,7 +175,6 @@ export async function readProgress(
|
||||
issueItems.length > 0;
|
||||
const hasActive =
|
||||
jobs.some((job) => ["pending", "running"].includes(job.status)) ||
|
||||
taskItems.some((task) => ["pending", "running"].includes(task.status)) ||
|
||||
(processing.state === "ok" &&
|
||||
(processing.data.pending > 0 || processing.data.running > 0));
|
||||
|
||||
@@ -229,11 +215,6 @@ export async function readProgress(
|
||||
summary_state: summaryState,
|
||||
summary:
|
||||
processing.state === "ok" ? projectProcessing(processing.data) : null,
|
||||
tasks_state: tasksState,
|
||||
tasks: taskItems.slice(0, options.limit).map(projectTask),
|
||||
returned: Math.min(taskItems.length, options.limit),
|
||||
validated: taskItems.length,
|
||||
output_truncated: taskItems.length > options.limit,
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -289,10 +270,7 @@ export function renderProgressText(document: ProgressDocument): string {
|
||||
? `记忆整理:待处理 ${document.generation.summary.pending} · 进行中 ${document.generation.summary.running} · 等待模型 ${document.generation.summary.waiting_for_model} · 失败 ${document.generation.summary.failed}`
|
||||
: "记忆整理:暂时无法读取",
|
||||
);
|
||||
if (
|
||||
document.imports.output_truncated ||
|
||||
document.generation.output_truncated
|
||||
) {
|
||||
if (document.imports.output_truncated) {
|
||||
lines.push("这里只显示汇总;完整记录请使用 --json。");
|
||||
}
|
||||
lines.push("");
|
||||
@@ -353,11 +331,6 @@ export function unreadProgress(
|
||||
generation: {
|
||||
summary_state: failed,
|
||||
summary: null,
|
||||
tasks_state: failed,
|
||||
tasks: [],
|
||||
returned: 0,
|
||||
validated: 0,
|
||||
output_truncated: false,
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -368,7 +341,11 @@ function validated<T>(
|
||||
):
|
||||
| { state: "ok"; data: T }
|
||||
| { state: Exclude<ReadState["state"], "ok">; code: string } {
|
||||
if (outcome.state !== "ok") return outcome;
|
||||
if (outcome.state !== "ok") {
|
||||
// A Core refusal of the request itself reads as an invalid answer here.
|
||||
const state = outcome.state === "rejected" ? "invalid" : outcome.state;
|
||||
return { state, code: outcome.code };
|
||||
}
|
||||
const data = validate(outcome.data);
|
||||
return data === null
|
||||
? { state: "invalid", code: "response_invalid" }
|
||||
@@ -465,7 +442,3 @@ function projectProcessing(
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
function projectTask(task: ValidatedMemoryTask): Record<string, unknown> {
|
||||
return { ...task };
|
||||
}
|
||||
@@ -129,18 +129,6 @@ const RESPONSES = new Map<string, { status: number; data: unknown }>([
|
||||
},
|
||||
},
|
||||
],
|
||||
[
|
||||
"/api/v1/memory-tasks",
|
||||
{
|
||||
status: 200,
|
||||
data: [
|
||||
memoryTask("task-pending", "pending"),
|
||||
memoryTask("task-running", "running"),
|
||||
memoryTask("task-model", "waiting_for_model", "model_missing"),
|
||||
memoryTask("task-failed", "failed", "provider_unavailable"),
|
||||
],
|
||||
},
|
||||
],
|
||||
[
|
||||
"/api/v1/import-jobs/job-running",
|
||||
{ status: 200, data: importJob("job-running", "running") },
|
||||
@@ -282,9 +270,9 @@ describe("CLI-19 progress and issue reads", () => {
|
||||
imported_messages: 20,
|
||||
totals_state: { state: "ok" },
|
||||
});
|
||||
expect(
|
||||
report.generation.tasks.map((task: { status: string }) => task.status),
|
||||
).toEqual(["pending", "running", "waiting_for_model", "failed"]);
|
||||
// The retired per-task list is no longer read or reported.
|
||||
expect(report.generation).not.toHaveProperty("tasks");
|
||||
expect(requestedPaths()).not.toContain("/api/v1/memory-tasks");
|
||||
expect(streams.stdout()).not.toMatch(/percent|percentage|%/i);
|
||||
});
|
||||
|
||||
@@ -374,7 +362,7 @@ describe("CLI-19 progress and issue reads", () => {
|
||||
expect(streams.stdout()).not.toContain("SYNTHETIC_FOREIGN_ISSUE");
|
||||
});
|
||||
|
||||
it("bounds each task list and emits an explicit truncation receipt", async () => {
|
||||
it("bounds the import job list and emits an explicit truncation receipt", async () => {
|
||||
const streams = captureStreams();
|
||||
try {
|
||||
expect(await run(["progress", "--limit", "1", "--json"], {})).toBe(1);
|
||||
@@ -388,12 +376,6 @@ describe("CLI-19 progress and issue reads", () => {
|
||||
validated: 3,
|
||||
output_truncated: true,
|
||||
});
|
||||
expect(report.generation.tasks).toHaveLength(1);
|
||||
expect(report.generation).toMatchObject({
|
||||
returned: 1,
|
||||
validated: 4,
|
||||
output_truncated: true,
|
||||
});
|
||||
});
|
||||
|
||||
it("streams bounded progress snapshots as serial NDJSON", async () => {
|
||||
@@ -428,7 +410,6 @@ describe("CLI-19 progress and issue reads", () => {
|
||||
"/api/v1/import-jobs",
|
||||
"/api/v1/sources",
|
||||
"/api/v1/memory-processing",
|
||||
"/api/v1/memory-tasks",
|
||||
]) {
|
||||
expect(requestedPaths().filter((item) => item === path)).toHaveLength(2);
|
||||
}
|
||||
@@ -500,7 +481,7 @@ describe("CLI-19 progress and issue reads", () => {
|
||||
it("preserves denied, unavailable and malformed sections in one partial report", async () => {
|
||||
transport.get.mockImplementation(async (url: string) => {
|
||||
if (url === "/api/v1/import-jobs") return { status: 503, data: {} };
|
||||
if (url === "/api/v1/memory-tasks") return { status: 403, data: {} };
|
||||
if (url === "/api/v1/sources") return { status: 403, data: {} };
|
||||
if (url === "/api/v1/memory-processing") {
|
||||
return { status: 200, data: { pending: { secret: "nested" } } };
|
||||
}
|
||||
@@ -521,7 +502,7 @@ describe("CLI-19 progress and issue reads", () => {
|
||||
state: "invalid",
|
||||
code: "response_invalid",
|
||||
});
|
||||
expect(report.generation.tasks_state).toEqual({
|
||||
expect(report.imports.totals_state).toEqual({
|
||||
state: "denied",
|
||||
code: "http_403",
|
||||
});
|
||||
@@ -574,7 +555,6 @@ describe("CLI-19 progress and issue reads", () => {
|
||||
"/api/v1/live-sync",
|
||||
"/api/v1/import-jobs",
|
||||
"/api/v1/memory-processing",
|
||||
"/api/v1/memory-tasks",
|
||||
].some((allowed) => path === allowed || path.startsWith(`${allowed}/`)),
|
||||
),
|
||||
).toBe(true);
|
||||
@@ -634,28 +614,6 @@ function importJob(
|
||||
};
|
||||
}
|
||||
|
||||
function memoryTask(
|
||||
taskId: string,
|
||||
status: string,
|
||||
error: string | null = null,
|
||||
) {
|
||||
return {
|
||||
task_id: taskId,
|
||||
request_id: `request-${taskId}`,
|
||||
user_id: "local",
|
||||
kind: "extract",
|
||||
input_ref: "synthetic",
|
||||
status,
|
||||
profile_id: "profile-1",
|
||||
profile_revision: 1,
|
||||
profile_snapshot: {},
|
||||
attempts: status === "failed" ? 2 : 0,
|
||||
last_error_code: error,
|
||||
created_at: "T1",
|
||||
updated_at: "T2",
|
||||
};
|
||||
}
|
||||
|
||||
function requestedPaths(): string[] {
|
||||
return (transport.get.mock.calls as [string][]).map(([path]) => path);
|
||||
}
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { renderStatusText, type StatusDocument } from "./status-report.js";
|
||||
import { composeDoctorReport, renderDoctorText } from "./doctor-report.js";
|
||||
import {
|
||||
composeStatusReport,
|
||||
renderStatusText,
|
||||
type StatusDocument,
|
||||
} from "./status-report.js";
|
||||
|
||||
describe("status default output", () => {
|
||||
it("formats blocked task objects recursively instead of printing object coercions", () => {
|
||||
@@ -35,3 +40,40 @@ describe("status default output", () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("status and doctor on an unfinished upgrade migration", () => {
|
||||
it("names the migration log and the issues page", async () => {
|
||||
const report = await composeStatusReport({
|
||||
dataDir: "/tmp/isolated/.one-memory",
|
||||
runtime: { state: "stopped", code: "sdk_migration_required" },
|
||||
background: null,
|
||||
reader: null,
|
||||
});
|
||||
|
||||
expect(report.runtime.log_file).toBe(
|
||||
"/tmp/isolated/.one-memory/logs/sdk-migration.log",
|
||||
);
|
||||
expect(report.runtime.issues_url).toBe(
|
||||
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new",
|
||||
);
|
||||
const text = renderStatusText(report, "MemHub");
|
||||
expect(text).toContain("sdk_migration_required");
|
||||
expect(text).toContain("/tmp/isolated/.one-memory/logs/sdk-migration.log");
|
||||
|
||||
const doctor = renderDoctorText(composeDoctorReport(report, null));
|
||||
expect(doctor).toContain(
|
||||
"/tmp/isolated/.one-memory/logs/sdk-migration.log",
|
||||
);
|
||||
expect(doctor).not.toContain("帮助与诊断");
|
||||
});
|
||||
|
||||
it("adds no log path for an ordinary stopped runtime", async () => {
|
||||
const report = await composeStatusReport({
|
||||
dataDir: "/tmp/isolated/.one-memory",
|
||||
runtime: { state: "stopped", code: "runtime_not_running" },
|
||||
background: null,
|
||||
reader: null,
|
||||
});
|
||||
expect(report.runtime.log_file).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -1,5 +1,4 @@
|
||||
import type {
|
||||
HomeSummary,
|
||||
LiveSyncHealth,
|
||||
MemoryProcessingSummary,
|
||||
ModelProfile,
|
||||
@@ -9,8 +8,10 @@ import type {
|
||||
BackgroundHealth,
|
||||
RuntimeStatus,
|
||||
} from "@one-memory/runtime-control";
|
||||
import { sdkMigrationLogPath } from "@one-memory/runtime-control";
|
||||
import type { CoreReader, ReadOutcome } from "./core-client.js";
|
||||
import {
|
||||
type SdkHomeSummary,
|
||||
validateHomeSummary,
|
||||
validateLiveSync,
|
||||
validateModelProfiles,
|
||||
@@ -19,6 +20,9 @@ import {
|
||||
} from "./core-responses.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
const MIGRATION_ISSUES_URL =
|
||||
"https://github.com/Tencent-Hunyuan/Hy-Memory/issues/new";
|
||||
|
||||
export type SectionState =
|
||||
| "ok"
|
||||
| "denied"
|
||||
@@ -42,6 +46,9 @@ export interface StatusDocument {
|
||||
host?: string;
|
||||
port?: number;
|
||||
version?: string;
|
||||
/** Set only when an unfinished upgrade migration is why nothing serves. */
|
||||
log_file?: string;
|
||||
issues_url?: string;
|
||||
};
|
||||
bridge: {
|
||||
state: BackgroundHealth["state"] | "not_checked";
|
||||
@@ -102,7 +109,7 @@ export async function composeStatusReport(options: {
|
||||
reader<unknown>("/api/v1/live-sync"),
|
||||
reader<unknown>("/api/v1/memory-processing"),
|
||||
reader<unknown>("/api/v1/model-profiles"),
|
||||
reader<unknown>("/api/v1/home-summary"),
|
||||
reader<unknown>("/api/v2/home-summary"),
|
||||
]);
|
||||
sections.sources = section(sources, validateSources, describeSources);
|
||||
sections.sync = section(sync, validateLiveSync, describeSync);
|
||||
@@ -153,6 +160,12 @@ export async function composeStatusReport(options: {
|
||||
host: options.runtime.host,
|
||||
port: options.runtime.port,
|
||||
version: options.runtime.version,
|
||||
...(options.runtime.code.startsWith("sdk_migration_")
|
||||
? {
|
||||
log_file: sdkMigrationLogPath(options.dataDir),
|
||||
issues_url: MIGRATION_ISSUES_URL,
|
||||
}
|
||||
: {}),
|
||||
},
|
||||
bridge,
|
||||
sections,
|
||||
@@ -177,6 +190,13 @@ export function renderStatusText(
|
||||
): string {
|
||||
const lines = [
|
||||
`${heading}:${runtimeLine(report)}`,
|
||||
...(report.runtime.log_file
|
||||
? [
|
||||
`升级迁移没有完成(错误码:${report.runtime.code}),MemHub 不会启动。`,
|
||||
`请把迁移日志附在 GitHub 反馈里发给我们:${report.runtime.log_file}`,
|
||||
`反馈地址:${report.runtime.issues_url}`,
|
||||
]
|
||||
: []),
|
||||
`资料空间:${report.data_dir}`,
|
||||
`应用连接:${BRIDGE_LABELS[report.bridge.state]}`,
|
||||
];
|
||||
@@ -347,7 +367,7 @@ function describeModel(profiles: ModelProfile[]): Record<string, unknown> {
|
||||
};
|
||||
}
|
||||
|
||||
function describeMemories(summary: HomeSummary): Record<string, unknown> {
|
||||
function describeMemories(summary: SdkHomeSummary): Record<string, unknown> {
|
||||
return {
|
||||
memory_count: summary.memory_count,
|
||||
};
|
||||
|
||||
@@ -102,6 +102,10 @@ export function uiSessionFailure(code: string): { code: string } {
|
||||
|
||||
export function uiExitCode(result: UiResult): number {
|
||||
if (result.state === "opened") return 0;
|
||||
// A stopped Runtime is the same exit 3 every other command reports.
|
||||
if (result.state === "blocked" && result.code === "runtime_not_running") {
|
||||
return 3;
|
||||
}
|
||||
if (result.state === "blocked") return 4;
|
||||
return 1;
|
||||
}
|
||||
|
||||
@@ -1,479 +0,0 @@
|
||||
import { mkdtemp, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
import { run } from "./cli.js";
|
||||
|
||||
const transport = vi.hoisted(() => ({
|
||||
get: vi.fn(),
|
||||
post: vi.fn(),
|
||||
put: vi.fn(),
|
||||
patch: vi.fn(),
|
||||
delete: vi.fn(),
|
||||
}));
|
||||
vi.mock("axios", () => ({
|
||||
default: { create: vi.fn(() => transport), get: vi.fn() },
|
||||
}));
|
||||
|
||||
const runtime = vi.hoisted(() => ({
|
||||
inspectRuntimeStatus: vi.fn(),
|
||||
discoverRuntimeAttachment: vi.fn(),
|
||||
}));
|
||||
vi.mock("@one-memory/runtime-control", async (importOriginal) => ({
|
||||
...(await importOriginal<typeof import("@one-memory/runtime-control")>()),
|
||||
...runtime,
|
||||
}));
|
||||
|
||||
const EVENT = "11111111-1111-4111-8111-111111111111";
|
||||
const UPLOAD = "22222222-2222-4222-8222-222222222222";
|
||||
const NOTE = "33333333-3333-4333-8333-333333333333";
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
// clearAllMocks keeps prior mock implementations, so a mockResolvedValue
|
||||
// from an earlier test would leak into the next one and fake its response.
|
||||
transport.get.mockReset();
|
||||
transport.post.mockReset();
|
||||
transport.put.mockReset();
|
||||
transport.patch.mockReset();
|
||||
transport.delete.mockReset();
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "running",
|
||||
code: "runtime_verified",
|
||||
instanceId: "instance-1",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
version: "1.2.3",
|
||||
});
|
||||
runtime.discoverRuntimeAttachment.mockResolvedValue({
|
||||
instanceId: "instance-1",
|
||||
apiToken: "synthetic-runtime-token",
|
||||
host: "127.0.0.1",
|
||||
port: 8765,
|
||||
servicePid: 4242,
|
||||
health: {
|
||||
status: "ok",
|
||||
service: "one-memory-core",
|
||||
version: "1.2.3",
|
||||
instance_id: "instance-1",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
describe("calls clear", () => {
|
||||
it("reports runtime_stopped (exit 3) without attempting a write", async () => {
|
||||
runtime.inspectRuntimeStatus.mockResolvedValue({
|
||||
state: "stopped",
|
||||
code: "runtime_not_running",
|
||||
});
|
||||
const result = await capturedRun([
|
||||
"calls",
|
||||
"clear",
|
||||
"--all",
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(3);
|
||||
expect(JSON.parse(result.stdout).code).toBe("runtime_not_running");
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("previews without contacting Core", async () => {
|
||||
const result = await capturedRun([
|
||||
"calls",
|
||||
"clear",
|
||||
"--event",
|
||||
EVENT,
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
const document = JSON.parse(result.stdout);
|
||||
expect(document.state).toBe("preview");
|
||||
expect(document.plan.kind).toBe("call_events_clear");
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
expect(transport.delete).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("sends the exact Core body with --confirm", async () => {
|
||||
transport.post.mockResolvedValue({ status: 200, data: { removed: 1 } });
|
||||
const result = await capturedRun([
|
||||
"calls",
|
||||
"clear",
|
||||
"--event",
|
||||
EVENT,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
"/api/v1/call-events/clear",
|
||||
{ event_ids: [EVENT], user_confirmed: true },
|
||||
undefined,
|
||||
);
|
||||
expect(JSON.parse(result.stdout).state).toBe("committed");
|
||||
});
|
||||
|
||||
it("clears all without naming events", async () => {
|
||||
transport.post.mockResolvedValue({ status: 200, data: { removed: 4 } });
|
||||
const result = await capturedRun([
|
||||
"calls",
|
||||
"clear",
|
||||
"--all",
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
"/api/v1/call-events/clear-all",
|
||||
{ user_confirmed: true },
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
it("rejects a malformed event id without contacting Core", async () => {
|
||||
const result = await capturedRun([
|
||||
"calls",
|
||||
"clear",
|
||||
"--event",
|
||||
"not-a-uuid",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(2);
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("never reports committed for a malformed success body", async () => {
|
||||
transport.post.mockResolvedValue({ status: 200, data: { wrong: true } });
|
||||
const result = await capturedRun([
|
||||
"calls",
|
||||
"clear",
|
||||
"--event",
|
||||
EVENT,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(1);
|
||||
const document = JSON.parse(result.stdout);
|
||||
expect(document.state).toBe("unknown");
|
||||
expect(document.code).toBe("ui_write_receipt_invalid");
|
||||
});
|
||||
});
|
||||
|
||||
describe.skip("retired import-file", () => {
|
||||
it("never uploads without --confirm", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "om-import-"));
|
||||
const file = path.join(directory, "export.json");
|
||||
await writeFile(file, '{"conversations":[]}', { mode: 0o600 });
|
||||
const result = await capturedRun([
|
||||
"import-file",
|
||||
"upload",
|
||||
"--file",
|
||||
file,
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
const document = JSON.parse(result.stdout);
|
||||
expect(document.state).toBe("preview");
|
||||
expect(document.plan.operation_id).toBeDefined();
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("uploads the file with a deterministic operation id on confirm", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "om-import-"));
|
||||
const file = path.join(directory, "export.json");
|
||||
await writeFile(file, '{"conversations":[]}', { mode: 0o600 });
|
||||
transport.post.mockResolvedValue({
|
||||
status: 201,
|
||||
data: {
|
||||
schema_version: 1,
|
||||
operation_id: "ignored-here",
|
||||
state: "completed",
|
||||
upload_id: UPLOAD,
|
||||
upload_state: "staged",
|
||||
},
|
||||
});
|
||||
const preview = await capturedRun([
|
||||
"import-file",
|
||||
"upload",
|
||||
"--file",
|
||||
file,
|
||||
"--json",
|
||||
]);
|
||||
const plan = JSON.parse(preview.stdout).plan;
|
||||
transport.post.mockResolvedValue({
|
||||
status: 201,
|
||||
data: {
|
||||
schema_version: 1,
|
||||
operation_id: plan.operation_id,
|
||||
state: "completed",
|
||||
upload_id: UPLOAD,
|
||||
upload_state: "staged",
|
||||
},
|
||||
});
|
||||
const result = await capturedRun([
|
||||
"import-file",
|
||||
"upload",
|
||||
"--file",
|
||||
file,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(0);
|
||||
const url = transport.post.mock.calls.at(-1)?.[0] as string;
|
||||
expect(url).toContain("/api/v1/import-files?");
|
||||
expect(url).toContain(`operation_id=${plan.operation_id}`);
|
||||
expect(url).toContain("filename=export.json");
|
||||
expect(JSON.parse(result.stdout).state).toBe("committed");
|
||||
});
|
||||
|
||||
it("commits and discards only with explicit confirmation", async () => {
|
||||
const preview = await capturedRun([
|
||||
"import-file",
|
||||
"commit",
|
||||
UPLOAD,
|
||||
"--json",
|
||||
]);
|
||||
expect(preview.code).toBe(0);
|
||||
expect(JSON.parse(preview.stdout).state).toBe("preview");
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: {
|
||||
upload_id: UPLOAD,
|
||||
filename: "x.json",
|
||||
vendor: "generic",
|
||||
source_id: "generic-export",
|
||||
messages: 0,
|
||||
conversations: 0,
|
||||
human_messages: 0,
|
||||
agent_messages: 0,
|
||||
skipped_records: 0,
|
||||
ignored_files: 0,
|
||||
checksum: "a".repeat(64),
|
||||
created_at: "T",
|
||||
committed: true,
|
||||
},
|
||||
});
|
||||
const committed = await capturedRun([
|
||||
"import-file",
|
||||
"commit",
|
||||
UPLOAD,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
expect(committed.code).toBe(0);
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
`/api/v1/import-files/${UPLOAD}/commit`,
|
||||
{ confirmed: true },
|
||||
undefined,
|
||||
);
|
||||
|
||||
transport.delete.mockResolvedValue({ status: 204, data: "" });
|
||||
const discarded = await capturedRun([
|
||||
"import-file",
|
||||
"discard",
|
||||
UPLOAD,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
expect(discarded.code).toBe(0);
|
||||
expect(JSON.parse(discarded.stdout).state).toBe("committed");
|
||||
expect(transport.delete).toHaveBeenCalledWith(
|
||||
`/api/v1/import-files/${UPLOAD}`,
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
it("maps a Core 404 to a stable rejected code, not a fake success", async () => {
|
||||
transport.post.mockResolvedValue({
|
||||
status: 404,
|
||||
data: { detail: "import_file_not_found" },
|
||||
});
|
||||
const result = await capturedRun([
|
||||
"import-file",
|
||||
"commit",
|
||||
UPLOAD,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
expect(result.code).toBe(1);
|
||||
const document = JSON.parse(result.stdout);
|
||||
expect(document.state).toBe("rejected");
|
||||
expect(document.code).toBe("import_file_not_found");
|
||||
});
|
||||
});
|
||||
|
||||
describe("notes / extraction / graph / manual extract", () => {
|
||||
it("adds a note from a bounded file and re-verifies content on apply", async () => {
|
||||
const directory = await mkdtemp(path.join(tmpdir(), "om-note-"));
|
||||
const content = path.join(directory, "note.txt");
|
||||
await writeFile(content, "记住这个决定", { mode: 0o600 });
|
||||
const preview = await capturedRun([
|
||||
"notes",
|
||||
"add",
|
||||
"--content-file",
|
||||
content,
|
||||
"--json",
|
||||
]);
|
||||
expect(preview.code).toBe(0);
|
||||
const planFile = path.join(directory, "plan.json");
|
||||
await writeFile(planFile, preview.stdout, { mode: 0o600 });
|
||||
transport.post.mockResolvedValue({
|
||||
status: 201,
|
||||
// A realistic Core `MemoryNoteReceipt`: the shared parser requires
|
||||
// `scope` and `memory_ids` too, so a minimal `{evidence_id,state}` stub
|
||||
// is a malformed body, not a success.
|
||||
data: {
|
||||
evidence_id: NOTE,
|
||||
text: "记住这个决定",
|
||||
scope: "all-apps",
|
||||
created_at: "2026-09-19T00:00:00Z",
|
||||
state: "pending",
|
||||
task_id: null,
|
||||
memory_ids: [],
|
||||
error: null,
|
||||
},
|
||||
});
|
||||
const applied = await capturedRun([
|
||||
"notes",
|
||||
"apply",
|
||||
"--plan-file",
|
||||
planFile,
|
||||
"--content-file",
|
||||
content,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
expect(applied.code).toBe(0);
|
||||
const body = transport.post.mock.calls.at(-1)?.[1] as Record<
|
||||
string,
|
||||
unknown
|
||||
>;
|
||||
expect(body.text).toBe("记住这个决定");
|
||||
expect(body.scope).toBe("all-apps");
|
||||
expect(JSON.parse(applied.stdout).state).toBe("committed");
|
||||
});
|
||||
|
||||
it.skip("plans then applies the extraction window", async () => {
|
||||
const preview = await capturedRun([
|
||||
"extraction",
|
||||
"set",
|
||||
"--window",
|
||||
"6m",
|
||||
"--json",
|
||||
]);
|
||||
expect(preview.code).toBe(0);
|
||||
expect(JSON.parse(preview.stdout).state).toBe("preview");
|
||||
expect(transport.put).not.toHaveBeenCalled();
|
||||
transport.put.mockResolvedValue({
|
||||
status: 200,
|
||||
data: { window_preset: "6m" },
|
||||
});
|
||||
const applied = await capturedRun([
|
||||
"extraction",
|
||||
"set",
|
||||
"--window",
|
||||
"6m",
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
expect(applied.code).toBe(0);
|
||||
expect(transport.put).toHaveBeenCalledWith(
|
||||
"/api/v1/extraction-settings",
|
||||
{ window_preset: "6m" },
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
it("does not call graph width without --confirm", async () => {
|
||||
const preview = await capturedRun([
|
||||
"graph",
|
||||
"width",
|
||||
"--decision",
|
||||
"rebuild_with_new",
|
||||
"--json",
|
||||
]);
|
||||
expect(preview.code).toBe(0);
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: { archived_path: "/tmp/old", exported_path: null, mismatch: null },
|
||||
});
|
||||
const applied = await capturedRun([
|
||||
"graph",
|
||||
"width",
|
||||
"--decision",
|
||||
"rebuild_with_new",
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
expect(applied.code).toBe(0);
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
"/api/v1/graph-width",
|
||||
{ decision: "rebuild_with_new" },
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
it.skip("plans manual extraction and applies it only on confirm", async () => {
|
||||
const preview = await capturedRun([
|
||||
"processing",
|
||||
"extract",
|
||||
"--evidence",
|
||||
NOTE,
|
||||
"--json",
|
||||
]);
|
||||
expect(preview.code).toBe(0);
|
||||
expect(JSON.parse(preview.stdout).state).toBe("preview");
|
||||
expect(transport.post).not.toHaveBeenCalled();
|
||||
transport.post.mockResolvedValue({
|
||||
status: 200,
|
||||
data: { promoted: 1, batches_planned: 2, batches_current: 1 },
|
||||
});
|
||||
const applied = await capturedRun([
|
||||
"processing",
|
||||
"extract",
|
||||
"--evidence",
|
||||
NOTE,
|
||||
"--confirm",
|
||||
"--json",
|
||||
]);
|
||||
expect(applied.code).toBe(0);
|
||||
expect(transport.post).toHaveBeenCalledWith(
|
||||
"/api/v1/memory-processing/manual-extract",
|
||||
{ evidence_ids: [NOTE] },
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
async function capturedRun(arguments_: string[]) {
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
const stdoutSpy = vi
|
||||
.spyOn(process.stdout, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stdout += String(chunk);
|
||||
return true;
|
||||
});
|
||||
const stderrSpy = vi
|
||||
.spyOn(process.stderr, "write")
|
||||
.mockImplementation((chunk) => {
|
||||
stderr += String(chunk);
|
||||
return true;
|
||||
});
|
||||
try {
|
||||
return {
|
||||
code: await run(arguments_, {
|
||||
ONE_MEMORY_DATA_DIR: "/synthetic/data",
|
||||
}),
|
||||
stdout,
|
||||
stderr,
|
||||
};
|
||||
} finally {
|
||||
stdoutSpy.mockRestore();
|
||||
stderrSpy.mockRestore();
|
||||
}
|
||||
}
|
||||
@@ -1,501 +0,0 @@
|
||||
/**
|
||||
* The remaining Web Console mutating operations the CLI did not expose:
|
||||
* 端内新增记忆 (memory-notes), extraction window, graph width, and manual
|
||||
* extraction.
|
||||
*
|
||||
* Core owns every rule; this module only validates the input it is about to
|
||||
* send, performs one explicit two-phase write, and normalises the receipt so
|
||||
* the CLI never reports a write as committed without evidence.
|
||||
*/
|
||||
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import {
|
||||
buildMemoryNoteSaveRequest,
|
||||
MemoryNoteInputError,
|
||||
parseExtractionSettingsReceipt,
|
||||
parseGraphWidthDecisionReceipt,
|
||||
parseManualExtractReceipt,
|
||||
parseMemoryNoteReceipt,
|
||||
} from "@one-memory/operations";
|
||||
import type { CoreOperations, CorePutOperations } from "./core-operations.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
const DIGEST = /^[0-9a-f]{64}$/;
|
||||
const NOTE_MAX_CHARACTERS = 12_000;
|
||||
const EXTRACTION_PRESETS = ["3m", "6m", "1y", "all"] as const;
|
||||
const GRAPH_DECISIONS = [
|
||||
"keep_existing",
|
||||
"rebuild_with_new",
|
||||
"export_first",
|
||||
] as const;
|
||||
const NOTE_STATES = [
|
||||
"waiting_for_model",
|
||||
"waiting_for_permission",
|
||||
"pending",
|
||||
"running",
|
||||
"failed",
|
||||
"ready",
|
||||
"no_memories",
|
||||
] as const;
|
||||
|
||||
export type NoteState = (typeof NOTE_STATES)[number];
|
||||
|
||||
export type ExtendedWritePlan =
|
||||
| {
|
||||
schema_version: 1;
|
||||
kind: "memory_note_add";
|
||||
request_id: string;
|
||||
content_sha256: string;
|
||||
content_characters: number;
|
||||
digest: string;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
kind: "memory_note_process";
|
||||
evidence_id: string;
|
||||
digest: string;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
kind: "extraction_window_set";
|
||||
window_preset: (typeof EXTRACTION_PRESETS)[number];
|
||||
digest: string;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
kind: "graph_width_decide";
|
||||
decision: (typeof GRAPH_DECISIONS)[number];
|
||||
digest: string;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
kind: "manual_extract";
|
||||
evidence_ids: string[];
|
||||
digest: string;
|
||||
};
|
||||
|
||||
export type ExtendedWriteOutcome =
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "preview";
|
||||
plan: ExtendedWritePlan;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "committed" | "recovered";
|
||||
operation_id: string;
|
||||
receipt: Record<string, unknown>;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "rejected" | "unknown" | "invalid";
|
||||
operation_id?: string;
|
||||
code: string;
|
||||
};
|
||||
|
||||
function canonical(value: unknown): string {
|
||||
if (Array.isArray(value)) return `[${value.map(canonical).join(",")}]`;
|
||||
if (value !== null && typeof value === "object") {
|
||||
return `{${Object.entries(value as Record<string, unknown>)
|
||||
.sort(([left], [right]) => left.localeCompare(right))
|
||||
.map(([key, item]) => `${JSON.stringify(key)}:${canonical(item)}`)
|
||||
.join(",")}}`;
|
||||
}
|
||||
if (value === undefined) return "null";
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
|
||||
function digestOf(value: unknown): string {
|
||||
return createHash("sha256").update(canonical(value)).digest("hex");
|
||||
}
|
||||
|
||||
type Draft =
|
||||
| Omit<Extract<ExtendedWritePlan, { kind: "memory_note_add" }>, "digest">
|
||||
| Omit<Extract<ExtendedWritePlan, { kind: "memory_note_process" }>, "digest">
|
||||
| Omit<
|
||||
Extract<ExtendedWritePlan, { kind: "extraction_window_set" }>,
|
||||
"digest"
|
||||
>
|
||||
| Omit<Extract<ExtendedWritePlan, { kind: "graph_width_decide" }>, "digest">
|
||||
| Omit<Extract<ExtendedWritePlan, { kind: "manual_extract" }>, "digest">;
|
||||
|
||||
function preview(plan: Draft): ExtendedWriteOutcome {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "preview",
|
||||
plan: { ...plan, digest: digestOf(plan) } as ExtendedWritePlan,
|
||||
};
|
||||
}
|
||||
|
||||
function invalid(code: string): ExtendedWriteOutcome {
|
||||
return { schema_version: 1, product: PRODUCT, state: "invalid", code };
|
||||
}
|
||||
|
||||
function committed(
|
||||
operationId: string,
|
||||
receipt: Record<string, unknown>,
|
||||
state: "committed" | "recovered" = "committed",
|
||||
): ExtendedWriteOutcome {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state,
|
||||
operation_id: operationId,
|
||||
receipt,
|
||||
};
|
||||
}
|
||||
|
||||
function rejected(
|
||||
operationId: string | undefined,
|
||||
code: string,
|
||||
): ExtendedWriteOutcome {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "rejected",
|
||||
...(operationId === undefined ? {} : { operation_id: operationId }),
|
||||
code,
|
||||
};
|
||||
}
|
||||
|
||||
function unknown(
|
||||
operationId: string | undefined,
|
||||
code: string,
|
||||
): ExtendedWriteOutcome {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "unknown",
|
||||
...(operationId === undefined ? {} : { operation_id: operationId }),
|
||||
code,
|
||||
};
|
||||
}
|
||||
|
||||
// --- memory-notes ---------------------------------------------------------
|
||||
|
||||
export function planMemoryNoteAdd(text: string): ExtendedWriteOutcome {
|
||||
// The plan keeps only the note's identity, like `memories plan add`: the
|
||||
//正文 lives in a bounded file and is re-read and re-hashed at apply time, so
|
||||
// a plan file never carries the user's private text.
|
||||
if (!validNoteText(text)) return invalid("memory_note_text_invalid");
|
||||
const requestId = randomUUID();
|
||||
return preview({
|
||||
schema_version: 1,
|
||||
kind: "memory_note_add",
|
||||
request_id: requestId,
|
||||
content_sha256: createHash("sha256").update(text, "utf8").digest("hex"),
|
||||
content_characters: [...text].length,
|
||||
});
|
||||
}
|
||||
|
||||
export function validNoteText(text: string): boolean {
|
||||
return (
|
||||
text.length > 0 &&
|
||||
[...text].length <= NOTE_MAX_CHARACTERS &&
|
||||
text.trim().length > 0 &&
|
||||
![...text].some((character) => {
|
||||
const code = character.charCodeAt(0);
|
||||
return code <= 8 || (code >= 11 && code <= 31) || code === 127;
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
export function planMemoryNoteProcess(
|
||||
evidenceId: string,
|
||||
): ExtendedWriteOutcome {
|
||||
return UUID.test(evidenceId)
|
||||
? preview({
|
||||
schema_version: 1,
|
||||
kind: "memory_note_process",
|
||||
evidence_id: evidenceId,
|
||||
})
|
||||
: invalid("memory_note_id_invalid");
|
||||
}
|
||||
|
||||
function noteReceipt(
|
||||
operationId: string,
|
||||
data: { evidence_id: string; state: NoteState },
|
||||
): ExtendedWriteOutcome {
|
||||
return committed(operationId, { ...data });
|
||||
}
|
||||
|
||||
export async function executeMemoryNoteAdd(
|
||||
plan: Extract<ExtendedWritePlan, { kind: "memory_note_add" }>,
|
||||
operations: CoreOperations,
|
||||
text: string,
|
||||
): Promise<ExtendedWriteOutcome> {
|
||||
if (
|
||||
!validNoteText(text) ||
|
||||
createHash("sha256").update(text, "utf8").digest("hex") !==
|
||||
plan.content_sha256 ||
|
||||
[...text].length !== plan.content_characters
|
||||
) {
|
||||
return invalid("memory_note_content_mismatch");
|
||||
}
|
||||
// The shared builder owns the save body (request_id/text/scope=all-apps), so
|
||||
// the CLI and the Web console post the same three fields. Its own text check
|
||||
// already ran above, so a throw here means an unusable plan.
|
||||
let request: ReturnType<typeof buildMemoryNoteSaveRequest>;
|
||||
try {
|
||||
request = buildMemoryNoteSaveRequest({
|
||||
request_id: plan.request_id,
|
||||
text,
|
||||
scope: "all-apps",
|
||||
} as Parameters<typeof buildMemoryNoteSaveRequest>[0]);
|
||||
} catch (error) {
|
||||
if (error instanceof MemoryNoteInputError) {
|
||||
return invalid("memory_note_content_mismatch");
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
const result = await operations.post<unknown>(request.path, request.body);
|
||||
if (result.state === "rejected") {
|
||||
return rejected(plan.request_id, result.code);
|
||||
}
|
||||
if (result.state === "unknown") {
|
||||
return unknown(plan.request_id, result.code);
|
||||
}
|
||||
const receipt = asNoteReceipt(result.data);
|
||||
return receipt === null
|
||||
? unknown(plan.request_id, "ui_write_receipt_invalid")
|
||||
: noteReceipt(plan.request_id, receipt);
|
||||
}
|
||||
|
||||
export async function executeMemoryNoteProcess(
|
||||
plan: Extract<ExtendedWritePlan, { kind: "memory_note_process" }>,
|
||||
operations: CoreOperations,
|
||||
): Promise<ExtendedWriteOutcome> {
|
||||
const result = await operations.post<unknown>(
|
||||
`/api/v1/memory-notes/${plan.evidence_id}/process`,
|
||||
{},
|
||||
);
|
||||
if (result.state === "rejected") {
|
||||
return rejected(plan.evidence_id, result.code);
|
||||
}
|
||||
if (result.state === "unknown") {
|
||||
return unknown(plan.evidence_id, result.code);
|
||||
}
|
||||
const receipt = asNoteReceipt(result.data);
|
||||
return receipt === null
|
||||
? unknown(plan.evidence_id, "ui_write_receipt_invalid")
|
||||
: noteReceipt(plan.evidence_id, receipt);
|
||||
}
|
||||
|
||||
function asNoteReceipt(
|
||||
value: unknown,
|
||||
): { evidence_id: string; state: NoteState } | null {
|
||||
// The shared parser is the single judge of a note receipt; a malformed body
|
||||
// returns null and the caller reports `unknown`, never a saved note.
|
||||
const receipt = parseMemoryNoteReceipt(value);
|
||||
return receipt === null
|
||||
? null
|
||||
: { evidence_id: receipt.evidence_id, state: receipt.state as NoteState };
|
||||
}
|
||||
|
||||
// --- extraction window / graph width / manual extract ---------------------
|
||||
|
||||
export function planExtractionWindowSet(
|
||||
windowPreset: string,
|
||||
): ExtendedWriteOutcome {
|
||||
return (EXTRACTION_PRESETS as readonly string[]).includes(windowPreset)
|
||||
? preview({
|
||||
schema_version: 1,
|
||||
kind: "extraction_window_set",
|
||||
window_preset: windowPreset as (typeof EXTRACTION_PRESETS)[number],
|
||||
})
|
||||
: invalid("extraction_window_preset_invalid");
|
||||
}
|
||||
|
||||
export function planGraphWidthDecide(decision: string): ExtendedWriteOutcome {
|
||||
return (GRAPH_DECISIONS as readonly string[]).includes(decision)
|
||||
? preview({
|
||||
schema_version: 1,
|
||||
kind: "graph_width_decide",
|
||||
decision: decision as (typeof GRAPH_DECISIONS)[number],
|
||||
})
|
||||
: invalid("graph_width_decision_invalid");
|
||||
}
|
||||
|
||||
export function planManualExtract(evidenceIds: string[]): ExtendedWriteOutcome {
|
||||
const ids = [...new Set(evidenceIds)];
|
||||
return ids.length === evidenceIds.length && ids.every((id) => UUID.test(id))
|
||||
? preview({
|
||||
schema_version: 1,
|
||||
kind: "manual_extract",
|
||||
evidence_ids: ids,
|
||||
})
|
||||
: invalid("manual_extract_evidence_ids_invalid");
|
||||
}
|
||||
|
||||
export async function executeExtractionWindowSet(
|
||||
plan: Extract<ExtendedWritePlan, { kind: "extraction_window_set" }>,
|
||||
operations: CorePutOperations,
|
||||
): Promise<ExtendedWriteOutcome> {
|
||||
const result = await operations.put<unknown>("/api/v1/extraction-settings", {
|
||||
window_preset: plan.window_preset,
|
||||
});
|
||||
if (result.state === "rejected") {
|
||||
return rejected(plan.window_preset, result.code);
|
||||
}
|
||||
if (result.state === "unknown") {
|
||||
return unknown(plan.window_preset, result.code);
|
||||
}
|
||||
// The shared parser also enforces that the receipt echoes the requested
|
||||
// preset, so a response belonging to another write cannot pass.
|
||||
const parsed = parseExtractionSettingsReceipt(
|
||||
result.data,
|
||||
plan.window_preset,
|
||||
);
|
||||
if (parsed === null) {
|
||||
return unknown(plan.window_preset, "ui_write_receipt_invalid");
|
||||
}
|
||||
return committed(plan.window_preset, { window_preset: plan.window_preset });
|
||||
}
|
||||
|
||||
export async function executeGraphWidthDecide(
|
||||
plan: Extract<ExtendedWritePlan, { kind: "graph_width_decide" }>,
|
||||
operations: CoreOperations,
|
||||
): Promise<ExtendedWriteOutcome> {
|
||||
const result = await operations.post<unknown>("/api/v1/graph-width", {
|
||||
decision: plan.decision,
|
||||
});
|
||||
if (result.state === "rejected") {
|
||||
return rejected(plan.decision, result.code);
|
||||
}
|
||||
if (result.state === "unknown") {
|
||||
return unknown(plan.decision, result.code);
|
||||
}
|
||||
const receipt = parseGraphWidthDecisionReceipt(result.data);
|
||||
if (receipt === null) {
|
||||
return unknown(plan.decision, "ui_write_receipt_invalid");
|
||||
}
|
||||
return committed(plan.decision, {
|
||||
archived_path: receipt.archived_path ?? null,
|
||||
exported_path: receipt.exported_path ?? null,
|
||||
mismatch: receipt.mismatch ?? null,
|
||||
});
|
||||
}
|
||||
|
||||
export async function executeManualExtract(
|
||||
plan: Extract<ExtendedWritePlan, { kind: "manual_extract" }>,
|
||||
operations: CoreOperations,
|
||||
): Promise<ExtendedWriteOutcome> {
|
||||
const result = await operations.post<unknown>(
|
||||
"/api/v1/memory-processing/manual-extract",
|
||||
{ evidence_ids: plan.evidence_ids },
|
||||
);
|
||||
if (result.state === "rejected") {
|
||||
return rejected("manual_extract", result.code);
|
||||
}
|
||||
if (result.state === "unknown") {
|
||||
return unknown("manual_extract", result.code);
|
||||
}
|
||||
const receipt = parseManualExtractReceipt(result.data);
|
||||
if (receipt === null) {
|
||||
return unknown("manual_extract", "ui_write_receipt_invalid");
|
||||
}
|
||||
return committed("manual_extract", {
|
||||
promoted: receipt.promoted,
|
||||
batches_planned: receipt.batches_planned,
|
||||
batches_current: receipt.batches_current,
|
||||
});
|
||||
}
|
||||
|
||||
export function validateExtendedWritePlan(
|
||||
value: unknown,
|
||||
): ExtendedWritePlan | null {
|
||||
if (value === null || typeof value !== "object" || Array.isArray(value)) {
|
||||
return null;
|
||||
}
|
||||
const candidate = value as Record<string, unknown>;
|
||||
if (candidate.schema_version !== 1 || typeof candidate.kind !== "string") {
|
||||
return null;
|
||||
}
|
||||
const { digest, ...withoutDigest } = candidate;
|
||||
if (typeof digest !== "string" || digest !== digestOf(withoutDigest)) {
|
||||
return null;
|
||||
}
|
||||
switch (candidate.kind) {
|
||||
case "memory_note_add":
|
||||
return typeof candidate.request_id === "string" &&
|
||||
UUID.test(candidate.request_id) &&
|
||||
typeof candidate.content_sha256 === "string" &&
|
||||
DIGEST.test(candidate.content_sha256) &&
|
||||
Number.isSafeInteger(candidate.content_characters) &&
|
||||
(candidate.content_characters as number) >= 1 &&
|
||||
(candidate.content_characters as number) <= NOTE_MAX_CHARACTERS &&
|
||||
Object.keys(candidate).length === 6
|
||||
? (candidate as unknown as ExtendedWritePlan)
|
||||
: null;
|
||||
case "memory_note_process":
|
||||
return typeof candidate.evidence_id === "string" &&
|
||||
UUID.test(candidate.evidence_id) &&
|
||||
Object.keys(candidate).length === 4
|
||||
? (candidate as unknown as ExtendedWritePlan)
|
||||
: null;
|
||||
case "extraction_window_set":
|
||||
return (EXTRACTION_PRESETS as readonly string[]).includes(
|
||||
String(candidate.window_preset),
|
||||
) && Object.keys(candidate).length === 4
|
||||
? (candidate as unknown as ExtendedWritePlan)
|
||||
: null;
|
||||
case "graph_width_decide":
|
||||
return (GRAPH_DECISIONS as readonly string[]).includes(
|
||||
String(candidate.decision),
|
||||
) && Object.keys(candidate).length === 4
|
||||
? (candidate as unknown as ExtendedWritePlan)
|
||||
: null;
|
||||
case "manual_extract":
|
||||
return Array.isArray(candidate.evidence_ids) &&
|
||||
candidate.evidence_ids.every(
|
||||
(id) => typeof id === "string" && UUID.test(id),
|
||||
) &&
|
||||
Object.keys(candidate).length === 4
|
||||
? (candidate as unknown as ExtendedWritePlan)
|
||||
: null;
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function extendedWriteExitCode(document: ExtendedWriteOutcome): number {
|
||||
return ["preview", "committed", "recovered"].includes(document.state) ? 0 : 1;
|
||||
}
|
||||
|
||||
export function renderExtendedWriteText(
|
||||
document: ExtendedWriteOutcome,
|
||||
): string {
|
||||
if (document.state === "preview") {
|
||||
const lines = ["以下操作尚未执行;加 --confirm 后才会写入:"];
|
||||
const plan = document.plan;
|
||||
switch (plan.kind) {
|
||||
case "memory_note_add":
|
||||
lines.push(`新增端内记忆:${plan.content_characters} 字符`);
|
||||
break;
|
||||
case "memory_note_process":
|
||||
lines.push(`处理端内记忆:${plan.evidence_id}`);
|
||||
break;
|
||||
case "extraction_window_set":
|
||||
lines.push(`设置提取窗口:${plan.window_preset}`);
|
||||
break;
|
||||
case "graph_width_decide":
|
||||
lines.push(`图谱宽度决定:${plan.decision}`);
|
||||
break;
|
||||
case "manual_extract":
|
||||
lines.push(`手动提取:${plan.evidence_ids.length} 条证据`);
|
||||
break;
|
||||
}
|
||||
lines.push(`plan_digest:${plan.digest}`);
|
||||
return `${lines.join("\n")}\n`;
|
||||
}
|
||||
if (document.state === "committed" || document.state === "recovered") {
|
||||
return `${document.state === "recovered" ? "已从回执恢复" : "已提交"}:${document.operation_id}\n${JSON.stringify(document.receipt)}\n`;
|
||||
}
|
||||
const code = "code" in document ? document.code : "ui_write_state_invalid";
|
||||
return `操作未完成:${document.state}(${code})\n`;
|
||||
}
|
||||
@@ -1,144 +0,0 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
|
||||
import type {
|
||||
CoreDeleteOperations,
|
||||
CoreOperations,
|
||||
} from "./core-operations.js";
|
||||
import {
|
||||
executeCallEventsClear,
|
||||
executeCallEventsClearAll,
|
||||
executeImportFileCommit,
|
||||
executeImportFileDiscard,
|
||||
planCallEventsClear,
|
||||
planCallEventsClearAll,
|
||||
planImportFileCommit,
|
||||
planImportFileDiscard,
|
||||
planImportFileUpload,
|
||||
validateUiWritePlan,
|
||||
} from "./ui-write-flows.js";
|
||||
|
||||
const EVENT = "11111111-1111-4111-8111-111111111111";
|
||||
const UPLOAD = "22222222-2222-4222-8222-222222222222";
|
||||
|
||||
function planOf(value: ReturnType<typeof planCallEventsClear>) {
|
||||
if (value.state !== "preview") throw new Error("expected preview");
|
||||
return value.plan;
|
||||
}
|
||||
|
||||
describe("UI write plans", () => {
|
||||
it("permits only valid call-event ids and rejects duplicates", () => {
|
||||
expect(planCallEventsClear([EVENT]).state).toBe("preview");
|
||||
expect(planCallEventsClear([]).state).toBe("invalid");
|
||||
expect(planCallEventsClear([EVENT, EVENT]).state).toBe("invalid");
|
||||
});
|
||||
|
||||
it("plans all remaining UI write operations", () => {
|
||||
expect(planCallEventsClearAll().state).toBe("preview");
|
||||
expect(planImportFileCommit(UPLOAD).state).toBe("preview");
|
||||
expect(planImportFileDiscard(UPLOAD).state).toBe("preview");
|
||||
expect(
|
||||
planImportFileUpload({
|
||||
path: "/tmp/export.json",
|
||||
bytes: 42,
|
||||
sha256: "a".repeat(64),
|
||||
inputKind: "raw_conversation",
|
||||
}).state,
|
||||
).toBe("preview");
|
||||
});
|
||||
|
||||
it("rejects a tampered plan digest", async () => {
|
||||
const plan = planOf(planCallEventsClear([EVENT]));
|
||||
expect(validateUiWritePlan(plan)).toEqual(plan);
|
||||
expect(validateUiWritePlan({ ...plan, event_ids: [UPLOAD] })).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("UI write execution", () => {
|
||||
it("posts clear with Core's confirmation field", async () => {
|
||||
const post = vi.fn(async () => ({
|
||||
state: "committed" as const,
|
||||
data: { removed: 1 },
|
||||
}));
|
||||
const plan = planOf(planCallEventsClear([EVENT]));
|
||||
if (plan.kind !== "call_events_clear") throw new Error("kind");
|
||||
const result = await executeCallEventsClear(plan, {
|
||||
post,
|
||||
} as unknown as CoreOperations);
|
||||
expect(result.state).toBe("committed");
|
||||
expect(post).toHaveBeenCalledWith("/api/v1/call-events/clear", {
|
||||
event_ids: [EVENT],
|
||||
user_confirmed: true,
|
||||
});
|
||||
});
|
||||
|
||||
it("posts clear-all with Core's confirmation field", async () => {
|
||||
const post = vi.fn(async () => ({
|
||||
state: "committed" as const,
|
||||
data: { removed: 2 },
|
||||
}));
|
||||
const plan = planOf(planCallEventsClearAll());
|
||||
if (plan.kind !== "call_events_clear_all") throw new Error("kind");
|
||||
await executeCallEventsClearAll(plan, {
|
||||
post,
|
||||
} as unknown as CoreOperations);
|
||||
expect(post).toHaveBeenCalledWith("/api/v1/call-events/clear-all", {
|
||||
user_confirmed: true,
|
||||
});
|
||||
});
|
||||
|
||||
it("uses POST commit and DELETE discard endpoints", async () => {
|
||||
const post = vi.fn(async () => ({
|
||||
state: "committed" as const,
|
||||
data: {
|
||||
upload_id: UPLOAD,
|
||||
filename: "x.json",
|
||||
vendor: "generic",
|
||||
source_id: "generic-export",
|
||||
messages: 0,
|
||||
conversations: 0,
|
||||
human_messages: 0,
|
||||
agent_messages: 0,
|
||||
skipped_records: 0,
|
||||
ignored_files: 0,
|
||||
checksum: "a".repeat(64),
|
||||
created_at: "T",
|
||||
committed: true,
|
||||
},
|
||||
}));
|
||||
const del = vi.fn(async <T>() => ({
|
||||
state: "committed" as const,
|
||||
data: {} as T,
|
||||
}));
|
||||
const operations = {
|
||||
post,
|
||||
delete: del,
|
||||
} as unknown as CoreOperations & CoreDeleteOperations;
|
||||
const commit = planOf(planImportFileCommit(UPLOAD));
|
||||
if (commit.kind !== "import_file_commit") throw new Error("kind");
|
||||
await executeImportFileCommit(commit, operations);
|
||||
expect(post).toHaveBeenCalledWith(`/api/v1/import-files/${UPLOAD}/commit`, {
|
||||
confirmed: true,
|
||||
});
|
||||
const discard = planOf(planImportFileDiscard(UPLOAD));
|
||||
if (discard.kind !== "import_file_discard") throw new Error("kind");
|
||||
expect((await executeImportFileDiscard(discard, operations)).state).toBe(
|
||||
"committed",
|
||||
);
|
||||
expect(del).toHaveBeenCalledWith(`/api/v1/import-files/${UPLOAD}`);
|
||||
});
|
||||
|
||||
it("treats a malformed success receipt as unknown", async () => {
|
||||
const plan = planOf(planCallEventsClear([EVENT]));
|
||||
if (plan.kind !== "call_events_clear") throw new Error("kind");
|
||||
const result = await executeCallEventsClear(plan, {
|
||||
post: vi.fn(async () => ({
|
||||
state: "committed" as const,
|
||||
data: { wrong: true },
|
||||
})),
|
||||
} as unknown as CoreOperations);
|
||||
expect(result).toMatchObject({
|
||||
state: "unknown",
|
||||
code: "ui_write_receipt_invalid",
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -1,555 +0,0 @@
|
||||
/**
|
||||
* The Web Console's remaining mutating operations, exposed to the CLI without
|
||||
* duplicating Core semantics. Each operation is planned first; callers apply
|
||||
* the exact plan they reviewed.
|
||||
*/
|
||||
|
||||
import { createHash } from "node:crypto";
|
||||
import { constants } from "node:fs";
|
||||
import { type FileHandle, open } from "node:fs/promises";
|
||||
import type {
|
||||
CoreDeleteOperations,
|
||||
CoreOperations,
|
||||
CoreUploadOperations,
|
||||
} from "./core-operations.js";
|
||||
import { PRODUCT } from "./version.js";
|
||||
|
||||
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||
const INPUT_KINDS = ["raw_conversation", "memory_content"] as const;
|
||||
const MAX_UPLOAD_BYTES = 128 * 1024 * 1024;
|
||||
|
||||
export type UiWritePlan =
|
||||
| {
|
||||
schema_version: 1;
|
||||
kind: "call_events_clear";
|
||||
event_ids: string[];
|
||||
digest: string;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
kind: "call_events_clear_all";
|
||||
digest: string;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
kind: "import_file_commit";
|
||||
upload_id: string;
|
||||
digest: string;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
kind: "import_file_discard";
|
||||
upload_id: string;
|
||||
digest: string;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
kind: "import_file_upload";
|
||||
operation_id: string;
|
||||
path: string;
|
||||
filename: string;
|
||||
input_kind: (typeof INPUT_KINDS)[number];
|
||||
bytes: number;
|
||||
sha256: string;
|
||||
digest: string;
|
||||
};
|
||||
|
||||
export type UiWriteOutcome =
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "preview";
|
||||
plan: UiWritePlan;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "committed" | "recovered";
|
||||
operation_id: string;
|
||||
receipt: Record<string, unknown>;
|
||||
}
|
||||
| {
|
||||
schema_version: 1;
|
||||
product: string;
|
||||
state: "rejected" | "unknown" | "invalid";
|
||||
operation_id?: string;
|
||||
code: string;
|
||||
receipt?: Record<string, unknown>;
|
||||
};
|
||||
|
||||
function digestOf(value: unknown): string {
|
||||
return createHash("sha256").update(canonical(value)).digest("hex");
|
||||
}
|
||||
|
||||
function canonical(value: unknown): string {
|
||||
if (Array.isArray(value)) return `[${value.map(canonical).join(",")}]`;
|
||||
if (value !== null && typeof value === "object") {
|
||||
return `{${Object.entries(value as Record<string, unknown>)
|
||||
.sort(([left], [right]) => left.localeCompare(right))
|
||||
.map(([key, item]) => `${JSON.stringify(key)}:${canonical(item)}`)
|
||||
.join(",")}}`;
|
||||
}
|
||||
if (value === undefined) return "null";
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
|
||||
export type UiWritePlanDraft =
|
||||
| Omit<Extract<UiWritePlan, { kind: "call_events_clear" }>, "digest">
|
||||
| Omit<Extract<UiWritePlan, { kind: "call_events_clear_all" }>, "digest">
|
||||
| Omit<Extract<UiWritePlan, { kind: "import_file_commit" }>, "digest">
|
||||
| Omit<Extract<UiWritePlan, { kind: "import_file_discard" }>, "digest">
|
||||
| Omit<Extract<UiWritePlan, { kind: "import_file_upload" }>, "digest">;
|
||||
|
||||
function preview(plan: UiWritePlanDraft): UiWriteOutcome {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "preview",
|
||||
plan: { ...plan, digest: digestOf(plan) } as UiWritePlan,
|
||||
};
|
||||
}
|
||||
|
||||
function invalid(code: string): UiWriteOutcome {
|
||||
return { schema_version: 1, product: PRODUCT, state: "invalid", code };
|
||||
}
|
||||
|
||||
function committed(
|
||||
operationId: string,
|
||||
receipt: Record<string, unknown>,
|
||||
state: "committed" | "recovered" = "committed",
|
||||
): UiWriteOutcome {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state,
|
||||
operation_id: operationId,
|
||||
receipt,
|
||||
};
|
||||
}
|
||||
|
||||
export function planCallEventsClear(
|
||||
eventIds: readonly string[],
|
||||
): UiWriteOutcome {
|
||||
if (
|
||||
eventIds.length < 1 ||
|
||||
eventIds.length > 100 ||
|
||||
new Set(eventIds).size !== eventIds.length ||
|
||||
eventIds.some((id) => !UUID.test(id))
|
||||
) {
|
||||
return invalid("call_event_ids_invalid");
|
||||
}
|
||||
return preview({
|
||||
schema_version: 1,
|
||||
kind: "call_events_clear",
|
||||
event_ids: [...eventIds],
|
||||
});
|
||||
}
|
||||
|
||||
export function planCallEventsClearAll(): UiWriteOutcome {
|
||||
return preview({ schema_version: 1, kind: "call_events_clear_all" });
|
||||
}
|
||||
|
||||
export function planImportFileCommit(uploadId: string): UiWriteOutcome {
|
||||
return UUID.test(uploadId)
|
||||
? preview({
|
||||
schema_version: 1,
|
||||
kind: "import_file_commit",
|
||||
upload_id: uploadId,
|
||||
})
|
||||
: invalid("import_file_id_invalid");
|
||||
}
|
||||
|
||||
export function planImportFileDiscard(uploadId: string): UiWriteOutcome {
|
||||
return UUID.test(uploadId)
|
||||
? preview({
|
||||
schema_version: 1,
|
||||
kind: "import_file_discard",
|
||||
upload_id: uploadId,
|
||||
})
|
||||
: invalid("import_file_id_invalid");
|
||||
}
|
||||
|
||||
export function planImportFileUpload(input: {
|
||||
path: string;
|
||||
bytes: number;
|
||||
sha256: string;
|
||||
inputKind: "raw_conversation" | "memory_content";
|
||||
}): UiWriteOutcome {
|
||||
const filename = input.path.split(/[/\\]/).pop() ?? "";
|
||||
if (
|
||||
input.path.length === 0 ||
|
||||
!Number.isSafeInteger(input.bytes) ||
|
||||
input.bytes < 0 ||
|
||||
input.bytes > MAX_UPLOAD_BYTES ||
|
||||
!/^[0-9a-f]{64}$/.test(input.sha256) ||
|
||||
!INPUT_KINDS.includes(input.inputKind) ||
|
||||
filename.length === 0 ||
|
||||
[...filename].length > 200 ||
|
||||
!/\.(?:zip|json|md|txt)$/i.test(filename)
|
||||
) {
|
||||
return invalid("import_file_upload_invalid");
|
||||
}
|
||||
return preview({
|
||||
schema_version: 1,
|
||||
kind: "import_file_upload",
|
||||
operation_id: fileUploadOperationId(
|
||||
filename,
|
||||
input.inputKind,
|
||||
input.sha256,
|
||||
),
|
||||
path: input.path,
|
||||
filename,
|
||||
input_kind: input.inputKind,
|
||||
bytes: input.bytes,
|
||||
sha256: input.sha256,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Read one regular, non-symlinked export without buffering past the 128 MiB
|
||||
* limit. Returns the bytes plus the checksum the plan records, so apply can
|
||||
* prove the file it uploads is the one the caller reviewed.
|
||||
*/
|
||||
export async function readImportFileUpload(
|
||||
filePath: string,
|
||||
inputKind: "raw_conversation" | "memory_content",
|
||||
): Promise<{ plan: UiWritePlan; bytes: Buffer } | null> {
|
||||
let handle: FileHandle;
|
||||
try {
|
||||
handle = await open(
|
||||
filePath,
|
||||
constants.O_RDONLY |
|
||||
(constants.O_NONBLOCK ?? 0) |
|
||||
(constants.O_NOFOLLOW ?? 0),
|
||||
);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
const stats = await handle.stat();
|
||||
if (!stats.isFile() || stats.nlink !== 1 || stats.size > MAX_UPLOAD_BYTES) {
|
||||
return null;
|
||||
}
|
||||
const buffer = Buffer.alloc(Number(stats.size));
|
||||
let offset = 0;
|
||||
while (offset < buffer.length) {
|
||||
const chunk = await handle.read(
|
||||
buffer,
|
||||
offset,
|
||||
buffer.length - offset,
|
||||
offset,
|
||||
);
|
||||
if (chunk.bytesRead === 0) break;
|
||||
offset += chunk.bytesRead;
|
||||
}
|
||||
if (offset !== buffer.length) return null;
|
||||
const checksum = createHash("sha256").update(buffer).digest("hex");
|
||||
const result = planImportFileUpload({
|
||||
path: filePath,
|
||||
bytes: buffer.byteLength,
|
||||
sha256: checksum,
|
||||
inputKind,
|
||||
});
|
||||
if (result.state !== "preview") return null;
|
||||
return { plan: result.plan, bytes: buffer };
|
||||
} catch {
|
||||
return null;
|
||||
} finally {
|
||||
await handle.close().catch(() => undefined);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Core keys upload idempotency by `operation_id` and rejects a replay whose
|
||||
* fingerprint differs. Deriving it from the file's own identity means a retry
|
||||
* after a dropped response resumes the same staging operation instead of
|
||||
* creating a second upload; a genuinely different payload maps to a different
|
||||
* operation and cannot overwrite the first.
|
||||
*/
|
||||
function fileUploadOperationId(
|
||||
filename: string,
|
||||
inputKind: (typeof INPUT_KINDS)[number],
|
||||
checksum: string,
|
||||
): string {
|
||||
const hex = createHash("sha256")
|
||||
.update(JSON.stringify([filename, inputKind, checksum]))
|
||||
.digest("hex");
|
||||
return [
|
||||
hex.slice(0, 8),
|
||||
hex.slice(8, 12),
|
||||
`4${hex.slice(13, 16)}`,
|
||||
`${(8 + (parseInt(hex[16]!, 16) % 4)).toString(16)}${hex.slice(17, 20)}`,
|
||||
hex.slice(20, 32),
|
||||
].join("-");
|
||||
}
|
||||
|
||||
export function validateUiWritePlan(value: unknown): UiWritePlan | null {
|
||||
if (value === null || typeof value !== "object" || Array.isArray(value)) {
|
||||
return null;
|
||||
}
|
||||
const candidate = value as Record<string, unknown>;
|
||||
if (candidate.schema_version !== 1 || typeof candidate.kind !== "string") {
|
||||
return null;
|
||||
}
|
||||
const { digest, ...withoutDigest } = candidate;
|
||||
if (typeof digest !== "string" || digest !== digestOf(withoutDigest)) {
|
||||
return null;
|
||||
}
|
||||
switch (candidate.kind) {
|
||||
case "call_events_clear":
|
||||
return Array.isArray(candidate.event_ids) &&
|
||||
candidate.event_ids.length >= 1 &&
|
||||
candidate.event_ids.length <= 100 &&
|
||||
new Set(candidate.event_ids).size === candidate.event_ids.length &&
|
||||
candidate.event_ids.every(
|
||||
(id) => typeof id === "string" && UUID.test(id),
|
||||
)
|
||||
? (candidate as unknown as UiWritePlan)
|
||||
: null;
|
||||
case "call_events_clear_all":
|
||||
return Object.keys(candidate).length === 3
|
||||
? (candidate as unknown as UiWritePlan)
|
||||
: null;
|
||||
case "import_file_commit":
|
||||
case "import_file_discard":
|
||||
return typeof candidate.upload_id === "string" &&
|
||||
UUID.test(candidate.upload_id) &&
|
||||
Object.keys(candidate).length === 4
|
||||
? (candidate as unknown as UiWritePlan)
|
||||
: null;
|
||||
case "import_file_upload":
|
||||
return typeof candidate.operation_id === "string" &&
|
||||
UUID.test(candidate.operation_id) &&
|
||||
typeof candidate.path === "string" &&
|
||||
typeof candidate.filename === "string" &&
|
||||
candidate.filename === candidate.path.split(/[/\\]/).pop() &&
|
||||
/^[0-9a-f]{64}$/.test(String(candidate.sha256)) &&
|
||||
INPUT_KINDS.includes(
|
||||
candidate.input_kind as (typeof INPUT_KINDS)[number],
|
||||
) &&
|
||||
Number.isSafeInteger(candidate.bytes) &&
|
||||
(candidate.bytes as number) >= 0 &&
|
||||
(candidate.bytes as number) <= 128 * 1024 * 1024
|
||||
? (candidate as unknown as UiWritePlan)
|
||||
: null;
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function uiWriteExitCode(document: UiWriteOutcome): number {
|
||||
return ["preview", "committed", "recovered"].includes(document.state) ? 0 : 1;
|
||||
}
|
||||
|
||||
export function renderUiWriteText(document: UiWriteOutcome): string {
|
||||
if (document.state === "preview") {
|
||||
return `以下操作尚未执行;加 --confirm 后才会写入:\n${document.plan.kind}\nplan_digest:${document.plan.digest}\n`;
|
||||
}
|
||||
if (document.state === "committed" || document.state === "recovered") {
|
||||
return `已提交:${document.operation_id}\n${JSON.stringify(document.receipt)}\n`;
|
||||
}
|
||||
const code = "code" in document ? document.code : "ui_write_state_invalid";
|
||||
return `操作未完成:${document.state}(${code})\n`;
|
||||
}
|
||||
|
||||
export async function executeCallEventsClear(
|
||||
plan: Extract<UiWritePlan, { kind: "call_events_clear" }>,
|
||||
operations: CoreOperations,
|
||||
): Promise<UiWriteOutcome> {
|
||||
const result = await operations.post<unknown>("/api/v1/call-events/clear", {
|
||||
event_ids: plan.event_ids,
|
||||
user_confirmed: true,
|
||||
});
|
||||
return outcome(result, plan.event_ids.join(","), ["removed"]);
|
||||
}
|
||||
|
||||
export async function executeCallEventsClearAll(
|
||||
plan: Extract<UiWritePlan, { kind: "call_events_clear_all" }>,
|
||||
operations: CoreOperations,
|
||||
): Promise<UiWriteOutcome> {
|
||||
const result = await operations.post<unknown>(
|
||||
"/api/v1/call-events/clear-all",
|
||||
{ user_confirmed: true },
|
||||
);
|
||||
return outcome(result, "all", ["removed"]);
|
||||
}
|
||||
|
||||
export async function executeImportFileCommit(
|
||||
plan: Extract<UiWritePlan, { kind: "import_file_commit" }>,
|
||||
operations: CoreOperations,
|
||||
): Promise<UiWriteOutcome> {
|
||||
const result = await operations.post<unknown>(
|
||||
`/api/v1/import-files/${plan.upload_id}/commit`,
|
||||
{ confirmed: true },
|
||||
);
|
||||
return outcome(result, plan.upload_id, [
|
||||
"upload_id",
|
||||
"filename",
|
||||
"vendor",
|
||||
"source_id",
|
||||
"messages",
|
||||
"conversations",
|
||||
"human_messages",
|
||||
"agent_messages",
|
||||
"skipped_records",
|
||||
"ignored_files",
|
||||
"checksum",
|
||||
"created_at",
|
||||
"committed",
|
||||
]);
|
||||
}
|
||||
|
||||
export async function executeImportFileDiscard(
|
||||
plan: Extract<UiWritePlan, { kind: "import_file_discard" }>,
|
||||
operations: CoreDeleteOperations,
|
||||
): Promise<UiWriteOutcome> {
|
||||
const result = await operations.delete<unknown>(
|
||||
`/api/v1/import-files/${plan.upload_id}`,
|
||||
);
|
||||
// Core answers 204 with no body. A successful delete is a committed
|
||||
// operation even though there is no receipt to validate; requiring a body
|
||||
// here would report every real discard as `ui_write_receipt_invalid`.
|
||||
return outcome(result, plan.upload_id, null);
|
||||
}
|
||||
|
||||
/** Upload one local file through the same recoverable Core staging flow the
|
||||
* Web console uses. The operation id is part of the plan, so a retry after a
|
||||
* dropped response resumes the same staging record instead of duplicating it.
|
||||
*/
|
||||
export async function executeImportFileUpload(
|
||||
plan: Extract<UiWritePlan, { kind: "import_file_upload" }>,
|
||||
operations: CoreUploadOperations,
|
||||
): Promise<UiWriteOutcome> {
|
||||
// Re-read the file at apply time and re-derive its identity. A plan that
|
||||
// names a path whose content changed after review is refused rather than
|
||||
// uploading something the caller never saw.
|
||||
const reread = await readImportFileUpload(plan.path, plan.input_kind);
|
||||
if (reread === null) {
|
||||
return invalid("import_file_upload_unreadable");
|
||||
}
|
||||
const bytes = reread.bytes;
|
||||
const checksum = createHash("sha256").update(bytes).digest("hex");
|
||||
if (
|
||||
bytes.byteLength !== plan.bytes ||
|
||||
checksum !== plan.sha256 ||
|
||||
reread.plan.digest !== plan.digest
|
||||
) {
|
||||
return invalid("import_file_upload_content_mismatch");
|
||||
}
|
||||
const result = await operations.uploadImportFile<unknown>({
|
||||
bytes,
|
||||
filename: plan.filename,
|
||||
inputKind: plan.input_kind,
|
||||
operationId: plan.operation_id,
|
||||
});
|
||||
if (result.state === "rejected") {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "rejected",
|
||||
operation_id: plan.operation_id,
|
||||
code: result.code,
|
||||
};
|
||||
}
|
||||
if (result.state === "unknown") {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "unknown",
|
||||
operation_id: plan.operation_id,
|
||||
code: result.code,
|
||||
};
|
||||
}
|
||||
const receipt =
|
||||
result.data !== null &&
|
||||
typeof result.data === "object" &&
|
||||
!Array.isArray(result.data)
|
||||
? (result.data as Record<string, unknown>)
|
||||
: null;
|
||||
if (
|
||||
receipt === null ||
|
||||
receipt.operation_id !== plan.operation_id ||
|
||||
typeof receipt.state !== "string"
|
||||
) {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "unknown",
|
||||
operation_id: plan.operation_id,
|
||||
code: "ui_write_receipt_invalid",
|
||||
};
|
||||
}
|
||||
// Core reports the staged upload as `completed`; `pending`/`not_found` here
|
||||
// means the operation record did not settle, so the result is not known.
|
||||
if (receipt.state !== "completed") {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "unknown",
|
||||
operation_id: plan.operation_id,
|
||||
code: `import_upload_${receipt.state}`,
|
||||
};
|
||||
}
|
||||
const uploadId =
|
||||
typeof receipt.upload_id === "string" ? receipt.upload_id : undefined;
|
||||
if (uploadId === undefined || !UUID.test(uploadId)) {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "unknown",
|
||||
operation_id: plan.operation_id,
|
||||
code: "ui_write_receipt_invalid",
|
||||
};
|
||||
}
|
||||
return committed(plan.operation_id, {
|
||||
operation_id: plan.operation_id,
|
||||
upload_id: uploadId,
|
||||
upload_state: receipt.upload_state ?? null,
|
||||
});
|
||||
}
|
||||
|
||||
function outcome(
|
||||
result:
|
||||
| Awaited<ReturnType<CoreOperations["post"]>>
|
||||
| Awaited<ReturnType<CoreDeleteOperations["delete"]>>,
|
||||
operationId: string,
|
||||
required: readonly string[] | null,
|
||||
): UiWriteOutcome {
|
||||
if (result.state === "rejected") {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "rejected",
|
||||
operation_id: operationId,
|
||||
code: result.code,
|
||||
};
|
||||
}
|
||||
if (result.state === "unknown") {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "unknown",
|
||||
operation_id: operationId,
|
||||
code: result.code,
|
||||
};
|
||||
}
|
||||
if (required === null) {
|
||||
return committed(operationId, {});
|
||||
}
|
||||
const data =
|
||||
result.data !== null &&
|
||||
typeof result.data === "object" &&
|
||||
!Array.isArray(result.data)
|
||||
? (result.data as Record<string, unknown>)
|
||||
: null;
|
||||
if (data === null || required.some((key) => !Object.hasOwn(data, key))) {
|
||||
return {
|
||||
schema_version: 1,
|
||||
product: PRODUCT,
|
||||
state: "unknown",
|
||||
operation_id: operationId,
|
||||
code: "ui_write_receipt_invalid",
|
||||
};
|
||||
}
|
||||
return committed(operationId, data);
|
||||
}
|
||||
@@ -13,7 +13,16 @@ const GLOBAL_OPTIONS: readonly CommandOption[] = [
|
||||
summary: "显示帮助;可与 --json 组合输出机器可读目录。",
|
||||
},
|
||||
{ flag: "--version, -V", summary: "显示 CLI 版本。" },
|
||||
{ flag: "--json", summary: "以 JSON 输出到 stdout,供脚本与 Agent 消费。" },
|
||||
{
|
||||
flag: "--json",
|
||||
summary:
|
||||
"以 JSON 输出到 stdout,供脚本与 Agent 消费;结果与失败都在 stdout,只有参数错误写 stderr。",
|
||||
},
|
||||
{
|
||||
flag: "--data-dir <path>",
|
||||
summary:
|
||||
"本次使用的 MemHub 数据目录(绝对路径);默认 ONE_MEMORY_DATA_DIR > 安装回执 > ~/.one-memory。",
|
||||
},
|
||||
];
|
||||
|
||||
export function renderGeneralHelp(): string {
|
||||
@@ -33,6 +42,9 @@ export function renderGeneralHelp(): string {
|
||||
(exit) => ` ${exit.code} ${pad(exit.name, 17)}${exit.summary}`,
|
||||
),
|
||||
"",
|
||||
"确认约定:只有不可逆操作(memories delete、models remove、delete-data、backup restore、graph-width --decision rebuild_with_new)需要 --yes;不加时只说明将要做什么,不做任何改动,exit 0。delete-data 与 backup restore 还要用 --confirm <path> 再写一遍目标路径。install/upgrade/uninstall/service-install/service-uninstall 不加 --yes 时只出计划。其它写命令直接执行。",
|
||||
"异步操作返回 ID,用对应命令查进度:sources receipt、models receipt、memories add-status、write status。",
|
||||
"目前仅支持 macOS 与 Linux 连接本机 Runtime。",
|
||||
"stdout 只输出数据,进度与诊断写入 stderr。",
|
||||
`运行 ${PRODUCT} help <command> 查看单个命令的参数。`,
|
||||
"",
|
||||
@@ -55,6 +67,8 @@ export function renderCommandHelp(command: CommandDescriptor): string {
|
||||
);
|
||||
}
|
||||
lines.push(
|
||||
"Global options: --json、--data-dir <path>(见 memhub help)。",
|
||||
"",
|
||||
"Exit codes:",
|
||||
...EXIT_CODES.filter((exit) => command.exit_codes.includes(exit.code)).map(
|
||||
(exit) => ` ${exit.code} ${pad(exit.name, 17)}${exit.summary}`,
|
||||
|
||||
@@ -12,7 +12,7 @@ dependencies = [
|
||||
"fastapi>=0.141.1",
|
||||
"filelock>=3.32.4",
|
||||
# Public SDK version validated against MemHub's permission and recall contracts.
|
||||
"hy-memory==2.0.5",
|
||||
"hy-memory==2.0.7",
|
||||
# chromadb arrives through the HY SDK; locked here so an unrelated upgrade
|
||||
# cannot move the SDK's store. Built-in embedding consumes numpy directly.
|
||||
"chromadb==1.5.9",
|
||||
|
||||
+715
-1716
File diff suppressed because it is too large.
Load diff
@@ -15,7 +15,7 @@ from one_memory.call_event_store import (
|
||||
CallTool,
|
||||
HostId,
|
||||
)
|
||||
from one_memory.context_models import ContextSearchResponse, MemoryContextItem, RawContextItem
|
||||
from one_memory.context_models import SdkContextSearchResponse
|
||||
from one_memory.external_memory_access import ExternalMemoryAccess
|
||||
from one_memory.recall import RecallReceipt
|
||||
|
||||
@@ -41,75 +41,27 @@ def _host_id(access: ExternalMemoryAccess | None) -> HostId | None:
|
||||
return cast(HostId, host) if host in HOSTS else None
|
||||
|
||||
|
||||
def context_call_event(
|
||||
result: ContextSearchResponse,
|
||||
def sdk_context_call_event(
|
||||
result: SdkContextSearchResponse,
|
||||
access: ExternalMemoryAccess | None,
|
||||
*,
|
||||
memory_searched: bool,
|
||||
timed_out: bool,
|
||||
) -> CallEvent:
|
||||
references: list[CallReference] = []
|
||||
seen: set[tuple[str, str, int]] = set()
|
||||
for item in result.items:
|
||||
if isinstance(item, RawContextItem):
|
||||
candidates = [
|
||||
CallReference(
|
||||
kind="raw",
|
||||
id=message.evidence_id,
|
||||
revision=message.revision,
|
||||
source_id=item.source_id,
|
||||
)
|
||||
for message in item.messages
|
||||
]
|
||||
elif isinstance(item, MemoryContextItem):
|
||||
candidates = [
|
||||
CallReference(
|
||||
kind="memory", id=item.memory_id, revision=item.revision, source_id=None
|
||||
)
|
||||
]
|
||||
else:
|
||||
continue
|
||||
for reference in candidates:
|
||||
identity = (reference.kind, reference.id, reference.revision)
|
||||
if identity not in seen:
|
||||
seen.add(identity)
|
||||
references.append(reference)
|
||||
timed_out = any(
|
||||
path.reason == "timeout_budget" for path in (result.paths.raw, result.paths.memory)
|
||||
# SDK items carry no Evidence/Governance revision, and the reference
|
||||
# resolver reads the legacy store, so the event records counts only.
|
||||
raw_count = sum(item.kind == "raw" for item in result.items)
|
||||
memory_count = sum(item.kind == "memory" for item in result.items)
|
||||
state: CallState = "timed_out" if timed_out else result.state
|
||||
return _metadata_event(
|
||||
tool="om_recall" if memory_searched else "om_conversations",
|
||||
state=state,
|
||||
elapsed_ms=result.total_ms,
|
||||
access=access,
|
||||
raw_count=raw_count,
|
||||
memory_count=memory_count,
|
||||
event_id=UUID(result.request_id),
|
||||
)
|
||||
state: CallState = (
|
||||
"partial" if timed_out and references else "timed_out" if timed_out else result.state
|
||||
)
|
||||
return CallEvent.model_validate(
|
||||
{
|
||||
"event_id": UUID(result.request_id),
|
||||
"occurred_at": datetime.now(UTC),
|
||||
"host_id": _host_id(access),
|
||||
"installation_id": UUID(access.installation_id) if access else None,
|
||||
"tool": _context_tool(result),
|
||||
"state": state,
|
||||
"stage": "core_result",
|
||||
"delivery": "unknown",
|
||||
"adoption": "unknown",
|
||||
"host_session_id": None,
|
||||
"injection_elapsed_ms": None,
|
||||
"elapsed_ms": result.total_ms,
|
||||
"raw_count": sum(isinstance(item, RawContextItem) for item in result.items),
|
||||
"memory_count": sum(isinstance(item, MemoryContextItem) for item in result.items),
|
||||
"references": tuple(references),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def record_context_result(
|
||||
store: CallEventStore,
|
||||
result: ContextSearchResponse,
|
||||
access: ExternalMemoryAccess | None,
|
||||
) -> None:
|
||||
store.record(context_call_event(result, access))
|
||||
|
||||
|
||||
def _context_tool(result: ContextSearchResponse) -> CallTool:
|
||||
if result.paths.memory.status not in {"not_requested", "not_authorized"}:
|
||||
return "om_recall"
|
||||
return "om_conversations"
|
||||
|
||||
|
||||
def _metadata_event(
|
||||
@@ -142,26 +94,6 @@ def _metadata_event(
|
||||
)
|
||||
|
||||
|
||||
def conversation_call_event(
|
||||
result: object,
|
||||
access: ExternalMemoryAccess | None,
|
||||
elapsed_ms: float,
|
||||
) -> CallEvent:
|
||||
hits = tuple(getattr(result, "hits", ()) or ())
|
||||
conversations = {
|
||||
conversation_id
|
||||
for hit in hits
|
||||
if (conversation_id := getattr(hit, "conversation_id", None))
|
||||
}
|
||||
return _metadata_event(
|
||||
tool="om_conversations",
|
||||
state="grounded" if hits else "empty",
|
||||
elapsed_ms=elapsed_ms,
|
||||
access=access,
|
||||
raw_count=min(len(conversations), 20),
|
||||
)
|
||||
|
||||
|
||||
def ask_call_event(
|
||||
result: object,
|
||||
access: ExternalMemoryAccess | None,
|
||||
@@ -255,8 +187,9 @@ def record_legacy_result(
|
||||
host_session_id=None,
|
||||
injection_elapsed_ms=None,
|
||||
elapsed_ms=elapsed_ms,
|
||||
raw_count=0,
|
||||
memory_count=len(receipt.hits),
|
||||
# A raw-fallback recall returned conversation excerpts, not memories.
|
||||
raw_count=len(receipt.hits) if receipt.mode == "raw" else 0,
|
||||
memory_count=0 if receipt.mode == "raw" else len(receipt.hits),
|
||||
references=(),
|
||||
)
|
||||
)
|
||||
@@ -1,6 +1,5 @@
|
||||
import os
|
||||
import signal
|
||||
import socket
|
||||
import sys
|
||||
|
||||
# A handed-off capture ends itself: the hook that gave it the prompt has
|
||||
@@ -29,10 +28,13 @@ def _end_by(seconds: str | None) -> None:
|
||||
if __name__ == "__main__" and sys.argv[1:2] == ["capture"]:
|
||||
_end_by(os.environ.get("ONE_MEMORY_CAPTURE_DEADLINE_S"))
|
||||
|
||||
import contextlib # noqa: E402
|
||||
import errno # noqa: E402
|
||||
import json # noqa: E402
|
||||
import re # noqa: E402
|
||||
import time # noqa: E402
|
||||
from collections.abc import Iterator, Sequence # noqa: E402
|
||||
from pathlib import Path # noqa: E402
|
||||
from typing import TextIO # noqa: E402
|
||||
|
||||
from one_memory.config import load_settings # noqa: E402
|
||||
@@ -68,6 +70,9 @@ def migration_failure_code(
|
||||
return "sdk_migration_disk_space_insufficient"
|
||||
for item in chain:
|
||||
code = getattr(item, "code", "")
|
||||
# `sdk_data_plane_write_job_input_missing:<job_id>`: the name is the
|
||||
# code; the id is for the log, not for the dialog.
|
||||
code = code.split(":", 1)[0] if isinstance(code, str) else code
|
||||
if not isinstance(code, str) or not _SAFE_MIGRATION_CODE.match(code):
|
||||
continue
|
||||
if code.startswith(("sdk_data_plane_", "sdk_migration_")):
|
||||
@@ -103,12 +108,8 @@ def run(arguments: Sequence[str], *, output: TextIO | None = None) -> int:
|
||||
if not arguments or list(arguments) in (
|
||||
["serve"],
|
||||
["serve", "--maintenance"],
|
||||
["serve", "--reader"],
|
||||
):
|
||||
serve(
|
||||
maintenance=list(arguments) == ["serve", "--maintenance"],
|
||||
reader=list(arguments) == ["serve", "--reader"],
|
||||
)
|
||||
serve(maintenance=list(arguments) == ["serve", "--maintenance"])
|
||||
return 0
|
||||
command, *options = arguments
|
||||
if command == "restore":
|
||||
@@ -117,29 +118,17 @@ def run(arguments: Sequence[str], *, output: TextIO | None = None) -> int:
|
||||
return sdk_migrate(options, stream)
|
||||
if command == "sdk-migration-status":
|
||||
return sdk_migration_status(options, stream)
|
||||
if command == "boundary" and not options:
|
||||
from one_memory.workbuddy_boundary import write_hook
|
||||
|
||||
return write_hook(stream, sys.stdin)
|
||||
if command == "boundary" and options == ["--peek"]:
|
||||
from one_memory.workbuddy_boundary import write_peek
|
||||
|
||||
return write_peek(stream, sys.stdin)
|
||||
if command == "boundary" and len(options) == 2 and options[0] == "--ack":
|
||||
from one_memory.workbuddy_boundary import run_confirm
|
||||
|
||||
return run_confirm(options[1])
|
||||
if command == "capture" and len(options) == 1:
|
||||
from one_memory.live_capture import capture_stdio
|
||||
|
||||
return capture_stdio(options[0], sys.stdin, stream)
|
||||
if command not in {"status", "doctor"} or options not in ([], ["--json"]):
|
||||
stream.write(
|
||||
"Usage: one-memory [serve [--maintenance|--reader]|status [--json]|doctor [--json]"
|
||||
"Usage: one-memory [serve [--maintenance]|status [--json]|doctor [--json]"
|
||||
"|restore --source <managed-backup> --yes [--json]"
|
||||
"|sdk-migrate --runtime-build <build> --yes [--json]"
|
||||
"|sdk-migration-status [--json]"
|
||||
"|boundary [--peek|--ack TOKEN]|capture SOURCE]\n"
|
||||
"|capture SOURCE]\n"
|
||||
)
|
||||
return 2
|
||||
|
||||
@@ -312,7 +301,26 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
|
||||
stream.write("one-memory: sdk-migrate 需要 --runtime-build <build>\n")
|
||||
return 2
|
||||
|
||||
from one_memory.sdk_migration_log import authority_snapshot, write_migration_log
|
||||
|
||||
settings = load_settings()
|
||||
database_path = settings.data_dir / "one-memory.sqlite3"
|
||||
started = time.monotonic()
|
||||
write_migration_log(
|
||||
settings.data_dir,
|
||||
writer="core-sdk-migrate",
|
||||
event="migrate.invoked",
|
||||
outcome="started",
|
||||
details={
|
||||
"runtime_build": runtime_build,
|
||||
"confirmed": confirmed,
|
||||
# The Runtime the upgrade replaced, as the orchestrator read it
|
||||
# from that Runtime's install manifest; never guessed.
|
||||
"source_version": _migration_source("ONE_MEMORY_MIGRATION_SOURCE_VERSION"),
|
||||
"source_build": _migration_source("ONE_MEMORY_MIGRATION_SOURCE_BUILD"),
|
||||
**authority_snapshot(database_path),
|
||||
},
|
||||
)
|
||||
status = inspect_runtime(settings.data_dir)
|
||||
if status.state != "stopped":
|
||||
document = {
|
||||
@@ -321,6 +329,14 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
|
||||
"code": "sdk_migration_runtime_running",
|
||||
"runtime_state": status.state,
|
||||
}
|
||||
write_migration_log(
|
||||
settings.data_dir,
|
||||
writer="core-sdk-migrate",
|
||||
event="migrate.blocked",
|
||||
outcome="blocked",
|
||||
code="sdk_migration_runtime_running",
|
||||
details={"runtime_state": status.state},
|
||||
)
|
||||
_write_restore(stream, as_json, document, "Runtime 仍在运行;不能迁移。")
|
||||
return 4
|
||||
if not confirmed:
|
||||
@@ -328,7 +344,13 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
|
||||
_write_restore(stream, as_json, document, "预览通过;加 --yes 后执行迁移。")
|
||||
return 4
|
||||
|
||||
from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE, HY_MEMORY_VERSION
|
||||
from one_memory.hy_runtime_config import (
|
||||
HY_MEMORY_NAMESPACE,
|
||||
HY_MEMORY_VERSION,
|
||||
configure_hy_runtime_environment,
|
||||
)
|
||||
|
||||
configure_hy_runtime_environment(settings.data_dir)
|
||||
from one_memory.sdk_migration_runner import run_startup_migration
|
||||
|
||||
step_names = {
|
||||
@@ -340,23 +362,45 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
|
||||
"cleanup": "正在删除旧表……",
|
||||
}
|
||||
|
||||
# Everything before the first progress step (authority read, profile
|
||||
# read, resumable identity, legacy id scan, model-profile check) is the
|
||||
# preflight. A failure is logged with the phase it happened in.
|
||||
phase = {"name": "preflight", "started": time.monotonic()}
|
||||
|
||||
def progress(step: str) -> None:
|
||||
now = time.monotonic()
|
||||
write_migration_log(
|
||||
settings.data_dir,
|
||||
writer="core-sdk-migrate",
|
||||
event="phase.finished",
|
||||
outcome="succeeded",
|
||||
details={"phase": phase["name"], "duration_ms": (now - phase["started"]) * 1000},
|
||||
)
|
||||
phase["name"], phase["started"] = step, now
|
||||
write_migration_log(
|
||||
settings.data_dir,
|
||||
writer="core-sdk-migrate",
|
||||
event="phase.started",
|
||||
outcome="started",
|
||||
details={"phase": step},
|
||||
)
|
||||
if as_json:
|
||||
stream.write(
|
||||
json.dumps({"schema_version": 1, "state": "running", "step": step}) + "\n"
|
||||
)
|
||||
stream.write(json.dumps({"schema_version": 1, "state": "running", "step": step}) + "\n")
|
||||
else:
|
||||
stream.write(f"{step_names.get(step, step)}\n")
|
||||
stream.flush()
|
||||
|
||||
restore: dict[str, object] = {}
|
||||
try:
|
||||
result = run_startup_migration(
|
||||
data_dir=settings.data_dir,
|
||||
runtime_build=runtime_build,
|
||||
sdk_version=HY_MEMORY_VERSION,
|
||||
namespace=HY_MEMORY_NAMESPACE,
|
||||
progress=progress,
|
||||
)
|
||||
with _migration_lock(settings.data_dir):
|
||||
result = run_startup_migration(
|
||||
data_dir=settings.data_dir,
|
||||
runtime_build=runtime_build,
|
||||
sdk_version=HY_MEMORY_VERSION,
|
||||
namespace=HY_MEMORY_NAMESPACE,
|
||||
progress=progress,
|
||||
restore_report=restore,
|
||||
)
|
||||
except Exception as error:
|
||||
from one_memory.sdk_migration_verify import MemoryReadbackFailed
|
||||
|
||||
@@ -364,27 +408,176 @@ def sdk_migrate(options: Sequence[str], stream: TextIO) -> int:
|
||||
failed: dict[str, object] = {"schema_version": 1, "state": "failed", "code": code}
|
||||
if isinstance(error, MemoryReadbackFailed):
|
||||
failed["readback"] = error.report.as_document()
|
||||
_write_restore(stream, as_json, failed, f"迁移未完成:{code}")
|
||||
# The migration's own code stays the result; the restore is secondary.
|
||||
failed["data_restored"] = restore.get("state") == "restored"
|
||||
if restore.get("state") == "failed":
|
||||
failed["restore_code"] = restore.get("code")
|
||||
write_migration_log(
|
||||
settings.data_dir,
|
||||
writer="core-sdk-migrate",
|
||||
event="migrate.failed",
|
||||
outcome="failed",
|
||||
code=code,
|
||||
error=error,
|
||||
details={
|
||||
"phase": phase["name"],
|
||||
"phase_duration_ms": (time.monotonic() - phase["started"]) * 1000,
|
||||
"duration_ms": (time.monotonic() - started) * 1000,
|
||||
# A generic fallback code means the cause is only in `error`.
|
||||
"code_is_fallback": code == "sdk_migration_preflight_failed"
|
||||
and not _names_code(error, code),
|
||||
**({"readback": failed["readback"]} if "readback" in failed else {}),
|
||||
"restore": restore.get("state", "not_attempted"),
|
||||
**({"restore_code": restore["code"]} if "code" in restore else {}),
|
||||
**authority_snapshot(database_path),
|
||||
},
|
||||
)
|
||||
restored_text = ";数据已恢复到升级前的状态" if failed["data_restored"] else ""
|
||||
_write_restore(stream, as_json, failed, f"迁移未完成:{code}{restored_text}")
|
||||
return 1
|
||||
document = result.as_document()
|
||||
write_migration_log(
|
||||
settings.data_dir,
|
||||
writer="core-sdk-migrate",
|
||||
event="migrate.finished",
|
||||
outcome="succeeded",
|
||||
migration_id=result.migration_id,
|
||||
details={
|
||||
"state": result.state,
|
||||
"last_phase": phase["name"],
|
||||
"duration_ms": (time.monotonic() - started) * 1000,
|
||||
"result": document,
|
||||
**authority_snapshot(database_path),
|
||||
},
|
||||
)
|
||||
messages = {
|
||||
"not_required": "不需要迁移。",
|
||||
"complete": "迁移已完成;遗留的旧表或向量已处理。",
|
||||
"cutover": "迁移完成:记忆数据已以 SDK 为准。",
|
||||
}
|
||||
_write_restore(stream, as_json, result.as_document(), messages[result.state])
|
||||
_write_restore(stream, as_json, document, messages[result.state])
|
||||
return 0
|
||||
|
||||
|
||||
_SAFE_SOURCE = re.compile(r"^[0-9A-Za-z][0-9A-Za-z.+\-]{0,63}$")
|
||||
|
||||
|
||||
def _migration_source(name: str) -> str:
|
||||
value = os.environ.get(name, "").strip()
|
||||
return value if _SAFE_SOURCE.fullmatch(value) else "unknown"
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def _migration_lock(data_dir: Path) -> Iterator[None]:
|
||||
"""One `sdk-migrate` per home at a time.
|
||||
|
||||
Desktop, CLI install/upgrade and the Runtime service each migrate before
|
||||
serving, and a login-started service can be migrating when the person
|
||||
opens the app. The second waits here and then finds the work done (or
|
||||
resumes it); two runs never interleave.
|
||||
"""
|
||||
from one_memory.sdk_migration_log import write_migration_log
|
||||
|
||||
data_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||
descriptor = os.open(data_dir / "sdk-migration.lock", os.O_RDWR | os.O_CREAT, 0o600)
|
||||
try:
|
||||
if os.name == "nt":
|
||||
import msvcrt
|
||||
|
||||
def acquire(blocking: bool) -> None:
|
||||
while True:
|
||||
try:
|
||||
os.lseek(descriptor, 0, os.SEEK_SET)
|
||||
msvcrt.locking(descriptor, msvcrt.LK_NBLCK, 1)
|
||||
return
|
||||
except OSError:
|
||||
if not blocking:
|
||||
raise
|
||||
time.sleep(0.5)
|
||||
|
||||
def release() -> None:
|
||||
os.lseek(descriptor, 0, os.SEEK_SET)
|
||||
msvcrt.locking(descriptor, msvcrt.LK_UNLCK, 1)
|
||||
else:
|
||||
import fcntl
|
||||
|
||||
def acquire(blocking: bool) -> None:
|
||||
fcntl.flock(descriptor, fcntl.LOCK_EX | (0 if blocking else fcntl.LOCK_NB))
|
||||
|
||||
def release() -> None:
|
||||
fcntl.flock(descriptor, fcntl.LOCK_UN)
|
||||
|
||||
try:
|
||||
acquire(False)
|
||||
except OSError:
|
||||
write_migration_log(
|
||||
data_dir,
|
||||
writer="core-sdk-migrate",
|
||||
event="migrate.waiting_for_lock",
|
||||
outcome="started",
|
||||
)
|
||||
waited = time.monotonic()
|
||||
acquire(True)
|
||||
write_migration_log(
|
||||
data_dir,
|
||||
writer="core-sdk-migrate",
|
||||
event="migrate.lock_acquired",
|
||||
outcome="succeeded",
|
||||
details={"waited_ms": (time.monotonic() - waited) * 1000},
|
||||
)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
release()
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
|
||||
def _names_code(error: BaseException, code: str) -> bool:
|
||||
return any(
|
||||
str(getattr(item, "code", "")).split(":", 1)[0] == code for item in _exception_chain(error)
|
||||
)
|
||||
|
||||
|
||||
def sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
|
||||
"""Read-only probe: does this home need `sdk-migrate`?"""
|
||||
try:
|
||||
return _sdk_migration_status(options, stream)
|
||||
except Exception as error:
|
||||
from one_memory.sdk_migration_log import write_migration_log
|
||||
|
||||
# The orchestrator only sees a non-zero exit and reports
|
||||
# `sdk_migration_status_unavailable`; the cause is recorded here.
|
||||
write_migration_log(
|
||||
load_settings().data_dir,
|
||||
writer="core-sdk-migration-status",
|
||||
event="status.failed",
|
||||
outcome="failed",
|
||||
code="sdk_migration_status_unavailable",
|
||||
error=error,
|
||||
)
|
||||
raise
|
||||
|
||||
|
||||
def _sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
|
||||
if list(options) not in ([], ["--json"]):
|
||||
stream.write("one-memory: sdk-migration-status 只接受 --json\n")
|
||||
return 2
|
||||
as_json = "--json" in options
|
||||
settings = load_settings()
|
||||
database_path = settings.data_dir / "one-memory.sqlite3"
|
||||
if not database_path.is_file():
|
||||
from one_memory.sdk_migration_restore import restore_pending
|
||||
|
||||
if restore_pending(settings.data_dir):
|
||||
# A restore was killed between its renames; the database may be
|
||||
# elsewhere. Only `sdk-migrate` may finish it.
|
||||
document: dict[str, object] = {
|
||||
"schema_version": 1,
|
||||
"authority": "legacy",
|
||||
"state": "required",
|
||||
}
|
||||
elif not database_path.is_file():
|
||||
document = {
|
||||
"schema_version": 1,
|
||||
"authority": "sdk",
|
||||
"state": "not_required",
|
||||
@@ -409,15 +602,21 @@ def sdk_migration_status(options: Sequence[str], stream: TextIO) -> int:
|
||||
data_dir=settings.data_dir,
|
||||
namespace=HY_MEMORY_NAMESPACE,
|
||||
)
|
||||
or (
|
||||
profile is not None
|
||||
and legacy_vector_collections(settings.data_dir, profile)
|
||||
)
|
||||
or (profile is not None and legacy_vector_collections(settings.data_dir, profile))
|
||||
)
|
||||
state = "required" if leftovers else "not_required"
|
||||
else:
|
||||
state = "required"
|
||||
document = {"schema_version": 1, "authority": authority.value, "state": state}
|
||||
from one_memory.sdk_migration_log import write_migration_log
|
||||
|
||||
write_migration_log(
|
||||
settings.data_dir,
|
||||
writer="core-sdk-migration-status",
|
||||
event="status.probed",
|
||||
outcome="succeeded",
|
||||
details={"authority": document["authority"], "state": document["state"]},
|
||||
)
|
||||
if as_json:
|
||||
stream.write(f"{json.dumps(document, ensure_ascii=False)}\n")
|
||||
else:
|
||||
@@ -437,30 +636,72 @@ def _status_text(command: str, status: RuntimeStatus) -> str:
|
||||
return f"MemHub Runtime 状态不可验证:{status.code}。"
|
||||
|
||||
|
||||
def _reserve_reader_port(host: str, port: int) -> int:
|
||||
"""Resolve the port the Reader will listen on, including the ephemeral case.
|
||||
#: Exit status of a serving Core refused because the home is not on SDK
|
||||
#: authority (`EX_CONFIG`). The Runtime maps it to `sdk_migration_required`.
|
||||
SDK_MIGRATION_REQUIRED_EXIT = 78
|
||||
|
||||
The supervisor asks for an ephemeral port (`ONE_MEMORY_READER_PORT=0`) and
|
||||
then reads the real one back from stderr. Letting uvicorn bind port 0
|
||||
itself would give a port nobody can report, so claim a free port here by
|
||||
binding and immediately releasing it, then serve on that exact port.
|
||||
|
||||
def _refuse_unmigrated_home(data_dir: Path) -> None:
|
||||
"""Exit before binding when an existing home has not reached SDK authority.
|
||||
|
||||
A brand-new home has no database yet and is created on SDK authority by
|
||||
startup. Anything else that is not SDK is an upgrade that did not finish,
|
||||
and is never served: the migration must run first.
|
||||
"""
|
||||
if port != 0:
|
||||
return port
|
||||
probe = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
from one_memory.sdk_data_plane_migration import MemoryAuthority, SdkDataPlaneMigrationStore
|
||||
from one_memory.sdk_migration_log import authority_snapshot, write_migration_log
|
||||
from one_memory.sdk_migration_restore import restore_pending
|
||||
|
||||
database_path = data_dir / "one-memory.sqlite3"
|
||||
if restore_pending(data_dir):
|
||||
# A killed restore may have moved the database aside: serving now
|
||||
# would create an empty home over the user's data.
|
||||
write_migration_log(
|
||||
data_dir,
|
||||
writer="core-startup",
|
||||
event="core.refused_legacy",
|
||||
outcome="refused",
|
||||
code="sdk_migration_required",
|
||||
details={"stage": "serve_entry", "restore_pending": True},
|
||||
)
|
||||
print("sdk_migration_required", file=sys.stderr, flush=True)
|
||||
raise SystemExit(SDK_MIGRATION_REQUIRED_EXIT)
|
||||
if not database_path.is_file():
|
||||
return
|
||||
try:
|
||||
probe.bind((host, 0))
|
||||
return int(probe.getsockname()[1])
|
||||
finally:
|
||||
probe.close()
|
||||
authority = SdkDataPlaneMigrationStore(database_path).authority()
|
||||
except Exception as error: # noqa: BLE001 - an unreadable home is not served either
|
||||
write_migration_log(
|
||||
data_dir,
|
||||
writer="core-startup",
|
||||
event="core.refused_legacy",
|
||||
outcome="refused",
|
||||
code="sdk_migration_required",
|
||||
error=error,
|
||||
)
|
||||
print("sdk_migration_required", file=sys.stderr, flush=True)
|
||||
raise SystemExit(SDK_MIGRATION_REQUIRED_EXIT) from None
|
||||
if authority is MemoryAuthority.SDK:
|
||||
return
|
||||
write_migration_log(
|
||||
data_dir,
|
||||
writer="core-startup",
|
||||
event="core.refused_legacy",
|
||||
outcome="refused",
|
||||
code="sdk_migration_required",
|
||||
details={"stage": "serve_entry", **authority_snapshot(database_path)},
|
||||
)
|
||||
print("sdk_migration_required", file=sys.stderr, flush=True)
|
||||
raise SystemExit(SDK_MIGRATION_REQUIRED_EXIT)
|
||||
|
||||
|
||||
def serve(*, maintenance: bool = False, reader: bool = False) -> None:
|
||||
def serve(*, maintenance: bool = False) -> None:
|
||||
import uvicorn
|
||||
|
||||
from one_memory.diagnostic_log import configure_diagnostics, diagnostic_event
|
||||
from one_memory.hy_runtime_config import (
|
||||
apply_diagnostic_log_level,
|
||||
configure_hy_runtime_environment,
|
||||
diagnostic_log_level,
|
||||
)
|
||||
from one_memory.log_file import start_logging_to_file
|
||||
@@ -468,40 +709,13 @@ def serve(*, maintenance: bool = False, reader: bool = False) -> None:
|
||||
from one_memory.runtime import CoreRuntime, parent_pid
|
||||
|
||||
settings = load_settings()
|
||||
if reader:
|
||||
settings = settings.model_copy(
|
||||
update={
|
||||
"port": int(os.environ.get("ONE_MEMORY_READER_PORT", "0") or "0"),
|
||||
"api_token": settings.api_token,
|
||||
}
|
||||
)
|
||||
from one_memory.reader_app import ReaderSettings, create_reader_app
|
||||
|
||||
core_port = int(os.environ.get("ONE_MEMORY_CORE_PORT", "8765"))
|
||||
# The supervisor learns the Reader's port by watching stderr for
|
||||
# `reader_port:<n>`. With `ONE_MEMORY_READER_PORT=0` uvicorn would bind
|
||||
# an OS-chosen port that nothing reports back, so claim the port up
|
||||
# front and serve on that exact one.
|
||||
port = _reserve_reader_port(settings.host, settings.port)
|
||||
reader_settings = ReaderSettings(
|
||||
data_dir=settings.data_dir,
|
||||
host=settings.host,
|
||||
port=port,
|
||||
api_token=settings.api_token,
|
||||
core_port=core_port,
|
||||
core_api_token=os.environ.get("ONE_MEMORY_CORE_API_TOKEN", ""),
|
||||
)
|
||||
application = create_reader_app(reader_settings)
|
||||
print(f"reader_port:{port}", file=sys.stderr, flush=True)
|
||||
uvicorn.run(
|
||||
application,
|
||||
host=settings.host,
|
||||
port=port,
|
||||
reload=False,
|
||||
access_log=False,
|
||||
)
|
||||
return
|
||||
|
||||
if not maintenance:
|
||||
_refuse_unmigrated_home(settings.data_dir)
|
||||
# HY resolves some trace paths at import time and its pipeline path when
|
||||
# SharedRuntime starts. Bind both to the MemHub root before importing the
|
||||
# application graph; otherwise HY falls back to ~/.hy_memory while the
|
||||
# desktop Debug panel watches the managed data directory.
|
||||
configure_hy_runtime_environment(settings.data_dir)
|
||||
from one_memory.api import create_app
|
||||
|
||||
start_logging_to_file(settings.data_dir)
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
"""Authenticated composite read route, without generation or reindex side effects."""
|
||||
"""Authenticated SDK context search route, without generation or reindex side effects."""
|
||||
|
||||
import asyncio
|
||||
import sqlite3
|
||||
import time
|
||||
from collections.abc import Sequence
|
||||
from datetime import UTC, datetime
|
||||
@@ -11,37 +10,15 @@ from uuid import uuid4
|
||||
from fastapi import APIRouter, HTTPException, Request, Response
|
||||
from fastapi.params import Depends
|
||||
|
||||
from one_memory.background_work import (
|
||||
in_interactive_read,
|
||||
in_read_pool,
|
||||
memory_read_pool,
|
||||
raw_read_pool,
|
||||
)
|
||||
from one_memory.call_event_recorder import context_call_event
|
||||
from one_memory.call_event_recorder import sdk_context_call_event
|
||||
from one_memory.context_access import resolve_context_authority
|
||||
from one_memory.context_memory import (
|
||||
merge_memory_candidates,
|
||||
read_memory_context,
|
||||
revalidate_memory_context,
|
||||
)
|
||||
from one_memory.context_models import (
|
||||
ContextItem,
|
||||
ContextKind,
|
||||
ContextLaneResult,
|
||||
ContextSearchRequest,
|
||||
ContextSearchResponse,
|
||||
ContextSemanticStatus,
|
||||
MemoryContextItem,
|
||||
RawContextItem,
|
||||
SdkContextItem,
|
||||
SdkContextSearchRequest,
|
||||
SdkContextSearchResponse,
|
||||
SdkMemoryContextItem,
|
||||
SdkRawContextItem,
|
||||
)
|
||||
from one_memory.context_raw import read_raw_context, revalidate_raw_context
|
||||
from one_memory.context_search import search_context
|
||||
from one_memory.diagnostic_log import diagnostic_event, diagnostic_exception
|
||||
from one_memory.external_memory_access import external_access
|
||||
from one_memory.hy_runtime_config import HY_MEMORY_READER
|
||||
from one_memory.hy_sdk_runtime import sdk_relevance_floor
|
||||
@@ -51,387 +28,8 @@ from one_memory.hy_sdk_types import (
|
||||
parse_sdk_search,
|
||||
)
|
||||
from one_memory.imports.storage import EvidenceStore
|
||||
from one_memory.semantic_deadline import hy_wait_budget, search_within
|
||||
|
||||
CALL_EVENT_RESPONSE_BUDGET_MS = 250
|
||||
RESPONSE_SERIALIZATION_RESERVE_MS = 100
|
||||
INTERACTIVE_TOTAL_BUDGET_MS = 900
|
||||
RAW_LANE_BUDGET_SECONDS = 0.3
|
||||
MEMORY_LANE_BUDGET_SECONDS = 0.75
|
||||
MEMORY_FINALIZE_RESERVE_SECONDS = 0.1
|
||||
|
||||
|
||||
def context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
router = APIRouter(prefix="/api/v1/context", dependencies=dependencies)
|
||||
|
||||
@router.post("/search", response_model=ContextSearchResponse)
|
||||
async def search(
|
||||
value: ContextSearchRequest, request: Request, response: Response
|
||||
) -> ContextSearchResponse:
|
||||
response.headers["Cache-Control"] = "no-store"
|
||||
started = time.monotonic()
|
||||
search_budget_ms = min(
|
||||
INTERACTIVE_TOTAL_BUDGET_MS,
|
||||
max(50, value.timeout_ms - RESPONSE_SERIALIZATION_RESERVE_MS),
|
||||
)
|
||||
deadline = started + search_budget_ms / 1000
|
||||
access = external_access(request.headers)
|
||||
try:
|
||||
database = cast(EvidenceStore, request.app.state.evidence_store).database_path
|
||||
authority = await asyncio.wait_for(
|
||||
in_interactive_read(
|
||||
resolve_context_authority, value, access, database, deadline=deadline
|
||||
),
|
||||
timeout=max(0, deadline - time.monotonic()),
|
||||
)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception:
|
||||
# Authority could not be established; do not try an unbounded local lane.
|
||||
raise HTTPException(503, "context_authority_unavailable") from None
|
||||
|
||||
lane_observations: dict[ContextKind, ContextLaneResult] = {}
|
||||
lane_failures: dict[ContextKind, tuple[str, str]] = {}
|
||||
stage_timings: dict[str, float] = {}
|
||||
|
||||
async def raw(end: float) -> ContextLaneResult:
|
||||
lane_end = min(end, time.monotonic() + RAW_LANE_BUDGET_SECONDS)
|
||||
read = asyncio.create_task(
|
||||
in_read_pool(
|
||||
raw_read_pool(),
|
||||
read_raw_context,
|
||||
database,
|
||||
value,
|
||||
source_ids=authority.raw_sources,
|
||||
deadline=lane_end,
|
||||
)
|
||||
)
|
||||
done, _ = await asyncio.wait({read}, timeout=max(0, lane_end - time.monotonic()))
|
||||
if read not in done:
|
||||
read.cancel()
|
||||
read.add_done_callback(
|
||||
lambda task: task.exception() if not task.cancelled() else None
|
||||
)
|
||||
result = ContextLaneResult(
|
||||
status="degraded",
|
||||
reason="timeout_budget",
|
||||
searched_channels=("lexical", "semantic"),
|
||||
timed_out=True,
|
||||
truncated_by=("timeout_budget",),
|
||||
)
|
||||
else:
|
||||
result = read.result()
|
||||
lane_observations["raw"] = result
|
||||
return result
|
||||
|
||||
async def memory(end: float) -> ContextLaneResult:
|
||||
lane_end = min(end, time.monotonic() + MEMORY_LANE_BUDGET_SECONDS)
|
||||
target = getattr(request.app.state, "recall_target", None)
|
||||
lexical = asyncio.create_task(
|
||||
in_read_pool(
|
||||
memory_read_pool(),
|
||||
read_memory_context,
|
||||
database,
|
||||
value,
|
||||
scopes=authority.memory_scopes,
|
||||
deadline=lane_end,
|
||||
semantic_ids=(),
|
||||
semantic_outcome="absent",
|
||||
)
|
||||
)
|
||||
semantic_hits, outcome = await search_within(
|
||||
getattr(target, "semantic_search", None),
|
||||
value.query,
|
||||
hy_wait_budget(lane_end - time.monotonic()),
|
||||
)
|
||||
done, _ = await asyncio.wait({lexical}, timeout=max(0, lane_end - time.monotonic()))
|
||||
if lexical not in done:
|
||||
lexical.cancel()
|
||||
lexical.add_done_callback(
|
||||
lambda task: task.exception() if not task.cancelled() else None
|
||||
)
|
||||
result = ContextLaneResult(
|
||||
status="unavailable",
|
||||
reason="timeout_budget",
|
||||
searched_channels=("lexical", "semantic"),
|
||||
semantic_status=cast(
|
||||
ContextSemanticStatus,
|
||||
{
|
||||
"absent": "not_searched",
|
||||
"answered": "completed",
|
||||
"failed": "failed",
|
||||
"timed_out": "timed_out",
|
||||
}[outcome],
|
||||
),
|
||||
timed_out=True,
|
||||
truncated_by=("timeout_budget",),
|
||||
)
|
||||
lane_observations["memory"] = result
|
||||
return result
|
||||
try:
|
||||
lexical_result = lexical.result()
|
||||
except (sqlite3.Error, OSError, ValueError) as error:
|
||||
diagnostic_event(
|
||||
level="warn",
|
||||
component="context.search",
|
||||
event="context.lexical_memory_failed",
|
||||
outcome="degraded",
|
||||
error_code="lexical_memory_read_failed",
|
||||
**diagnostic_exception(error),
|
||||
)
|
||||
lexical_result = ContextLaneResult(
|
||||
status="unavailable",
|
||||
reason="read_unavailable",
|
||||
searched_channels=("lexical",),
|
||||
failure_site="memory_lexical",
|
||||
error_class=type(error).__name__,
|
||||
)
|
||||
if outcome == "absent":
|
||||
result = lexical_result
|
||||
elif outcome in {"failed", "timed_out"}:
|
||||
truncated_by = list(lexical_result.truncated_by)
|
||||
has_lexical_memory = bool(lexical_result.items)
|
||||
if (
|
||||
outcome == "timed_out"
|
||||
and not has_lexical_memory
|
||||
and "timeout_budget" not in truncated_by
|
||||
):
|
||||
truncated_by.append("timeout_budget")
|
||||
result = lexical_result.model_copy(
|
||||
update={
|
||||
"status": "degraded",
|
||||
"reason": "lexical_only" if has_lexical_memory else "timeout_budget",
|
||||
"truncated_by": tuple(truncated_by),
|
||||
"searched_channels": ("lexical", "semantic"),
|
||||
"semantic_status": "timed_out" if outcome == "timed_out" else "failed",
|
||||
"timed_out": outcome == "timed_out" and not has_lexical_memory,
|
||||
}
|
||||
)
|
||||
else:
|
||||
semantic_request = value.model_copy(update={"query": ""})
|
||||
enriched = asyncio.create_task(
|
||||
in_interactive_read(
|
||||
read_memory_context,
|
||||
database,
|
||||
semantic_request,
|
||||
scopes=authority.memory_scopes,
|
||||
deadline=lane_end,
|
||||
semantic_hits=semantic_hits,
|
||||
semantic_outcome=outcome,
|
||||
)
|
||||
)
|
||||
done, _ = await asyncio.wait(
|
||||
{enriched},
|
||||
timeout=max(
|
||||
0,
|
||||
lane_end - MEMORY_FINALIZE_RESERVE_SECONDS - time.monotonic(),
|
||||
),
|
||||
)
|
||||
if enriched not in done:
|
||||
enriched.cancel()
|
||||
enriched.add_done_callback(
|
||||
lambda task: task.exception() if not task.cancelled() else None
|
||||
)
|
||||
result = lexical_result.model_copy(
|
||||
update={
|
||||
"status": "degraded",
|
||||
"reason": "lexical_only",
|
||||
"searched_channels": ("lexical", "semantic"),
|
||||
"semantic_status": "completed",
|
||||
"timed_out": False,
|
||||
}
|
||||
)
|
||||
else:
|
||||
try:
|
||||
semantic_result = enriched.result()
|
||||
except (sqlite3.Error, OSError, ValueError) as error:
|
||||
diagnostic_event(
|
||||
level="warn",
|
||||
component="context.search",
|
||||
event="context.semantic_enrichment_failed",
|
||||
outcome="degraded",
|
||||
error_code="semantic_enrichment_read_failed",
|
||||
**diagnostic_exception(error),
|
||||
)
|
||||
result = lexical_result.model_copy(
|
||||
update={
|
||||
"status": "degraded",
|
||||
"reason": "lexical_only",
|
||||
"searched_channels": ("lexical", "semantic"),
|
||||
"semantic_status": "completed",
|
||||
"timed_out": False,
|
||||
}
|
||||
)
|
||||
lane_observations["memory"] = result
|
||||
return result
|
||||
if semantic_result.status == "unavailable" and lexical_result.items:
|
||||
result = lexical_result.model_copy(
|
||||
update={
|
||||
"status": "degraded",
|
||||
"reason": "lexical_only",
|
||||
"searched_channels": ("lexical", "semantic"),
|
||||
"semantic_status": "completed",
|
||||
"timed_out": False,
|
||||
}
|
||||
)
|
||||
lane_observations["memory"] = result
|
||||
return result
|
||||
semantic_items = tuple(
|
||||
item
|
||||
for item in semantic_result.items
|
||||
if isinstance(item, MemoryContextItem)
|
||||
)
|
||||
lexical_items = tuple(
|
||||
item for item in lexical_result.items if isinstance(item, MemoryContextItem)
|
||||
)
|
||||
combined_count = len(
|
||||
{item.memory_id for item in (*semantic_items, *lexical_items)}
|
||||
)
|
||||
merged = merge_memory_candidates(
|
||||
lexical_items,
|
||||
semantic_items,
|
||||
limit=value.limit + 1,
|
||||
)
|
||||
truncated_by = list(semantic_result.truncated_by)
|
||||
if combined_count > len(merged) and "count_budget" not in truncated_by:
|
||||
truncated_by.append("count_budget")
|
||||
result = semantic_result.model_copy(
|
||||
update={
|
||||
"items": tuple(merged),
|
||||
# An answered semantic search may legitimately
|
||||
# return no IDs. Lexical can still have a grounded
|
||||
# hit; retaining semantic's `empty` status while
|
||||
# attaching that hit violates the lane contract
|
||||
# and makes the aggregator reject the whole lane.
|
||||
"status": "ok" if merged else semantic_result.status,
|
||||
"truncated_by": tuple(truncated_by),
|
||||
}
|
||||
)
|
||||
lane_observations["memory"] = result
|
||||
return result
|
||||
|
||||
async def finalizer(
|
||||
kind: ContextKind, items: tuple[ContextItem, ...], end: float
|
||||
) -> tuple[ContextItem, ...]:
|
||||
if kind == "raw":
|
||||
return await in_interactive_read(
|
||||
revalidate_raw_context,
|
||||
database,
|
||||
tuple(item for item in items if isinstance(item, RawContextItem)),
|
||||
source_ids=authority.raw_sources,
|
||||
deadline=end,
|
||||
)
|
||||
return await in_interactive_read(
|
||||
revalidate_memory_context,
|
||||
database,
|
||||
tuple(item for item in items if isinstance(item, MemoryContextItem)),
|
||||
scopes=authority.memory_scopes,
|
||||
deadline=end,
|
||||
)
|
||||
|
||||
result = await search_context(
|
||||
value,
|
||||
raw_reader=raw if authority.raw_allowed else None,
|
||||
memory_reader=memory if authority.memory_allowed else None,
|
||||
finalizer=finalizer,
|
||||
deadline=deadline,
|
||||
timing_observer=stage_timings.update,
|
||||
failure_observer=lambda kind, site, error_class: lane_failures.__setitem__(
|
||||
kind, (site, error_class)
|
||||
),
|
||||
)
|
||||
final = result.model_copy(update={"total_ms": (time.monotonic() - started) * 1000})
|
||||
serialization_started = time.monotonic()
|
||||
final.model_dump_json()
|
||||
stage_timings["serialization_ms"] = (time.monotonic() - serialization_started) * 1000
|
||||
for phase, duration in stage_timings.items():
|
||||
diagnostic_event(
|
||||
level="info",
|
||||
component="context.search",
|
||||
event="context_stage.completed",
|
||||
outcome="succeeded",
|
||||
request_id=final.request_id,
|
||||
duration_ms=max(0, round(duration)),
|
||||
details={"phase": phase},
|
||||
)
|
||||
for kind in ("raw", "memory"):
|
||||
path = getattr(final.paths, kind)
|
||||
observed = lane_observations.get(kind)
|
||||
failure = lane_failures.get(kind)
|
||||
lexical_hits = (
|
||||
sum(
|
||||
1
|
||||
for item in observed.items
|
||||
if isinstance(item, MemoryContextItem) and "lexical" in item.matched_by
|
||||
)
|
||||
if observed is not None and kind == "memory"
|
||||
else 0
|
||||
)
|
||||
semantic_hits = (
|
||||
sum(
|
||||
1
|
||||
for item in observed.items
|
||||
if isinstance(item, MemoryContextItem)
|
||||
and "governed_semantic" in item.matched_by
|
||||
)
|
||||
if observed is not None and kind == "memory"
|
||||
else 0
|
||||
)
|
||||
timed_out = path.reason == "timeout_budget" or (
|
||||
observed.timed_out if observed is not None else False
|
||||
)
|
||||
diagnostic_failure_site = failure[0] if failure is not None else None
|
||||
diagnostic_error_class = failure[1] if failure is not None else None
|
||||
if observed is not None:
|
||||
diagnostic_failure_site = diagnostic_failure_site or observed.failure_site
|
||||
diagnostic_error_class = diagnostic_error_class or observed.error_class
|
||||
diagnostic_event(
|
||||
level="warn" if path.status in {"degraded", "unavailable"} else "info",
|
||||
component="context.search",
|
||||
event="context_lane.completed",
|
||||
outcome=path.status,
|
||||
request_id=final.request_id,
|
||||
duration_ms=round(path.elapsed_ms),
|
||||
details={
|
||||
"operation_kind": kind,
|
||||
"mode": ",".join(observed.searched_channels) if observed else "none",
|
||||
"status": path.status,
|
||||
"index_complete": observed.index_complete if observed else None,
|
||||
**(
|
||||
{"lexical_hits": lexical_hits, "semantic_hits": semantic_hits}
|
||||
if kind == "memory"
|
||||
else {}
|
||||
),
|
||||
"reason": path.reason,
|
||||
"truncated": path.truncated,
|
||||
"timed_out": timed_out,
|
||||
**(
|
||||
{
|
||||
"failure_site": diagnostic_failure_site,
|
||||
"error_class": diagnostic_error_class,
|
||||
}
|
||||
if diagnostic_failure_site is not None
|
||||
and diagnostic_error_class is not None
|
||||
else {}
|
||||
),
|
||||
},
|
||||
)
|
||||
store = getattr(request.app.state, "call_event_store", None)
|
||||
if store is not None and value.query != "__om_probe__":
|
||||
event = context_call_event(final, access)
|
||||
try:
|
||||
await asyncio.to_thread(
|
||||
store.record,
|
||||
event,
|
||||
busy_timeout_ms=CALL_EVENT_RESPONSE_BUDGET_MS,
|
||||
)
|
||||
response.headers["X-One-Memory-Call-Recorded"] = "true"
|
||||
except Exception:
|
||||
response.headers["X-One-Memory-Call-Recorded"] = "false"
|
||||
request.app.state.call_event_recording_error = True
|
||||
return final
|
||||
|
||||
return router
|
||||
|
||||
|
||||
def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
@@ -550,6 +148,29 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
)
|
||||
return tuple(items)
|
||||
|
||||
async def record(final: SdkContextSearchResponse, *, timed_out: bool) -> None:
|
||||
# Same boundary as v2 ask: only a host's call is history. The
|
||||
# renderer's own searches are the person reading their library.
|
||||
store = getattr(request.app.state, "call_event_store", None)
|
||||
if store is None or access is None or value.query == "__om_probe__":
|
||||
return
|
||||
event = sdk_context_call_event(
|
||||
final,
|
||||
access,
|
||||
memory_searched=authority.memory_allowed,
|
||||
timed_out=timed_out,
|
||||
)
|
||||
try:
|
||||
await asyncio.to_thread(
|
||||
store.record,
|
||||
event,
|
||||
busy_timeout_ms=CALL_EVENT_RESPONSE_BUDGET_MS,
|
||||
)
|
||||
response.headers["X-One-Memory-Call-Recorded"] = "true"
|
||||
except Exception:
|
||||
response.headers["X-One-Memory-Call-Recorded"] = "false"
|
||||
request.app.state.call_event_recording_error = True
|
||||
|
||||
lanes = (raw_lane(), memory_lane())
|
||||
try:
|
||||
raw_items, memory_items = await asyncio.wait_for(
|
||||
@@ -557,7 +178,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
timeout=value.timeout_ms / 1000,
|
||||
)
|
||||
except TimeoutError:
|
||||
return SdkContextSearchResponse(
|
||||
final = SdkContextSearchResponse(
|
||||
request_id=str(uuid4()),
|
||||
state="unavailable",
|
||||
items=(),
|
||||
@@ -565,6 +186,8 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
truncated=True,
|
||||
total_ms=(time.monotonic() - started) * 1000,
|
||||
)
|
||||
await record(final, timed_out=True)
|
||||
return final
|
||||
partial = isinstance(raw_items, Exception) or isinstance(memory_items, Exception)
|
||||
raw_candidates = (
|
||||
() if isinstance(raw_items, Exception) else cast(tuple[SdkContextItem, ...], raw_items)
|
||||
@@ -596,7 +219,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
truncated = True
|
||||
selected.append(item)
|
||||
used += len(item.content)
|
||||
return SdkContextSearchResponse(
|
||||
final = SdkContextSearchResponse(
|
||||
request_id=str(uuid4()),
|
||||
state=("partial" if partial else "grounded" if selected else "empty"),
|
||||
items=tuple(selected),
|
||||
@@ -604,5 +227,7 @@ def sdk_context_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
truncated=truncated,
|
||||
total_ms=(time.monotonic() - started) * 1000,
|
||||
)
|
||||
await record(final, timed_out=False)
|
||||
return final
|
||||
|
||||
return router
|
||||
@@ -1,464 +0,0 @@
|
||||
"""Bounded two-lane orchestration; readers and finalizers own trusted read access."""
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from collections.abc import Awaitable, Callable, Mapping
|
||||
from dataclasses import dataclass, field
|
||||
from uuid import uuid4
|
||||
|
||||
from pydantic import TypeAdapter
|
||||
|
||||
from one_memory.context_models import (
|
||||
ContextItem,
|
||||
ContextKind,
|
||||
ContextLaneResult,
|
||||
ContextLimit,
|
||||
ContextPath,
|
||||
ContextPaths,
|
||||
ContextSearchRequest,
|
||||
ContextSearchResponse,
|
||||
MemoryContextItem,
|
||||
RawContextItem,
|
||||
)
|
||||
|
||||
Reader = Callable[[float], Awaitable[ContextLaneResult]]
|
||||
Finalizer = Callable[
|
||||
[ContextKind, tuple[ContextItem, ...], float], Awaitable[tuple[ContextItem, ...]]
|
||||
]
|
||||
TimingObserver = Callable[[Mapping[str, float]], None]
|
||||
FailureObserver = Callable[[ContextKind, str, str], None]
|
||||
_KINDS: tuple[ContextKind, ...] = ("raw", "memory")
|
||||
_ITEMS = TypeAdapter(tuple[ContextItem, ...])
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Lane:
|
||||
result: ContextLaneResult
|
||||
elapsed_ms: float = 0
|
||||
limits: list[ContextLimit] = field(default_factory=list)
|
||||
|
||||
|
||||
def _identity(item: ContextItem) -> tuple[str, ...]:
|
||||
if isinstance(item, RawContextItem):
|
||||
return (item.kind, item.source_id, item.conversation_id, item.anchor_evidence_id)
|
||||
return (item.kind, item.memory_id)
|
||||
|
||||
|
||||
def _validate_items(kind: ContextKind, items: tuple[ContextItem, ...]) -> None:
|
||||
if len(items) > 21 or any(item.kind != kind for item in items):
|
||||
raise ValueError("invalid lane items")
|
||||
for item in items:
|
||||
if isinstance(item, RawContextItem) and item.anchor_evidence_id not in {
|
||||
message.evidence_id for message in item.messages
|
||||
}:
|
||||
raise ValueError("missing raw anchor")
|
||||
|
||||
|
||||
async def _read(reader: Reader, kind: ContextKind, deadline: float) -> ContextLaneResult:
|
||||
value = await reader(deadline)
|
||||
# Revalidate even BaseModel instances: model_construct/model_copy bypass validation.
|
||||
result = ContextLaneResult.model_validate(value.model_dump())
|
||||
_validate_items(kind, result.items)
|
||||
if result.status not in ("ok", "empty", "degraded", "unavailable"):
|
||||
raise ValueError("reader cannot decide requested authority")
|
||||
if result.status in ("empty", "unavailable") and result.items:
|
||||
raise ValueError("non-result lane has items")
|
||||
return result
|
||||
|
||||
|
||||
async def _finalize(
|
||||
finalizer: Finalizer, kind: ContextKind, items: tuple[ContextItem, ...], deadline: float
|
||||
) -> tuple[ContextItem, ...]:
|
||||
value = await finalizer(kind, items, deadline)
|
||||
if not isinstance(value, tuple):
|
||||
raise ValueError("invalid finalization result")
|
||||
checked = _ITEMS.validate_python([item.model_dump() for item in value])
|
||||
_validate_items(kind, checked)
|
||||
if not {_identity(item) for item in checked} <= {_identity(item) for item in items}:
|
||||
raise ValueError("finalizer introduced an unselected identity")
|
||||
return checked
|
||||
|
||||
|
||||
async def _collect(
|
||||
jobs: dict[ContextKind, Awaitable[object]], deadline: float
|
||||
) -> tuple[
|
||||
dict[ContextKind, object],
|
||||
dict[ContextKind, BaseException],
|
||||
set[ContextKind],
|
||||
dict[ContextKind, float],
|
||||
]:
|
||||
"""Cancel and await every owned task on deadline, failure or caller cancellation."""
|
||||
|
||||
durations: dict[ContextKind, float] = {}
|
||||
|
||||
async def measured(kind: ContextKind, job: Awaitable[object]) -> object:
|
||||
started = time.monotonic()
|
||||
try:
|
||||
value = await job
|
||||
if time.monotonic() >= deadline:
|
||||
raise TimeoutError
|
||||
return value
|
||||
finally:
|
||||
durations[kind] = (time.monotonic() - started) * 1000
|
||||
|
||||
tasks = {kind: asyncio.create_task(measured(kind, job)) for kind, job in jobs.items()}
|
||||
values: dict[ContextKind, object] = {}
|
||||
failed: dict[ContextKind, BaseException] = {}
|
||||
timed_out: set[ContextKind] = set()
|
||||
try:
|
||||
if not tasks:
|
||||
return values, failed, timed_out, durations
|
||||
done, pending = await asyncio.wait(
|
||||
tasks.values(), timeout=max(0, deadline - time.monotonic())
|
||||
)
|
||||
for kind, task in tasks.items():
|
||||
if task in pending:
|
||||
timed_out.add(kind)
|
||||
elif task in done:
|
||||
if not task.cancelled() and isinstance(task.exception(), TimeoutError):
|
||||
timed_out.add(kind)
|
||||
elif task.cancelled() or task.exception() is not None:
|
||||
failed[kind] = task.exception() or asyncio.CancelledError()
|
||||
else:
|
||||
values[kind] = task.result()
|
||||
return values, failed, timed_out, durations
|
||||
finally:
|
||||
for task in tasks.values():
|
||||
if not task.done():
|
||||
task.cancel()
|
||||
if tasks:
|
||||
# Cooperative readers finish immediately and still get their
|
||||
# normal cleanup. A bridge that suppresses cancellation must not
|
||||
# hold a host response past its hard deadline while an underlying
|
||||
# SDK thread winds down.
|
||||
cleanup: asyncio.Future[list[object]] = asyncio.gather(
|
||||
*tasks.values(), return_exceptions=True
|
||||
)
|
||||
current = asyncio.current_task()
|
||||
if current is not None and current.cancelling():
|
||||
await cleanup
|
||||
else:
|
||||
try:
|
||||
await asyncio.wait_for(asyncio.shield(cleanup), timeout=0.05)
|
||||
except TimeoutError:
|
||||
cleanup.add_done_callback(_consume_cleanup)
|
||||
|
||||
|
||||
def _consume_cleanup(task: asyncio.Future[list[object]]) -> None:
|
||||
if not task.cancelled():
|
||||
task.exception()
|
||||
|
||||
|
||||
def _deduplicate(
|
||||
items: tuple[ContextItem, ...], limits: list[ContextLimit]
|
||||
) -> tuple[ContextItem, ...]:
|
||||
unique: dict[tuple[str, ...], ContextItem] = {}
|
||||
for item in items:
|
||||
key = _identity(item)
|
||||
previous = unique.get(key)
|
||||
if isinstance(previous, MemoryContextItem) and isinstance(item, MemoryContextItem):
|
||||
if item.revision != previous.revision:
|
||||
# Never graft older evidence onto a newer memory version.
|
||||
unique[key] = item if item.revision > previous.revision else previous
|
||||
continue
|
||||
evidence = tuple(dict.fromkeys((*previous.evidence_ids, *item.evidence_ids)))
|
||||
if len(evidence) > 128:
|
||||
limits.append("metadata_budget")
|
||||
unique[key] = previous.model_copy(
|
||||
update={
|
||||
"evidence_ids": evidence[:128],
|
||||
"truncated": previous.truncated or item.truncated or len(evidence) > 128,
|
||||
}
|
||||
)
|
||||
elif isinstance(previous, RawContextItem) and isinstance(item, RawContextItem):
|
||||
messages = {message.evidence_id: message for message in previous.messages}
|
||||
for message in item.messages:
|
||||
if (
|
||||
message.evidence_id not in messages
|
||||
or message.revision > messages[message.evidence_id].revision
|
||||
):
|
||||
messages[message.evidence_id] = message
|
||||
if len(messages) > 3:
|
||||
limits.append("metadata_budget")
|
||||
# The first window already contains the anchor; dict order preserves it.
|
||||
unique[key] = previous.model_copy(
|
||||
update={
|
||||
"messages": tuple(messages.values())[:3],
|
||||
"truncated": previous.truncated or item.truncated or len(messages) > 3,
|
||||
}
|
||||
)
|
||||
else:
|
||||
unique[key] = item
|
||||
# Reserve each raw anchor for its own window before deduplicating context messages.
|
||||
anchors = {
|
||||
(item.source_id, item.conversation_id, item.anchor_evidence_id)
|
||||
for item in unique.values()
|
||||
if isinstance(item, RawContextItem)
|
||||
}
|
||||
seen: set[tuple[str, ...]] = set()
|
||||
result: list[ContextItem] = []
|
||||
for item in unique.values():
|
||||
if isinstance(item, MemoryContextItem):
|
||||
result.append(
|
||||
item.model_copy(update={"evidence_ids": tuple(dict.fromkeys(item.evidence_ids))})
|
||||
)
|
||||
continue
|
||||
retained_messages = []
|
||||
for message in item.messages:
|
||||
key = (item.source_id, item.conversation_id, message.evidence_id)
|
||||
if key in seen or (key in anchors and message.evidence_id != item.anchor_evidence_id):
|
||||
continue
|
||||
seen.add(key)
|
||||
retained_messages.append(message)
|
||||
result.append(item.model_copy(update={"messages": tuple(retained_messages)}))
|
||||
return tuple(result)
|
||||
|
||||
|
||||
def _characters(item: ContextItem) -> int:
|
||||
if isinstance(item, MemoryContextItem):
|
||||
return len(item.content)
|
||||
return sum(len(message.text) for message in item.messages)
|
||||
|
||||
|
||||
def _shares(lengths: list[int], budget: int) -> list[int]:
|
||||
"""Water-fill allocations so a large first item cannot consume every character."""
|
||||
allocations = [0] * len(lengths)
|
||||
pending = list(range(len(lengths)))
|
||||
while pending and budget:
|
||||
share, remainder = divmod(budget, len(pending))
|
||||
next_pending = []
|
||||
for position, index in enumerate(pending):
|
||||
added = min(lengths[index] - allocations[index], share + (position < remainder))
|
||||
allocations[index] += added
|
||||
budget -= added
|
||||
if allocations[index] < lengths[index]:
|
||||
next_pending.append(index)
|
||||
pending = next_pending
|
||||
return allocations
|
||||
|
||||
|
||||
def _clip(item: ContextItem, budget: int) -> ContextItem:
|
||||
if _characters(item) <= budget:
|
||||
return item
|
||||
if isinstance(item, MemoryContextItem):
|
||||
return item.model_copy(update={"content": item.content[:budget], "truncated": True})
|
||||
# The first raw message is the lexical anchor. Give it the first share of
|
||||
# the budget, then water-fill the adjacent context; otherwise a long
|
||||
# neighbor can consume the whole window and hide the answer.
|
||||
anchor = item.messages[0]
|
||||
anchor_budget = min(len(anchor.text), budget)
|
||||
remaining = budget - anchor_budget
|
||||
adjacent = list(item.messages[1:])
|
||||
shares = [anchor_budget, *_shares([len(message.text) for message in adjacent], remaining)]
|
||||
return item.model_copy(
|
||||
update={
|
||||
"truncated": True,
|
||||
"messages": tuple(
|
||||
message.model_copy(
|
||||
update={
|
||||
"text": message.text[:share],
|
||||
"truncated": message.truncated or len(message.text) > share,
|
||||
}
|
||||
)
|
||||
for message, share in zip(item.messages, shares, strict=True)
|
||||
),
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
async def search_context(
|
||||
request: ContextSearchRequest,
|
||||
*,
|
||||
raw_reader: Reader | None,
|
||||
memory_reader: Reader | None,
|
||||
finalizer: Finalizer,
|
||||
deadline: float | None = None,
|
||||
timing_observer: TimingObserver | None = None,
|
||||
failure_observer: FailureObserver | None = None,
|
||||
) -> ContextSearchResponse:
|
||||
request = ContextSearchRequest.model_validate(request.model_dump())
|
||||
started = time.monotonic()
|
||||
deadline = min(
|
||||
deadline if deadline is not None else float("inf"), started + request.timeout_ms / 1000
|
||||
)
|
||||
lanes: dict[ContextKind, _Lane] = {}
|
||||
jobs: dict[ContextKind, Awaitable[object]] = {}
|
||||
readers: dict[ContextKind, Reader | None] = {"raw": raw_reader, "memory": memory_reader}
|
||||
for kind, reader in readers.items():
|
||||
if kind not in request.kinds:
|
||||
lanes[kind] = _Lane(ContextLaneResult(status="not_requested", reason="not_requested"))
|
||||
elif reader is None:
|
||||
lanes[kind] = _Lane(
|
||||
ContextLaneResult(status="not_authorized", reason="permission_required")
|
||||
)
|
||||
elif time.monotonic() >= deadline:
|
||||
lanes[kind] = _Lane(
|
||||
ContextLaneResult(status="degraded", reason="timeout_budget", timed_out=True),
|
||||
limits=["timeout_budget"],
|
||||
)
|
||||
else:
|
||||
jobs[kind] = _read(reader, kind, deadline)
|
||||
values, failed, timed_out, read_ms = await _collect(jobs, deadline)
|
||||
for kind in jobs:
|
||||
result = values.get(kind)
|
||||
if kind in timed_out:
|
||||
result = ContextLaneResult(
|
||||
status="degraded",
|
||||
reason="timeout_budget",
|
||||
truncated_by=("timeout_budget",),
|
||||
timed_out=True,
|
||||
)
|
||||
elif kind in failed or not isinstance(result, ContextLaneResult):
|
||||
error = failed.get(kind)
|
||||
if failure_observer is not None:
|
||||
failure_observer(
|
||||
kind,
|
||||
"collect",
|
||||
type(error).__name__ if error is not None else "InvalidLaneResult",
|
||||
)
|
||||
result = ContextLaneResult(
|
||||
status="unavailable",
|
||||
reason="read_unavailable",
|
||||
failure_site="collect",
|
||||
error_class=type(error).__name__ if error is not None else "InvalidLaneResult",
|
||||
)
|
||||
lanes[kind] = _Lane(result, read_ms.get(kind, 0), list(dict.fromkeys(result.truncated_by)))
|
||||
|
||||
final_jobs: dict[ContextKind, Awaitable[object]] = {}
|
||||
for kind in jobs:
|
||||
lane = lanes[kind]
|
||||
if lane.result.status not in ("ok", "empty", "degraded"):
|
||||
continue
|
||||
if time.monotonic() >= deadline:
|
||||
lane.result = ContextLaneResult(
|
||||
status="degraded", reason="timeout_budget", timed_out=True
|
||||
)
|
||||
lane.limits.append("timeout_budget")
|
||||
else:
|
||||
final_jobs[kind] = _finalize(finalizer, kind, lane.result.items, deadline)
|
||||
final_values, final_failed, final_timed_out, revalidation_ms = await _collect(
|
||||
final_jobs, deadline
|
||||
)
|
||||
for kind in final_jobs:
|
||||
lane = lanes[kind]
|
||||
lane.elapsed_ms += revalidation_ms.get(kind, 0)
|
||||
if kind in final_timed_out:
|
||||
lane.result = ContextLaneResult(
|
||||
status="unavailable", reason="timeout_budget", timed_out=True
|
||||
)
|
||||
lane.limits.append("timeout_budget")
|
||||
elif kind in final_failed:
|
||||
error = final_failed[kind]
|
||||
if failure_observer is not None:
|
||||
failure_observer(kind, "finalize", type(error).__name__)
|
||||
lane.result = ContextLaneResult(
|
||||
status="unavailable",
|
||||
reason="read_unavailable",
|
||||
failure_site="finalize",
|
||||
error_class=type(error).__name__,
|
||||
)
|
||||
else:
|
||||
final_items = _ITEMS.validate_python(final_values[kind])
|
||||
changed = {_identity(item): item for item in lane.result.items} != {
|
||||
_identity(item): item for item in final_items
|
||||
}
|
||||
lane.result = lane.result.model_copy(
|
||||
update={
|
||||
"items": final_items,
|
||||
"status": "degraded" if changed else lane.result.status,
|
||||
"reason": ("evidence_changed" if kind == "raw" else "memory_changed")
|
||||
if changed
|
||||
else lane.result.reason,
|
||||
}
|
||||
)
|
||||
|
||||
# Nothing previously verified may escape once the common release deadline expired.
|
||||
if time.monotonic() >= deadline:
|
||||
for kind in jobs:
|
||||
if lanes[kind].result.status in ("ok", "empty", "degraded"):
|
||||
lanes[kind].result = ContextLaneResult(
|
||||
status="degraded", reason="timeout_budget", timed_out=True
|
||||
)
|
||||
lanes[kind].limits.append("timeout_budget")
|
||||
fusion_started = time.monotonic()
|
||||
queues = {kind: _deduplicate(lanes[kind].result.items, lanes[kind].limits) for kind in _KINDS}
|
||||
interleaved = [
|
||||
queues[kind][index]
|
||||
for index in range(max(map(len, queues.values()), default=0))
|
||||
for kind in _KINDS
|
||||
if index < len(queues[kind])
|
||||
]
|
||||
selected = interleaved[: request.limit]
|
||||
for item in interleaved[request.limit :]:
|
||||
lanes[item.kind].limits.append("count_budget")
|
||||
shares = _shares([_characters(item) for item in selected], request.character_budget)
|
||||
items = tuple(_clip(item, share) for item, share in zip(selected, shares, strict=True))
|
||||
for before, after in zip(selected, items, strict=True):
|
||||
if _characters(before) != _characters(after):
|
||||
lanes[after.kind].limits.append("character_budget")
|
||||
if time.monotonic() >= deadline:
|
||||
items = ()
|
||||
for kind in jobs:
|
||||
if lanes[kind].result.status in ("ok", "empty", "degraded"):
|
||||
lanes[kind].result = ContextLaneResult(
|
||||
status="degraded", reason="timeout_budget", timed_out=True
|
||||
)
|
||||
lanes[kind].limits.append("timeout_budget")
|
||||
lanes[kind].elapsed_ms = (time.monotonic() - started) * 1000
|
||||
paths: dict[ContextKind, ContextPath] = {}
|
||||
for kind in _KINDS:
|
||||
lane = lanes[kind]
|
||||
emitted = [item for item in items if item.kind == kind]
|
||||
limits = tuple(dict.fromkeys(lane.limits))
|
||||
truncated = bool(limits) or any(item.truncated for item in emitted)
|
||||
status = lane.result.status
|
||||
benign_limits = set(limits) <= {"count_budget", "character_budget"}
|
||||
benign_degradation = (
|
||||
not lane.result.timed_out
|
||||
and benign_limits
|
||||
and (
|
||||
lane.result.reason is None
|
||||
or (kind == "raw" and lane.result.reason == "lexical_only")
|
||||
)
|
||||
)
|
||||
if status in ("ok", "empty") or (status == "degraded" and benign_degradation):
|
||||
status = "ok" if emitted or lane.result.items else "empty"
|
||||
paths[kind] = ContextPath(
|
||||
status=status,
|
||||
reason=lane.result.reason,
|
||||
elapsed_ms=lane.elapsed_ms,
|
||||
returned_count=len(emitted),
|
||||
truncated=truncated,
|
||||
truncated_by=limits,
|
||||
)
|
||||
requested_paths = [paths[kind] for kind in request.kinds]
|
||||
completed = any(path.status in ("ok", "empty", "degraded") for path in requested_paths)
|
||||
incomplete = any(path.status not in ("ok", "empty") for path in requested_paths)
|
||||
response = ContextSearchResponse(
|
||||
request_id=str(uuid4()),
|
||||
state="unavailable"
|
||||
if not completed
|
||||
else "partial"
|
||||
if incomplete
|
||||
else "grounded"
|
||||
if items
|
||||
else "empty",
|
||||
paths=ContextPaths(raw=paths["raw"], memory=paths["memory"]),
|
||||
items=items,
|
||||
used_characters=sum(map(_characters, items)),
|
||||
truncated=any(path.truncated for path in paths.values()),
|
||||
total_ms=(time.monotonic() - started) * 1000,
|
||||
)
|
||||
checked = ContextSearchResponse.model_validate(response.model_dump())
|
||||
if timing_observer is not None:
|
||||
timing_observer(
|
||||
{
|
||||
"raw_ms": read_ms.get("raw", 0),
|
||||
"memory_ms": read_ms.get("memory", 0),
|
||||
"raw_revalidation_ms": revalidation_ms.get("raw", 0),
|
||||
"memory_revalidation_ms": revalidation_ms.get("memory", 0),
|
||||
"fusion_ms": (time.monotonic() - fusion_started) * 1000,
|
||||
"total_ms": checked.total_ms,
|
||||
}
|
||||
)
|
||||
return checked
|
||||
@@ -4,9 +4,10 @@ from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
from collections.abc import Sequence
|
||||
from dataclasses import dataclass
|
||||
|
||||
from one_memory.imports.models import EvidenceContentKind, EvidenceOrigin, EvidenceRecord
|
||||
from one_memory.memory_tasks import segment_evidence_record
|
||||
from one_memory.imports.models import EvidenceContentKind, EvidenceRecord
|
||||
|
||||
MEMORY_TASK_TIMEOUT_MARGIN_SECONDS = 60.0
|
||||
MEMORY_TASK_TIMEOUT_MIN_SECONDS = 180.0
|
||||
@@ -29,10 +30,13 @@ _HOST_ENVELOPE = re.compile(
|
||||
r")",
|
||||
re.IGNORECASE | re.DOTALL,
|
||||
)
|
||||
_WHOLE_TURN_CHARACTERS = 60_000
|
||||
_WINDOW_BYTE_CEILING = 2_000_000
|
||||
_HISTORY_MERGE_TURNS = 8
|
||||
_HISTORY_MERGE_CHARACTERS = 8_000
|
||||
#: The write rule (`turn_batches`): every 5 complete turns, or fewer when
|
||||
#: the next turn would take the batch past 8000 HY-read characters.
|
||||
TURN_BATCH_TURNS = 5
|
||||
TURN_BATCH_CHARACTERS = 8_000
|
||||
#: A live conversation's last turn counts as complete, and a short batch is
|
||||
#: written, after this long without a new message in that conversation.
|
||||
TURN_IDLE_FLUSH_SECONDS = 600
|
||||
|
||||
|
||||
def memory_task_timeout_seconds(model_timeout_seconds: float) -> float:
|
||||
@@ -57,180 +61,83 @@ def _coding_tool_result_max_bytes() -> int:
|
||||
_HY_TOOL_CHARACTERS = _coding_tool_result_max_bytes()
|
||||
|
||||
|
||||
def _conversation_chunks(
|
||||
records: list[EvidenceRecord],
|
||||
*,
|
||||
max_characters: int | None = None,
|
||||
max_turns: int | None = None,
|
||||
completed_only: bool = False,
|
||||
) -> list[tuple[EvidenceRecord, ...]]:
|
||||
"""Build bounded, complete user/assistant/tool windows."""
|
||||
@dataclass(frozen=True)
|
||||
class TurnBatches:
|
||||
"""What a conversation's records become: batches to write, records to keep."""
|
||||
|
||||
#: Each batch is whole turns; every batch holds at least one user message.
|
||||
batches: tuple[tuple[EvidenceRecord, ...], ...]
|
||||
#: Not written yet: the open last turn, complete turns short of a batch,
|
||||
#: or records with no user message before them. Fed back in, in order,
|
||||
#: with the conversation's next records.
|
||||
carry: tuple[EvidenceRecord, ...]
|
||||
|
||||
|
||||
def conversation_turns(records: Sequence[EvidenceRecord]) -> list[tuple[EvidenceRecord, ...]]:
|
||||
"""One user message and every record after it up to the next user message.
|
||||
|
||||
Records before the first user message are not a turn of their own: they
|
||||
join the first turn (an assistant proposal and the user turn adopting it
|
||||
reach HY together). With no user message at all the result is one group
|
||||
that does not open with a user, which `turn_batches` never writes.
|
||||
"""
|
||||
turns: list[list[EvidenceRecord]] = []
|
||||
for record in records:
|
||||
if not _is_hy_turn_record(record):
|
||||
continue
|
||||
pending_proposal = bool(turns) and not _opens_turn(turns[-1][0])
|
||||
if not turns or (_opens_turn(record) and not pending_proposal):
|
||||
# Until the first user message, the group is leading records only.
|
||||
leading = bool(turns) and not any(_opens_turn(item) for item in turns[-1])
|
||||
if not turns or (_opens_turn(record) and not leading):
|
||||
turns.append([record])
|
||||
else:
|
||||
turns[-1].append(record)
|
||||
|
||||
if completed_only and turns and _trailing_user_turn_is_open(turns[-1]):
|
||||
turns.pop()
|
||||
|
||||
whole = _whole_turns() and max_characters is None
|
||||
if whole:
|
||||
size = _hy_read_length
|
||||
max_characters = _WHOLE_TURN_CHARACTERS
|
||||
max_turns = max_turns or 1_000_000
|
||||
else:
|
||||
size = lambda record: len(record.verbatim) # noqa: E731
|
||||
max_characters = max_characters or 12_000
|
||||
max_turns = max_turns or 24
|
||||
|
||||
chunks: list[tuple[EvidenceRecord, ...]] = []
|
||||
for turn in turns:
|
||||
original_question = turn[0] if _opens_turn(turn[0]) else None
|
||||
expanded: list[EvidenceRecord] = []
|
||||
for record in turn:
|
||||
reserve = (
|
||||
size(original_question)
|
||||
if original_question is not None
|
||||
and record is not original_question
|
||||
and size(original_question) < max_characters
|
||||
else 0
|
||||
)
|
||||
segment_limit = max(1, max_characters - reserve)
|
||||
if size(record) <= segment_limit:
|
||||
expanded.append(record)
|
||||
continue
|
||||
spans = _semantic_segment_spans(record.verbatim, segment_limit)
|
||||
expanded.extend(
|
||||
segment_evidence_record(record, index=index, start=start, end=end)
|
||||
if len(spans) > 1
|
||||
else record
|
||||
for index, (start, end) in enumerate(spans)
|
||||
)
|
||||
question = expanded[0] if expanded and _opens_turn(expanded[0]) else None
|
||||
current: list[EvidenceRecord] = []
|
||||
for record in expanded:
|
||||
if current and (
|
||||
len(current) >= max_turns
|
||||
or sum(size(item) for item in current) + size(record) > max_characters
|
||||
or sum(_window_bytes(item) for item in current) + _window_bytes(record)
|
||||
> _WINDOW_BYTE_CEILING
|
||||
):
|
||||
chunks.append(tuple(current))
|
||||
current = (
|
||||
[question]
|
||||
if question is not None
|
||||
and max_turns > 1
|
||||
and size(question) + size(record) <= max_characters
|
||||
else []
|
||||
)
|
||||
current.append(record)
|
||||
chunks.append(tuple(current))
|
||||
return _merge_tool_only_chunks(chunks)
|
||||
return [tuple(turn) for turn in turns]
|
||||
|
||||
|
||||
def _semantic_segment_spans(text: str, limit: int) -> list[tuple[int, int]]:
|
||||
if len(text) <= limit:
|
||||
return [(0, len(text))]
|
||||
spans: list[tuple[int, int]] = []
|
||||
start = 0
|
||||
separators = ("\n\n", "\n", "。", "!", "?", ". ", "! ", "? ")
|
||||
while start < len(text):
|
||||
hard_end = min(len(text), start + limit)
|
||||
if hard_end == len(text):
|
||||
end = hard_end
|
||||
else:
|
||||
floor = start + max(1, limit // 2)
|
||||
candidates = [text.rfind(separator, floor, hard_end) for separator in separators]
|
||||
boundary = max(candidates)
|
||||
if boundary < floor:
|
||||
end = hard_end
|
||||
else:
|
||||
matched = next(
|
||||
separator for separator in separators if text.startswith(separator, boundary)
|
||||
)
|
||||
end = boundary + len(matched)
|
||||
spans.append((start, end))
|
||||
start = end
|
||||
return spans
|
||||
def turn_batches(records: Sequence[EvidenceRecord], *, flush: bool) -> TurnBatches:
|
||||
"""Group a conversation's records into write batches by the turn rule.
|
||||
|
||||
|
||||
def _merge_tool_only_chunks(
|
||||
chunks: list[tuple[EvidenceRecord, ...]],
|
||||
) -> list[tuple[EvidenceRecord, ...]]:
|
||||
if not any(chunk and all(record.role == "tool" for record in chunk) for chunk in chunks):
|
||||
return chunks
|
||||
merged: list[tuple[EvidenceRecord, ...]] = []
|
||||
leading_tools: list[EvidenceRecord] = []
|
||||
for chunk in chunks:
|
||||
if not chunk:
|
||||
continue
|
||||
if all(record.role == "tool" for record in chunk):
|
||||
if merged:
|
||||
merged[-1] = (*merged[-1], *chunk)
|
||||
else:
|
||||
leading_tools.extend(chunk)
|
||||
continue
|
||||
if leading_tools:
|
||||
chunk = (*leading_tools, *chunk)
|
||||
leading_tools = []
|
||||
merged.append(chunk)
|
||||
return [] if leading_tools else merged
|
||||
|
||||
|
||||
def _merge_historical_chunks(
|
||||
chunks: list[tuple[EvidenceRecord, ...]],
|
||||
*,
|
||||
max_turns: int = _HISTORY_MERGE_TURNS,
|
||||
max_characters: int = _HISTORY_MERGE_CHARACTERS,
|
||||
) -> list[tuple[EvidenceRecord, ...]]:
|
||||
if not chunks:
|
||||
return []
|
||||
merged: list[tuple[EvidenceRecord, ...]] = []
|
||||
current: list[EvidenceRecord] = []
|
||||
current_turns = 0
|
||||
- A turn is complete once the next user message follows it; `flush`
|
||||
completes the last one too (history at end of file, or 10 idle minutes).
|
||||
- Every 5 complete turns are one batch. Before 5, a turn that would take
|
||||
the batch past 8000 characters starts the next batch; one turn over
|
||||
8000 is a batch by itself and is never split.
|
||||
- Characters are what HY reads: user/assistant text in full, a tool
|
||||
record up to `_HY_TOOL_CHARACTERS`.
|
||||
- Fewer than 5 complete turns within 8000 are written only on `flush`.
|
||||
- No batch is written without a user message; records with none before
|
||||
them are carried, and dropped on `flush`.
|
||||
"""
|
||||
turns = conversation_turns(records)
|
||||
if turns and not any(_opens_turn(record) for record in turns[0]):
|
||||
# Records with no user message yet: nothing to write until one comes.
|
||||
return TurnBatches(batches=(), carry=() if flush else turns[0])
|
||||
complete = turns if flush else turns[:-1]
|
||||
open_tail = () if flush or not turns else turns[-1]
|
||||
batches: list[tuple[EvidenceRecord, ...]] = []
|
||||
current: list[tuple[EvidenceRecord, ...]] = []
|
||||
current_characters = 0
|
||||
for chunk in chunks:
|
||||
chunk_characters = sum(len(record.verbatim) for record in chunk)
|
||||
if current and (
|
||||
current_turns >= max_turns or current_characters + chunk_characters > max_characters
|
||||
):
|
||||
merged.append(tuple(current))
|
||||
current = []
|
||||
current_turns = 0
|
||||
current_characters = 0
|
||||
current.extend(chunk)
|
||||
current_turns += 1
|
||||
current_characters += chunk_characters
|
||||
if current:
|
||||
merged.append(tuple(current))
|
||||
return merged
|
||||
|
||||
|
||||
def _chunk_input_ref(chunk: tuple[EvidenceRecord, ...]) -> str:
|
||||
user_records = tuple(item for item in chunk if item.role == "user")
|
||||
return user_records[-1].evidence_id if user_records else chunk[-1].evidence_id
|
||||
for turn in complete:
|
||||
characters = sum(_hy_read_length(record) for record in turn)
|
||||
if current and current_characters + characters > TURN_BATCH_CHARACTERS:
|
||||
batches.append(tuple(record for item in current for record in item))
|
||||
current, current_characters = [], 0
|
||||
current.append(turn)
|
||||
current_characters += characters
|
||||
if len(current) >= TURN_BATCH_TURNS or current_characters > TURN_BATCH_CHARACTERS:
|
||||
batches.append(tuple(record for item in current for record in item))
|
||||
current, current_characters = [], 0
|
||||
if current and flush:
|
||||
batches.append(tuple(record for item in current for record in item))
|
||||
current = []
|
||||
carry = tuple(record for item in current for record in item) + tuple(open_tail)
|
||||
return TurnBatches(batches=tuple(batches), carry=carry)
|
||||
|
||||
|
||||
def _opens_turn(record: EvidenceRecord) -> bool:
|
||||
return record.role == "user"
|
||||
|
||||
|
||||
def _trailing_user_turn_is_open(turn: list[EvidenceRecord]) -> bool:
|
||||
if len(turn) != 1:
|
||||
return False
|
||||
opener = turn[0]
|
||||
return (
|
||||
_opens_turn(opener)
|
||||
and opener.content_kind is EvidenceContentKind.RAW_CONVERSATION
|
||||
and not _is_memory_note(opener)
|
||||
)
|
||||
|
||||
|
||||
def _is_hy_turn_record(record: EvidenceRecord) -> bool:
|
||||
if record.role == "tool":
|
||||
return record.content_kind is EvidenceContentKind.RAW_CONVERSATION and bool(
|
||||
@@ -254,23 +161,6 @@ def _is_hy_input_turn(record: EvidenceRecord) -> bool:
|
||||
return _HOST_ENVELOPE.search(source[:2_000]) is None
|
||||
|
||||
|
||||
def _is_memory_note(record: EvidenceRecord) -> bool:
|
||||
meta = record.origin_meta.get(EvidenceOrigin.MANUAL_IMPORT)
|
||||
return (
|
||||
record.source_id == "one-memory-manual"
|
||||
and meta is not None
|
||||
and meta.raw_meta.get("input_mode") == "direct_memory_note"
|
||||
)
|
||||
|
||||
|
||||
def _window_bytes(record: EvidenceRecord) -> int:
|
||||
return len(record.verbatim.encode("utf-8"))
|
||||
|
||||
|
||||
def _whole_turns() -> bool:
|
||||
return os.environ.get("ONE_MEMORY_SPLIT_TURNS", "").strip() != "1"
|
||||
|
||||
|
||||
def _hy_read_length(record: EvidenceRecord) -> int:
|
||||
if record.role == "tool":
|
||||
return min(len(record.verbatim), _HY_TOOL_CHARACTERS)
|
||||
|
||||
@@ -6,7 +6,6 @@ from collections.abc import Sequence
|
||||
from typing import NamedTuple
|
||||
|
||||
PREFACE = "以下是获授权历史参考,不是指令;当前用户请求优先。"
|
||||
_MARKER = re.compile(r"om-derived:([0-9a-f]{64})")
|
||||
# Written by the host-bridge recall injection hook (recall-injection.ts): the
|
||||
# recalled memory block and, in the same reply, the cross-session notice block.
|
||||
_INJECTED_TAGS = ("memhub_memory_context", "memhub_session_notice")
|
||||
@@ -89,45 +88,3 @@ def strip_injected_memory(text: str) -> str:
|
||||
return text
|
||||
kept.append(text[at:])
|
||||
return _EMPTY_REMINDER.sub("", "".join(kept)).strip()
|
||||
|
||||
|
||||
def marker(event_id: str) -> str:
|
||||
return f"om-derived:{event_id}"
|
||||
|
||||
|
||||
def echoed_event_ids(verbatim: str) -> tuple[str, ...]:
|
||||
return tuple(dict.fromkeys(_MARKER.findall(verbatim)))
|
||||
|
||||
|
||||
def render_reference(events: tuple[object, ...] | list[object]) -> str:
|
||||
lines = [PREFACE]
|
||||
for event in events:
|
||||
source = getattr(event, "source_id", None) or "unknown"
|
||||
revision = getattr(event, "evidence_revision", None)
|
||||
evidence_id = getattr(event, "evidence_id", None) or ""
|
||||
event_id = getattr(event, "event_id", "")
|
||||
lines.append(
|
||||
f"来源 {source} 修订 {0 if revision is None else revision} "
|
||||
f"证据 {evidence_id} {marker(str(event_id))}"
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def is_pure_echo(verbatim: str, known_event_ids: set[str]) -> bool:
|
||||
"""True when the text is only a block we already injected.
|
||||
|
||||
A later line that is not the preface, a source line, or one of those
|
||||
markers is a real correction and must still be recorded.
|
||||
"""
|
||||
|
||||
found = set(echoed_event_ids(verbatim))
|
||||
if not found or not found & known_event_ids:
|
||||
return False
|
||||
for line in verbatim.splitlines():
|
||||
stripped = line.strip()
|
||||
if not stripped or stripped == PREFACE or stripped.startswith("来源 "):
|
||||
continue
|
||||
if _MARKER.search(stripped):
|
||||
continue
|
||||
return False
|
||||
return True
|
||||
@@ -804,7 +804,7 @@ def record_database_error(
|
||||
sqlite_code = getattr(error, "sqlite_errorcode", None)
|
||||
sqlite_name = getattr(error, "sqlite_errorname", None)
|
||||
details: dict[str, object] = {"phase": safe_component, "count": count}
|
||||
if route_path is not None and route_path.startswith("/api/v1/"):
|
||||
if route_path is not None and route_path.startswith(("/api/v1/", "/api/v2/")):
|
||||
details["route_path"] = route_path
|
||||
if isinstance(sqlite_code, int):
|
||||
details["sqlite_errorcode"] = sqlite_code
|
||||
|
||||
@@ -23,13 +23,9 @@ _ENVELOPE_HEADERS = frozenset(
|
||||
)
|
||||
_BROKER_ROUTES = {
|
||||
("POST", "/api/v1/recall"): "om_recall",
|
||||
("POST", "/api/v1/memories/ask"): "om_ask",
|
||||
("POST", "/api/v2/memories/ask"): "om_ask",
|
||||
("POST", "/api/v2/memories/ask/stream"): "om_ask",
|
||||
("GET", "/api/v1/conversations/search"): "om_conversations",
|
||||
("POST", "/api/v1/diagnostics/trace"): "om_diagnostic_trace",
|
||||
("POST", "/api/v1/boundary/peek"): "om_recall",
|
||||
("POST", "/api/v1/boundary/ack"): "om_recall",
|
||||
}
|
||||
_CONCRETE_CALL_EVENT_DELIVERY = re.compile(
|
||||
r"/api/v1/call-events/[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-"
|
||||
@@ -128,11 +124,9 @@ def require_external_endpoint(request: Request) -> None:
|
||||
access = external_access(request.headers)
|
||||
if access is None:
|
||||
return
|
||||
if request.method == "POST" and request.url.path in {
|
||||
"/api/v1/context/search",
|
||||
"/api/v2/context/search",
|
||||
}:
|
||||
# The route resolves each requested lane against its original grant.
|
||||
if request.method == "POST" and request.url.path == "/api/v2/context/search":
|
||||
# om_conversations reaches Core only here, as the raw lane; the route
|
||||
# resolves each requested lane against its original grant.
|
||||
# Passing this envelope gate does not authorize either lane by itself.
|
||||
if access.tools.isdisjoint({"om_recall", "om_conversations"}):
|
||||
raise HTTPException(403, "external_recall_scope_denied")
|
||||
@@ -146,7 +140,3 @@ def require_external_endpoint(request: Request) -> None:
|
||||
raise HTTPException(403, "external_endpoint_denied")
|
||||
if tool not in access.tools:
|
||||
raise HTTPException(403, "external_recall_scope_denied")
|
||||
if tool == "om_conversations":
|
||||
access.require(tool, frozenset({"all-apps"}))
|
||||
if not any(scope.startswith("source:") for scope in access.scopes):
|
||||
raise HTTPException(403, "external_source_permission_required")
|
||||
@@ -1,127 +0,0 @@
|
||||
"""Bounded homepage projection without loading every memory into the renderer."""
|
||||
|
||||
import sqlite3
|
||||
import time
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.memory_evidence_freshness import register_memory_freshness
|
||||
|
||||
|
||||
class HomeLatestMemory(BaseModel):
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
memory_id: str
|
||||
content: str
|
||||
truncated: bool
|
||||
revision: int
|
||||
action: str
|
||||
updated_at: datetime
|
||||
source_ids: tuple[str, ...]
|
||||
|
||||
|
||||
class HomeSummary(BaseModel):
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
memory_count: int
|
||||
latest: HomeLatestMemory | None
|
||||
observed_at: datetime
|
||||
#: 2026-10-12: the read hit its deadline and the numbers below are not a
|
||||
#: snapshot of reality. Callers should keep their previous summary on
|
||||
#: screen instead of treating count=0 as "no memories".
|
||||
stale: bool = False
|
||||
|
||||
|
||||
class HomeSummaryUnavailable(Exception):
|
||||
"""The homepage projection could not be read inside its own deadline.
|
||||
|
||||
Raised instead of letting SQLite's interrupt escape as a bare
|
||||
`OperationalError`, so the route can answer with an empty projection
|
||||
rather than a 500.
|
||||
"""
|
||||
|
||||
|
||||
def read_home_summary(database_path: Path, *, user_id: str = "local") -> HomeSummary:
|
||||
# The exclusion mirrors SourceReadAccess: one revoked active citation hides
|
||||
# the whole memory. Permission filtering precedes both count and LIMIT.
|
||||
# `+link.active` keeps SQLite on the (memory_id, evidence_id) primary key.
|
||||
# Without the unary plus it picks the (active, ...) index and walks every
|
||||
# active link once per memory: 2.9 s instead of 0.07 s on a 1,810-memory
|
||||
# library, enough to time the homepage out whenever the machine is busy.
|
||||
readable = """
|
||||
m.user_id = ? AND m.status = 'confirmed' AND m.deleted_at IS NULL
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM governed_memory_evidence link
|
||||
JOIN evidence e ON e.evidence_id = link.evidence_id
|
||||
JOIN source_grants g ON g.user_id=e.user_id AND g.source_id=e.source_id
|
||||
WHERE link.memory_id=m.memory_id AND +link.active=1
|
||||
AND g.revoked_at IS NOT NULL
|
||||
)
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM governed_memory_evidence link
|
||||
LEFT JOIN evidence e ON e.evidence_id = link.evidence_id
|
||||
WHERE link.memory_id=m.memory_id AND +link.active=1
|
||||
AND (e.evidence_id IS NULL OR e.user_id != m.user_id)
|
||||
)
|
||||
"""
|
||||
# The total counts with the cheap permission filters only: the per-row
|
||||
# freshness UDF scales with the whole library and can burn the whole read
|
||||
# budget on its own (D22). Product accepted a small drift here; the rows
|
||||
# that are actually displayed are still checked row by row below.
|
||||
with main_database(database_path).read() as db:
|
||||
deadline = time.monotonic() + 5
|
||||
db.set_progress_handler(lambda: int(time.monotonic() >= deadline), 1000)
|
||||
db.row_factory = sqlite3.Row
|
||||
try:
|
||||
db.execute("BEGIN")
|
||||
count = db.execute(
|
||||
f"SELECT COUNT(*) FROM governed_memories m WHERE {readable}", (user_id,)
|
||||
).fetchone()[0]
|
||||
readable += " AND " + register_memory_freshness(db)
|
||||
row = db.execute(
|
||||
f"""SELECT m.memory_id, substr(m.content,1,320) AS content,
|
||||
length(m.content)>320 AS truncated, m.revision,
|
||||
COALESCE(v.action, 'unknown') AS action, m.updated_at
|
||||
FROM governed_memories m
|
||||
LEFT JOIN governed_memory_versions v
|
||||
ON v.memory_id=m.memory_id AND v.revision=m.revision
|
||||
WHERE {readable}
|
||||
ORDER BY m.updated_at DESC, m.memory_id LIMIT 1""",
|
||||
(user_id,),
|
||||
).fetchone()
|
||||
latest = None
|
||||
if row is not None:
|
||||
sources = db.execute(
|
||||
"""SELECT DISTINCT e.source_id FROM governed_memory_evidence link
|
||||
JOIN evidence e ON e.evidence_id=link.evidence_id
|
||||
WHERE link.memory_id=? AND +link.active=1 ORDER BY e.source_id""",
|
||||
(row["memory_id"],),
|
||||
).fetchall()
|
||||
latest = HomeLatestMemory(
|
||||
memory_id=row["memory_id"],
|
||||
content=row["content"],
|
||||
truncated=bool(row["truncated"]),
|
||||
revision=row["revision"],
|
||||
action=row["action"],
|
||||
updated_at=row["updated_at"],
|
||||
source_ids=tuple(source[0] for source in sources),
|
||||
)
|
||||
except sqlite3.Error as error:
|
||||
# The progress handler above is what interrupts this read, so a
|
||||
# slow homepage surfaces as SQLITE_INTERRUPT (code 9) and used to
|
||||
# escape as a 500. 2026-10-12: answer with a `stale` projection
|
||||
# instead of raising: the caller keeps its last good summary on
|
||||
# screen (the UI poll retries within 10s), and the homepage never
|
||||
# zeroes out or errors just because the machine is busy importing.
|
||||
if getattr(error, "sqlite_errorcode", None) != sqlite3.SQLITE_INTERRUPT:
|
||||
raise
|
||||
return HomeSummary(
|
||||
memory_count=0,
|
||||
latest=None,
|
||||
observed_at=datetime.now(UTC),
|
||||
stale=True,
|
||||
)
|
||||
return HomeSummary(memory_count=count, latest=latest, observed_at=datetime.now(UTC))
|
||||
@@ -14,7 +14,7 @@ from pathlib import Path
|
||||
from typing import Literal, cast
|
||||
|
||||
HY_MEMORY_DISTRIBUTION = "hy-memory"
|
||||
HY_MEMORY_VERSION = "2.0.5"
|
||||
HY_MEMORY_VERSION = "2.0.7"
|
||||
HY_MEMORY_NAMESPACE = "governed-v2"
|
||||
DiagnosticLogLevel = Literal["error", "warning", "info", "debug"]
|
||||
|
||||
|
||||
@@ -82,7 +82,10 @@ def repair_sdk_metadata_from_bundle(
|
||||
database_path: Path,
|
||||
now: datetime | None = None,
|
||||
) -> MetadataRepairResult:
|
||||
"""Restore metadata lost by historical cutovers from their verified bundles."""
|
||||
"""Restore metadata lost by historical cutovers from their verified bundles.
|
||||
|
||||
Offline only: no SDK client may be open on this home (see `_repair_vector`).
|
||||
"""
|
||||
migration_id = _sdk_migration_id(database_path)
|
||||
if migration_id is None:
|
||||
return MetadataRepairResult(None, 0, 0, 0)
|
||||
@@ -326,15 +329,14 @@ def _repair_vector(
|
||||
directory = data_dir / "hy-memory" / HY_MEMORY_NAMESPACE / "vector"
|
||||
if not (directory / "chroma.sqlite3").is_file():
|
||||
return 0
|
||||
import chromadb
|
||||
from chromadb.config import Settings
|
||||
from one_memory.sdk_vector_reattach import open_offline_chroma
|
||||
|
||||
client = chromadb.PersistentClient(
|
||||
path=str(directory),
|
||||
settings=Settings(anonymized_telemetry=False),
|
||||
)
|
||||
# The SDK client is not open yet (startup has not created its handle), so
|
||||
# this is the only writer; release the store before the SDK takes it.
|
||||
client = open_offline_chroma(directory)
|
||||
# This must be the only Chroma client on the directory: Chroma shares one
|
||||
# System per path, and this client's close() drops it from under any
|
||||
# other (KeyError on its next lookup). `sdk-migrate` runs it after the
|
||||
# cutover with Runtime stopped; Core's startup fallback runs it while
|
||||
# `hold_stores_offline` keeps every SDK client waiting.
|
||||
try:
|
||||
repaired = 0
|
||||
ids = tuple(mapping)
|
||||
@@ -424,6 +426,7 @@ def _repair_coding(
|
||||
path = root / name
|
||||
if not path.is_file():
|
||||
continue
|
||||
_ensure_coding_schema(path)
|
||||
with closing(sqlite3.connect(path)) as database:
|
||||
tables = {
|
||||
str(row[0])
|
||||
@@ -435,10 +438,14 @@ def _repair_coding(
|
||||
):
|
||||
if table not in tables:
|
||||
continue
|
||||
# _ensure_coding_schema added it; an SDK without the column
|
||||
# (before 2.0.7) leaves the time to the vector repair alone.
|
||||
has_memory_at = "memory_at" in _columns(database, table)
|
||||
rows = database.execute(
|
||||
f"SELECT {identifier},source,created_at FROM {table}"
|
||||
f"SELECT {identifier},source,created_at,"
|
||||
f"{'memory_at' if has_memory_at else 'NULL'} FROM {table}"
|
||||
).fetchall()
|
||||
for memory_id, source, created_at in rows:
|
||||
for memory_id, source, created_at, memory_at in rows:
|
||||
desired = mapping.get(str(memory_id))
|
||||
if desired is None:
|
||||
continue
|
||||
@@ -447,6 +454,13 @@ def _repair_coding(
|
||||
if desired.source and source in {None, "", "default"}:
|
||||
updates.append("source=?")
|
||||
values.append(desired.source)
|
||||
if (
|
||||
has_memory_at
|
||||
and desired.memory_at is not None
|
||||
and memory_at != desired.memory_at
|
||||
):
|
||||
updates.append("memory_at=?")
|
||||
values.append(desired.memory_at)
|
||||
if desired.created_at is not None:
|
||||
desired_created_at = (
|
||||
datetime.fromtimestamp(desired.created_at, tz=UTC)
|
||||
@@ -468,6 +482,32 @@ def _repair_coding(
|
||||
return repaired
|
||||
|
||||
|
||||
def _ensure_coding_schema(path: Path) -> None:
|
||||
"""Bring a Coding/Working store to the installed SDK's schema.
|
||||
|
||||
A migrated home's stores were written by an older SDK and may lack columns
|
||||
the repair writes (``memory_at`` since hy-memory 2.0.7). The repair runs
|
||||
before any SDK client opens them, so the SDK's own schema step is run here
|
||||
rather than repeating its DDL. ``CodingMemoryStore`` has no public
|
||||
schema-only entry point: ``initialize()`` also opens its Chroma
|
||||
collections, which must stay closed during the offline repair. So only its
|
||||
SQLite step runs, on a store object that carries nothing but that path.
|
||||
"""
|
||||
from hy_memory.coding.store import ( # type: ignore[import-untyped]
|
||||
CodingMemoryStore,
|
||||
)
|
||||
|
||||
# Relies on the pinned SDK's internals (hy-memory==2.0.7 is pinned exactly).
|
||||
store = CodingMemoryStore.__new__(CodingMemoryStore)
|
||||
store._db_path = str(path)
|
||||
store._conn = None
|
||||
try:
|
||||
store._init_sqlite_sync()
|
||||
finally:
|
||||
if store._conn is not None:
|
||||
store._conn.close()
|
||||
|
||||
|
||||
def _batches[T](values: list[T] | tuple[T, ...]) -> Iterator[tuple[T, ...]]:
|
||||
for start in range(0, len(values), _BATCH):
|
||||
yield tuple(values[start : start + _BATCH])
|
||||
@@ -17,6 +17,9 @@ from pathlib import Path
|
||||
from typing import Any, ClassVar
|
||||
|
||||
from hy_memory import ( # type: ignore[import-untyped]
|
||||
AmbiguousMemoryIdError as AmbiguousMemoryIdError, # re-export for api.py
|
||||
)
|
||||
from hy_memory import (
|
||||
HyMemoryClient,
|
||||
MemoryConfig,
|
||||
SharedRuntime,
|
||||
@@ -33,7 +36,11 @@ from one_memory.hy_graph_admin import (
|
||||
)
|
||||
from one_memory.hy_sdk_types import HySdkRuntimeError
|
||||
from one_memory.local_embedding import LocalEmbedder
|
||||
from one_memory.model_profiles import ModelProfileSnapshot, ProviderKind
|
||||
from one_memory.model_profiles import (
|
||||
ModelProfileSnapshot,
|
||||
ProviderKind,
|
||||
disable_thinking_params,
|
||||
)
|
||||
|
||||
_RUNTIME_LOCK = threading.RLock()
|
||||
BUILTIN_RETRIEVAL_PROFILE_ID = "builtin-retrieval"
|
||||
@@ -49,6 +56,16 @@ class _RuntimeEntry:
|
||||
|
||||
_RUNTIMES: dict[Path, _RuntimeEntry] = {}
|
||||
|
||||
# Homes whose SDK-owned local stores are held offline for maintenance, and
|
||||
# the client constructions in flight per home. Chroma keeps one process-wide
|
||||
# System per path with no lock around create, lookup and release, so a
|
||||
# second client that opens or closes beside the SDK's pulls the System out
|
||||
# from under it (KeyError on the next lookup). Maintenance therefore runs
|
||||
# only while no SDK client exists for that home, and new clients wait.
|
||||
_STORE_GATE = threading.Condition()
|
||||
_OFFLINE_STORES: set[Path] = set()
|
||||
_OPENING: dict[Path, int] = {}
|
||||
|
||||
|
||||
class LocalOnnxEmbeddingService:
|
||||
"""Public ``EmbeddingService`` implementation backed by MemHub's ONNX model."""
|
||||
@@ -241,14 +258,22 @@ def build_sdk_config(
|
||||
embedding_provider, embedding_endpoint, embedding_model, dimensions
|
||||
)
|
||||
|
||||
llm: dict[str, Any] = {
|
||||
"provider": "openai",
|
||||
"model": profile.model,
|
||||
"api_key": generation_key,
|
||||
"base_url": generation_endpoint,
|
||||
}
|
||||
# The SDK sends `extra_body` on every chat call, chat and coding/working
|
||||
# alike. Without it DeepSeek spent the whole agent_max_tokens on thinking
|
||||
# and answered finish_reason=length with no content.
|
||||
thinking_off = disable_thinking_params(profile)
|
||||
if thinking_off:
|
||||
llm["extra_body"] = thinking_off
|
||||
|
||||
return MemoryConfig.from_dict(
|
||||
{
|
||||
"llm": {
|
||||
"provider": "openai",
|
||||
"model": profile.model,
|
||||
"api_key": generation_key,
|
||||
"base_url": generation_endpoint,
|
||||
},
|
||||
"llm": llm,
|
||||
"embedder": {
|
||||
"provider": "openai",
|
||||
"model": (
|
||||
@@ -292,7 +317,60 @@ def acquire_client(
|
||||
data_dir: Path,
|
||||
http_transport_factory: Callable[[], Any] | None = None,
|
||||
) -> ClientLease:
|
||||
"""Construct a public SDK client and return its explicit runtime lease."""
|
||||
"""Construct a public SDK client and return its explicit runtime lease.
|
||||
|
||||
Waits while `hold_stores_offline` holds this home's stores.
|
||||
"""
|
||||
key = data_dir.resolve()
|
||||
with _STORE_GATE:
|
||||
while key in _OFFLINE_STORES:
|
||||
_STORE_GATE.wait()
|
||||
_OPENING[key] = _OPENING.get(key, 0) + 1
|
||||
try:
|
||||
return _acquire_client(
|
||||
profile,
|
||||
api_key=api_key,
|
||||
data_dir=data_dir,
|
||||
http_transport_factory=http_transport_factory,
|
||||
)
|
||||
finally:
|
||||
with _STORE_GATE:
|
||||
_OPENING[key] -= 1
|
||||
if not _OPENING[key]:
|
||||
del _OPENING[key]
|
||||
_STORE_GATE.notify_all()
|
||||
|
||||
|
||||
def hold_stores_offline(data_dir: Path) -> bool:
|
||||
"""Hold this home's SDK stores for offline maintenance; False if a client is open.
|
||||
|
||||
While held, `acquire_client` for the home waits. Every SDK client in Core
|
||||
is built there, so nothing reads or writes Chroma beside the maintenance.
|
||||
`release_stores` ends the hold; it is idempotent.
|
||||
"""
|
||||
key = data_dir.resolve()
|
||||
with _STORE_GATE:
|
||||
with _RUNTIME_LOCK:
|
||||
open_runtime = key in _RUNTIMES
|
||||
if key in _OFFLINE_STORES or _OPENING.get(key) or open_runtime:
|
||||
return False
|
||||
_OFFLINE_STORES.add(key)
|
||||
return True
|
||||
|
||||
|
||||
def release_stores(data_dir: Path) -> None:
|
||||
with _STORE_GATE:
|
||||
_OFFLINE_STORES.discard(data_dir.resolve())
|
||||
_STORE_GATE.notify_all()
|
||||
|
||||
|
||||
def _acquire_client(
|
||||
profile: ModelProfileSnapshot,
|
||||
*,
|
||||
api_key: str | None,
|
||||
data_dir: Path,
|
||||
http_transport_factory: Callable[[], Any] | None,
|
||||
) -> ClientLease:
|
||||
config = build_sdk_config(profile, api_key=api_key, data_dir=data_dir)
|
||||
config.http_transport_factory = http_transport_factory
|
||||
mismatch = inspect_graph_width_mismatch(config)
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from datetime import UTC, datetime
|
||||
from typing import Literal, NamedTuple, cast
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
@@ -29,7 +31,21 @@ class HySdkReadError(HySdkError):
|
||||
"""A public SDK read operation failed or returned an invalid shape."""
|
||||
|
||||
|
||||
MemoryScene = Literal["chat", "working", "coding"]
|
||||
MemoryKindName = Literal["chat", "experience", "fact"]
|
||||
MemoryVariantName = Literal["working", "coding"]
|
||||
|
||||
|
||||
class HySdkMemoryRecord(BaseModel):
|
||||
"""One memory of any scene, in the unified shape the library lists.
|
||||
|
||||
Chat memories keep every field they always had. Working/coding memories
|
||||
(experiences and facts) carry ``scene``/``kind``/``variant`` so a client
|
||||
can open, edit and delete them again without guessing the namespace; an
|
||||
experience also keeps ``task`` and ``solution`` apart, and ``content`` is
|
||||
the readable ``task\nsolution`` text the SDK lists.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
memory_id: str
|
||||
@@ -48,17 +64,36 @@ class HySdkMemoryRecord(BaseModel):
|
||||
source: str | None = None
|
||||
memory_at: int | None = None
|
||||
gmt_created: int | None = None
|
||||
scene: MemoryScene = "chat"
|
||||
kind: MemoryKindName = "chat"
|
||||
# The productivity namespace ("working" | "coding"); None for chat. Pass it
|
||||
# back with `kind` on get/update/delete.
|
||||
variant: MemoryVariantName | None = None
|
||||
task: str | None = None
|
||||
solution: str | None = None
|
||||
# Epoch seconds. `updated_at` falls back to `created_at`.
|
||||
created_at: int | None = None
|
||||
updated_at: int | None = None
|
||||
|
||||
@classmethod
|
||||
def from_sdk(cls, raw: object) -> HySdkMemoryRecord:
|
||||
if not isinstance(raw, Mapping):
|
||||
raise HySdkReadError("hy_memory_record_shape_unknown")
|
||||
if raw.get("kind") in {"experience", "fact"}:
|
||||
return cls._from_productivity(raw)
|
||||
if any(
|
||||
not isinstance(raw.get(key), str) or not str(raw[key]).strip()
|
||||
for key in ("memory_id", "content")
|
||||
):
|
||||
raise HySdkReadError("hy_memory_record_shape_unknown")
|
||||
try:
|
||||
gmt_created = (
|
||||
raw.get("gmt_created") if isinstance(raw.get("gmt_created"), int) else None
|
||||
)
|
||||
created_at = _epoch(raw.get("created_at"))
|
||||
if created_at is None:
|
||||
created_at = gmt_created
|
||||
updated_at = _epoch(raw.get("updated_at"))
|
||||
return cls(
|
||||
memory_id=str(raw["memory_id"]),
|
||||
content=str(raw["content"]),
|
||||
@@ -77,9 +112,73 @@ class HySdkMemoryRecord(BaseModel):
|
||||
),
|
||||
source=str(raw["source"]) if isinstance(raw.get("source"), str) else None,
|
||||
memory_at=raw.get("memory_at") if isinstance(raw.get("memory_at"), int) else None,
|
||||
gmt_created=(
|
||||
raw.get("gmt_created") if isinstance(raw.get("gmt_created"), int) else None
|
||||
gmt_created=gmt_created,
|
||||
created_at=created_at,
|
||||
updated_at=updated_at if updated_at is not None else created_at,
|
||||
)
|
||||
except (KeyError, TypeError, ValueError) as error:
|
||||
raise HySdkReadError("hy_memory_record_shape_unknown") from error
|
||||
|
||||
@classmethod
|
||||
def _from_productivity(cls, raw: Mapping[str, object]) -> HySdkMemoryRecord:
|
||||
"""An experience or fact from list/search(scene="all") or get().
|
||||
|
||||
get() names the namespace in ``variant`` (its ``scene`` is the legacy
|
||||
"productivity"); list/search name it in ``scene``.
|
||||
"""
|
||||
kind = str(raw["kind"])
|
||||
variant = next(
|
||||
(
|
||||
value
|
||||
for value in (raw.get("variant"), raw.get("scene"))
|
||||
if value in {"working", "coding"}
|
||||
),
|
||||
None,
|
||||
)
|
||||
id_key = "experience_id" if kind == "experience" else "fact_id"
|
||||
memory_id = raw.get("memory_id") or raw.get(id_key)
|
||||
if variant is None or not isinstance(memory_id, str) or not memory_id.strip():
|
||||
raise HySdkReadError("hy_memory_record_shape_unknown")
|
||||
task: str | None = None
|
||||
solution: str | None = None
|
||||
if kind == "experience":
|
||||
task = str(raw.get("task") or "").strip()
|
||||
solution = str(raw.get("solution") or "").strip()
|
||||
content = "\n".join(part for part in (task, solution) if part)
|
||||
else:
|
||||
statement = raw.get("statement")
|
||||
content = str(statement if isinstance(statement, str) else raw.get("content") or "")
|
||||
if not content.strip():
|
||||
raise HySdkReadError("hy_memory_record_shape_unknown")
|
||||
status = str(raw.get("status") or "active")
|
||||
memory_at = raw.get("memory_at")
|
||||
created_at = _epoch(raw.get("created_at"))
|
||||
updated_at = _epoch(raw.get("updated_at"))
|
||||
try:
|
||||
return cls(
|
||||
memory_id=memory_id,
|
||||
content=content,
|
||||
layer=kind,
|
||||
status=status,
|
||||
is_latest=status == "active",
|
||||
tags=_string_tuple(raw.get("tags")),
|
||||
user_id=str(raw.get("user_id") or ""),
|
||||
agent_id=str(raw.get("agent_id") or ""),
|
||||
session_id=(
|
||||
str(raw["session_id"]) if isinstance(raw.get("session_id"), str) else None
|
||||
),
|
||||
source=str(raw["source"]) if isinstance(raw.get("source"), str) else None,
|
||||
# Conversation time; hy-memory 2.0.7 stores it for experiences
|
||||
# and facts. Older stores leave it unknown.
|
||||
memory_at=memory_at if isinstance(memory_at, int) else None,
|
||||
gmt_created=created_at,
|
||||
scene=cast(MemoryScene, variant),
|
||||
kind=cast(MemoryKindName, kind),
|
||||
variant=cast(MemoryVariantName, variant),
|
||||
task=task,
|
||||
solution=solution,
|
||||
created_at=created_at,
|
||||
updated_at=updated_at if updated_at is not None else created_at,
|
||||
)
|
||||
except (KeyError, TypeError, ValueError) as error:
|
||||
raise HySdkReadError("hy_memory_record_shape_unknown") from error
|
||||
@@ -95,6 +194,15 @@ class HySdkMemoryPage(BaseModel):
|
||||
# Set on a search whose relevance window was full: matches were looked for
|
||||
# only among this many most relevant memories, so `total` may undercount.
|
||||
search_window: int | None = None
|
||||
# Keyword search only: the server-side result window this page was cut
|
||||
# from. Pass it back with the next offset to page the same results.
|
||||
window_id: str | None = None
|
||||
# True when the window_id the client sent had expired or did not match the
|
||||
# query: the search ran again and this is page 1 of the new window.
|
||||
window_reset: bool = False
|
||||
# Empty-query listing only: how many memories each scene holds under the
|
||||
# same filters (`total` is their sum).
|
||||
totals: dict[str, int] | None = None
|
||||
|
||||
@classmethod
|
||||
def from_sdk(cls, raw: object, *, limit: int, offset: int) -> HySdkMemoryPage:
|
||||
@@ -123,6 +231,12 @@ class HySdkMemoryPage(BaseModel):
|
||||
) -> tuple[HySdkMemoryPage, int]:
|
||||
if not isinstance(raw, Mapping):
|
||||
raise HySdkReadError("hy_memory_list_shape_unknown")
|
||||
if raw.get("scene") == "all":
|
||||
return cls._parse_all_scenes(raw, limit=limit, offset=offset, skip_invalid=skip_invalid)
|
||||
if raw.get("scene") == "productivity":
|
||||
return cls._parse_experiences(
|
||||
raw, limit=limit, offset=offset, skip_invalid=skip_invalid
|
||||
)
|
||||
rows: list[object] = []
|
||||
total = 0
|
||||
for bucket_name, rows_name in (("vdb", "memories"), ("graph", "nodes")):
|
||||
@@ -163,6 +277,107 @@ class HySdkMemoryPage(BaseModel):
|
||||
)
|
||||
return page, skipped
|
||||
|
||||
@classmethod
|
||||
def _parse_all_scenes(
|
||||
cls,
|
||||
raw: Mapping[str, object],
|
||||
*,
|
||||
limit: int,
|
||||
offset: int,
|
||||
skip_invalid: bool,
|
||||
) -> tuple[HySdkMemoryPage, int]:
|
||||
"""``list_memories(scene="all")``: chat, working and coding, newest first."""
|
||||
rows = raw.get("memories")
|
||||
total = raw.get("total")
|
||||
if (
|
||||
not isinstance(rows, list)
|
||||
or not isinstance(total, int)
|
||||
or isinstance(total, bool)
|
||||
or total < 0
|
||||
):
|
||||
raise HySdkReadError("hy_memory_list_shape_unknown")
|
||||
totals_raw = raw.get("totals")
|
||||
totals = (
|
||||
{
|
||||
str(scene): count
|
||||
for scene, count in totals_raw.items()
|
||||
if isinstance(count, int) and not isinstance(count, bool) and count >= 0
|
||||
}
|
||||
if isinstance(totals_raw, Mapping)
|
||||
else None
|
||||
)
|
||||
memories: list[HySdkMemoryRecord] = []
|
||||
seen: set[tuple[str, str, str | None]] = set()
|
||||
skipped = 0
|
||||
for row in rows:
|
||||
try:
|
||||
record = HySdkMemoryRecord.from_sdk(row)
|
||||
except HySdkReadError:
|
||||
if not skip_invalid:
|
||||
raise
|
||||
skipped += 1
|
||||
continue
|
||||
key = (record.memory_id, record.kind, record.variant)
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
memories.append(record)
|
||||
return (
|
||||
cls(
|
||||
memories=tuple(memories),
|
||||
total=total,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
totals=totals,
|
||||
),
|
||||
skipped,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def _parse_experiences(
|
||||
cls,
|
||||
raw: Mapping[str, object],
|
||||
*,
|
||||
limit: int,
|
||||
offset: int,
|
||||
skip_invalid: bool,
|
||||
) -> tuple[HySdkMemoryPage, int]:
|
||||
"""``list_memories(scene="working"|"coding")``: that scene's experiences.
|
||||
|
||||
The SDK names the scene once, in ``variant``; each row is a bare
|
||||
experience, so the kind and variant are added before parsing.
|
||||
"""
|
||||
rows = raw.get("experiences")
|
||||
total = raw.get("total")
|
||||
variant = raw.get("variant")
|
||||
if (
|
||||
not isinstance(rows, list)
|
||||
or not isinstance(total, int)
|
||||
or isinstance(total, bool)
|
||||
or total < 0
|
||||
or variant not in {"working", "coding"}
|
||||
):
|
||||
raise HySdkReadError("hy_memory_list_shape_unknown")
|
||||
memories: list[HySdkMemoryRecord] = []
|
||||
skipped = 0
|
||||
for row in rows:
|
||||
try:
|
||||
if not isinstance(row, Mapping):
|
||||
raise HySdkReadError("hy_memory_record_shape_unknown")
|
||||
record = HySdkMemoryRecord.from_sdk(
|
||||
{**row, "kind": "experience", "variant": variant}
|
||||
)
|
||||
except HySdkReadError:
|
||||
if not skip_invalid:
|
||||
raise
|
||||
skipped += 1
|
||||
continue
|
||||
memories.append(record)
|
||||
return (
|
||||
cls(memories=tuple(memories), total=total, limit=limit, offset=offset),
|
||||
skipped,
|
||||
)
|
||||
|
||||
|
||||
class HySdkRawRecord(BaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
@@ -327,38 +542,79 @@ def parse_sdk_search(raw: object, *, error_code: str) -> tuple[SemanticMemoryHit
|
||||
return tuple(found)
|
||||
|
||||
|
||||
def library_keyword_hits(raw: object, query: str) -> tuple[tuple[str, ...], int]:
|
||||
"""Ids in an SDK search window whose text contains every query term.
|
||||
class LibraryHit(NamedTuple):
|
||||
"""One keyword match in a library search window, with its namespace."""
|
||||
|
||||
memory_id: str
|
||||
kind: MemoryKindName
|
||||
variant: MemoryVariantName | None
|
||||
|
||||
|
||||
def library_keyword_hits(raw: object, query: str) -> tuple[tuple[LibraryHit, ...], int]:
|
||||
"""Memories in an SDK search window whose text contains every query term.
|
||||
|
||||
The local SDK store has no keyword channel (Chroma's keyword search is
|
||||
empty), so search is pure vector similarity: a word in the library can
|
||||
score below the floor, and a word in none of it still scores above. The
|
||||
library search asks the SDK for its most relevant window and keeps only the
|
||||
rows that actually contain the words, most relevant first. Returns those ids
|
||||
and how many distinct rows the window held.
|
||||
rows that actually contain the words, most relevant first. Returns those
|
||||
hits and how many distinct rows the window held.
|
||||
|
||||
A ``search(scene="all")`` window mixes chat memories with working/coding
|
||||
experiences and facts; their text is the unified ``content`` (an
|
||||
experience's task and solution, a fact's statement). The fused order is
|
||||
kept as the SDK ranked it.
|
||||
"""
|
||||
rows = _reply_rows(raw, "hy_memory_search_shape_unknown")
|
||||
terms = query.casefold().split()
|
||||
seen: set[str] = set()
|
||||
matched: list[str] = []
|
||||
for item in sorted(rows, key=_score, reverse=True):
|
||||
memory_id = item.get("memory_id") or item.get("id")
|
||||
if not isinstance(memory_id, str) or not memory_id or memory_id in seen:
|
||||
if isinstance(raw, Mapping) and raw.get("scene") == "all":
|
||||
rows = raw.get("memories")
|
||||
if not isinstance(rows, list) or any(not isinstance(row, Mapping) for row in rows):
|
||||
raise HySdkReadError("hy_memory_search_shape_unknown")
|
||||
ordered: list[Mapping[str, object]] = list(rows)
|
||||
else:
|
||||
ordered = sorted(
|
||||
_reply_rows(raw, "hy_memory_search_shape_unknown"), key=_score, reverse=True
|
||||
)
|
||||
seen: set[LibraryHit] = set()
|
||||
matched: list[LibraryHit] = []
|
||||
for item in ordered:
|
||||
hit = _library_hit(item)
|
||||
if hit is None or hit in seen:
|
||||
continue
|
||||
seen.add(memory_id)
|
||||
content = item.get("content")
|
||||
seen.add(hit)
|
||||
tags = item.get("tags")
|
||||
text = " ".join(
|
||||
[
|
||||
content if isinstance(content, str) else "",
|
||||
*(tag for tag in (tags if isinstance(tags, list) else ()) if isinstance(tag, str)),
|
||||
]
|
||||
).casefold()
|
||||
parts = [item.get(key) for key in ("content", "task", "solution", "statement")] + list(
|
||||
tags if isinstance(tags, list) else ()
|
||||
)
|
||||
text = " ".join(part for part in parts if isinstance(part, str)).casefold()
|
||||
if all(term in text for term in terms):
|
||||
matched.append(memory_id)
|
||||
matched.append(hit)
|
||||
return tuple(matched), len(seen)
|
||||
|
||||
|
||||
def _library_hit(item: Mapping[str, object]) -> LibraryHit | None:
|
||||
kind = item.get("kind")
|
||||
if kind in {"experience", "fact"}:
|
||||
variant = item.get("variant") if item.get("variant") in {"working", "coding"} else None
|
||||
variant = variant or (
|
||||
item.get("scene") if item.get("scene") in {"working", "coding"} else None
|
||||
)
|
||||
memory_id = item.get("memory_id") or item.get(
|
||||
"experience_id" if kind == "experience" else "fact_id"
|
||||
)
|
||||
if variant is None or not isinstance(memory_id, str) or not memory_id:
|
||||
return None
|
||||
return LibraryHit(
|
||||
memory_id,
|
||||
"experience" if kind == "experience" else "fact",
|
||||
"coding" if variant == "coding" else "working",
|
||||
)
|
||||
memory_id = item.get("memory_id") or item.get("id")
|
||||
if not isinstance(memory_id, str) or not memory_id:
|
||||
return None
|
||||
return LibraryHit(memory_id, "chat", None)
|
||||
|
||||
|
||||
def parse_sdk_raw_search(raw: object) -> tuple[HySdkRawHit, ...]:
|
||||
return tuple(
|
||||
HySdkRawHit.from_sdk_search(row)
|
||||
@@ -366,6 +622,25 @@ def parse_sdk_raw_search(raw: object) -> tuple[HySdkRawHit, ...]:
|
||||
)
|
||||
|
||||
|
||||
def _epoch(value: object) -> int | None:
|
||||
"""Epoch seconds from an int or an ISO-8601 string (naive means UTC)."""
|
||||
if isinstance(value, bool):
|
||||
return None
|
||||
if isinstance(value, int):
|
||||
return value
|
||||
if isinstance(value, float):
|
||||
return int(value)
|
||||
if isinstance(value, str) and value.strip():
|
||||
try:
|
||||
parsed = datetime.fromisoformat(value.strip())
|
||||
except ValueError:
|
||||
return None
|
||||
if parsed.tzinfo is None:
|
||||
parsed = parsed.replace(tzinfo=UTC)
|
||||
return int(parsed.timestamp())
|
||||
return None
|
||||
|
||||
|
||||
def _string_tuple(value: object) -> tuple[str, ...]:
|
||||
if not isinstance(value, list):
|
||||
return ()
|
||||
|
||||
@@ -662,11 +662,15 @@ class ImportRunner:
|
||||
and self.sdk_queue is not None
|
||||
and event.cursor.read_to_end
|
||||
):
|
||||
# History ends at this end of file, so its last turn
|
||||
# and a short batch are written. Live polling keeps
|
||||
# them for the next increment or the idle flush.
|
||||
self.sdk_queue.finalize_sdk_file(
|
||||
user_id=current.user_id,
|
||||
source_id=current.source_id,
|
||||
file_key=event.cursor.file_key,
|
||||
connection=connection,
|
||||
flush=not self._live_import,
|
||||
)
|
||||
continue
|
||||
if self.on_stable_evidence is not None and event.cursor.read_to_end:
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
"""Short-lived result windows for the memory library's keyword search.
|
||||
|
||||
A library search asks the SDK once for a bounded relevance window and keeps
|
||||
the keyword matches. Paging must cut every page from that same list: running
|
||||
the search again per page can rank differently and duplicate or skip memories
|
||||
between pages. The matches are kept here under an opaque ``window_id`` that
|
||||
the client passes back with the next offset.
|
||||
|
||||
A window is bound to the filters it was searched with (query, source,
|
||||
statuses, scene). It expires after ``ttl_seconds`` without being read, and the
|
||||
store keeps at most ``max_windows`` windows, dropping the least recently read
|
||||
first.
|
||||
Only memory identities are kept, never content: each page is read fresh from
|
||||
the SDK, so an edit or a delete shows on the next page read.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import secrets
|
||||
import threading
|
||||
import time
|
||||
from collections import OrderedDict
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from one_memory.hy_sdk_types import LibraryHit
|
||||
|
||||
DEFAULT_WINDOW_TTL_SECONDS = 300.0
|
||||
DEFAULT_MAX_WINDOWS = 32
|
||||
|
||||
WindowKey = tuple[str, str | None, tuple[str, ...], str]
|
||||
|
||||
|
||||
def window_key(
|
||||
query: str, source: str | None, statuses: list[str], scene: str = "all"
|
||||
) -> WindowKey:
|
||||
return (query, source, tuple(sorted(set(statuses))), scene)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SearchWindow:
|
||||
window_id: str
|
||||
key: WindowKey
|
||||
hits: tuple[LibraryHit, ...]
|
||||
# Set when the SDK window was full, so matches outside it were not looked
|
||||
# for and `total` may undercount.
|
||||
search_window: int | None
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Entry:
|
||||
window: SearchWindow
|
||||
last_read: float = field(default=0.0)
|
||||
|
||||
|
||||
class LibrarySearchWindows:
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
ttl_seconds: float = DEFAULT_WINDOW_TTL_SECONDS,
|
||||
max_windows: int = DEFAULT_MAX_WINDOWS,
|
||||
clock: Callable[[], float] = time.monotonic,
|
||||
) -> None:
|
||||
self._ttl = ttl_seconds
|
||||
self._max = max(1, max_windows)
|
||||
self._clock = clock
|
||||
self._entries: OrderedDict[str, _Entry] = OrderedDict()
|
||||
self._lock = threading.Lock()
|
||||
|
||||
def get(self, window_id: str, key: WindowKey) -> SearchWindow | None:
|
||||
"""The window, if it is still live and was searched with `key`."""
|
||||
with self._lock:
|
||||
now = self._clock()
|
||||
self._expire(now)
|
||||
entry = self._entries.get(window_id)
|
||||
if entry is None or entry.window.key != key:
|
||||
return None
|
||||
entry.last_read = now
|
||||
self._entries.move_to_end(window_id)
|
||||
return entry.window
|
||||
|
||||
def put(
|
||||
self,
|
||||
key: WindowKey,
|
||||
hits: tuple[LibraryHit, ...],
|
||||
search_window: int | None,
|
||||
) -> SearchWindow:
|
||||
window = SearchWindow(
|
||||
window_id=secrets.token_urlsafe(12),
|
||||
key=key,
|
||||
hits=hits,
|
||||
search_window=search_window,
|
||||
)
|
||||
with self._lock:
|
||||
now = self._clock()
|
||||
self._expire(now)
|
||||
self._entries[window.window_id] = _Entry(window=window, last_read=now)
|
||||
while len(self._entries) > self._max:
|
||||
self._entries.popitem(last=False)
|
||||
return window
|
||||
|
||||
def __len__(self) -> int:
|
||||
with self._lock:
|
||||
return len(self._entries)
|
||||
|
||||
def _expire(self, now: float) -> None:
|
||||
stale = [
|
||||
window_id
|
||||
for window_id, entry in self._entries.items()
|
||||
if now - entry.last_read >= self._ttl
|
||||
]
|
||||
for window_id in stale:
|
||||
del self._entries[window_id]
|
||||
@@ -79,9 +79,9 @@ SDK_MIGRATION_CAPTURE_FENCE = ".sdk-migration-capture-fence.json"
|
||||
|
||||
|
||||
def database_path() -> Path:
|
||||
from one_memory.workbuddy_boundary import database_path as boundary_database
|
||||
from one_memory.config import load_settings
|
||||
|
||||
return boundary_database()
|
||||
return load_settings().data_dir / "one-memory.sqlite3"
|
||||
|
||||
|
||||
def capture_stdio(source_id: str, stdin: TextIO, output: TextIO) -> int:
|
||||
@@ -105,10 +105,17 @@ def capture_stdio(source_id: str, stdin: TextIO, output: TextIO) -> int:
|
||||
|
||||
|
||||
def sdk_migration_capture_fenced(data_dir: Path) -> bool:
|
||||
"""Ignore stale fences; only a live Desktop migration pauses hook writes."""
|
||||
"""Ignore stale fences; a live migration or a failed one pauses hook writes.
|
||||
|
||||
A failed upgrade migration leaves the fence `frozen` (no pid): the home was
|
||||
put back as it was before the upgrade and takes no writes until the next
|
||||
migration attempt, which replaces the fence and lifts it on success.
|
||||
"""
|
||||
marker = data_dir / SDK_MIGRATION_CAPTURE_FENCE
|
||||
try:
|
||||
document = json.loads(marker.read_text(encoding="utf-8"))
|
||||
if isinstance(document, dict) and document.get("frozen") is True:
|
||||
return True
|
||||
pid = int(document["pid"])
|
||||
if pid < 1:
|
||||
raise ValueError("invalid migration fence pid")
|
||||
@@ -249,24 +256,19 @@ def record_capture_rejection(
|
||||
|
||||
|
||||
def _submit_live_memory(store: EvidenceStore, record: EvidenceRecord, *, changed: bool) -> None:
|
||||
"""Add the record to its conversation's live assembling row.
|
||||
|
||||
The turn rule decides what is written (`turn_batches`): a turn is
|
||||
complete when the next user message arrives, batches are 5 turns or
|
||||
8000 characters, and Core writes the rest after 10 quiet minutes.
|
||||
"""
|
||||
if not changed or not _sync_open(store.database_path, record.source_id):
|
||||
return
|
||||
from one_memory.conversation_processing import _conversation_chunks
|
||||
from one_memory.memory_server.durable_queue import DurableWriteQueue
|
||||
from one_memory.memory_server.unified_entry import UnifiedWriteIntake
|
||||
|
||||
intake = UnifiedWriteIntake(DurableWriteQueue(store.database_path, evidence=store))
|
||||
for chunk in _conversation_chunks(
|
||||
store.list_conversation(record.evidence_id),
|
||||
max_turns=12,
|
||||
completed_only=True,
|
||||
):
|
||||
if (
|
||||
record in chunk
|
||||
and any(item.role == "user" for item in chunk)
|
||||
and any(item.role == "assistant" for item in chunk)
|
||||
):
|
||||
intake.submit_records(chunk, priority="live")
|
||||
DurableWriteQueue(store.database_path, evidence=store).assemble_live_records(
|
||||
(record,), file_key=f"hook:{record.conversation_id}"
|
||||
)
|
||||
|
||||
|
||||
def _conversation(payload: dict[str, object]) -> str | None:
|
||||
|
||||
+80
-188
@@ -1,5 +1,13 @@
|
||||
"""Add one memory directly: `POST /api/v2/memories` and its status read.
|
||||
|
||||
The public API has no "notes": a hand-added memory is filed as manual Evidence
|
||||
and turned into memories by the durable write queue. Storage keeps its
|
||||
pre-rename identifiers (`memory_note_requests`, source `one-memory-manual`,
|
||||
`input_mode=direct_memory_note`, conversation `note:<request_id>`) because they
|
||||
are persisted and feed evidence ids; renaming them would change stored data.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import sqlite3
|
||||
from collections.abc import Sequence
|
||||
from datetime import UTC, datetime
|
||||
@@ -28,7 +36,7 @@ from one_memory.memory_tasks import MemoryTaskStatus, MemoryTaskStore
|
||||
from one_memory.secrets import SecretStoreUnavailableError
|
||||
|
||||
SOURCE_ID = "one-memory-manual"
|
||||
MemoryNoteState = Literal[
|
||||
MemoryAddState = Literal[
|
||||
"waiting_for_model",
|
||||
"waiting_for_permission",
|
||||
"pending",
|
||||
@@ -39,7 +47,7 @@ MemoryNoteState = Literal[
|
||||
]
|
||||
|
||||
|
||||
class MemoryNoteInput(BaseModel):
|
||||
class MemoryAddInput(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
request_id: UUID
|
||||
@@ -50,39 +58,25 @@ class MemoryNoteInput(BaseModel):
|
||||
@classmethod
|
||||
def meaningful_text(cls, value: str) -> str:
|
||||
if not value.strip():
|
||||
raise ValueError("memory_note_empty")
|
||||
raise ValueError("memory_add_empty")
|
||||
return value
|
||||
|
||||
|
||||
class MemoryNoteReceipt(BaseModel):
|
||||
class MemoryAddReceipt(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
evidence_id: str
|
||||
text: str
|
||||
scope: Literal["all-apps"] = "all-apps"
|
||||
created_at: datetime
|
||||
state: MemoryNoteState
|
||||
state: MemoryAddState
|
||||
task_id: str | None = None
|
||||
memory_ids: list[str] = Field(default_factory=list)
|
||||
error: str | None = None
|
||||
|
||||
|
||||
class SavedMemoryNoteEvidence(BaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
available: Literal[True] = True
|
||||
evidence_id: str
|
||||
source_id: str
|
||||
revision: int
|
||||
imported_at: datetime
|
||||
role: str
|
||||
written_by: WrittenBy
|
||||
created_at: datetime | None
|
||||
verbatim: str
|
||||
|
||||
|
||||
class LegacyMemoryNoteReader:
|
||||
"""Read pre-cutover note receipts without retaining any write runtime."""
|
||||
class LegacyMemoryAdditionReader:
|
||||
"""Read pre-cutover addition receipts without retaining any write runtime."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
@@ -93,7 +87,7 @@ class LegacyMemoryNoteReader:
|
||||
self.governance = governance
|
||||
|
||||
|
||||
class MemoryNoteStore:
|
||||
class MemoryAdditionStore:
|
||||
def __init__(
|
||||
self,
|
||||
evidence: EvidenceStore,
|
||||
@@ -106,44 +100,52 @@ class MemoryNoteStore:
|
||||
self.evidence = evidence
|
||||
self.processor = processor
|
||||
# D52: the production composition root passes a UnifiedWriteIntake so a
|
||||
# manual note reaches the durable queue like every other entry. It stays
|
||||
# manual addition reaches the durable queue like every other entry. It stays
|
||||
# optional so isolated tests can keep constructing the store directly.
|
||||
self.intake = intake
|
||||
self.sdk_authority = sdk_authority
|
||||
self.write_queue = write_queue
|
||||
|
||||
def save(self, note: MemoryNoteInput) -> EvidenceRecord:
|
||||
def save(self, addition: MemoryAddInput) -> EvidenceRecord:
|
||||
now = datetime.now(UTC)
|
||||
value = EvidenceInput(
|
||||
source_id=SOURCE_ID,
|
||||
conversation_id=f"note:{note.request_id}",
|
||||
message_id=str(note.request_id),
|
||||
conversation_id=f"note:{addition.request_id}",
|
||||
message_id=str(addition.request_id),
|
||||
role="user",
|
||||
written_by=WrittenBy.PERSON,
|
||||
created_at=now,
|
||||
verbatim=note.text,
|
||||
verbatim=addition.text,
|
||||
origin_meta={
|
||||
EvidenceOrigin.MANUAL_IMPORT: OriginMetadata(
|
||||
raw_meta={
|
||||
"input_mode": "direct_memory_note",
|
||||
"recall_scope": note.scope,
|
||||
"recall_scope": addition.scope,
|
||||
}
|
||||
)
|
||||
},
|
||||
)
|
||||
if self.sdk_authority:
|
||||
if self.intake is None or self.write_queue is None:
|
||||
raise RuntimeError("sdk_memory_note_intake_unavailable")
|
||||
raise RuntimeError("sdk_memory_add_intake_unavailable")
|
||||
record = materialize_write_input(value, observed_at=now)
|
||||
with main_database(self.write_queue.database_path).read() as database:
|
||||
# No grant row is written here: a missing row means allowed,
|
||||
# only an explicit revocation refuses (same as the claim gate).
|
||||
grant = database.execute(
|
||||
"SELECT revoked_at FROM source_grants WHERE user_id='local' AND source_id=?",
|
||||
(SOURCE_ID,),
|
||||
).fetchone()
|
||||
existing = database.execute(
|
||||
"SELECT request_id FROM memory_note_requests WHERE note_id=?",
|
||||
(record.evidence_id,),
|
||||
).fetchone()
|
||||
if grant is not None and grant[0] is not None:
|
||||
raise ValueError("memory_add_source_revoked")
|
||||
if existing is not None:
|
||||
current = self.require(record.evidence_id)
|
||||
if current.verbatim != record.verbatim:
|
||||
raise ValueError("memory_note_request_conflict")
|
||||
raise ValueError("memory_add_conflict")
|
||||
return current
|
||||
receipt = self.intake.submit_records(
|
||||
(record,),
|
||||
@@ -173,13 +175,13 @@ class MemoryNoteStore:
|
||||
(SOURCE_ID,),
|
||||
).fetchone()
|
||||
if grant and grant[0] is not None:
|
||||
raise ValueError("memory_note_source_revoked")
|
||||
raise ValueError("memory_add_source_revoked")
|
||||
existing = db.execute(
|
||||
"SELECT verbatim FROM evidence WHERE evidence_id=?", (value.evidence_id,)
|
||||
).fetchone()
|
||||
if existing:
|
||||
if existing[0] != note.text:
|
||||
raise ValueError("memory_note_request_conflict")
|
||||
if existing[0] != addition.text:
|
||||
raise ValueError("memory_add_conflict")
|
||||
else:
|
||||
self.evidence.put(value, connection=db)
|
||||
db.execute(
|
||||
@@ -232,10 +234,10 @@ class MemoryNoteStore:
|
||||
(SOURCE_ID,),
|
||||
).fetchone()
|
||||
if grant is None or grant[0] is not None:
|
||||
raise ValueError("memory_note_source_revoked")
|
||||
raise ValueError("memory_add_source_revoked")
|
||||
return record
|
||||
|
||||
def receipt(self, evidence_id: str) -> MemoryNoteReceipt:
|
||||
def receipt(self, evidence_id: str) -> MemoryAddReceipt:
|
||||
record = self.require(evidence_id)
|
||||
if self.sdk_authority:
|
||||
assert self.write_queue is not None
|
||||
@@ -246,7 +248,7 @@ class MemoryNoteStore:
|
||||
).fetchone()[0]
|
||||
durable = self.write_queue.get(str(request_id))
|
||||
durable_state = durable.state if durable is not None else "failed"
|
||||
mapped: MemoryNoteState = {
|
||||
mapped: MemoryAddState = {
|
||||
"pending": "pending",
|
||||
"running": "running",
|
||||
"waiting_for_model": "waiting_for_model",
|
||||
@@ -255,7 +257,7 @@ class MemoryNoteStore:
|
||||
"ready" if durable is not None and durable.sdk_memory_ids else "no_memories"
|
||||
),
|
||||
}.get(durable_state, "failed")
|
||||
return MemoryNoteReceipt(
|
||||
return MemoryAddReceipt(
|
||||
evidence_id=evidence_id,
|
||||
text=sanitize_model_input(record.verbatim),
|
||||
created_at=record.created_at or record.imported_at,
|
||||
@@ -264,7 +266,7 @@ class MemoryNoteStore:
|
||||
error=durable.detail if durable is not None else "write_missing",
|
||||
)
|
||||
if self.processor is None:
|
||||
raise RuntimeError("legacy_memory_note_processor_unavailable")
|
||||
raise RuntimeError("legacy_memory_add_processor_unavailable")
|
||||
task = self.processor.tasks.task_for_evidence(evidence_id)
|
||||
memory_ids = []
|
||||
if task:
|
||||
@@ -272,7 +274,7 @@ class MemoryNoteStore:
|
||||
memory = self.processor.governance.get(output.memory_id, user_id="local")
|
||||
if memory and memory.status is MemoryStatus.CONFIRMED:
|
||||
memory_ids.append(memory.memory_id)
|
||||
state: MemoryNoteState
|
||||
state: MemoryAddState
|
||||
if task is None or task.status == MemoryTaskStatus.WAITING_FOR_MODEL:
|
||||
state = "waiting_for_model"
|
||||
elif task.status == MemoryTaskStatus.WAITING_FOR_PERMISSION:
|
||||
@@ -288,7 +290,7 @@ class MemoryNoteStore:
|
||||
else:
|
||||
state = "failed"
|
||||
error = task.last_error_code if task else None
|
||||
return MemoryNoteReceipt(
|
||||
return MemoryAddReceipt(
|
||||
evidence_id=record.evidence_id,
|
||||
text=sanitize_model_input(record.verbatim),
|
||||
created_at=record.created_at or record.imported_at,
|
||||
@@ -298,33 +300,7 @@ class MemoryNoteStore:
|
||||
error=error,
|
||||
)
|
||||
|
||||
def list(self) -> list[MemoryNoteReceipt]:
|
||||
if self.sdk_authority:
|
||||
assert self.write_queue is not None
|
||||
with main_database(self.write_queue.database_path).read() as database:
|
||||
ids = [
|
||||
str(row[0])
|
||||
for row in database.execute(
|
||||
"SELECT note_id FROM memory_note_requests ORDER BY created_at DESC LIMIT 50"
|
||||
)
|
||||
]
|
||||
return [self.receipt(key) for key in ids]
|
||||
with main_database(self.evidence.database_path).read() as db:
|
||||
ids = [
|
||||
row[0]
|
||||
for row in db.execute(
|
||||
"SELECT e.evidence_id,o.raw_meta_json FROM evidence e "
|
||||
"JOIN source_grants g USING(user_id,source_id) "
|
||||
"JOIN evidence_origins o USING(evidence_id) WHERE e.user_id='local' "
|
||||
"AND e.source_id=? AND g.revoked_at IS NULL AND o.origin='manual_import' "
|
||||
"ORDER BY e.imported_at DESC,e.evidence_id LIMIT 50",
|
||||
(SOURCE_ID,),
|
||||
)
|
||||
if json.loads(row[1]).get("input_mode") == "direct_memory_note"
|
||||
]
|
||||
return [self.receipt(key) for key in ids]
|
||||
|
||||
def receipt_for_request(self, request_id: UUID) -> MemoryNoteReceipt:
|
||||
def receipt_for_request(self, request_id: UUID) -> MemoryAddReceipt:
|
||||
identity = str(request_id)
|
||||
if self.sdk_authority:
|
||||
probe = EvidenceInput(
|
||||
@@ -352,149 +328,65 @@ class MemoryNoteStore:
|
||||
return self.receipt(str(row[0]))
|
||||
|
||||
|
||||
def memory_note_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
router = APIRouter(
|
||||
prefix="/api/v1/memory-notes", tags=["memory-notes"], dependencies=dependencies
|
||||
)
|
||||
def memory_add_router(dependencies: Sequence[Depends]) -> APIRouter:
|
||||
"""`POST /api/v2/memories` and `GET /api/v2/memories/additions/{request_id}`.
|
||||
|
||||
def store(request: Request) -> MemoryNoteStore:
|
||||
return cast(MemoryNoteStore, request.app.state.memory_note_store)
|
||||
Both paths are outside the single-segment `/api/v2/memories/{memory_id}`
|
||||
routes; external (enveloped) callers are refused by
|
||||
`require_external_endpoint` because neither is a broker read route.
|
||||
"""
|
||||
router = APIRouter(prefix="/api/v2/memories", tags=["memories"], dependencies=dependencies)
|
||||
|
||||
async def schedule(
|
||||
notes: MemoryNoteStore,
|
||||
record: EvidenceRecord,
|
||||
*,
|
||||
readmit: bool = False,
|
||||
) -> MemoryNoteReceipt:
|
||||
def store(request: Request) -> MemoryAdditionStore:
|
||||
return cast(MemoryAdditionStore, request.app.state.memory_addition_store)
|
||||
|
||||
async def schedule(additions: MemoryAdditionStore, record: EvidenceRecord) -> None:
|
||||
try:
|
||||
if notes.sdk_authority:
|
||||
return await asyncio.to_thread(notes.receipt, record.evidence_id)
|
||||
if notes.processor is None:
|
||||
raise RuntimeError("legacy_memory_note_processor_unavailable")
|
||||
if notes.intake is None:
|
||||
if additions.sdk_authority:
|
||||
return
|
||||
if additions.processor is None:
|
||||
raise RuntimeError("legacy_memory_add_processor_unavailable")
|
||||
if additions.intake is None:
|
||||
# Isolated tests construct the store without a queue.
|
||||
await notes.processor.enqueue_conversation_off_thread(record)
|
||||
await additions.processor.enqueue_conversation_off_thread(record)
|
||||
else:
|
||||
# Production: submit to the durable queue first. The drain runs
|
||||
# the same supervisor path, so create/process semantics are
|
||||
# unchanged; replaying the same request returns the original
|
||||
# `entry:<evidence_id>` job instead of a second HY call.
|
||||
if readmit:
|
||||
# `/{evidence_id}/process` is the explicit retry signal.
|
||||
# It also revives pre-upgrade permission waiters.
|
||||
await asyncio.to_thread(notes.intake.queue.readmit_parked)
|
||||
submitted = await asyncio.to_thread(
|
||||
notes.intake.submit_evidence,
|
||||
# Production: submit to the durable queue. Replaying the same
|
||||
# request returns the original `entry:<evidence_id>` job
|
||||
# instead of a second HY call.
|
||||
await asyncio.to_thread(
|
||||
additions.intake.submit_evidence,
|
||||
record,
|
||||
first_round=1,
|
||||
last_round=1,
|
||||
priority="live",
|
||||
)
|
||||
if readmit:
|
||||
await asyncio.to_thread(
|
||||
notes.intake.readmit_cancelled_evidence,
|
||||
record,
|
||||
job_id=submitted.record.job_id,
|
||||
first_round=1,
|
||||
last_round=1,
|
||||
priority="live",
|
||||
)
|
||||
except SecretStoreUnavailableError:
|
||||
saved = await asyncio.to_thread(notes.receipt, record.evidence_id)
|
||||
return saved.model_copy(
|
||||
update={"state": "waiting_for_model", "error": "secure_storage_unavailable"}
|
||||
)
|
||||
return await asyncio.to_thread(notes.receipt, record.evidence_id)
|
||||
# The receipt already reports the saved text; the queue retries
|
||||
# once secure storage is back.
|
||||
return
|
||||
|
||||
@router.post("", response_model=MemoryNoteReceipt, status_code=201)
|
||||
async def create(
|
||||
note: MemoryNoteInput,
|
||||
@router.post("", response_model=MemoryAddReceipt, status_code=201)
|
||||
async def add_memory(
|
||||
addition: MemoryAddInput,
|
||||
request: Request,
|
||||
background: BackgroundTasks,
|
||||
) -> MemoryNoteReceipt:
|
||||
notes = store(request)
|
||||
) -> MemoryAddReceipt:
|
||||
additions = store(request)
|
||||
try:
|
||||
record = await asyncio.to_thread(notes.save, note)
|
||||
record = await asyncio.to_thread(additions.save, addition)
|
||||
except ValueError as error:
|
||||
raise HTTPException(409, str(error)) from error
|
||||
saved = await asyncio.to_thread(notes.receipt, record.evidence_id)
|
||||
background.add_task(schedule, notes, record)
|
||||
saved = await asyncio.to_thread(additions.receipt, record.evidence_id)
|
||||
background.add_task(schedule, additions, record)
|
||||
return saved
|
||||
|
||||
@router.get("", response_model=list[MemoryNoteReceipt])
|
||||
def listing(request: Request) -> list[MemoryNoteReceipt]:
|
||||
return store(request).list()
|
||||
|
||||
@router.get("/operations/{request_id}", response_model=MemoryNoteReceipt)
|
||||
def operation(request_id: UUID, request: Request) -> MemoryNoteReceipt:
|
||||
@router.get("/additions/{request_id}", response_model=MemoryAddReceipt)
|
||||
def memory_add_status(request_id: UUID, request: Request) -> MemoryAddReceipt:
|
||||
try:
|
||||
return store(request).receipt_for_request(request_id)
|
||||
except KeyError as error:
|
||||
raise HTTPException(404, "memory_note_operation_not_found") from error
|
||||
|
||||
@router.get("/{evidence_id}", response_model=MemoryNoteReceipt)
|
||||
def get(evidence_id: str, request: Request) -> MemoryNoteReceipt:
|
||||
try:
|
||||
return store(request).receipt(evidence_id)
|
||||
except KeyError as error:
|
||||
raise HTTPException(404, "memory_note_not_found") from error
|
||||
raise HTTPException(404, "memory_add_not_found") from error
|
||||
except ValueError as error:
|
||||
raise HTTPException(409, str(error)) from error
|
||||
|
||||
@router.get("/{evidence_id}/evidence", response_model=SavedMemoryNoteEvidence)
|
||||
def evidence(evidence_id: str, request: Request) -> SavedMemoryNoteEvidence:
|
||||
try:
|
||||
notes = store(request)
|
||||
if notes.sdk_authority:
|
||||
raise HTTPException(409, "memory_note_evidence_retired")
|
||||
with notes.evidence._connect() as connection:
|
||||
record = notes.evidence._get_with_connection(connection, evidence_id)
|
||||
metadata = record.origin_meta.get(EvidenceOrigin.MANUAL_IMPORT) if record else None
|
||||
if (
|
||||
record is None
|
||||
or record.user_id != "local"
|
||||
or record.source_id != SOURCE_ID
|
||||
or metadata is None
|
||||
or metadata.raw_meta.get("input_mode") != "direct_memory_note"
|
||||
):
|
||||
raise KeyError(evidence_id)
|
||||
grant = connection.execute(
|
||||
"SELECT revoked_at FROM source_grants WHERE user_id='local' AND source_id=?",
|
||||
(SOURCE_ID,),
|
||||
).fetchone()
|
||||
if grant is None or grant[0] is not None:
|
||||
raise ValueError("memory_note_source_revoked")
|
||||
result = SavedMemoryNoteEvidence(
|
||||
evidence_id=record.evidence_id,
|
||||
source_id=record.source_id,
|
||||
revision=record.revision,
|
||||
imported_at=record.imported_at,
|
||||
role=record.role,
|
||||
written_by=record.written_by,
|
||||
created_at=record.created_at,
|
||||
verbatim=record.verbatim,
|
||||
)
|
||||
# The returned body and reference share one snapshot. Recheck authority
|
||||
# after that read so a revoked source is not released from a stale view.
|
||||
notes.require(evidence_id)
|
||||
return result
|
||||
except KeyError as error:
|
||||
raise HTTPException(404, "memory_note_not_found") from error
|
||||
except ValueError as error:
|
||||
raise HTTPException(409, str(error)) from error
|
||||
|
||||
@router.post("/{evidence_id}/process", response_model=MemoryNoteReceipt)
|
||||
async def process(evidence_id: str, request: Request) -> MemoryNoteReceipt:
|
||||
notes = store(request)
|
||||
try:
|
||||
record = await asyncio.to_thread(notes.require, evidence_id)
|
||||
except KeyError as error:
|
||||
raise HTTPException(404, "memory_note_not_found") from error
|
||||
except ValueError as error:
|
||||
raise HTTPException(409, str(error)) from error
|
||||
# This route is the explicit "the condition that parked it may have
|
||||
# changed" signal (the person just granted consent or fixed the model),
|
||||
# which is when `readmit_parked` is meant to run.
|
||||
return await schedule(notes, record, readmit=True)
|
||||
|
||||
return router
|
||||
@@ -1,342 +0,0 @@
|
||||
"""Read-only local library projection over the existing governed memory store."""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.legacy_lifecycle_types import HyLifecycle
|
||||
from one_memory.memory_evidence_freshness import register_memory_freshness
|
||||
from one_memory.memory_governance import (
|
||||
GovernedMemory,
|
||||
MemoryGovernanceStore,
|
||||
MemoryStatus,
|
||||
lifecycle_predicate,
|
||||
)
|
||||
|
||||
#: Read budget for one library page. 2026-10-12: raised from 5s (which starved
|
||||
#: under a concurrent history import and surfaced as 503) to 15s, still well
|
||||
#: under the 30s client timeout; past it the read degrades to a `stale` page.
|
||||
READ_BUDGET_SECONDS = 15
|
||||
|
||||
|
||||
class MemorySourceTime(BaseModel):
|
||||
"""When the conversation a memory came from happened (D96).
|
||||
|
||||
`basis` is `conversation` when the cited messages carry their own time,
|
||||
`import` when only the import time is known; the UI words them apart.
|
||||
"""
|
||||
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
at: str
|
||||
basis: Literal["conversation", "import"]
|
||||
|
||||
|
||||
class MemoryLibraryPage(BaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
items: list[GovernedMemory]
|
||||
total: int = Field(ge=0)
|
||||
offset: int = Field(ge=0)
|
||||
limit: int = Field(ge=1, le=500)
|
||||
source_ids_by_memory: dict[str, list[str]] = Field(default_factory=dict)
|
||||
source_time_by_memory: dict[str, MemorySourceTime] = Field(default_factory=dict)
|
||||
#: True when the read budget ran out mid-page (typically during a history
|
||||
#: import): the counts and rows above are a partial snapshot, not an
|
||||
#: authoritative view. Clients should show "importing" instead of an error.
|
||||
stale: bool = False
|
||||
|
||||
|
||||
def _sort_by_conversation() -> bool:
|
||||
"""D99: the list reads as a timeline of the conversations, newest first.
|
||||
|
||||
A memory generated today from a three-week-old dialogue belongs next to
|
||||
that week, not at the top. ONE_MEMORY_LIBRARY_SORT=updated brings back the
|
||||
old "newest generated first" order without a code change.
|
||||
"""
|
||||
return os.environ.get("ONE_MEMORY_LIBRARY_SORT", "").strip().lower() != "updated"
|
||||
|
||||
|
||||
def _origin_join() -> str:
|
||||
"""The newest cited message per memory, as one joined aggregate.
|
||||
|
||||
A correlated subquery would re-scan the link table for every candidate row;
|
||||
this is a single pass. Memories with no readable citation keep their old
|
||||
position through COALESCE.
|
||||
"""
|
||||
if not _sort_by_conversation():
|
||||
return ""
|
||||
return """LEFT JOIN (
|
||||
SELECT link.memory_id AS memory_id,
|
||||
MAX(COALESCE(e.created_at, e.imported_at)) AS origin_at
|
||||
FROM governed_memory_evidence link
|
||||
JOIN evidence e ON e.evidence_id = link.evidence_id
|
||||
WHERE link.active = 1
|
||||
GROUP BY link.memory_id
|
||||
) origin ON origin.memory_id = m.memory_id"""
|
||||
|
||||
|
||||
def _order_by() -> str:
|
||||
if not _sort_by_conversation():
|
||||
return "m.updated_at DESC, m.memory_id ASC"
|
||||
# Conversations that share a timestamp keep the generated order between
|
||||
# themselves; ties would otherwise fall back to the id and look arbitrary.
|
||||
return "COALESCE(origin.origin_at, m.updated_at) DESC, m.updated_at DESC, m.memory_id ASC"
|
||||
|
||||
|
||||
class _LibraryQuery(BaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
|
||||
user_id: str = Field(min_length=1, max_length=256)
|
||||
query: str | None = None
|
||||
status: MemoryStatus | None = None
|
||||
active_only: bool = False
|
||||
source: str | None = Field(default=None, min_length=1)
|
||||
offset: int = Field(default=0, ge=0, le=2**63 - 1, strict=True)
|
||||
limit: int = Field(default=100, ge=1, le=500, strict=True)
|
||||
|
||||
|
||||
def read_conversation_memories(
|
||||
database_path: Path,
|
||||
*,
|
||||
user_id: str,
|
||||
source_id: str,
|
||||
conversation_id: str,
|
||||
workspace: str | None,
|
||||
limit: int = 100,
|
||||
) -> list[GovernedMemory]:
|
||||
"""Current memories backed by any active Evidence in one conversation."""
|
||||
with main_database(database_path).read() as db:
|
||||
db.row_factory = sqlite3.Row
|
||||
freshness = register_memory_freshness(db)
|
||||
rows = db.execute(
|
||||
f"""
|
||||
SELECT DISTINCT m.* FROM governed_memories m
|
||||
JOIN governed_memory_evidence link ON link.memory_id=m.memory_id AND link.active=1
|
||||
JOIN evidence e ON e.evidence_id=link.evidence_id
|
||||
WHERE m.user_id=? AND m.status='confirmed' AND m.deleted_at IS NULL
|
||||
AND e.user_id=? AND e.source_id=? AND e.conversation_id=?
|
||||
AND e.workspace IS ? AND {freshness}
|
||||
ORDER BY m.updated_at DESC, m.memory_id
|
||||
LIMIT ?
|
||||
""",
|
||||
(user_id, user_id, source_id, conversation_id, workspace, limit),
|
||||
).fetchall()
|
||||
return MemoryGovernanceStore.memories_from_rows(db, rows)
|
||||
|
||||
|
||||
def read_memory_library(
|
||||
database_path: Path,
|
||||
*,
|
||||
user_id: str = "local",
|
||||
query: str | None = None,
|
||||
status: MemoryStatus | None = None,
|
||||
active_only: bool = False,
|
||||
source: str | None = None,
|
||||
offset: int = 0,
|
||||
limit: int = 100,
|
||||
lifecycle: HyLifecycle | None = None,
|
||||
) -> MemoryLibraryPage:
|
||||
"""Read one page and its exact readable total from the same SQLite snapshot.
|
||||
|
||||
This is the local UI policy, not an external agent authorization entry point.
|
||||
Missing grants do not revoke already stored content; explicit revocations do.
|
||||
Read/parse failures propagate rather than masquerading as an empty library.
|
||||
Which memories are current is HY's `lifecycle` when given, as in
|
||||
`MemoryGovernanceStore.list`.
|
||||
"""
|
||||
request = _LibraryQuery(
|
||||
user_id=user_id,
|
||||
query=query,
|
||||
status=status,
|
||||
active_only=active_only,
|
||||
source=source,
|
||||
offset=offset,
|
||||
limit=limit,
|
||||
)
|
||||
if request.active_only and request.status is not None:
|
||||
raise ValueError("active_only_conflicts_with_status")
|
||||
clauses = ["m.user_id = ?"]
|
||||
values: list[str | int] = [request.user_id]
|
||||
selected, selected_values = lifecycle_predicate(
|
||||
lifecycle, MemoryStatus.CONFIRMED if request.active_only else request.status, alias="m"
|
||||
)
|
||||
if selected is not None:
|
||||
clauses.append(selected)
|
||||
values.extend(selected_values)
|
||||
if request.status != MemoryStatus.DELETED:
|
||||
clauses.append("m.deleted_at IS NULL")
|
||||
|
||||
# SourceReadAccess hides the whole memory if any active citation is revoked.
|
||||
# Do this before COUNT/LIMIT; a readable citation cannot rescue a mixed memory.
|
||||
clauses.append("""NOT EXISTS (
|
||||
SELECT 1 FROM governed_memory_evidence link
|
||||
JOIN evidence e ON e.evidence_id = link.evidence_id
|
||||
JOIN source_grants g ON g.user_id = e.user_id AND g.source_id = e.source_id
|
||||
WHERE link.memory_id = m.memory_id AND +link.active = 1
|
||||
AND g.revoked_at IS NOT NULL
|
||||
)""")
|
||||
# Governance normally guarantees ownership and referential integrity. Fail
|
||||
# closed for broken active links instead of exposing another user's citation.
|
||||
clauses.append("""NOT EXISTS (
|
||||
SELECT 1 FROM governed_memory_evidence link
|
||||
LEFT JOIN evidence e ON e.evidence_id = link.evidence_id
|
||||
WHERE link.memory_id = m.memory_id AND +link.active = 1
|
||||
AND (e.evidence_id IS NULL OR e.user_id != m.user_id)
|
||||
)""")
|
||||
if request.query:
|
||||
clauses.append("m.content LIKE ? ESCAPE '\\'")
|
||||
escaped = request.query.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
|
||||
values.append(f"%{escaped}%")
|
||||
if request.source is not None:
|
||||
# Start from the compact active-link index, then resolve each Evidence
|
||||
# identity by primary key. The old correlated EXISTS let SQLite scan
|
||||
# every Evidence row for every candidate memory on large libraries.
|
||||
clauses.append("""m.memory_id IN (
|
||||
SELECT link.memory_id FROM governed_memory_evidence link
|
||||
JOIN evidence e ON e.evidence_id = link.evidence_id
|
||||
WHERE link.active = 1 AND e.user_id = ? AND e.source_id = ?
|
||||
)""")
|
||||
values.extend((request.user_id, request.source))
|
||||
predicate = " AND ".join(clauses)
|
||||
|
||||
def _interrupted(error: Exception) -> bool:
|
||||
return isinstance(error, sqlite3.OperationalError) and "interrupted" in str(error).lower()
|
||||
|
||||
with main_database(database_path).read() as db:
|
||||
# 2026-10-12: 5s starved the page whenever a history import saturated
|
||||
# SQLite, and the caller turned the deadline abort into a 503, so the
|
||||
# memory page read as broken for the whole import. 15s stays well under
|
||||
# the 30s client timeout; past it we return a `stale` page instead of
|
||||
# an error — an incomplete list is honest, an error is not.
|
||||
deadline = time.monotonic() + READ_BUDGET_SECONDS
|
||||
db.set_progress_handler(lambda: int(time.monotonic() >= deadline), 1000)
|
||||
db.row_factory = sqlite3.Row
|
||||
db.execute("BEGIN")
|
||||
# The total counts with the cheap permission filters only: the per-row
|
||||
# freshness UDF scales with the whole library and can burn the whole
|
||||
# read budget on its own (D22). Product accepted a small drift here;
|
||||
# the rows that are actually displayed are still checked row by row.
|
||||
try:
|
||||
total = db.execute(
|
||||
f"SELECT COUNT(*) FROM governed_memories m WHERE {predicate}", values
|
||||
).fetchone()[0]
|
||||
except sqlite3.OperationalError as error:
|
||||
if not _interrupted(error):
|
||||
raise
|
||||
return MemoryLibraryPage(
|
||||
items=[],
|
||||
total=0,
|
||||
offset=request.offset,
|
||||
limit=request.limit,
|
||||
stale=True,
|
||||
)
|
||||
predicate += " AND " + register_memory_freshness(db)
|
||||
try:
|
||||
if request.query:
|
||||
current = db.execute(
|
||||
f"""SELECT m.* FROM governed_memories m {_origin_join()}
|
||||
WHERE {predicate}
|
||||
ORDER BY {_order_by()}""",
|
||||
values,
|
||||
).fetchall()
|
||||
rows = current[request.offset : request.offset + request.limit]
|
||||
else:
|
||||
rows = db.execute(
|
||||
f"""SELECT m.* FROM governed_memories m {_origin_join()}
|
||||
WHERE {predicate}
|
||||
ORDER BY {_order_by()} LIMIT ? OFFSET ?""",
|
||||
[*values, request.limit, request.offset],
|
||||
).fetchall()
|
||||
except sqlite3.OperationalError as error:
|
||||
if not _interrupted(error):
|
||||
raise
|
||||
# The count already succeeded: report it, flag the page stale.
|
||||
return MemoryLibraryPage(
|
||||
items=[],
|
||||
total=total,
|
||||
offset=request.offset,
|
||||
limit=request.limit,
|
||||
stale=True,
|
||||
)
|
||||
# Static read helper only: constructing the Store would initialize schema.
|
||||
items = MemoryGovernanceStore.memories_from_rows(db, rows, lifecycle=lifecycle)
|
||||
# Only the authorized page participates; source filters select memories,
|
||||
# not a subset of their provenance. Keep this in the count/page snapshot.
|
||||
source_ids_by_memory: dict[str, list[str]] = {item.memory_id: [] for item in items}
|
||||
# D96: the list shows when the cited conversation happened, not when
|
||||
# the memory was generated (which is "today" for a fresh import). Read
|
||||
# in the same query as the sources, so the page costs no extra round
|
||||
# trip and only this page's IDs are looked at, never the whole library.
|
||||
conversation_at: dict[str, str] = {}
|
||||
imported_at: dict[str, str] = {}
|
||||
if items:
|
||||
try:
|
||||
for link in db.execute(
|
||||
# D104: the conversation time is when the person spoke, not
|
||||
# when the turn ended. A memory's evidence covers the whole
|
||||
# round (question, tool calls, reply), so MAX() lands on the
|
||||
# reply and reads as a later, contradictory time next to the
|
||||
# cited source. Prefer the earliest person-side evidence; only
|
||||
# fall back to the earliest of any kind when the round has no
|
||||
# person-authored row at all.
|
||||
"""SELECT link.memory_id, e.source_id,
|
||||
COALESCE(
|
||||
MIN(
|
||||
CASE WHEN e.role = 'user'
|
||||
AND e.role = 'user'
|
||||
THEN e.created_at END
|
||||
),
|
||||
MIN(e.created_at)
|
||||
) AS conversation_at,
|
||||
MAX(e.imported_at) AS imported_at
|
||||
FROM governed_memory_evidence link
|
||||
JOIN evidence e ON e.evidence_id = link.evidence_id
|
||||
WHERE link.active = 1 AND e.user_id = ?
|
||||
AND link.memory_id IN (SELECT value FROM json_each(?))
|
||||
GROUP BY link.memory_id, e.source_id
|
||||
ORDER BY link.memory_id, e.source_id""",
|
||||
(request.user_id, json.dumps(list(source_ids_by_memory))),
|
||||
):
|
||||
memory_id = link["memory_id"]
|
||||
source_ids_by_memory[memory_id].append(link["source_id"])
|
||||
for seen, value in (
|
||||
(conversation_at, link["conversation_at"]),
|
||||
(imported_at, link["imported_at"]),
|
||||
):
|
||||
if value and str(value) > seen.get(memory_id, ""):
|
||||
seen[memory_id] = str(value)
|
||||
except sqlite3.OperationalError as error:
|
||||
if not _interrupted(error):
|
||||
raise
|
||||
# Page rows are already read; ship them without source maps
|
||||
# rather than failing the whole request at the last step.
|
||||
return MemoryLibraryPage(
|
||||
items=items,
|
||||
total=total,
|
||||
offset=request.offset,
|
||||
limit=request.limit,
|
||||
source_ids_by_memory=source_ids_by_memory,
|
||||
stale=True,
|
||||
)
|
||||
source_time_by_memory = {
|
||||
memory_id: MemorySourceTime(at=conversation_at[memory_id], basis="conversation")
|
||||
if memory_id in conversation_at
|
||||
else MemorySourceTime(at=imported_at[memory_id], basis="import")
|
||||
for memory_id in source_ids_by_memory
|
||||
if memory_id in conversation_at or memory_id in imported_at
|
||||
}
|
||||
return MemoryLibraryPage(
|
||||
items=items,
|
||||
total=total,
|
||||
offset=request.offset,
|
||||
limit=request.limit,
|
||||
source_ids_by_memory=source_ids_by_memory,
|
||||
source_time_by_memory=source_time_by_memory,
|
||||
)
|
||||
@@ -1,180 +0,0 @@
|
||||
"""Bounded, permission-aware receipt projection; never initialize a write store."""
|
||||
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
from one_memory.database_access import main_database
|
||||
|
||||
#: `legacy` is a confirmation recorded before memories stopped waiting for
|
||||
#: one; it changed nothing that still exists, and `migrated_from` says so.
|
||||
Operation = Literal["create", "correct", "set_scopes", "undo", "delete", "hy_reconcile", "legacy"]
|
||||
ResultStatus = Literal["confirmed", "deleted"]
|
||||
MigratedFrom = Literal["confirm", "reject"]
|
||||
OPERATIONS = {"create", "correct", "set_scopes", "undo", "delete", "hy_reconcile", "legacy"}
|
||||
STATUSES = {"confirmed", "deleted"}
|
||||
#: Which operation each pre-v2 one now reads as.
|
||||
MIGRATED_OPERATIONS = {"confirm": "legacy", "reject": "delete"}
|
||||
#: A memory may move on to these after an earlier receipt was written; the
|
||||
#: receipt still reports what that request committed, not the current status.
|
||||
CURRENT_STATUSES = STATUSES | {"superseded"}
|
||||
|
||||
|
||||
class MemoryMutationReceiptForbidden(RuntimeError):
|
||||
def __init__(self) -> None:
|
||||
super().__init__("memory_mutation_receipt_forbidden")
|
||||
|
||||
|
||||
class MemoryMutationReceiptUnavailable(RuntimeError):
|
||||
def __init__(self) -> None:
|
||||
super().__init__("memory_mutation_receipt_unavailable")
|
||||
|
||||
|
||||
class MemoryMutationReceipt(BaseModel):
|
||||
model_config = ConfigDict(frozen=True, extra="forbid")
|
||||
schema_version: Literal[2] = 2
|
||||
request_id: str
|
||||
state: Literal["not_found", "committed"]
|
||||
operation: Operation | None = None
|
||||
memory_id: str | None = None
|
||||
result_revision: int | None = None
|
||||
result_status: ResultStatus | None = None
|
||||
memory_state: Literal["present", "deleted"] | None = None
|
||||
#: The operation as it was stored before v2, when it has no v2 name of its own.
|
||||
migrated_from: MigratedFrom | None = None
|
||||
|
||||
|
||||
def _identity(value: str, maximum: int) -> bool:
|
||||
return isinstance(value, str) and 0 < len(value) <= maximum
|
||||
|
||||
|
||||
def _read_snapshot(
|
||||
database_path: Path, request_id: str, user_id: str
|
||||
) -> tuple[MemoryMutationReceipt, tuple[object, ...]]:
|
||||
with main_database(database_path).read(row_factory=sqlite3.Row) as db:
|
||||
db.execute("PRAGMA query_only=ON")
|
||||
db.execute("BEGIN")
|
||||
rows = db.execute(
|
||||
"SELECT user_id,request_id,memory_id,operation,result_revision "
|
||||
"FROM governed_memory_mutations WHERE user_id=? AND request_id=? LIMIT 2",
|
||||
(user_id, request_id),
|
||||
).fetchall()
|
||||
if not rows:
|
||||
return MemoryMutationReceipt(request_id=request_id, state="not_found"), ()
|
||||
if len(rows) != 1:
|
||||
raise MemoryMutationReceiptUnavailable()
|
||||
mutation = rows[0]
|
||||
memory_id, operation, revision = (
|
||||
mutation["memory_id"],
|
||||
mutation["operation"],
|
||||
mutation["result_revision"],
|
||||
)
|
||||
if (
|
||||
not _identity(memory_id, 512)
|
||||
or operation not in OPERATIONS
|
||||
or type(revision) is not int
|
||||
or not 1 <= revision <= 2**53 - 1
|
||||
):
|
||||
raise MemoryMutationReceiptUnavailable()
|
||||
current = db.execute(
|
||||
"SELECT user_id,revision,status,deleted_at FROM governed_memories WHERE memory_id=?",
|
||||
(memory_id,),
|
||||
).fetchone()
|
||||
if (
|
||||
current is None
|
||||
or current["user_id"] != user_id
|
||||
or type(current["revision"]) is not int
|
||||
or current["revision"] < revision
|
||||
or current["status"] not in CURRENT_STATUSES
|
||||
or (current["status"] == "deleted") != (current["deleted_at"] is not None)
|
||||
):
|
||||
raise MemoryMutationReceiptUnavailable()
|
||||
|
||||
# Versions have no historical evidence snapshot. Check every retained link,
|
||||
# not only active ones, before disclosing even operation/ID metadata.
|
||||
links = db.execute(
|
||||
"SELECT l.evidence_id,l.active,e.user_id,e.source_id,e.revision,"
|
||||
"g.granted_at,g.revoked_at FROM governed_memory_evidence l "
|
||||
"LEFT JOIN evidence e ON e.evidence_id=l.evidence_id "
|
||||
"LEFT JOIN source_grants g ON g.user_id=e.user_id AND g.source_id=e.source_id "
|
||||
"WHERE l.memory_id=? ORDER BY l.evidence_id LIMIT 10001",
|
||||
(memory_id,),
|
||||
).fetchall()
|
||||
if not links or len(links) > 10000:
|
||||
raise MemoryMutationReceiptUnavailable()
|
||||
for link in links:
|
||||
if (
|
||||
link["user_id"] != user_id
|
||||
or link["source_id"] is None
|
||||
or link["active"] not in (0, 1)
|
||||
):
|
||||
raise MemoryMutationReceiptUnavailable()
|
||||
if link["revoked_at"] is not None:
|
||||
raise MemoryMutationReceiptForbidden()
|
||||
versions = db.execute(
|
||||
"SELECT memory_id,revision,action,status,migrated_from FROM governed_memory_versions "
|
||||
"WHERE memory_id=? AND revision=? LIMIT 2",
|
||||
(memory_id, revision),
|
||||
).fetchall()
|
||||
if len(versions) != 1:
|
||||
raise MemoryMutationReceiptUnavailable()
|
||||
version = versions[0]
|
||||
if version["action"] != operation or version["status"] not in STATUSES:
|
||||
raise MemoryMutationReceiptUnavailable()
|
||||
migrated_from = version["migrated_from"]
|
||||
if migrated_from is not None and MIGRATED_OPERATIONS.get(migrated_from) != operation:
|
||||
raise MemoryMutationReceiptUnavailable()
|
||||
if (operation == "legacy") != (migrated_from == "confirm"):
|
||||
raise MemoryMutationReceiptUnavailable()
|
||||
required_status = {
|
||||
"create": "confirmed",
|
||||
"legacy": "confirmed",
|
||||
"delete": "deleted",
|
||||
}
|
||||
if operation in required_status and version["status"] != required_status[operation]:
|
||||
raise MemoryMutationReceiptUnavailable()
|
||||
if (operation == "create") != (revision == 1):
|
||||
raise MemoryMutationReceiptUnavailable()
|
||||
if (
|
||||
operation in {"correct", "set_scopes", "hy_reconcile", "undo"}
|
||||
and version["status"] != "confirmed"
|
||||
):
|
||||
raise MemoryMutationReceiptUnavailable()
|
||||
receipt = MemoryMutationReceipt(
|
||||
request_id=request_id,
|
||||
state="committed",
|
||||
operation=operation,
|
||||
memory_id=memory_id,
|
||||
result_revision=revision,
|
||||
result_status=version["status"],
|
||||
memory_state="deleted" if current["status"] == "deleted" else "present",
|
||||
migrated_from=migrated_from,
|
||||
)
|
||||
return receipt, (
|
||||
tuple(mutation),
|
||||
tuple(current),
|
||||
tuple(version),
|
||||
tuple(tuple(link) for link in links),
|
||||
)
|
||||
|
||||
|
||||
def read_memory_mutation_receipt(
|
||||
database_path: Path, *, request_id: str, user_id: str = "local"
|
||||
) -> MemoryMutationReceipt:
|
||||
"""History only, never current authorization, HY cleanup or safe-retry proof."""
|
||||
if not _identity(request_id, 256):
|
||||
raise ValueError("invalid_memory_mutation_request_id")
|
||||
if not _identity(user_id, 256):
|
||||
raise ValueError("invalid_memory_mutation_user_id")
|
||||
try:
|
||||
first = _read_snapshot(database_path, request_id, user_id)
|
||||
current = _read_snapshot(database_path, request_id, user_id)
|
||||
if first != current:
|
||||
raise MemoryMutationReceiptUnavailable()
|
||||
return current[0]
|
||||
except MemoryMutationReceiptForbidden:
|
||||
raise
|
||||
except (sqlite3.Error, ValueError, TypeError, KeyError, IndexError, OSError):
|
||||
raise MemoryMutationReceiptUnavailable() from None
|
||||
@@ -24,10 +24,11 @@ from collections.abc import Awaitable, Callable
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from typing import Any, Literal
|
||||
from uuid import uuid4
|
||||
|
||||
from one_memory.background_work import in_the_background
|
||||
from one_memory.conversation_processing import TURN_IDLE_FLUSH_SECONDS, turn_batches
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.diagnostic_log import diagnostic_event
|
||||
from one_memory.imports.models import EvidenceOrigin, EvidenceRecord
|
||||
@@ -316,6 +317,12 @@ LEGACY_MEMORY_WRITE_JOB_COLUMNS = (
|
||||
|
||||
_FIFO_BLOCKING_STATES = (PENDING, RUNNING, WAITING_FOR_PERMISSION, WAITING_FOR_MODEL)
|
||||
EVENT_TIME_WINDOW = timedelta(minutes=10)
|
||||
#: Sources whose jobs are a person adding a memory by hand ("添加记忆"), not
|
||||
#: history. History is digested strictly old to new through the event-time
|
||||
#: window; a note is stamped "now", so inside that window it waited behind the
|
||||
#: whole unprocessed history. These jobs are claimed ahead of the window,
|
||||
#: still one at a time per source and in order within their session.
|
||||
DIRECT_WRITE_SOURCES = frozenset({"one-memory-manual"})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -888,16 +895,30 @@ class DurableWriteQueue:
|
||||
priority: str,
|
||||
connection: sqlite3.Connection,
|
||||
) -> None:
|
||||
"""Append fresh-install input to an assembling queue row.
|
||||
"""Append input to its conversation's assembling queue row.
|
||||
|
||||
The row is not claimable until the scanner commits EOF and finalizes
|
||||
it. No Evidence table is involved.
|
||||
The row is not claimable. `finalize_sdk_file` turns what is complete
|
||||
into write jobs (`turn_batches`) and keeps the rest here for the next
|
||||
increment; `flush_idle_assemblies` writes a live row's rest once the
|
||||
conversation has been quiet. `updated_at` moves only when a record is
|
||||
new or revised, so a re-read does not keep a quiet conversation open.
|
||||
No Evidence table is involved.
|
||||
"""
|
||||
assembly_key = (
|
||||
f"assembly:{record.user_id}:{record.source_id}:"
|
||||
f"{hashlib.sha256(file_key.encode()).hexdigest()}:"
|
||||
f"{hashlib.sha256(record.conversation_id.encode()).hexdigest()}"
|
||||
)
|
||||
session_key = internal_session_id_for(record.conversation_id, record.source_id)
|
||||
written = connection.execute(
|
||||
"SELECT 1 FROM memory_write_jobs AS job, json_each(job.evidence_ids_json) AS ref "
|
||||
"WHERE job.session_key=? AND job.state!='assembling' AND ref.value=? LIMIT 1",
|
||||
(session_key, record.evidence_id),
|
||||
).fetchone()
|
||||
if written is not None:
|
||||
# A re-read of a record a job already holds: staging it again
|
||||
# would regroup it with later turns into a second job.
|
||||
return
|
||||
row = connection.execute(
|
||||
"SELECT job_id,payload_json,priority FROM memory_write_jobs "
|
||||
"WHERE idempotency_key=? AND state='assembling'",
|
||||
@@ -908,8 +929,20 @@ class DurableWriteQueue:
|
||||
payload = json.loads(str(row["payload_json"]))
|
||||
records = list(payload.get("input_records") or [])
|
||||
dumped = record.model_dump(mode="json")
|
||||
if not any(item.get("evidence_id") == record.evidence_id for item in records):
|
||||
index = next(
|
||||
(
|
||||
position
|
||||
for position, item in enumerate(records)
|
||||
if item.get("evidence_id") == record.evidence_id
|
||||
),
|
||||
None,
|
||||
)
|
||||
changed = index is None or records[index] != dumped
|
||||
if index is None:
|
||||
records.append(dumped)
|
||||
elif changed:
|
||||
# A revision replaces the carried copy in place.
|
||||
records[index] = dumped
|
||||
now = datetime.now(UTC).isoformat()
|
||||
payload_json = json.dumps(
|
||||
{
|
||||
@@ -920,15 +953,13 @@ class DurableWriteQueue:
|
||||
sort_keys=True,
|
||||
)
|
||||
if row is not None:
|
||||
merged_priority = "live" if "live" in (row["priority"], priority) else "normal"
|
||||
if not changed and merged_priority == row["priority"]:
|
||||
return
|
||||
connection.execute(
|
||||
"UPDATE memory_write_jobs SET payload_json=?,priority=?,updated_at=? "
|
||||
"WHERE job_id=?",
|
||||
(
|
||||
payload_json,
|
||||
"live" if row["priority"] == "live" or priority == "live" else "normal",
|
||||
now,
|
||||
str(row["job_id"]),
|
||||
),
|
||||
"UPDATE memory_write_jobs SET payload_json=?,priority=?,"
|
||||
"updated_at=CASE WHEN ? THEN ? ELSE updated_at END WHERE job_id=?",
|
||||
(payload_json, merged_priority, int(changed), now, str(row["job_id"])),
|
||||
)
|
||||
return
|
||||
queue_seq = int(
|
||||
@@ -946,7 +977,7 @@ class DurableWriteQueue:
|
||||
(
|
||||
str(uuid4()),
|
||||
queue_seq,
|
||||
internal_session_id_for(record.conversation_id, record.source_id),
|
||||
session_key,
|
||||
record.source_id,
|
||||
assembly_key,
|
||||
assembly_key,
|
||||
@@ -976,92 +1007,215 @@ class DurableWriteQueue:
|
||||
source_id: str,
|
||||
file_key: str,
|
||||
connection: sqlite3.Connection,
|
||||
flush: bool = True,
|
||||
) -> tuple[str, ...]:
|
||||
"""Write what one file's assembling rows hold that the turn rule completes.
|
||||
|
||||
`flush` (history at end of file) completes the last turn and writes a
|
||||
short batch; without it (live polling) those stay in the row.
|
||||
"""
|
||||
rows = connection.execute(
|
||||
"SELECT * FROM memory_write_jobs WHERE state='assembling' "
|
||||
"AND source_key=? ORDER BY queue_seq",
|
||||
(source_id,),
|
||||
).fetchall()
|
||||
finalized = []
|
||||
finalized: list[str] = []
|
||||
for row in rows:
|
||||
payload = json.loads(str(row["payload_json"]))
|
||||
if payload.get("file_key") != file_key:
|
||||
continue
|
||||
records = tuple(
|
||||
EvidenceRecord.model_validate(item)
|
||||
for item in payload.get("input_records") or []
|
||||
if item.get("user_id") == user_id
|
||||
finalized.extend(
|
||||
self._write_assembled_turns(connection, row, user_id=user_id, flush=flush)
|
||||
)
|
||||
if not records:
|
||||
connection.execute(
|
||||
"DELETE FROM memory_write_jobs WHERE job_id=?",
|
||||
(str(row["job_id"]),),
|
||||
return tuple(finalized)
|
||||
|
||||
def assemble_live_records(
|
||||
self,
|
||||
records: tuple[EvidenceRecord, ...],
|
||||
*,
|
||||
file_key: str,
|
||||
) -> tuple[str, ...]:
|
||||
"""Stage live records and write what is complete, in one transaction.
|
||||
|
||||
For live capture outside the import runner (host hooks): the same
|
||||
assembling rows, turn rule and idle flush as live file polling.
|
||||
"""
|
||||
if not records:
|
||||
return ()
|
||||
first = records[0]
|
||||
with (
|
||||
main_database(self.database_path).write(row_factory=sqlite3.Row) as connection,
|
||||
connection,
|
||||
):
|
||||
for record in records:
|
||||
self.stage_sdk_input(
|
||||
record, file_key=file_key, priority="live", connection=connection
|
||||
)
|
||||
continue
|
||||
first = records[0]
|
||||
session_id = internal_session_id_for(
|
||||
first.conversation_id,
|
||||
first.source_id,
|
||||
)
|
||||
material = ":".join(f"{item.evidence_id}:{item.content_hash}" for item in records)
|
||||
request_id = f"turn:v2:{hashlib.sha256(material.encode()).hexdigest()}"
|
||||
existing = connection.execute(
|
||||
"SELECT job_id FROM memory_write_jobs WHERE idempotency_key=? AND job_id!=?",
|
||||
(request_id, str(row["job_id"])),
|
||||
).fetchone()
|
||||
if existing is not None:
|
||||
connection.execute(
|
||||
"DELETE FROM memory_write_jobs WHERE job_id=?",
|
||||
(str(row["job_id"]),),
|
||||
)
|
||||
finalized.append(str(existing["job_id"]))
|
||||
continue
|
||||
request = WriteRequest(
|
||||
internal_session_id=session_id,
|
||||
return self.finalize_sdk_file(
|
||||
user_id=first.user_id,
|
||||
source_id=first.source_id,
|
||||
batch_identity=BatchIdentity(
|
||||
internal_session_id=session_id,
|
||||
first_round=1,
|
||||
last_round=1,
|
||||
),
|
||||
evidence_ids=[item.evidence_id for item in records],
|
||||
input_records=records,
|
||||
idempotency_key=request_id,
|
||||
priority="live" if str(row["priority"]) == "live" else "normal",
|
||||
event_at=max(
|
||||
(item.created_at for item in records if item.created_at),
|
||||
default=first.imported_at,
|
||||
),
|
||||
time_quality="exact" if any(item.created_at for item in records) else "approximate",
|
||||
file_key=file_key,
|
||||
connection=connection,
|
||||
flush=False,
|
||||
)
|
||||
|
||||
def flush_idle_assemblies(self, *, now: datetime | None = None) -> tuple[str, ...]:
|
||||
"""Write the rest of every live conversation quiet for 10 minutes.
|
||||
|
||||
Its last turn counts as complete and a batch short of 5 turns is
|
||||
written. History rows are left alone: their end of file flushes them.
|
||||
"""
|
||||
current = now or datetime.now(UTC)
|
||||
cutoff = (current - timedelta(seconds=TURN_IDLE_FLUSH_SECONDS)).isoformat()
|
||||
query = (
|
||||
"SELECT * FROM memory_write_jobs WHERE state='assembling' AND priority='live' "
|
||||
"AND updated_at<=? ORDER BY queue_seq"
|
||||
)
|
||||
# A read first: most ticks find nothing and must not take the writer.
|
||||
with main_database(self.database_path).read(row_factory=sqlite3.Row) as connection:
|
||||
if connection.execute(query, (cutoff,)).fetchone() is None:
|
||||
return ()
|
||||
finalized: list[str] = []
|
||||
with (
|
||||
main_database(self.database_path).write(row_factory=sqlite3.Row) as connection,
|
||||
connection,
|
||||
):
|
||||
for row in connection.execute(query, (cutoff,)).fetchall():
|
||||
finalized.extend(self._write_assembled_turns(connection, row, flush=True))
|
||||
return tuple(finalized)
|
||||
|
||||
def _write_assembled_turns(
|
||||
self,
|
||||
connection: sqlite3.Connection,
|
||||
row: sqlite3.Row,
|
||||
*,
|
||||
flush: bool,
|
||||
user_id: str | None = None,
|
||||
) -> list[str]:
|
||||
"""Turn one assembling row's complete batches into jobs; keep the carry."""
|
||||
payload = json.loads(str(row["payload_json"]))
|
||||
records = tuple(
|
||||
EvidenceRecord.model_validate(item)
|
||||
for item in payload.get("input_records") or []
|
||||
if user_id is None or item.get("user_id") == user_id
|
||||
)
|
||||
grouped = turn_batches(records, flush=flush)
|
||||
priority: Literal["live", "normal"] = "live" if str(row["priority"]) == "live" else "normal"
|
||||
job_ids = [
|
||||
self._insert_turn_batch(connection, batch, priority=priority)
|
||||
for batch in grouped.batches
|
||||
]
|
||||
if not grouped.carry:
|
||||
connection.execute(
|
||||
"UPDATE memory_write_jobs SET session_key=?,batch_id=?,"
|
||||
"idempotency_key=?,payload_json=?,evidence_ids_json=?,state='pending',"
|
||||
"event_at=?,time_quality=?,updated_at=? WHERE job_id=?",
|
||||
"DELETE FROM memory_write_jobs WHERE job_id=?", (str(row["job_id"]),)
|
||||
)
|
||||
elif grouped.batches or len(grouped.carry) != len(records):
|
||||
# `updated_at` stays: the idle clock counts new messages only.
|
||||
connection.execute(
|
||||
"UPDATE memory_write_jobs SET payload_json=? WHERE job_id=?",
|
||||
(
|
||||
session_id,
|
||||
request.batch_identity.batch_id,
|
||||
request_id,
|
||||
json.dumps(
|
||||
request.model_dump(
|
||||
mode="json",
|
||||
exclude={"event_at", "time_quality"},
|
||||
),
|
||||
{
|
||||
**payload,
|
||||
"input_records": [
|
||||
record.model_dump(mode="json") for record in grouped.carry
|
||||
],
|
||||
},
|
||||
sort_keys=True,
|
||||
),
|
||||
json.dumps(list(request.evidence_ids)),
|
||||
(
|
||||
request.event_at.isoformat()
|
||||
if request.event_at
|
||||
else datetime.now(UTC).isoformat()
|
||||
),
|
||||
request.time_quality,
|
||||
datetime.now(UTC).isoformat(),
|
||||
str(row["job_id"]),
|
||||
),
|
||||
)
|
||||
finalized.append(str(row["job_id"]))
|
||||
return tuple(finalized)
|
||||
return job_ids
|
||||
|
||||
def _insert_turn_batch(
|
||||
self,
|
||||
connection: sqlite3.Connection,
|
||||
records: tuple[EvidenceRecord, ...],
|
||||
*,
|
||||
priority: Literal["live", "normal"],
|
||||
) -> str:
|
||||
"""One pending job for one batch of whole turns, keyed by its content.
|
||||
|
||||
Same records, same key: a re-read returns the job already written.
|
||||
"""
|
||||
first = records[0]
|
||||
session_id = internal_session_id_for(first.conversation_id, first.source_id)
|
||||
material = ":".join(f"{item.evidence_id}:{item.content_hash}" for item in records)
|
||||
request_id = f"turn:v2:{hashlib.sha256(material.encode()).hexdigest()}"
|
||||
existing = connection.execute(
|
||||
"SELECT job_id FROM memory_write_jobs WHERE idempotency_key=?",
|
||||
(request_id,),
|
||||
).fetchone()
|
||||
if existing is not None:
|
||||
return str(existing["job_id"])
|
||||
request = WriteRequest(
|
||||
internal_session_id=session_id,
|
||||
source_id=first.source_id,
|
||||
batch_identity=BatchIdentity(
|
||||
internal_session_id=session_id,
|
||||
first_round=1,
|
||||
last_round=1,
|
||||
),
|
||||
evidence_ids=[item.evidence_id for item in records],
|
||||
input_records=records,
|
||||
idempotency_key=request_id,
|
||||
priority=priority,
|
||||
)
|
||||
# The batch's time is its first user message's, as for every other job.
|
||||
event_at, time_quality = self._event_time_for_request(request)
|
||||
now = datetime.now(UTC)
|
||||
job_id = str(uuid4())
|
||||
queue_seq = int(
|
||||
connection.execute(
|
||||
"SELECT COALESCE(MAX(queue_seq),0)+1 FROM memory_write_jobs"
|
||||
).fetchone()[0]
|
||||
)
|
||||
event = event_at or now
|
||||
if event.tzinfo is None:
|
||||
event = event.replace(tzinfo=UTC)
|
||||
connection.execute(
|
||||
"INSERT INTO memory_write_jobs ("
|
||||
"job_id,queue_seq,session_key,source_key,batch_id,idempotency_key,"
|
||||
"payload_json,evidence_ids_json,state,priority,attempts,max_attempts,"
|
||||
"lease_owner,lease_expires_at,hy_invoked,last_error,event_at,time_quality,"
|
||||
"created_at,updated_at,sdk_raw_ids_json,sdk_memory_ids_json"
|
||||
") VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
|
||||
(
|
||||
job_id,
|
||||
queue_seq,
|
||||
session_id,
|
||||
first.source_id,
|
||||
request.batch_identity.batch_id,
|
||||
request_id,
|
||||
json.dumps(
|
||||
request.model_dump(mode="json", exclude={"event_at", "time_quality"}),
|
||||
sort_keys=True,
|
||||
),
|
||||
json.dumps(list(request.evidence_ids)),
|
||||
PENDING,
|
||||
priority,
|
||||
0,
|
||||
3,
|
||||
None,
|
||||
None,
|
||||
0,
|
||||
None,
|
||||
event.astimezone(UTC).isoformat(),
|
||||
time_quality,
|
||||
now.isoformat(),
|
||||
now.isoformat(),
|
||||
"[]",
|
||||
"[]",
|
||||
),
|
||||
)
|
||||
if not self.sdk_authority:
|
||||
# Legacy catch-up reads coverage from these references.
|
||||
connection.executemany(
|
||||
"INSERT INTO memory_write_job_evidence (job_id, evidence_id) VALUES (?,?)",
|
||||
((job_id, evidence_id) for evidence_id in request.evidence_ids),
|
||||
)
|
||||
return job_id
|
||||
|
||||
def _freeze_request_inputs(self, request: WriteRequest) -> WriteRequest:
|
||||
if request.input_records or self.evidence is None:
|
||||
@@ -2494,6 +2648,9 @@ class DurableWriteDrain:
|
||||
#: a model outage clears on its own, and a queue that never goes idle
|
||||
#: must still re-admit the write once the wait has elapsed.
|
||||
self._last_model_readmit: datetime | None = None
|
||||
#: A live conversation quiet for 10 minutes has its last turn and short
|
||||
#: batch written (`flush_idle_assemblies`); checked on the same timer.
|
||||
self._last_assembly_flush: datetime | None = None
|
||||
#: Idle polls do not need to hit the single writer every 50ms. A
|
||||
#: 1m Evidence table makes a miss expensive enough that the tail
|
||||
#: call-event write can be behind it; back off only while idle and
|
||||
@@ -2601,6 +2758,39 @@ class DurableWriteDrain:
|
||||
placeholders = ",".join("?" for _ in enabled_sources)
|
||||
source_clause = f"AND candidate.source_key IN ({placeholders}) "
|
||||
source_params = tuple(sorted(enabled_sources))
|
||||
direct_sources = ",".join("?" for _ in DIRECT_WRITE_SOURCES)
|
||||
# Same eligibility as the window claim below (free lease, no
|
||||
# running job of its source, nothing earlier in its session), but
|
||||
# not bound to the window: a note must not wait for history.
|
||||
direct_rows = connection.execute(
|
||||
"SELECT candidate.job_id,candidate.source_key "
|
||||
"FROM memory_write_jobs AS candidate "
|
||||
"WHERE candidate.state=? "
|
||||
"AND (candidate.lease_expires_at IS NULL OR candidate.lease_expires_at < ?) "
|
||||
+ source_clause
|
||||
+ f"AND candidate.source_key IN ({direct_sources}) "
|
||||
"AND NOT EXISTS ("
|
||||
"SELECT 1 FROM memory_write_jobs AS held "
|
||||
"WHERE held.source_key=candidate.source_key AND held.state=? "
|
||||
"AND held.lease_owner IS NOT NULL AND held.lease_expires_at>=?) "
|
||||
"AND NOT EXISTS ("
|
||||
"SELECT 1 FROM memory_write_jobs AS earlier "
|
||||
"WHERE earlier.session_key=candidate.session_key "
|
||||
f"AND earlier.state IN ({blocking}) "
|
||||
"AND (earlier.event_at<candidate.event_at OR "
|
||||
"(earlier.event_at=candidate.event_at "
|
||||
"AND earlier.queue_seq<candidate.queue_seq))) "
|
||||
"ORDER BY candidate.queue_seq",
|
||||
(
|
||||
PENDING,
|
||||
now.isoformat(),
|
||||
*source_params,
|
||||
*sorted(DIRECT_WRITE_SOURCES),
|
||||
RUNNING,
|
||||
now.isoformat(),
|
||||
*_FIFO_BLOCKING_STATES,
|
||||
),
|
||||
).fetchall()
|
||||
window = connection.execute(
|
||||
"SELECT window_anchor,window_end FROM memory_write_schedule_state WHERE singleton=1"
|
||||
).fetchone()
|
||||
@@ -2654,6 +2844,7 @@ class DurableWriteDrain:
|
||||
if active is None:
|
||||
connection.execute("DELETE FROM memory_write_schedule_state WHERE singleton=1")
|
||||
window = None
|
||||
anchor: Any = None
|
||||
if window is None:
|
||||
anchor = connection.execute(
|
||||
"SELECT candidate.event_at "
|
||||
@@ -2682,8 +2873,10 @@ class DurableWriteDrain:
|
||||
*_FIFO_BLOCKING_STATES,
|
||||
),
|
||||
).fetchone()
|
||||
if anchor is None:
|
||||
if anchor is None and not direct_rows:
|
||||
return ()
|
||||
has_window = window is not None or anchor is not None
|
||||
if window is None and anchor is not None:
|
||||
anchor_at = datetime.fromisoformat(str(anchor[0]))
|
||||
if anchor_at.tzinfo is None:
|
||||
anchor_at = anchor_at.replace(tzinfo=UTC)
|
||||
@@ -2695,39 +2888,41 @@ class DurableWriteDrain:
|
||||
"VALUES (1,?,?,?)",
|
||||
(anchor_text, end_text, now.isoformat()),
|
||||
)
|
||||
rows = connection.execute(
|
||||
"SELECT candidate.job_id,candidate.source_key "
|
||||
"FROM memory_write_jobs AS candidate "
|
||||
"WHERE candidate.state=? "
|
||||
"AND (candidate.lease_expires_at IS NULL OR candidate.lease_expires_at < ?) "
|
||||
+ source_clause
|
||||
+ "AND candidate.event_at>=? AND candidate.event_at<? "
|
||||
"AND NOT EXISTS ("
|
||||
"SELECT 1 FROM memory_write_jobs AS held "
|
||||
"WHERE held.source_key=candidate.source_key AND held.state=? "
|
||||
"AND held.lease_owner IS NOT NULL AND held.lease_expires_at>=?) "
|
||||
"AND NOT EXISTS ("
|
||||
"SELECT 1 FROM memory_write_jobs AS earlier "
|
||||
"WHERE earlier.session_key=candidate.session_key "
|
||||
f"AND earlier.state IN ({blocking}) "
|
||||
"AND (earlier.event_at<candidate.event_at OR "
|
||||
"(earlier.event_at=candidate.event_at "
|
||||
"AND earlier.queue_seq<candidate.queue_seq))) "
|
||||
"ORDER BY candidate.event_at,candidate.queue_seq",
|
||||
(
|
||||
PENDING,
|
||||
now.isoformat(),
|
||||
*source_params,
|
||||
anchor_text,
|
||||
end_text,
|
||||
RUNNING,
|
||||
now.isoformat(),
|
||||
*_FIFO_BLOCKING_STATES,
|
||||
),
|
||||
).fetchall()
|
||||
rows: list[Any] = []
|
||||
if has_window:
|
||||
rows = connection.execute(
|
||||
"SELECT candidate.job_id,candidate.source_key "
|
||||
"FROM memory_write_jobs AS candidate "
|
||||
"WHERE candidate.state=? "
|
||||
"AND (candidate.lease_expires_at IS NULL OR candidate.lease_expires_at < ?) "
|
||||
+ source_clause
|
||||
+ "AND candidate.event_at>=? AND candidate.event_at<? "
|
||||
"AND NOT EXISTS ("
|
||||
"SELECT 1 FROM memory_write_jobs AS held "
|
||||
"WHERE held.source_key=candidate.source_key AND held.state=? "
|
||||
"AND held.lease_owner IS NOT NULL AND held.lease_expires_at>=?) "
|
||||
"AND NOT EXISTS ("
|
||||
"SELECT 1 FROM memory_write_jobs AS earlier "
|
||||
"WHERE earlier.session_key=candidate.session_key "
|
||||
f"AND earlier.state IN ({blocking}) "
|
||||
"AND (earlier.event_at<candidate.event_at OR "
|
||||
"(earlier.event_at=candidate.event_at "
|
||||
"AND earlier.queue_seq<candidate.queue_seq))) "
|
||||
"ORDER BY candidate.event_at,candidate.queue_seq",
|
||||
(
|
||||
PENDING,
|
||||
now.isoformat(),
|
||||
*source_params,
|
||||
anchor_text,
|
||||
end_text,
|
||||
RUNNING,
|
||||
now.isoformat(),
|
||||
*_FIFO_BLOCKING_STATES,
|
||||
),
|
||||
).fetchall()
|
||||
selected: list[str] = []
|
||||
selected_sources: set[str] = set()
|
||||
for row in rows:
|
||||
for row in (*direct_rows, *rows):
|
||||
source_key = str(row["source_key"])
|
||||
if source_key in selected_sources:
|
||||
continue
|
||||
@@ -2988,6 +3183,15 @@ class DurableWriteDrain:
|
||||
self.queue.readmit_model_waiters,
|
||||
operation_name="readmit_model_waiters",
|
||||
)
|
||||
if (
|
||||
self._last_assembly_flush is None
|
||||
or tick_started - self._last_assembly_flush >= self._reap_interval
|
||||
):
|
||||
self._last_assembly_flush = tick_started
|
||||
await self._run_idle_maintenance(
|
||||
self.queue.flush_idle_assemblies,
|
||||
operation_name="flush_idle_assemblies",
|
||||
)
|
||||
if self.reconcile_unknown is not None and (
|
||||
self._last_unknown_reconcile is None
|
||||
or tick_started - self._last_unknown_reconcile
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""A3-01: the HTTP entry to the unified write path.
|
||||
|
||||
`POST /api/v1/memory-writes` accepts a write and returns a request id;
|
||||
`GET /api/v1/memory-writes/{id}` reports its outcome.
|
||||
`POST /api/v2/memory-writes` accepts OpenAI-style messages and returns a
|
||||
request id; `GET /api/v2/memory-writes/{id}` reports its outcome.
|
||||
|
||||
Two rules shape this endpoint, both consequences of A0-03's measurement that HY
|
||||
has no idempotency key:
|
||||
@@ -45,17 +45,6 @@ from one_memory.memory_server.registry import WriteRegistry, WriteState
|
||||
from one_memory.memory_server.submit import WriteConflictError
|
||||
|
||||
|
||||
class MemoryWriteInput(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
internal_session_id: str = Field(min_length=1, max_length=128)
|
||||
first_round: int = Field(ge=1)
|
||||
last_round: int = Field(ge=1)
|
||||
evidence_ids: list[str] = Field(min_length=1)
|
||||
priority: Literal["live", "normal"] = "normal"
|
||||
idempotency_key: str | None = Field(default=None, max_length=256)
|
||||
|
||||
|
||||
class MemoryWriteReceipt(BaseModel):
|
||||
request_id: str
|
||||
state: WriteState
|
||||
@@ -121,67 +110,6 @@ def _registry(
|
||||
return cast(WriteRegistry | DurableWriteRegistry, registry)
|
||||
|
||||
|
||||
def router(dependencies: list[Any] | None = None) -> APIRouter:
|
||||
instance = APIRouter(
|
||||
prefix="/api/v1/memory-writes",
|
||||
tags=["memory-writes"],
|
||||
dependencies=dependencies or [],
|
||||
)
|
||||
|
||||
@instance.post("", response_model=MemoryWriteReceipt, status_code=202)
|
||||
def submit(payload: MemoryWriteInput, request: Request) -> MemoryWriteReceipt:
|
||||
registry = _registry(request)
|
||||
batch = BatchIdentity(
|
||||
internal_session_id=payload.internal_session_id,
|
||||
first_round=payload.first_round,
|
||||
last_round=payload.last_round,
|
||||
)
|
||||
request_contract = WriteRequest(
|
||||
internal_session_id=payload.internal_session_id,
|
||||
batch_identity=batch,
|
||||
evidence_ids=payload.evidence_ids,
|
||||
idempotency_key=payload.idempotency_key or batch.batch_id,
|
||||
priority=payload.priority,
|
||||
)
|
||||
try:
|
||||
record, enqueued = registry.submit(request_contract)
|
||||
except WriteConflictError as error:
|
||||
raise HTTPException(409, str(error)) from error
|
||||
except LookupError as error:
|
||||
# Evidence-first: the write points at Evidence the business
|
||||
# database does not hold. Rejecting it here is a caller error, not
|
||||
# a server fault, and it must not create a queue row.
|
||||
raise HTTPException(422, str(error)) from error
|
||||
return MemoryWriteReceipt(
|
||||
request_id=record.request_id,
|
||||
state=record.state,
|
||||
created_at=record.created_at,
|
||||
updated_at=record.updated_at,
|
||||
detail="" if enqueued else "already accepted; not enqueued again",
|
||||
)
|
||||
|
||||
@instance.get("/{request_id}", response_model=MemoryWriteReceipt)
|
||||
def status(request_id: str, request: Request) -> MemoryWriteReceipt:
|
||||
registry = _registry(request, write=False)
|
||||
try:
|
||||
record = registry.get(request_id)
|
||||
except DurableQueueSchemaUnavailableError as error:
|
||||
raise HTTPException(503, str(error)) from error
|
||||
if record is None:
|
||||
raise HTTPException(404, "unknown request id")
|
||||
return MemoryWriteReceipt(
|
||||
request_id=record.request_id,
|
||||
state=record.state,
|
||||
created_at=record.created_at,
|
||||
updated_at=record.updated_at,
|
||||
detail=record.detail,
|
||||
raw_ids=record.raw_ids,
|
||||
memory_ids=record.memory_ids,
|
||||
)
|
||||
|
||||
return instance
|
||||
|
||||
|
||||
def sdk_router(dependencies: list[Any] | None = None) -> APIRouter:
|
||||
instance = APIRouter(
|
||||
prefix="/api/v2/memory-writes",
|
||||
|
||||
@@ -98,23 +98,6 @@ class UnifiedWriteIntake:
|
||||
enqueued=durable.enqueued,
|
||||
)
|
||||
|
||||
def readmit_cancelled_evidence(
|
||||
self,
|
||||
record: EvidenceRecord,
|
||||
*,
|
||||
job_id: str,
|
||||
first_round: int,
|
||||
last_round: int,
|
||||
priority: WritePriority = "normal",
|
||||
) -> int:
|
||||
request = self._evidence_request(
|
||||
record,
|
||||
first_round=first_round,
|
||||
last_round=last_round,
|
||||
priority=priority,
|
||||
)
|
||||
return self.queue.readmit_cancelled_permission_confirmed(job_id, request)
|
||||
|
||||
@staticmethod
|
||||
def _evidence_request(
|
||||
record: EvidenceRecord,
|
||||
|
||||
@@ -1136,20 +1136,31 @@ class ModelProfileStore:
|
||||
)
|
||||
|
||||
|
||||
def disable_thinking_params(
|
||||
profile: ModelProfile | ModelProfileSnapshot,
|
||||
) -> dict[str, object]:
|
||||
"""Provider-specific thinking-off params, empty unless the profile opts in.
|
||||
|
||||
The one mapping for every request Core builds itself and for the SDK's
|
||||
`llm.extra_body`, so a model that thinks on one path thinks on none.
|
||||
"""
|
||||
if not profile.disable_thinking:
|
||||
return {}
|
||||
if profile.provider_kind is ProviderKind.OPENAI_COMPATIBLE:
|
||||
if urlparse(profile.base_url or "").hostname == "api.deepseek.com":
|
||||
return {"thinking": {"type": "disabled"}}
|
||||
return {"reasoning_effort": "none"}
|
||||
if profile.provider_kind is ProviderKind.OLLAMA:
|
||||
return {"think": False}
|
||||
return {}
|
||||
|
||||
|
||||
def attach_disable_thinking(
|
||||
profile: ModelProfile | ModelProfileSnapshot,
|
||||
payload: dict[str, object],
|
||||
) -> dict[str, object]:
|
||||
"""Attach provider-specific thinking-off params when the profile opts in."""
|
||||
if not profile.disable_thinking:
|
||||
return payload
|
||||
if profile.provider_kind is ProviderKind.OPENAI_COMPATIBLE:
|
||||
if urlparse(profile.base_url or "").hostname == "api.deepseek.com":
|
||||
payload["thinking"] = {"type": "disabled"}
|
||||
else:
|
||||
payload["reasoning_effort"] = "none"
|
||||
elif profile.provider_kind is ProviderKind.OLLAMA:
|
||||
payload["think"] = False
|
||||
payload.update(disable_thinking_params(profile))
|
||||
return payload
|
||||
|
||||
|
||||
|
||||
@@ -1,623 +0,0 @@
|
||||
"""Read-only process: serves the same read routes as Core without write access.
|
||||
|
||||
Opens the database in `mode=ro` with `PRAGMA query_only=ON`; every store
|
||||
operation here goes through `main_database().read()`, which already enforces
|
||||
this. No HY SDK, no Chroma, no ONNX runtime, no keychain: the imports that
|
||||
would pull them in are simply not reached, and the routes below never call
|
||||
into a write path.
|
||||
|
||||
Reads that need HY (semantic search, graph overlay) are delegated to the
|
||||
Core's internal endpoints; the Core is the only process that owns the
|
||||
vector stores. Side effects (call events, feedback) are queued and sent
|
||||
asynchronously, never blocking the read response.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import builtins
|
||||
import contextlib
|
||||
import logging
|
||||
import os
|
||||
import sqlite3
|
||||
import time
|
||||
from collections.abc import AsyncIterator, Collection, Mapping, Sequence
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, cast
|
||||
|
||||
from fastapi import Depends, FastAPI, HTTPException, Query, Request
|
||||
from pydantic import BaseModel, ConfigDict, SecretStr
|
||||
|
||||
from one_memory.database_access import main_database
|
||||
from one_memory.home_summary import (
|
||||
HomeSummary,
|
||||
HomeSummaryUnavailable,
|
||||
read_home_summary,
|
||||
)
|
||||
from one_memory.memory_governance import GovernedMemory, MemoryStatus
|
||||
from one_memory.memory_library import MemoryLibraryPage, read_memory_library
|
||||
from one_memory.recall import RecallRequest, _KeywordCandidates
|
||||
from one_memory.semantic_deadline import SemanticMemoryHit
|
||||
from one_memory.source_read_access import SourceReadAccess
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
MAX_READER_RSS_BYTES = 250 * 1024 * 1024
|
||||
|
||||
_PROBE_QUERY = "__om_probe__"
|
||||
|
||||
|
||||
def _check_imports() -> None:
|
||||
"""Fail fast if any forbidden module is already loaded."""
|
||||
forbidden = ("hy_memory", "chromadb", "onnxruntime", "keyring")
|
||||
import sys
|
||||
|
||||
loaded = [name for name in forbidden if name in sys.modules]
|
||||
if loaded:
|
||||
raise ImportError(f"reader_process_loaded_forbidden_modules: {', '.join(loaded)}")
|
||||
|
||||
|
||||
def _memory_readable_sql() -> str:
|
||||
return """
|
||||
m.user_id = ? AND m.status = 'confirmed' AND m.deleted_at IS NULL
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM governed_memory_evidence link
|
||||
JOIN evidence e ON e.evidence_id = link.evidence_id
|
||||
JOIN source_grants g ON g.user_id=e.user_id AND g.source_id=e.source_id
|
||||
WHERE link.memory_id=m.memory_id AND +link.active=1
|
||||
AND g.revoked_at IS NOT NULL
|
||||
)
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM governed_memory_evidence link
|
||||
LEFT JOIN evidence e ON e.evidence_id = link.evidence_id
|
||||
WHERE link.memory_id=m.memory_id AND +link.active=1
|
||||
AND (e.evidence_id IS NULL OR e.user_id != m.user_id)
|
||||
)
|
||||
"""
|
||||
|
||||
|
||||
class ReaderSettings(BaseModel):
|
||||
model_config = ConfigDict(frozen=True)
|
||||
|
||||
data_dir: Path
|
||||
host: str = "127.0.0.1"
|
||||
port: int = 0 # 0 = ephemeral
|
||||
api_token: SecretStr
|
||||
core_port: int
|
||||
core_api_token: str
|
||||
|
||||
|
||||
class SideEffectForwarder:
|
||||
"""Queues read-path side effects and forwards them to Core.
|
||||
|
||||
Reads in the Reader must stay read-only: the database is opened with
|
||||
`query_only`, so a call event cannot be written here. Instead each read
|
||||
that would have recorded one enqueues it, and a background task posts it
|
||||
to Core's internal endpoint. The response never waits for it; a Core that
|
||||
is unavailable drops it and counts the drop.
|
||||
"""
|
||||
|
||||
def __init__(self, settings: ReaderSettings) -> None:
|
||||
self.settings = settings
|
||||
self.queue: asyncio.Queue[dict[str, object]] = asyncio.Queue(maxsize=256)
|
||||
self.dropped = 0
|
||||
self.task: asyncio.Task[None] | None = None
|
||||
|
||||
def enqueue(self, event: dict[str, object]) -> None:
|
||||
try:
|
||||
self.queue.put_nowait(event)
|
||||
except asyncio.QueueFull:
|
||||
self.dropped += 1
|
||||
|
||||
def start(self) -> None:
|
||||
if self.task is None or self.task.done():
|
||||
self.task = asyncio.ensure_future(self._drain())
|
||||
|
||||
def stop(self) -> None:
|
||||
if self.task is not None and not self.task.done():
|
||||
self.task.cancel()
|
||||
|
||||
async def _drain(self) -> None:
|
||||
import httpx
|
||||
|
||||
while True:
|
||||
event = await self.queue.get()
|
||||
try:
|
||||
async with httpx.AsyncClient(
|
||||
headers={
|
||||
"Authorization": f"Bearer {self.settings.core_api_token}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
timeout=5.0,
|
||||
proxy=None,
|
||||
) as client:
|
||||
await client.post(
|
||||
f"http://127.0.0.1:{self.settings.core_port}/internal/call-event",
|
||||
json=event,
|
||||
)
|
||||
except Exception:
|
||||
# A dropped side effect is never a failed read: recall already
|
||||
# answered. Counted so a persistently unavailable Core is
|
||||
# visible rather than silent.
|
||||
self.dropped += 1
|
||||
|
||||
|
||||
async def _core_get(
|
||||
settings: ReaderSettings,
|
||||
path: str,
|
||||
timeout_s: float = 5.0,
|
||||
params: Mapping[
|
||||
str, str | int | float | bool | None | Sequence[str | int | float | bool | None]
|
||||
]
|
||||
| None = None,
|
||||
) -> Any:
|
||||
"""Call the Core's internal read endpoint for data only Core can serve.
|
||||
|
||||
`params` is passed to httpx rather than interpolated into the path: a
|
||||
query containing `&`, `#` or a space would otherwise be cut short or
|
||||
read as a fragment, silently searching for the wrong thing.
|
||||
"""
|
||||
import httpx
|
||||
|
||||
url = f"http://127.0.0.1:{settings.core_port}{path}"
|
||||
async with httpx.AsyncClient(
|
||||
headers={"Authorization": f"Bearer {settings.core_api_token}"},
|
||||
timeout=timeout_s,
|
||||
proxy=None,
|
||||
) as client:
|
||||
response = await client.get(url, params=params)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
|
||||
class _ReaderGovernance:
|
||||
"""Minimal read interface over the governance store for recall."""
|
||||
|
||||
def __init__(self, database_path: Path) -> None:
|
||||
self.database_path = database_path
|
||||
|
||||
def list(
|
||||
self,
|
||||
*,
|
||||
user_id: str,
|
||||
status: MemoryStatus | None = None,
|
||||
query: str | None = None,
|
||||
limit: int = 100,
|
||||
offset: int = 0,
|
||||
) -> list[GovernedMemory]:
|
||||
"""One page of memories, read through the Core's own read function.
|
||||
|
||||
`MemoryGovernanceStore.list` cannot be used directly: constructing the
|
||||
store runs schema DDL on the write connection, and the Reader opens
|
||||
with `query_only`. `read_memory_library` is the same read expressed as
|
||||
a static SQL read (revoked sources excluded, per-row freshness checked
|
||||
for the rows it returns, cheap total for the rest), so the Reader and
|
||||
the Core cannot drift on what "readable" means.
|
||||
"""
|
||||
page = read_memory_library(
|
||||
self.database_path,
|
||||
user_id=user_id,
|
||||
query=query,
|
||||
status=status,
|
||||
offset=offset,
|
||||
limit=limit,
|
||||
)
|
||||
return list(page.items)
|
||||
|
||||
def find_confirmed_matching_terms(
|
||||
self,
|
||||
*,
|
||||
user_id: str,
|
||||
terms: Collection[str],
|
||||
limit: int = 200,
|
||||
scopes: Collection[str] | None = None,
|
||||
) -> tuple[builtins.list[GovernedMemory], bool]:
|
||||
if not 1 <= limit <= 2_000:
|
||||
raise ValueError("invalid candidate limit")
|
||||
cleaned = tuple(dict.fromkeys(term.strip() for term in terms if term.strip()))
|
||||
if not cleaned or scopes is not None and not scopes:
|
||||
return [], False
|
||||
clauses = ["m.user_id=?", "m.status='confirmed'", "m.deleted_at IS NULL"]
|
||||
values: builtins.list[object] = [user_id]
|
||||
matches = []
|
||||
for term in cleaned[:32]:
|
||||
matches.append("m.content LIKE ? ESCAPE '\\'")
|
||||
escaped = term.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_")
|
||||
values.append(f"%{escaped}%")
|
||||
clauses.append(f"({' OR '.join(matches)})")
|
||||
if scopes is not None:
|
||||
clauses.append(
|
||||
"EXISTS (SELECT 1 FROM json_each(m.scopes_json) "
|
||||
f"WHERE json_each.value IN ({','.join('?' for _ in scopes)}))"
|
||||
)
|
||||
values.extend(scopes)
|
||||
values.append(limit + 1)
|
||||
with main_database(self.database_path).read(row_factory=sqlite3.Row) as db:
|
||||
rows = db.execute(
|
||||
f"SELECT m.* FROM governed_memories m WHERE {' AND '.join(clauses)} "
|
||||
"ORDER BY m.updated_at DESC,m.memory_id LIMIT ?",
|
||||
values,
|
||||
).fetchall()
|
||||
items = self._memories_from_rows(db, rows[:limit])
|
||||
return items, len(rows) > limit or len(cleaned) > 32
|
||||
|
||||
def get(self, memory_id: str, *, user_id: str) -> GovernedMemory | None:
|
||||
memory = _get_memory_direct(self.database_path, memory_id, user_id)
|
||||
return memory if isinstance(memory, GovernedMemory) else None
|
||||
|
||||
def sources_behind_memories(
|
||||
self,
|
||||
*,
|
||||
user_id: str,
|
||||
memory_ids: Collection[str],
|
||||
) -> dict[str, frozenset[str]]:
|
||||
if not memory_ids:
|
||||
return {}
|
||||
identifiers = tuple(memory_ids)
|
||||
placeholders = ",".join("?" for _ in identifiers)
|
||||
with main_database(self.database_path).read(row_factory=sqlite3.Row) as db:
|
||||
rows = db.execute(
|
||||
"SELECT link.memory_id,e.source_id FROM governed_memory_evidence link "
|
||||
"JOIN evidence e ON e.evidence_id=link.evidence_id "
|
||||
f"WHERE link.active=1 AND e.user_id=? AND link.memory_id IN ({placeholders})",
|
||||
(user_id, *identifiers),
|
||||
).fetchall()
|
||||
found: dict[str, set[str]] = {}
|
||||
for row in rows:
|
||||
found.setdefault(str(row["memory_id"]), set()).add(str(row["source_id"]))
|
||||
return {key: frozenset(values) for key, values in found.items()}
|
||||
|
||||
@staticmethod
|
||||
def _memories_from_rows(
|
||||
db: sqlite3.Connection, rows: Sequence[sqlite3.Row]
|
||||
) -> builtins.list[GovernedMemory]:
|
||||
from one_memory.memory_governance import MemoryGovernanceStore
|
||||
|
||||
return MemoryGovernanceStore.memories_from_rows(db, builtins.list(rows))
|
||||
|
||||
|
||||
def _conversation_hit(row: Any) -> dict[str, Any]:
|
||||
"""One stored row as a search hit.
|
||||
|
||||
The author travels with the row. A client that is not told who wrote a row
|
||||
has to invent an author, and the invention is always "the user", which
|
||||
files an assistant turn under the person who was talking to it.
|
||||
"""
|
||||
return {
|
||||
"evidence_id": row["evidence_id"],
|
||||
"conversation_id": row["conversation_id"],
|
||||
"source_id": row["source_id"],
|
||||
"workspace": row["workspace"],
|
||||
"project": row["project"],
|
||||
"created_at": row["created_at"],
|
||||
"excerpt": row["verbatim"][:200] if row["verbatim"] else "",
|
||||
# Both columns are NOT NULL on `evidence`, so a stored row always
|
||||
# knows; a summary of a whole conversation has no single author and
|
||||
# never reaches here.
|
||||
"role": row["role"],
|
||||
"written_by": row["written_by"],
|
||||
}
|
||||
|
||||
|
||||
def create_reader_app(settings: ReaderSettings) -> FastAPI:
|
||||
"""A read-only FastAPI application for the Reader process."""
|
||||
_check_imports()
|
||||
database_path = settings.data_dir / "one-memory.sqlite3"
|
||||
if not database_path.exists():
|
||||
raise FileNotFoundError(f"reader_database_missing: {database_path}")
|
||||
|
||||
# Validate schema version without writing.
|
||||
from one_memory.memory_server.migrations import SCHEMA_TABLE
|
||||
from one_memory.schema_gate import BUILD_SCHEMA_VERSION
|
||||
|
||||
with main_database(database_path).read() as db:
|
||||
tables = {
|
||||
str(row[0]) for row in db.execute("SELECT name FROM sqlite_master WHERE type='table'")
|
||||
}
|
||||
if SCHEMA_TABLE in tables:
|
||||
row = db.execute(f"SELECT MAX(version) FROM {SCHEMA_TABLE}").fetchone()
|
||||
found = int(row[0]) if row and row[0] is not None else 0
|
||||
if found > BUILD_SCHEMA_VERSION:
|
||||
raise RuntimeError(f"reader_schema_too_new: {found} > {BUILD_SCHEMA_VERSION}")
|
||||
elif not tables:
|
||||
raise RuntimeError("reader_database_empty")
|
||||
|
||||
access = SourceReadAccess(database_path)
|
||||
forwarder = SideEffectForwarder(settings)
|
||||
governance = _ReaderGovernance(database_path)
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(_: FastAPI) -> AsyncIterator[None]:
|
||||
monitor = asyncio.get_running_loop().create_task(_memory_monitor())
|
||||
forwarder.start()
|
||||
yield
|
||||
monitor.cancel()
|
||||
forwarder.stop()
|
||||
|
||||
async def _memory_monitor() -> None:
|
||||
import resource
|
||||
import sys
|
||||
|
||||
while True:
|
||||
await asyncio.sleep(30)
|
||||
usage = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
|
||||
rss_bytes = usage if sys.platform == "darwin" else usage * 1024
|
||||
if rss_bytes > MAX_READER_RSS_BYTES:
|
||||
logger.error(
|
||||
"reader_memory_limit_exceeded: %d bytes > %d",
|
||||
rss_bytes,
|
||||
MAX_READER_RSS_BYTES,
|
||||
)
|
||||
os._exit(1)
|
||||
|
||||
app = FastAPI(title="MemHub Reader", lifespan=lifespan)
|
||||
app.state.settings = settings
|
||||
app.state.database_path = database_path
|
||||
|
||||
expected_token = settings.api_token.get_secret_value()
|
||||
|
||||
def require_auth(request: Request) -> None:
|
||||
import hmac
|
||||
|
||||
origin = request.headers.get("origin")
|
||||
if origin is not None and origin not in (
|
||||
"one-memory://app",
|
||||
"http://localhost:5173",
|
||||
):
|
||||
raise HTTPException(403, "origin_not_allowed")
|
||||
authorization = request.headers.get("authorization", "")
|
||||
scheme, _, token = authorization.partition(" ")
|
||||
if not (scheme.lower() == "bearer" and hmac.compare_digest(token, expected_token)):
|
||||
raise HTTPException(401, "invalid_local_token")
|
||||
|
||||
protected = [Depends(require_auth)]
|
||||
|
||||
@app.get("/api/v1/home-summary", response_model=HomeSummary, dependencies=protected)
|
||||
def home_summary() -> HomeSummary:
|
||||
try:
|
||||
return read_home_summary(database_path)
|
||||
except HomeSummaryUnavailable:
|
||||
# Belt and braces: read_home_summary answers stale itself, but if
|
||||
# that ever changes, never let the homepage 500.
|
||||
return HomeSummary(
|
||||
memory_count=0, latest=None, observed_at=datetime.now(UTC), stale=True
|
||||
)
|
||||
|
||||
@app.get("/api/v1/memory-library", response_model=MemoryLibraryPage, dependencies=protected)
|
||||
def memory_library(
|
||||
query: str | None = Query(default=None, max_length=512),
|
||||
status: str | None = None,
|
||||
active_only: bool = False,
|
||||
source: str | None = Query(default=None, min_length=1),
|
||||
offset: int = Query(default=0, ge=0),
|
||||
limit: int = Query(default=100, ge=1, le=500),
|
||||
) -> MemoryLibraryPage:
|
||||
from one_memory.memory_governance import MemoryStatus as MS
|
||||
|
||||
ms = MS(status) if status else None
|
||||
return read_memory_library(
|
||||
database_path,
|
||||
user_id="local",
|
||||
query=query,
|
||||
status=ms,
|
||||
active_only=active_only,
|
||||
source=source,
|
||||
offset=offset,
|
||||
limit=limit,
|
||||
)
|
||||
|
||||
@app.get("/api/v1/memories", dependencies=protected)
|
||||
def list_memories(
|
||||
status: str | None = None,
|
||||
query: str | None = None,
|
||||
limit: int = Query(default=100, ge=1, le=500),
|
||||
offset: int = Query(default=0, ge=0),
|
||||
) -> list[dict[str, Any]]:
|
||||
from one_memory.memory_governance import MemoryStatus as MS
|
||||
|
||||
ms = MS(status) if status else None
|
||||
memories = governance.list(
|
||||
user_id="local", status=ms, query=query, limit=limit, offset=offset
|
||||
)
|
||||
visible = access.memories(memories)
|
||||
return [m.model_dump(mode="json") for m in visible]
|
||||
|
||||
@app.get("/api/v1/memories/graph", dependencies=protected)
|
||||
async def memory_graph(
|
||||
limit: int = Query(default=160, ge=1, le=300),
|
||||
) -> dict[str, Any]:
|
||||
memories = governance.list(user_id="local", status=MemoryStatus.CONFIRMED, limit=500)
|
||||
projected = access.memories(memories[:limit])
|
||||
nodes: dict[str, dict[str, Any]] = {}
|
||||
edges: list[dict[str, Any]] = []
|
||||
source_ids: set[str] = set()
|
||||
with main_database(database_path).read(row_factory=sqlite3.Row) as db:
|
||||
for memory in projected:
|
||||
node_id = f"memory:{memory.memory_id}"
|
||||
nodes[node_id] = {
|
||||
"node_id": node_id,
|
||||
"node_type": "memory",
|
||||
"label": memory.content,
|
||||
"memory_id": memory.memory_id,
|
||||
"kind": memory.kind.value
|
||||
if hasattr(memory.kind, "value")
|
||||
else str(memory.kind),
|
||||
"status": memory.status.value
|
||||
if hasattr(memory.status, "value")
|
||||
else str(memory.status),
|
||||
}
|
||||
evidence_ids = list(memory.evidence_ids)
|
||||
if evidence_ids:
|
||||
placeholders = ",".join("?" for _ in evidence_ids)
|
||||
sql = (
|
||||
f"SELECT DISTINCT source_id FROM evidence"
|
||||
f" WHERE evidence_id IN ({placeholders})"
|
||||
)
|
||||
for row in db.execute(
|
||||
sql,
|
||||
evidence_ids,
|
||||
):
|
||||
sid = str(row["source_id"])
|
||||
source_ids.add(sid)
|
||||
for sid in sorted(source_ids):
|
||||
source_node_id = f"source:{sid}"
|
||||
nodes[source_node_id] = {
|
||||
"node_id": source_node_id,
|
||||
"node_type": "source",
|
||||
"label": sid,
|
||||
"source_id": sid,
|
||||
}
|
||||
for memory in projected:
|
||||
if any(_source_for_evidence(db, eid) == sid for eid in memory.evidence_ids):
|
||||
memory_node_id = f"memory:{memory.memory_id}"
|
||||
edges.append(
|
||||
{
|
||||
"edge_id": f"{source_node_id}->{memory_node_id}",
|
||||
"source_node_id": source_node_id,
|
||||
"target_node_id": memory_node_id,
|
||||
}
|
||||
)
|
||||
# Fetch HY graph overlay from Core (not available in Reader).
|
||||
hy_overlay = None
|
||||
with contextlib.suppress(Exception):
|
||||
hy_overlay = await _core_get(
|
||||
settings,
|
||||
"/internal/graph-snapshot",
|
||||
timeout_s=5.0,
|
||||
params={"limit": limit},
|
||||
)
|
||||
if hy_overlay and not hy_overlay.get("consulted", False):
|
||||
hy_overlay = None
|
||||
return {
|
||||
"limit": limit,
|
||||
"total": len(projected),
|
||||
"nodes": tuple(nodes.values()),
|
||||
"edges": edges,
|
||||
**({"hy_overlay": hy_overlay} if hy_overlay else {}),
|
||||
}
|
||||
|
||||
@app.get("/api/v1/memories/{memory_id}", dependencies=protected)
|
||||
def get_memory(memory_id: str) -> dict[str, Any]:
|
||||
memory = _get_memory_direct(database_path, memory_id, "local")
|
||||
if memory is None:
|
||||
raise HTTPException(404, "memory_not_found")
|
||||
if not access.memories([memory]):
|
||||
raise HTTPException(403, "memory_source_revoked")
|
||||
return cast(dict[str, Any], memory.model_dump(mode="json"))
|
||||
|
||||
@app.post("/api/v1/recall", response_model=None, dependencies=protected)
|
||||
async def recall(request: RecallRequest) -> dict[str, Any]:
|
||||
from one_memory.recall import GovernedRecallTarget
|
||||
|
||||
started = time.monotonic()
|
||||
target = GovernedRecallTarget(
|
||||
governance=governance,
|
||||
semantic_search=None,
|
||||
resolve_workspace=None,
|
||||
filter_memories=access.memories,
|
||||
)
|
||||
|
||||
if request.query == _PROBE_QUERY:
|
||||
return target.recall(request, user_id="local").model_dump(mode="json")
|
||||
deadline = time.monotonic() + request.timeout_ms / 1_000
|
||||
semantic_ids: tuple[str, ...] = ()
|
||||
try:
|
||||
remaining = max(0.1, deadline - time.monotonic())
|
||||
result = await asyncio.wait_for(
|
||||
_core_get(
|
||||
settings,
|
||||
"/internal/semantic-ids",
|
||||
timeout_s=remaining,
|
||||
params={"query": request.query, "limit": request.limit},
|
||||
),
|
||||
timeout=remaining,
|
||||
)
|
||||
semantic_ids = tuple(str(value) for value in result.get("ids", []) if value)
|
||||
except Exception:
|
||||
pass
|
||||
readable = target._readable_scopes(request)
|
||||
# Keyword candidates from read-only SQL.
|
||||
keywords = target._keyword_candidates(
|
||||
request, user_id="local", readable=readable, deadline=deadline
|
||||
)
|
||||
receipt = target._rank(
|
||||
request,
|
||||
user_id="local",
|
||||
readable=readable,
|
||||
semantic_hits={
|
||||
memory_id: SemanticMemoryHit(
|
||||
memory_id,
|
||||
1.0 - index / max(1, len(semantic_ids)),
|
||||
)
|
||||
for index, memory_id in enumerate(semantic_ids)
|
||||
},
|
||||
keywords=keywords
|
||||
if isinstance(keywords, _KeywordCandidates)
|
||||
else _KeywordCandidates((), False, False),
|
||||
deadline=deadline,
|
||||
timed_out=False,
|
||||
)
|
||||
forwarder.enqueue(
|
||||
{
|
||||
"tool": "om_recall",
|
||||
"query": request.query,
|
||||
"scope": request.scope,
|
||||
"state": receipt.state.value,
|
||||
"memory_count": len(receipt.hits),
|
||||
"elapsed_ms": (time.monotonic() - started) * 1000,
|
||||
}
|
||||
)
|
||||
return receipt.model_dump(mode="json")
|
||||
|
||||
@app.get("/api/v1/conversations/search", dependencies=protected)
|
||||
def search_conversations(
|
||||
query: str = Query(min_length=1, max_length=512),
|
||||
limit: int = Query(default=20, ge=1, le=100),
|
||||
) -> dict[str, Any]:
|
||||
if not database_path.exists():
|
||||
raise HTTPException(503, "conversation_search_unavailable")
|
||||
with main_database(database_path).read(row_factory=sqlite3.Row) as db:
|
||||
try:
|
||||
rows = db.execute(
|
||||
"""SELECT * FROM evidence
|
||||
WHERE user_id = 'local' AND verbatim LIKE ? ESCAPE '\\'
|
||||
ORDER BY imported_at DESC LIMIT ?""",
|
||||
(f"%{query}%", limit),
|
||||
).fetchall()
|
||||
except Exception:
|
||||
raise HTTPException(503, "conversation_search_unavailable") from None
|
||||
hits = [_conversation_hit(row) for row in rows]
|
||||
return {
|
||||
"hits": hits,
|
||||
"empty_reason": None if hits else "no_matches",
|
||||
"imported_messages": len(hits),
|
||||
}
|
||||
|
||||
@app.get("/health")
|
||||
def health() -> dict[str, str]:
|
||||
return {"status": "ok", "service": "one-memory-reader"}
|
||||
|
||||
return app
|
||||
|
||||
|
||||
def _get_memory_direct(database_path: Path, memory_id: str, user_id: str) -> Any:
|
||||
from one_memory.memory_governance import _memory_from_row, _memory_links
|
||||
|
||||
with main_database(database_path).read(row_factory=sqlite3.Row) as db:
|
||||
row = db.execute(
|
||||
"SELECT * FROM governed_memories WHERE memory_id = ? AND user_id = ?",
|
||||
(memory_id, user_id),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
links = _memory_links(db, [memory_id])
|
||||
return _memory_from_row(row, *(tuple(vals) for vals in links[memory_id]))
|
||||
|
||||
|
||||
def _source_for_evidence(db: sqlite3.Connection, evidence_id: str) -> str | None:
|
||||
row = db.execute(
|
||||
"SELECT source_id FROM evidence WHERE evidence_id = ?", (evidence_id,)
|
||||
).fetchone()
|
||||
return str(row["source_id"]) if row else None
|
||||
@@ -81,7 +81,11 @@ class RecallHit(BaseModel):
|
||||
score: float = Field(ge=0, le=1)
|
||||
#: HY already rendered memory time/evolution/capsule context into content.
|
||||
#: Hosts must not append their own governance timestamp in this case.
|
||||
content_format: Literal["plain", "hy-sdk"] = "plain"
|
||||
#: "raw" is an excerpt of an imported conversation, not a memory.
|
||||
content_format: Literal["plain", "hy-sdk", "raw"] = "plain"
|
||||
#: Raw excerpts only: the app and conversation the excerpt came from.
|
||||
source: str | None = None
|
||||
session_id: str | None = None
|
||||
|
||||
|
||||
class RecallReceipt(BaseModel):
|
||||
@@ -103,6 +107,11 @@ class RecallReceipt(BaseModel):
|
||||
#: whether the answer is that nothing was found or that reading was not
|
||||
#: allowed.
|
||||
excluded_sources: tuple[str, ...] = ()
|
||||
#: "raw" when recall searched the imported conversations instead of
|
||||
#: memories, because no generation model is configured or no memory
|
||||
#: exists yet (`fallback_reason`). Raw hits are excerpts, not memories.
|
||||
mode: Literal["memory", "raw"] = "memory"
|
||||
fallback_reason: Literal["model_unavailable", "no_memories"] | None = None
|
||||
|
||||
|
||||
class RecallTarget(Protocol):
|
||||
|
||||
@@ -0,0 +1,149 @@
|
||||
"""Recall that degrades to the original conversations instead of answering empty.
|
||||
|
||||
MemHub_recall searches the memories the SDK distilled from conversations. That
|
||||
needs a configured generation model to have produced any, and a library that
|
||||
holds some. Without either, the memory search can only come back empty, so
|
||||
recall searches the SDK's raw layer (the imported conversations themselves)
|
||||
instead: plain retrieval over what was imported, the same lane
|
||||
`/api/v2/context/search` reads, under the same raw-source authority.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from collections.abc import Awaitable, Callable
|
||||
from datetime import UTC, datetime
|
||||
from typing import Literal
|
||||
|
||||
from one_memory.hy_sdk_types import HySdkRawHit
|
||||
from one_memory.recall import (
|
||||
PROBE_QUERY,
|
||||
RecallHit,
|
||||
RecallReceipt,
|
||||
RecallRequest,
|
||||
RecallState,
|
||||
TruncationBudget,
|
||||
)
|
||||
|
||||
FallbackReason = Literal["model_unavailable", "no_memories"]
|
||||
|
||||
#: Raw sources this caller may read, or None for every source (a trusted local
|
||||
#: caller). An empty set means the caller may read no raw conversation.
|
||||
RawSources = frozenset[str] | None
|
||||
|
||||
|
||||
async def recall_with_fallback(
|
||||
request: RecallRequest,
|
||||
*,
|
||||
model_ready: Callable[[], Awaitable[bool]],
|
||||
recall_memories: Callable[[RecallRequest], Awaitable[RecallReceipt]],
|
||||
has_memories: Callable[[], Awaitable[bool]],
|
||||
raw_sources: Callable[[], Awaitable[RawSources]],
|
||||
search_raw: Callable[[str, int, RawSources], Awaitable[tuple[HySdkRawHit, ...]]],
|
||||
) -> RecallReceipt:
|
||||
"""Memory recall when a model and memories exist, otherwise the raw layer.
|
||||
|
||||
The library check runs only after an empty memory search: with memories
|
||||
present and nothing relevant, the answer stays an honest empty memory
|
||||
result rather than a different kind of hit.
|
||||
"""
|
||||
if request.query == PROBE_QUERY:
|
||||
return await recall_memories(request)
|
||||
started = time.monotonic()
|
||||
if await model_ready():
|
||||
receipt = await recall_memories(request)
|
||||
if receipt.hits or await has_memories():
|
||||
return receipt
|
||||
reason: FallbackReason = "no_memories"
|
||||
else:
|
||||
reason = "model_unavailable"
|
||||
remaining = request.timeout_ms / 1_000 - (time.monotonic() - started)
|
||||
return await _raw_recall(
|
||||
request,
|
||||
reason=reason,
|
||||
remaining_seconds=remaining,
|
||||
raw_sources=raw_sources,
|
||||
search_raw=search_raw,
|
||||
)
|
||||
|
||||
|
||||
async def _raw_recall(
|
||||
request: RecallRequest,
|
||||
*,
|
||||
reason: FallbackReason,
|
||||
remaining_seconds: float,
|
||||
raw_sources: Callable[[], Awaitable[RawSources]],
|
||||
search_raw: Callable[[str, int, RawSources], Awaitable[tuple[HySdkRawHit, ...]]],
|
||||
) -> RecallReceipt:
|
||||
def receipt(
|
||||
hits: tuple[RecallHit, ...] = (),
|
||||
*,
|
||||
used: int = 0,
|
||||
truncated_by: TruncationBudget | None = None,
|
||||
unexamined: int = 0,
|
||||
) -> RecallReceipt:
|
||||
return RecallReceipt(
|
||||
state=RecallState.GROUNDED if hits else RecallState.EMPTY,
|
||||
query=request.query,
|
||||
scope=request.scope,
|
||||
hits=hits,
|
||||
used_characters=used,
|
||||
truncated=truncated_by is not None,
|
||||
truncated_by=truncated_by,
|
||||
unexamined_count=unexamined,
|
||||
mode="raw",
|
||||
fallback_reason=reason,
|
||||
)
|
||||
|
||||
sources = await raw_sources()
|
||||
if sources is not None and not sources:
|
||||
# The caller may read no original conversation: an empty answer, not
|
||||
# a wider grant than recall had.
|
||||
return receipt()
|
||||
if remaining_seconds <= 0:
|
||||
return receipt(truncated_by=TruncationBudget.TIMEOUT)
|
||||
try:
|
||||
found = await asyncio.wait_for(
|
||||
search_raw(request.query, request.limit, sources),
|
||||
timeout=remaining_seconds,
|
||||
)
|
||||
except TimeoutError:
|
||||
return receipt(truncated_by=TruncationBudget.TIMEOUT)
|
||||
hits: list[RecallHit] = []
|
||||
used = 0
|
||||
truncated_by: TruncationBudget | None = None
|
||||
for index, hit in enumerate(found):
|
||||
if sources is not None and hit.source not in sources:
|
||||
continue
|
||||
if len(hits) >= request.limit:
|
||||
truncated_by = TruncationBudget.COUNT
|
||||
return receipt(
|
||||
tuple(hits),
|
||||
used=used,
|
||||
truncated_by=truncated_by,
|
||||
unexamined=len(found) - index,
|
||||
)
|
||||
if used + len(hit.content) > request.character_budget:
|
||||
truncated_by = TruncationBudget.CHARACTER
|
||||
continue
|
||||
hits.append(
|
||||
RecallHit(
|
||||
memory_id=hit.raw_id,
|
||||
content=hit.content,
|
||||
kind="raw",
|
||||
scopes=(),
|
||||
evidence_ids=(),
|
||||
updated_at=(
|
||||
datetime.fromtimestamp(hit.memory_at, tz=UTC)
|
||||
if hit.memory_at is not None
|
||||
else datetime(1970, 1, 1, tzinfo=UTC)
|
||||
),
|
||||
score=hit.score,
|
||||
content_format="raw",
|
||||
source=hit.source,
|
||||
session_id=hit.session_id,
|
||||
)
|
||||
)
|
||||
used += len(hit.content)
|
||||
return receipt(tuple(hits), used=used, truncated_by=truncated_by)
|
||||
@@ -41,6 +41,20 @@ class MigrationProgress:
|
||||
phase: str
|
||||
|
||||
|
||||
def _ledger_log(database_path: Path, event: str, migration_id: str, **details: object) -> None:
|
||||
"""Every ledger transition goes to `sdk-migration.log`; never raises."""
|
||||
from one_memory.sdk_migration_log import write_migration_log
|
||||
|
||||
write_migration_log(
|
||||
database_path.parent,
|
||||
writer="core-sdk-migrate",
|
||||
event=event,
|
||||
outcome="succeeded",
|
||||
migration_id=migration_id,
|
||||
details=details,
|
||||
)
|
||||
|
||||
|
||||
class SdkDataPlaneMigrationStore:
|
||||
def __init__(self, database_path: Path) -> None:
|
||||
self.database_path = database_path
|
||||
@@ -151,7 +165,16 @@ class SdkDataPlaneMigrationStore:
|
||||
elif backup_bundle_id and existing["backup_bundle_id"] != backup_bundle_id:
|
||||
raise DataPlaneMigrationError("sdk_data_plane_migration_bundle_conflict")
|
||||
connection.execute("COMMIT")
|
||||
return self.progress(migration_id)
|
||||
progress = self.progress(migration_id)
|
||||
_ledger_log(
|
||||
self.database_path,
|
||||
"ledger.started",
|
||||
migration_id,
|
||||
created=existing is None,
|
||||
state=progress.state,
|
||||
phase=progress.phase,
|
||||
)
|
||||
return progress
|
||||
|
||||
def set_phase(self, migration_id: str, *, phase: str, state: str = "running") -> None:
|
||||
with main_database(self.database_path).write(autocommit=True) as connection:
|
||||
@@ -164,6 +187,7 @@ class SdkDataPlaneMigrationStore:
|
||||
if cursor.rowcount != 1:
|
||||
raise DataPlaneMigrationError("sdk_data_plane_migration_not_mutable")
|
||||
connection.execute("COMMIT")
|
||||
_ledger_log(self.database_path, "ledger.phase_set", migration_id, phase=phase, state=state)
|
||||
|
||||
def progress(self, migration_id: str) -> MigrationProgress:
|
||||
with main_database(self.database_path).read(row_factory=sqlite3.Row) as connection:
|
||||
@@ -257,6 +281,7 @@ class SdkDataPlaneMigrationStore:
|
||||
(migration_id, now),
|
||||
)
|
||||
connection.execute("COMMIT")
|
||||
_ledger_log(self.database_path, "ledger.authority_committed", migration_id, authority="sdk")
|
||||
marker = self.cutover()
|
||||
if marker is None:
|
||||
raise DataPlaneMigrationError("sdk_data_plane_cutover_missing")
|
||||
|
||||
@@ -0,0 +1,268 @@
|
||||
"""The one log support asks for when an upgrade did not reach SDK authority.
|
||||
|
||||
`<data_dir>/logs/sdk-migration.log` is JSON Lines, shared with the TypeScript
|
||||
orchestrator (`packages/runtime-control/src/sdk-migration-log.ts`), which writes
|
||||
the same line shape. A person attaches this single file to a report, so every
|
||||
line says who wrote it, which build, which migration and what happened.
|
||||
|
||||
It must never carry user content: no memory or conversation text, no provider
|
||||
response, no exception message (a message can quote either). Errors are logged
|
||||
as their type, a sanitised code and traceback frames as `file:line:function`.
|
||||
Detail values are numbers, booleans and short identifier-shaped strings only;
|
||||
anything else is dropped rather than escaped.
|
||||
|
||||
Writing is synchronous and flushed: a migration is rare and the line that
|
||||
explains a failure is worth more than the milliseconds. A write failure is
|
||||
swallowed, because logging must never be the reason a migration or a startup
|
||||
fails.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import traceback
|
||||
from collections.abc import Iterator, Mapping
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
LOG_NAME = "sdk-migration.log"
|
||||
#: Rotated to `sdk-migration.log.1` beyond this; one generation is kept.
|
||||
MAX_BYTES = 4 * 1024 * 1024
|
||||
|
||||
_SAFE_TEXT = re.compile(r"^[A-Za-z0-9_.:/@+\-]{0,160}$")
|
||||
_CODE = re.compile(r"^[a-z][a-z0-9]*(?:[._-][a-z0-9]+){0,15}$")
|
||||
_SUBJECT = re.compile(r"^[A-Za-z0-9_.\-]{1,96}$")
|
||||
_MAX_ITEMS = 32
|
||||
_MAX_DEPTH = 4
|
||||
_MAX_FRAMES = 24
|
||||
_FRAME = re.compile(r"^[A-Za-z0-9_.\-]{1,96}:\d{1,7}:<?[A-Za-z0-9_]{1,96}>?$")
|
||||
|
||||
|
||||
def log_path(data_dir: Path) -> Path:
|
||||
return data_dir / "logs" / LOG_NAME
|
||||
|
||||
|
||||
def _safe(value: object, depth: int = 0) -> Any:
|
||||
if value is None or isinstance(value, bool):
|
||||
return value
|
||||
if isinstance(value, int):
|
||||
return value
|
||||
if isinstance(value, float):
|
||||
return round(value, 3) if value == value and abs(value) != float("inf") else None
|
||||
if isinstance(value, str):
|
||||
return value if _SAFE_TEXT.fullmatch(value) else "<redacted>"
|
||||
if depth >= _MAX_DEPTH:
|
||||
return "<nested>"
|
||||
if isinstance(value, Mapping):
|
||||
result: dict[str, Any] = {}
|
||||
for index, (key, item) in enumerate(value.items()):
|
||||
if index >= _MAX_ITEMS:
|
||||
result["<truncated>"] = len(value) - _MAX_ITEMS
|
||||
break
|
||||
if isinstance(key, str) and _SAFE_TEXT.fullmatch(key):
|
||||
result[key] = _safe(item, depth + 1)
|
||||
return result
|
||||
if isinstance(value, (list, tuple, set, frozenset)):
|
||||
items = list(value)
|
||||
safe = [_safe(item, depth + 1) for item in items[:_MAX_ITEMS]]
|
||||
if len(items) > _MAX_ITEMS:
|
||||
safe.append(f"<+{len(items) - _MAX_ITEMS}>")
|
||||
return safe
|
||||
return "<unsupported>"
|
||||
|
||||
|
||||
def _chain(error: BaseException) -> Iterator[BaseException]:
|
||||
seen: set[int] = set()
|
||||
current: BaseException | None = error
|
||||
while current is not None and id(current) not in seen:
|
||||
seen.add(id(current))
|
||||
yield current
|
||||
current = current.__cause__ or current.__context__
|
||||
|
||||
|
||||
def describe_error(error: BaseException) -> dict[str, object]:
|
||||
"""Type, code and frames of an exception chain; never its message."""
|
||||
chain: list[dict[str, object]] = []
|
||||
for item in _chain(error):
|
||||
name = type(item).__name__
|
||||
entry: dict[str, object] = {"type": name if _SAFE_TEXT.fullmatch(name) else "<unknown>"}
|
||||
code = getattr(item, "code", None)
|
||||
if isinstance(code, str):
|
||||
# `<code>:<opaque id>` names the row that failed, e.g. a write job.
|
||||
name, _, subject = code.partition(":")
|
||||
if _CODE.fullmatch(name):
|
||||
entry["code"] = name
|
||||
if subject and _SUBJECT.fullmatch(subject):
|
||||
entry["subject"] = subject
|
||||
message = str(item)
|
||||
# A message that is itself a stable code (the codebase's convention
|
||||
# for domain errors) is safe to keep; anything else is content.
|
||||
if _CODE.fullmatch(message):
|
||||
entry["code"] = entry.get("code") or message
|
||||
errno = getattr(item, "errno", None)
|
||||
if isinstance(errno, int):
|
||||
entry["errno"] = errno
|
||||
if isinstance(item, OSError) and item.filename is not None:
|
||||
# A path names a file, not its contents, but home directories can
|
||||
# name a person: keep only the file name.
|
||||
entry["file"] = Path(str(item.filename)).name
|
||||
frames = traceback.extract_tb(item.__traceback__)[-_MAX_FRAMES:]
|
||||
# Code locations, not content: built here from file names, line
|
||||
# numbers and function names (`<module>`, `<lambda>` included).
|
||||
entry["frames"] = [
|
||||
location
|
||||
for frame in frames
|
||||
if _FRAME.fullmatch(
|
||||
location := f"{Path(frame.filename).name}:{frame.lineno}:{frame.name}"
|
||||
)
|
||||
]
|
||||
chain.append(entry)
|
||||
if len(chain) >= 6:
|
||||
break
|
||||
return {"chain": chain}
|
||||
|
||||
|
||||
_PY_FRAME = re.compile(r"^[A-Za-z0-9_][A-Za-z0-9_.\-]*\.py:\d{1,7}:[A-Za-z_][A-Za-z0-9_]*$")
|
||||
|
||||
|
||||
def error_frames(error: BaseException, *, limit: int = 16) -> tuple[str, ...]:
|
||||
"""The raised error's innermost `file.py:line:function` frames, for a stack field.
|
||||
|
||||
Library frames are kept (a KeyError raised inside Chroma is located by
|
||||
them); `<module>`-style names are dropped because stack fields refuse them.
|
||||
"""
|
||||
chain = describe_error(error)["chain"]
|
||||
frames = chain[0].get("frames", ()) if chain else () # type: ignore[index,union-attr]
|
||||
return tuple(frame for frame in frames if _PY_FRAME.fullmatch(frame))[-limit:]
|
||||
|
||||
|
||||
def error_chain_text(error: BaseException) -> str:
|
||||
"""`describe_error` as one log token: `Type@file:line:function>...;Cause@...`.
|
||||
|
||||
For a plain text log (core.log). Types and frames only, never a message.
|
||||
"""
|
||||
return ";".join(
|
||||
f"{entry['type']}@{'>'.join(entry.get('frames', ()))}" # type: ignore[arg-type]
|
||||
for entry in describe_error(error)["chain"] # type: ignore[union-attr]
|
||||
)
|
||||
|
||||
|
||||
def _build() -> dict[str, object]:
|
||||
from one_memory import __version__
|
||||
|
||||
build: dict[str, object] = {"core_version": __version__}
|
||||
for key, name in (
|
||||
("ONE_MEMORY_VERSION", "app_version"),
|
||||
("ONE_MEMORY_BUILD_NUMBER", "build_number"),
|
||||
("ONE_MEMORY_COMMIT", "build_commit"),
|
||||
("ONE_MEMORY_MIGRATION_CALLER", "caller"),
|
||||
("ONE_MEMORY_MIGRATION_RUNTIME_BUILD", "runtime_build"),
|
||||
):
|
||||
value = os.environ.get(key, "").strip()
|
||||
if value:
|
||||
build[name] = value
|
||||
return _safe(build)
|
||||
|
||||
|
||||
def write_migration_log(
|
||||
data_dir: Path,
|
||||
*,
|
||||
writer: str,
|
||||
event: str,
|
||||
outcome: str,
|
||||
migration_id: str | None = None,
|
||||
details: Mapping[str, object] | None = None,
|
||||
error: BaseException | None = None,
|
||||
code: str | None = None,
|
||||
) -> None:
|
||||
"""Append one line. Never raises."""
|
||||
try:
|
||||
line: dict[str, object] = {
|
||||
"schema_version": 1,
|
||||
"ts": datetime.now(UTC).isoformat(),
|
||||
"pid": os.getpid(),
|
||||
"writer": writer,
|
||||
"event": event,
|
||||
"outcome": outcome,
|
||||
**_build(),
|
||||
}
|
||||
if migration_id is not None:
|
||||
line["migration_id"] = migration_id
|
||||
if code is not None:
|
||||
line["code"] = code
|
||||
if details:
|
||||
line["details"] = dict(details)
|
||||
safe_line = _safe(line)
|
||||
if not isinstance(safe_line, dict):
|
||||
return
|
||||
if error is not None:
|
||||
# Already content-free by construction; `_safe` would redact the
|
||||
# `<module>` frames.
|
||||
safe_line["error"] = describe_error(error)
|
||||
destination = log_path(data_dir)
|
||||
destination.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||
try:
|
||||
if destination.stat().st_size > MAX_BYTES:
|
||||
os.replace(destination, destination.with_name(f"{LOG_NAME}.1"))
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
payload = (json.dumps(safe_line, ensure_ascii=False) + "\n").encode("utf-8")
|
||||
descriptor = os.open(destination, os.O_WRONLY | os.O_APPEND | os.O_CREAT, 0o600)
|
||||
try:
|
||||
os.write(descriptor, payload)
|
||||
os.fsync(descriptor)
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
except Exception: # noqa: BLE001 - logging must never break a migration
|
||||
return
|
||||
|
||||
|
||||
def migration_rows(database_path: Path) -> list[dict[str, object]]:
|
||||
"""The recorded migration attempts, newest first; ids, states and phases only."""
|
||||
import sqlite3
|
||||
|
||||
try:
|
||||
connection = sqlite3.connect(f"file:{database_path}?mode=ro", uri=True, timeout=1)
|
||||
except sqlite3.Error:
|
||||
return []
|
||||
try:
|
||||
rows = connection.execute(
|
||||
"SELECT migration_id,state,phase,created_at,updated_at "
|
||||
"FROM sdk_data_plane_migrations ORDER BY created_at DESC LIMIT 8"
|
||||
).fetchall()
|
||||
except sqlite3.Error:
|
||||
return []
|
||||
finally:
|
||||
connection.close()
|
||||
return [
|
||||
{
|
||||
"migration_id": row[0],
|
||||
"state": row[1],
|
||||
"phase": row[2],
|
||||
# Older builds stored `YYYY-MM-DD HH:MM:SS`; keep it identifier-shaped.
|
||||
"created_at": str(row[3]).replace(" ", "T"),
|
||||
"updated_at": str(row[4]).replace(" ", "T"),
|
||||
}
|
||||
for row in rows
|
||||
]
|
||||
|
||||
|
||||
def authority_snapshot(database_path: Path) -> dict[str, object]:
|
||||
"""Authority and recorded attempts, read without changing the database."""
|
||||
if not database_path.is_file():
|
||||
return {"database": "absent"}
|
||||
snapshot: dict[str, object] = {"database": "present"}
|
||||
try:
|
||||
from one_memory.sdk_data_plane_migration import SdkDataPlaneMigrationStore
|
||||
|
||||
authority, authority_migration = SdkDataPlaneMigrationStore(database_path).authority_state()
|
||||
snapshot["authority"] = authority.value
|
||||
snapshot["authority_migration_id"] = authority_migration
|
||||
except Exception as error: # noqa: BLE001 - a snapshot must not fail its caller
|
||||
snapshot["authority"] = "unreadable"
|
||||
snapshot["authority_error"] = type(error).__name__
|
||||
snapshot["migrations"] = migration_rows(database_path)
|
||||
return snapshot
|
||||
@@ -0,0 +1,504 @@
|
||||
"""Put a home back exactly as it was when a failed `sdk-migrate` started.
|
||||
|
||||
A migration that fails before SDK authority is committed has already changed
|
||||
the home: schema migrations on `one-memory.sqlite3`, legacy work moved into the
|
||||
durable queue, renamed Chroma collections under `hy-memory/governed-v2`, ledger
|
||||
rows. The fixed release that follows must start from the original data, not
|
||||
from that half-way state, so the runner restores the verified full-home bundle
|
||||
it took before the first change (`restore point`).
|
||||
|
||||
The restore is staged and then swapped in with renames:
|
||||
|
||||
1. the bundle is verified (every file hashed against its manifest);
|
||||
2. its database, config and SDK root are copied next to their live places and
|
||||
verified again;
|
||||
3. a journal lists every rename, then the live items (database with its
|
||||
`-wal`/`-shm`, `config.json`, `hy-memory/governed-v2`) are renamed into
|
||||
`backups/sdk-migration-failed/` and the staged copies are renamed in.
|
||||
|
||||
A rename either happened or did not, and each step is idempotent, so a process
|
||||
killed during step 3 is finished by the next run (`finish_interrupted_restore`)
|
||||
before anything else reads the home. Core refuses to serve and the status
|
||||
probe reports `required` while a journal exists.
|
||||
|
||||
After SDK authority is committed nothing is restored: the home already belongs
|
||||
to the SDK and the legacy tables may be gone.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import errno
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
|
||||
from one_memory.database_access import close_main_databases
|
||||
from one_memory.hy_runtime_config import HY_MEMORY_NAMESPACE
|
||||
from one_memory.sdk_data_plane_migration import DataPlaneMigrationError
|
||||
|
||||
RESTORE_POINT = ".sdk-migration-restore-point.json"
|
||||
RESTORE_STAGING = ".sdk-migration-restore"
|
||||
JOURNAL = "journal.json"
|
||||
FAILED_STATE = "sdk-migration-failed"
|
||||
RESTORED_MARK = "restored.json"
|
||||
DATABASE = "one-memory.sqlite3"
|
||||
CONFIG = "config.json"
|
||||
_DATABASE_SIDECARS = ("-wal", "-shm", "-journal")
|
||||
_WRITER = "core-sdk-migrate"
|
||||
|
||||
|
||||
class MigrationRestoreFailed(DataPlaneMigrationError):
|
||||
"""The pre-migration data could not be put back; the home stays blocked."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RestoreResult:
|
||||
bundle_id: str
|
||||
files: int
|
||||
bytes: int
|
||||
duration_ms: float
|
||||
|
||||
def as_document(self) -> dict[str, object]:
|
||||
return {
|
||||
"bundle_id": self.bundle_id,
|
||||
"files": self.files,
|
||||
"bytes": self.bytes,
|
||||
"duration_ms": self.duration_ms,
|
||||
}
|
||||
|
||||
|
||||
def _home(data_dir: Path) -> Path:
|
||||
return data_dir.resolve()
|
||||
|
||||
|
||||
def _staging(home: Path) -> Path:
|
||||
return home / RESTORE_STAGING
|
||||
|
||||
|
||||
def _journal(home: Path) -> Path:
|
||||
return _staging(home) / JOURNAL
|
||||
|
||||
|
||||
def _sdk_staging(home: Path) -> Path:
|
||||
return home / "hy-memory" / f".{HY_MEMORY_NAMESPACE}.restore-staging"
|
||||
|
||||
|
||||
def failed_state_path(data_dir: Path) -> Path:
|
||||
return _home(data_dir) / "backups" / FAILED_STATE
|
||||
|
||||
|
||||
def _write_json(path: Path, document: dict[str, object]) -> None:
|
||||
temporary = path.with_name(f"{path.name}.{os.getpid()}.tmp")
|
||||
temporary.write_text(json.dumps(document, sort_keys=True) + "\n", encoding="utf-8")
|
||||
os.chmod(temporary, 0o600)
|
||||
_fsync(temporary)
|
||||
temporary.replace(path)
|
||||
_fsync_directory(path.parent)
|
||||
|
||||
|
||||
def _fsync(path: Path) -> None:
|
||||
descriptor = os.open(path, os.O_RDONLY)
|
||||
try:
|
||||
os.fsync(descriptor)
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
|
||||
def _fsync_directory(path: Path) -> None:
|
||||
# Windows cannot open a directory; its renames are durable on return.
|
||||
try:
|
||||
_fsync(path)
|
||||
except OSError:
|
||||
return
|
||||
|
||||
|
||||
def _log(
|
||||
data_dir: Path,
|
||||
event: str,
|
||||
outcome: str,
|
||||
*,
|
||||
migration_id: str | None,
|
||||
code: str | None = None,
|
||||
error: BaseException | None = None,
|
||||
details: dict[str, object] | None = None,
|
||||
) -> None:
|
||||
from one_memory.sdk_migration_log import write_migration_log
|
||||
|
||||
write_migration_log(
|
||||
data_dir,
|
||||
writer=_WRITER,
|
||||
event=event,
|
||||
outcome=outcome,
|
||||
migration_id=migration_id,
|
||||
code=code,
|
||||
error=error,
|
||||
details=details,
|
||||
)
|
||||
|
||||
|
||||
# --- restore point -----------------------------------------------------------
|
||||
|
||||
|
||||
def record_restore_point(data_dir: Path, *, migration_id: str, bundle_id: str) -> None:
|
||||
"""Name the verified bundle that holds this home as the migration found it.
|
||||
|
||||
Written before the first change, so a run killed part-way is rolled back
|
||||
by the next one instead of resumed.
|
||||
"""
|
||||
_write_json(
|
||||
_home(data_dir) / RESTORE_POINT,
|
||||
{
|
||||
"schema_version": 1,
|
||||
"migration_id": migration_id,
|
||||
"bundle_id": bundle_id,
|
||||
"created_at": datetime.now(UTC).isoformat(),
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def read_restore_point(data_dir: Path) -> dict[str, str] | None:
|
||||
try:
|
||||
document = json.loads((_home(data_dir) / RESTORE_POINT).read_text(encoding="utf-8"))
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
except (OSError, ValueError) as error:
|
||||
raise MigrationRestoreFailed("sdk_migration_restore_point_invalid") from error
|
||||
if (
|
||||
not isinstance(document, dict)
|
||||
or document.get("schema_version") != 1
|
||||
or not isinstance(document.get("migration_id"), str)
|
||||
or not isinstance(document.get("bundle_id"), str)
|
||||
):
|
||||
raise MigrationRestoreFailed("sdk_migration_restore_point_invalid")
|
||||
return {"migration_id": document["migration_id"], "bundle_id": document["bundle_id"]}
|
||||
|
||||
|
||||
def clear_restore_point(data_dir: Path) -> None:
|
||||
(_home(data_dir) / RESTORE_POINT).unlink(missing_ok=True)
|
||||
|
||||
|
||||
def restore_pending(data_dir: Path) -> bool:
|
||||
"""A restore was interrupted between its renames; nothing may read the home."""
|
||||
return _journal(_home(data_dir)).is_file()
|
||||
|
||||
|
||||
# --- restore -----------------------------------------------------------------
|
||||
|
||||
|
||||
def _sha256(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as handle:
|
||||
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def _release_handles() -> None:
|
||||
"""Close what this process holds on the home; Windows cannot rename open files."""
|
||||
close_main_databases()
|
||||
chromadb = sys.modules.get("chromadb")
|
||||
if chromadb is None:
|
||||
return
|
||||
try:
|
||||
from chromadb.api.shared_system_client import SharedSystemClient
|
||||
|
||||
SharedSystemClient.clear_system_cache()
|
||||
except Exception: # noqa: BLE001 - best effort; a held file fails the rename instead
|
||||
return
|
||||
|
||||
|
||||
def _verify_copy(copy: Path, expected: list[dict[str, object]], prefix: str) -> None:
|
||||
"""The staged tree holds exactly the bundle's files under `prefix`."""
|
||||
wanted = {
|
||||
str(item["path"])[len(prefix) :]: item
|
||||
for item in expected
|
||||
if str(item["path"]).startswith(prefix)
|
||||
}
|
||||
present = (
|
||||
{path.relative_to(copy).as_posix() for path in copy.rglob("*") if path.is_file()}
|
||||
if copy.is_dir()
|
||||
else set()
|
||||
)
|
||||
if present != set(wanted):
|
||||
raise MigrationRestoreFailed("sdk_migration_restore_verify_failed")
|
||||
for relative, item in wanted.items():
|
||||
path = copy / relative
|
||||
if path.stat().st_size != int(str(item["bytes"])) or _sha256(path) != item["sha256"]:
|
||||
raise MigrationRestoreFailed("sdk_migration_restore_verify_failed")
|
||||
|
||||
|
||||
def _plan(home: Path, *, config_staged: bool) -> list[tuple[str, str]]:
|
||||
"""Every rename of the swap, as paths relative to the home, in order."""
|
||||
failed = Path("backups") / FAILED_STATE
|
||||
staging = Path(RESTORE_STAGING)
|
||||
sdk_live = Path("hy-memory") / HY_MEMORY_NAMESPACE
|
||||
steps: list[tuple[Path, Path]] = [
|
||||
(Path(f"{DATABASE}{suffix}"), failed / "management" / f"{DATABASE}{suffix}")
|
||||
for suffix in _DATABASE_SIDECARS
|
||||
]
|
||||
steps += [
|
||||
(Path(DATABASE), failed / "management" / DATABASE),
|
||||
(Path(CONFIG), failed / "management" / CONFIG),
|
||||
(sdk_live, failed / "sdk" / HY_MEMORY_NAMESPACE),
|
||||
(staging / "management" / DATABASE, Path(DATABASE)),
|
||||
(_sdk_staging(home).relative_to(home), sdk_live),
|
||||
]
|
||||
if config_staged:
|
||||
steps.append((staging / "management" / CONFIG, Path(CONFIG)))
|
||||
return [(source.as_posix(), target.as_posix()) for source, target in steps]
|
||||
|
||||
|
||||
def _apply(
|
||||
home: Path,
|
||||
steps: list[list[str]] | list[tuple[str, str]],
|
||||
done: list[tuple[Path, Path]],
|
||||
) -> None:
|
||||
"""Run each rename once; a step whose source is already gone is skipped.
|
||||
|
||||
`done` collects the renames this call made, so a caller can undo them.
|
||||
"""
|
||||
for source_name, target_name in steps:
|
||||
source, target = home / source_name, home / target_name
|
||||
source_exists = source.exists() or source.is_symlink()
|
||||
target_exists = target.exists() or target.is_symlink()
|
||||
if not source_exists:
|
||||
continue
|
||||
if target_exists:
|
||||
raise MigrationRestoreFailed("sdk_migration_restore_conflict")
|
||||
target.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||
source.replace(target)
|
||||
done.append((source, target))
|
||||
for directory in (home, home / "hy-memory"):
|
||||
_fsync_directory(directory)
|
||||
|
||||
|
||||
def _finish(home: Path, bundle_id: str) -> None:
|
||||
# The restore point goes first: a kill after it leaves only the journal,
|
||||
# whose steps are all done, and the next run just tidies up.
|
||||
clear_restore_point(home)
|
||||
shutil.rmtree(_sdk_staging(home), ignore_errors=True)
|
||||
_journal(home).unlink(missing_ok=True)
|
||||
shutil.rmtree(_staging(home), ignore_errors=True)
|
||||
bundle = home / "backups" / f"{bundle_id}.sdk-bundle"
|
||||
try:
|
||||
# A later verified bundle supersedes this one (`prune_restored_bundles`).
|
||||
_write_json(
|
||||
bundle / RESTORED_MARK,
|
||||
{"schema_version": 1, "restored_at": datetime.now(UTC).isoformat()},
|
||||
)
|
||||
except OSError:
|
||||
return
|
||||
|
||||
|
||||
def restore_pre_migration_state(
|
||||
data_dir: Path,
|
||||
*,
|
||||
bundle_id: str,
|
||||
migration_id: str | None,
|
||||
reason: str,
|
||||
) -> RestoreResult:
|
||||
"""Put the restore-point bundle back in place of the half-migrated home.
|
||||
|
||||
The caller holds the migration lock with Runtime stopped and host capture
|
||||
fenced. Raises `MigrationRestoreFailed`; before the swap the home is left as
|
||||
the failure left it, and the restore point stays for the next run.
|
||||
"""
|
||||
home = _home(data_dir)
|
||||
started = time.monotonic()
|
||||
_log(
|
||||
home,
|
||||
"restore.started",
|
||||
"started",
|
||||
migration_id=migration_id,
|
||||
details={"bundle_id": bundle_id, "reason": reason},
|
||||
)
|
||||
try:
|
||||
result = _restore(home, bundle_id=bundle_id, started=started)
|
||||
except BaseException as error:
|
||||
code = getattr(error, "code", None)
|
||||
if not (isinstance(code, str) and code.startswith("sdk_migration_restore")):
|
||||
code = (
|
||||
"sdk_migration_restore_disk_space_insufficient"
|
||||
if isinstance(error, OSError) and error.errno in {errno.ENOSPC, errno.EDQUOT}
|
||||
else "sdk_migration_restore_failed"
|
||||
)
|
||||
_log(
|
||||
home,
|
||||
"restore.failed",
|
||||
"failed",
|
||||
migration_id=migration_id,
|
||||
code=code,
|
||||
error=error,
|
||||
details={
|
||||
"bundle_id": bundle_id,
|
||||
"reason": reason,
|
||||
"duration_ms": (time.monotonic() - started) * 1000,
|
||||
"swap_pending": restore_pending(home),
|
||||
},
|
||||
)
|
||||
if not isinstance(error, Exception) or (
|
||||
isinstance(error, MigrationRestoreFailed) and error.code == code
|
||||
):
|
||||
raise
|
||||
raise MigrationRestoreFailed(code) from error
|
||||
_log(
|
||||
home,
|
||||
"restore.finished",
|
||||
"succeeded",
|
||||
migration_id=migration_id,
|
||||
details={**result.as_document(), "reason": reason},
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def _restore(home: Path, *, bundle_id: str, started: float) -> RestoreResult:
|
||||
from one_memory.sdk_migration_bundle import inspect_sdk_migration_bundle
|
||||
|
||||
if restore_pending(home):
|
||||
raise MigrationRestoreFailed("sdk_migration_restore_pending")
|
||||
manifest = inspect_sdk_migration_bundle(data_dir=home, bundle_id=bundle_id)
|
||||
bundle = home / "backups" / f"{bundle_id}.sdk-bundle"
|
||||
listed = manifest.get("files")
|
||||
files = [item for item in listed if isinstance(item, dict)] if isinstance(listed, list) else []
|
||||
total = sum(int(str(item["bytes"])) for item in files)
|
||||
if shutil.disk_usage(home).free < total:
|
||||
raise MigrationRestoreFailed("sdk_migration_restore_disk_space_insufficient")
|
||||
_release_handles()
|
||||
|
||||
staging = _staging(home)
|
||||
sdk_staging = _sdk_staging(home)
|
||||
shutil.rmtree(staging, ignore_errors=True)
|
||||
shutil.rmtree(sdk_staging, ignore_errors=True)
|
||||
(staging / "management").mkdir(parents=True, mode=0o700)
|
||||
try:
|
||||
shutil.copy2(bundle / "management" / DATABASE, staging / "management" / DATABASE)
|
||||
config_staged = manifest.get("config") is not None
|
||||
if config_staged:
|
||||
shutil.copy2(bundle / "management" / CONFIG, staging / "management" / CONFIG)
|
||||
_verify_copy(staging / "management", files, "management/")
|
||||
sdk_source = bundle / "sdk" / HY_MEMORY_NAMESPACE
|
||||
if not sdk_source.is_dir() or sdk_source.is_symlink():
|
||||
raise MigrationRestoreFailed("sdk_migration_restore_bundle_incomplete")
|
||||
sdk_staging.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copytree(sdk_source, sdk_staging, symlinks=True)
|
||||
_verify_copy(sdk_staging, files, f"sdk/{HY_MEMORY_NAMESPACE}/")
|
||||
for path in (staging / "management").iterdir():
|
||||
_fsync(path)
|
||||
except BaseException:
|
||||
shutil.rmtree(staging, ignore_errors=True)
|
||||
shutil.rmtree(sdk_staging, ignore_errors=True)
|
||||
raise
|
||||
|
||||
failed = failed_state_path(home)
|
||||
# One failed state is kept for diagnosis; the newest replaces the last.
|
||||
shutil.rmtree(failed, ignore_errors=True)
|
||||
if failed.exists():
|
||||
raise MigrationRestoreFailed("sdk_migration_restore_failed_state_busy")
|
||||
steps = _plan(home, config_staged=config_staged)
|
||||
_write_json(
|
||||
_journal(home),
|
||||
{"schema_version": 1, "bundle_id": bundle_id, "steps": [list(step) for step in steps]},
|
||||
)
|
||||
done: list[tuple[Path, Path]] = []
|
||||
try:
|
||||
_apply(home, steps, done)
|
||||
except BaseException as error:
|
||||
if not isinstance(error, Exception):
|
||||
# Interrupted (a signal tearing the process down): leave the
|
||||
# journal; the next run finishes the swap forward.
|
||||
raise
|
||||
try:
|
||||
for source, target in reversed(done):
|
||||
target.replace(source)
|
||||
except OSError:
|
||||
# The journal stays: the next run finishes the swap forward.
|
||||
raise MigrationRestoreFailed("sdk_migration_restore_swap_interrupted") from error
|
||||
_journal(home).unlink(missing_ok=True)
|
||||
shutil.rmtree(staging, ignore_errors=True)
|
||||
shutil.rmtree(sdk_staging, ignore_errors=True)
|
||||
raise
|
||||
_finish(home, bundle_id)
|
||||
return RestoreResult(
|
||||
bundle_id=bundle_id,
|
||||
files=len(files),
|
||||
bytes=total,
|
||||
duration_ms=(time.monotonic() - started) * 1000,
|
||||
)
|
||||
|
||||
|
||||
def finish_interrupted_restore(data_dir: Path, *, migration_id: str | None = None) -> bool:
|
||||
"""Complete a swap a killed process left between its renames. True if it did."""
|
||||
home = _home(data_dir)
|
||||
journal = _journal(home)
|
||||
if not journal.is_file():
|
||||
return False
|
||||
started = time.monotonic()
|
||||
try:
|
||||
document = json.loads(journal.read_text(encoding="utf-8"))
|
||||
steps = document["steps"]
|
||||
bundle_id = str(document["bundle_id"])
|
||||
if not isinstance(steps, list) or not all(
|
||||
isinstance(step, list) and len(step) == 2 for step in steps
|
||||
):
|
||||
raise ValueError("journal steps invalid")
|
||||
_release_handles()
|
||||
_apply(home, steps, [])
|
||||
_finish(home, bundle_id)
|
||||
except Exception as error:
|
||||
code = (
|
||||
error.code
|
||||
if isinstance(error, MigrationRestoreFailed)
|
||||
else "sdk_migration_restore_resume_failed"
|
||||
)
|
||||
_log(
|
||||
home,
|
||||
"restore.failed",
|
||||
"failed",
|
||||
migration_id=migration_id,
|
||||
code=code,
|
||||
error=error,
|
||||
details={"reason": "interrupted_swap", "swap_pending": True},
|
||||
)
|
||||
if isinstance(error, MigrationRestoreFailed):
|
||||
raise
|
||||
raise MigrationRestoreFailed(code) from error
|
||||
_log(
|
||||
home,
|
||||
"restore.finished",
|
||||
"succeeded",
|
||||
migration_id=migration_id,
|
||||
details={
|
||||
"bundle_id": bundle_id,
|
||||
"reason": "interrupted_swap",
|
||||
"duration_ms": (time.monotonic() - started) * 1000,
|
||||
},
|
||||
)
|
||||
return True
|
||||
|
||||
|
||||
def prune_restored_bundles(data_dir: Path, *, keep: set[str]) -> int:
|
||||
"""Drop bundles already restored, once a newer verified bundle exists.
|
||||
|
||||
Each holds the same pre-migration home a newer bundle now holds (capture
|
||||
stayed fenced since), so it is only a second copy of it.
|
||||
"""
|
||||
backups = _home(data_dir) / "backups"
|
||||
removed = 0
|
||||
for mark in backups.glob(f"*.sdk-bundle/{RESTORED_MARK}"):
|
||||
bundle = mark.parent
|
||||
if bundle.name.removesuffix(".sdk-bundle") in keep:
|
||||
continue
|
||||
shutil.rmtree(bundle, ignore_errors=True)
|
||||
removed += 0 if bundle.exists() else 1
|
||||
return removed
|
||||
|
||||
|
||||
def discard_failed_state(data_dir: Path) -> None:
|
||||
"""The failed half-migrated copy is only useful until a migration succeeds."""
|
||||
shutil.rmtree(failed_state_path(data_dir), ignore_errors=True)
|
||||
Loaded 100 of 371 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user